Literature clustering method based on citation network and large language model analysis

Abstract

The invention relates to the technical field of literature clustering, in particular to a literature clustering method based on a citation network and large language model analysis, which comprises the following steps of: obtaining and preprocessing literatures; clustering is carried out on the reference literature questions; constructing a clustering quality evaluation mechanism; screening highly cited literatures and cited literatures; carrying out abstract analysis and panoramic scanning; and generating a structured review and visualization. According to the method, a large language model is used for carrying out clustering analysis on reference literature topics, finding out mainstream clusters, ensuring comprehensiveness and systematicness of the review content, screening out highly-introduced literatures from the mainstream clusters, ensuring high quality and authority of the review content, analyzing the abstract of the literatures by using the large language model, extracting important topics, and improving the accuracy of the review content. The accuracy and depth of the review content are ensured, and the comprehensiveness, accuracy, systematicness and efficiency of literature review can be remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of document clustering technology, specifically a document clustering method based on citation network and large language model analysis. Background Technology

[0002] In the field of scientific research, quickly understanding the research context of a specific topic and capturing its research hotspots and future development trends is crucial for subsequent research. Traditional methods mainly rely on manual reading and summarizing of each article, which is inefficient and struggles to cope with the massive increase in the volume of literature today. In recent years, with the rapid development of natural language processing technology, large language models have made significant progress in text processing and analysis, enabling them to efficiently handle massive amounts of text data.

[0003] However, existing tools (such as those based on word frequency statistics, topic modeling (LDA), or simply using LLM to summarize single articles) still suffer from systematic deficiencies, insufficient accuracy / depth, and a lack of evolutionary analysis. This is mainly manifested in the fact that existing tools struggle to automatically identify and organize multiple mainstream research directions or subfields within a field (clustering), easily overlooking important branches, leading to fragmented review content and an inability to systematically trace the research trajectory of a specific topic; tools relying solely on word frequency statistics or simple topic models may fail to capture the core content of the research.

[0004] Tools that only summarize individual articles struggle to understand the connections and evolution between them. Conversely, having large language models directly summarize a vast amount of literature can easily create a knowledge illusion. Existing tools also struggle to automatically track the dynamic development of research hotspots, hindering an accurate grasp of research trends.

[0005] Based on this, the present invention provides a document clustering method based on citation network and large language model analysis to solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a document clustering method based on citation network and large language model analysis to solve the problems mentioned in the background.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention proposes a document clustering method based on citation networks and large language model analysis, comprising the following steps:

[0009] S1. Perform literature acquisition and preprocessing;

[0010] S2. Cluster the titles of the references;

[0011] S3. Construct a clustering quality assessment mechanism;

[0012] S4. Screen highly cited and citing documents;

[0013] S5. Abstract Analysis and Panoramic Scan;

[0014] S6. Generate structured reviews and visualizations.

[0015] Optionally, the implementation process of step S1 is as follows:

[0016] S1.1. Input a literature collection R1 on a specific topic;

[0017] S1.2. Set the time range and document type selection criteria (e.g., only retain journal articles and conference papers) to obtain the selected document set R2; extract the reference information of all documents in R2 to form the reference set R3;

[0018] S1.3. Perform data cleaning on R3: remove duplicate reference entries; remove reference entries with missing or invalid titles (such as those with only placeholders).

[0019] Optionally, the implementation process of step S2 is as follows:

[0020] S2.1. Users select a clustering algorithm (such as K-means, DBSCAN, hierarchical clustering) or use the system's default recommendation;

[0021] S2.2. Process all document titles in R3 using a Large Language Model (LLM): Convert each title into a high-dimensional semantic embedding vector; input the set of embedding vectors into the selected clustering algorithm to automatically identify document clusters;

[0022] S2.3. For each cluster, use LLM to analyze the semantics of all topics within the cluster; generate keywords (e.g., 3-5) that summarize the core research direction of the cluster as cluster labels.

[0023] Optionally, the implementation process of step S3 is as follows:

[0024] S3.1. Assess clustering quality using modularity (Q-value) and silhouette coefficient (S-value), and set quality thresholds (e.g., Q>0.3, S>0.5);

[0025] S3.2. Identify clusters that reach the threshold as “mainstream cluster set R4” and present them to the user.

[0026] Optionally, the implementation process of step S4 is as follows:

[0027] S4.1. Users set high-citation literature screening criteria (such as minimum citation frequency threshold N);

[0028] S4.2. For each cluster in R4: select documents with a citation frequency ≥ N as highly cited documents (representing the research foundation);

[0029] S4.3. Retrieve citing references (representing the latest developments) from R2 or the extended reference database that cite highly cited references.

[0030] Optionally, the implementation process of step S5 is as follows:

[0031] S5.1. For each cluster and its literature collection in R4: group by time period, including grouping by publication year (or custom time window);

[0032] S5.2.LLM Deep Semantic Analysis: Design prompts (e.g., "Analyze the collection of abstracts, extract 1-3 core research topics, and describe their importance"); input literature abstracts from various time periods to extract core research topics and descriptions;

[0033] S5.3. Dynamic Evolution Analysis: Integrate results from all time periods, compare topic changes (new additions, disappearances, merging, differentiation) using LLM, and mark the evolution trend.

[0034] Optionally, the implementation process of step S6 is as follows:

[0035] S6.1. Text Generation: Based on cluster labels, highly cited literature, citing literature, and topic evolution results, structured review texts are generated through predefined templates or LLM instructions (such as "using cluster labels as chapter titles, highly cited literature to introduce the basics, citing literature to reflect the progress, and describing the evolution in chronological order").

[0036] S6.2. Visualization: Generate graphs, citation network clustering graphs (showing mainstream clustering relationships), topic evolution timeline graphs, and highly cited literature impact graphs.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] This invention utilizes a large language model to perform cluster analysis on the titles of references, identify mainstream clusters to ensure the comprehensiveness and systematicity of the review content, select highly cited literature from the major mainstream clusters to ensure the high quality and authority of the review content, and analyze the abstracts of the literature using a large language model to extract important topics to ensure the accuracy and depth of the review content. This invention can significantly improve the comprehensiveness, accuracy, systematicity, and efficiency of literature reviews. Attached Figure Description

[0039] Figure 1 This is a flowchart of the document clustering method based on citation network and large language model analysis of the present invention;

[0040] Figure 2 This is a schematic diagram of the content mapping mechanism of the present invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1, please refer to Figure 1 This invention proposes a document clustering method based on citation network and large language model analysis, comprising the following steps:

[0043] S1. Perform literature acquisition and preprocessing;

[0044] S2. Cluster the titles of the references;

[0045] S3. Construct a clustering quality assessment mechanism;

[0046] S4. Screen highly cited and citing documents;

[0047] S5. Abstract Analysis and Panoramic Scan;

[0048] S6. Generate structured reviews and visualizations.

[0049] In this embodiment, it should also be noted that the implementation process of step S1 is as follows:

[0050] S1.1. Input a literature collection R1 on a specific topic;

[0051] S1.2. Set the time range and document type selection criteria (e.g., only retain journal articles and conference papers) to obtain the selected document set R2; extract the reference information of all documents in R2 to form the reference set R3;

[0052] S1.3. Perform data cleaning on R3: remove duplicate reference entries; remove reference entries with missing or invalid titles (such as those with only placeholders).

[0053] In this embodiment, it should also be noted that the implementation process of step S2 is as follows:

[0054] S2.1. Users select a clustering algorithm (such as K-means, DBSCAN, hierarchical clustering) or use the system's default recommendation;

[0055] S2.2. Process all document titles in R3 using a Large Language Model (LLM): Convert each title into a high-dimensional semantic embedding vector; input the set of embedding vectors into the selected clustering algorithm to automatically identify document clusters;

[0056] S2.3. For each cluster, use LLM to analyze the semantics of all topics within the cluster; generate keywords (e.g., 3-5) that summarize the core research direction of the cluster as cluster labels.

[0057] In this embodiment, it should also be noted that the implementation process of step S3 is as follows:

[0058] S3.1. Assess clustering quality using modularity (Q-value) and silhouette coefficient (S-value), and set quality thresholds (e.g., Q>0.3, S>0.5);

[0059] S3.2. Identify clusters that reach the threshold as “mainstream cluster set R4” and present them to the user.

[0060] In this embodiment, it should also be noted that the implementation process of step S4 is as follows:

[0061] S4.1. Users set high-citation literature screening criteria (such as minimum citation frequency threshold N);

[0062] S4.2. For each cluster in R4: select documents with a citation frequency ≥ N as highly cited documents (representing the research foundation);

[0063] S4.3. Retrieve citing references (representing the latest developments) from R2 or the extended reference database that cite highly cited references.

[0064] In this embodiment, it should also be noted that the implementation process of step S5 is as follows:

[0065] S5.1. For each cluster and its literature collection in R4: group by time period, including grouping by publication year (or custom time window);

[0066] S5.2.LLM Deep Semantic Analysis: Design prompts (e.g., "Analyze the collection of abstracts, extract 1-3 core research topics, and describe their importance"); input literature abstracts from various time periods to extract core research topics and descriptions;

[0067] S5.3. Dynamic Evolution Analysis: Integrate results from all time periods, compare topic changes (new additions, disappearances, merging, differentiation) using LLM, and mark the evolution trend.

[0068] In this embodiment, it should also be noted that the implementation process of step S6 is as follows:

[0069] S6.1. Text Generation: Based on cluster labels, highly cited literature, citing literature, and topic evolution results, structured review texts are generated through predefined templates or LLM instructions (such as "using cluster labels as chapter titles, highly cited literature to introduce the basics, citing literature to reflect the progress, and describing the evolution in chronological order").

[0070] S6.2. Visualization: Generate graphs, citation network clustering graphs (showing mainstream clustering relationships), topic evolution timeline graphs, and highly cited literature impact graphs.

[0071] Example 2, please refer to Figure 1 and Figure 2 In practical applications, this invention utilizes a document clustering method based on citation networks and large language model analysis. Specifically, its steps are as follows:

[0072] (1). Literature Acquisition and Preprocessing:

[0073] User-inputted or provided literature collection R1 on a specific topic;

[0074] Set timeframes and document type selection criteria based on research needs:

[0075] For example: by retaining only journal articles and conference papers and removing non-research literature such as reviews and editorials, we obtain the filtered literature set R2;

[0076] Extract the reference information of all documents in reference set R2 to form reference set R3;

[0077] Data cleaning was performed on the reference set R3, including: removing duplicate reference entries; removing reference entries with missing or invalid titles, including those with only placeholders such as "Reference" or "Citedby", to ensure the quality of input data for subsequent processing.

[0078] (2). Reference title clustering:

[0079] Users can choose one of the supported clustering algorithms: K-means, DBSCAN, hierarchical clustering, or be recommended by the system default.

[0080] Large Language Model (LLM) is used to process the titles of all references in reference set R3: LLM converts each reference title into a high-dimensional semantic embedding vector to capture the deep semantic information of the title;

[0081] The obtained set of embedding vectors is input into the clustering algorithm selected by the user for cluster analysis, which automatically identifies multiple document clusters;

[0082] After generating clusters, for each cluster, LLM is used to analyze the semantics of all questions within that cluster, and representative keywords that best summarize the core research direction of that cluster are generated as cluster labels. LLM is then asked to output 3-5 keywords based on prompt words.

[0083] (3). Clustering quality assessment:

[0084] The quality of the clusters generated in step 2 is evaluated using two key metrics: modularity (Q-value) and silhouette coefficient (S-value).

[0085] Set quality thresholds Q>0.3 and S>0.5, and identify clusters that reach or exceed the thresholds as successful clusters with good cohesion and discriminative power, forming the mainstream cluster set R4;

[0086] Present the mainstream cluster set R4 and its labels to the user;

[0087] (4) Screening of highly cited and citing documents in mainstream clusters:

[0088] User-defined or system-provided filtering criteria for highly cited literature:

[0089] For example: the minimum citation frequency threshold N, N can be input by the user according to the characteristics of the field or a default value recommended by the system based on the overall citation distribution of the document set R2;

[0090] For each cluster in the mainstream cluster set R4: within the scope of references contained in the cluster, select references with a citation frequency ≥ N as highly cited references representing the research foundation of this research direction;

[0091] At the same time, in the literature set R2 or the extended literature database, all literature that cited the highly cited literature is retrieved as research literature representing the latest progress and application of this research direction;

[0092] Ensure that the selected literature has high research value and impact;

[0093] (5). Abstract Analysis and Panoramic Scan:

[0094] For each cluster in the mainstream cluster set R4 and its selected set of highly cited and citing documents: First, these documents are grouped according to the publication year or a user-specified time window, such as by year, two-year period, or five-year period.

[0095] Secondly, specific LLM prompts are designed for the collection of literature abstracts for each time period;

[0096] For example: "Please analyze the following collection of abstracts, extract 1-3 core research topics in this research direction, summarize each topic with a short title and a sentence, and point out the importance or novelty of the topic in this time period."

[0097] LLM was used for deep semantic analysis to extract the core research topics and their descriptions in this research direction during this time period;

[0098] Finally, by integrating the analysis results from all time periods, LLM automatically labels and describes the dynamic evolution trend of research topics by comparing topic changes in different time periods, including the addition, disappearance, merging, and differentiation of topics.

[0099] (6) Structured review generation and visualization:

[0100] Based on the results obtained from the aforementioned steps: mainstream clusters and their labels, the set of highly cited and citing documents for each cluster, research topics and evolution analysis for each time period, generated through predefined structured templates or specific LLM instructions;

[0101] For example, the instruction requires "using cluster labels as chapter titles, highly cited literature as the research basis for introduction, citing literature to reflect the latest progress, organizing the topic evolution description according to timeline, and generating coherent literature review chapters";

[0102] Transform structured information into coherent and systematic literature review texts;

[0103] Simultaneously, corresponding visualization maps can be generated, such as: citation network clustering maps, which show the mainstream clusters and their relationships; topic evolution timeline maps, which show the rise and fall of topics under various research directions; and highly cited literature influence maps, which intuitively present the field structure and evolution path in the form of interactive charts.

[0104] Example: This example demonstrates the complete implementation process of "a literature clustering method based on citation network and large language model analysis", using "the application of blockchain in supply chain management" as an application scenario, and fully presents the technical chain from data input to result output.

[0105] Step 1. Implementation Environment Configuration:

[0106] This method is implemented in the following technical environments:

[0107] Operating System: Windows 10 Professional / Ubuntu 20.04LTS;

[0108] Programming language: Python 3.10;

[0109] The core technology stack is shown in Table 1:

[0110]

[0111] Table 1

[0112] Large language model:

[0113] OpenAI GPT-4: Used for semantic analysis and text generation;

[0114] Local deployment option: LLaMA 2 7B Quantized Edition, loaded via HuggingFace

[0115] Step 2. Input data structure:

[0116] Construct an initial reference set R1, which includes the following core fields: reference ID, title, abstract, year, citation frequency, and references;

[0117] For application scenario examples, please refer to Table 2:

[0118]

[0119] Table 2 shows the preprocessed reference set R3:

[0120]

[0121]

[0122] #Retrieving Cited References

[0123] citing_docs=get_citing_documents(high_cited)

[0124] Design structured prompts for dynamic analysis:

[0125] The system command is:

[0126] You are a professional research analysis assistant. Please provide the following literature abstracts grouped by year:

[0127] 1) Identify three core research topics;

[0128] 2) Analyze the temporal evolution characteristics of each topic;

[0129] 3) Identify the technological turning point;

[0130] [Input Data]

[0131] 2018-2019:

[0132] Abstract 1: Blockchain applied to food cold chain traceability...

[0133] Abstract 2: Smart contracts simplify the procurement process...

[0134] 2020-2022:

[0135] Abstract 3: Cross-organizational blockchain data sharing...

[0136] Abstract 4: Smart Contract Settlement in Supply Chain Finance...

[0137] Please see Figure 2 Establish a content mapping mechanism:

[0138] Output result:

[0139] "Blockchain has formed three mainstream research directions in the field of supply chain management: traceability technology (profile coefficient 0.62), smart contract applications (profile coefficient 0.58), and cross-organizational collaboration (profile coefficient 0.55). Research in 2018-2019 focused on food cold chain traceability, with the representative paper 'Blockchain-based Cold Chain Monitoring' (cited 42 times) establishing the basic technical framework. After 2020, research shifted to smart contract-driven cross-enterprise settlement, and the collaborative model proposed in the paper 'Blockchain Supply Chain Finance Applications' was cited in 15 subsequent studies..."

[0140] Through the above steps, the specific advantages and beneficial effects of the method of the present invention include:

[0141] Enhancing the comprehensiveness of literature reviews: This invention, based on cluster analysis of reference networks, comprehensively covers all directions of the research topic, ensuring that the review content encompasses all aspects of the topic and avoiding omissions of important research directions. Rigorous data cleaning (duplicate removal, elimination of invalid headings) and cluster quality assessment (Q-value, S-value) guarantee the reliability of the clustering results. Furthermore, by selecting highly cited literature (research foundation / authoritative) and citing literature (latest progress / application), it not only covers the research foundation of the field but also reflects the latest research progress, further enhancing the comprehensiveness of the review.

[0142] Improving the accuracy of literature reviews: This invention utilizes Large Language Modeling (LLM) to perform semantic embedding transformation and clustering of literature titles, and deep semantic analysis of the literature abstract collection (based on time periods and specific cue words). This enables more accurate identification of domain structure and extraction of the core research topics, ensuring that the review content accurately reflects the core content and research trends of the literature. Furthermore, by filtering citation relationships, highly cited and citing literature are prioritized, guaranteeing the authority of the literature, further improving the accuracy of the review, and effectively reducing the risk of LLM generating illusions when processing a large number of documents.

[0143] Enhancing the systematic nature of literature reviews: This invention categorizes literature by research direction through cluster analysis and organizes the review content using a structured generation method (cluster labels → chapter titles, highly cited literature → research foundation, citing literature → latest developments, topic evolution → timeline description), ensuring a clear logical structure and hierarchy. Visualized maps (such as cluster diagrams and topic evolution diagrams) further enhance the systematic presentation. In-depth analysis of highly cited literature and topic evolution highlights research hotspots and trends in the field, making the review content more targeted and practical.

[0144] Improving the efficiency of literature reviews: This invention employs an automated process (data cleaning, LLM embedding generation, clustering calculation, quality assessment, citation screening, LLM abstract analysis, and structured generation), utilizing a large language model to process literature data and combining it with clustering algorithms for literature classification and analysis. This effectively replaces a significant amount of manual reading, classification, and summarization work, greatly reducing human intervention and significantly improving the efficiency of literature review generation. Simultaneously, through automated tools and methods, it can quickly respond to new literature data, promptly update and generate the latest literature reviews, ensuring the timeliness of the review content.

[0145] Reduced labor costs: By introducing automated tools and methods, this invention reduces reliance on manual labor, especially in the stages of literature cleaning, classification, core topic extraction, trend analysis and text generation, which significantly reduces the labor costs in the literature review generation process.

[0146] Enhancing the quality and credibility of literature reviews: This invention improves the quality and credibility of literature reviews by screening highly cited literature (based on a user-defined threshold N) to ensure that the review content includes the most representative and influential research results in the field. The inclusion of cited literature reflects the latest developments in the field, while the in-depth analysis of the abstract set and the tracking of its evolution over time by the Large Language Model (LLM) further enhances the depth, breadth, and timeliness of the review content.

[0147] Revealing the Structure and Evolution of a Research Field: This invention not only efficiently generates high-quality literature review texts, but also clearly and structurally presents the overall architecture (mainstream clusters / directions) of a research field and the specific evolutionary paths of research topics from each direction through cluster analysis and visualization techniques (citation network cluster maps, topic evolution timeline maps). This provides researchers with more intuitive and in-depth insights, helping them quickly understand the development trajectory, current hotspots, and future trends of the field. This is the key value that distinguishes this invention from traditional literature review methods.

[0148] In summary, this invention, through a series of innovative methods and techniques, can significantly improve the comprehensiveness, accuracy, systematicness, efficiency, and credibility of literature reviews, reduce human resource costs, and provide deeper domain insights and strong support for academic research and practical applications.

[0149] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0150] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A document clustering method based on citation network and large language model analysis, characterized in that, Includes the following steps: S1. Perform literature acquisition and preprocessing; S2. Cluster the titles of the references; S3. Construct a clustering quality assessment mechanism; S4. Screening highly cited and citing documents; S5. Abstract Analysis and Panoramic Scan; S6. Generate structured reviews and visualizations.

2. The document clustering method based on citation network and large language model analysis according to claim 1, characterized in that, The implementation process of step S1 is as follows: S1.

1. Input a literature collection R1 on a specific topic; S1.

2. Set the time range and document type selection criteria to obtain the selected document set R2; extract the reference information of all documents in R2 to form the reference set R3; S1.

3. Perform data cleaning on R3: remove duplicate reference entries; remove reference entries with missing or invalid titles.

3. The document clustering method based on citation network and large language model analysis according to claim 2, characterized in that, The implementation process of step S2 is as follows: S2.

1. The user selects a clustering algorithm or uses the system's default recommendation; S2.

2. Process all document titles in R3 using a large language model: convert each title into a high-dimensional semantic embedding vector; input the set of embedding vectors into the selected clustering algorithm to automatically identify document clusters; S2.

3. For each cluster, use LLM to analyze the semantics of all topics within the cluster; generate keywords that summarize the core research direction of the cluster as cluster labels.

4. The document clustering method based on citation network and large language model analysis according to claim 3, characterized in that, The implementation process of step S3 is as follows: S3.

1. Evaluate clustering quality using modularity and silhouette coefficient, and set a quality threshold; S3.

2. Identify clusters that reach the threshold as "mainstream cluster set R4" and present them to the user.

5. The document clustering method based on citation network and large language model analysis according to claim 4, characterized in that, The implementation process of step S4 is as follows: S4.

1. Users set criteria for selecting highly cited literature; S4.

2. For each cluster in R4: select documents with a citation frequency ≥ N as highly cited documents; S4.

3. Retrieve citing references that cite highly cited references in R2 or the extended reference database.

6. The document clustering method based on citation network and large language model analysis according to claim 5, characterized in that, The implementation process of step S5 is as follows: S5.

1. For each cluster and its literature set in R4: group by time period, including grouping by publication year; S5.2.LLM Deep Semantic Analysis: Design prompt words; input literature abstracts for each time period to extract core research topics and descriptions; S5.

3. Dynamic Evolution Analysis: Integrate results from all time periods, compare topic changes using LLM, and mark evolution trends.

7. The document clustering method based on citation network and large language model analysis according to claim 6, characterized in that, The implementation process of step S6 is as follows: S6.

1. Text Generation: Based on clustering tags, highly cited literature, citing literature, and topic evolution results, structured review texts are generated using predefined templates or LLM instructions. S6.

2. Visualization: Generate graphs, citation network clustering graphs, topic evolution timeline graphs, and highly cited literature impact graphs.

Images