Long document intelligent analysis method and system based on large language model and graph database

Abstract

The application relates to a long document intelligent analysis method and system based on a large language model and a graph database. The method comprises the following steps: inputting a long document into an edge computing unit to perform semantic segmentation, and performing structured information extraction by using a large language model to perform entity extraction, relationship type induction, triple modeling and hierarchical structure recognition; injecting the generated triple data into a graph database in batches to dynamically construct a knowledge graph; performing a multi-dimensional graph native analysis task on the knowledge graph; visualizing the analysis result or converting the analysis result into a natural language summary output, and comparing the result with the original long document to automatically generate a supporting conclusion paragraph index. The application combines the deep semantic understanding ability of the large language model with the efficient correlation calculation ability of the graph database, realizes deep insight into the long document without complex manual intervention, solves the drawbacks of the traditional natural language processing pipeline, and improves the accuracy of the long document intelligent analysis and the traceability of the conclusion.

Description

Technical Field

[0001] The present invention relates to the cross - technical field of artificial intelligence and knowledge graphs, and particularly to a long - document intelligent analysis method and system based on large - language models and graph databases. Background Art

[0002] With the acceleration of the digitalization process, a large number of long documents (such as classic literary works, legal documents, corporate annual reports, etc.) urgently need to be deeply mined. Traditional text analysis methods (such as keyword search, simple statistics) are difficult to reveal the complex entity relationships and event logics inside long documents.

[0003] However, when the existing technology conducts intelligent analysis of long documents, there are usually the following limitations: First, the analysis dimension is single. The results of information extraction by existing deep - learning - based natural language processing (NLP) models are mostly flat lists, which are difficult to support cross - entity, cross - event, and cross - document linkage insights. Second, the association mining is shallow. The traditional knowledge graph construction process depends on rigid rules or small - scale models and cannot effectively model the complex semantic relationships and hierarchical structures in long documents. Third, the processing efficiency is low and the results are easily lost. Especially when facing literary works with millions of words such as "Dream of the Red Chamber" or ultra - long texts such as multi - year company annual reports, traditional NLP pipelines are prone to information breaks and context loss problems, and there is a lack of traceability verification of the credibility of analysis conclusions. Therefore, there is still a need for further improvement in long - document intelligent analysis methods. Summary of the Invention

[0004] Based on the above description, the present invention provides a long - document intelligent analysis method and system based on large - language models and graph databases, which are used to solve the technical problems that the existing technology relies on flat extraction, lacks deep - level association calculation, and is difficult to ensure the credibility of results, resulting in low efficiency and accuracy of long - document analysis.

[0005] To achieve the above object, the first aspect of the present invention provides a long - document intelligent analysis method based on large - language models and graph databases, and the method includes: Input the long document to be analyzed into a large - language model, and the large - language model divides the long document into blocks and performs information extraction on each text block to generate structured triple data containing entities, relationships, and attributes; Batch - import the generated triple data into a graph database, perform conflict detection and fusion processing, and construct and store a dynamic knowledge graph; In response to the user's analysis instruction, call the graph query language of the graph database and perform multi - dimensional analysis tasks on the dynamic knowledge graph; The analysis results of the multi-dimensional analysis task are converted into visual charts or natural language summaries and output. At the same time, a large language model is called to compare the analysis results with the original long document, and the original text paragraph index and content supporting the analysis results are automatically generated.

[0006] As a preferred embodiment, the step of extracting information from each text block to generate structured triplet data containing entities, relations, and attributes includes: Multiple rounds of structured information extraction tasks are executed sequentially, and the information extraction tasks include at least named entity recognition, entity disambiguation, relation extraction, event extraction, and sentiment analysis. Each information extraction stage employs embedded contextual information and dynamic prompts that enforce JSON Schema specifications to constrain the output format and content of the large language model.

[0007] As a preferred embodiment, the step of batch importing the generated triplet data into a graph database for conflict detection and fusion processing, and constructing and storing a dynamic knowledge graph includes: Conflict detection is performed between newly imported triples and existing nodes and relationships in the graph, and conflicting data is then fused. Add attribute tags to nodes and relationships, including the chapter in which they first appear and the time when the event occurs.

[0008] As a preferred embodiment, the multi-dimensional analysis task is performed on the dynamic knowledge graph, and the multi-dimensional analysis task includes at least person relationship network analysis, event development tracking and cross-document data comparison analysis. The analysis of the personal relationship network includes: using a centrality algorithm to calculate the importance of a person in the network, using a community detection algorithm to identify the community divisions between people, and inferring the implicit relationships between people; The event development tracing includes: sorting events according to their timestamps to form a timeline, constructing a causal chain or temporal chain of events, and identifying key turning points.

[0009] As a preferred embodiment, the analysis results of the multi-dimensional analysis task are converted into visual charts or natural language summaries and output; simultaneously, a large language model is invoked to compare the analysis results with the original long document, automatically generating an index of original text paragraphs and content supporting the analysis results, thus achieving the traceability of the conclusions, including: The analysis results can be rendered into an interactive relationship graph, timeline, or comparison table, or a large language model can be called to summarize the graph query results into a natural language summary. The source tracing mechanism is activated simultaneously, and the large language model is called again to compare and analyze the conclusion with the original long document. Contextual evidence supporting the conclusion is extracted, and the corresponding original text paragraph index is generated to achieve closed-loop verification of the conclusion and evidence.

[0010] As a preferred embodiment, the sequential execution of multiple rounds of structured information extraction tasks includes: Send the first prompt word, instructing the large language model to extract all entities and their attributes and return them in a preset format; Send the second prompt word to instruct the large language model to extract all unique relation type names; Send a third prompt word to dynamically embed the entity list and the relation type list, and instruct the large language model to construct specific relation triples between entities; Sending the fourth prompt word instructs the large language model to identify and construct a hierarchical relationship system in the long document based on the triples.

[0011] A second aspect of the present invention provides an intelligent long document analysis system based on a large language model and graph database, comprising: Document preprocessing module: responsible for semantically segmenting the long document to be analyzed and preparing it for input into the large language model; Language model invocation module: responsible for sending dynamic prompt words to the large language model, performing multi-stage information extraction, and receiving the returned structured triple data; Graph Database Interaction Module: Responsible for batch importing the generated triple data into the graph database, performing conflict detection and attribute fusion, constructing a dynamic knowledge graph and performing graph query operations; Multidimensional analysis engine module: responsible for calling the graph query language of the graph database, performing multidimensional analysis tasks such as interpersonal relationship networks and event tracking, and generating analysis results; Results presentation module: responsible for converting analysis results into visual charts or natural language summaries and outputting them to the user; The source tracing module is used to call a large language model to compare the analysis results with the original document and generate paragraph indexes and content that support the conclusions.

[0012] Another aspect of the present invention provides a computer-readable storage medium storing a program for intelligent analysis of long documents based on a large language model and a graph database. When the program for intelligent analysis of long documents based on a large language model and a graph database is executed by a processor, it implements the steps of the method for intelligent analysis of long documents based on a large language model and a graph database.

[0013] Compared with the prior art, the technical solution of this application has the following beneficial technical effects: High-precision knowledge extraction: This application effectively suppresses the illusion and format drift problems of large language models in long text processing through a four-stage pipeline and schema constraint hint engineering, and achieves accurate structured extraction of complex and implicit knowledge (such as family lineage, organizational structure, and causal event chains).

[0014] Highly efficient relational computation: This application relies on the native graph storage and query capabilities of graph databases to support millisecond-level traversal and aggregation of millions of triples, solving the performance bottleneck of traditional vector or relational databases when processing highly connected data.

[0015] Deep and multi-dimensional insights: This application not only supports single-dimensional analysis, but also enables cross-linked analysis of multiple elements such as "people, events, time and data", which greatly enhances the richness of the analytical perspective.

[0016] Explainability and Conclusion Source Tracing: This application achieves a "conclusion-evidence" closed loop through dual-channel output of visualization and natural language, supplemented by a source tracing mechanism of original text paragraphs launched synchronously by the system, providing users with intelligent decision support that combines depth and credibility. Attached Figure Description

[0017] Figure 1 This is the overall flowchart of the method in Example 1. Detailed Implementation

[0018] Reference Figure 1 A method for intelligent analysis of long documents based on large language models and graph databases, comprising the following steps: Step 1: Input the long document to be analyzed into the large language model. The large language model divides the long document into multiple text blocks, and performs information extraction on each text block to generate structured triple data containing entities, relations and attributes.

[0019] Specifically, in this step, multiple rounds of structured information extraction tasks are sequentially performed on each text block during information extraction. These information extraction tasks include at least named entity recognition, entity disambiguation, relation extraction, event extraction, and sentiment analysis. Each information extraction stage employs dynamic prompts with embedded contextual information and enforced JSON Schema specifications to constrain the output format and content of the large language model.

[0020] Specifically, performing multi-round structured information extraction tasks includes: Send the first prompt word, instructing the large language model to extract all entities and their attributes and return them in a preset format; Send the second prompt word to instruct the large language model to extract all unique relation type names; Send a third prompt word to dynamically embed the entity list and the relation type list, and instruct the large language model to construct specific relation triples between entities; Sending the fourth prompt word instructs the large language model to identify and construct a hierarchical relationship system in the long document based on the triples.

[0021] For example, long documents can be chunked by chapters or semantic units. For each text chunk, a carefully crafted dynamic prompt is sent to a large language model (such as GPT-4, Claude, or an open-source LLM), instructing it to perform multiple NLP pipeline tasks. This pipeline sequentially covers entity extraction, relation type induction, triple modeling, and hierarchical structure recognition. For example, for a chapter of *Dream of the Red Chamber*, the prompt could be set as: "Please extract all character entities and location entities from the following text, and identify the [kinship relationships, social relationships] between characters and the [events] that occur, in the format (entity 1, relationship, entity 2 / event)". The model ultimately returns structured triple data that follows a mandatory JSON Schema format.

[0022] Step 2: Import the generated triplet data into the graph database in batches, perform conflict detection and fusion processing, and construct and store the dynamic knowledge graph.

[0023] In this step, conflict detection is performed between the newly imported triples and existing nodes and relationships in the graph, and conflict data is fused. Attribute tags, including the chapter in which the node first appears and the time when the event occurs, are added to the nodes and relationships.

[0024] For example, all generated triplet data can be imported into a graph database (such as Neo4j or Memgraph) via a batch import interface, and the database automatically constructs a scalable dynamic knowledge graph. During the import process, the system simultaneously performs entity disambiguation, conflict detection, and attribute fusion operations. The system will disambiguate entities with the same name but different meanings (such as whether "Baoyu" refers to Jia Baoyu or Zhen Baoyu) and establish a unique ID. At the same time, rich attribute tags are added to nodes and relationships, such as the "first chapter of a character" and the "time of occurrence" of an event, to ensure the consistency and integrity of the knowledge graph.

[0025] Step 3: In response to the user's analysis command, invoke the graph query language of the graph database to perform multi-dimensional analysis tasks on the dynamic knowledge graph.

[0026] This step involves multi-dimensional analysis tasks, including at least the analysis of interpersonal relationships, the tracing of event developments, and cross-document data comparison and analysis.

[0027] The analysis of the personal relationship network includes: using a centrality algorithm to calculate the importance of a person in the network, using a community detection algorithm to identify the community divisions between people, and inferring the implicit relationships between people; The event development tracing includes: sorting events according to their timestamps to form a timeline, constructing a causal chain or temporal chain of events, and identifying key turning points.

[0028] For example, in response to a user's instruction, the system converts the user's instruction into a graph database query language (Cypher) to perform specific tasks, including: (1) Analysis of character relationships: Execute the Cypher query MATCH (p1:Person)-[r]->(p2:Person) WHERE p1.name = 'Jia Baoyu' RETURN p2.name, type(r) to obtain the direct associated entities and relationship types of the target character; on this basis, further call the built-in graph data science algorithm library of the graph database to perform deep network analysis on the constructed dynamic knowledge graph, specifically including: a) Node importance evaluation based on the PageRank algorithm: The system first creates a subgraph projection in memory consisting of all Person nodes and their associated relationships, and configures the relationships as undirected connections to reflect the reciprocity of the social network. Subsequently, call the PageRank algorithm to calculate this projection graph. This algorithm generates a global importance score (PageRank Score) for each character node by iteratively evaluating the degree to which each node is pointed to by other high-importance nodes. The higher the score of a character, the more central the hub position in the entire relationship network. The system finally returns a list of characters sorted in descending order of scores, which is used to identify the global key characters in addition to the direct associations.

[0029] b) Community structure discovery based on the Louvain algorithm: The system reuses the above subgraph projection and calls the Louvain community discovery algorithm. This algorithm automatically divides the character network into several communities with dense internal connections and sparse external connections by optimizing the modularity of the graph. The algorithm assigns a community identifier (Community ID) to each character node. By aggregating nodes with the same community identifier, the system can identify implicit social circles such as "the core family of the Jia Mansion" and "members of the Grand View Garden Poetry Club". This process not only reveals the explicit two-person relationships, but also discovers the group structures composed of multiple people with common attributes or interaction patterns.

[0030] By integrating the direct relationship query results, the PageRank global importance ranking, and the Louvain community division information, the system can comprehensively depict the core social circle of the target character (such as Jia Baoyu), and infer his roles and influences in different communities, so as to achieve a multi-dimensional and in-depth analysis of the complex character relationship network.

[0031] (2) Discovery of event context: The system executes the query `MATCH (e:Event) WHERE e.novel = 'Dream of the Red Chamber' RETURN e ORDER BY e.timestamp` to obtain a global event sequence sorted by timestamp. Based on this, the system further utilizes the path query capabilities of the graph database to deeply mine the causal dependencies between event nodes. The specific implementation is as follows: First, during the knowledge graph construction phase, in addition to recording the basic attributes of events (such as name, timestamp, and location), causal relationship edges between events are explicitly identified and established through information extraction tasks using a large language model (e.g., relationship types CAUSES or LEADS_TO). These causal relationship triples (such as (Event_A)-[:CAUSES]->(Event_B)) are imported into the graph database together to form a directed graph structure of event evolution.

[0032] Secondly, to identify key turning points, the system employs a variable-length path query technique. For any starting event or a user-specified key event, the following Cypher query is executed: MATCH path = (start:Event {name: 'Event'})-[:CAUSES 1..5]->(end:Event) WHERE start.timestamp<= end.timestamp RETURN nodes(path) AS event_chain, relationships(path) AS causal_links ORDER BY length(path) DESC Among them, [:CAUSES [1..5] indicates a search for causal paths of length 1 to 5 hops, effectively avoiding noisy paths caused by over-reasoning. This query can return a complete chain of events starting from the initial event and linked by causal relationships.

[0033] Furthermore, the system assesses the influence of all discovered causal paths. Its "betweenness centrality" is calculated by statistically analyzing the frequency with which each event node appears as an intermediate node (neither the starting point nor the ending point) in all causal paths. Higher frequency indicates that the event plays a crucial role in multiple causal chains, serving as a central pivot or major turning point in the plot. This assessment can be efficiently performed using the betweenness algorithm in the Graph Data Science Library (GDS).

[0034] Ultimately, the system integrates the timeline sorting results, causal path chains, and key turning point identification conclusions to generate a comprehensive depiction of the event's development. It outputs an ordered timeline view or causal network diagram, thereby revealing the internal logic and driving mechanism of the event's evolution in a long document.

[0035] (3) Data comparison and analysis: For long documents with multiple versions (such as company annual reports from different years), the system first constructs independent knowledge subgraphs for each document that are isolated from each other but have a consistent structure. Based on this, a structured comparison analysis is performed across subgraphs. The specific implementation method is as follows: a) Construction and labeling of independent subgraphs: In the knowledge graph construction phase of step two, the system assigns a unique document context identifier (e.g., doc_id: "2021_AR") to each input document (e.g., "2021 Annual Report" and "2022 Annual Report"). This identifier is appended as an attribute to all entity nodes and relationships extracted from that document. Subsequently, in the graph database, logically isolated subgraph management is implemented in the following way: All nodes (such as FinancialMetric, BusinessSegment, Executive, etc.) carry the doc_id attribute of their source document; when performing batch import, the system can selectively write triples under the same doc_id into the same named graph space (if the graph database supports it, such as Neo4j's Composite Database, or by distinguishing them by tags / attributes); when querying, data from specific years can be dynamically filtered through the WHERE clause to achieve the effect of "virtual subgraph".

[0036] b) Comparison of key financial indicators across years: The system predefines a set of core financial indicator node types (such as Revenue, NetProfit, R&D_Expense, etc.) and ensures that indicators with the same name in subgraphs of different years have the same node labels and attribute structures (such as value, currency, unit). Using graph query language, changes in indicators can be efficiently extracted and compared horizontally. This query directly returns the absolute value of revenue and the year-on-year growth rate for two years. Similar patterns can be extended to any structured indicator such as profit, debt ratio, and cash flow.

[0037] c) Comparison of the graph structure of business logic evolution: Beyond numerical comparisons, the system further analyzes structural changes in business logic relationships. For example, by examining increases or decreases in the "business segment - revenue contribution" relationship, it identifies shifts in strategic focus. Furthermore, the system can calculate the graph edit distance or subgraph isomorphism differences between two subgraphs to quantify the stability or degree of change in the overall business architecture. For critical paths (such as "R&D investment → patent output → new product revenue"), path existence checks determine whether the innovation chain continues or breaks.

[0038] d) Visualization and interpretation of differences: Ultimately, the system integrates the aforementioned numerical differences with structural changes to generate interactive comparative views (such as side-by-side bar charts, difference heatmaps, and animations of relationship edge increases and decreases). It also automatically generates natural language summaries using a large language model (e.g., "A new 'new energy vehicle' business segment was added in 2022, with R&D investment increasing by 35% year-on-year, but it has not yet generated significant revenue contribution"). Simultaneously, the source tracing module automatically locates specific paragraphs in the original annual report describing the relevant strategic adjustments, ensuring the verifiability and interpretability of the analysis conclusions.

[0039] Step 4: Convert the analysis results of the multi-dimensional analysis task into visual charts or natural language summaries and output them. At the same time, call the large language model to compare the analysis results with the original long document and automatically generate the original text paragraph index and content to support the analysis results.

[0040] In this step, the analysis results are rendered as an interactive relationship graph, timeline, or comparison table, or a large language model is called to summarize the graph query results into a natural language summary; The source tracing mechanism is activated simultaneously, and the large language model is called again to compare and analyze the conclusion with the original long document. Contextual evidence supporting the conclusion is extracted, and the corresponding original text paragraph index is generated to achieve closed-loop verification of the conclusion and evidence.

[0041] The analysis results can be rendered into interactive relationship diagrams, timelines, or comparison tables using front-end tools. Alternatively, the LLM can summarize the query results into a natural language summary (e.g., "Jia Baoyu's core social circle consists of Lin Daiyu, Xue Baochai, and Wang Xifeng..."). Simultaneously, the system initiates a source tracing mechanism, again calling the LLM to compare the analysis conclusions with the original long document, automatically generating an index of supporting original text paragraphs and contextual evidence for the user, thus achieving comprehensive traceability of the analysis conclusions and a closed loop of evidence.

[0042] Example 2: A long document intelligent analysis system based on a large language model and graph database includes: Document preprocessing module: responsible for semantically segmenting the long document to be analyzed and preparing it for input into the large language model; Language model invocation module: responsible for sending dynamic prompt words to the large language model, performing multi-stage information extraction, and receiving the returned structured triple data; Graph Database Interaction Module: Responsible for batch importing the generated triple data into the graph database, performing conflict detection and attribute fusion, constructing a dynamic knowledge graph and performing graph query operations; Multidimensional analysis engine module: responsible for calling the graph query language of the graph database, performing multidimensional analysis tasks such as interpersonal relationship networks and event tracking, and generating analysis results; Results presentation module: responsible for converting analysis results into visual charts or natural language summaries and outputting them to the user; The source tracing module is used to call a large language model to compare the analysis results with the original document and generate paragraph indexes and content that support the conclusions.

[0043] Example 3: A computer-readable storage medium is used in the aforementioned intelligent analysis system for long documents based on a large language model and graph database, characterized in that a computer program is stored on the computer-readable storage medium.

[0044] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent analysis of long documents based on large language models and graph databases, characterized by: The long document to be analyzed is input into a large language model, which divides the long document into blocks and extracts information from each text block to generate structured triplet data containing entities, relations and attributes. The generated triplet data is imported into a graph database in batches for conflict detection and fusion processing, and a dynamic knowledge graph is constructed and stored. In response to the user's analysis command, the graph query language of the graph database is invoked to perform multi-dimensional analysis tasks on the dynamic knowledge graph; The analysis results of the multi-dimensional analysis task are converted into visual charts or natural language summaries and output. At the same time, a large language model is called to compare the analysis results with the original long document, and the original text paragraph index and content supporting the analysis results are automatically generated.

2. The intelligent long document analysis method based on a large language model and graph database according to claim 1, characterized in that, The process of extracting information from each text block to generate structured triplet data containing entities, relations, and attributes includes: Multiple rounds of structured information extraction tasks are executed sequentially, and the information extraction tasks include at least named entity recognition, entity disambiguation, relation extraction, event extraction, and sentiment analysis. Each information extraction stage employs embedded contextual information and dynamic prompts that enforce JSON Schema specifications to constrain the output format and content of the large language model.

3. The intelligent long document analysis method based on a large language model and graph database according to claim 1, characterized in that, The step of batch importing the generated triplet data into a graph database for conflict detection and fusion processing, and constructing and storing a dynamic knowledge graph includes: Conflict detection is performed between newly imported triples and existing nodes and relationships in the graph, and conflicting data is then fused. Add attribute tags to nodes and relationships, including the chapter in which they first appear and the time when the event occurs.

4. The intelligent long document analysis method based on a large language model and graph database according to claim 1, characterized in that, The multi-dimensional analysis task is performed on the dynamic knowledge graph, and the multi-dimensional analysis task includes at least person relationship network analysis, event development tracking and cross-document data comparison analysis. The analysis of the personal relationship network includes: using a centrality algorithm to calculate the importance of a person in the network, using a community detection algorithm to identify the community divisions between people, and inferring the implicit relationships between people; The event development tracing includes: sorting events according to their timestamps to form a timeline, constructing a causal chain or temporal chain of events, and identifying key turning points.

5. The intelligent long document analysis method based on a large language model and graph database according to claim 1, characterized in that, The process involves converting the analysis results of the multi-dimensional analysis task into visual charts or natural language summaries and outputting them; simultaneously, a large language model is invoked to compare the analysis results with the original long document, automatically generating original text paragraph indexes and content to support the analysis results, thus ensuring the traceability of the conclusions, including: The analysis results can be rendered into an interactive relationship graph, timeline, or comparison table, or a large language model can be called to summarize the graph query results into a natural language summary. The source tracing mechanism is activated simultaneously, and the large language model is called again to compare and analyze the conclusion with the original long document. Contextual evidence supporting the conclusion is extracted, and the corresponding original text paragraph index is generated to achieve closed-loop verification of the conclusion and evidence.

6. The intelligent long document analysis method based on a large language model and graph database according to claim 2, characterized in that, The sequential execution of multiple rounds of structured information extraction tasks includes: Send the first prompt word, instructing the large language model to extract all entities and their attributes and return them in a preset format; Send the second prompt word to instruct the large language model to extract all unique relation type names; Send a third prompt word to dynamically embed the entity list and the relation type list, and instruct the large language model to construct specific relation triples between entities; Sending the fourth prompt word instructs the large language model to identify and construct a hierarchical relationship system in the long document based on the triples.

7. A long document intelligent analysis system based on large language models and graph databases, characterized in that: include: Document preprocessing module: responsible for semantically segmenting the long document to be analyzed and preparing it for input into the large language model; Language model invocation module: responsible for sending dynamic prompt words to the large language model, performing multi-stage information extraction, and receiving the returned structured triple data; Graph Database Interaction Module: Responsible for batch importing the generated triple data into the graph database, performing conflict detection and attribute fusion, constructing a dynamic knowledge graph and performing graph query operations; Multidimensional analysis engine module: responsible for calling the graph query language of the graph database, performing multidimensional analysis tasks such as interpersonal relationship networks and event tracking, and generating analysis results; Results presentation module: responsible for converting analysis results into visual charts or natural language summaries and outputting them to the user; The source tracing module is used to call a large language model to compare the analysis results with the original document and generate paragraph indexes and content that support the conclusions.

Images