Question answering method and system based on logic graph enhanced unstructured long text

Abstract

The present application relates to a kind of unstructured long text question answering method and system based on logical atlas enhancement, combine semantic block clustering, logic relationship modeling, form semantic atlas enhanced question answering mechanism.The system is encoded to paragraph by Sentence-BERT, based on Louvain algorithm identifies multilevel semantic block, again by trained logic relationship classifier (for example, based on RoBERTa-large such as model fine-tuning) to the pair of semantic block is carried out multi-type logic relationship identification, constructs the structured logic atlas including cause and effect, support, contrast and the like, finally joint atlas representation and context input large language model generates structured, strong explanation answer, significantly improves long text in long-distance dependent understanding and cause and effect chain generation capability, applicable to annual report analysis, policy interpretation, scientific research literature and the like complex scene.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a method and system for answering unstructured long text questions based on logic graph enhancement. Background Technology

[0002] Among existing document question answering technologies, summary-based question answering (such as BART and Pegasus models) and RAG (retrieval augmentation generation) methods are currently the mainstream solutions, but they still face significant technical bottlenecks when dealing with complex document tasks. The specific challenges can be broken down as follows:

[0003] (1) The semantic fragmentation problem caused by fixed window truncation of long documents is prominent. Due to the limited length of the model input sequence, when processing long documents that exceed the window, it is necessary to truncate the text segments to a fixed length. This processing method is prone to severing the semantic connection between paragraphs. For example, in legal documents, the logical connection between the definition of the preceding clause and the applicable circumstances in the following clause may be truncated, causing the model to be unable to capture the remote dependency relationship between definition and application, which in turn leads to the omission of key related information in the question and answer results, or even semantic contradictions.

[0004] (2) The model lacks the ability to model semantic evolution and implicit logical chains. Document content often exhibits dynamic semantic evolution, such as the progressive process of hypothesis formulation, experimental verification, and conclusion derivation in scientific research papers, and contains implicit non-explicit logical chains, such as the implicit association between event background, indirect impact, and subsequent results in historical documents. Existing methods mostly focus on explicit keyword matching or local semantic extraction, failing to construct a dynamic modeling framework adapted to semantic evolution, and also failing to effectively identify and connect implicit logical nodes, resulting in question-and-answer results that only remain at the level of surface information extraction, making it difficult to form in-depth answers that conform to the logical context of the document.

[0005] (3) It does not support the generation of structured causal graphs driven by logical relationships, resulting in poor interpretability of the output. High-quality document Q&A requires a clear logical link between what the answer is based on and why the conclusion is reached, while existing methods only output natural language answers without constructing structured causal relationships. For example, in medical case Q&A, it is impossible to generate a causal graph of symptom A-indicator B-conclusion C. This unstructured output makes it impossible to trace the source of the answer's information and to explain the logical process of the conclusion derivation, leading users to doubt the credibility of the Q&A results, which is especially unsuitable for professional fields with high interpretability requirements.

[0006] In summary, there is currently a lack of a question-answering method for unstructured long texts that can solve or partially solve the aforementioned problems. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a question-answering method and system for unstructured long text based on logic graph enhancement, so as to solve or partially solve the problems of semantic silos, logical omissions and structural ambiguity in existing question-answering methods when facing unstructured long text scenarios.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] One aspect of the present invention provides an unstructured long text question answering method based on logic graph enhancement, comprising the following steps:

[0010] The original document is retrieved and split into paragraph semantic vector sequences based on line breaks and semantic punctuation.

[0011] Based on the semantic vector sequence of paragraphs, the similarity between each pair of paragraphs is calculated, and the similarity is filtered based on a dynamic threshold to construct a semantic association graph across paragraphs;

[0012] Based on the semantic association graph, multi-level semantic blocks are obtained through graph community detection and clustering.

[0013] The user query is obtained and vectorized to obtain the query vector. The similarity between the query vector and the center vector of each semantic block is calculated. The initial set of highly relevant semantic blocks is selected, and the layer to which the semantic block with the highest similarity belongs is taken as the target layer.

[0014] Based on the target hierarchy, starting from the initial set of semantic blocks, the logical relationships between semantic blocks and the extended set of semantic blocks are obtained through an iterative process of candidate search, relation identification, and condition expansion.

[0015] Traverse the extended semantic block set, using the text content of the semantic blocks as nodes and high-confidence logical relationships as edges to construct a logical graph;

[0016] Using the query vector and the logical graph as input to the large language model, the answer to the user query is obtained.

[0017] As a preferred technical solution, the process of splitting paragraph semantic vector sequences based on line breaks and semantic punctuation includes the following steps:

[0018] Based on the original document, it is converted into plain text through optical character recognition and / or parsing, and then split into paragraph sequences based on line breaks and semantic punctuation.

[0019] Based on the paragraph sequence, the vectors of each paragraph are calculated using the Sentence-BERT model and normalized to obtain the paragraph semantic vector sequence.

[0020] As a preferred technical solution, the process of obtaining the initial semantic block set and the target level includes the following steps:

[0021] The query vector is obtained by encoding the user's natural language query using the SBERT model.

[0022] Calculate the cosine similarity between the query vector and the center vector of each semantic block in each level, and select the top-K related semantic blocks as the initial semantic block set;

[0023] Search all levels for the semantic block with the highest similarity to the query vector, and use the corresponding level as the target level.

[0024] As a preferred technical solution, based on the target level, starting from the initial set of semantic blocks, the process of obtaining the logical relationships between semantic blocks and expanding the set of semantic blocks through an iterative process of candidate search, relation identification, and condition expansion includes the following steps:

[0025] Initialize collection queue to be explored ,in The initial set of semantic blocks, based on the current queue to be explored. and set Perform an iterative process of candidate search, relation identification, and condition expansion until the queue is reached. The set is empty at this time. To expand the semantic block set,

[0026] In the iteration of candidate search-relationship identification-condition expansion, for the queue Each semantic block in Perform the following steps:

[0027] Candidate Search: Target Hierarchy from Multi-Level Semantic Blocks In the middle, choose and Adjacent semantic blocks that are directly connected by an edge are added to the candidate semantic block list. ;

[0028] Relationship identification: For each If semantic block With semantic blocks Relationship Given an unknown object, its text pairs are input to a trained logical relation classifier, and the output relation category is... and confidence level ;

[0029] Condition extension: In response to identifying conditions that satisfy and The valid logical relationship, and Not yet The middle will Add to collection ,in The threshold value is used.

[0030] As a preferred technical solution, the process of constructing the logical graph includes the following steps:

[0031] Iterate through the extended semantic block set, for each semantic block Create a node object, where each node includes a semantic block. The complete text content will be used to add the generated node object to the node list of the graph;

[0032] The logical relationships obtained by traversing the identification are used to determine whether there is a relationship with a confidence level higher than a preset value. If so, an edge object is created. The edge object includes the identifier of the source semantic block, the identifier of the target semantic block, the logical relationship, and the confidence level. The edge object is added to the edge list of the graph to realize the construction of the logical graph.

[0033] As a preferred technical solution, the process of obtaining multi-level semantic blocks through graph community detection and clustering includes the following steps:

[0034] Based on the initial semantic association graph, through multiple rounds of Louvain graph community iteration, the result of each iteration is taken as a semantic block of one level, resulting in multi-level semantic blocks.

[0035] As a preferred technical solution, the semantic association graph is a sparse weighted undirected semantic graph, with paragraphs as nodes, filtered paragraph pairs as edges, and filtered similarity as weights.

[0036] As a preferred technical solution, the dynamic threshold is:

[0037]

[0038] in, These are the mean and standard deviation of the similarity, respectively. These are adjustment parameters used to control the sparsity of edges in the graph.

[0039] As a preferred technical solution, the similarity between each pair of paragraphs is the cosine similarity.

[0040] Another aspect of the present invention provides an unstructured long text question answering system based on logic graph enhancement, for implementing the aforementioned unstructured long text question answering method, the system comprising:

[0041] The document preprocessing and paragraph semantic vector encoding module is used to obtain the original document and split it into paragraph semantic vector sequences based on line breaks and semantic punctuation.

[0042] The cross-segment semantic similarity graph construction module is used to calculate the similarity between every two paragraphs based on the paragraph semantic vector sequence, filter the similarity based on a dynamic threshold, and construct a cross-segment semantic association graph.

[0043] Semantic block dynamic clustering and segmentation is used to obtain multi-level semantic blocks based on the semantic association graph through graph community detection clustering.

[0044] The user query vectorization and semantic block retrieval module is used to obtain user queries and perform vectorization processing to obtain query vectors, calculate the similarity between the query vectors and the center vectors of each semantic block, filter the initial set of highly relevant semantic blocks, and take the level to which the semantic block with the highest similarity belongs as the target level.

[0045] Dynamic multi-type relation identification and expansion is used to obtain the logical relations between semantic blocks and the expanded semantic block set based on the target hierarchy, starting from the initial set of semantic blocks, through an iterative process of candidate search-relation identification-condition expansion;

[0046] The logical graph construction and serialization module is used to traverse and expand the set of semantic blocks, using the text content of the semantic blocks as nodes and high-confidence logical relationships as edges to construct a logical graph.

[0047] The large language model question-answering generation module uses the query vector and the logical graph as input to the large language model to obtain answers to user queries.

[0048] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0049] (1) Overcoming semantic fragmentation: In response to the semantic island problem in existing solutions, this invention segments the input text, constructs a semantic association graph across paragraphs, and obtains multi-level semantic blocks through graph community detection and clustering. This aggregates semantically related non-continuous paragraphs into coherent semantic blocks, thus overcoming the semantic fragmentation caused by fixed window truncation.

[0050] (2) Fully identify the logical relationships between paragraphs: In response to the problem of logical deficiencies in existing solutions, this invention obtains a highly relevant initial set of semantic blocks and a target level based on multi-level semantic blocks. Through an iterative process of candidate search, relationship identification, and condition expansion, the logical relationships between semantic blocks and the expanded set of semantic blocks are obtained. In the iteration, multiple types of implicit logical relationships between semantic blocks are obtained through logical relationship identification, and the complex logical chain inside the document is reconstructed based on the graph propagation algorithm, providing a deep reasoning basis for question answering.

[0051] (3) The generated answer structure is clear: In response to the problem of structural ambiguity in existing solutions, this invention traverses and expands the set of semantic blocks, uses the text content of the semantic blocks as nodes and high-confidence logical relationships as edges to construct a logical graph. The query vector and the logical graph are used as inputs to the large language model to obtain the answer to the user query. By using the logical graph as an enhanced context input to the large language model, the generated answer structure is clear, logically rigorous, and highly interpretable, effectively suppressing model illusion. Attached Figure Description

[0052] Figure 1 This is a flowchart of the unstructured long text question answering method based on logic graph enhancement in the embodiment;

[0053] Figure 2 This is a schematic diagram of the input stage in the embodiment;

[0054] Figure 3 This is a schematic diagram illustrating clustering and detection in the embodiment;

[0055] Figure 4 This is a schematic diagram of the logic graph construction in the embodiment;

[0056] Figure 5 This is a schematic diagram illustrating the generation of answers in the example. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0058] Example 1

[0059] To address the problems existing in the aforementioned prior art, this embodiment provides an unstructured long text question answering method based on logical graph enhancement. It aims to enhance question answer generation through cross-segment semantic clustering and multi-granularity semantic block modeling, cross-semantic block logical relationship identification and graph construction, and graph and context fusion, so as to output answers with strong semantic relevance, logical modeling ability, and interpretability.

[0060] See Figure 1 The method includes the following steps:

[0061] Step S1: Document preprocessing and paragraph semantic vector encoding.

[0062] Get the original document After being converted into plain text by Optical Character Recognition (OCR) or a parser, it is split into paragraph sequences based on line breaks and semantic punctuation.

[0063]

[0064] Sentence-BERT is used to calculate paragraph vectors:

[0065]

[0066] in This indicates that the MPNet model supports words. The embedded vector is used to obtain the overall paragraph vector through average pooling. .

[0067] To ensure uniform vector scaling, the vectors for each paragraph are normalized:

[0068]

[0069] Step S2: Construction of cross-segment semantic similarity graph and adaptive threshold setting.

[0070] Specifically, this includes steps S201-S203:

[0071] Step S201: Construct a paragraph similarity matrix.

[0072] Calculate the cosine similarity matrix between all pairs of paragraphs.

[0073]

[0074] Step S202: Set the dynamic threshold.

[0075]

[0076] in, These are the mean and standard deviation of the similarity, respectively. The parameters are adjusted to control the sparsity of the edges in the graph.

[0077] Step S203: Construction of cross-paragraph semantic association graph.

[0078] Retain satisfaction Construct a sparse weighted undirected semantic graph using the edges. This reflects the semantic relationship structure across paragraphs within a document.

[0079] in Corresponding document paragraphs, For the corresponding edge set,

[0080] Step S3: Dynamic clustering and segmentation of semantic blocks.

[0081] Breaking through the limitations of fixed order and length, the Louvain graph community detection algorithm is used to cluster semantic blocks in the semantic graph structure, outputting multi-level semantic blocks:

[0082]

[0083] For clustering and grouping, This represents the clustering hierarchy. Different clustering levels satisfy both fine-grained and coarse-grained requirements, and support the dynamic selection of appropriate semantic blocks during queries.

[0084] Specifically, starting from the initial graph G, use L The Louvain graph community is iterated through rounds, and the results of each round of iteration are output. L Layer semantic blocks.

[0085]

[0086] Step S4: User query vectorization and semantic block retrieval

[0087] User natural language query Encoded as a vector using the SBERT model:

[0088]

[0089] from L In the hierarchical semantic block structure, semantic blocks at the appropriate level are adaptively selected for retrieval, and the query center vector is calculated for each semantic block. Similarity:

[0090]

[0091] Select the top-K relevant semantic block set , as the initial set of semantic blocks.

[0092] from L In the layered semantic block structure, the target level is adaptively selected for retrieval. The adaptive selection process specifically involves: first, calculating the query vector q and the relationship between the query vector q and each level (l = 1, 2, ..., ...). L Each semantic block in ) The cosine similarity sim(q, ) of the center vectors Then, find the semantic block with the highest similarity to the query vector q across all levels. Ultimately, the semantic blocks with the highest similarity will be selected. To which level Determined as target level .

[0093] Step S5: Dynamic multi-type relation identification and expansion based on logical propagation.

[0094] From the initial semantic block Starting from this point, the logical relationships are identified and the set is expanded through an iterative process to cover the complete logical chain.

[0095] Specifically, including:

[0096] (1) Initialization: Let the extended set be queue to be explored

[0097] (2) Iterative expansion: For queues Each semantic block in Perform the following operations:

[0098] 1. Candidate Search: Based on the semantic graph structure constructed above, from the target level... In the middle, choose and Adjacent semantic blocks that are directly connected by an edge are added to the candidate semantic block list. .

[0099] 2. Relationship identification: For each If the relationship If the text is unknown, it is input into a trained logical relation classifier (such as a classification model fine-tuned based on a pre-trained model like RoBERTa-large), and the output is the relation category. and confidence level .

[0100] 3. Condition expansion: If a valid logical relationship is identified ( ), and it has not yet been in In the middle, then Add to sets and queues.

[0101]

[0102] (3) Termination: When the queue The iteration terminates when the value is empty. This is the final set of extended logical semantic blocks, and all valid logical relationships within it have been obtained.

[0103] Step S6: Logical graph construction and serialization.

[0104] The expanded semantic block set For each block and the logical relationship identified in step S5, a structured directed graph is constructed. As a logical graph .

[0105] Specifically, traverse the extended logical semantic block set. For each semantic block in the set Create a node object, with each node containing the complete text content of the semantic block. Add the generated node objects to the nodes list of the graph; iterate through the logical relationships identified in step S5. If a relationship with a confidence level higher than a preset threshold exists, create an edge object and populate it with source_id (corresponding to...). ), target_id (corresponding) Logical relationships and confidence levels Add the edge object to the graph's edges list. Finally, complete the logical graph. Serialization to JSON format facilitates input of large language models.

[0106] Step S7: Generate question-and-answer format using the large language model.

[0107] Combining query and logic graph It calls a large model to generate question-and-answer answers and outputs natural language responses and logic graph visualization results.

[0108] The following concrete example will provide a more detailed explanation of this method. See also... Figures 2-5 It includes the input phase, clustering and detection, logical graph construction, and answer generation.

[0109] (1) Input stage

[0110] See Figure 2 It includes the following steps:

[0111] 1. Input the original annual report PDF, and through OCR and segmentation, obtain the following paragraphs (example of excerpted paragraph content):

[0112] Paragraph 1: In 2023, the company achieved operating revenue of xx billion yuan, a year-on-year increase of 5%.

[0113] Paragraph 2: Affected by the sharp rise in raw material prices, the company's operating costs increased by 20% year-on-year.

[0114] Paragraph 3: Sales expenses increased by 15% due to increased marketing activities, while administrative expenses remained stable.

[0115] Paragraph 4: This year's net profit is XX billion yuan, a decrease of 30% compared to the previous year, mainly due to cost pressures.

[0116] Paragraph 5: The Board believes that the current macroeconomic environment poses challenges to the industry and that cost control needs to be strengthened.

[0117] 2. Obtain the normalized paragraph vector through SBERT encoding.

[0118] 3. Calculate the pairwise cosine similarity of the paragraphs to obtain the similarity matrix:

[0119] sim(paragraph 1, paragraph 2) = 0.5095

[0120] sim(paragraph 1, paragraph 3) = 0.5177

[0121] sim(paragraph 1, paragraph 4) = 0.6476

[0122] sim(paragraph 1, paragraph 5) = 0.4234

[0123] sim(paragraph 2, paragraph 3) = 0.6227

[0124] sim(paragraph 2, paragraph 4) = 0.5657

[0125] sim(paragraph 2, paragraph 5) = 0.5468

[0126] sim(paragraph 3, paragraph 4) = 0.5466

[0127] sim(paragraph 3, paragraph 5) = 0.5348

[0128] sim(paragraph 4, paragraph 5) = 0.5439

[0129] 4. Through dynamic threshold ( ) Filter and keep the edges (paragraph 1, paragraph 3), (paragraph 1, paragraph 4), (paragraph 2, paragraph 3), (paragraph 2, paragraph 4), (paragraph 2, paragraph 5), (paragraph 3, paragraph 4), (paragraph 3, paragraph 5), (paragraph 4, paragraph 5).

[0130] 5. Construct a sparse weighted undirected semantic association graph.

[0131] (2) Clustering and Retrieval

[0132] See Figure 3 It includes the following steps:

[0133] 1. Based on the sparse weighted undirected semantic association graph obtained in step (1), multi-level semantic blocks are obtained through Louvain multi-round clustering:

[0134] Level 1: {Paragraphs 1, 2, 3, 4, 5}

[0135] Level 2: {Block A = [Paragraph 1, 4], Block B = [Paragraph 2, 3], Block C = [Paragraph 5]}

[0136] 2. Obtain user queries for "reasons for the decline in net profit".

[0137] 3. Obtain the query vector through SBERT encoding.

[0138] 4. Calculate the cosine similarity between the query vector q and the center vectors of each semantic block:

[0139] The highest similarity score for Level 1 (paragraph) is sim(q, paragraph 4) = 0.85.

[0140] The highest similarity at level 2 (semantic block) is sim(q, block A) = 0.91.

[0141] 5. Selecting Top-K initial blocks and determining the target level:

[0142] Initial block: {block A}

[0143] Target level: Level 2 (because the highest global similarity of 0.91 appears in this level).

[0144] (3) Logical graph construction

[0145] See Figure 4 It includes the following steps:

[0146] 1. Initialize based on the initial block and target level obtained in step (2): Extend set = {block A}, queue = [block A].

[0147] 2. Check if the queue is empty. If not, proceed to step 3. If yes, proceed to step 8.

[0148] 3. Retrieve blocks from the queue and search for candidates: In the target hierarchy, find semantic blocks adjacent to the current block:

[0149] Block A's adjacent block: Block B.

[0150] 4. Relationship identification (if the relationship is unknown): Input (block A text, block B text) to the RoBERTa logical relation classifier, output block B-causality-block A, confidence = 0.92.

[0151] 5. Determine if the confidence level is greater than the threshold. If yes, proceed to step 6; otherwise, proceed to step 2.

[0152] 6. Add block B to the extended set and queue:

[0153] Extended set = {block A, block B}.

[0154] 7. Execute step 2.

[0155] 8. Traverse the expanded collection and create nodes:

[0156] Node 1 (Block A)

[0157] Node 2 (Block B).

[0158] 9. Traverse logical relationships and create directed edges:

[0159] Edge: Block B - Causal (0.92) - Block A

[0160] 10. Serialize the logical graph (JSON format).

[0161] (4) Answer generation

[0162] See Figure 5 It includes the following steps:

[0163] 1. Based on the serialized logic graph obtained in step (3), construct a large language model (LLM). Input: query + logic graph:

[0164] Query: Reasons for the decline in net profit

[0165] Graph: {nodes[…], edges: [block B - block A]}

[0166] 2. Generate structured, explanatory answers:

[0167] The main reason for the decline in net profit was the significant increase in operating costs (+20%), which constitutes a direct causal relationship.

[0168] This method significantly improves the performance of unstructured long documents in deep question answering scenarios, achieving the following results:

[0169] (1) Overcoming the problem of semantic silos: Through semantic graph community discovery, semantically related non-continuous paragraphs are aggregated into coherent semantic blocks, overcoming the semantic fragmentation caused by fixed window truncation.

[0170] (2) Overcoming the problem of missing logic: By using a trained logical relationship classifier to identify multiple types of implicit logical relationships between semantic blocks, and by reconstructing the complex logical chain inside the document based on the graph propagation algorithm, it provides a deep reasoning basis for question answering.

[0171] (3) Overcoming structural ambiguity: Using the logic graph as an enhanced context input to the large language model makes the generated answer structure clear, logically rigorous, and highly interpretable, effectively suppressing model illusion.

[0172] Example 2

[0173] Based on Example 1, this example provides an unstructured long text question answering system based on logic graph enhancement, used to implement the unstructured long text question answering method of Example 1. The system includes:

[0174] (1) Document preprocessing and paragraph semantic vector encoding module, used to obtain the original document and split it into paragraph semantic vector sequences based on line breaks and semantic punctuation.

[0175] (2) Cross-segment semantic similarity graph construction module, which is used to calculate the similarity between each two paragraphs based on the paragraph semantic vector sequence, filter the similarity based on the dynamic threshold, and construct a cross-segment semantic association graph.

[0176] (3) Semantic block dynamic clustering and segmentation, which is used to obtain multi-level semantic blocks by clustering through graph community detection based on the semantic association graph.

[0177] (4) User query vectorization and semantic block retrieval module, used to obtain user query and perform vectorization processing to obtain query vector, calculate the similarity between the query vector and the center vector of each semantic block, filter the initial set of highly relevant semantic blocks, and take the level to which the semantic block with the highest similarity belongs as the target level.

[0178] (5) Dynamic multi-type relation identification and expansion, which is used to obtain the logical relationship between semantic blocks and the expanded semantic block set based on the target level, starting from the initial semantic block set, through an iterative process of candidate search-relation identification-condition expansion.

[0179] (6) Logical graph construction and serialization module, used to traverse and expand the set of semantic blocks, using the text content of the semantic blocks as nodes and the logical relationships with high confidence as edges to construct a logical graph.

[0180] (7) The large language model question answering generation module uses the query vector and the logical graph as input to the large language model to obtain the answer to the user query.

[0181] In summary, this invention combines semantic block clustering, logical relationship modeling, and semantic graph enhancement to create a question-answering mechanism. The system encodes paragraphs using Sentence-BERT, identifies multi-level semantic blocks based on the Louvain algorithm, and then uses a trained logical relationship classifier (e.g., fine-tuned based on RoBERTa-large models) to identify multiple types of logical relationships between paired semantic blocks. This constructs a structured logical graph containing causal, supporting, and contrastive relationships. Finally, the system combines the graph representation with a large language model based on the context input to generate structured and highly interpretable answers. This significantly improves the ability to understand long-range dependencies and generate causal chains in long texts, making it suitable for complex scenarios such as annual report analysis, policy interpretation, and scientific literature.

[0182] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A question-answering method for unstructured long texts based on logical graph enhancement, characterized in that, Includes the following steps: The original document is retrieved and split into paragraph semantic vector sequences based on line breaks and semantic punctuation. Based on the semantic vector sequence of paragraphs, the similarity between each pair of paragraphs is calculated, and the similarity is filtered based on a dynamic threshold to construct a semantic association graph across paragraphs; Based on the semantic association graph, multi-level semantic blocks are obtained through graph community detection and clustering. The user query is obtained and vectorized to obtain the query vector. The similarity between the query vector and the center vector of each semantic block is calculated. The initial set of highly relevant semantic blocks is selected, and the layer to which the semantic block with the highest similarity belongs is taken as the target layer. Based on the target hierarchy, starting from the initial set of semantic blocks, the logical relationships between semantic blocks and the extended set of semantic blocks are obtained through an iterative process of candidate search, relation identification, and condition expansion. Traverse the extended semantic block set, using the text content of the semantic blocks as nodes and high-confidence logical relationships as edges to construct a logical graph; Using the query vector and the logical graph as input to the large language model, the answer to the user query is obtained. Based on the target hierarchy, starting from the initial set of semantic blocks, the process of obtaining the logical relationships between semantic blocks and expanding the set of semantic blocks through an iterative process of candidate search, relation identification, and condition expansion includes the following steps: Initialize collection queue to be explored ,in The initial set of semantic blocks, based on the current queue to be explored. and set Perform an iterative process of candidate search, relation identification, and condition expansion until the queue is reached. The set is empty at this time. To expand the semantic block set, In the iteration of candidate search-relationship identification-condition expansion, for the queue Each semantic block in Perform the following steps: Candidate Search: Target Hierarchy from Multi-Level Semantic Blocks In the middle, choose and Adjacent semantic blocks that are directly connected by an edge are added to the candidate semantic block list. ; Relationship identification: For each If semantic block With semantic blocks Relationship Given an unknown object, its text pairs are input to a trained logical relation classifier, and the output relation category is... and confidence level ; Condition extension: In response to identifying conditions that satisfy and The valid logical relationship, and Not yet The middle will Add to collection ,in The threshold value is used.

2. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The process of splitting paragraph semantic vector sequences based on newline characters and semantic punctuation includes the following steps: Based on the original document, it is converted into plain text through optical character recognition and / or parsing, and then split into paragraph sequences based on line breaks and semantic punctuation. Based on the paragraph sequence, the vectors of each paragraph are calculated using the Sentence-BERT model and normalized to obtain the paragraph semantic vector sequence.

3. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The process of obtaining the initial semantic block set and the target hierarchy includes the following steps: The query vector is obtained by encoding the user's natural language query using the SBERT model. Calculate the cosine similarity between the query vector and the center vector of each semantic block in each level, and select the top-K related semantic blocks as the initial semantic block set; Search all levels for the semantic block with the highest similarity to the query vector, and use the corresponding level as the target level.

4. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The process of constructing the logical graph includes the following steps: Iterate through the extended semantic block set, for each semantic block Create a node object, where each node includes a semantic block. The complete text content will be used to add the generated node object to the node list of the graph; The logical relationships obtained by traversing the identification are used to determine whether there is a relationship with a confidence level higher than a preset value. If so, an edge object is created. The edge object includes the identifier of the source semantic block, the identifier of the target semantic block, the logical relationship, and the confidence level. The edge object is added to the edge list of the graph to realize the construction of the logical graph.

5. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The process of obtaining multi-level semantic blocks through graph community detection and clustering includes the following steps: Based on the initial semantic association graph, through multiple rounds of Louvain graph community iteration, the result of each iteration is taken as a semantic block of one level, resulting in multi-level semantic blocks.

6. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The semantic association graph is a sparse weighted undirected semantic graph, with paragraphs as nodes, filtered paragraph pairs as edges, and filtered similarity as weights.

7. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The dynamic threshold is: in, These are the mean and standard deviation of the similarity, respectively. These are adjustment parameters used to control the sparsity of edges in the graph.

8. The unstructured long text question answering method based on logical graph enhancement according to claim 1, characterized in that, The similarity between each pair of paragraphs is the cosine similarity.

9. A question-answering system for unstructured long texts based on logic graph enhancement, characterized in that, For implementing the unstructured long text question answering method as described in any one of claims 1-8, the system includes: The document preprocessing and paragraph semantic vector encoding module is used to obtain the original document and split it into paragraph semantic vector sequences based on line breaks and semantic punctuation. The cross-segment semantic similarity graph construction module is used to calculate the similarity between every two paragraphs based on the paragraph semantic vector sequence, filter the similarity based on a dynamic threshold, and construct a cross-segment semantic association graph. Semantic block dynamic clustering and segmentation is used to obtain multi-level semantic blocks based on the semantic association graph through graph community detection clustering. The user query vectorization and semantic block retrieval module is used to obtain user queries and perform vectorization processing to obtain query vectors, calculate the similarity between the query vectors and the center vectors of each semantic block, filter the initial set of highly relevant semantic blocks, and take the level to which the semantic block with the highest similarity belongs as the target level. Dynamic multi-type relation identification and expansion is used to obtain the logical relations between semantic blocks and the expanded semantic block set based on the target hierarchy, starting from the initial set of semantic blocks, through an iterative process of candidate search-relation identification-condition expansion; The logical graph construction and serialization module is used to traverse and expand the set of semantic blocks, using the text content of the semantic blocks as nodes and high-confidence logical relationships as edges to construct a logical graph. The large language model question-answering generation module uses the query vector and the logical graph as input to the large language model to obtain answers to user queries.

Images