A Question Answering Method and System for Structured Long Documents

Abstract

The present invention relates to the technical field of large language models, and specifically provides a question-answering method for structured long documents, which includes the following steps: S1, Parse documents in different formats, and construct structured metadata of the documents according to the parsing results; S2, Divide the documents into multiple text segments, perform vectorization processing on each text segment, and store them in a dedicated vector database; S3, Construct multiple text content acquisition tools respectively for extracting text content from different parts of the documents; Design and implement a vector-based retrieval tool for finding text segments related to the user's query in the vector database; S4, Construct an Agent that includes multiple text content acquisition tools and retrieval tools, and intelligently select text content acquisition tools or retrieval tools for the user's question to obtain relevant text content required for the LLM to answer questions; S5, After obtaining the relevant text content, analyze the relevant text content through the LLM to generate a final answer.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and specifically to a question-answering method and system for structured long documents. Background Art

[0002] In today's technical field, large language models (LLMs) face a significant challenge when dealing with document question-answering tasks: when the length of the document exceeds the context length that the model can handle, its performance often drops significantly. To address this issue, existing research has mostly focused on retrieval enhancement techniques, that is, retrieving text fragments relevant to the question from the document and converting them into plain text for processing. However, in practical applications, documents such as PDF files, text files, and web pages often have rich structural features, including different page layouts, complex table forms, and clear chapter divisions. When users ask questions based on these structural features, existing plain text processing methods are difficult to effectively capture and utilize this information. For example, users may ask about the data relationships in a specific table or request a detailed explanation of a concept in a certain chapter. At this time, simple text conversion will lose a large amount of structural information, resulting in the model being unable to accurately locate the key to the question and thus unable to give a satisfactory answer. Therefore, how to effectively combine the structural information of the document for intelligent question-answering has become an important bottleneck restricting the further development of LLMs in document question-answering tasks. Summary of the Invention

[0003] One aspect of the present invention provides a question-answering method and system for structured long documents to solve the problems raised in the above background art.

[0004] To achieve the above object, in the first aspect, the present invention provides the following technical solutions:

[0005] A question-answering method for structured long documents includes the following steps:

[0006] S1. Parse documents in different formats and construct structured metadata of the document according to the parsing results;

[0007] S2. Divide the document into multiple text fragments, perform vectorization processing on each text fragment, and store them in a dedicated vector database;

[0008] S3. Construct multiple text content acquisition tools respectively for extracting text content from different parts of the document; design and implement a vector-based retrieval tool for finding text fragments relevant to the user query in the vector database;

[0009] S4. Construct an Agent that includes multiple text content acquisition tools and retrieval tools, and intelligently select a text content acquisition tool or a retrieval tool for the user's question to obtain the relevant text content required for the LLM to answer the question;

[0010] S5. After obtaining the relevant text content, use the LLM to analyze the relevant text content to generate a final answer.

[0011] Preferably, step S1 specifically includes:

[0012] S11. For PDF format documents, extract the position information and content of various elements such as pages, chapters, tables, and images; for DOC and DOCX format documents, obtain the text content and identify the chapter and paragraph divisions; for image format documents, convert the content in the image into editable text;

[0013] S12. Construct a structured metadata that includes the pages, chapters, tables, and images of the document based on the parsing results.

[0014] Preferably, step S2 specifically includes:

[0015] S21. The system automatically identifies the titles and paragraphs in the document, and reasonably divides the document according to semantics by analyzing the text font style and paragraph spacing;

[0016] S22. Use a text embedding model to encode each text segment in the document and convert it into a vector representation of a fixed dimension. The vectorization results of each text segment will be stored in a dedicated vector database.

[0017] Preferably, step S3 specifically includes:

[0018] S31. To handle different forms of information requirements in the document, multiple text content acquisition tools for specific document structures are constructed, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools;

[0019] S32. The retrieval tool retrieves in the vector database to find the text segment most relevant to the query.

[0020] Preferably, the page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; the chapter extraction tool is used to obtain the complete text of a specific chapter according to the title hierarchy; the table extraction tool is used to extract the row and column data and header information of the table according to the table identifier in the user's question; the image extraction tool is used to extract the text content and its context description in the image according to the specified image identifier in the user's question.

[0021] Preferably, step 32 specifically includes:

[0022] The retrieval tool performs text preprocessing on the user's question, including removing noise, tokenization, and removing stop words;

[0023] By using a text embedding model, the user's query is converted into a vector representation;

[0024] Using this vector representation, a retrieval is performed in the vector database to find the text fragments most relevant to the query for subsequent processing and answer generation.

[0025] Preferably, the construction of the Agent including the above tools in step S4 includes:

[0026] S41. Use the ReAct framework to construct an Agent that includes multiple text content acquisition tools and retrieval tools;

[0027] S42. The Agent semantically analyzes the user's question through the LLM to identify the core information and structural requirements of the question;

[0028] S43. According to the semantic analysis result, the Agent dynamically selects tools to complete information acquisition. When the user's question contains structured features such as page numbers, chapter title levels, table identifiers, or image identifiers, the Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool, or image extraction tool; for unstructured semantic queries, it will select a vector-based retrieval tool to achieve precise matching in the semantic space through the text embedding model;

[0029] S44. The Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate the information.

[0030] Preferably, the ReAct framework includes:

[0031] Enhance the original action space A through the language model space L, and the enhanced action space ;

[0032] For the actions from the language model space , the Agent will perform reasoning based on the current environment to obtain useful information, and then support subsequent reasoning and action execution; based on the weight-cured language model, use the tool definition and examples of problem-solving given in the prompt words to generate the thought and action sequences for problem-solving.

[0033] Preferably, step S5 specifically includes:

[0034] After obtaining the relevant text content, the LLM performs in-depth understanding and reasoning processing based on the specific questions of the user and in combination with the extracted document content to generate answers that meet the semantic requirements.

[0035] In a second aspect, the present invention provides a question-answering system for structured long documents, which is applicable to the question-answering method for structured long documents as described above, and includes:

[0036] A document parsing module: used to parse documents in different formats and construct structured metadata of the documents;

[0037] A document partitioning module: used to partition a document into multiple text segments, perform vectorization processing on each text segment, and store them in a dedicated vector database;

[0038] An Agent: includes multiple text content acquisition tools and retrieval tools, and is used to intelligently select a text content acquisition tool or a retrieval tool for a user question to obtain the relevant text content required for the LLM to answer the question;

[0039] An LLM: used to generate answers that meet the semantic requirements after obtaining the relevant text content.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] By constructing metadata containing multi-dimensional structural information such as document pages, chapters, images, and tables, the present invention can effectively utilize the structural information of the document. Compared with traditional methods, this technology can more accurately understand and process complex document structures, and constructs multiple text content acquisition tools for obtaining text content of pages, chapters, images, and tables, and can select the retrieval scope and content according to the specific requirements of the user question; by constructing an Agent containing multiple text content acquisition tools and retrieval tools, this Agent can automatically classify the user question and call the corresponding text content acquisition tool or retrieval tool, which can not only improve the accuracy of the answer, but also flexibly adjust the workflow according to different types of question-answering requirements to provide more personalized and accurate answers. Users can not only obtain answers through traditional question-answering methods, but also ask questions about certain specific content based on the structure of the document, enhancing the flexibility and convenience of the interaction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a question-answering method for structured long documents provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of structured metadata of a document in a question-answering method for structured long documents provided by an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of generating a thought and action sequence for problem solving in a question - answering method for structured long documents provided for an embodiment of the present invention. Detailed implementation manners

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] Figure 1 Flow schematic diagram of a question - answering method for structured long documents provided for an embodiment of the present invention. As Figure 1 shown, the embodiment of the present invention provides a question - answering method for structured long documents, including the following steps:

[0047] S1. Analyze documents in different formats, and construct structured metadata of the documents according to the analysis results;

[0048] S2. Divide the document into multiple text segments, perform vectorization processing on each text segment, and store them in a dedicated vector database;

[0049] S3. Construct multiple text content acquisition tools respectively for extracting text content in different parts of the document; design and implement a vector - based retrieval tool for finding text segments related to the user's query in the vector database;

[0050] S4. Construct an Agent including multiple text content acquisition tools and retrieval tools, and intelligently select text content acquisition tools and retrieval tools for the user's question to obtain relevant text content required for the LLM to answer the question;

[0051] S5. After obtaining the relevant text content, generate a final answer through the LLM by analyzing the relevant text content.

[0052] Figure 2 Schematic diagram of the structured metadata of the document in the question - answering method for structured long documents provided for an embodiment of the present invention. In an embodiment of the present invention, step S1 specifically includes:

[0053] S11. For PDF - format documents, extract the position information and content of various elements such as pages, chapters, tables, and images; for DOC and DOCX - format documents, obtain the text content and identify chapter and paragraph divisions; for image - format documents, convert the content in the image into editable text;

[0054] S12. Construct a structured metadata that includes the pages, chapters, tables, and images of the document according to the parsing results.

[0055] An example of the structured metadata is as Figure 2 shown. By constructing metadata that includes multi-dimensional structural information such as document pages, chapters, images, and tables, the present invention can effectively utilize the structural information of the document. Compared with traditional methods, this technology can more accurately understand and process complex document structures, and constructs multiple text content acquisition tools for obtaining the text content of pages, chapters, images, and tables, and can select the retrieval scope and content according to the specific requirements of the user's question.

[0056] In an embodiment of the present invention, step S2 specifically includes:

[0057] S21. The system automatically identifies the titles and paragraphs in the document, and reasonably segments the document (such as paragraphs, sentences, etc.) according to semantics by analyzing the text font styles (titles usually have specific font sizes, bold, etc.) and paragraph spacing;

[0058] S22. Use a text embedding model to encode each text segment in the document and convert it into a vector representation of a fixed dimension. The vectorization results of each text segment will be stored in a dedicated vector database. This can achieve fast matching and efficient retrieval in subsequent query processes to accurately find the content relevant to the user's question.

[0059] In an embodiment of the present invention, step S3 specifically includes:

[0060] S31. To address different forms of information requirements in the document, multiple text content acquisition tools for specific document structures are constructed, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools;

[0061] S32. The retrieval tool retrieves in the vector database to find the text segment most relevant to the query.

[0062] Furthermore, the page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; for editable documents (such as PDF / DOCX), use a parsing library (such as PyMuPDF, python-docx) to directly read the physical page structure of the document and extract the text according to the page number index; for scanned PDF / image documents, use OCR technology to identify the image-based page content;

[0063] The chapter extraction tool is used to obtain the complete text of a specific chapter according to the title hierarchy; the chapter extraction tool first obtains the starting page and ending page of the chapter by parsing the structured metadata; combines the formatting rules (such as title font styles, numbering patterns) with the semantic model to extract all the text content of the chapter;

[0064] The table extraction tool is used to extract the row and column data and header information of a table according to the table identifier in the user's question; the table extraction tool first locates the page where the table is located by parsing the structured metadata in S11; for editable documents, directly parse the table tags (such as those in DOCX

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] t

[0077] Figure 3

[0078]

[0079] Figure 3

[0080]

[0081] Figure 1

[0082]

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] Element); for unstructured documents, use deep learning models or rule engines to detect table areas and convert them into structured data; the image extraction tool is used to extract the text content and its context description within the image according to the image identifier specified in the user's question; the image extraction tool first locates the page where the image is located and the associated title by parsing the structured metadata in S11; subsequently, it uses OCR technology to identify the text in the image and returns the OCR results and the context information related to the picture (such as the image title, adjacent paragraph text). Further, step 32 specifically includes: the retrieval tool performs text preprocessing on the user's question, including removing noise, tokenization, and removing stop words; by using a text embedding model, the user's query is converted into a vector representation; using this vector representation to perform retrieval in the vector database to find the text segment most relevant to the query for subsequent processing and generating answers. In an embodiment of the present invention, the construction of the Agent including the above tools in step S4 includes: S41. Use the ReAct framework to construct an Agent including multiple text content acquisition tools and retrieval tools; S42. The Agent performs semantic parsing on the user's question through the LLM to identify the core information and structural requirements of the question; S43. According to the semantic parsing result, the Agent dynamically selects tools to complete information acquisition, specifically manifested as: when the user's question contains structured features such as page numbers, chapter title levels, table identifiers, or image identifiers, the Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool, or image extraction tool; for unstructured semantic queries, it selects a vector-based retrieval tool to achieve precise matching in the semantic space through a text embedding model; S44. The Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate the information. This Agent can automatically select the appropriate text content acquisition tool or retrieval tool according to the question raised by the user for document content extraction and retrieval, and integrate the extracted content and provide it to the LLM to generate the final answer. The general process of the traditional Agent interacting with the environment is that at time step, the environmental observation received by the Agent is, and then it takes corresponding actions based on the policy, where is the context of the Agent. The learning process of the policy generally uses the policy gradient algorithm, and this process involves a large amount of computation. A schematic diagram of generating a thought and action sequence for solving problems in a question-answering method for structured long documents provided by an embodiment of the present invention.In an embodiment of the present invention, the ReAct framework includes: enhancing the original action space A through the language model space L to obtain an enhanced action space; for actions from the language model space, the Agent will reason based on the current environment to obtain useful information, thereby supporting subsequent reasoning and action execution; a language model based on weight solidification. As shown, the thought and action sequences for problem-solving are generated by giving tool definitions and examples of problem-solving in the prompt words. Specifically, the Agent first semantically parses the user's question through the LLM to identify the core information and structural requirements of the question. Specifically, the LLM will perform natural language understanding on the user's question to identify the intent, entities, and keywords of the question. For example, if the user asks "Please explain the table data on page 5", the Agent will identify the two key pieces of information, "page 5" and "table data". Then, based on the semantic parsing result of the question, the Agent will further analyze the user's requirements for the document structure, such as whether it is necessary to extract the content of a specific page, the text of a certain chapter, or the data of a certain table; after the question is parsed, the Agent will select an appropriate text content acquisition tool according to the parsing result to extract the document content. The logic for selecting the text content acquisition tool is based on the specific structural information mentioned in the user's question. For example, if the page number is clearly mentioned in the question (such as "page 5"), the Agent will select a page extraction tool; if the chapter title or chapter number is mentioned in the question (such as "Detailed Implementation Manner"), the Agent will select a chapter extraction tool; if the table identifier is mentioned in the question (such as "Table 1"), the Agent will select a table extraction tool; if the image identifier is mentioned in the question (such as ""), the Agent will select an image extraction tool; after the text content acquisition tool is selected, the Agent will call the corresponding text content acquisition tool to extract relevant content from the document. If the user's question involves multiple parts (such as requiring both chapter content and table data), the Agent will switch between different tools and integrate the information. For example, if the question requires analyzing a certain table in combination with the content of the second chapter, the Agent will first call the chapter extraction tool to obtain the text content of the second chapter, and then call the table extraction tool to obtain the data of that table. Then, the Agent will integrate the content extracted by different tools to form a complete reference material. This Agent design based on the ReAct framework improves the flexibility and accuracy of the question-answering system compared to traditional retrieval enhancement algorithms, and can also dynamically adjust the workflow according to different types of question-answering requirements to provide more personalized and accurate answers. In an embodiment of the present invention, step S5 specifically includes: after the LLM obtains the relevant text content, according to the user's specific question, combined with the extracted document content, it performs in-depth understanding and reasoning processing to generate an answer that meets the semantic requirements.For example, for the question "Please explain the tabular data on page 5 and analyze it in combination with the content of the second chapter", the Agent will extract the tabular data on page 5 and the text content of the second chapter, and integrate this information and pass it to the LLM to generate a detailed explanation combining the tabular data and the chapter text. The embodiments of the present invention provide a question-answering system for structured long documents, applicable to the question-answering method for structured long documents as described above, including: a document parsing module: for parsing documents in different formats and constructing the structured metadata of the documents; a document partitioning module: for partitioning the document into multiple text segments, performing vectorization processing on each text segment and storing it in a dedicated vector database; an Agent: including multiple text content acquisition tools and retrieval tools, for intelligently selecting text content acquisition tools or retrieval tools for user questions to obtain the relevant text content required for the LLM to answer questions; an LLM: for generating an answer that meets the semantic requirements after obtaining the relevant text content. The question-answering system for structured long documents provided by the embodiments of the present invention builds an Agent including multiple text content acquisition tools and retrieval tools, which can automatically classify user questions and call the corresponding text content acquisition tools or retrieval tools, not only improving the accuracy of answers, but also flexibly adjusting the workflow according to different types of question-answering requirements to provide more personalized and accurate answers. Users can not only obtain answers through traditional question-answering methods, but also ask questions about certain specific content based on the structure of the document, enhancing the flexibility and convenience of the interaction process. Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A question answering method for structured long documents, characterized in that, Including the following steps: S1. Parse documents in different formats and construct structured metadata of the documents according to the parsing results; S2. Divide the document into multiple text fragments, perform vectorization processing on each text fragment, and store them in a dedicated vector database; S3. Construct multiple text content acquisition tools respectively for extracting text content from different parts of the document; design and implement a vector-based retrieval tool for finding text fragments related to the user's query in the vector database; S4. Construct an Agent that includes multiple text content acquisition tools and retrieval tools, and intelligently select a text content acquisition tool or a retrieval tool for the user's question to obtain relevant text content required for the LLM to answer the question; S5. After obtaining the relevant text content, analyze the relevant text content through the LLM to generate a final answer; The construction of the Agent including the above tools described in step S4 includes: S41. Use the ReAct framework to construct an Agent that includes multiple text content acquisition tools and retrieval tools; S42. The Agent semantically parses the user's question through the LLM to identify the core information and structural requirements of the question; S43. The Agent dynamically selects tools to complete information acquisition according to the semantic parsing results. When the user's question contains structured features such as page numbers, chapter title levels, table identifiers, or image identifiers, the Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool, or image extraction tool; for unstructured semantic queries, select the vector-based retrieval tool to achieve precise matching in the semantic space through the text embedding model; S44. The Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate the information; The ReAct framework includes: Enhance the original action space A through the language model space L, and the enhanced action space ; For actions from the language model space , the Agent will reason based on the current environment to obtain useful information, thereby supporting subsequent reasoning and action execution; based on a weight-cured language model, tool definitions and examples of problem-solving are given in the prompt to generate a thought and action sequence for problem-solving.

2. The question-answering method for structured long documents according to claim 1, wherein Step S1 specifically includes: S11. For PDF format documents, extract the position information and content of various elements such as pages, chapters, tables, and images; for DOC and DOCX format documents, obtain the text content and identify chapter and paragraph divisions; for image format documents, convert the content in the image into editable text; S12. Construct a structured metadata including the pages, chapters, tables, and images of the document according to the parsing results.

3. The question-and-answer method for structured long documents according to claim 1, characterized in that Step S2 specifically includes: S21. The system automatically identifies the titles and paragraphs in the document, and reasonably segments the document according to semantics by analyzing the text font style and paragraph spacing; S22. Use a text embedding model to encode each text fragment in the document and convert it into a vector representation of a fixed dimension. The vectorization result of each text fragment will be stored in a dedicated vector database.

4. The question-answering method for structured long documents according to claim 1, characterized in that, Step S3 specifically includes: S31. To cope with different forms of information requirements in the document, multiple text content acquisition tools for specific document structures are constructed, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools; S32. The retrieval tool conducts a retrieval in the vector database to find the text fragment most relevant to the query.

5. The question-answering method for structured long documents according to claim 4, wherein The page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; the chapter extraction tool is used to obtain the complete text of a specific chapter according to the title hierarchy; the table extraction tool is used to extract the row and column data and the table header information of the table according to the table identifier in the user's question; the image extraction tool is used to extract the text content and its context description within the image according to the specified image identifier in the user's question.

6. The question-answering method for structured long documents according to claim 4, wherein Step 32 specifically includes: The retrieval tool performs text preprocessing on the user's question, including noise removal, tokenization, and stop word removal; By using a text embedding model, the user query is converted into a vector representation; Using this vector representation to conduct a retrieval in the vector database to find the text fragment most relevant to the query for subsequent processing and answer generation.

7. The question answering method for structured long documents according to claim 1, characterized in that, Step S5 specifically includes: After obtaining the relevant text content, the LLM performs in-depth understanding and reasoning processing according to the specific question of the user in combination with the extracted document content to generate an answer that meets the semantic requirements.

8. A question answering system for structured long documents, characterized in that, Applicable to the question-answering method for structured long documents described in any one of claims 1-7, including: The document parsing module: used to parse documents in different formats and construct the structured metadata of the documents; The document partitioning module: used to partition the document into multiple text fragments, perform vectorization processing on each text fragment, and store them in a dedicated vector database; Agent: contains multiple text content acquisition tools and retrieval tools, used to intelligently select text content acquisition tools for the user's question to obtain the relevant text content required for the LLM to answer questions; Construct an Agent using the ReAct framework that includes multiple text content acquisition tools and retrieval tools; the ReAct framework includes: enhancing the original action space A through the language model space L, and the enhanced action space ; for actions from the language model space , the Agent will perform reasoning based on the current environment to obtain useful information, thereby supporting subsequent reasoning and action execution; based on a weight-cured language model, use tool definitions and problem-solving examples in the prompt to generate a problem-solving thought and action sequence; LLM: used to generate an answer that meets the semantic requirements after obtaining the relevant text content.

Images