Question and answer method and system for structured long document

Abstract

The invention relates to the technical field of large language models, in particular to a question and answer method for a structured long document, which comprises the following steps of: S1, analyzing documents in different formats, and constructing structured metadata of the documents according to analysis results; s2, dividing the document into a plurality of text segments, vectorizing each text segment, and storing the vectorized text segments in a special vector database; s3, respectively constructing a plurality of text content acquisition tools for extracting text contents of different parts in the document; designing and implementing a vector-based retrieval tool, wherein the vector-based retrieval tool is used for searching text fragments related to user query in a vector database; s4, constructing an Agent containing a plurality of text content acquisition tools and retrieval tools, and intelligently selecting the text content acquisition tools or the retrieval tools according to the user questions to acquire related text contents required by the LLM to answer the questions; and S5, after the related text content is acquired, analyzing the related text content through LLM to generate a final answer.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a question-answering method and system for structured long documents. Background Art

[0002] In today's technical field, large language models (LLMs) face a significant challenge when dealing with document question answering tasks: when the length of the document exceeds the context length that the model can handle, its performance often drops significantly. To address this problem, existing research mostly focuses on retrieval enhancement technology, that is, retrieving text fragments related to the question from the document and converting them into plain text for processing. However, in practical applications, documents such as PDF files, text files, and web pages often have rich structural features, including different page layouts, complex table forms, and clear chapter divisions. When users ask questions based on these structural features, existing plain text processing methods are difficult to effectively capture and utilize this information. For example, users may ask about the data relationship in a specific table, or request a detailed explanation of a concept in a certain chapter. At this time, simple text conversion will lose a lot of structural information, resulting in the model being unable to accurately locate the key to the problem and thus unable to give a satisfactory answer. Therefore, how to effectively combine the structural information of the document for intelligent question answering has become an important bottleneck restricting the further development of LLM in document question answering tasks. Summary of the invention

[0003] One aspect of the present invention provides a question-answering method and system for structured long documents to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above objectives, in a first aspect, the present invention provides the following technical solutions: A question answering method for a structured long document comprises the following steps: S1. Parse documents of different formats and construct structured metadata of the documents based on the parsing results; S2, dividing the document into multiple text segments, vectorizing each text segment, and storing them in a dedicated vector database; S3. Build multiple text content acquisition tools to extract text content from different parts of the document; design and implement vector-based retrieval tools to find text fragments related to user queries in the vector database; S4, building an agent that includes multiple text content acquisition tools and retrieval tools, intelligently selecting text content acquisition tools or retrieval tools according to user questions to obtain relevant text content required by LLM to answer questions; S5. After obtaining the relevant text content, the LLM is used to analyze the relevant text content and generate the final answer.

[0005] Preferably, step S1 specifically includes: S11. For documents in PDF format, extract the location information and content of various elements such as pages, chapters, tables, and images; for documents in DOC and DOCX format, obtain text content and identify chapter and paragraph divisions; for documents in image format, convert the content in the image into editable text; S12. Construct a structured metadata including the document's pages, chapters, tables, and images based on the parsing results.

[0006] Preferably, step S2 specifically includes: S21. The system automatically identifies titles and paragraphs in the document, analyzes the text font style and paragraph spacing, and divides the document into reasonable segments based on semantics; S22. Use the text embedding model to encode each text fragment in the document and convert it into a vector representation of fixed dimension. The vectorization result of each text fragment will be stored in a dedicated vector database.

[0007] Preferably, step S3 specifically includes: S31. In order to cope with different forms of information needs in documents, a number of text content acquisition tools for specific document structures were built, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools; S32. The search tool searches the vector database to find the text fragments most relevant to the query.

[0008] Preferably, the page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; the chapter extraction tool is used to obtain the complete text of a specific chapter according to the title hierarchy; the table extraction tool is used to extract the row and column data and header information of the table according to the table identifier in the user's question; the image extraction tool is used to extract the text content and its context description in the image according to the image identifier specified in the user's question.

[0009] Preferably, step 32 specifically includes: The search tool performs text preprocessing on user questions, including noise removal, tokenization, and stop word removal; Convert user queries into vector representations by using text embedding models; The vector representation is used to search in the vector database to find the text fragments most relevant to the query for subsequent processing and answer generation.

[0010] Preferably, the step S4 of constructing an Agent including the above-mentioned tools includes: S41. Use the ReAct framework to build an agent that includes multiple text content acquisition tools and retrieval tools; S42, Agent uses LLM to semantically analyze the user's questions and identify the core information and structural requirements of the questions; S43, Agent dynamically selects tools to complete information acquisition based on the semantic analysis results. When the user's question contains structured features such as page number, chapter title level, table mark or image mark, Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool or image extraction tool; for unstructured semantic queries, it selects vector-based retrieval tools to achieve accurate matching of semantic space through text embedding models; S44, the Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate the information.

[0011] Preferably, the ReAct framework includes: The original action space A is enhanced by the language model space L. ; For actions from the language model space , the Agent will be based on the current environment Reasoning is performed to obtain useful information, thereby supporting subsequent reasoning and action execution; based on the weighted language model, tool definitions and problem-solving examples are given in the prompt words to generate thought and action sequences for solving problems.

[0012] Preferably, step S5 specifically includes: After obtaining relevant text content, LLM conducts in-depth understanding and reasoning processing based on the user's specific questions and the extracted document content to generate answers that meet semantic requirements.

[0013] In a second aspect, the present invention provides a question-answering system for a structured long document, which is applicable to the question-answering method for a structured long document as described above, comprising: Document parsing module: used to parse documents of different formats and build structured metadata of documents; Document segmentation module: used to divide a document into multiple text segments, vectorize each text segment and store it in a dedicated vector database; Agent: contains multiple text content acquisition tools and retrieval tools, which are used to intelligently select text content acquisition tools or retrieval tools according to user questions to obtain relevant text content required by LLM to answer questions; LLM: Used to obtain relevant text content and generate answers that meet semantic requirements.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention can effectively utilize the structural information of the document by constructing metadata containing multi-dimensional structural information such as document pages, chapters, images, tables, etc. Compared with traditional methods, this technology can more accurately understand and process complex document structures, and construct multiple text content acquisition tools for acquiring the text content of pages, chapters, images, and tables, and can select the scope and content of retrieval according to the specific needs of user questions; by constructing an Agent containing multiple text content acquisition tools and retrieval tools, the Agent can automatically classify user questions and call corresponding text content acquisition tools or retrieval tools, which can not only improve the accuracy of answers, but also flexibly adjust workflows according to different types of question and answer requirements, and provide more personalized and accurate answers. Users can not only obtain answers through traditional question and answer methods, but also ask questions about certain specific content based on the structure of the document, which enhances the flexibility and convenience of the interactive process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a question-answering method for a structured long document provided by an embodiment of the present invention; Figure 2 A schematic diagram of document structured metadata in a question-answering method for a structured long document provided by an embodiment of the present invention; Figure 3 A schematic diagram of generating a thought and action sequence for solving a problem in a question-answering method for a structured long document provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Figure 1 The following is a flow chart of a question-answering method for a structured long document provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a question-answering method for a structured long document, comprising the following steps: S1. Parse documents of different formats and construct structured metadata of the documents based on the parsing results; S2, dividing the document into multiple text segments, vectorizing each text segment, and storing them in a dedicated vector database; S3. Build multiple text content acquisition tools to extract text content from different parts of the document; design and implement vector-based retrieval tools to find text fragments related to user queries in the vector database; S4, build an agent containing multiple text content acquisition tools and retrieval tools, and intelligently select text content acquisition tools and retrieval tools according to user questions to obtain relevant text content required by LLM to answer questions; S5. After obtaining the relevant text content, the LLM is used to analyze the relevant text content and generate the final answer.

[0018] Figure 2 A schematic diagram of document structured metadata in a question-answering method for structured long documents provided in an embodiment of the present invention. In one embodiment of the present invention, step S1 specifically includes: S11. For documents in PDF format, extract the location information and content of various elements such as pages, chapters, tables, and images; for documents in DOC and DOCX format, obtain text content and identify chapter and paragraph divisions; for documents in image format, convert the content in the image into editable text; S12. Construct a structured metadata including the document's pages, chapters, tables, and images based on the parsing results.

[0019] Examples of structured metadata include Figure 2 As shown. The present invention can effectively utilize the structural information of the document by constructing metadata containing multi-dimensional structural information such as document pages, chapters, images, tables, etc. Compared with traditional methods, this technology can more accurately understand and process complex document structures, and construct multiple text content acquisition tools for acquiring the text content of pages, chapters, images, and tables, which can select the scope and content of the retrieval according to the specific needs of the user's questions.

[0020] In one embodiment of the present invention, step S2 specifically includes: S21. The system automatically identifies titles and paragraphs in the document, analyzes the text font style (titles usually have specific font size, bold, etc.), paragraph spacing, and divides the document into reasonable segments (such as paragraphs, sentences, etc.) based on semantics; S22. Use the text embedding model to encode each text fragment in the document and convert it into a fixed-dimensional vector representation. The vectorization result of each text fragment will be stored in a dedicated vector database. This will enable fast matching and efficient retrieval in subsequent query processes, so as to accurately find content related to user questions.

[0021] In one embodiment of the present invention, step S3 specifically includes: S31. In order to cope with different forms of information needs in documents, a number of text content acquisition tools for specific document structures were built, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools; S32. The search tool searches the vector database to find the text fragments most relevant to the query.

[0022] Furthermore, the page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; for editable documents (such as PDF / DOCX), the parsing library (such as PyMuPDF, python-docx) is used to directly read the physical page structure of the document and extract text by page number index; for scanned PDF / image documents, the image page content is recognized through OCR technology; The chapter extraction tool is used to obtain the complete text of a specific chapter based on the title hierarchy. The chapter extraction tool first obtains the start and end pages of the chapter by parsing the structured metadata. It then extracts the entire text content of the chapter by combining formatting rules (such as title font style, numbering pattern) with the semantic model. The table extraction tool is used to extract the row and column data and header information of the table according to the table identifier in the user's question. The table extraction tool first locates the page where the table is located by parsing the structured metadata in S11. For editable documents, the table tag (such as DOCX's

[0023]

[0024]

[0025]

[0026] t

[0027] Figure 3 Figure 3

[0028] Figure 1

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] element); for unstructured documents, use a deep learning model or rule engine to detect table areas and convert them into structured data; the image extraction tool is used to extract the text content and its contextual description in the image according to the image identifier specified in the user's question; the image extraction tool first locates the page where the image is located and the associated title by parsing the structured metadata in S11; then the text in the image is recognized through OCR technology, and the OCR result and contextual information related to the image (such as the image title and adjacent paragraph text) are returned. Going further, step 32 specifically includes: the retrieval tool performs text preprocessing on the user's question, including noise removal, tokenization, and stop word removal; the user query is converted into a vector representation by using a text embedding model; the vector representation is used to search in the vector database to find the text fragment most relevant to the query for subsequent processing and answer generation. In one embodiment of the present invention, the construction of the Agent containing the above-mentioned tools described in step S4 includes: S41, using the ReAct framework to construct an Agent containing multiple text content acquisition tools and retrieval tools; S42, the Agent performs semantic analysis on the user's question through LLM to identify the core information and structural requirements of the question; S43, the Agent dynamically selects tools to complete information acquisition based on the semantic analysis results, which is specifically manifested as: when the user's question contains structured features such as page number, chapter title level, table mark or image mark, the Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool or image extraction tool; for unstructured semantic queries, a vector-based retrieval tool is selected to achieve accurate matching of semantic space through a text embedding model; S44, the Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate information. The Agent can automatically select a suitable text content acquisition tool or retrieval tool to extract and retrieve the document content according to the question raised by the user, and integrate the extracted content and provide it to LLM to generate the final answer. The process of traditional agent-environment interaction is generally that the agent receives the environmental observation at the time step , and then takes the corresponding action based on the strategy, where is the context of the agent. The strategy learning process generally uses the policy gradient algorithm, which involves a lot of calculations. A schematic diagram of generating a thought and action sequence for solving a problem in a question-answering method for a structured long document provided by an embodiment of the present invention.In one embodiment of the present invention, the ReAct framework includes: enhancing the original action space A through the language model space L, and enhancing the action space; for the action from the language model space, the agent will infer based on the current environment to obtain useful information, thereby supporting subsequent reasoning and action execution; based on the language model with weight solidification, as shown, the tool definition and problem-solving examples given in the prompt words are used to generate thought and action sequences for solving the problem. Specifically, the agent first performs semantic parsing on the user's question through LLM to identify the core information and structural requirements of the question. Specifically, LLM performs natural language understanding on the user's question to identify the intention, entity and keywords of the question. For example, if the user asks "Please explain the table data on page 5", the agent will identify the two key information "page 5" and "table data". Then, the agent will further analyze the user's requirements for the document structure based on the semantic parsing results of the question, such as whether it is necessary to extract the content of a specific page, the text of a chapter or the data of a table; after the question parsing is completed, the agent will select a suitable text content acquisition tool to extract the document content based on the parsing results. The logic of selecting the text content acquisition tool is based on the specific structural information mentioned in the user's question. For example, if the question explicitly mentions the page number (such as "Page 5"), the Agent will select the page extraction tool; if the question mentions the chapter title or chapter number (such as "Specific Implementation Method"), the Agent will select the chapter extraction tool; if the question mentions the table identifier (such as "Table 1"), the Agent will select the table extraction tool; if the question mentions the image identifier (such as ""), the Agent will select the image extraction tool; after the text content acquisition tool is selected, the Agent will call the corresponding text content acquisition tool to extract relevant content from the document. If the user's question involves multiple parts (such as requiring chapter content and table data at the same time), the Agent will switch between different tools and integrate information. For example, if the question requires an analysis of a table in combination with the content of the second chapter, the Agent will first call the chapter extraction tool to obtain the text content of the second chapter, and then call the table extraction tool to obtain the data of the table. Then, the Agent will integrate the content extracted by different tools to form a complete reference. This Agent design based on the ReAct framework improves the flexibility and accuracy of the question-answering system compared to traditional retrieval enhancement algorithms, and can also dynamically adjust the workflow according to different types of question-answering needs to provide more personalized and accurate answers. In one embodiment of the present invention, step S5 specifically includes: after obtaining the relevant text content, LLM performs deep understanding and reasoning processing based on the user's specific questions and in combination with the extracted document content to generate an answer that meets the semantic requirements.For example, for the question "Please explain the table data on page 5 and analyze it in conjunction with the content of the second chapter", the Agent will extract the table data on page 5 and the text content of the second chapter, and integrate this information and pass it to the LLM to generate a detailed explanation that combines the table data and the chapter text. An embodiment of the present invention provides a question-answering system for structured long documents, which is suitable for the question-answering method for structured long documents as described above, including: a document parsing module: used to parse documents of different formats and construct structured metadata for the documents; a document segmentation module: used to divide the document into multiple text segments, vectorize each text segment and store it in a dedicated vector database; the Agent: contains multiple text content acquisition tools and retrieval tools, which are used to intelligently select text content acquisition tools or retrieval tools for user questions to obtain relevant text content required for the LLM to answer the question; the LLM: used to generate answers that meet semantic requirements after obtaining relevant text content. The question-answering system for structured long documents provided by the embodiments of the present invention constructs an Agent including multiple text content acquisition tools and retrieval tools, which can automatically classify user questions and call corresponding text content acquisition tools or retrieval tools, which can not only improve the accuracy of answers, but also flexibly adjust the workflow according to different types of question-answering needs, and provide more personalized and accurate answers. Users can not only get answers through traditional question-answering methods, but also ask questions about certain specific content according to the structure of the document, which enhances the flexibility and convenience of the interactive process. Although the embodiments of the present invention have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A question-answering method for structured long documents, characterized in that: The following steps are involved: S1. Parse documents of different formats and construct structured metadata of the documents based on the parsing results; S2, dividing the document into multiple text segments, vectorizing each text segment, and storing them in a dedicated vector database; S3. Build multiple text content acquisition tools to extract text content from different parts of the document; design and implement vector-based retrieval tools to find text fragments related to user queries in the vector database; S4, building an agent that includes multiple text content acquisition tools and retrieval tools, intelligently selecting text content acquisition tools or retrieval tools according to user questions to obtain relevant text content required by LLM to answer questions; S5. After obtaining the relevant text content, the LLM is used to analyze the relevant text content and generate the final answer.

2. The question-answering method for structured long documents according to claim 1, characterized in that: Step S1 specifically includes: S11. For documents in PDF format, extract the location information and content of various elements such as pages, chapters, tables, and images; for documents in DOC and DOCX format, obtain text content and identify chapter and paragraph divisions; for documents in image format, convert the content in the image into editable text; S12. Construct a structured metadata including the document's pages, chapters, tables, and images based on the parsing results.

3. The question-answering method for structured long documents according to claim 1, characterized in that: Step S2 specifically includes: S21. The system automatically identifies titles and paragraphs in the document, analyzes the text font style and paragraph spacing, and divides the document into reasonable segments based on semantics; S22. Use the text embedding model to encode each text fragment in the document and convert it into a vector representation of fixed dimension. The vectorization result of each text fragment will be stored in a dedicated vector database.

4. The question-answering method for structured long documents according to claim 1, characterized in that: Step S3 specifically includes: S31. In order to cope with different forms of information needs in documents, a number of text content acquisition tools for specific document structures were built, including page extraction tools, chapter extraction tools, table extraction tools, and image extraction tools; S32. The search tool searches the vector database to find the text fragments most relevant to the query.

5. The question-answering method for structured long documents according to claim 4, characterized in that: The page extraction tool is used to extract the complete text content of the specified page according to the page number in the user's question; the chapter extraction tool is used to obtain the complete text of a specific chapter according to the title level; the table extraction tool is used to extract the row and column data and header information of the table according to the table identifier in the user's question; the image extraction tool is used to extract the text content and context description in the image according to the image identifier specified in the user's question.

6. The question-answering method for structured long documents according to claim 4, characterized in that: Step 32 specifically includes: The search tool performs text preprocessing on user questions, including noise removal, tokenization, and stop word removal; Convert user queries into vector representations by using text embedding models; The vector representation is used to search in the vector database to find the text fragments most relevant to the query for subsequent processing and answer generation.

7. The question-answering method for structured long documents according to claim 1, characterized in that: The construction of the Agent containing the above tools described in step S4 includes: S41. Use the ReAct framework to build an agent that includes multiple text content acquisition tools and retrieval tools; S42, Agent uses LLM to semantically analyze the user's questions and identify the core information and structural requirements of the questions; S43. Agent dynamically selects tools to complete information acquisition based on the semantic analysis results. When the user's question contains structured features such as page number, chapter title level, table mark or image mark, the Agent will intelligently select the corresponding page extraction tool, chapter extraction tool, table extraction tool or image extraction tool; for unstructured semantic queries, it selects vector-based retrieval tools to achieve accurate matching of semantic space through text embedding models; S44, the Agent calls the corresponding text content acquisition tool or retrieval tool to extract relevant content from the document. If the user's question involves multiple parts, the Agent will switch between different tools and integrate the information.

8. The question-answering method for structured long documents according to claim 7, characterized in that: The ReAct framework includes: The original action space A is enhanced by the language model space L. ; For actions from the language model space , the Agent will be based on the current environment Reasoning is performed to obtain useful information, thereby supporting subsequent reasoning and action execution; based on the weighted language model, tool definitions and problem-solving examples are given in the prompt words to generate thought and action sequences for solving problems.

9. The question-answering method for structured long documents according to claim 1, characterized in that: Step S5 specifically includes: After obtaining relevant text content, LLM conducts in-depth understanding and reasoning processing based on the user's specific questions and the extracted document content to generate answers that meet semantic requirements.

10. A question-answering system for structured long documents, characterized in that: A question-answering method for a structured long document according to any one of claims 1 to 9, comprising: Document parsing module: used to parse documents of different formats and build structured metadata of documents; Document segmentation module: used to divide a document into multiple text segments, vectorize each text segment and store it in a dedicated vector database; Agent: contains multiple text content acquisition tools and retrieval tools, which are used to intelligently select text content acquisition tools based on user questions to obtain relevant text content required by LLM to answer questions; LLM: Used to obtain relevant text content and generate answers that meet semantic requirements.

Images