Large language model long text question answering method and system based on hybrid context compression technology

Abstract

The invention belongs to the field of text questioning and answering, and relates to a large language model long text questioning and answering method and system based on a hybrid context compression technology. The method comprises the following steps: analyzing and preprocessing a document, and converting an unstructured original text into a normalized text paragraph set; classifying questions of the long text question and answer scene based on a large language model; according to the question type and the preprocessed text, adaptively selecting the most suitable context compression method to compress the long text to obtain a compressed context; and generating an answer to the question based on a large language model by using the compressed context. According to the method, the advantages of two context compression technologies are integrated, the problem of context window limitation of processing a long text by a large language model is effectively solved, high-quality question and answer performance is kept while computing resource consumption and processing delay of the compression technologies are reduced, and the limitation of a single compression method on different types of problems is relieved.

Description

Technical Field

[0001] This invention belongs to the field of text question answering, specifically involving a technique that integrates two context compression techniques to solve long text question answering for large language models. It aims to improve the limitation of the context window size of a single large model when processing ultra-long texts, and enhance the model's performance in long text question answering involving characteristics such as cross-paragraph and multi-hop reasoning. Background Technology

[0002] This invention aims to solve the context window limitation and performance problems of large language models in long text question answering by combining two context compression techniques. The closest implementation schemes to this invention include the retrieval enhancement generation system LightRAG and the single context compression strategy system LongLLMLingua. The following is a detailed introduction to the two schemes.

[0003] LightRAG proposes a retrieval enhancement generation system that integrates graph construction and two-layer retrieval. The system first uses a pre-trained large-scale language model to segment the original text and automatically extract entities and their relationships, thereby constructing a knowledge graph with global semantic connections and generating a key-value pair index for efficient retrieval. After a user submits a question, the system uses keyword extraction technology to break down the query into two types of retrieval requests: one targeting specific details and the other targeting overall concepts. On the one hand, low-level retrieval locates specific entity information and relational data in the text; on the other hand, high-level retrieval captures the global theme and abstract context. Subsequently, the system integrates these multi-dimensional retrieval results, and the pre-trained large-scale language model performs a deep understanding of the context, ultimately generating an answer that matches the user's intent. To adapt to the dynamically changing information environment, LightRAG also designs an incremental update mechanism, allowing new data to be quickly integrated into the system without rebuilding the entire index graph. Each module undergoes specialized data preprocessing and targeted fine-tuning: the graph construction module is trained using a large amount of templated entity relationship annotation data, while the question-answering generation module continuously optimizes the generation effect using carefully selected high-quality question-answer samples. The overall process of LightRAG is as follows: Figure 1 As shown.

[0004] LongLLMLingua proposes a context compression system to address the performance degradation, high computational cost, and positional information bias issues of large language models in long text scenarios. The system mainly consists of two modules: a question-aware context compression module and an information reordering and subsequence recovery module. The context compression module first uses a pre-trained small language model and a question-aware coarse-to-fine compression strategy to initially screen and compress the original long prompts. In the coarse-grained stage, the system dynamically determines the compression ratio based on question relevance and the distribution of key information in the document, and uses contrastive perplexity to assess the importance of each document. Then, in the fine-grained stage, an iterative lexical-level compression method is used to dynamically allocate the compression budget, ensuring that while significantly reducing the number of lexical units, the information crucial to the question is preserved to the greatest extent. Next, the information reordering and subsequence recovery module reorders the compressed documents according to the density of key information, mitigating the positional information bias caused by excessively long prompts, and uses a subsequence recovery strategy to ensure that the large language model can accurately restore the key details in the original prompts when generating answers. The overall process of LongLLMLingua is as follows: Figure 2 As shown, this method, during training, achieves the goal of significantly improving inference accuracy while greatly reducing computational cost and response latency in various long text application scenarios by performing instruction tuning and data distillation pre-training on coarse-grained and fine-grained compression strategies respectively.

[0005] Existing technologies for solving long-text question answering in large language models include retrieval-enhanced generation methods and context compression methods. Retrieval-enhanced generation systems typically rely on simple text segmentation and vector matching to simplify long texts by retrieving relevant information from the question, thus enabling long-text question answering for large language models. However, this method struggles to fully capture the complex semantic relationships between entities in the text, leading to incomplete information acquisition and potential breaks in the reasoning chain during multi-step inference tasks. Context compression systems typically employ strategies based on information entropy or summarization, directly shortening text length and transforming long-text question answering into a regular text question answering task. However, context compression is prone to losing key information or introducing unnecessary redundancy, making it difficult to meet the needs of long-text question answering. Summary of the Invention

[0006] To address the problems of existing technologies, this invention proposes a long-text question-answering method for large language models based on hybrid context compression technology. This invention employs a question-oriented hybrid context compression strategy, combining two different compression techniques: semantic relationship graph-based and question-related statement-based filtering. It can automatically select the most suitable compression method based on the question type. For factual questions, the semantic relationship graph-based method captures semantic connections and multi-hop relationships in long texts; for non-factual questions, the question-related statement-based method efficiently extracts relevant information. In special cases, the results of both methods can be combined. The proposed method integrates the advantages of two context compression techniques, effectively solving the context window limitation problem in processing long texts with large language models. While reducing the computational resource consumption and processing latency of compression techniques, it maintains high-quality question-answering performance and alleviates the limitations of single compression methods on different question types.

[0007] The technical solution adopted in this invention is as follows:

[0008] A long text question answering method based on a large language model using hybrid context compression technology includes the following steps:

[0009] Perform document parsing and preprocessing to convert unstructured raw text into a set of normalized text paragraphs;

[0010] Classify questions in long text question-answering scenarios based on a large language model;

[0011] Based on the question type and the preprocessed text, the most suitable context compression method is adaptively selected to compress the long text and obtain the compressed context.

[0012] Using compressed context, answers to questions are generated based on a large language model.

[0013] Furthermore, the document parsing and preprocessing supports input processing of various document formats, including plain text, PDF, and Word formats. Preprocessing includes removing redundant whitespace characters, segmenting paragraphs, and recognizing chapter structures.

[0014] Furthermore, the classification of questions in long text question-answering scenarios based on the large language model includes: when receiving user input questions, the large language model classifies the questions into two types: factual question-answering questions and non-factual question-answering questions; factual question-answering questions refer to questions that require retrieving, extracting, or synthesizing objective factual information from the text, and usually require establishing a complete reasoning chain; non-factual question-answering questions focus on the expression of ideas, the articulation of viewpoints, or the creation of content.

[0015] Furthermore, the adaptive selection of the most suitable context compression method based on the question type and the preprocessed text to obtain the compressed context includes:

[0016] For factual questions, a context compression method based on semantic relationship graphs is used to capture semantic associations and multi-hop relationships in long texts;

[0017] For non-factual questions, a context compression method based on filtering relevant statements is used to efficiently extract relevant information;

[0018] If the context compression method based on semantic relationship graphs fails to form a complete logical chain or the number of graph search hops exceeds a preset threshold, the system will automatically switch to the context compression method based on question-related statements and merge the results of the two methods to maximize the retention of key information.

[0019] Furthermore, the context compression method based on semantic relation graphs includes:

[0020] Long texts are segmented semantically, and then a graph structure is constructed using the semantic block information. Each node represents a key element and its related atomic facts, and the edges between nodes represent semantic relationships.

[0021] Intelligent planning and retrieval based on user questions employs chain thinking to search for information paths related to the question in the graph, and summarizes the retrieved key information into a compressed context.

[0022] Furthermore, the context compression method based on filtering question-related statements includes:

[0023] A lightweight transformer encoder is used to score the relevance of each sentence in the text to the question, and the sentences are classified into three categories: directly relevant, indirectly relevant, and irrelevant.

[0024] Retain both directly and indirectly related sentences, and reorganize them in the order of "directly related first, then indirectly related" to form a compressed context.

[0025] Furthermore, the step of generating the answer to the question based on a large language model using the compressed context includes:

[0026] The compressed context and user questions are integrated into a standard format prompt, which is then passed to the selected large language model through a unified API interface.

[0027] Based on the reasoning results from the large language model, the compressed text and the final answer are presented to the user.

[0028] A long text question-answering system based on a large language model using hybrid context compression technology, comprising:

[0029] The document parsing and preprocessing module is used to parse and preprocess documents, converting unstructured raw text into a set of normalized text paragraphs;

[0030] The question type judgment module based on the large language model is used to classify questions in long text question-answering scenarios based on the large language model.

[0031] The problem-oriented hybrid context compression module is used to adaptively select the most suitable context compression method to compress long texts based on the problem type and the preprocessed text, and obtain the compressed context.

[0032] The answer generation module based on the large language model is used to generate answers to questions by utilizing compressed context and based on the large language model.

[0033] The beneficial effects of this invention are as follows:

[0034] Compared to existing technologies, one advantage of this invention is that it eliminates the need for additional base model pre-training or fine-tuning. By training only with lightweight prompt word engineering and a context compression model, it significantly improves the ability of large language models to process long texts. This makes the system easy to deploy and maintain, and allows for flexible compatibility with various base models. Furthermore, the hybrid compression architecture of this invention can intelligently select the most suitable compression strategy for specific question types, exhibiting greater versatility and adaptability compared to single compression methods. On another note, the large language model long text question answering method using hybrid context compression technology proposed in this invention achieves more beneficial results on the common long text question answering dataset LongBench. Attached Figure Description

[0035] Figure 1 This is a flowchart of the workflow of the existing search enhancement generation system, LightRAG.

[0036] Figure 2 This is a flowchart of the existing single-context compression strategy system, LongLLMLingua.

[0037] Figure 3 This is a flowchart of the long text question answering method based on hybrid context compression technology of the present invention. Detailed Implementation

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0039] The technical solution of the present invention is as follows Figure 3As shown, it mainly includes four core steps: document parsing and preprocessing, question type determination based on a large language model, question-oriented mixed context compression, and answer generation based on a large language model.

[0040] This invention aims to address the context window limitations and performance issues of large language models when processing extremely long texts. By adaptively selecting the most suitable context compression strategy, long texts are compressed into more concise prompts while retaining key information, enabling the large language model to efficiently answer questions based on the compressed context. This invention selectively employs two different context compression techniques: a context compression method based on semantic relationship graphs and a context compression method based on filtering question-related statements. The choice of technique is determined through question type analysis, and finally, the results of long text question answering are generated based on the context compression results. In the specific implementation, the semantic relationship graph construction and intelligent planning exploration are implemented using existing large language models. The specific technical solution is as follows.

[0041] Step 1: Document parsing and preprocessing.

[0042] The text processed in this step refers to user-provided text that requires question-and-answer analysis. This text typically originates from user-uploaded document files or directly input long text content, often exceeding the context window limit of a large language model. These texts form question-and-answer pairs with the user's question, meaning the user hopes to answer a specific question based on this long text content. For example, a user might upload a technical report and then ask, "What are the main technical innovations mentioned in this report?" In this case, the technical report is the "text" to be processed. This invention extracts text content based on existing document parsing technologies, supporting input processing of various document formats, including plain text, PDF, and Word. For PDF documents, the system uses pdfplumber for parsing; for Word documents, the system uses python-docx for content extraction; and for plain text files, the system reads them directly. Subsequently, the system preprocesses the extracted text content, including removing redundant whitespace characters, segmenting paragraphs, and identifying chapter structures, converting the unstructured raw text into a standardized set of text paragraphs, preparing for the subsequent context compression stage.

[0043] Step 2: Determine the question type based on the large language model.

[0044] This invention categorizes questions in different long-text question-answering scenarios based on a large language model. When a user inputs a question, the large language model classifies it into factual and non-factual questions. Factual questions require retrieving, extracting, or synthesizing objective factual information from the text; these questions typically have clear, fact-based answers that can be derived through text analysis and reasoning. Non-factual questions, on the other hand, focus more on expressing ideas, stating viewpoints, or creating content. This classifier is implemented by using prompt words to guide the large language model in classifying questions. In this step, the large language model can be of any size.

[0045] Step 3: Problem-oriented hybrid context compression technology.

[0046] In the hybrid context compression technology of this invention, "compression" refers to extracting and simplifying information from the standardized text paragraph set after preprocessing in step 1, converting long texts into short texts that retain key information, in order to adapt to the context window limitations of large language models.

[0047] Based on the classification results of the question type by the large language model in step 2, this invention adaptively selects the most suitable context compression strategy:

[0048] (1) When the question is a factual question, the system calls a context compression method based on semantic relationship graphs. This method first divides the long text into semantic blocks, and then uses the semantic block information to construct a graph structure. Each node represents a key element and its related atomic facts, and the edges between nodes represent semantic relationships. Subsequently, the system performs intelligent planning and retrieval based on the user's question, and uses chain thinking to search for information paths related to the question in the graph. Finally, the retrieved key information is summarized into compressed context.

[0049] (2) When the question is a non-factual question, the system invokes a context compression method based on filtering relevant statements. This method uses a lightweight transformer encoder to score the relevance of each sentence in the text to the question, classifying sentences into three categories: directly relevant, indirectly relevant, and irrelevant. The system retains the directly relevant and indirectly relevant sentences and reassembles them in an optimized order (first directly relevant, then indirectly relevant) to form the compressed context.

[0050] (3) In special cases, if the semantic relationship graph-based method fails to form a complete logical chain (the number of graph search hops exceeds a preset threshold), the system will automatically switch to the question-related statement-based filtering method and merge the results of the two methods to maximize the retention of key information. The final text obtained is a shorter reference text after context compression. The large language model used in this step can be any pre-trained large language model.

[0051] Step 4: Response generation based on a large language model.

[0052] This invention generates answers based on a large language model (MLM) after context compression of the text. During the generation phase, the system integrates the compressed context with the user's question into standard-format prompts, which are then passed to the selected MLM model via a unified API interface. Based on the inference results of the MLM model, the system presents the compressed text and the final answer to the user. The system also supports directly inputting the original long text into the MLM model for question-and-answer processing. In this step, the selected MLM model can be of any size.

[0053] The technical solution of the present invention will be illustrated below through an example of a company's annual report Q&A.

[0054] Suppose a user uploads a 15,000-word annual report of a technology company and asks the question: "What is the company's investment strategy and technology layout in the field of artificial intelligence?"

[0055] Step 1: Document Parsing and Preprocessing

[0056] The system uses PDF parsing technology to extract the text content of the annual report, removes redundant information such as headers, footers, and chart titles, and divides the document into 87 standardized paragraphs, including chapters on company overview, business development, financial status, and technology investment.

[0057] Step 2: Problem type determination based on large language model

[0058] The system inputs the user's question, "What are the company's investment strategies and technology layout in the field of artificial intelligence?" into the large language model for analysis. The result is that it is a factual question because the question requires the extraction of objective investment data, technical information and other factual content from the document.

[0059] Step 3: Problem-oriented hybrid context compression techniques

[0060] Since the question was determined to be a factual question, the system invoked a context compression method based on semantic relationship graphs:

[0061] The 87 paragraphs were semantically divided into 16 semantic blocks containing information such as "technology investment", "R&D layout", "partners", and "future plans". Key elements and corresponding atomic facts were extracted from the semantic blocks to construct a semantic relationship graph, which includes related nodes such as "artificial intelligence investment", "R&D budget", "technology team", and "product line". Based on the user's question, a graph search was performed to identify multiple related nodes including "AI investment amount", "technology R&D direction", and "partnership relationship". The relevant information was summarized to form a compressed context of about 800 words.

[0062] Step 4: Response generation based on a large language model

[0063] The system combines the compressed 800-word context with the user's question to form prompts, which are then input into a large language model to generate the final answer: "According to the annual report, the company has invested approximately 230 million yuan in the field of artificial intelligence, mainly focusing on: 1) natural language processing technology research and development; 2) computer vision applications; 3) establishing AI laboratories with three key universities..."

[0064] Through the technical solution of this invention, a long document originally containing 15,000 words is compressed into 800 words of key information, while retaining the core content, enabling the large language model to answer user questions efficiently and accurately.

[0065] The key and innovative aspect of this invention lies in proposing a problem-oriented hybrid context compression method. This method incorporates two approaches: context compression based on semantic relationship graphs and context compression based on filtering question-related statements. It adaptively selects the most suitable compression strategy according to the question type. The hybrid context compression strategy innovatively combines two complementary methods: semantic relationship graph-based and question-related statement-based methods. The former excels at handling factual questions requiring multi-hop reasoning, while the latter offers higher efficiency and versatility in general question-answering scenarios. By intelligently selecting and combining these two methods, the system can adapt to a wider range of question types. The adaptive compression method selection automatically chooses the most suitable compression method for the current question, eliminating the need for manual parameter settings or strategy selection by the user, significantly improving the system's usability and intelligence.

[0066] Compared to existing technologies, one advantage of this invention is that it does not require additional base model pre-training or fine-tuning. It significantly improves the ability of large language models to process long texts through lightweight prompt word engineering and context compression model training alone. This makes the system easy to deploy and maintain, and flexibly compatible with various base models. Furthermore, the hybrid compression architecture of this invention can intelligently select the most suitable compression strategy for specific question types, exhibiting greater versatility and adaptability compared to single compression methods. On the other hand, the large language model long text question answering method proposed in this invention, using hybrid context compression technology, achieves more beneficial results on the common long text question answering dataset LongBench. To fully analyze the beneficial effects of this invention, the performance of different methods is compared in Table 1.

[0067] Table 1. Performance comparison of different methods on LongBench

[0068]

[0069] Compared to existing methods LightRAG and LongLLMLingua, the method of this invention achieves superior average performance on LongBench. On the subsets of SingleDoc and MultiDoc question answering, which require strong reasoning capabilities, the method of this invention significantly outperforms other related techniques. This is because the designed context compression technique based on text relationship graphs excels at handling long text multi-hop reasoning tasks. For other subsets of summarization, few-shot learning, localization, and code synthesis—Summ, FewShot, Synth, and Code—the method of this invention has its advantages and disadvantages compared to the other two methods. A major reason is that the two compression techniques of the hybrid context compression strategy are good at different tasks, and question type judgment based on a large language model cannot completely assign different questions to the most suitable methods.

[0070] In the technical implementation of this invention, a problem-oriented hybrid context compression method is currently adopted. This method adaptively selects compression strategies to handle different types of questions, requiring the context compression technology to be applicable to question-answering tasks in different domains. This part can be replaced by a general-domain context compression technology. By training an adaptive context compression model and fine-tuning it for different types of tasks, a solution for long text question answering using a single compression method for large language models can be provided.

[0071] Another embodiment of the present invention provides a long text question-answering system based on a large language model using hybrid context compression technology, comprising:

[0072] The document parsing and preprocessing module is used to parse and preprocess documents, converting unstructured raw text into a set of normalized text paragraphs;

[0073] The question type judgment module based on the large language model is used to classify questions in long text question-answering scenarios based on the large language model.

[0074] The problem-oriented hybrid context compression module is used to adaptively select the most suitable context compression method to compress long texts based on the problem type and the preprocessed text, and obtain the compressed context.

[0075] The answer generation module based on the large language model is used to generate answers to questions by utilizing compressed context and based on the large language model.

[0076] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0077] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0078] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0079] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A long text question answering method based on a large language model using hybrid context compression technology, characterized in that, Includes the following steps: Perform document parsing and preprocessing to convert unstructured raw text into a set of normalized text paragraphs; Classify questions in long text question-answering scenarios based on a large language model; Based on the question type and the preprocessed text, the most suitable context compression method is adaptively selected to compress the long text and obtain the compressed context. Using compressed context, answers to questions are generated based on a large language model.

2. The method according to claim 1, characterized in that, The document parsing and preprocessing supports input processing of various document formats, including plain text, PDF, and Word formats. Preprocessing includes removing redundant whitespace characters, segmenting paragraphs, and recognizing chapter structures.

3. The method according to claim 1, characterized in that, The classification of questions in long text question-answering scenarios based on the large language model includes: when receiving user input questions, the large language model classifies the questions into two types: factual question-answering questions and non-factual question-answering questions; factual question-answering questions refer to questions that require retrieving, extracting, or synthesizing objective factual information from the text, and usually require establishing a complete reasoning chain; non-factual question-answering questions focus on the expression of ideas, the articulation of viewpoints, or the creation of content.

4. The method according to claim 1, characterized in that, The process of adaptively selecting the most suitable context compression method based on the problem type and preprocessed text to obtain the compressed context includes: For factual questions, a context compression method based on semantic relationship graphs is used to capture semantic associations and multi-hop relationships in long texts; For non-factual questions, a context compression method based on filtering relevant statements is used to efficiently extract relevant information; If the context compression method based on semantic relationship graphs fails to form a complete logical chain or the number of graph search hops exceeds a preset threshold, the system will automatically switch to the context compression method based on question-related statements and merge the results of the two methods to maximize the retention of key information.

5. The method according to claim 4, characterized in that, The context compression method based on semantic relation graphs includes: Long texts are segmented semantically, and then a graph structure is constructed using the semantic block information. Each node represents a key element and its related atomic facts, and the edges between nodes represent semantic relationships. Intelligent planning and retrieval based on user questions employs chain thinking to search for information paths related to the question in the graph, and summarizes the retrieved key information into a compressed context.

6. The method according to claim 4, characterized in that, The context compression method based on filtering related statements includes: A lightweight transformer encoder is used to score the relevance of each sentence in the text to the question, and the sentences are classified into three categories: directly relevant, indirectly relevant, and irrelevant. Retain both directly and indirectly related sentences, and reorganize them in the order of "directly related first, then indirectly related" to form a compressed context.

7. The method according to claim 1, characterized in that, The process of generating answers to questions based on a large language model using compressed context includes: The compressed context and user questions are integrated into a standard format prompt, which is then passed to the selected large language model through a unified API interface. Based on the reasoning results from the large language model, the compressed text and the final answer are presented to the user.

8. A long text question-answering system based on a large language model using hybrid context compression technology, characterized in that, include: The document parsing and preprocessing module is used to parse and preprocess documents, converting unstructured raw text into a set of normalized text paragraphs; The question type judgment module based on the large language model is used to classify questions in long text question-answering scenarios based on the large language model. The problem-oriented hybrid context compression module is used to adaptively select the most suitable context compression method to compress long texts based on the problem type and the preprocessed text, and obtain the compressed context. The answer generation module based on the large language model is used to generate answers to questions by utilizing compressed context and based on the large language model.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images