A reasoning system and method that can be used for long text tasks

Abstract

The application discloses a reasoning system and method which can be used for long text tasks, and the reasoning system comprises an information extraction and compression module and a reasoning module, wherein the information extraction and compression module is used for scanning background text, splicing an input query with the background text, appending a preset number of compression guide marks at the tail, compressing the spliced text, encoding the compression guide marks, and extracting the last layer of hidden states after coding as implicit fact marks; and the reasoning module is used for reasoning based on the implicit fact marks compressed by the information extraction and compression module. The reasoning system is used for reasoning, the knowledge extraction and the logical reasoning process are decoupled, the reasoning module can skip the original redundant text and directly perform efficient reasoning, and therefore, the reasoning delay is reduced while the effective context window is significantly expanded.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a reasoning system and method applicable to long text tasks. Background Technology

[0002] The performance of Large Language Models (LLMs) in handling knowledge-intensive tasks is largely limited by the static features of their pre-training corpora. Since the models can only grasp the information covered during the training phase, they often struggle to guarantee the accuracy of their reasoning when faced with real-time updates or domain knowledge they haven't yet learned. Therefore, to overcome the limitations of pre-training corpora and further improve the models' intelligence and practicality, they need to be able to dynamically acquire and utilize new knowledge for complex reasoning. Currently, this is mostly achieved through input augmentation techniques or parametric knowledge editing techniques. Input augmentation techniques essentially involve providing knowledge as contextual input to the model to assist in reasoning, and are mainly divided into two categories: Retrieval-augmented Generation (RAG) techniques and cue word compression techniques. RAG suffers from insufficient accuracy due to limitations in retrieval machine performance. Furthermore, the overall performance of the RAG system is highly sensitive to the top-k setting in the retrieval stage, i.e., the number of relevant text segments retrieved, k. This parameter inevitably creates a trade-off between "information coverage" and "noise introduction." Specifically, a smaller top-k is prone to missing key information relevant to the question, while a larger top-k introduces a large amount of irrelevant context, thus interfering with downstream generation and inference processes. Cue word compression techniques are divided into hard compression and soft compression. Hard compression directly selects and filters tokens, a discrete decision-making method that easily leads to information omissions and semantic incompleteness. Soft compression, on the other hand, compresses the input contextual knowledge into the latent space before providing it to the inference model. Soft compression methods are mostly static compression, easily losing key task information relevant to the query, and this static mode suffers from high information distortion rates at high compression ratios. Common soft compression methods, such as AutoCompressor, Gist Tokens, and ICAE, attempt to simultaneously complete context compression and downstream inference within a single language model. Among these, AutoCompressor generates compact implicit representations to approximate the original context semantics by training the model; Gist Tokens introduce learnable summary tokens to compress long inputs into fixed-length representations for subsequent inference; ICAE employs an autoencoder-like approach, enabling the model to learn reconstructive compression of long texts within the context. Additionally, there are soft compression methods based on the (RAG) paradigm, such as xRAG and COCOM, which reduce input length and improve the efficiency of RAG systems by compressing retrieved documents into latent space vectors using an encoder.The latest soft compression method is E2LLM, which uses a lightweight encoder to segment long inputs and transform them into latent representations for downstream inference. Context-cascaded compression (C3) uses two different large language models for compression and reconstruction respectively. Although it performs well in compression and reconstruction, it still has a gap between understanding compressed representations and achieving effective inference due to the lack of specific adaptation for downstream inference tasks.

[0003] Parametric knowledge editing techniques include methods that directly edit the model's internal parameters and methods that train external pluggable knowledge modules. Internalizing knowledge through parameter fine-tuning or knowledge editing often disrupts the model's inherent reasoning logic and carries the risk of catastrophic forgetting. Therefore, methods that train pluggable knowledge modules, such as MemoryDecoder and MLP Memory, are preferred. These modules store knowledge in trainable parametric memory modules and are pre-trained with retrieval behavior as the target, thus supporting efficient knowledge access without modifying the main model parameters. Training external pluggable knowledge modules decouples external knowledge from the model parameters, pre-storing knowledge in the knowledge modules through training, allowing for plug-and-play use during inference. However, this approach cannot handle long, real-time inputs and constantly updated knowledge, and its training cost is high. Furthermore, most existing training methods mimic the behavior of the retrieval system, so their performance is still limited by the retrieval system itself.

[0004] In summary, the reasoning ability of existing large language models in knowledge-intensive tasks is significantly limited by the static nature of pre-training corpora, making it difficult to effectively utilize real-time updated or uncovered external knowledge. Existing input augmentation techniques generally face problems such as noisy retrieval results, omission of key information, and information distortion under high compression rates in practical applications, while parametric knowledge editing or fine-tuning methods are prone to destroying the original reasoning ability of the model and suffer from drawbacks such as catastrophic forgetting and high training costs. Summary of the Invention

[0005] To address some or all of the problems of existing technologies, and in order to achieve efficient acquisition, dynamic compression, and accurate utilization of external knowledge without modifying model parameters, thereby improving the accuracy and efficiency of large language models in complex reasoning tasks, the first aspect of this invention provides a reasoning system applicable to long text tasks, comprising:

[0006] The information extraction and compression module is used to scan background text, extract the required information based on the task request, and compress it.

[0007] The reasoning module is used to perform reasoning based on the information extracted and compressed by the information extraction and compression module.

[0008] Furthermore, the information extraction and compression module is used for:

[0009] The input query is concatenated with the background text, and a preset number of compressed guide marks are appended at the end;

[0010] The concatenated text is compressed in order to encode the compression guide mark;

[0011] Extract the last hidden state after encoding as an implicit fact marker.

[0012] Furthermore, compression is achieved through a bucket compression mechanism.

[0013] Furthermore, the reasoning system also includes:

[0014] The text preprocessing module is used to divide the background text into several semantically coherent blocks and then input them into the information extraction and compression module. The information extraction and compression module processes each block in parallel to obtain the implicit fact markers of each block.

[0015] Furthermore, the text preprocessing module includes a recursive character text splitter.

[0016] Furthermore, the reasoning system also includes:

[0017] The mapping module is used to map the information compressed by the information extraction and compression module to align it to the embedding space of the inference module.

[0018] Furthermore, the mapping module includes a three-layer multilayer perceptron projector.

[0019] Based on the aforementioned reasoning system, a second aspect of the present invention provides a reasoning method applicable to long text tasks, comprising:

[0020] The input query is concatenated with the background text, and a preset number of compressed guide marks are appended at the end;

[0021] The concatenated text is compressed in order to encode the compression guide mark;

[0022] Extract the last hidden state after encoding as an implicit fact marker;

[0023] Reasoning is based on the implicit fact markers.

[0024] Furthermore, the concatenated text is compressed based on a bucket compression mechanism.

[0025] Furthermore, the reasoning method also includes:

[0026] The background text is divided into several semantically coherent blocks;

[0027] Each block is processed in parallel to obtain the implicit fact tag for each block;

[0028] The implicit fact tags of each block are concatenated in the original document order, and block boundary tags are inserted between adjacent implicit fact tags.

[0029] The concatenated implicit fact tags are mapped to fact embeddings, and reasoning is performed based on these fact embeddings.

[0030] Furthermore, the reasoning method also includes training the reasoning system before reasoning, including:

[0031] The frozen inference module provides reconstruction signals, which are then used to train the information extraction and compression modules to generate the potential fact representations that best support document reconstruction.

[0032] By providing reconstruction signals through the frozen inference module, introducing query input, and using specified supporting evidence as supervision signals, the information extraction and compression modules are retrained to enable them to extract and compress query-related information from redundant documents.

[0033] Based on standard language modeling loss, the inference model is optimized to enable it to extract parsed information, compress implicit fact embeddings generated by the module, and combine queries to generate accurate and context-aware answers.

[0034] This invention discloses a reasoning system and method for long text tasks. It decouples knowledge extraction from the core reasoning process, dynamically compressing document blocks into high-density implicit fact tags based on queries using a lightweight knowledge model. These tags are then projected into the embedding space of a large-scale reasoning model, enabling efficient reasoning without processing the original redundant text. This significantly expands the effective context window while drastically reducing inference latency and maintaining high-precision logical expression. This dynamic compression mode preserves more task-relevant core information at a high compression rate and greatly reduces the impact of redundant and irrelevant information on the reasoning model. Attached Figure Description

[0035] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0036] Figure 1 This diagram illustrates the structure of an inference system for long text tasks according to an embodiment of the present invention.

[0037] Figure 2This diagram illustrates a flowchart of an inference method for long text tasks according to an embodiment of the present invention.

[0038] Figure 3 This diagram illustrates a comparison of end-to-end first-word response times for different methods under progressively increasing input lengths.

[0039] Figure 4 This diagram illustrates a process of training a reasoning system for long text tasks according to an embodiment of the present invention. Detailed Implementation

[0040] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or in conjunction with other alternatives and / or additional methods or components. In other instances, well-known structures or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific numbers and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details. Furthermore, it should be understood that the embodiments shown in the drawings are illustrative representations and are not necessarily drawn to scale.

[0041] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0042] It should be noted that the embodiments of the present invention describe the method steps in a specific order; however, this is only for illustrating the specific embodiment and not for limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to actual needs.

[0043] To address the significant limitations of existing large language models in reasoning tasks due to the static nature of pre-training corpora, which hinders their ability to effectively utilize real-time updated or uncovered external knowledge, this invention provides a reasoning system and method applicable to long text tasks. This system decouples knowledge extraction from the core reasoning process, enabling efficient acquisition, dynamic compression, and accurate utilization of external knowledge without modifying model parameters. This improves the accuracy and efficiency of large language models in complex reasoning tasks. The reasoning system and method employ a dual-model decoupled architecture, DRIFT, consisting of a lightweight knowledge model and a large-scale reasoning model. This explicitly separates knowledge extraction from the core reasoning process, thereby enhancing the processing efficiency and reasoning accuracy of large language models in scenarios with extremely long contexts.

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0045] Figure 1 This diagram illustrates the structure of an inference system for long text tasks according to an embodiment of the present invention. Figure 1 As shown, a reasoning system for long text tasks includes an information extraction and compression module 101 and a reasoning module 102. The information extraction and compression module 101 is primarily used for deep scanning of massive amounts of background text, extracting and compressing relevant information in conjunction with the task request. The reasoning module 102 is used to perform complex logical reasoning based on the compressed high-value information. This clearly defined architecture eliminates the need for the reasoning module to directly access redundant original text, thus greatly alleviating its computational burden and reducing interference from massive amounts of irrelevant information.

[0046] In one embodiment of the present invention, the information extraction and compression module 101 includes a lightweight knowledge model. The information extraction and compression module 101 concatenates the input query with the background text and appends a preset number of compression tokens to the end. Then, it uses the lightweight knowledge model... The query-related information in the concatenated text is deeply compressed to encode the compression guide markers, and the encoded last-layer hidden states are extracted as implicit fact markers. In embodiments of the present invention, the lightweight knowledge model... The base model can be any base model, but Qwen2.5-Instruct-3B is preferred.

[0047] In one embodiment of the present invention, the inference module 102 includes a large-scale reasoning model. The large-scale inference model The system includes an embedding layer and other layers. The embedding layer converts the input query into query embeddings and concatenates them with compressed information output from the information extraction and compression module 101 to form a continuous input stream. This stream is then input to other layers for reasoning and outputs a response. In embodiments of the present invention, the large-scale reasoning model... The base model can be any base model, such as Mistral-7B-Instruct-v0.2 or Qwen2.5-72B-Instruct, etc.

[0048] In one embodiment of the present invention, the inference system further includes a text preprocessing module 103, which divides the background text into several semantically coherent chunks and inputs them into the information extraction and compression module 101. In another embodiment of the present invention, the text preprocessing module 103 includes a recursive character text splitter, which divides a long document X into a series of highly semantically coherent chunks C=(C1,C2,...,C...) by prioritizing the identification of natural semantic boundaries such as paragraphs (\n\n) and sentences. K To match the training window, in one embodiment of the invention, the size of each block is set to 8,192 tokens.

[0049] After dividing the text into several blocks, it is input into the information extraction and compression module 101 for knowledge compression. In one embodiment of the present invention, the information extraction and compression module 101 first combines the input query with each block C. j The blocks are spliced ​​together, and a preset number of compression guide markers <|CPS|> are added to the end of each block. Then, the lightweight knowledge model is used to... The compressed bootstrap marker <|CPS|> is encoded in parallel, and its last hidden state is extracted as the implicit fact marker T of the corresponding block. j :

[0050] ,

[0051] Where Q represents query, d represents the number of compressed tags, and d represents the dimension of the hidden layer.

[0052] After completing the parallel compression of each block, the information extraction and compression module 101 generates the implicit fact marker T for each block. j Concatenating the documents in their original order yields a global implicit fact tag sequence. To preserve the physical structure of the documents in the latent space, in one embodiment of the invention, the information extraction and compression module 101 concatenates adjacent tag sequences T... j With T j+1 Insert block boundary markers between them, wherein the block boundary markers may be, for example, specific double newline characters (\n\n).

[0053] Since two text segments of the same length will inevitably contain different amounts of information related to the query, a bucketed compression mechanism is introduced in one embodiment of this invention to optimize information encoding efficiency. Unlike traditional compression that uses a fixed ratio, such as 8:1, bucketed compression maps inputs belonging to the same length range to a fixed output length calculated based on the upper bound of that range by pre-setting multiple word length range buckets, such as 64-128, 128-256, etc. By artificially introducing discretization of the output length, bucketed compression effectively prevents the model from exhibiting "shortcut learning" behavior that only focuses on positional distribution, forcing the model to truly understand the sparse key evidence in the input content. In one embodiment of this invention, the output length S after bucketed compression... bucket Follow these principles:

[0054] ,

[0055] Where b(n) represents the upper bound of the bucket containing n input tokens, and c is a predefined compression ratio. In one embodiment of the present invention, the value of c is preferably 32.

[0056] In one embodiment of the present invention, the inference system further includes a mapping module 104, which is used to map the information compressed by the information extraction and compression module 101 to align it to the embedding space of the inference module 102. In one embodiment of the present invention, the mapping module 104 includes a three-layer MLP projector π, which can non-linearly map the globally implicit fact tokens obtained by concatenation to transform them into implicit fact embeddings E=(e1,e2,...,e...) that can be directly recognized by the inference module 102. ξ The fact embedding E and the query embedding E(Q) are concatenated to form a continuous input stream, which is then input into the large-scale inference model. The large-scale inference model By processing these highly condensed fact units that have already been query-aligned in the latent space, accurate question-answering responses for extremely long texts are achieved with minimal resource consumption.

[0057] .

[0058] It should be understood that in some other embodiments of the present invention, other small networks may be used instead of the three-layer MLP projector π.

[0059] By employing a lightweight knowledge model set to form a large-scale reasoning model to compose the reasoning system, the knowledge extraction and logical reasoning processes are explicitly separated. The knowledge model dynamically compresses redundant background text into query-related implicit fact tags, which are then aligned to the embedding space of the reasoning model via a projector. This allows the reasoning model to skip the original redundant text and directly perform efficient reasoning, thereby significantly expanding the effective context window while reducing reasoning latency.

[0060] Based on the reasoning system described above, Figure 2 This diagram illustrates a flowchart of an inference method for long text tasks according to an embodiment of the present invention. Figure 2 As shown, an inference method applicable to long text tasks includes:

[0061] First, in step 201, text preprocessing. For very long text tasks, the input long document X can first be preprocessed using a text preprocessing module such as a recursive character text segmenter. The text preprocessing module divides the document X into a series of semantically highly coherent blocks C=(C1,C2,...,C...) by prioritizing the identification of natural semantic boundaries such as paragraphs (\n\n) and sentences. K This ensures that the document retains its inherent structural integrity during compression. To match the training window, in one embodiment of the invention, the size of each block is set to 8,192 tokens. For shorter text tasks, this step can be omitted.

[0062] Next, in step 202, knowledge extraction and compression are performed. Based on the input document or preprocessed blocks, knowledge extraction and compression are conducted. Specifically, the input query is concatenated with the document or block, and a preset number of compression guide markers are appended to the end. Then, deep compression is performed on the information related to the query, the compression guide markers are encoded, and the last layer of hidden states after encoding is extracted as implicit fact markers. For blocks obtained from preprocessing of very long text tasks, parallel encoding can be performed. Then, the implicit fact markers of each block are concatenated according to the original document order to obtain a global implicit fact marker sequence, and block boundary markers are inserted between adjacent implicit fact markers. As mentioned earlier, in one embodiment of the present invention, compression is performed based on a bucket compression strategy. Specifically, by using predefined term length buckets, such as 64-128, 128-256, etc., input blocks belonging to the same length range are uniformly mapped to a fixed-dimensional output calculated based on the upper bound of the bucket. It can effectively prevent the model from exhibiting "shortcut learning" behaviors such as positional bias or over-averaging during training, ensuring that the model can prioritize the identification and abstraction of core semantic content, and improving robustness in scenarios with sparse evidence distribution.

[0063] Finally, in step 203, reasoning is performed. Reasoning is conducted based on the compressed implicit fact tags or the global implicit fact tag sequence to obtain an answer. In one embodiment of the invention, the concatenated implicit fact tags can first be mapped to fact embeddings, and then the fact embeddings are concatenated with the query embeddings as input to the reasoning model for reasoning.

[0064] To verify the performance and efficiency of the inference system and method, evaluations were conducted on several representative long-text benchmark sets, and end-to-end first-to-first-to-token (TTFT) analysis was performed. Performance verification primarily involved evaluating the system and method on multiple representative long-text benchmark sets using different approaches. These methods include the DRIFT method from the embodiments of this invention, and several baseline methods based on Mistral-7B-v0.2, including the retrieval enhancement generation method NaiveRAG, the hard compression method LLM Lingua-2, the soft compression methods ICAE, COCOM, and xRAG, as well as the vanilla LLM models Mistral-7B-v0.2 and Qwen2.5-Instruct-7B. DRIFT includes both Mistral-7B-v0.2-based DRIFT and Qwen2.5-Instruct-7B-based DRIFT. The long text benchmark set includes BAMBOO (16k), the QA Subset of L-Eval, the Sum Subset of L-Eval, LongBench-v2, and LoCoMo, which are used to perform complex tasks such as multi-document understanding, multi-hop reasoning, and long-range dialogue memory. Among them, the following tasks were performed based on BAMBOO (16k): AltQA (a numerical question answering task for Wikipedia), MeetQA (a question answering task for conference proceedings), and PaperQA (a question answering task for academic papers). Classical extractive question answering tasks Natural Questions (NQ), NarrativeQA (NarQA), and Coursera (Crse) were performed based on a subset of L-Eval question answering methods. Multi-domain conference summarization tasks QMSum (QMS) and hotel review summarization tasks SPACE (SPC) were performed based on a subset of L-Eval. T1 head-to-head reasoning question answering, T2 multi-hop reasoning question answering, T3 temporal reasoning question answering, and T4 open-domain reasoning question answering were performed based on LoCoMo. LongBench-v2 was used for long text application tasks, divided into Short, Medium, and Long categories based on the input document length. Each task was tested using one or more compression ratios. During the evaluation process, accuracy was calculated using LLM-Judge based on Qwen2.5-Instruct-72B for simple question-and-answer questions. For abstracting, the ROUGE-L score was used as the metric, and the average metric was calculated for the same test set under the same compression ratio across different tasks. Experimental results are shown in Table 1.

[0065]

[0066] Table 1

[0067] As can be seen, DRIFT significantly outperforms existing technologies such as LLMlingua-2, ICAE, COCOM, and xRAG in all benchmark tests. Particularly in the LongBench-v2 task, DRIFT based on Mistral-7B-v0.2 significantly improved accuracy from 20.87% of the original model to 29.22% with a 32x compression ratio, demonstrating its superior inference accuracy in long text contexts. On the LoCoMo long-range dialogue memory benchmark, DRIFT maintained strong performance even when other compression algorithms failed. In extreme compression scenarios, DRIFT exhibited excellent stability, maintaining an average accuracy of 56.89% even with 128x compression on the BAMBOO(16k) task, far exceeding COCOM's 26.57% at the same compression ratio. Furthermore, cross-model generalization experiments show that when Qwen2.5-Instruct-7B is used as the base, DRIFT still achieves good performance on various benchmarks, further verifying the applicability of this architecture across different model families. It has been verified that the aforementioned inference system and method can achieve better long text inference accuracy under high compression ratios. Under the same model base conditions, through a decoupled dual-model architecture and dynamic compression mechanism, the limitation of traditional soft compression methods that experience a sharp performance drop at extreme compression ratios can be overcome. Experimental results show that through query-aware dynamic compression and deep alignment of implicit fact tags, the aforementioned inference system and method can significantly reduce information redundancy while more accurately preserving and reinforcing key facts relevant to the task.

[0068] End-to-end first-word response time analysis covered various input lengths from 32k to 256k words. It employed the methods DRIFT, Base Model, NaiveRAG (a retrieval enhancement generation method), LLMlingua-2 (hard compression method), and ICAE, COCOM, and xRAG (soft compression methods) from the embodiments of this invention for testing. Experimental results are as follows: Figure 3As shown in the figure, the vertical axis is logarithmic for easier comparison. It can be seen that DRIFT exhibits excellent scalability as the input size increases. Specifically, when processing ultra-long documents with 256k words, it achieves an average speedup of 7 times in first-word generation compared to the baseline model processing the complete original text, with a TTFT of only about 8 seconds. As the input length increases, the latency of full-context processing increases dramatically and non-linearly, while the latency growth curve of DRIFT is much flatter, indicating that the longer the input sequence, the more significant the computation time saved by DRIFT through decoupled reasoning. Verification shows that the aforementioned reasoning system and method significantly reduce the latency and computational overhead of ultra-long context reasoning. After applying parallel knowledge compression and decoupled reasoning modes, the system can dynamically extract key facts based on the query, eliminating the need for large-scale reasoning models to process hundreds of thousands of original words, thus greatly reducing the computational time of the pre-filling stage. Compared with existing full-context processing strategies and traditional retrieval enhancement generation methods, the aforementioned reasoning system and method exhibit superior scalability when processing continuously increasing input sizes, and its efficiency ranks among the top of all algorithms.

[0069] In one embodiment of the present invention, in order to achieve effective decoupling and alignment of knowledge and reasoning, the training process of the reasoning system is decomposed into three progressive stages, gradually allowing the knowledge model and the reasoning model to learn the desired capabilities. For example... Figure 4 As shown, in one embodiment of the present invention, the training of the reasoning system includes three stages: latent fact reconstruction pre-training (LFRP), query-aware dynamic compression fine-tuning (QAFT-DC), and query-aware question-answering fine-tuning (QAFT-QA).

[0070] In Latent Fact Reconstruction Pre-training (LFRP), a lightweight knowledge model is trained to accurately compress and encode the original document content into a high-fidelity latent fact representation. This representation is then reconstructed using a frozen, large-scale inference model, which acts as a frozen decoder, providing only the reconstruction signal. The knowledge model is optimized to generate the latent fact representation that best supports document reconstruction. The training loss function for this stage is defined as:

[0071] ,

[0072] in, It is a lightweight knowledge model. This represents the projector, and X represents the original document. For the t-th token in the original document X, This represents the reconstruction instructions input to the inference model. for and based on The fact embedding obtained by transformation, This represents all tokens preceding the first token in the original document X. express based on , , The obtained output results and A consistent probability is obtained by taking the logarithm of the probability and then inverting it, transforming maximizing the probability into minimizing the loss. In the first stage, the gradient is only backpropagated to the knowledge model and the projector, enabling them to learn high-fidelity fact encodings.

[0073] The Query-Aware Dynamic Compression Fine-Tuning (QAFT-DC) phase primarily trains the knowledge model to extract key evidence based on specific queries. Unlike the first phase, this phase introduces query input and uses specific supporting evidence as supervision signals to train the knowledge model to initially possess the ability to accurately extract and efficiently compress query-related information from redundant documents. Its reconstruction target is narrowed down from complete text to labeled supporting evidence X. evi The dynamic compression capability of the model is explicitly trained using evidence-supervised signals, and the loss function is:

[0074] ,

[0075] in, It is a lightweight knowledge model. Indicates projector, To support the k-th token in the evidence, This represents the instruction input to the inference model to answer the question. Embedding of the dynamic facts obtained from the transformation, Supporting evidence X evi All tokens preceding the k-th token in the list. express based on , , The obtained output results and The probability of consistency is obtained by taking the logarithm of the probability and then inverting it, which transforms maximizing the probability into minimizing the loss.

[0076] The Query-Aware Question Answering Fine-Tuning (QAFT-QA) phase is the only phase that updates the inference model's parameters. Its purpose is to teach the inference model to utilize compressed latent facts for downstream question answering tasks. As the final end-to-end optimization phase, the inference model is unlocked to learn how to deeply parse the implicit fact embeddings generated by the knowledge model and combine them with queries to generate accurate and context-appropriate answers. This phase uses standard language modeling loss for end-to-end optimization.

[0077] ,

[0078] in, For large-scale inference models, It is a lightweight knowledge model. For the j-th token in the answer test set A, This represents the instruction input to the inference model to answer the question. The transformed dynamic fact embedding is E(Q), where E(Q) is the input query embedding. This represents all tokens preceding the j-th token in the answer (Answer) A. express based on , E (Q) The obtained output results and The probability of consistency is obtained by taking the logarithm of the probability and then inverting it, which transforms maximizing the probability into minimizing the loss.

[0079] By employing a progressive optimization path and phased training, the knowledge model acquires high-fidelity fact encoding capabilities, and the inference model learns to parse implicit representations of the latent space. Furthermore, a lexical-level curriculum learning strategy (CurriculumLearning) addresses the convergence challenge in long document training, thereby achieving collaborative optimization between knowledge alignment during training and task generation during the inference phase. To verify the contribution of the three-stage collaborative training task to the overall system capability, an ablation implementation was performed, specifically removing the LFRP, QAFT-QA, and QAFT-DC stages from the training process. The trained model was then tested on BAMBOO, LongBench-v2, and LoCoMo, and compared with the DRIFT model obtained from the three-stage training. The results are shown in Table 2. Here, w / o LFRP indicates training with the LFRP stage removed, w / o QAFT-DC indicates training with the QAFT-DC stage removed, and w / o QAFT-QA indicates training with the QAFT-QA stage removed.

[0080]

[0081] Table 2

[0082] It can be seen that QAFT-QA is the core of the three stages. Removing this stage will lead to a significant decrease in the model's reasoning ability, for example, the accuracy on the LoCoMo task drops from 57.73% to 36.89%. LFRP can ensure the fidelity of the knowledge model's encoding of the original information, while QAFT-DC significantly optimizes the efficiency of extracting key evidence in the latent space.

[0083] In one embodiment of the invention, the training process uses the English Wikipedia snapshot from November 1, 2023, as the original corpus to construct a large-scale document-question-answering-evidence dataset containing over 300,000 instances. For the pre-training task, 640,000 original text segments are used, each segment serving as a reconstruction sample; for the query-aware fine-tuning task, 240,000 targeted samples are generated, containing questions, answers, and evidence. The entire dataset covers various sequence lengths from 64 to 8192 lexical units and is evenly distributed across different length buckets to ensure the model's robustness to various context windows. During QA pair generation, Qwen2.5-72B-Instruct is used as the generator, and a random slicing sampling strategy is implemented to avoid positional bias in the model, ensuring that relevant information appears at random locations within the document. The generated data includes questions, standard answers, and supporting evidence, and the question types cover multiple choice, true / false, and short answer questions to enhance the model's generalization ability. To ensure data quality, a large model is used as the evaluator, and the generated data is rigorously filtered based on five criteria: relevance, correctness, clarity, fidelity, and sufficiency. Finally, a lexical-level curriculum learning strategy is employed, gradually guiding model training according to document length from shortest to longest, effectively addressing the technical challenge of convergence difficulties in training extremely long sequences. It should be understood that in other embodiments of this invention, other similar public datasets or self-constructed datasets can also be used as training datasets.

[0084] This invention discloses a reasoning system and method for long text tasks. It decouples knowledge extraction from the core reasoning process, dynamically compressing document blocks into high-density implicit fact tags based on queries using a lightweight knowledge model. These tags are then projected into the embedding space of a large-scale reasoning model, enabling efficient reasoning without processing the original redundant text. This significantly expands the effective context window while drastically reducing inference latency and maintaining high-precision logical expression. This dynamic compression mode preserves more task-relevant core information at a high compression rate and greatly reduces the impact of redundant and irrelevant information on the reasoning model.

[0085] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.

Claims

1. A reasoning system applicable to long text tasks, characterized in that, include: The information extraction and compression module is configured to scan the background text, concatenate the input query with the background text, and append a preset number of compression guide marks at the end. The concatenated text is compressed using a bucket compression mechanism to encode the compression guide marks, and the last hidden state after encoding is extracted as an implicit fact mark. The reasoning module is configured to perform reasoning based on the implicit fact tags compressed by the information extraction and compression module.

2. The reasoning system as described in claim 1, characterized in that, Also includes: The text preprocessing module is configured to divide the background text into several semantically coherent blocks and then input them into the information extraction and compression module. The information extraction and compression module is configured to process each block in parallel to obtain the implicit fact tags of each block.

3. The reasoning system as described in claim 2, characterized in that, The text preprocessing module includes a recursive character text splitter.

4. The reasoning system as described in claim 1, characterized in that, Also includes: The mapping module is configured to map the information compressed by the information extraction and compression module to align it to the embedding space of the inference module.

5. The reasoning system as described in claim 4, characterized in that, The mapping module includes a three-layer multi-layer sensor projector.

6. A reasoning method applicable to long text tasks, characterized in that, Reasoning using the reasoning system as described in any one of claims 1 to 5 includes: The input query is concatenated with the background text, and a preset number of compressed guide marks are appended at the end; The concatenated text is compressed using a bucket compression mechanism to encode the compression guide marker; Extract the last hidden state after encoding as an implicit fact marker; Reasoning is based on the implicit fact markers.

7. The reasoning method as described in claim 6, characterized in that, Also includes: The background text is divided into several semantically coherent blocks; Each block is processed in parallel to obtain the implicit fact tag for each block; The implicit fact tags of each block are concatenated in the original document order, and block boundary tags are inserted between adjacent implicit fact tags. The concatenated implicit fact tags are mapped to fact embeddings, and reasoning is performed based on these fact embeddings.

8. The reasoning method as described in claim 6, characterized in that, This also includes training the reasoning system before reasoning, including: The frozen inference module provides reconstruction signals, which are then used to train the information extraction and compression modules to generate the potential fact representations that best support document reconstruction. By providing reconstruction signals through the frozen inference module, introducing query input, and using specified supporting evidence as supervision signals, the information extraction and compression modules are retrained to enable them to extract and compress query-related information from redundant documents. Based on standard language modeling loss, the inference model is optimized to enable it to extract parsed information, compress implicit fact embeddings generated by the module, and combine queries to generate accurate and context-aware answers.

Images