Language model retrieval enhancement method based on long document
By employing digital perception negative sampling and consistency reinforcement learning, the retrieval system's ability to discriminate digital facts and the generator's robustness are enhanced. This addresses the issues of insufficient model discriminability of key digital information and inaccurate generation results in long document numerical reasoning, thereby achieving efficient long document numerical reasoning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
Existing technologies fail to fully recognize the importance of numerical facts in long document numerical reasoning, resulting in insufficient model discernment of key numerical information, and the generated results lack interpretability and robustness.
A retrieval system is trained using a negative sampling strategy based on digital perception, while a generator is trained using a consistency-based reinforcement learning and target procedure enhancement strategy. This improves the retrieval system's ability to discriminate digital facts and enhances the robustness and accuracy of the generator.
It significantly improves the accuracy of the retrieval system and the execution accuracy of the generator, enhancing the overall performance of numerical reasoning for long documents and reaching the current state-of-the-art level.
Smart Images
Figure CN121808012A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of language model inference, and particularly relates to a language model retrieval enhancement method based on long documents. BACKGROUND
[0002] With the advent of the information age, long document analysis has become a challenging task. Long documents often contain a large amount of information and complex semantic relationships, bringing many difficulties to the analysis process. Document length is a significant obstacle. Long documents can contain a large amount of text content, involving multiple topics and viewpoints. Processing long documents requires a large amount of computing resources and time, and may require segmentation or chunking of the text for better understanding and analysis. The information in long documents has context dependence. The understanding of one part may depend on the content of another part. Therefore, long document analysis needs to consider context dependence to ensure accurate capture of semantic relationships in the document. This is crucial for extracting accurate information and identifying topics in the document.
[0003] Taking financial document analysis as an example, the analysis and understanding of financial documents play a crucial role in assessing the performance of enterprises. Inadequate or inappropriate analysis can result in billions of dollars in cost losses. In order to facilitate high-quality and timely decision-making, professionals such as analysts or investors need to perform complex quantitative analysis to sift information from financial reports. This analysis requires advanced reasoning between different types of data sources (structured and unstructured) and performs complex numerical reasoning, such as comparing profitability or growth financial ratios.
[0004] The characteristics of financial document analysis are reflected in its ability to handle and interpret large amounts of long text and complex data. This analysis involves in-depth mining and understanding of various structured and unstructured data such as corporate financial statements, market trends, industry reports, etc. The importance of financial document analysis not only lies in its direct impact on the performance of enterprises, but also in its impact on investment decisions. High-quality financial analysis can help decision-makers identify investment opportunities and potential risks, thereby optimizing resource allocation and improving economic efficiency. However, in the face of growing data volume and analysis complexity, traditional manual analysis methods have been difficult to meet current needs. Therefore, it is particularly urgent to develop technologies and systems that can be used for long document numerical reasoning and automatically perform these complex analysis tasks to ensure the efficiency and accuracy of financial analysis and support more reasonable and timely decision-making.
[0005] In the field of long document numerical reasoning, two main approaches are adopted: end-to-end method and retrieval-generation two-stage method. The end-to-end method relies on pre-trained models designed specifically for long documents (such as Longformer), which adjust the attention mechanism to reduce computational cost, enabling the model to handle long text inputs. In addition, this method fine-tunes the model on large-scale datasets to generate executable programs to output answers. However, the results produced by this method often lack interpretability and are highly sensitive to the quality of the dataset, affecting the robustness of the model. On the other hand, the retrieval-generation method decomposes long documents into multiple text blocks and converts them into text vectors to retrieve the most relevant text vectors to the question, providing the basis for answering the question. After retrieving relevant documents, these documents are input into the generator along with the question to further generate programs to obtain answers. This framework is also widely used in open-domain question answering, reasoning, and other tasks.
[0006] Despite this, existing research on long document numerical reasoning still has two key omissions. First, existing methods fail to fully recognize the importance of numerical facts and treat all facts equally. Intuitively, numerical facts are a direct source of parameters for generating programs, so they have higher importance. SUMMARY
[0007] The purpose of the present invention is to address the above technical problems existing in the prior art and provide a long document-based language model retrieval enhancement method, referred to as APOLLO. For the retriever, a number-aware negative sampling strategy is used for training, enabling the retriever to better distinguish key numerical facts. For the generator, a consistency-based reinforcement learning and a target program enhancement strategy based on program execution result consistency are designed to improve the robustness and accuracy of the generator.
[0008] A long document-based language model retrieval enhancement method, comprising the following steps:
[0009] 1) Number-aware negative sampling training: select numerical facts Q from long documents to form a numerical fact set; sample from the numerical fact set as negative samples along with positive samples for training the retriever, and train the retriever to prioritize and distinguish numerical facts; the sampling method is complexity-based sampling to enhance the retriever's ability to distinguish numerical facts;
[0010] 2) Target program enhancement: when training the generator, use data augmentation techniques to expand the training dataset, and perform data augmentation based on seed data artificially constructed in the original training set to improve the robustness of the generator training;
[0011] 3) Consistency-based reinforcement learning: Introduce a reinforcement learning mechanism during the generator training process, using the execution result of the generation program as a reward to train the generator and improve the accuracy of the generator.
[0012] In step 1), the digital-aware negative sampling training includes the following steps:
[0013] 1.1) Data Preprocessing: Extract all facts containing numbers from the long document dataset as a set of numerical facts. For each question in a document, filter out positive samples (i.e., facts directly related to the question and containing key numbers) and a preliminary set of negative samples (i.e., facts that do not contain key numbers or are not directly related to the question).
[0014] 1.2) Numerical perception negative sampling: APOLLO adopts complexity-based sampling: select the most complex facts from the set of numerical facts. The complexity here is measured by the number of numbers contained in the facts. Calculate the number of numbers contained in each fact in the initial negative sample set, and select the facts with the most numbers as complexity negative samples. The complexity here can be adjusted according to actual needs, such as selecting facts with more than a certain threshold number of numbers.
[0015] 1.3) Training the retrieval engine: The retrieval engine is based on a sequence binary classification model. Question q and each fact... Connect to a BERT transformer. Use pooling and linear layers to obtain a score for each fact. :
[0016]
[0017] in ∈F, cls() extracts the hidden vector of BERT at the [CLS] label. This is the projection vector. The Adam optimizer (initial learning rate 9e-6, 10 training epochs) is used for training, combining positive and negative samples. The retrieval engine is optimized using the Triplet Loss function.
[0018]
[0019] The margin is a manually set boundary value of 0.5.
[0020] In step 2), a standard program artificially constructed from the original training set is used as seed data. Specific methods for enhancing the target program include transformation, addition / subtraction, multiplication / division, and multiplication-division. For programs containing addition and multiplication operators, the transformation method randomly swaps the parameter positions to generate new program samples. The addition / subtraction method adds or subtracts a small random constant (this constant should be much smaller than the program's calculation result to avoid altering the outcome) to generate new program samples. The multiplication / division method is similar to addition / subtraction, but multiplies or divides the program by a random constant close to 1 at the end to ensure the result remains approximately unchanged. The multiplication-division method selects a constant term in the program and performs a multiplication or division operation on it (again ensuring the result remains approximately unchanged) to generate new program samples.
[0021] The expanded training dataset involves applying one or more of the data augmentation methods described above to each program in the original training set to generate multiple new program samples. These new samples are then added to the original training set to form the expanded training dataset.
[0022] In step 3), the consistency-based reinforcement learning does not use cross-entropy training at each step of the program generation, but instead obtains the next token by sampling from the output distribution. At the end of the generation process, the generated program is executed and compared with the correct answer to determine the reward. Let Gg denote the program generated by the generator, and GT denote the standard program corresponding to the question. The reward R(Gg, GT) is defined as:
[0023]
[0024] The reinforcement learning loss is the program that can be generated. The derivation of the policy gradient for the negative expected reward is as follows:
[0025]
[0026] Where θ is the policy network, i.e., the decoder of the generator. This indicates the selection of a flag during decoding time step t. The probability of.
[0027] Training stabilization strategy:
[0028] Pre-training: First, fine-tune the generator using cross-entropy loss (to avoid the cold start problem). The loss function is as follows:
[0029] .
[0030] Monte Carlo sampling: Sample tokens from the output distribution at each decoding step (non-greedy decoding), and calculate the reward after generating the complete program.
[0031] Gradient clipping: restricting the gradient norm.
[0032] After adopting the above solution, the beneficial effects of the present invention are:
[0033] (1) The retrieval system is optimized by using a negative sampling training strategy based on digital perception, which results in a stable improvement in the accuracy of the retrieval system. This enables the retrieval system to retrieve key digital information from long documents in a priority and accurate manner, providing a solid foundation for subsequent numerical reasoning; and it also results in a stable improvement in the accuracy of the retrieval system.
[0034] (2) In the generator training, a consistency-based reinforcement learning and target program augmentation strategy is proposed. Through data augmentation, the generator can learn more diverse program expressions to improve execution accuracy and generator robustness.
[0035] (3) Through reinforcement learning, the generator can learn how to generate programs that can be executed correctly and obtain high rewards, thereby improving its execution accuracy and generation efficiency.
[0036] (4) The method of the present invention achieves execution accuracy of 71.07 and 78.76 and program accuracy of 68.94 and 77.19 on the real-world long document datasets FinQA and ConvFinQA, respectively, reaching the state-of-the-art level. Attached Figure Description
[0037] Figure 1 This is a schematic diagram of an embodiment of the present invention.
[0038] Figure 2 This is a flowchart illustrating an embodiment of the present invention. Detailed Implementation
[0039] The technical solution and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.
[0040] Figure 1This example illustrates numerical reasoning in a long document, providing an overview of a company's financial data from 2015 to 2017, including information on the company's assets, liabilities, equity, and interest coverage ratio, particularly regarding long-term and short-term debt. The question is posed: "What percentage of total debt was long-term debt in 2017?" Gold Fact indicates that, based on the table data, long-term debt in 2017 was $3.794 billion ($3,794 million), while supplementary text indicates that total debt in 2017 was $4.7 billion. The calculation was performed using two equivalent procedural methods: the Gold Program first converted 4.7 billion to 4700 million units (Multiply(4.7, Const_1000)) and then calculated the ratio (Divide(3794, #0)); the Consistency Program first calculated 3794 ÷ 4.7 (Divide(3794, 4.7)) and then converted the units (Divide(#0, Const_1000)). Both methods yielded a result of 0.81. It is worth noting that the program uses #0 to reference intermediate results and Const_1000 as a unit conversion constant. Although the order of operations differs (unit conversion first or last), functional consistency is achieved through numerical awareness, and non-numerical text paragraphs do not affect the core calculation process.
[0041] Based on the above examples, the specific process of the embodiments of the present invention is as follows: Figure 2 As shown. The terms included are defined as follows:
[0042] • Teacher Forcing: During the generator pre-training phase, the reference program token is used to force input to the decoder.
[0043] • Combination: A collection of variant programs generated by enhancing the target program.
[0044] · / : Positive / negative sample set.
[0045] · Cross-entropy loss.
[0046] · : Reinforcement learning loss.
[0047] The embodiments of the present invention include the following steps:
[0048] Step 1 employs a number-aware negative sampling strategy. The retrieval engine is trained using both positive and negative samples, whereas previous work simply sampled negative facts randomly from a given long-format document. However, since number reasoning is directly related to numbers, intuitively, facts without numbers contribute less. Figure 1 As shown, the two green-highlighted facts are both numerical facts, and they are the primary source of parameters in the standard procedure. In contrast, the red-highlighted non-numerical facts contribute less to the golden plan and the answer. APOLLO expects the retrieval machine to focus more on numerical facts. To effectively train the retrieval machine, this invention employs a number-aware negative sampling strategy, which involves extracting numerical facts from a set of negative facts in a long-format document and sampling a subset of these numerical facts. This subset is then used as a set of number-aware negative facts, along with the set of positive facts, to train the retrieval machine, thereby enhancing its ability to distinguish numerical facts. For the number-aware negative sampling strategy, APOLLO employs complexity sampling: selecting the most complex facts from the set of negative facts (here, complexity is measured by the number of digits contained in the fact).
[0049] Step 2: Expand the dataset and improve the model's robustness using target program augmentation. Ten program computation operators are generated, including addition, subtraction, multiplication, and division. This invention employs four target program augmentation methods: transformation method, addition / subtraction, multiplication / division, and multiplication-division. The specific details of these four methods are as follows: Transformation method: For addition and multiplication operators, these two parameters are interchangeable. Heuristically, for programs that have these two operators in practice, these two parameters can be swapped to increase the amount of data. Furthermore, programs with n operators, as well as addition or multiplication operators, can be created. Multiple new samples are used for training. Addition and subtraction: For the same program, adding or subtracting a random constant at the end does not change the result. Multiplication and division: Similar to the previous construction, multiplying or dividing by a random constant at the end does not change the result. Multiplication-division: The same program performs multiplication or division on a constant, but it does not change the result.
[0050] Step 3: Use consistency-based reinforcement learning to improve the model's execution accuracy. In consistency-based reinforcement learning, cross-entropy is not performed at every step of the program generation process. Instead of training, the next token is obtained by sampling from the output distribution. At the end of the generation process, the generated program is executed and compared with the correct answer to determine the reward. The training process of consistency-based reinforcement learning is as follows: Figure 2As shown. Although the standard procedure and the generated procedure differ at the character level, they are identical at the consistency level, thus producing positive rewards. However, if only cross-entropy is used for training, the decoded and generated prediction procedure may unfairly penalize the model parameters.
[0051] The specific steps can be as follows:
[0052] 3.1 Model Pre-training
[0053] Use cross-entropy Loss fine-tuning generator.
[0054] 3.2 Enhanced learning and training:
[0055] In each training round, for the input question and document, the generator produces a candidate program.
[0056] Execute the candidate program and compare the result with the correct answer to calculate the reward value. If the result is the same as or very close to the correct answer, a high reward is given; otherwise, a low reward or penalty is given.
[0057] Reinforcement learning algorithms such as policy gradient are used to update the generator's model parameters based on the reward value.
[0058] 3.3 Consistency considerations:
[0059] When calculating rewards, not only the accuracy of the program's execution results are considered, but also the similarity between the program and the standard program in terms of consistency level. If two programs differ in expression but are functionally identical (i.e., produce the same execution results), a higher reward is given.
[0060] 3.4 Iterative Training:
[0061] Repeat the above steps until the generator's performance on the validation set no longer improves significantly or the preset number of training rounds is reached.
[0062] This invention allows models to analyze information in long documents, build programs to solve given problems, and obtain the final answer by executing the programs. The FinQA and ConvFinQA datasets are benchmark datasets specifically designed to evaluate a model's ability to perform numerical reasoning on long-format financial documents. They contain a large number of real-world financial reports, announcements, and other documents, along with complex numerical reasoning problems based on these documents. These problems require models not only to understand the document content but also to extract key numerical information and construct appropriate mathematical expressions or programs to solve them. This invention validates its performance on the real-world long-document datasets FinQA and ConvFinQA. Experimental results show that APOLLO achieves state-of-the-art performance on both the FinQA and ConvFinQA datasets. On the FinQA test set, APOLLO achieves an execution accuracy (Exe Acc) of 71.07% and a program accuracy (the rate at which the generated program perfectly matches the reference program's operators and operands, Prog Acc) of 68.94%, significantly outperforming GPT-4 (Exe Acc 68.79%) and TABT5 (Exe Acc 68.00%). On the ConvFinQA test set, APOLLO further improved execution accuracy to 78.76% and program accuracy to 77.19%.
[0063] The digital-aware negative sampling strategy, by focusing on training with digital facts, significantly improves the model's sensitivity and processing ability to digital information, thus achieving better results in numerical reasoning tasks. Through data augmentation techniques, this invention effectively expands the training dataset, increasing the model's generalization ability and robustness. This helps the model maintain high accuracy even when facing complex, unseen problems. Consistency-based reinforcement learning not only focuses on the correctness of the program execution result but also considers the similarity between the program and the standard answer at the consistency level. This helps the model generate more accurate and reasonable programs, improving the overall performance of the model.
[0064] In summary, this invention has achieved remarkable results in long document numerical reasoning tasks through a series of innovative methods and techniques, reaching the current state-of-the-art level.
[0065] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.
Claims
1. A language model-based retrieval enhancement method for long documents, characterized in that... Includes the following steps: 1) Negative sampling training based on digital perception: facts containing numbers are filtered out from long documents to form a set of digital facts; samples are taken from this set of digital facts as negative samples, which are used together with positive samples to train the retrieval machine, so that the retrieval machine prioritizes and distinguishes digital facts; the sampling method is complexity-based sampling to enhance the retrieval machine's ability to distinguish digital facts; 2) Target program enhancement: When training the generator, data augmentation techniques are used to expand the training dataset. Data augmentation is performed based on artificially constructed seed data in the original training set, thereby improving the robustness of generator training. 3) Consistency-based reinforcement learning: Introduce a reinforcement learning mechanism during the generator training process, using the execution result of the generation program as a reward to train the generator and improve the accuracy of the generator.
2. The language model-based retrieval enhancement method for long documents according to claim 1, characterized in that... In step 1), the digital-aware negative sampling training includes the following steps: 1.1) Data preprocessing: Extract all facts containing numbers from the long document dataset as a set of numerical facts; for each question in a document, filter out positive samples and a preliminary set of negative samples from the document; 1.2) Numerical perceptual negative sampling: APOLLO adopts complexity-based sampling: select the most complex facts from the set of numerical facts. Here, complexity is measured by the number of numbers contained in the facts. Calculate the number of numbers contained in each fact in the initial negative sample set, and select the facts with the most numbers as complexity negative samples. 1.3) Training the retrieval engine: The retrieval engine is based on a sequence binary classification model; question q and each fact Connect to a BERT transformer; use pooling layers and linear layers to obtain a score for each fact. : in ∈F, cls() extracts the hidden vector of BERT at the [CLS] label. It is a projection vector; the Adam optimizer is used, and it is trained using both positive and negative samples; the retrieval engine is optimized using the Triplet Loss function. Here, margin is a manually set boundary value.
3. The method for enhancing retrieval based on a language model of long documents according to claim 1, characterized in that... In step 2), a standard program artificially constructed in the original training set is used as seed data. The specific methods for enhancing the target program include transformation, addition and subtraction, multiplication and division, and multiplication-division. For programs containing addition and multiplication operators, the transformation method randomly swaps the positions of their parameters to generate new program samples. The addition and subtraction method adds or subtracts a small random constant at the end of the program to generate new program samples. The multiplication and division method is similar to addition and subtraction, but multiplies or divides a random constant close to 1 at the end of the program to ensure that the result remains approximately unchanged. The multiplication-division method selects a constant term in the program and performs multiplication or division operations on it to generate new program samples.
4. The language model-based retrieval enhancement method for long documents according to claim 1, characterized in that... In step 2), the expanded training dataset involves applying data augmentation methods to each program in the original training set to generate multiple new program samples, which are then added to the original training set to form the expanded training dataset.
5. The language model-based retrieval enhancement method for long documents according to claim 1, characterized in that... In step 3), the consistency-based reinforcement learning obtains the next token by sampling from the output distribution; at the end of the generation process, the generated program is executed and compared with the correct answer to determine the reward; let Gg denote the program generated by the generator, and GT denote the standard program corresponding to the question; the reward R(Gg, GT) is defined as: The reinforcement learning loss is the program that can be generated. The derivation of the policy gradient for the negative expected reward is as follows: Where θ is the policy network, i.e., the decoder of the generator. This indicates the selection of a flag during decoding time step t. The probability of.