Method for improving multi-hop reasoning capability of long context language model

Abstract

According to the method for improving the multi-hop reasoning ability of the long context language model, an attribution reasoning mechanism is introduced, and the model is required to provide clear attribution information in the reasoning process. The method comprises the following steps of: generating a preliminary attribution annotation by utilizing 5-shot CoQ prompt, and constructing a high-quality attribution annotation data set 'MuSiQuality-Attribute'; a multi-task learning framework and a data enhancement strategy are combined, answer prediction, CoT generation and reference recognition tasks are included, fine adjustment is conducted on the model, and an improved model capable of executing attribution reasoning is generated. Experiments prove that the method is effective in a multi-hop reasoning benchmark test, the performance of the method is close to that of a proprietary model, and excellent adaptability and competitiveness are shown.

Description

Technical Field

[0001] This invention relates to text processing and question-answering techniques for language models, and in particular to a method for improving the multi-hop reasoning capability of long-context language models. Background Technology

[0002] Long context modeling has received considerable attention due to its importance in applications requiring broad understanding and generative capabilities (Lewis et al., 2020; Liu et al., 2023b). Proposed techniques for long context modeling (Chen et al., 2023a; Peng et al., 2023; Chen et al., 2023b) have achieved encouraging results on existing benchmarks (An et al., 2023; Bai et al., 2023).

[0003] Recent advances in long text modeling have enhanced the ability of language models (LMs) to handle complex tasks in various natural language processing (NLP) applications. Despite these advancements, this invention finds that these models struggle with multi-hop reasoning and their performance degrades in the presence of noisy contexts. Small, long-context models, in particular, remain challenging in reasoning.

[0004] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This invention reveals that these models underperform in multi-hop reasoning tasks, which require models to navigate and synthesize information from different sources to answer complex questions. Benchmarks such as LongBench (Bai et al., 2023) and experimental results from this invention show that these long-context models outperform leading multi-hop reasoning systems (Zhang et al., 2023). The reasons for the insufficient effectiveness of multi-hop reasoning are not yet fully understood.

[0006] This invention argues that the limitations of multi-hop reasoning observed in long-context models stem from two main problems: the inability to identify relevant information in noisy contexts (Liu et al., 2023a) and the difficulty in effectively integrating knowledge within context, especially for small-scale models (Zheng et al., 2023a). To address these challenges, this invention introduces attribution-based reasoning, a method that forces models to support their reasoning by linking claims to relevant contextual fragments (such as citations (Gao et al., 2023) or direct quotes (Menick et al., 2022)). This approach not only guides the model to perform targeted information retrieval to pinpoint the location of relevant context, thereby reducing noise, but also ensures that its responses are rooted in the source material.

[0007] The main objective of this invention is to address the problems existing in the background art described above and to provide a method for improving the multi-hop reasoning capability of long-context language models.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for improving the multi-hop reasoning capability of long-context language models includes the following steps:

[0010] Prepare a basic language model with long context processing capabilities. This model should have a broad pre-trained knowledge base and introduce an attribution reasoning mechanism to require the model to provide explicit attribution information for each assertion when performing multi-hop reasoning.

[0011] We use a language model to generate attribution-based annotations and employ the 5-shot CoQ hint method to create preliminary attribution annotations, thus constructing an attribution annotation dataset.

[0012] A multi-task learning framework is constructed, including answer prediction, CoT (Copy of Thought) generation, and reference recognition tasks, to enhance the model's reasoning ability, and data augmentation strategies are implemented for the attribution annotation dataset.

[0013] Using the data-augmented attribution annotation dataset, the base language model is fine-tuned in conjunction with the multi-task learning framework to generate an improved model capable of performing attribution inference. During the fine-tuning process, the improved model is made to integrate the CoC (Coreference Chain) and CoQ (Coreference Query) chains, and the model parameters are adjusted based on feedback from the multi-task learning framework and the data augmentation strategy to maximize the effectiveness of attribution inference.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for multi-hop reasoning capability of the improved long-context language model.

[0015] A computer program product includes a computer program that, when executed by a processor, implements the method for multi-hop reasoning capabilities of the improved long-context language model.

[0016] The present invention has the following beneficial effects:

[0017] This invention proposes a novel method—Reasoning with Attributions—which encourages language models to provide attributions for each assertion during the reasoning process, effectively improving the multi-hop reasoning capabilities of long-context language models. Experiments on three multi-hop reasoning datasets validate the method, demonstrating its effectiveness and adaptability using both proprietary and open-source models. Furthermore, this invention proposes a method to enhance reasoning capabilities through fine-tuning, providing a labeled dataset with attribution and a specialized training strategy. The fine-tuned model exhibits competitive performance on multi-hop reasoning benchmarks, approaching that of proprietary language models such as ChatGPT and Claude-instant.

[0018] The comprehensive experimental results of this invention confirm the effectiveness and adaptability of the method on various multi-hop inference benchmarks.

[0019] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0020] Figure 1 The differences between the main LA task and the three auxiliary tasks of this invention are shown.

[0021] Figure 2 The exact match (EM) results for different models at various noise levels are shown in three multi-hop inference datasets.

[0022] Figure 3 The multi-hop inference performance of AttrLoRA is compared with its reference precision and recall.

[0023] Figure 4 The effect of scaling and fine-tuning the data size is shown. Detailed Implementation

[0024] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0025] This invention proposes a method to improve the multi-hop reasoning capabilities of long-context language models, enhancing the potential of language models to perform both reasoning and attribution simultaneously through learning. The invention improves the multi-hop reasoning capabilities of long-context models by focusing on attribution. In the embodiments described below, a dedicated dataset, MuSiQue-Attribute, is created by annotating the multi-hop reasoning dataset MuSiQue (Trivedi et al., 2022) with ChatGPT (Brown et al., 2020) to fine-tune the model on this dual capability. The invention proposes a powerful learning strategy that leverages multi-task learning and data augmentation to fully utilize these annotations. Experiments on five long-context models tested on three multi-hop reasoning datasets and two general instruction following datasets demonstrate that, despite slight trade-offs in other capabilities, the Vicuna-7B model (Zheng et al., 2023b) fine-tuned according to this invention significantly outperforms benchmarks of similar size across all aspects, averaging over 20 points, and even surpasses ChatGPT and Claude-instant on MuSiQue.

[0026] The innovative contributions of this invention can be summarized as follows:

[0027] This invention introduces attribution-based reasoning, an innovative reasoning paradigm that enhances the performance and robustness of long-context models in multi-hop reasoning tasks.

[0028] This invention provides MuSiQue-Attribute, an attribution-annotated multi-hop inference dataset, to support further research in this field.

[0029] This invention develops a customized learning strategy that combines a novel auxiliary task for multi-task learning with a tailored data augmentation technique.

[0030] Through rigorous testing on three multi-hop inference datasets and two general instruction-compliant benchmarks, using both proprietary and open-source models, this invention demonstrates that the finely tuned Vicuna-7B model achieves multi-hop inference performance comparable to ChatGPT with minimal impact on other capabilities. Furthermore, the model exhibits adaptability to varying degrees of contextual noise, highlighting the effectiveness of the proposed method.

[0031] This invention provides a method for improving the multi-hop reasoning capability of long-context language models, comprising the following steps:

[0032] S1: Select and prepare the base language model: Prepare a base language model with long context processing capabilities. The model should have a broad pre-trained knowledge base and introduce an attribution reasoning mechanism to require the model to provide explicit attribution information for each assertion when performing multi-hop reasoning.

[0033] S2: Design and build an attribution annotation dataset: Generate attribution-based annotations using a language model (such as ChatGPT) and create initial attribution annotations using the 5-shot CoQ hint method; Build a high-quality attribution annotation dataset "MuSiQue-Attribute" by filtering out erroneous annotations;

[0034] S3: Establish a multi-task learning framework and data augmentation strategies: Construct a multi-task learning framework, including answer prediction, chain of thoughts (CoT) generation, and reference recognition tasks, to enhance the model's reasoning ability, and implement data augmentation strategies, such as interference sample sampling and document shuffling, for the attribution annotation dataset to improve the model's adaptability to noisy contexts.

[0035] S4: Fine-tuning the model and optimizing parameters: Using the data-augmented attribution annotation dataset, the base language model is fine-tuned in conjunction with the multi-task learning framework to generate an improved model capable of performing attribution inference. During the fine-tuning process, the model is made to accurately integrate the chain of citations (CoC) and the chain of quotations (CoQ), and the model parameters are adjusted based on the feedback from the multi-task learning framework and the data augmentation strategy to maximize the effectiveness of attribution inference.

[0036] In a preferred embodiment, the attribution reasoning mechanism includes: employing the concept of Chain of Thoughts (CoT) to decompose the complex multi-hop problem into two manageable tasks: identifying relevant information in the context and constructing a credible claim based on that information; a citation chain CoC mechanism, wherein during the reasoning process, the model is prompted to refer to citations corresponding to each step of the thought chain to ensure that the claim is associated with a specific part of the context; and a citation chain CoQ mechanism, which further requires that in each reasoning step, the model includes direct citations of the cited material to enhance the credibility and clarity of the claim; and through fine-tuning, the model is able to: accurately integrate the citation chain CoC and the citation chain CoQ; and explicitly link the claim to relevant fragments in the context through citation prompts or direct citations.

[0037] In a preferred embodiment, in the citation chain mechanism, the citation directly points to the specific document or information source supporting the claim through index tags; in the citation chain mechanism, the direct citation explicitly indicates the cited text content and source through quotation marks and index tags.

[0038] In a preferred embodiment, the design and construction of the attribution annotation dataset specifically includes: selecting instances: randomly selecting multiple instances (5,000 in this example) from the responsive training set to ensure that the generated attribution annotation dataset has sufficient diversity and representativeness; generating attribution annotations: using a language model (such as ChatGPT) to provide 5-shot CoQ (Chain-of-Quote) prompts to generate attribution-based annotations for the randomly selected instances; creating thought chains: creating attribution-based thought chains (CoT) for the instances to ensure that each reasoning step has corresponding citations or references to support it, thereby providing a clear chain of evidence.

[0039] In this way, the attribution annotation dataset can provide the model with rich multi-hop inference instances with explicit attribution information, thereby improving the model's performance in multi-hop inference tasks.

[0040] In a preferred embodiment, the design and construction of the attribution annotation dataset further includes: after generating the annotations, implementing a filtering process to exclude the following erroneous annotations: incorrect answers: the model's predicted answer is inconsistent with the reference answer, which usually indicates an error in the chain of events (CoT); non-existent attributions: fictitious references or references that do not match the actual context, indicating that the model is hallucinating; incorrect references: references that do not match manually identified supporting facts, suggesting a flaw in the attribution; duplicate references: redundant references that violate the requirement of multi-hop sourcing from multiple documents; extreme references: references that are too short (less than five words) or too long (spanning the entire document) and lack practicality.

[0041] The above filtering process ensures the high quality and accuracy of the attribution annotation dataset, thereby providing reliable training material for the model.

[0042] In a preferred embodiment, the fine-tuning of the base language model using a multi-task learning framework includes implementing the following auxiliary tasks on top of the main attribution learning task (LA): Answer prediction (AP) task: This task focuses on directly predicting the answer without explicit reasoning; it aims to help the language model internalize the reasoning required for simple questions, where chain thinking (CoT) is not required; CoT generation (CG) task: This task trains the model to generate chain thinking (CoT) before providing the answer; it aims to cultivate the language model's ability to reason explicitly and systematically across multiple pieces of information on complex questions; Quotation recognition (QI) task: This task trains the model to identify key quotations required for reasoning; it aims to fine-tune the language model's ability to filter out irrelevant details and focus on relevant fragments in the text, thereby improving the accuracy of reasoning. The above auxiliary tasks are combined with the main attribution learning task (LA) for training to enhance the language model's attribution-based reasoning ability. This multi-task learning method improves the overall performance of the model in multi-hop reasoning tasks.

[0043] In a preferred embodiment, the data augmentation strategy package implemented for the attribution annotation dataset Includes: interference samples This sampling: Design and implement a interference sample sampling strategy, by randomly selecting different numbers of... The method modifies the position of relevant documents in the context and the total number of documents, which is irrelevant to the quantity of documents. It aims to simulate the fluctuating noise level encountered in real-world scenarios and effectively train language models to cope with noisy environments. Document shuffling: A document shuffling strategy is designed to remove surface positional cues that may lead to inference bias by rearranging documents. This ensures that the model does not learn to associate the order of relevant documents with a fixed sequence of thought chains, thereby reducing bias in the inference process.

[0044] Furthermore, the aforementioned data augmentation strategies are applied sequentially in each training instance to ensure that the model is fully exposed to various noise levels and document arrangements during training, thereby enhancing its generalization ability. In this way, the method ensures that the attribution annotation dataset provides diverse inputs during training, thereby improving the robustness and adaptability of the model in multi-hop inference tasks.

[0045] The following describes specific embodiments of the present invention.

[0046] Attributional reasoning

[0047] Large language models often get stuck in context-irrelevant situations, a phenomenon known as the "Lost in the Middle," which has been observed in various Natural Language Processing (NLP) tasks, such as multi-document QA (Liuet al., 2023a) and mathematical reasoning (Shi et al., 2023). This problem is also evident in multi-hop reasoning, which will be discussed later in this paper. Figure 2 This will be explained in the following section. Previous research has noted this problem but has not decoded its underlying mechanisms. For example, while Liu et al. (2023a) found that introducing the query before the context can help retrieve information from the context more effectively, they did not achieve improvements in question-answering (QA) performance through this query-aware approach. As mentioned in the first part of this invention, the reason may lie not only in the challenge of retrieval but also in the complexity of effectively applying the retrieved knowledge.

[0048] To address the aforementioned problems, this invention introduces attribution reasoning, a strategy that requires language models to associate claims made during reasoning with specific parts of the provided context. This implicit requirement effectively decomposes a complex multi-hop problem into two more manageable tasks: identifying relevant information in the context and constructing credible claims based on that information.

[0049] This invention employs the concept of Chain-of-Thought (CoT) reasoning (Wei et al., 2022) to create two distinct variants consistent with the attribution-based approach of this invention: Chain-of-Citation (CoC) and Chain-of-Quote (CoQ). In CoC, the model is prompted to refer to the citation corresponding to each step of the thought chain. CoQ further requires the model to include a direct quote of the cited material at each reasoning step. Table 1 provides an example highlighting the nuances of these approaches.

[0050] Table 1

[0051]

[0052] Table 1: An example of CoT and the two reasoning and attribution methods of this invention: CoC and CoQ. This invention emphasizes the differences between these methods; for example, answers are marked in green, citations in orange, and references in blue.

[0053] Table 2

[0054]

[0055] Table 2: Exact Match (EM) and F1 scores for ChatGPT and Claude-instant on multi-hop inference datasets, using 5-shot hints such as MuSiQue, 2WikiMultiHopQA (2Wiki for short), and HotpotQA. Best results are bolded. AO indicates that the model only predicts the answer.

[0056] Table 2 compares the effectiveness of applying CoT, CoC, and CoQ on two proprietary long context models (LMs): ChatGPT (Brown et al., 2020) and Claude-instant (Bai et al., 2022). In this study, ChatGPT consistently refers to gpt-3.5-turbo-1106, while Claude-instant refers to claude-instant-1.2. The results show that CoC and CoQ generally outperform CoT, indicating that attribution-based reasoning improves the accuracy and coherence of the model's (the model's) reasoning process. CoQ appears to perform slightly worse than CoC, possibly due to the increased complexity of generating accurate quotations.

[0057] It is noteworthy that even when CoT degrades AO-only performance, CoC not only mitigates this decline but also outperforms AO benchmarks. This demonstrates the potential of CoC as a powerful inference method. The methods by which this invention has achieved success on various open-source models are further elaborated in Section 4.3. Based on these insights, this invention adopts CoC as the primary inference format in subsequent chapters.

[0058] Dataset Construction

[0059] Based on the data in Tables 2 and 5, the analysis of this invention confirms that while attribution reasoning has potential, smaller open-source long-context language models are significantly inferior to their proprietary models in multi-hop reasoning tasks. To address this issue, this invention investigates whether training these models to perform attributions can improve their reasoning capabilities.

[0060] One obstacle in this process is the lack of attribution annotations in existing multi-hop inference benchmarks. To fill this gap, this invention generates new annotations by applying 5-shot CoQ hints to ChatGPT. This is to create a chain of attributions (CoT) for 5,000 instances randomly selected from the answerable training set. While CoC is generally superior to CoQ, this invention chooses CoQ for annotation because it provides more detailed information. This richness not only benefits the evaluation of annotation quality but also facilitates the fine-tuning process discussed in Section 3.

[0061] Table 3: Occurrence Rate of Different Error Types

[0062]

[0063] Table 4: Statistical Data of MuSiQue-Attribute

[0064]

[0065] After generating the annotations, this invention implements a filtering process to exclude any erroneous annotations, including:

[0066] • Incorrect answer: The model's predicted answer is inconsistent with the reference answer, which usually indicates an error in the chain (CoT).

[0067] • Non-existent attribution: Fictitious references or references that do not fit the actual context indicate that the model is creating an illusion.

[0068] • Incorrect citation: The citation does not match the supporting facts identified manually, suggesting a flaw in the attribution.

[0069] • Redundant citations: Redundant citations violate the requirement of multi-hop retrieval of sources from multiple documents.

[0070] • Extreme quotations: Quotes that are too brief (less than five words) or too lengthy (spanning the entire document) lack practicality.

[0071] Table 3 shows the significant incidence of each error type, which may negatively impact fine-tuning results. After filtering, this invention yields a training dataset of 1,358 samples, termed MuSiQue-Attribute. Statistics for the MuSiQue-Attribute training set are listed in Table 4. Notably, the hop count distribution in MuSiQue-Attribute is skewed. This skewness arises both from the fact that chains (CoTs) generated for problems with higher hop counts are more prone to errors, and from the fact that such problems constitute a smaller proportion of the original MuSiQue training set.

[0072] Learning Attribution Through Reasoning

[0073] An intuitive approach to enhancing the multi-hop reasoning capabilities of language models (LMs) is to perform... Fine-tuning, using the meticulously crafted MuSiQue-Attribute of this invention, teaches them to incorporate attribution into their reasoning process, specifically, to generate CoCs. While this approach is simple, subsequent analysis in Section 4.4 shows that this direct method fails to produce robust results.

[0074] Multi-task learning: In addition to simply fine-tuning the language model using MuSiQue-Attributes to learn attribution in reasoning (denoted as LA), this invention proposes three auxiliary tasks as simplified analogies to LA. These tasks are designed to train the LM in conjunction with LA to enhance its attribution-based reasoning capabilities:

[0075] • Answer Prediction (AP): This task focuses on directly predicting the answer without requiring an explicit reasoning process. AP aims to help LM internalize the reasoning needed for simple questions where CoT is not required.

[0076] • CoT Generation (CG): In the CG task, the model is trained to generate CoTs before providing the answer. This aims to cultivate the LM's ability to reason explicitly and systematically across multiple pieces of information on complex problems.

[0077] • Quotation Recognition (QI): This task trains the model to identify key quotations needed for inference. QI aims to fine-tune the LM's ability to filter out irrelevant details and focus on relevant segments of the text, thereby improving the accuracy of inference.

[0078] Figure 1 This explains the differences between the main LA task and the three auxiliary tasks of the present invention.

[0079] Data Augmentation: A recognized limitation of directly fine-tuning the MuSiQue-Attribute is that the model may become biased, such as favoring certain relevant document positions (Liu et al., 2023a), being sensitive to a fixed number of documents, or only adapting to a narrow noise level. To combat these biases, this invention designs the following data augmentation strategies:

[0080] • Interference sample sampling: By randomly selecting different numbers of irrelevant documents, this invention modifies the position of relevant documents in the context and the total number of documents. This method also simulates the fluctuating noise levels encountered in real-world scenarios, effectively training the language model to cope with noisy environments.

[0081] • Document shuffling: Rearranging documents helps remove any surface positional cues that could lead to biased reasoning. For example, this ensures the model doesn't learn to associate the order of related documents with a fixed sequence of thought chains. These data augmentation strategies are applied sequentially in each training instance.

[0082] experiment

[0083] Dataset

[0084] The effectiveness of the proposed method in multi-hop inference was evaluated on the following datasets: HotpotQA (Yang et al., 2018), 2WikiMultiHopQA (abbreviated as 2Wiki) (Hoetal., (2020) and MuSiQue (Trivedi et al., 2022). For each problem, a context consisting of shuffled relevant and irrelevant documents is provided, with the irrelevant documents being the distractors from the official retrieval. Evaluation is performed using the validation and test sets provided by Trivedi et al. (2023), containing 100 and 500 examples, respectively. Results are derived from the average of three independent trials, each with a different random seed.

[0085] Table 5: Exact Match (EM) Results on Three Multi-Hop Inference Datasets

[0086]

[0087] The best small-scale long-context LM results are shown in bold, while the best baseline results are shown in underline.

[0088] Model

[0089] Select the following long text baseline model:

[0090] -ChatGPT (Brown et al., 2020). Select gpt-3.5-turbo-1106, which supports 16K standards. Token The window size.

[0091] -Claude-instant (Bai et al., 2022). Select claude-instant-1.2 with a window size of 100K tokens.

[0092] -LongChat (Li et al., 2023a). Using longchat-7b-16k, a finely tuned 7BLLaMA model (Touvron et al., 2023a). Its window size is 16K tokens.

[0093] -LongLoRA (Chen et al., 2023b). Using LongAlpaca-7B-16k, the window size is 16K.

[0094] -Vicuna (Zheng et al., 2023b). Using vicuna-7b-v1.5-16k, which is a 7B fine-tuned LLaMA-2 model (Touvron et al., 2023b). It supports a window size of 16K tokens.

[0095] Five-shot examples were used to prompt all models to evaluate their multi-hop inference performance. These five examples were randomly selected from 20 labeled training examples provided by Trivedi et al. (2023). If the input length exceeded the window size, the last example was discarded until the input length was met.

[0096] The proposed model, AttrLoRA, is fine-tuned on vicuna-7b-v1.5-16k using LoRA (Hu et al., 2022), following the hyperparameters used in FastChat (Zheng et al., 2023b). For its training data, this invention augments the data for all tasks in Part III, doubling the training data except for the QI task. Notably, this invention randomly samples a similar amount of instruction fine-tuning data from the Alpaca dataset (Taori et al., 2023) and mixes it with the inference data. The purpose of this instruction fine-tuning data is to minimize the risk of hindering Vicuna's existing capabilities before fine-tuning.

[0097] Main results

[0098] Validity and robustness of reasoning and attribution

[0099] Table 5 highlights the effectiveness of the CoC hint on three multi-turn inference datasets and compares it with five benchmarks. CoC outperforms CoT in 77% of the evaluation cases (ignoring instances with near-zero model performance). Notably, Claude-instant performs well on AO, but its performance degrades when using CoT. However, CoC not only mitigates this degradation but also achieves results comparable to AO, demonstrating the robustness of attribution-based inference.

[0100] Figure 2The exact match (EM) results for different models at various noise levels are shown on three multi-hop inference datasets. Notably, all models, except for AttrLoRA of this invention, use 5 example hints. A higher noise ratio indicates that more distractors, i.e., irrelevant documents, are presented in the test instance and the background in the teaching.

[0101] Table 6 Results based on general instruction compliance standards

[0102]

[0103] "+Alpaca data" is the Vicuna-7B model that is further fine-tuned based on Alpaca data.

[0104] Performance comparison of AttrLoRA and proprietary models

[0105] Table 5 shows a comparison between the zero-shot performance of AttrLoRA of the present invention and the 5-shot performance from various baselines. AttrLoRA outperforms baselines of similar size by an average of 20 points. It outperforms two significant proprietary models on MuSiQue and delivers competitive results on two other benchmarks.

[0106] Specifically, AttrLoRA and AO outperform CoT on 2Wiki and also surpass CoT on HotpotQA. This is because these datasets are relatively simple, and explicit reasoning does not significantly improve model performance. For example, on these datasets, CoT's advantage is significantly smaller than MuSiQue on ChatGPT and Claude-instant. Furthermore, Jiang and Bansal (2019) point out that over half of the "bridge-type" questions in HotpotQA contain shortcuts, allowing answers to be located through keyword matching, thus bypassing the required two-step reasoning. Similarly, 2Wiki questions, being constructed from a finite set of rules, are predictable, thus simplifying the task of LMs (language models). Another reason is that AttrLoRA is trained on MuSiQue-Attribute, a dataset that does not contain all types of questions found in 2Wiki and HotpotQA, such as "comparison-type" questions.

[0107] AttrLoRA's adaptability to noisy contexts

[0108] A key aspect of AttrLoRA is its robustness to contextual noise. To investigate this, Figure 2The performance of AttrLoRA under different levels of synthetic noise is demonstrated. This synthetic noise is achieved by adding varying numbers of random, irrelevant documents to the context. Data shows that while the performance of the baseline model significantly decreases with increasing noise—for example, Vicuna's score on MuSiQue drops by more than 30 points—AttrLoRA exhibits stronger adaptability, reducing the noise by only about 10 points.

[0109] The impact of attribution learning on general competence

[0110] This invention not only studies multi-hop reasoning, but also examines how attribution learning affects AttrLoRA. General instruction following capability compared to the Vicuna baseline after fine-tuning. Table 6 shows results from the two instruction following benchmarks, indicating that fine-tuning slightly impairs capabilities in the 7B model except for multi-hop inference due to capacity limitations. However, deeper analysis reveals that over 98% of the performance degradation is attributable to fine-tuning using Alpaca data (“+Alpaca Data”), while the loss from multi-hop inference data is less than 2%. This is because the quality of Alpaca data is lower than that of Vicuna; the former is synthesized from single-turn GPT-3 data, while the latter includes multi-turn human-computer dialogue.

[0111] analyze

[0112] Figure 3 The multi-hop inference performance of AttrLoRA is compared with its reference precision and recall.

[0113] AttrLoRA's attribution quality

[0114] Based on Gao et al. (2023), this invention carefully examined the citation precision and recall of AttrLoRA, such as Figure 3 As shown, the model exhibits high accuracy, demonstrating its ability to correctly attribute statements to relevant documents. However, the moderate recall suggests that AttrLoRA does not always identify all relevant documents, possibly due to inconsistent inference patterns observed in the model on the MuSiQue-Attribute dataset.

[0115] The effectiveness of multi-task learning

[0116] The ablation studies in Table 7 evaluated the multi-task learning method of the present invention.

[0117] Table 7 Ablation Study of Multi-Task Learning

[0118]

[0119] The results show that Vicuna's inference ability is significantly enhanced after fine-tuning using the dataset of this invention ("+AP"). However, training Vicuna to generate CoT ("+CG") produces mixed results: it has some benefit on the MuSiQue dataset, but negatively impacts the results on 2Wiki and HotpotQA. This may be because the latter dataset is relatively simple, where simpler questions and shortcuts reduce the effectiveness of complex inference strategies. Notably, integrating the LA task ("+LA") mitigates the performance degradation associated with CoT and significantly improves MuSiQue performance. This means that attribution allows the model to handle complex questions without hindering its ability to handle simple queries. Finally, the addition of the QI task ("+QI")... This appears to further enhance the model's multi-hop reasoning ability, highlighting the multi-task learning frame of this invention. shelf value.

[0120] Table 8: Ablation studies on data augmentation.

[0121]

[0122] The effectiveness of data augmentation

[0123] The impact of the proposed data augmentation strategy is further explored. Since the QI task alone is insufficient to train the model for multi-hop inference, it was deliberately omitted. Table 8 shows that including augmented data generally improves model performance across different datasets. However, augmenting the CG and LA data did not yield improvements on 2Wiki. In this case, the model can easily learn from the limited amount of annotated data due to the simplicity of automatically generated questions within 2Wiki. Conversely, on MuSiQue and HotpotQA, which contain more complex and diverse human-created questions, the model needs to be exposed to larger datasets to adapt to the diversity of question formulations.

[0124] Figure 4 This illustrates the effect of scaling and fine-tuning the data size.

[0125] Scaling and fine-tuning the validity of data

[0126] exist Figure 4In this study, we investigated how the expansion of fine-tuning data affects model performance. Clearly, introducing additional data steadily improved the performance of MuSiQue and 2Wiki, while for HotpotQA, optimal results were achieved using only 60% of the data. This fact suggests that complex question answering systems involving more inference steps, such as MuSiQue and 2Wiki, require larger datasets. An interesting finding is that approximately 85% of the peak performance was achieved using only 20% of the data. This highlights the efficiency of fine-tuning: even a small subset of multi-hop inference examples can significantly improve the model's inference capabilities.

[0127] Case Studies

[0128] Table 9 presents a comparative case study in which both Vicuna and AttrLoRA were prompted to generate CoCs. In the example, AttrLoRA successfully generated coherent chained thought (CoT) and accurately attributed it to each claim. In contrast, Vicuna's generated answers did not involve explicit reasoning.

[0129] Table 9: Two examples from the MuSiQue test set

[0130]

[0131] Vicuna's prediction failed, while AttrLoRA's prediction succeeded. References to this invention are manually highlighted in blue, and citations are highlighted in orange. Correct predictions are marked with √, and incorrect predictions with ×.

[0132] The differences and advantages of this invention compared to existing technologies are as follows:

[0133] Multi-hop reasoning

[0134] In open-domain question answering, multi-hop reasoning requires synthesizing and analyzing discrete facts from different documents to form an answer. Key datasets in this field include HotpotQA (Yang et al., 2018), 2Wiki (Ho et al., 2020), and MuSiQue (Trivedi et al., 2022), which primarily employ reading comprehension frameworks where authors provide pre-retrieved documents. Traditional methods typically use a selector-reader model (Zhang et al., 2023; Zhu et al., 2021), where the selector's task is to identify relevant documents from the provided document set, and the reader constructs the answer based on these selections.

[0135] However, recent research has shifted to a paradigm utilizing long-context models (long-context LMs, Khot et al., 2023; Trivedi et al., 2023). In this approach, the selector is replaced by a long-context LM that processes all retrieved documents, acting as a reader. This invention focuses specifically on using attributions within this long-context LM framework to enhance the performance of multi-hop reasoning.

[0136] Context utilization

[0137] Recent developments in long-context mathematical reasoning (LMs) have shown promising promise (Li et al., 2023a; Zheng et al., 2023b; Chen et al., 2023b). However, these models often struggle with noisy contexts. Shi et al. (2023) demonstrated that redundant statements can significantly interfere with mathematical reasoning. Liu et al. (2023a) showed that models exhibit bias towards relevant document location in multi-document QA. Wu et al. (2024) showed that LMs are susceptible to interference from retrieved irrelevant input.

[0138] To mitigate the impact of irrelevant context, Shi et al. (2023) suggest that the model ignores this type of information. And adopt Self-consistency technique (Wang et al., 2023). Creswell et al. (2023) proposed a... Two-stage approaches focus on fact selection prior to inference. Yu et al. (2023) introduced Chain-of-Note, proposing to examine the relevance of documents before providing answers. Meanwhile, Yoran et al. (2023) examined automated data generation to train more robust models. This invention contributes to the field by investigating attribution use as a novel approach to effective contextual utilization.

[0139] Language model attribution

[0140] In language models, attribution constitutes an emerging research area, primarily aimed at identifying and mitigating hallucinations (Li et al., 2023b). One research approach focuses on post-retrieval responses: models provide responses based on retrieval results with citation attributions (Nakano et al., 2021; Menick et al., 2022; Gao et al., 2023).

[0141] This invention, based on language model attribution, focuses on reducing multi-hop reasoning rather than illusions. Furthermore, it delves into optimizing training methods to maximize the effectiveness of attribution annotations.

[0142] In summary, this invention proposes an innovative method that significantly improves the performance of long-context language models (LMs) in multi-hop reasoning tasks and enhances the model's ability to handle complex problems through a series of strategies. The core of this method is "attributional reasoning," which requires the model to provide explicit attributional information during reasoning. This mechanism effectively solves the problem of identifying relevant information in noisy contexts and improves the efficiency of knowledge integration, especially for smaller models.

[0143] The technological innovation mechanism of this invention is mainly reflected in the following aspects:

[0144] 1. Introduction of attributional reasoning: By requiring the model to cite specific evidence during the reasoning process, the transparency and accuracy of the reasoning are improved.

[0145] 2. Creation of the attribution annotation dataset “MuSiQue-Attribute”: This fills the gap in attribution annotation in existing datasets and provides rich, high-quality examples for model training.

[0146] 3. Application of multi-task learning framework: By combining auxiliary tasks such as answer prediction, thought chain generation and reference recognition, the model's reasoning ability is comprehensively improved.

[0147] 4. Data augmentation strategy design: By using techniques such as perturbation sampling and document shuffling, the model's dependence on specific context layout is reduced, and the model's robustness to noise is enhanced.

[0148] Compared with existing methods, the advantages of the present invention mainly include:

[0149] - In multi-hop inference benchmarks, the improved model "AttrLoRA" significantly outperforms the baseline model of the same size and, in some cases, approaches or surpasses the proprietary model.

[0150] - It demonstrates strong robustness to noisy contexts, maintaining high performance even in high-noise environments.

[0151] - The open-source compatibility of this method ensures its accessibility and applicability, enabling it to be widely applied to different models and application scenarios.

[0152] - Through customized training strategies and data augmentation, the model is able to learn and adapt to multi-hop inference tasks more effectively.

[0153] In terms of application value, this invention provides a powerful framework for long-context logical reasoning (LLM), applicable to fields such as question-answering systems, data analysis, and research automation. It can handle and answer complex questions requiring the synthesis of information from multiple sources. Through this invention, LLM can more effectively perform multi-hop reasoning tasks, improving the accuracy and reliability of automated information processing and contributing significant technological advancements to the field of artificial intelligence.

[0154] In summary, this invention introduces an attribution-integrated reasoning paradigm that significantly improves the reasoning capabilities of long-context models. Furthermore, this invention proposes a new dataset with attribution annotations and investigates training strategies for multi-hop reasoning. Comprehensive experiments on five models and five benchmarks validate the superiority of the proposed method in improving multi-hop reasoning performance.

[0155] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0156] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0157] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0158] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk drive or magnetic tape drive. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0159] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and other methods may be used in actual implementation. The division method, such as: multiple units or components can be combined, or It can be integrated into another system. Some features may be ignored or not performed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed may be indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or other forms.

[0160] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0162] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0164] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0165] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0166] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0167] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should be noted that the specific implementation of this invention is not limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of this invention, and all such modifications, with the same performance or purpose, should be considered within the scope of protection of this invention.

Claims

1. A method for improving the multi-hop reasoning capability of long-context language models, characterized in that, Includes the following steps: Prepare a basic language model with long context processing capabilities. This model should have a broad pre-trained knowledge base and introduce an attribution reasoning mechanism to require the model to provide explicit attribution information for each assertion when performing multi-hop reasoning. We use a language model to generate attribution-based annotations and employ the 5-shot CoQ hint method to create preliminary attribution annotations, thus constructing an attribution annotation dataset. A multi-task learning framework is constructed, including answer prediction, CoT (Copy of Thought) generation, and reference recognition tasks, to enhance the model's reasoning ability, and data augmentation strategies are implemented for the attribution annotation dataset. Using the data-augmented attribution annotation dataset, the base language model is fine-tuned in conjunction with the multi-task learning framework to generate an improved model capable of performing attribution inference. During the fine-tuning process, the improved model is made to integrate the CoC (Coreference Chain) and CoQ (Coreference Query) chains, and the model parameters are adjusted based on feedback from the multi-task learning framework and the data augmentation strategy to maximize the effectiveness of attribution inference.

2. The method as described in claim 1, characterized in that, The attribution reasoning mechanism includes: By adopting the concept of Mind Chain of Thought (CoT), the complex multi-hop problem is decomposed into two manageable tasks: identifying relevant information in the context and constructing credible claims based on that information. The reference chain CoC mechanism, in which the model is prompted to refer to the citations corresponding to each step of the thought chain during the reasoning process, to ensure that the claim is relevant to a specific part of the context; The Citation Chain CoQ mechanism further requires that the model include direct citations of the cited material in each reasoning step to enhance the credibility and clarity of the claims; Through fine-tuning, the model is able to: accurately integrate the citation chain CoC and the citation chain CoQ; and explicitly link claims to relevant fragments in the context through citation hints or direct quotations.

3. The method as described in claim 2, characterized in that, In the citation chain mechanism, the citation directly points to the specific document or information source supporting the claim through index tags; in the citation chain mechanism, the direct citation explicitly indicates the cited text content and source through quotation marks and index tags.

4. The method according to any one of claims 1 to 3, characterized in that, The design and construction of the attribution annotation dataset specifically includes: Randomly select multiple instances from the responsive training set; Using a language model for 5-shot CoQ prompts, attribution-based annotations are generated for the randomly selected instances; Create a thought chain (CoT) with attribution for the instance, so that each reasoning step has a corresponding reference or citation to support it.

5. The method according to any one of claims 1 to 4, characterized in that, The design and construction of the attribution annotation dataset also includes: after generating annotations, implementing a filtering process to exclude the following erroneous annotations: Incorrect answer: The model's predicted answer is inconsistent with the reference answer; Attribution for non-existence: fictitious references or references that do not fit the actual context; Incorrect reference: The reference does not match the manually identified supporting facts; Duplicated references: Redundant references violate the requirement of multi-hop retrieval of sources from multiple documents; Extreme citations: Citations that are too brief or too lengthy.

6. The method according to any one of claims 1 to 5, characterized in that, The fine-tuning of the basic language model using a multi-task learning framework includes implementing the following auxiliary tasks on top of the main attribution learning task (LA): Answer prediction task: The answer prediction task focuses on directly predicting the answer without requiring an explicit reasoning process; CoT Generation Task: The CoT generation task trains the model to generate a thought chain CoT before providing an answer; Reference recognition task: The reference recognition task trains a model to identify key references required for inference.

7. The method according to any one of claims 1 to 6, characterized in that, The data augmentation strategies implemented for the attribution annotation dataset include: Interference sample sampling: By randomly selecting different numbers of irrelevant documents, the position of relevant documents in the context and the total number of documents are modified; Document shuffling: By rearranging documents, surface location clues that may lead to biased reasoning are removed.

8. The method as described in claim 7, characterized in that, The above data augmentation strategies are applied sequentially in each training instance to ensure that the model is fully exposed to various noise levels and document arrangements during training.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for improving the multi-hop reasoning capability of a long-context language model as described in any one of claims 1-8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for improving the multi-hop reasoning capability of a long-context language model as described in any one of claims 1-8.

Images