Retrieval enhancement generation system for instruction following alignment

Abstract

The invention provides a retrieval enhancement generation system for instruction compliance alignment. Comprising two parts: an instruction synthesis stage and an instruction query synthesis stage; in the instruction synthesis stage, a seed instruction is handwritten firstly, then instruction combination and verification are carried out, and then the quality of an automatically generated instruction is verified; in the instruction query synthesis stage, firstly, random instruction query combination is performed, then instruction query enhancement and dual verification are performed, a sampling rejection strategy is adopted, and then a dual verification process is performed on instruction query data: based on compiler feedback, the compliance of enhancement output is evaluated by using an existing verification function; and checking the consistency of the large model: finally, automatically obtaining a large-scale and high-quality VIF-RAG-QA data set. The method can be seamlessly connected to diversified instruction following and retrieval enhancement to generate an evaluation benchmark, so that combined evaluation is realized.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and more particularly, to an instruction-compliant aligned retrieval enhancement generation system. Background Art

[0002] In real-world user-model interaction scenarios, users often do not follow standard interaction templates when asking questions, but instead impose diverse instructions on the model output to meet specific task requirements. Therefore, having a strong instruction following capability is the application basis of large language models and retrieval-augmented generation systems (RAG). The core goal of instruction following is to enable the model to adapt to the diverse intentions of users, which has attracted widespread attention in the large language model community. Existing technical solutions mainly explore the two parts of large language models and retrieval-augmented generation.

[0003] Instruction following alignment of large models: Instruction following capability is one of the most core basic capabilities of large language models. Existing work can be divided into two categories. The first group focuses on how to comprehensively evaluate the instruction following capability of existing large models. Some pioneering work such as C-Eval and MT-bench is committed to comprehensively evaluating the model's ability to follow general natural instructions. A series of work represented by IFEval[1] aims to focus on the fine-grained evaluation of specific instruction constraints and verify them based on rules. It is currently the most widely used instruction following evaluation benchmark for large language models. Furthermore, FollowBench, Infobench, CFbench, Complexbench, and RefuteBench have made more complex designs in terms of instruction difficulty, domain, combination type, and task form, and have proposed more stringent standards for the instruction following capability of large models. In addition to evaluation, another series of work continues to focus on how to improve the instruction following alignment of large models. The simplest and most natural way is for annotators to manually design instructions and write responses. However, creating highly complex and diverse instructions is challenging for humans, and the high cost of annotation and verification also makes this process difficult to scale. To alleviate this problem, another class of methods aims to first synthesize diverse instruction data and let weaker models imitate the behavior of high-level language models such as GPT4's responses, thereby achieving strong-to-weak instruction compliance alignment.

[0004] Alignment of Retrieval-Augmented Generation: Retrieval-Augmented Generation (RAG) provides a promising solution to the problem of knowledge hallucination in large language models by retrieving relevant factual information. However, it remains a challenge to efficiently align the retrieved knowledge with the preferences of large language models. Many researchers have devoted themselves to studying re-ranker-based methods such as LRL, PRP, etc. to reduce noise information to bridge this gap. Furthermore, methods represented by FILCO achieve further alignment by filtering models through training data. In addition, preference alignment methods represented by KnowPAT integrate the preferences of large language models into training objectives to improve alignment. SelfRAG and MetaRAG use multiple rounds of retrieval and generation to optimize the output and achieve better alignment. Despite these advances, the diverse knowledge introduced by retrieval-augmented techniques still poses a significant challenge for large language models to handle complex instructions. This highlights the need for further exploration of effective instruction-following alignment in RAG systems.

[0005] Although existing instruction following techniques have achieved certain results and progress, in complex retrieval-augmented generation (RAG) scenarios, the diverse knowledge introduced by retrieval-augmented techniques poses a major challenge to LLMs in effectively processing complex instructions. To quantitatively analyze this problem, we compared the performance of the base model with that of the model after fine-tuning it on high-quality general and knowledge-based question-answering datasets. The fine-tuned version of the model shows strong capabilities in both instruction following and retrieval-augmented generation tasks (Mistral-base vs. Mistral-SFT). These capabilities do not always generalize well to instruction following tasks in retrieval-augmented generation scenarios, and may even conflict with other basic capabilities. Unfortunately, current research on instruction following in retrieval-augmented generation systems is still limited, which seriously hinders its application in real-world interactions. We summarize the above challenges into the following two points:

[0006] (1) How to comprehensively evaluate complex instruction-following capabilities in retrieval-augmented generation scenarios.

[0007] (2) How to achieve scalable and reliable instruction-following alignment in a retrieval-augmented generation system while avoiding conflicts with its underlying capabilities. Summary of the invention

[0008] The purpose of the embodiments of the present disclosure is to provide a retrieval enhancement generation system that follows alignment of instructions.

[0009] In a general aspect, a retrieval enhancement generation system for instruction-compliant alignment is provided, comprising two parts: an instruction synthesis stage and an instruction query synthesis stage; the instruction synthesis stage receives a small batch of text restriction instructions and outputs a large batch of text restrictions that have undergone multiple quality checks; in the instruction query synthesis stage, external input variables include three parts: a user's query question, a restriction instruction generated in the instruction synthesis stage, and relevant knowledge recalled by a dense retriever, and the output is an answer to the user's query question in the form of text that satisfies the user's complex instruction restrictions; finally, the synthesized alignment data is constructed into a universal alignment dataset;

[0010] The instruction synthesis stage firstly writes seed instructions manually, manually creates 15 single atomic instructions for each type of constraint; then performs instruction combination and verification, designs rules, and automatically combines atomic instructions into complex instructions constrained by multiple chain rules to obtain the initial seed instruction set D seed ; Then the instructions are rewritten and the quality is verified, using a supervised model GPT-4 with batches of 50 instructions per round, starting from the initial seed instruction set D seed Iterate and rewrite the instructions, perform K rounds of operations, and generate an enhanced instruction set D aug , and merge the seed sample with the enhanced sample to form the merged instruction set D ins =D seed ∪D aug , and remove duplicates; then verify the quality of the automatically generated instructions;

[0011] In the instruction query synthesis stage, random instruction query combination is first performed. For the retrieval enhancement generation domain, a query set Q is randomly selected from a hybrid QA data source, which includes open domain multi-hop and knowledge base QA scenarios. A dense retriever R is used to retrieve the top K relevant documents D for each query q∈Q from the external knowledge base. i , thus generating a data set For each instruction I, randomly select D RAG The K queries and their corresponding retrieved documents in , and combine them to get the RAG query set with IF constraints

[0012] For the general domain, for each instruction I∈D ins , randomly select K queries from ShareGPT and combine them with the instructions to build a general domain dataset D IF-General Finally, the instruction constraint query sets of these two fields are merged to form the final query set of VIF-RAG-QA, which is expressed as After that, we conduct command query enhancement and double verification, adopt the rejection sampling strategy, and use the supervision model to generate a single command-query pair. Generate K replies Thus we get {x,y x}∈

[0013] D VIF-RAG , and then a double verification process is adopted for the instruction query data: Based on compiler feedback: Use the existing verification function to evaluate the compliance of the enhanced output; Large model consistency check: Use the supervised model to score the match between the query and the instruction from 1 to 10, and filter out samples with a score lower than 8; Finally, a large-scale, high-quality VIF-RAG-QA dataset is automatically obtained.

[0014] The specific process of verifying the quality of automatically generated instructions is as follows: for each instruction I∈D ins , use the supervised model to generate K verification function codes and corresponding test cases And evaluate the quality of the instruction by analyzing the output of the executor E; for any function and test case The execution result is:

[0015]

[0016] Then calculate the accuracy Acc of each verification function based on K test samples func , and use K validation functions to evaluate the accuracy Acc of each test case case , the formula is as follows:

[0017]

[0018] Based on the above cross-metric criteria, each instruction is required to have at least one Acc func and Acc case More than 0.5; finally, get an automatically verified instruction set:

[0019]

[0020] Other samples that do not meet the criteria are discarded.

[0021] The specific process of using the existing verification function to evaluate the compliance of the enhanced output is that at least one response in the VIF-RAG data set must achieve an accuracy rate exceeding 0.5 in all verification functions, otherwise the sample is discarded.

[0022] The innovation of the embodiment of the present invention lies in:

[0023] (1) In order to achieve instruction compliance alignment in the retrieval-augmented generation scenario, a verifiable automatic instruction data synthesis framework for retrieval-augmented generation (VIF-RAG) is proposed. VIF-RAG combines enhanced rewriting with a diversified verification process, including compiler feedback-based verification and large model consistency verification, to synthesize high-quality instruction compliance alignment data from a very small number of seed instruction samples (<100) and expand to more than 100,000 samples.

[0024] (2) In order to solve the problem of lack of comprehensive evaluation of retrieval enhancement generation scenarios, a robust benchmark construction process is designed, which includes three steps: instruction sampling and collection, retrieval and reconstruction, and combination and verification. The unique advantage of this framework is that it can seamlessly connect to a variety of instruction following and retrieval enhancement generation evaluation benchmarks to achieve combined evaluation.

[0025] (3) Based on the above benchmark building framework, we propose FollowRAG, which is the first benchmark designed to comprehensively evaluate the ability of large language models to follow complex instructions in the retrieval-augmented generation (RAG) task. FollowRAG contains nearly 3,000 test samples, covering 4 knowledge-intensive question-answering benchmarks and 22 constraints. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects and features of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings.

[0027] Figure 1 is a schematic diagram showing a verifiable instruction compliance retrieval enhancement generation framework according to an embodiment of the present disclosure;

[0028] Figure 2 is a schematic diagram showing a FollowRAG evaluation benchmark according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0030] The features described herein can be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, which will be clear after understanding the disclosure of the present application.

[0031] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more.

[0032] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, the first member, first component, first region, first layer, or first portion referred to in the examples may also be referred to as the second member, second component, second region, second layer, or second portion.

[0033] In the specification, when an element (such as a layer, a region or a substrate) is described as being “on”, “connected to” or “coupled to” another element, the element may be directly “on”, “connected to” or “coupled to” another element, or one or more other elements may be present therebetween. Conversely, when an element is described as being “directly on”, “directly connected to” or “directly coupled to” another element, there may be no other elements present therebetween.

[0034] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs after understanding the present disclosure. Unless explicitly defined as such herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and should not be interpreted in an idealized or overly formal manner.

[0036] Furthermore, in the description of examples, when it is considered that a detailed description of a well-known related structure or function would cause vague interpretation of the present disclosure, such a detailed description will be omitted.

[0037] Figure 1 is a schematic diagram showing a retrieval enhancement generation system based on instruction-following alignment according to an embodiment of the present disclosure.

[0038] We propose a verifiable automatic instruction data synthesis framework (VIF-RAG) for retrieval augmentation generation scenarios. The core of VIF-RAG is that each step of automatic generation or combination is accompanied by a proper verification process. The VIF-RAG framework can be divided into two parts: the instruction synthesis phase and the instruction query synthesis phase, which scales from almost zero (<100) to more than 100,000 high-quality instruction query samples. Below we will discuss its specific implementation details in depth.

[0039] Specifically, in the instruction synthesis stage, the system receives small batches of text restriction instructions, which regulate the format and style of the answers during the user query process. The output of the system is a large batch of text restrictions that have undergone multiple quality checks, which are superior to the input instructions in terms of quality, richness and scope of application. In the instruction query synthesis stage, the external input variables include three parts: the user's query question, the restriction instructions generated in the instruction synthesis stage, and the relevant knowledge recalled by the dense retriever. The output of the system is presented in the form of text, which can accurately answer the user's query question and meet the user's complex instruction restrictions. Finally, the synthesized alignment data forms a flexible and versatile general alignment dataset. This dataset aims to improve the human instruction following ability of large language models and retrieval enhancement systems while ensuring the accuracy of the answers.

[0040] Instruction synthesis stage

[0041] 1Handwritten seed instructions

[0042] Data synthesis from hand-written a minimal seed instruction set Four basic constraint categories: format constraints, semantic constraints, knowledge constraints, and vocabulary constraints were used as the subject of the instructions. We only hired one human annotator to manually create 15 single atomic instructions for each type of constraint. It is worth noting that this is the only step in our data synthesis process that includes human supervision.

[0043] 2. Instruction combination and verification

[0044] Real-world instructions often involve multiple constraints. To address this complexity, we designed rules to automatically combine atomic instructions into diverse complex instructions:

[0045] (1) Multiple constraints: Figure 1As shown, we We randomly extract pairs of instructions from the training set and insert them into a constraint template. By directly connecting these instruction pairs, we create complex instructions with double and triple constraints. Such instructions require the model to generate results that satisfy multiple constraints simultaneously.

[0046] (2) Chain rule constraint: We designed a sequential condition constraint template and Atomic instructions are selected to form chain constraints. Formally, the chain consists of n tasks {T1, T2, ..., T n}, requiring the output of the model to complete these tasks in sequence.

[0047] (3) Verification: Randomly combining these atomic instructions can easily lead to conflicts between them (e.g., not using words containing the letter "I", using words ending with "ing"). These semantic conflicts are difficult to detect using simple natural language inference models. To evaluate the potential conflicts between these instructions, we use a strong supervised model to evaluate their consistency on a scale of 1 to 10. Samples with a score lower than 8 will be excluded to optimize our high-quality complex instruction set. Finally, we get the initial seed instruction set

[0048] 3. Instruction rewriting and quality verification

[0049] In order to automatically expand the instruction set, the instruction rewriting strategy is considered to be the most natural enhancement method and has received extensive attention in the fields of RAG and reasoning. A supervised model GPT-4 is used to train the instruction set from D seed Iterate and rewrite the instructions in the set, perform K rounds of operations, and generate an enhanced instruction set D aug Subsequently, we merge the seed sample with the enhanced sample to form the merged instruction set D ins =D seed ∪D aug , and remove duplicates.

[0050] Inspired by the work on compiler feedback, we use the powerful encoding capabilities of LLMs to help verify the quality of automatically generated instructions. Figure 1 As shown, for each instruction I∈D ins , use the supervised model to generate K verification function codes and corresponding test cases The quality of the instruction is evaluated by analyzing the output of the executor E. For any function and test case The execution result is:

[0051]

[0052] Therefore, the accuracy Acc of each verification function is calculated based on K test samples func , and use K validation functions to evaluate the accuracy Acc of each test case case The formula is as follows:

[0053]

[0054] Based on the above cross-metric criteria, each instruction is required to have at least one Acc func and Acc case More than 0.5. Finally, we get an automatically verified instruction set:

[0055]

[0056] Other samples that do not meet the criteria will be discarded.

[0057] Extensible command-query synthesis

[0058] 1 Random command-query combination

[0059] In practical interaction with RAG systems, achieving instruction-following (IF) alignment relies on effectively combining synthesized instructions with queries used by the RAG system. To achieve this goal, Figure 1 As shown, we first extract high-quality queries from two different data sources.

[0060] (1) Retrieval-enhanced generation domain: Building an effective retrieval-enhanced generation system requires preparing sufficient QA format data and containing relevant knowledge to enhance its knowledge-based interaction capabilities. Therefore, we randomly select a query set Q from a hybrid QA data source, which includes open domain multi-hop and knowledge base QA scenarios. Following the retrieve-then-read paradigm, we use a dense retriever R to retrieve the top K relevant documents D for each query q∈Q from an external knowledge base. i , thus generating a data set In addition, we randomly select D for each instruction I RAG The K queries and their corresponding retrieved documents in the query are combined to create a RAG query set with IF constraints.

[0061] (2) General domain: In addition to specific capabilities such as retrieval-enhanced generation, the retrieval-enhanced generation system must also have basic human-aligned capabilities to meet users' daily interaction needs. Therefore, ShareGPT provides real multi-round human dialogue data, which is our natural choice. Similar to the processing method in the retrieval-enhanced generation field, for each instruction I∈D insWe randomly select K queries from ShareGP T and combine them with instructions to construct a general domain dataset D IF-General Finally, we merge the instruction constraint query sets of these two domains to form the final query set of VIF-RAG-QA, which is expressed as

[0062] 2. Command-query enhancement and double verification

[0063] Under diverse instruction-following constraints, retrieving the original true answers to queries in augmented generation and general datasets becomes unreliable. To address this issue and improve the diversity of synthetic data, we adopt a rejection sampling strategy. Specifically, we use a supervised model to generate a Generate K replies Thus we get {x,y x}∈D VIF-RAG .

[0064] To further ensure comprehensive quality control of the synthetic dataset, we employ a two-fold validation process for the command-query data:

[0065] (1) Compiler feedback-based: To automatically verify whether the responses generated by the model comply with the constraints of the instruction-query sample, we use existing verification functions to evaluate the compliance of the augmented output. As described in the “Instruction Rewriting and Quality Verification” section, at least one response in the VIF-RAG dataset must achieve an accuracy (Acc_case) of more than 0.5 in all verification functions, otherwise the sample will be discarded.

[0066] (2) Large model consistency verification: We noticed that the combined instructions and queries often conflict. A simple example is that the query "Please write a short biography about a specific person" does not match the instruction "Strictly limit the answer to 10 words." Based on the previous instruction consistency verification, we use a supervised model to score the match between the query and the instruction from 1 to 10, and filter out samples with a score lower than 8. After two-stage verification, we automatically obtained a large-scale, high-quality VIF-RAG-QA dataset.

[0067] FollowRAG Benchmark

[0068] 1 Dataset Construction

[0069] Data Collection and Extraction: FollowRAG aims to evaluate the model's ability to follow user instructions in complex multi-document contexts. The definitions and examples of atomic instructions are collected from general instruction following (IF) datasets such as IFEval and FollowBen ch and verified by rules (e.g., code), excluding parts that are not relevant to the retrieval enhancement generation scenario. Finally, 22 types of instruction constraints are determined, covering aspects such as language, length, structure, and keywords.

[0070] Retrieval and instruction reconstruction: Figure 2 , using existing question-answering datasets such as Natural Questions as the basis for building FollowRAG samples. For queries extracted from the question-answering dataset, a complex instruction with 1 to 4 atomic instruction constraints needs to be generated. In order to enhance the diversity of atomic instruction representations, we adopt GPT-4o as the instruction generator. Specifically, for a given query, n instructions are first extracted from the atomic instruction set and conflict detection is performed. Subsequently, guided by examples, we prompt the large language model to generate new diverse instruction texts for each atomic instruction, along with parameters for evaluating instruction following.

[0071] Data combination and verification: Finally, the retrieved text, query, and atomic instructions are integrated to construct the sample input of FollowRAG. In order to avoid mechanically splicing queries and instructions together in a template manner, we prompt the supervised model to naturally combine multiple atomic instructions with queries into a coherent instruction-query paragraph. Then, we add the first K document paragraphs retrieved based on the query to the instruction-query paragraph to form a complete sample input.

[0072] 2. Evaluation and Statistics

[0073] After obtaining the model output, we evaluate it from two perspectives: instruction following and question answering under the retrieval-augmented generation paradigm:

[0074] (1) Instruction following: Using the verifiable characteristics of atomic instructions, we automatically evaluate the model’s compliance with each instruction through code verification, referring to the IFEval method. We then calculate the average pass rate of each atomic instruction in all samples to determine the instruction following score of FollowRAG.

[0075] (2) Retrieval-enhanced generation: Under the constraints of the new instructions, the target output of the model is different from the standard answer in the original question-answering dataset, making traditional indicators (such as exact match) invalid. To address this problem, the original standard answer is used as a reference, and GPT-4o is used to evaluate whether the model's output correctly answers the question. The scoring criteria are as follows: completely correct (1 point), partially correct (0.5 points), and completely wrong (0 points). The average score of all samples is used as the retrieval-enhanced generation score of FollowRAG.

[0076] As shown in the figure, FollowRAG is the first instruction following evaluation dataset for RAG scenarios, containing 2.8K samples, covering 22 fine-grained atomic instructions in 6 categories. The queries in FollowRAG come from 4 QA datasets, covering 3 types:

[0077] (1) Open domain QA: Natural Questions (NQ) and TriviaQA (TQA);

[0078] (2) Multi-hop QA: HotpotQA (HQA);

[0079] (3) Knowledge base QA: WebQuestionsSP (WebQSP).

[0080] To further construct instruction following tasks of different difficulties, FollowRAG contains 0.9K single and dual atomic instruction samples, and 0.5K complex multi-instruction samples, including 3 and 4 atomic instructions respectively.

[0081] This application proposes a verifiable automatic instruction data synthesis framework (VIF-RAG) in the retrieval-enhanced generation scenario. Its core design is that each step of automatic generation or combination is accompanied by multiple verification processes, including large model consistency verification and verification based on compiler feedback. The VIF-RAG framework can be roughly divided into two parts: the instruction synthesis stage and the instruction query synthesis stage, which expands from almost zero (<100) to more than 100,000 high-quality instruction query samples. Furthermore, in order to solve the problem of lack of comprehensive evaluation in the retrieval-enhanced generation scenario, we designed a robust benchmark construction process, including three steps of instruction sampling and collection, retrieval and reconstruction, and combination and verification. The unique advantage of this framework is that it can seamlessly connect to a variety of instruction following and retrieval-enhanced generation evaluation benchmarks to achieve combined evaluation. Based on the above benchmark construction framework, we proposed FollowRAG, which is the first benchmark designed to comprehensively evaluate the complex instruction following capabilities of large language models in retrieval-enhanced generation tasks. FollowRAG contains nearly 3,000 test samples, covering 4 knowledge-intensive question-answering benchmarks and 22 constraints.

[0082] Although some embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that modifications may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A retrieval enhancement generation system based on instruction-based alignment, characterized in that: It includes two parts: the instruction synthesis stage and the instruction query synthesis stage. The instruction synthesis stage receives small batches of text restriction instructions and outputs large batches of text restrictions that have undergone multiple quality checks. In the instruction query synthesis stage, the external input variables include three parts: the user's query question, the restriction instructions generated in the instruction synthesis stage, and the relevant knowledge recalled by the dense retriever. The output is output in the form of text to answer the user's query question that meets the user's complex instruction restrictions. Finally, the synthesized alignment data is constructed into a universal alignment dataset. The instruction synthesis stage firstly writes seed instructions manually, manually creates 15 single atomic instructions for each type of constraint; then performs instruction combination and verification, designs rules, and automatically combines atomic instructions into complex instructions constrained by multiple chain rules to obtain the initial seed instruction set D seed ; Then the instructions are rewritten and the quality is verified, using a supervised model GPT-4 with batches of 50 instructions per round, starting from the initial seed instruction set D seed Iterate and rewrite the instructions, perform K rounds of operations, and generate an enhanced instruction set D aug , and merge the seed sample with the enhanced sample to form the merged instruction set D ins =D seed ∪D aug , and remove duplicates; then verify the quality of the automatically generated instructions; In the instruction query synthesis stage, random instruction query combination is first performed. For the retrieval enhancement generation domain, a query set Q is randomly selected from a hybrid QA data source, which includes open domain multi-hop and knowledge base QA scenarios. A dense retriever R is used to retrieve the top K relevant documents D for each query q∈Q from the external knowledge base. i , thus generating a data set For each instruction I, randomly select D RAG The K queries and their corresponding retrieved documents in , and combine them to get the RAG query set with IF constraints For the general domain, for each instruction I∈D ins , randomly select K queries from ShareGPT and combine them with the instructions to build a general domain dataset D IF-General Finally, the instruction constraint query sets of these two fields are merged to form the final query set of VIF-RAG-QA, which is expressed as After that, we conduct command query enhancement and double verification, adopt the rejection sampling strategy, and use the supervision model to generate a single command-query pair. Generate K replies Thus we get {x,y x }∈D VIF-RAG , and then a double verification process is adopted for the instruction query data: Based on compiler feedback: Use the existing verification function to evaluate the compliance of the enhanced output; Large model consistency check: Use the supervised model to score the match between the query and the instruction from 1 to 10, and filter out samples with a score lower than 8; Finally, a large-scale, high-quality VIF-RAG-QA dataset is automatically obtained.

2. The retrieval enhancement generation system for instruction compliance alignment as claimed in claim 1, characterized in that: The specific process of verifying the quality of automatically generated instructions is as follows: for each instruction I∈D ins , use the supervised model to generate K verification function codes and corresponding test cases And evaluate the quality of the instruction by analyzing the output of the executor E; for any function and test case The execution result is: Then calculate the accuracy Acc of each verification function based on K test samples func , and use K validation functions to evaluate the accuracy Acc of each test case case , the formula is as follows: Based on the above cross-metric criteria, each instruction is required to have at least one Acc func and Acc case More than 0.5; finally, get an automatically verified instruction set: Other samples that do not meet the criteria are discarded.

3. The retrieval enhancement generation system for instruction compliance alignment as claimed in claim 1, characterized in that: The specific process of using the existing verification function to evaluate the compliance of the enhanced output is that at least one response in the VIF-RAG data set must achieve an accuracy rate of more than 0.5 in all verification functions, otherwise the sample is discarded.

Images