A training method of a unified example retriever for context learning
By adopting a unified example retrieval training method, and utilizing language model feedback and multivariate distribution sampling, the problem of high transfer and expansion costs of the retrieval on different tasks is solved, achieving significant performance improvement and good transfer performance across multiple tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2024-12-11
- Publication Date
- 2026-06-19
AI Technical Summary
In the prior art, retrieval machines for specific natural language processing tasks are costly to migrate and extend, and general-purpose retrieval machines have limited performance on different tasks, making it difficult to perform well on multiple tasks.
By designing a unified training method for example retrieval, the training signals of different tasks are unified into a list sorted by utilizing the feedback of the language model. Combined with multivariate sampling and loss function training, the training data is iteratively mined to achieve efficient training of the example retrieval.
Significant performance improvements and good transferability were achieved across different tasks, reducing the impact of high-resource tasks and improving the training performance of the example retrieval system.
Smart Images
Figure CN122241209A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of deep learning and natural language processing, and specifically relates to a training method for a unified example retrieval system for context learning. Background Technology
[0002] Large language models have demonstrated excellent context learning capabilities across a wide range of natural language processing tasks. By feeding a set of examples (input-output pairs related to a task) into a large language model, it can directly output predictions based on the test input. This novel natural language processing paradigm is called context learning, and its advantage is that multiple natural language tasks can be performed using the same language model's inference capabilities without requiring fine-tuning of its parameters. The performance of context learning is highly dependent on the provided examples, which has spurred research into example retrieval: given a test input, relevant examples are retrieved from the training set as examples for context learning.
[0003] There are two main methods for retrieving examples: (1) using a general-purpose retrieval engine; and (2) training a retrieval engine for a specific task. General-purpose retrieval engines, such as BM25 and Sentence-BERT, can retrieve examples that are textually or semantically similar to the test input and are suitable for a variety of natural language processing tasks. Training a retrieval engine for a specific task requires designing training signals for each task, such as output similarity. General-purpose retrieval engines perform much better than randomly selected examples, while retrieval engines for specific tasks outperform general-purpose retrieval engines on the tasks they have been trained on.
[0004] Task-specific retrieval systems offer excellent performance, but previous explorations have been limited to a narrow range of natural language processing tasks, such as semantic parsing and dialogue state tracking. Furthermore, transferring and extending these task-specific retrieval systems to various tasks is costly, requiring the design of specialized training signals for each task, and the number of retrieval systems increases with the number of tasks. Summary of the Invention
[0005] This invention is made to solve the above-mentioned problems, and aims to provide a training method for a unified example retrieval system for context learning.
[0006] This invention provides a method for training a unified example retrieval system for context learning. The example retrieval system provides examples for a language model to process tasks and includes the following steps:
[0007] Step S1: For each natural language processing task, given training samples (x, y) and a candidate set... Using the probability value of generating y as the scoring criterion, a language model is used to score each candidate sample in the candidate set, resulting in a score s(z) for each candidate sample.i Then, based on the scores of each candidate sample, they are reordered from highest to lowest.
[0008] Step S2: After obtaining the reordered candidate set, the candidate sets for different tasks are sampled using a probability distribution to obtain a training set. The example retrieval machine is then trained using the training set and a predetermined loss function.
[0009] Step S3: After training the example retrieval tool for a predetermined number of steps, use the example retrieval tool to retrieve examples from all samples in the training set. Score and rank the retrieved examples using a language model. The retrieved examples and the candidate set are used together as a new candidate set. Then, sample the new candidate set using a probability distribution to obtain a new training set. Continue training the example retrieval tool using the new training set and loss function. After iterative training, the training of the example retrieval tool is completed.
[0010] The training method for the unified example retrieval system for context learning provided by this invention may also have the following feature: In step S2, candidate sets for different tasks are sampled using a probability distribution, as shown in the following formula:
[0011]
[0012] In formula (1), T i For the i-th task, D Ti It is the i-th candidate set, and α is a predefined hyperparameter.
[0013] The training method for the unified example retrieval system for context learning provided by this invention may also have the following feature: wherein, in step S2, the loss function formula is as follows:
[0014] L=λ*L rank +(1-λ)*L ib (2)
[0015] In formula (2), λ is a predefined hyperparameter.
[0016]
[0017] In formula (3), sim(x,z) represents the similarity between x and z, calculated using cosine similarity.
[0018] In formula (4), z * X is the candidate ranked first, and Z is the training set for the entire batch, including the training set of X and the training sets of other training samples in the batch.
[0019] This invention also provides a method for example retrieval using the example retrieval trained by the above-mentioned unified example retrieval method for context learning, characterized by the following steps:
[0020] The example retrieval tool retrieves examples from the training set corresponding to the natural language processing task based on the test input of the natural language processing task.
[0021] The role and effect of invention
[0022] According to the training method of the unified example retrieval system for context learning of the present invention, firstly, the training signals of different natural language processing tasks are designed as list sorting forms through the feedback of the language model itself, thus unifying the example retrieval for different tasks; and secondly, the training set is obtained by sampling candidate sets of different sizes using a multinomial distribution, which can avoid the excessive influence of high-resource tasks during the training process, and the example retrieval system is trained by jointly using a list loss function and an intra-batch loss function, which can achieve better training results; in addition, during the training process, the present invention iteratively mines the training data using the example retrieval system itself after a certain number of training steps, and continues to train the example retrieval system with the newly mined data and the previous training data, thereby obtaining higher quality training data through iterative mining.
[0023] Compared to previous methods, the example retrieval trained by the unified example retrieval method for context learning of this invention can achieve significant performance improvements on different datasets for various tasks, and can achieve good transfer effects on different language models and number of examples. Attached Figure Description
[0024] Figure 1 This is a flowchart illustrating a training method for a unified example retrieval system for context learning, as described in an embodiment of the present invention.
[0025] Figure 2 This is a schematic diagram of the process by which a trained example retrieval system provides examples in an embodiment of the present invention;
[0026] Figure 3 This is a comparison of the performance of the trained example retrieval system in the natural language processing classification task in the embodiments of the present invention with that of previous methods;
[0027] Figure 4 This is a comparison of the performance of the trained example retrieval system in the embodiment of the present invention on the natural language processing generation task with previous methods. Detailed Implementation
[0028] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate a training method for a unified example retrieval system for context learning according to this invention.
[0029] <Example>
[0030] Figure 1 This is a flowchart illustrating a training method for a unified example retrieval system for context learning, as described in an embodiment of the present invention.
[0031] like Figure 1 As shown in this embodiment, a training method for a unified example retrieval system for context learning is provided. The example retrieval system is used to provide examples for the task to be processed by the language model, and includes the following steps:
[0032] Step S1: For each natural language processing task, given training samples (x, y) and a candidate set... (l is the size of the candidate set), using the probability of generating y as the scoring criterion, a language model is used to score each candidate sample in the candidate set, resulting in a score s(z) for each candidate sample. i Then, based on the scores of each candidate sample, they are reordered from high to low, thereby designing the training signals of different tasks into a unified list sorting form through the feedback of the language model.
[0033] Step S2: After obtaining the reordered candidate set, the candidate sets for different tasks are sampled using a probability distribution to obtain a training set. The example retrieval machine is then trained using the training set and a predetermined loss function.
[0034] In step S2, the candidate set for different tasks is sampled using a probability distribution, as shown in the following formula:
[0035]
[0036] In formula (1), T i For the i-th task, D Ti It is the i-th candidate set, and α is a predefined hyperparameter, which is 0.5 in this embodiment.
[0037] In step S2, the loss function formula is as follows:
[0038] L=λ*L rank +)1-λ)*L ib (2)
[0039] In formula (2), λ is a predefined hyperparameter.
[0040]
[0041] In formula (3), sim(x,z) represents the similarity between x and z, calculated using cosine similarity.
[0042] In formula (4), z * X is the candidate ranked first, and Z is the training set for the entire batch, including the training set of X and the training sets of other training samples in the batch.
[0043] Step S3: After training the example retrieval tool for a predetermined number of steps, use the example retrieval tool to retrieve examples from all samples in the training set. Score and sort the retrieved examples using a language model. The retrieved examples and the candidate set are used together as a new candidate set. Then, sample the new candidate set using a probability distribution to obtain a new training set. Continue training the example retrieval tool using the new training set and loss function. After three rounds of iterative training, the training of the example retrieval tool is completed.
[0044] Figure 2 This is a schematic diagram of the process by which a trained example retrieval system provides examples in an embodiment of the present invention.
[0045] like Figure 2 As shown, in this embodiment, the method for example retrieval using the example retrieval trainer obtained through the training method of the unified example retrieval trainer for context learning described above includes the following steps:
[0046] The example retrieval tool retrieves examples from the training set corresponding to the natural language processing task based on the test input of the natural language processing task. Further, the specific process of retrieving examples that provide context for the natural language processing task using the trained example retrieval tool is as follows:
[0047] For multiple different natural language processing tasks T i and its training set Z i The example retrieval unit first appends task-specific instructions ((x',y')∈Z) before the test input x and all samples x' in the training set. i The concatenated text is then encoded using an encoder, and cosine similarity is calculated to retrieve examples corresponding to the test input for context learning. These retrieved examples are then fed into the language model before the test input, and the language model's output is the predicted output.
[0048] In this embodiment, an example retrieval system (UDR) trained using the unified example retrieval system for context learning described in this embodiment is also used to provide examples for natural language processing classification tasks on different datasets. Examples are provided using Random, BM25, SBERT, Instructor, and EPR, and their performance is compared. Figure 3This is a comparison of the performance of the example retrieval system trained in the embodiments of the present invention on the classification task of natural language processing with previous methods.
[0049] like Figure 3 As shown, the example retrieval system trained by the unified example retrieval system for context learning in this embodiment can achieve significant performance improvements in classification tasks under different datasets.
[0050] In this embodiment, an example retrieval system (UDR) trained using the unified example retrieval system for context learning described in this embodiment is also used to provide examples for natural language processing generation tasks on different datasets. Examples are provided using Random, BM25, SBERT, Instructor, and EPR, and their performance is compared.
[0051] Figure 4 This is a comparison of the performance of the trained example retrieval system in the embodiment of the present invention on the natural language processing generation task with previous methods.
[0052] like Figure 4 As shown, the example retrieval trained by the unified example retrieval method for context learning in this embodiment can achieve significant performance improvements in generation tasks under different datasets.
[0053] In summary, compared with previous methods, the example retrieval trained by the unified example retrieval method for context learning proposed in this application achieves significant performance improvements on different datasets for various tasks, and demonstrates excellent performance across different tasks.
[0054] The role and effect of the embodiments
[0055] According to the training method of the unified example retrieval system for context learning involved in this embodiment, firstly, the training signals of different natural language processing tasks are designed as list sorting forms through the feedback of the language model itself, thus unifying the example retrieval for different tasks; and secondly, the training set is obtained by sampling candidate sets of different sizes using a multivariate distribution, which can avoid the excessive influence of high-resource tasks during the training process. Furthermore, the example retrieval system is trained by jointly using a list loss function and an intra-batch loss function, which can achieve better training results. In addition, during the training process, this embodiment uses the example retrieval system itself to iteratively mine training data after a certain number of training steps. The newly mined data is used together with the previous training data to continue training the example retrieval system. Through iterative mining, higher quality training data can be obtained.
[0056] Furthermore, based on the test results of this embodiment, it can be seen that, compared with previous methods, the example retrieval trained by the training method of the unified example retrieval for context learning in this embodiment can achieve significant performance improvements on different datasets of various tasks, and can achieve good transfer effects on different language models and number of examples.
[0057] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.
Claims
1. A method for training a unified example retrieval system for context learning, wherein the example retrieval system is used to provide examples for a task to be processed by a language model, characterized in that, Includes the following steps: Step S1: For each natural language processing task, given training samples (x, y) and a candidate set... Using the probability value of generating y as the scoring criterion, the language model is used to score each candidate sample in the candidate set, resulting in a score s(z) for each candidate sample. i Then, based on the scores of each candidate sample, they are reordered from highest to lowest. Step S2: After obtaining the reordered candidate set, the candidate set for different tasks is sampled using a probability distribution to obtain a training set. The example retrieval machine is then trained using the training set and a predetermined loss function. Step S3: After training the example retrieval tool for a predetermined number of steps, use the example retrieval tool to perform example retrieval on all samples in the training set. Score and sort the retrieved examples using the language model. The retrieved examples and the candidate set are used together as a new candidate set. Then, sample the new candidate set using the probability distribution to obtain a new training set. Continue training the example retrieval tool using the new training set and the loss function. After iterative training, the training of the example retrieval tool is completed.
2. The training method for a unified example retrieval system for context learning according to claim 1, characterized in that: in, In step S2, the candidate set for different tasks is sampled using a probability distribution, as shown in the following formula: In formula (1), T i For the i-th task, It is the i-th candidate set, and α is a predefined hyperparameter.
3. The training method for a unified example retrieval system for context learning according to claim 1, characterized in that: in, In step S2, the loss function formula is as follows: L=λ*L rank +(1-λ)*L ib (2) In formula (2), λ is a predefined hyperparameter. In formula (3), sim(x,z) represents the similarity between x and z, calculated using cosine similarity. In formula (4), z * X is the candidate ranked first, and Z is the training set for the entire batch, including the training set of X and the training sets of other training samples in the batch.
4. A method for performing example retrieval using an example retrieval trained by the training method for a unified example retrieval retrieval retrieval accredited by claims 1-3, characterized in that, Includes the following steps: The example retrieval machine retrieves examples corresponding to the test input of the natural language processing task from the training set corresponding to the natural language processing task.