Model training method and device, electronic equipment, storage medium and computer program product

By using training demonstrations and training instructions to generate adapter weights and adjust student models, the problem of high adaptability and computational cost for large language models in new tasks is solved, and the ability and adaptability of cross-tasks is improved.

CN118916693APending Publication Date: 2024-11-08INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410947007.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional large language models have limited rapid adaptability when facing new tasks, relying on instance training leads to high computational costs and insufficient ability to generalize across tasks.

Method used

By obtaining training input, training demonstration and training instructions in the training data, using the hypernetwork encoder and adapter generator to generate adapter weights, adjust the student model, and calculate the training loss in combination with the teacher model to train and fine-tune the model to reduce the number of models Tag data dependencies.

Benefits of technology

The model's rapid adaptation to new tasks and the improvement of cross-task generalization capabilities is achieved, which reduces computing overhead, saves computing costs, and improves the model's adaptability and flexibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916693A_ABST
    Figure CN118916693A_ABST
Patent Text Reader

Abstract

The invention relates to a model training method and device, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: inputting a training demonstration and a training instruction into a super network encoder; a coding result is input into an adapter generator; adjusting the student model based on the adapter weight; inputting the training input and the coding result into the adjusted student model; inputting the training data into a teacher model to obtain teacher output distribution; calculating training loss; and training the super network and the student model based on the training loss. Therefore, the adapter weight can be obtained by utilizing the adapter generator based on the training demonstration and the training instruction, that is, the adapter of the specific task can be automatically generated from the training instruction, the unseen task does not need to be retrained, the dependence on a large amount of marked data is reduced, and the cross-task generalization ability of the model can be improved. Moreover, the calculation overhead can be reduced, the calculation cost is saved, and the adaptability and flexibility of the model in practical application are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a model training method, device, electronic device, storage medium, and computer program product. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have become a powerful tool for solving various natural language processing tasks. By pre-training these models on large-scale datasets, they can capture rich language patterns and knowledge, thereby showing excellent performance on a variety of tasks. However, LLMs face some challenges in practical applications, especially in scenarios that require cross-task generalization.

[0003] Traditional LLMs training methods rely on a large number of labeled examples to fine-tune the model, which is often not feasible in the real world. This is because obtaining a large amount of labeled data is expensive and time-consuming for many tasks. In addition, when faced with unseen tasks, these models usually need to be retrained or adjusted, which limits their adaptability and flexibility. To address these problems, an instruction finetuning (IFT) based method is proposed to guide the model to complete a specific task by using natural language descriptions.

[0004] However, although IFT improves the model's cross-task generalization ability to a certain extent, it still relies on instance training, which limits the model's ability to quickly adapt when faced with new tasks. In addition, when dealing with unseen tasks, the IFT method often requires reprocessing the model, which is not only inefficient but also computationally expensive. Summary of the invention

[0005] The present disclosure provides a model training method, device, electronic device, storage medium and computer program product to at least solve the problem in the above-mentioned related technology that the IFT method relies on example training, which limits the model's ability to quickly adapt to new tasks and has high computational costs.

[0006] According to a first aspect of an embodiment of the present disclosure, a model training method is provided, comprising: obtaining training data, wherein the training data includes training input, training demonstration and training instruction, and the training demonstration is an example for guiding model training; inputting the training demonstration and the training instruction into a supernetwork encoder included in a supernetwork to obtain an encoding result; inputting the encoding result into an adapter generator included in the supernetwork to obtain an adapter weight; adjusting a student model based on the adapter weight to obtain an adjusted student model; inputting the training input and the encoding result into the adjusted student model to obtain a student output result and a student output distribution; inputting the training data into a teacher model to obtain a teacher output distribution, wherein the teacher model is a model obtained by training a large language model using examples; calculating a training loss based on the adapter weight, a target weight corresponding to the teacher model, the student output result, a true output label corresponding to the training input, the student output distribution and the teacher output distribution; adjusting the parameters of the adapter generator based on the training loss to train the supernetwork and the student model.

[0007] Optionally, the calculating of training loss based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the true output label corresponding to the training input, the student output distribution and the teacher output distribution includes: calculating weight loss based on the adapter weight and the target weight corresponding to the teacher model; calculating output result loss based on the student output result and the true output label corresponding to the training input; calculating output distribution loss based on the student output distribution and the teacher output distribution; calculating the weighted sum of the weight loss, the output result loss and the output distribution loss to obtain the training loss.

[0008] Optionally, before obtaining the training data, the training method further includes: obtaining pre-training data, wherein the pre-training data includes pre-training input and pre-training instructions, and the pre-training input corresponds to a pre-training true output label; inputting the pre-training input and the pre-training instruction into the super-network to obtain a pre-training output result; calculating the pre-training loss based on the pre-training output result and the pre-training true output label; adjusting the parameters of the super-network based on the pre-training loss to pre-train the super-network; and inputting the training demonstration and the training instruction into a super-network encoder contained in the super-network to obtain an encoding result, including: inputting the training demonstration and the training instruction into a super-network encoder contained in a pre-trained super-network to obtain the encoding result.

[0009] Optionally, obtaining pre-training data includes: obtaining a target text; segmenting the target text to obtain three segmented parts arranged in sequence; determining the first segmented part of the three segmented parts as the pre-training instruction, determining the second segmented part of the three segmented parts as the pre-training input, and determining the third segmented part of the three segmented parts as the pre-training true output label.

[0010] Optionally, before obtaining the training data, the training method further includes: obtaining teacher training data, wherein the teacher training data includes teacher training input, teacher training demonstration and teacher training instructions, and the teacher training input corresponds to a teacher training true output label; inputting the teacher training input, the teacher training demonstration and the teacher training instructions into the large language model to obtain a teacher output result; calculating the teacher training loss based on the teacher output result and the teacher training true output label; and adjusting the parameters of the large language model based on the teacher training loss to train the large language model to obtain the teacher model.

[0011] Optionally, the student model includes a student encoder and a student decoder, and adjusting the student model based on the adapter weight to obtain the adjusted student model includes: adjusting the student encoder and the student decoder based on the adapter weight to obtain the adjusted student encoder and the adjusted student decoder; inputting the training input and the encoding result into the adjusted student model to obtain the student output result and the student output distribution includes: inputting the training input and the encoding result into the adjusted student encoder, and fusing the training input and the encoding result through the adjusted student encoder to obtain a fusion result; splicing the fusion result with the encoding result to obtain a splicing result; inputting the splicing result into the adjusted student decoder to obtain the student output result and the student output distribution.

[0012] According to a second aspect of an embodiment of the present disclosure, a model training device is provided, comprising: a training data acquisition module, configured to acquire training data, wherein the training data includes training input, training demonstration and training instruction, and the training demonstration is an example for guiding model training; an encoding module, configured to input the training demonstration and the training instruction into a supernetwork encoder included in a supernetwork to obtain an encoding result; an adapter weight acquisition module, configured to input the encoding result into an adapter generator included in the supernetwork to obtain adapter weights; a student model adjustment module, configured to adjust the student model based on the adapter weights to obtain an adjusted student model; a student output acquisition module, configured to input the encoding result into an adapter generator included in the supernetwork to obtain adapter weights; The training input and the encoding result are input into the adjusted student model to obtain the student output result and the student output distribution; the teacher output distribution acquisition module is configured to input the training data into the teacher model to obtain the teacher output distribution, wherein the teacher model is a model obtained by training a large language model using examples; the training loss calculation module is configured to calculate the training loss based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the true output label corresponding to the training input, the student output distribution and the teacher output distribution; the super network training module is configured to adjust the parameters of the adapter generator based on the training loss to train the super network and the student model.

[0013] Optionally, the training loss calculation module is configured to: calculate the weight loss based on the adapter weight and the target weight corresponding to the teacher model; calculate the output result loss based on the student output result and the true output label corresponding to the training input; calculate the output distribution loss based on the student output distribution and the teacher output distribution; calculate the weighted sum of the weight loss, the output result loss and the output distribution loss to obtain the training loss.

[0014] Optionally, the training device of the model also includes: a pre-training data acquisition module, configured to acquire pre-training data, wherein the pre-training data includes pre-training input and pre-training instructions, and the pre-training input corresponds to a pre-training true output label; a pre-training output acquisition module, configured to input the pre-training input and the pre-training instruction into the super network to obtain a pre-training output result; a pre-training loss calculation module, configured to calculate the pre-training loss based on the pre-training output result and the pre-training true output label; a pre-training module, configured to adjust the parameters of the super network based on the pre-training loss to pre-train the super network; the encoding module is configured to: input the training demonstration and the training instruction into the super network encoder contained in the pre-trained super network to obtain the encoding result.

[0015] Optionally, the pre-training data acquisition module is configured to: acquire a target text; segment the target text to obtain three segmented parts arranged in sequence; determine the first segmented part of the three segmented parts as the pre-training instruction, determine the second segmented part of the three segmented parts as the pre-training input, and determine the third segmented part of the three segmented parts as the pre-training true output label.

[0016] Optionally, the training device of the model also includes: a teacher training data acquisition module, configured to acquire teacher training data, wherein the teacher training data includes teacher training input, teacher training demonstration and teacher training instructions, and the teacher training input corresponds to a teacher training true output label; a teacher output result acquisition module, configured to input the teacher training input, the teacher training demonstration and the teacher training instruction into the large language model to obtain a teacher output result; a teacher training loss calculation module, configured to calculate the teacher training loss based on the teacher output result and the teacher training true output label; a large language model training module, configured to adjust the parameters of the large language model based on the teacher training loss to train the large language model to obtain the teacher model.

[0017] Optionally, the student model includes a student encoder and a student decoder, and the student model adjustment module is configured to: adjust the student encoder and the student decoder based on the adapter weight to obtain an adjusted student encoder and an adjusted student decoder; the student output acquisition module is configured to: input the training input and the encoding result into the adjusted student encoder, and fuse the training input and the encoding result through the adjusted student encoder to obtain a fusion result; splice the fusion result with the encoding result to obtain a splicing result; input the splicing result into the adjusted student decoder to obtain the student output result and the student output distribution.

[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a training method for a model according to the present disclosure.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a training method for a model according to the present disclosure.

[0020] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the training method of the model according to the present disclosure is implemented. The technical solution provided by the embodiment of the present disclosure brings at least the following beneficial effects:

[0021] The training method of the model provided by the present disclosure can obtain adapter weights using an adapter generator based on training demonstrations and training instructions, that is, it can automatically generate adapters for specific tasks from training instructions without retraining for unseen tasks, thereby reducing the reliance on a large amount of labeled data and improving the cross-task generalization ability of the model. In addition, it can also reduce computing overhead and save computing costs, thereby improving the adaptability and flexibility of the model in practical applications.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0024] Figure 1 is a flowchart illustrating a method for training a model according to an exemplary embodiment of the present disclosure;

[0025] Figure 2 is a schematic diagram illustrating a training process of a model according to an exemplary embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram showing a comparison between learning from instructions according to an exemplary embodiment of the present disclosure and training through examples in related arts;

[0027] Figure 4 is a schematic diagram showing SNI experimental results according to an exemplary embodiment of the present disclosure;

[0028] Figure 5 is a schematic diagram showing the results of a P3 experiment according to an exemplary embodiment of the present disclosure;

[0029] Figure 6 is a schematic diagram showing the influence of the number of meta-training on performance according to an exemplary embodiment of the present disclosure;

[0030] Figure 7 is a schematic diagram of an ablation experiment showing various steps and components according to an exemplary embodiment of the present disclosure;

[0031] Figure 8is a block diagram illustrating a training apparatus for a model according to an exemplary embodiment of the present disclosure;

[0032] Fig. 9 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0035] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0036] In the related art, as mentioned above, although IFT improves the cross-task generalization ability of the model to a certain extent, it still relies on example training, which limits the model's ability to quickly adapt when facing new tasks. In addition, when dealing with unseen tasks, the IFT method often requires reprocessing the model, which is not only inefficient but also computationally expensive.

[0037] To overcome these limitations, researchers have begun to explore techniques such as meta-learning and hypernetworks to improve the generalization ability and processing efficiency of models. Meta-learning aims to enable models to quickly adapt to new tasks, while hypernetworks are used to dynamically generate model weights for specific tasks. However, these methods usually require complex network structures and additional training steps, and are less efficient when processing task instructions.

[0038] In order to solve the above problems existing in the related art, the present disclosure provides a training method, device, electronic device, storage medium and computer program product of the model, which can obtain adapter weights using an adapter generator based on training demonstrations and training instructions, that is, it can automatically generate adapters for specific tasks from training instructions without retraining unseen tasks, reducing the dependence on a large amount of labeled data, and improving the cross-task generalization ability of the model. In addition, it can also reduce computing overhead and save computing costs, and improve the adaptability and flexibility of the model in practical applications.

[0039] Figure 1 is a flowchart illustrating a training method of a model according to an exemplary embodiment of the present disclosure.

[0040] Reference Figure 1 In step 101, training data may be obtained, wherein the training data may include training input, training demonstrations, and training instructions, and the training demonstrations may be examples for guiding model training.

[0041] It should be noted that the training method of the model provided by the present disclosure can be applied to a variety of task scenarios, including but not limited to: "Question Answering Task Scenario", "Question Generation Task Scenario", "Text to Code Task Scenario" and "Program Execution Task Scenario". Among them, the "Question Answering Task Scenario" can be a question input to the model, and the model outputs the answer after analyzing the question; the "Question Generation Task Scenario" can be a piece of content input to the model, and then the model generates a corresponding question based on the content; the "Text to Code Task Scenario" can be a piece of text input to the model, and the model converts the text into code; the "Program Execution Task Scenario" can refer to inputting a program into the model, and then the model outputs the execution result of the program.

[0042] Exemplarily, in a "question-and-answer scenario", the above-mentioned "training instruction" may be: "solve a linear equation of two variables"; the above-mentioned "training demonstration" may include multiple pairs of question-and-answer examples, wherein each pair of question-and-answer examples may include a specific linear equation of two variables, the solution process of the linear equation of two variables, and the solution result of the linear equation of two variables; the above-mentioned "training input" may be a linear equation of two variables to be solved.

[0043] It should be noted that in the present disclosure, the above-mentioned training input and model output can be but not limited to: text form, picture form, video form, audio form, etc. The present disclosure does not make specific restrictions on the form of training input and model output.

[0044] Figure 2 is a schematic diagram illustrating a training process of a model according to an exemplary embodiment of the present disclosure. Figure 2 The training process of the model involves three parts: teacher model, student model and hypernetwork. The hypernetwork may include a hypernetwork encoder and an adapter generator, and the adapter generator may include a linear layer and a rectified linear unit (ReLU).

[0045] In step 102, the training demonstration and training instruction may be input into a hypernetwork encoder (Hypernetwork Encoder) included in the hypernetwork to obtain an encoding result. Exemplarily, the training demonstration and training instruction may be encoded into a continuous context representation by the hypernetwork encoder. In addition, the encoding method of the encoder may include, but is not limited to, linear predictive coding (LPC) and the principle of minimizing the perceptual weighted error, etc.

[0046] In step 103, the above encoding result can be input into the adapter generator (AdapterGenerator) contained in the hypernetwork to obtain the adapter weight (LoRA Weights). Exemplarily, a multi-layer perceptron (MLP) can be used as an adapter generator to convert the above encoding result into a LoRA adapter. In addition, the adapter generator can be independently trained in different task scenarios to generate the optimal adapter for different task scenarios.

[0047] In step 104, the student model may be adjusted based on the adapter weights to obtain an adjusted student model. Figure 2 The two arrows pointing from the adapter generator to the student model represent the insertion of the above adapter weights into the corresponding modules contained in the student model.

[0048] In step 105, the training input and the encoding result may be input into the adjusted student model to obtain the student output result (Student Output) and the student output distribution (Student Distribution).

[0049] According to an exemplary embodiment of the present disclosure, the student model may include a student encoder (StudentEncoder) and a student decoder (Student Decoder). The student encoder may include three parts, namely a multi-head attention layer (Multi-head attention), a cross attention layer (Cross Attention) and a feed-forward neural network (Feed-forward Neural Network, FFN).

[0050] The student encoder and the student decoder can be adjusted based on the adapter weights to obtain an adjusted student encoder and an adjusted student decoder. Figure 2 , the two arrows pointing from the adapter generator to the student encoder and student decoder represent the insertion of the above adapter weights into the student encoder and student decoder.

[0051] Specifically, the above training input and encoding result can be input into the adjusted student encoder, and the training input and encoding result can be fused through the adjusted student encoder to obtain a fusion result. Exemplarily, the above training input and encoding result can be fused (Fusion) through the cross attention layer contained in the student encoder to enrich the model's understanding and processing of task instructions. Next, the above fusion result can be concatenated (Concat) with the encoding result to obtain a concatenated result. Then, the obtained concatenated result can be input into the adjusted student decoder to obtain the student output result and the student output distribution.

[0052] For example, as mentioned above, in the “question-answering task scenario”, the “training input” may be a linear equation of two variables to be solved. Assume that the linear equation of two variables is: 2 +x+1=0, then the above “student output result” can be the student model itself for the two-variable linear equation: 2 +x+1=0 to obtain the calculation result.

[0053] In step 106, the training data may be input into a teacher model to obtain a teacher output distribution (TeacherDistribution), wherein the teacher model may be a model obtained by training a large language model (LLM) using examples. Exemplarily, the teacher model may be a model with relatively good performance obtained by training a large language model using a large number of examples in actual scenarios. Furthermore, the teacher model may correspond to a target weight, which may refer to a weight that enables the teacher model to have good performance.

[0054] Exemplarily, in the "question-answering task scenario", the target weight may refer to the weight that enables the teacher model to have excellent question-answering performance, that is, the target weight may refer to the weight that makes the output accuracy of the teacher model relatively high; in the "question generation task scenario", the target weight may refer to the weight that enables the teacher model to have excellent question generation performance, that is, the target weight may refer to the weight that makes the questions generated by the teacher model more reasonable; in the "text-to-code task scenario", the target weight may refer to the weight that enables the teacher model to have excellent conversion performance, that is, the target weight may refer to the weight that makes the conversion accuracy of the teacher model relatively high; in the "program execution task scenario", the target weight may refer to the weight that enables the teacher model to have excellent execution performance, that is, the target weight may refer to the weight that makes the execution accuracy of the teacher model relatively high, and so on.

[0055] In step 107, the training loss L can be calculated based on the adapter weights, the target weights corresponding to the teacher model, the student output results, the ground truth labels corresponding to the training inputs, the student output distribution, and the teacher output distribution. finetune As mentioned above, in the “question-answering task scenario”, the “training input” can be a linear equation to be solved. Assume that the linear equation is: 2 +x+1=0, then the above “true output label corresponding to the training input” can be for the binary linear equation: 2 +x+1=0 to obtain the truly correct result.

[0056] According to an exemplary embodiment of the present disclosure, the weight loss (L ins ); the output loss (L pred ); Based on the above student output distribution and teacher output distribution, the output distribution loss (L kl ). Then, the weighted sum of the weight loss, output result loss, and output distribution loss can be calculated to obtain the above training loss.

[0057] Exemplarily, the above training loss can be calculated by the following formula:

[0058] L finetune =L pred +μ1L kl +μ2L ins

[0059] Among them, μ1 is the weighted weight corresponding to the distribution loss, and μ2 is the weighted weight corresponding to the weight loss.

[0060] Furthermore, the above process of "calculating the output distribution loss based on the student output distribution and the teacher output distribution" may refer to the process of utilizing the consistency between the labels, output logic, and adapter parameters between the teacher model and the student model to enhance the understanding of instructions and generate more effective adapters for specific tasks through L2 regularization. Specifically, the knowledge distillation technique may be used, that is, using a strong baseline model (teacher model) as a teacher to align the task-specific model (teacher model) that has been trained through examples with the student model, which includes minimizing the Kullback-Leibler divergence between the teacher model and the student model and the mean square error between the adapter parameters to ensure that the student model can learn the behavior of the teacher model. It should be noted that the above "teacher model" may refer to a model with relatively good performance obtained by training a large language model using a complete combination of task instructions, input data, and task demonstrations.

[0061] In step 108, the parameters of the adapter generator may be adjusted based on the above training loss to train the hypernetwork and the student model.

[0062] It should be noted that the aforementioned steps 101 to 108 describe the process of fine-tuning the hypernetwork, that is, the process of obtaining a task-specific model through LoRA fine-tuning. As mentioned above, first, the training instructions can be converted into continuous context representations using the encoder of the hypernetwork. Then, the encoding results obtained by encoding the training instructions can be fused with the training input of the student model to utilize the instruction information in the decoding process. In addition, the adapter generator of the hypernetwork can generate LoRA parameters based on the encoding results, which are low-rank matrices used to adjust the model attention weight matrix.

[0063] It should be noted that LoRA (Low-rank Adaptation) tuning is one of the key steps to achieve task-specific models. It is an efficient parameter adjustment method that reduces the number of trainable parameters by decomposing the attention weight matrix of a large language model. Specifically, LoRA can represent the update of the attention weight matrix as the product of two low-rank matrices, so that the model can adapt to different task scenarios more flexibly while keeping the number of parameters relatively low. Here we assume that the model fine-tuned by LoRA on a specific task is the optimal model for the corresponding task. In this way, by fine-tuning the hypernetwork on the multi-task prompt dataset, it can be enabled to learn to generate optimal parameters from task instructions, that is, the optimal LoRA parameters can be dynamically generated according to the task instructions, thereby ensuring effective generalization to future unseen tasks, and thus adapting to the needs of different task scenarios.

[0064] The present disclosure provides a method for generating task adapters from instructions to enhance the cross-task generalization ability of a large language model, that is, a method for generating task adapters from instructions (TAGI). TAGI aims to improve the performance of LLMs on unseen tasks by imitating the way humans understand instructions to learn, that is, by imitating the way humans understand instructions to learn to enhance the cross-task generalization ability of LLMs when processing unseen tasks, which can reduce the dependence on a large number of labeled instances. In practical applications, the model trained by the TAGI method can surpass traditional meta-training models and other hypernetwork models. In addition, it can also significantly reduce computing resource requirements. Moreover, since TAGI does not require additional parameter updates or gradient backpropagation, it can also avoid the inefficiency caused by repeated encoding instructions during the reasoning process.

[0065] According to an exemplary embodiment of the present disclosure, before fine-tuning the supernetwork, pre-training can also be performed on the supernetwork. Specifically, pre-training data can be obtained, wherein the pre-training data can include pre-training input and pre-training instructions, and the pre-training input can correspond to a pre-training true output label. Then, the pre-training input and the pre-training instruction can be input into the supernetwork to obtain the pre-training output result. Next, the pre-training loss can be calculated based on the pre-training output result and the pre-training true output label. Then, the parameters of the supernetwork can be adjusted based on the pre-training loss to pre-train the supernetwork. Next, the training demonstration and the training instruction can be input into the supernetwork encoder contained in the pre-trained supernetwork to obtain the encoding result.

[0066] Furthermore, standard text pre-training data, such as the C4 dataset, can be used to pre-train the hypernetwork to enhance the model's ability to recognize and process instructions. Specifically, the hypernetwork can be pre-trained by minimizing the cross-entropy loss function to ensure that it can generate effective adapters related to the task. In this way, through pre-training, the hypernetwork can learn a wider range of instruction patterns, which helps the model to more effectively generate suitable adapters when faced with new and unseen tasks.

[0067] According to an exemplary embodiment of the present disclosure, a target text can be obtained, and then segmented to obtain three segmented parts arranged in sequence. Then, the first segmented part of the three segmented parts can be determined as a pre-training instruction, the second segmented part of the three segmented parts can be determined as a pre-training input, and the third segmented part of the three segmented parts can be determined as a pre-training true output label.

[0068] Specifically, refer to Figure 2, the target text (Pretraining Data) is divided into 3 parts, namely a, b, and c in order. Among them, "a" represents the pretraining instruction, "b" represents the pretraining input, and "c" represents the pretraining true output label. In this way, the pretraining loss can be calculated by comparing the pretraining output result of the hypernetwork and the pretraining true output label, and then the parameters of the hypernetwork can be adjusted based on the pretraining loss to pretrain the hypernetwork.

[0069] Further, as mentioned above, a large-scale general text dataset, such as the C4 dataset, can be selected to preprocess the hypernetwork, including sequence truncation and encoding. First, a hypernetwork can be constructed, which can include a hypernetwork encoder and an adapter generator. The hypernetwork encoder is responsible for converting task instructions into continuous context representations, and the adapter generator is responsible for generating efficient adapters, such as LoRA. Specifically, training can be started using randomly initialized hypernetwork weights, the input sequence is fed into the hypernetwork, and the adapter parameters (LoRAWeights) are generated by the hypernetwork encoder and adapter generator. It should be noted that pre-training helps the model learn more robust feature representations, reduce overfitting on specific datasets, and enable it to more effectively generate adapters for specific tasks in subsequent tasks, thereby improving the performance of the model on unseen tasks.

[0070] According to an exemplary embodiment of the present disclosure, before obtaining the training data, teacher training data may also be obtained, wherein the teacher training data may include teacher training input, teacher training demonstration and teacher training instruction, and the teacher training input may correspond to a teacher training real output label. Figure 2 , teacher training data (teacher data) can include three parts, namely teacher training instructions (Def.), teacher training demonstrations (Demos) and teacher training input (Source). Among them, the teacher training input can correspond to the teacher training real output label (target).

[0071] Then, the teacher training input, teacher training demonstration and teacher training instruction can be input into the large language model to obtain the teacher output result. Next, the teacher training loss can be calculated based on the teacher output result and the teacher training true output label. Then, the parameters of the large language model can be adjusted based on the teacher training loss to train the large language model and obtain the teacher model (teacher). Further, referring to Figure 2, the teacher model can include two parts: teacher encoder (teacherEncoder) and teacher decoder (teacherDecoder).

[0072] It should be noted that during the fine-tuning of the hypernetwork, the teacher encoder and teacher decoder included in the teacher model, and the student encoder and student decoder included in the student model are all in a frozen state. That is, in the present disclosure, the hypernetwork is fine-tuned on the basis of freezing the teacher model and the student model.

[0073] Figure 3 2 is a schematic diagram showing a comparison between learning from instructions according to an exemplary embodiment of the present disclosure and training through examples in related technologies. Figure 3 In the related art, the parameters of the task-specific model are directly adjusted by back-propagation to perform model training; in the present disclosure, the vanilla model is frozen, and the model training is performed only by fine-tuning the hypernetwork. That is, in the present disclosure, a method for generating task adapters from instructions (TAGI) is proposed. The TAGI method can improve the performance of LLMs on unseen tasks by simulating the process of human understanding of instructions, and at the same time, it can reduce the dependence on a large amount of labeled data. Specifically, since the TAGI method can automatically generate a model adapter for a specific task from a task instruction without retraining the unseen task, the TAGI method can improve the cross-task generalization ability of the model, reduce the demand for computing resources, and enhance the adaptability and flexibility of the model in practical applications.

[0074] In order to illustrate the effectiveness of the training method of the model provided by the present disclosure, the present disclosure is verified on multiple public multi-task datasets, for example, including but not limited to: Super-Natural Instructions (SNI) and P3 datasets. Among them, the SNI dataset may contain more than 1,600 task datasets, each of which may include a task definition and a fixed number of positive examples and negative examples. The P3 dataset may consist of 62 task datasets, and the model can be trained and tested on these tasks. Specifically, for SNI, the ROUGE-L indicator can be used to evaluate the text generated by the model on 119 unseen tasks. For P3, it can be evaluated based on the accuracy of multiple-choice questions. The experiment used T5-LM-Adapt and T0 models as initialization, and the T0 model was trained on multi-tasks on the P3 meta-training set.

[0075] For the SNI dataset: TAGI shows significant improvements over baseline models in both zero-shot and few-shot settings. Specifically, TAGI outperforms the meta-training model by 0-2.3 points while saving nearly 2.5 times the computational overhead (FLOPs), outperforms the decoder-only model GPT2 by 10 points, and outperforms OPT-13B by 7 points. In addition, compared to other hypernetwork models, TAGI can achieve the same computational overhead savings and outperform 2 points. Figure 4 is a schematic diagram showing SNI experimental results according to an exemplary embodiment of the present disclosure.

[0076] For the P3 dataset: TAGI also shows superior performance on the P3 evaluation set, significantly reducing the computational burden. Compared with the ICL-based method, TAGI only processes the instruction once, which can achieve comparable or higher accuracy while significantly reducing the computational overhead (×0.88 times). In addition to being about 1.2 points lower than FiD-ICL, it can be higher than other methods by more than 1 point. In addition, the ICL-based method takes 2-13.2 times the reasoning time of the standard method. Figure 5 is a schematic diagram showing the results of a P3 experiment according to an exemplary embodiment of the present disclosure.

[0077] It can be seen that the TAGI method provided by the present disclosure can effectively generate adapters on both of the above datasets to adapt to unseen tasks, which shows the effectiveness of TAGI in cross-task generalization. Specifically, compared with other hypernetwork-based methods, TAGI reduces the computational requirements by about 60%, while having comparable or better performance than other methods.

[0078] It should be noted that the number of meta-training tasks will also have a significant impact on the performance of the model. For example, the tasks can be sorted according to the number of data sets contained, and the tasks can be divided into different groups according to the number, for example, 6, 15, 30 and 60 tasks. In each group, a corresponding number of tasks can be selected from the meta-training set (Ttrain) for training. The experimental results show that as the number of meta-training tasks increases, the performance of the model also improves, indicating that more meta-training tasks can help the model learn a wider range of instructions and data patterns, thereby improving its performance on unseen tasks.

[0079] Furthermore, experiments have shown that when the model performs parameter alignment on meta-training tasks, the model performs relatively well even without explicit parameter alignment, and can also show a smaller relative standard deviation and smaller performance fluctuations with changes in the number of tasks. It can be seen that by increasing the number of meta-training tasks, the generalization ability of the model is enhanced, because the model can learn a wider range of instruction processing capabilities through more diverse tasks. However, although increasing the number of meta-training tasks can improve performance, it is also necessary to consider the feasibility and resource limitations of obtaining enough tasks in practical applications. Figure 6 is a schematic diagram illustrating the influence of the number of meta-trainings on performance according to an exemplary embodiment of the present disclosure.

[0080] The present disclosure also conducts ablation experiments to evaluate the contribution of each part in the TAGI model. The ablation experiment shows that the pre-training stage has a significant improvement in model performance, and the model without pre-training performs poorly in task execution. In addition, instruction fusion enhances the model's understanding and processing capabilities of instructions by fusing task instructions with input data, thereby improving performance. In the ablation experiment, removing specific loss terms (for example, L2 regularization, KL divergence, etc.) will lead to a decrease in performance, which indicates that these loss terms are beneficial for model training. The importance of each component in the TAGI model is demonstrated through ablation experiments, and the model is optimized accordingly. Experimental results show that pre-training, instruction fusion, and specific loss terms have significant positive effects on model performance. Figure 7 is a schematic diagram of an ablation experiment showing various steps and components according to an exemplary embodiment of the present disclosure.

[0081] Figure 8 is a block diagram illustrating a training apparatus 800 for a model according to an exemplary embodiment of the present disclosure.

[0082] Reference Figure 8The training device 800 of the model may include a training data acquisition module 801, an encoding module 802, an adapter weight acquisition module 803, a student model adjustment module 804, a student output acquisition module 805, a teacher output acquisition module 806, a training loss calculation module 807 and a super network training module 808.

[0083] The training data acquisition module 801 can acquire training data, wherein the training data can include training input, training demonstration and training instructions, and the training demonstration can be an example for guiding model training.

[0084] The encoding module 802 may input the training demonstration and training instruction into a hypernetwork encoder (Hypernetwork Encoder) included in the hypernetwork to obtain an encoding result. Exemplarily, the training demonstration and training instruction may be encoded into a continuous context representation by the hypernetwork encoder.

[0085] The adapter weight acquisition module 803 can input the above encoding results into the adapter generator (Adapter Generator) contained in the super network to obtain adapter weights (LoRAWeights). Exemplarily, a multi-layer perceptron (MLP) can be used as an adapter generator to convert the above encoding results into a LoRA adapter. In addition, the adapter generator can be independently trained in different task scenarios to generate the optimal adapter for different task scenarios.

[0086] The student model adjustment module 804 can adjust the student model based on the above adapter weights to obtain an adjusted student model.

[0087] The student output acquisition module 805 can input the above training input and the above encoding result into the adjusted student model to obtain the student output result (Student Output) and the student output distribution (Student Distribution).

[0088] According to an exemplary embodiment of the present disclosure, the student model may include a student encoder and a student decoder. The student encoder may include three parts, namely a multi-head attention layer, a cross attention layer and a feedforward neural network.

[0089] The student model adjustment module 804 can adjust the student encoder and the student decoder based on the adapter weights to obtain an adjusted student encoder and an adjusted student decoder. Figure 2 , the two arrows pointing from the adapter generator to the student encoder and student decoder represent the insertion of the above adapter weights into the student encoder and student decoder.

[0090] Specifically, the student output acquisition module 805 can input the above-mentioned training input and encoding result into the adjusted student encoder, and fuse the training input and encoding result through the adjusted student encoder to obtain a fusion result. Exemplarily, the above-mentioned training input and encoding result can be fused through the cross attention layer contained in the student encoder to enrich the model's understanding and processing of task instructions. Next, the student output acquisition module 805 can splice the above-mentioned fusion result with the encoding result to obtain a splicing result. Then, the student output acquisition module 805 can input the obtained splicing result into the adjusted student decoder to obtain the student output result and the student output distribution.

[0091] The teacher output acquisition module 806 can input the above training data into the teacher model to obtain the teacher output distribution, wherein the teacher model can be a model obtained by training the large language model using examples. Exemplarily, the teacher model can be a model with relatively good performance obtained by training the large language model using a large number of examples in actual scenarios. In addition, the teacher model can correspond to a target weight, which can refer to a weight that enables the teacher model to have good performance.

[0092] The training loss calculation module 807 can calculate the training loss L based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the ground truth corresponding to the training input, the student output distribution and the teacher output distribution. finetune .

[0093] According to an exemplary embodiment of the present disclosure, the training loss calculation module 807 can calculate the weight loss based on the target weight corresponding to the adapter weight and the teacher model; the training loss calculation module 807 can calculate the output result loss based on the student output result and the real output label corresponding to the training input; the training loss calculation module 807 can calculate the output distribution loss based on the student output distribution and the teacher output distribution. Then, the training loss calculation module 807 can calculate the weighted sum of the weight loss, the output result loss and the output distribution loss to obtain the training loss.

[0094] The hypernetwork training module 808 can adjust the parameters of the adapter generator based on the above training loss to train the hypernetwork and the student model.

[0095] According to an exemplary embodiment of the present disclosure, the training device 800 of the above-mentioned model may also include a pre-training data acquisition module, a pre-training output acquisition module, a pre-training loss calculation module and a pre-training module.

[0096] Pre-training can also be performed for the super network. Specifically, the pre-training data acquisition module can acquire pre-training data, wherein the pre-training data can include pre-training input and pre-training instructions, and the pre-training input can correspond to a pre-training true output label. Then, the pre-training output acquisition module can input the pre-training input and pre-training instructions into the super network to obtain the pre-training output result. Next, the pre-training loss calculation module can calculate the pre-training loss based on the pre-training output result and the pre-training true output label. Then, the pre-training module can adjust the parameters of the super network based on the pre-training loss to pre-train the super network. Next, the above-mentioned encoding module 802 can input the training demonstration and training instructions into the super network encoder contained in the pre-trained super network to obtain the encoding result.

[0097] Furthermore, standard text pre-training data, such as the C4 dataset, can be used to pre-train the hypernetwork to enhance the model's ability to recognize and process instructions. Specifically, the hypernetwork can be pre-trained by minimizing the cross-entropy loss function to ensure that it can generate effective adapters related to the task. In this way, through pre-training, the hypernetwork can learn a wider range of instruction patterns, which helps the model to more effectively generate suitable adapters when faced with new and unseen tasks.

[0098] According to an exemplary embodiment of the present disclosure, the above-mentioned pre-training data acquisition module can obtain the target text, and then segment the target text to obtain three segmented parts arranged in sequence. Then, the pre-training data acquisition module can determine the first segmented part of the three segments as the pre-training instruction, the second segmented part of the three segments as the pre-training input, and the third segmented part of the three segments as the pre-training true output label. It should be noted that pre-training helps the model learn a more robust feature representation, reduces overfitting of a specific data set, enables it to more effectively generate adapters for specific tasks in subsequent tasks, and improves the performance of the model on unseen tasks.

[0099] According to an exemplary embodiment of the present disclosure, the training device 800 of the above-mentioned model may also include a teacher training data acquisition module, a teacher output result acquisition module, a teacher training loss calculation module, and a large language model training module.

[0100] The teacher training data acquisition module can also acquire teacher training data, wherein the teacher training data can include teacher training input, teacher training demonstration and teacher training instructions, and the teacher training input can correspond to a teacher training real output label.

[0101] Then, the teacher output result acquisition module can input the teacher training input, the teacher training demonstration and the teacher training instruction into the large language model to obtain the teacher output result. Next, the teacher training loss calculation module can calculate the teacher training loss based on the teacher output result and the teacher training true output label. Then, the large language model training module can adjust the parameters of the large language model based on the teacher training loss to train the large language model and obtain the teacher model.

[0102] It should be noted that during the fine-tuning of the hypernetwork, the teacher encoder and teacher decoder included in the teacher model, and the student encoder and student decoder included in the student model are all in a frozen state. That is, in the present disclosure, the hypernetwork is fine-tuned on the basis of freezing the teacher model and the student model.

[0103] Fig. 9 is a block diagram illustrating an electronic device 900 according to an exemplary embodiment of the present disclosure.

[0104] Reference Fig. 9 The electronic device 900 includes at least one memory 901 and at least one processor 902, wherein the at least one memory 901 stores instructions, and when the instructions are executed by the at least one processor 902, a training method for a model according to an exemplary embodiment of the present disclosure is executed.

[0105] As an example, the electronic device 900 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 900 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 900 may also be part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0106] In the electronic device 900, the processor 902 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0107] The processor 902 may execute instructions or codes stored in the memory 901, wherein the memory 901 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0108] The memory 901 may be integrated with the processor 902, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 901 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 901 and the processor 902 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 902 can read files stored in the memory.

[0109] In addition, the electronic device 900 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 900 may be connected to each other via a bus and / or a network.

[0110] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the training method of the above-mentioned model. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0111] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program, which implements the training method of the model according to the present disclosure when the computer program is executed by a processor.

[0112] According to the training method, device, electronic device, storage medium and computer program product of the model disclosed in the present invention, the adapter weight can be obtained by using the adapter generator based on the training demonstration and training instructions, that is, the adapter of a specific task can be automatically generated from the training instructions without retraining the unseen tasks, which reduces the dependence on a large amount of labeled data and can improve the cross-task generalization ability of the model. In addition, it can also reduce the computing overhead and save computing costs, and improve the adaptability and flexibility of the model in practical applications.

[0113] According to an exemplary embodiment of the present disclosure, by fine-tuning the hypernetwork on a multi-task prompt dataset, it can be enabled to learn to generate optimal parameters from task instructions, that is, the optimal LoRA parameters can be dynamically generated according to the task instructions, thereby ensuring effective generalization of future unseen tasks, and thus adapting to the needs of different task scenarios.

[0114] According to an exemplary embodiment of the present disclosure, through pre-training, the super network can learn a wider range of instruction patterns, which helps the model to more effectively generate suitable adapters when facing new and unseen tasks. That is, pre-training helps the model learn more robust feature representations, reduce overfitting of specific data sets, and enable it to more effectively generate adapters for specific tasks in subsequent tasks, thereby improving the performance of the model on unseen tasks.

[0115] According to an exemplary embodiment of the present disclosure, a method for generating a task adapter in an instruction is proposed. The TAGI method can improve the performance of LLMs on unseen tasks by simulating the process of human understanding of instructions for learning, and at the same time, can reduce the dependence on a large amount of labeled data. Specifically, since the TAGI method can automatically generate a model adapter for a specific task from a task instruction without retraining the unseen task, the TAGI method can improve the cross-task generalization ability of the model, reduce the demand for computing resources, and enhance the adaptability and flexibility of the model in practical applications.

[0116] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0117] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A model training method, It is characterized in that include: Acquire training data, wherein the training data includes training input, training demonstration, and training instruction, wherein the training demonstration is an example for guiding model training; Inputting the training demonstration and the training instruction into a hypernetwork encoder included in the hypernetwork to obtain an encoding result; Inputting the encoding result into an adapter generator included in the super network to obtain an adapter weight; Adjust the student model based on the adapter weight to obtain an adjusted student model; Inputting the training input and the encoding result into the adjusted student model to obtain a student output result and a student output distribution; Inputting the training data into a teacher model to obtain a teacher output distribution, wherein the teacher model is a model obtained by training a large language model using examples; Calculating a training loss based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the true output label corresponding to the training input, the student output distribution, and the teacher output distribution; Parameters of the adapter generator are adjusted based on the training loss to train the hypernetwork and the student model.

2. The training method according to claim 1, It is characterized in that The calculating the training loss based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the true output label corresponding to the training input, the student output distribution and the teacher output distribution includes: Calculating a weight loss based on the adapter weight and the target weight corresponding to the teacher model; Calculate the output loss based on the student output and the true output label corresponding to the training input; Calculating output distribution loss based on the student output distribution and the teacher output distribution; A weighted sum of the weight loss, the output result loss, and the output distribution loss is calculated to obtain the training loss.

3. The training method according to claim 1, It is characterized in that Before acquiring the training data, the training method further includes: Acquire pre-training data, wherein the pre-training data includes pre-training input and pre-training instructions, and the pre-training input corresponds to a pre-training true output label; Inputting the pre-training input and the pre-training instruction into the super network to obtain a pre-training output result; Calculating the pre-training loss based on the pre-training output result and the pre-training true output label; Adjusting parameters of the hypernetwork based on the pre-training loss to pre-train the hypernetwork; The step of inputting the training demonstration and the training instruction into a hypernetwork encoder included in the hypernetwork to obtain an encoding result includes: The training demonstration and the training instruction are input into a super network encoder included in the pre-trained super network to obtain the encoding result.

4. The training method according to claim 3, It is characterized in that The obtaining of pre-training data comprises: Get the target text; Segmenting the target text to obtain three segmented parts arranged in sequence; The first segmented part of the three segments is determined as the pre-training instruction, the second segmented part of the three segments is determined as the pre-training input, and the third segmented part of the three segments is determined as the pre-training true output label.

5. The training method according to claim 1, It is characterized in that Before acquiring the training data, the training method further includes: Acquire teacher training data, wherein the teacher training data includes teacher training input, teacher training demonstration and teacher training instruction, and the teacher training input corresponds to a teacher training real output label; Inputting the teacher training input, the teacher training demonstration and the teacher training instruction into the large language model to obtain a teacher output result; Calculating the teacher training loss based on the teacher output result and the teacher training true output label; The parameters of the large language model are adjusted based on the teacher training loss to train the large language model to obtain the teacher model.

6. The training method according to claim 1, It is characterized in that The student model includes a student encoder and a student decoder, and the adjusting the student model based on the adapter weight to obtain the adjusted student model includes: Adjusting the student encoder and the student decoder based on the adapter weight to obtain an adjusted student encoder and an adjusted student decoder; The step of inputting the training input and the encoding result into the adjusted student model to obtain a student output result and a student output distribution includes: Inputting the training input and the encoding result into the adjusted student encoder, and fusing the training input and the encoding result through the adjusted student encoder to obtain a fusion result; Splicing the fusion result with the encoding result to obtain a splicing result; The splicing result is input into the adjusted student decoder to obtain the student output result and the student output distribution.

7. A training device for a model, It is characterized in that include: A training data acquisition module is configured to acquire training data, wherein the training data includes training input, training demonstration and training instruction, and the training demonstration is an example for guiding model training; An encoding module is configured to input the training demonstration and the training instruction into a supernetwork encoder included in the supernetwork to obtain an encoding result; an adapter weight acquisition module, configured to input the encoding result into an adapter generator included in the super network to obtain an adapter weight; a student model adjustment module, configured to adjust the student model based on the adapter weight to obtain an adjusted student model; A student output acquisition module is configured to input the training input and the encoding result into the adjusted student model to obtain a student output result and a student output distribution; A teacher output distribution acquisition module is configured to input the training data into a teacher model to obtain a teacher output distribution, wherein the teacher model is a model obtained by training a large language model using examples; a training loss calculation module configured to calculate the training loss based on the adapter weight, the target weight corresponding to the teacher model, the student output result, the true output label corresponding to the training input, the student output distribution and the teacher output distribution; A hypernetwork training module is configured to adjust parameters of the adapter generator based on the training loss to train the hypernetwork and the student model.

8. An electronic device, It is characterized in that include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the training method of the model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, It is characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the model as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the training method of the model according to any one of claims 1 to 6 is implemented.

Citation Information

Cited By

  • Perpetual visual language navigation method and system based on prototype evolution

    CN122192335A

  • Prototype-evolution-based lifelong visual language navigation method and system

    CN122192335B