Fine tuning training method and device for domain large language model, electronic equipment and medium

Abstract

The invention relates to a field large language model fine tuning training method and device, electronic equipment and a medium. The method comprises the steps that in the fine tuning training stage, original instruction data of multiple domain tasks are obtained, a single selection task data set is constructed, the single selection task data set comprises multiple pieces of single selection task data of the multiple domain tasks, data disturbance operation is conducted on original instruction topics in the single selection task data set, and the original instruction topics in the single selection task data set are obtained; identifying a key attention head of each domain task, obtaining a key attention head index table, and loading a target key attention head list corresponding to the target domain task based on the key attention head index table; and for the target domain task, carrying out key attention head training based on the target key attention head list, updating the weight parameter of each key attention head in the target key attention head list, and freezing the weight parameters of other network layers.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a method, device, electronic device and medium for fine-tuning a large language model in a field. Background Art

[0002] In recent years, artificial intelligence, especially natural language processing (NLP), has made significant progress, among which the development of large language models (LLMs) is particularly prominent. These models, such as the GPT series and the Llama series, can generate high-quality natural language text by training on massive text data and perform well in various NLP tasks. However, as the scale of models continues to expand, how to effectively train and fine-tune these models to adapt to different task requirements in specific fields has become an important challenge.

[0003] In related technologies, domain task training is performed through methods such as full parameter fine-tuning, LoRA (Low-Rank Adaptation) fine-tuning, and QLoRA (Quantized Low-Rank Adapter) fine-tuning. However, most of the existing fine-tuning methods focus on general tasks, but have weak adaptability to specific fields or subtasks. Since different fields have unique terminology, expressions, and knowledge structures, general models are often difficult to migrate directly to professional fields. For example, in highly specialized fields such as finance, taxation, etc., general large language models may not be able to understand and generate accurate context and content. Ordinary training methods may cause the general capabilities of the model to degrade. Although full parameter fine-tuning can improve the performance of specific tasks, it is easy to cause overfitting, causing the model to lose its original general capabilities.

[0004] Therefore, how to improve the model's domain capabilities while avoiding a reduction in the model's general capabilities is an issue that needs to be addressed urgently. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a method, device, electronic device and medium for fine-tuning a large domain language model.

[0006] In a first aspect, the present disclosure provides a method for fine-tuning a domain-wide language model, comprising:

[0007] In the fine-tuning training stage, original instruction data of multiple domain tasks are obtained to construct a single-choice task data set; the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: an original instruction title, a correct option and an incorrect option corresponding to the original instruction title;

[0008] Performing data perturbation operations on the original instruction questions in the single-choice task data set, identifying key attention heads for tasks in various fields, and obtaining a key attention head index table; the key attention head index table is used to represent a mapping relationship between multiple field tasks and multiple key attention head lists;

[0009] Based on the key attention head index table, load the target key attention head list corresponding to the target domain task;

[0010] For the target domain task, key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen.

[0011] As an optional implementation of the embodiment of the present disclosure, the obtaining of original instruction data of tasks in multiple fields and constructing a single-choice task data set includes:

[0012] For each piece of original instruction data, according to a mapping function, extract the user instruction of each piece of original instruction data as the original instruction title of the single-choice task data;

[0013] Inputting the original instruction data into a preset large language model to obtain the correct option of the single-choice task data;

[0014] Inputting the original instruction title and the correct option into the preset large language model to obtain the wrong option of the single-choice task data;

[0015] Constructing single-choice task data based on the original instruction title, the correct options, and the incorrect options;

[0016] Based on multiple single-choice task data from tasks in multiple fields, a single-choice task dataset is constructed.

[0017] As an optional implementation of the embodiment of the present disclosure, the data perturbation operation is performed on the original instruction questions in the single-choice task data set, the key attention heads of tasks in various fields are identified, and the key attention head index table is obtained, including:

[0018] For each domain task, the single-choice task data is input into the preset large language model, and the initial probability that the original instruction title of each single-choice task data predicted by the preset large language model is the correct option is obtained;

[0019] Performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbed single-choice task data;

[0020] Inputting the disturbance single-choice task data into the disturbance preset large language model, and obtaining the target probability that the disturbance preset large language model predicts that the original instruction title of each disturbance single-choice task data is the correct option;

[0021] According to the initial probability and the target probability, determining whether the target attention head in the perturbed preset large language model is a key attention head, so as to obtain a list of key attention heads for tasks in various fields;

[0022] According to the mapping relationship between multiple domain tasks and multiple key attention head index tables, a key attention head index table is obtained.

[0023] As an optional implementation of the embodiment of the present disclosure, before inputting the disturbance single-choice task data into the disturbance preset large language model, the method further includes:

[0024] For each attention head in the preset large language model, the original activation value of the current attention head is replaced with the perturbed activation value, and the original activation values ​​of other attention heads are kept unchanged to obtain the perturbed preset large language model.

[0025] As an optional implementation of the embodiment of the present disclosure, determining whether the target attention head in the perturbed preset large language model is a key attention head according to the initial probability and the target probability, so as to obtain a list of key attention heads for tasks in various fields, includes:

[0026] For each domain task, after replacing the perturbation activation value of each attention head, the difference between the initial probability and the target probability is obtained;

[0027] According to the difference between the initial probability and the target probability, a list of key attention heads for tasks in various fields is obtained.

[0028] As an optional implementation of the embodiment of the present disclosure, updating the weight parameters of each key attention head in the target key attention head list and freezing the weight parameters of other network layers include:

[0029] By setting the value of the gradient tracking attribute corresponding to the target key attention head to true, setting the value of the gradient tracking attribute corresponding to other key attention heads to false, updating the weight parameters of each key attention head in the target key attention head list, and freezing the weight parameters of other network layers.

[0030] As an optional implementation of the embodiment of the present disclosure, performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbed single-choice task data includes:

[0031] The original instruction questions in the single-choice task data set are subjected to sequence permutation and typos replacement operations to obtain disturbed single-choice task data.

[0032] In a second aspect, the present disclosure provides a domain-wide large language model fine-tuning training device, including:

[0033] A data set construction module is used to obtain original instruction data of multiple domain tasks and construct a single-choice task data set in the fine-tuning training stage; the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: an original instruction title, a correct option and an incorrect option corresponding to the original instruction title;

[0034] An index table acquisition module is used to perform data perturbation operations on the original instruction questions in the single-choice task data set, identify the key attention heads of various domain tasks, and obtain a key attention head index table; the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists;

[0035] A list loading module, used for loading a target key attention head list corresponding to a target domain task based on the key attention head index table;

[0036] The parameter updating module is used to perform key attention head training based on the target key attention head list for the target domain task, update the weight parameters of each key attention head in the target key attention head list, and freeze the weight parameters of other network layers.

[0037] As an optional implementation of the embodiment of the present disclosure, the data set construction module is specifically used to:

[0038] For each piece of original instruction data, according to a mapping function, extract the user instruction of each piece of original instruction data as the original instruction title of the single-choice task data;

[0039] Inputting the original instruction data into a preset large language model to obtain the correct option of the single-choice task data;

[0040] Inputting the original instruction title and the correct option into the preset large language model to obtain the wrong option of the single-choice task data;

[0041] Constructing single-choice task data based on the original instruction title, the correct options, and the incorrect options;

[0042] Based on multiple single-choice task data from tasks in multiple fields, a single-choice task dataset is constructed.

[0043] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module includes:

[0044] An initial probability acquisition unit, for each domain task, inputs the single-choice task data into the preset large language model, and obtains the initial probability that the preset large language model predicts that the original instruction title of each single-choice task data is the correct option;

[0045] A perturbation data acquisition unit, used for performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbation single-choice task data;

[0046] A target probability acquisition unit is used to input the disturbance single-choice task data into the disturbance preset large language model to obtain the target probability that the original instruction title of each disturbance single-choice task data predicted by the disturbance preset large language model is a correct option;

[0047] An attention head list acquisition unit, used to determine whether a target attention head in the perturbed preset large language model is a key attention head according to the initial probability and the target probability, so as to obtain a key attention head list for tasks in various fields;

[0048] The attention head index table acquisition unit is used to obtain the key attention head index table according to the mapping relationship between multiple domain tasks and multiple key attention head index tables.

[0049] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module further includes:

[0050] The perturbation model acquisition unit is used to replace the original activation value of the current attention head with the perturbation activation value for each attention head in the preset large language model, keep the original activation values ​​of other attention heads unchanged, and obtain the perturbation preset large language model.

[0051] As an optional implementation of the embodiment of the present disclosure, the attention head list acquisition unit is specifically used to:

[0052] For each domain task, after replacing the perturbation activation value of each attention head, the difference between the initial probability and the target probability is obtained;

[0053] According to the difference between the initial probability and the target probability, a list of key attention heads for tasks in various fields is obtained.

[0054] As an optional implementation of the embodiment of the present disclosure, the parameter updating module is specifically used to:

[0055] For the target domain task, key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen.

[0056] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module is further used to:

[0057] The original instruction questions in the single-choice task data set are subjected to sequence permutation and typos replacement operations to obtain disturbed single-choice task data.

[0058] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, it implements the domain large language model fine-tuning training method described in the first aspect or any implementation method of the first aspect.

[0059] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the domain large language model fine-tuning training method as described in any implementation of the first aspect.

[0060] Compared with the prior art, the technical solution provided by the disclosed embodiment has the following advantages: in the fine-tuning training stage, original instruction data of multiple domain tasks are obtained, and a single-choice task data set is constructed, wherein the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: an original instruction title, correct options and incorrect options corresponding to the original instruction title; data perturbation operations are performed on the original instruction titles in the single-choice task data set, key attention heads of each domain task are identified, and a key attention head index table is obtained, wherein the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists; based on the key attention head index table, a target key attention head list corresponding to the target domain task is loaded; for the target domain task, key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen. When fine-tuning instructions for training various domain subtasks based on the general model base, the data used for fine-tuning instructions is converted into a single-choice task suitable for key attention head recognition, and a mapping relationship between domain subtasks and key attention heads is constructed to locate the key attention heads that have the closest impact on the performance of the subtask, and fine-tune the attention heads corresponding to the subtasks, so as to improve the model's domain capabilities while avoiding a reduction in the model's general capabilities. In addition, during fine-tuning model training, the efficiency of domain model training is improved and the cost of domain model training is reduced by freezing the network structure that is not related to the training of each subtask or has an impact less than a certain threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0063] Figure 1 It is a flowchart of a method for fine-tuning a domain large language model provided by an embodiment of the present disclosure;

[0064] Figure 2 is a schematic diagram of converting instruction fine-tuning data into single-item selection data provided by an embodiment of the present disclosure;

[0065] Figure 3 It is a structural diagram of a domain large language model fine-tuning training device provided by an embodiment of the present disclosure;

[0066] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0069] Relational terms such as “first” and “second” in the description and claims of the present disclosure are merely used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0070] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0071] Glossary:

[0072] LLM: (Large Language Model) is an artificial intelligence model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, etc.

[0073] LoRA (Low-Rank Adaptation) is an efficient parameter fine-tuning technique for large language models (LLMs). Its core idea is to inject a trainable low-rank factorization matrix into each Transformer architecture layer based on the pre-trained model, instead of fine-tuning all model parameters, thereby significantly reducing the number of trainable parameters for downstream tasks. Low-rank factorization matrix is ​​a method of decomposing a large matrix into the product of two or more smaller, lower-rank matrices.

[0074] QLoRA (Quantized Low-Rank Adapter) is an efficient large language model (LLM) fine-tuning method that significantly reduces memory usage during fine-tuning by combining quantization technology and low-rank adapter (LoRA). QLoRA can reduce the fine-tuning of a 65B parameter model that originally required more than 780GB of GPU memory to a single 48GB GPU, while maintaining the same performance as 16-bit full parameter fine-tuning.

[0075] A general model foundation is a specific type of large language model (LLM) that is designed to handle multiple types of tasks rather than being specialized for a specific task. Such models usually have a high degree of generalization ability and can play a role in different application domains.

[0076] The instruction engineering method, also known as prompt engineering, involves how to ask questions to the big model, that is, to interact with the big model through carefully designed instructions to guide it to perform specific tasks. The core elements of instruction engineering include instructions, context, input data, and output prompts. Among them, instructions refer to the content that users require the model to generate. Context refers to providing additional contextual information to help the model respond better. Input data refers to the content or questions entered by the user. Output data refers to specifying the type or format of the output.

[0077] In the related art, domain task training is performed through methods such as full parameter fine-tuning, LoRA (Low-Rank Adaptation) fine-tuning, and QLoRA (Quantized Low-Rank Adapter) fine-tuning. However, most of the existing fine-tuning methods focus on general tasks, but have weak adaptability to specific fields or subtasks. Since different fields have unique terminology, expressions, and knowledge structures, general models are often difficult to migrate directly to professional fields. For example, in highly specialized fields such as finance and taxation, general large language models may not be able to understand and generate accurate context and content. Ordinary training methods may cause the general ability of the model to degrade. Although full parameter fine-tuning can improve the performance of specific tasks, it is easy to cause overfitting, causing the model to lose its original general ability. In addition, full parameter training is inefficient. Current fine-tuning methods usually require updating a large number of parameters, and even LoRA and QLoRA only partially alleviate this problem. For research institutions or enterprises with limited resources, the high computing cost and time consumption are obstacles that cannot be ignored. In addition, frequent updates of a large number of parameters may also lead to unnecessary computing overhead and increase the instability of the system.

[0078] In response to the above problems, the disclosed embodiments provide a training method for fine-tuning instructions for a large domain language model. When fine-tuning instructions for training subtasks in various domains based on a general model base, the subtask instruction fine-tuning data is constructed in the form of multiple-choice questions through a large model, thereby locating the model attention head that has the closest impact on the performance of the subtask, and performing fine-tuning training on the attention head corresponding to the subtask, thereby improving the model's domain capabilities while avoiding a reduction in the model's general capabilities. In addition, during the fine-tuning training of the model, the efficiency of domain model training is improved and the cost of domain model training is reduced by freezing the network structure that is irrelevant to the training of each subtask or whose impact is less than a certain threshold.

[0079] In some embodiments, Figure 1 As shown, a domain large language model fine-tuning training method is provided. This method is applied to the fine-tuning training process of the domain large language model, including the following steps S11-S14:

[0080] S11. During the fine-tuning training phase, original instruction data of tasks in multiple fields are obtained to construct a single-choice task dataset.

[0081] Specifically, by constructing the original instruction data of multiple domain tasks into a single-choice task dataset, the essence is to convert the original task data into a single-choice question format, which is a format that is convenient for key attention head recognition. Each domain task contains multiple pieces of original instruction data.

[0082] The single-choice task data set includes multiple single-choice task data of multiple field tasks, and each single-choice task data includes: an original instruction title, a correct option and an incorrect option corresponding to the original instruction title.

[0083] Raw instruction data refers to the data set used to fine-tune the model. These data sets contain a series of specific instructions and the corresponding expected outputs. These data are used to instruct the model on how to perform specific tasks based on the given instructions. Raw instruction data includes: instructions, additional information, and expected outputs. Instructions refer to the natural language text input of a given task, which is used to inform the model of the target task to be completed. Additional information is optional supplementary information used to provide context related to the current target task. The expected output is the target output generated based on the provided instructions and context, that is, the response.

[0084] Specifically, suppose there is a set of domain tasks T = {T1, T2, ..., T n}, where each task represents a specific subtask of the domain model to be trained. For each task T i , define the data set it contains as D i ={d i1 ,di2 ,…,d im}, where each piece of data d ij is an instance that needs to be processed. In order to convert these data into multiple-choice questions, a large language model LLM is introduced, which is used to generate corresponding multiple-choice question data based on the input original instruction fine-tuning data through instruction engineering.

[0085] For example, for each piece of data d ij , the original instruction content of the user identity is used as the question q ij , LLM combined with q ij Generate a correct answer with the assistant's original answer And three wrong options

[0086] In some embodiments, the above step S11 (obtaining original instruction data of tasks in multiple fields and constructing a single-choice task data set) can be implemented as follows:

[0087] a. For each piece of original instruction data, according to the mapping function, extract the user instruction of each piece of original instruction data as the original instruction title of the single-choice task data.

[0088] b. Input the original instruction data into a preset large language model to obtain the correct option of the single-choice task data.

[0089] c. Input the original instruction title and the correct option into the preset large language model to obtain the wrong option of the single-choice task data.

[0090] d. Constructing single-choice task data based on the original instruction title, the correct options, and the incorrect options.

[0091] e. Construct a single-choice task dataset based on multiple single-choice task data from multiple field tasks.

[0092] Specifically, the conversion process can be expressed by the following formula:

[0093] q ij =f(d ij )

[0094]

[0095] Among them, d ij represents any piece of data, and f represents a mapping function, which is used to extract the user instructions in the original data instance as the original instruction title q of the single-choice task data. ij , Represents the correct option for single-choice task data. Indicates the wrong options in the single-choice task data. LLM is a large model used to generate correct options and wrong options. First, the correct options are generated based on the original data. Then combine the topic q ij and the correct option Generate incorrect interference options. For example, you can generate incorrect options by adjusting the numerical value of the correct answer, changing the subject and object, etc. These incorrect options should be similar to the correct answer in numerical value or semantics to test the model's ability to recognize details.

[0096] For example, the financial field is used as an example. Figure 2 As shown, in the financial report analysis task, assuming that the original instruction data is Figure 2 The data on the left is obtained by extracting the original question, generating the correct option (2 million US dollars), and combining the question and the correct option to generate the wrong options (1 million US dollars, 1.8 million US dollars, and 8 million US dollars), and the final single-choice task data corresponding to the data is obtained.

[0097] S12. Perform data perturbation operations on the original instruction questions in the single-choice task data set, identify key attention heads for tasks in various fields, and obtain a key attention head index table.

[0098] Among them, the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists.

[0099] Specifically, identifying the attention heads that have the greatest impact on task performance is an important step to improve model performance while maintaining general capabilities. By analyzing the data of each subtask, we can determine the attention heads that play a key role in the model prediction process. For each domain task T i , has converted its original instruction dataset into multiple multiple-choice questions. On this basis, we first calculate the probability of the original model predicting the correct option, and then evaluate the impact of each attention head on the final prediction result by adding perturbations to the content of each option.

[0100] In some embodiments, the above step S12 (performing a data perturbation operation on the original instruction questions in the single-choice task data set, identifying the key attention heads of tasks in various fields, and obtaining a key attention head index table) can be implemented as follows:

[0101] A. For each domain task, the single-choice task data is input into the preset large language model to obtain the initial probability that the preset large language model predicts that the original instruction title of each single-choice task data is the correct option.

[0102] Specifically, for each converted data instance, the original model to be trained (i.e., the preset large language model) is used to calculate the probability of predicting it as the correct option. * , then the probability of the preset large language model predicting the correct option is:

[0103]

[0104] B. Performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbed single-choice task data.

[0105] In some embodiments, the original instruction questions in the single-choice task data set are subjected to sequence permutation and typos replacement operations to obtain disturbed single-choice task data.

[0106] Exemplarily, these disturbance operations include, but are not limited to, changing the order of options, replacing typos that do not affect semantic understanding, etc.

[0107] C. Input the disturbance single-choice task data into the disturbance preset large language model, and obtain the target probability that the disturbance preset large language model predicts that the original instruction title of each disturbance single-choice task data is the correct option.

[0108] In some embodiments, before executing the above step C (inputting the disturbance single-choice task data into the disturbance preset large language model, and obtaining the target probability that the disturbance preset large language model predicts that the original instruction title of each disturbance single-choice task data is the correct option), the following steps are further performed:

[0109] For each attention head in the preset large language model, the original activation value of the current attention head is replaced with the perturbed activation value, and the original activation values ​​of other attention heads are kept unchanged to obtain the perturbed preset large language model.

[0110] Specifically, the questions in the single-choice task data are perturbed by order permutation, typos replacement, etc. Assuming that the perturbed input is X′, traverse each layer L of the Transformer decoder model i Each attention head H in i ,It can be understood that each network layer includes multiple attention heads. The self-attention mechanism in Transformer is extended to multiple attention heads, each of which can learn different attention weights to better capture different types of relationships.

[0111] For each attention head, its original activation value is replaced with the perturbed activation value, while other attention heads are kept with the original activation values. Specifically, assuming that the original activation matrix is ​​A orig , the activation matrix after perturbation is Aperturb For a certain attention head H of a certain layer j , do the following:

[0112] That is, replace the activation value of the target attention head: A′ new (H j )=A perturb (H j )

[0113] Keep the activation values ​​of other attention heads unchanged:

[0114] D. According to the initial probability and the target probability, determine whether the target attention head in the perturbed preset large language model is a key attention head, so as to obtain a list of key attention heads for tasks in various fields.

[0115] In some embodiments, the above step D (determining whether the target attention head in the perturbed preset large language model is a key attention head according to the initial probability and the target probability, so as to obtain a list of key attention heads for tasks in various fields) can be implemented as follows:

[0116] For each domain task, after replacing the perturbation activation value of each attention head, the difference between the initial probability and the target probability is obtained;

[0117] According to the difference between the initial probability and the target probability, a list of key attention heads for tasks in various fields is obtained.

[0118] Specifically, use the model after replacing the activation value of the attention head (i.e., perturb the preset large language model) Calculate new predicted probabilities

[0119]

[0120] By comparing the i-th task T i , the original model predicts the probability P of the correct option a* and the attention head H ij The predicted probability of the correct option after the perturbation Calculate the difference between the two:

[0121]

[0122] Specifically, if the difference ΔP ij If it is greater than the preset threshold δ, it is confirmed that the attention head has a significant impact on the performance of the task in this field. Finally, through the above process, each field task T i The mapping relationship with the key attention head list M = {T i→{H i1 ,H i2 ,…,H in}}, where {H i1 ,H i2 ,…,H in} indicates that for task T i The most influential set of attention heads.

[0123] E. Obtain the key attention head index table based on the mapping relationship between multiple domain tasks and multiple key attention head index tables.

[0124] Specifically, according to the mapping relationship between multiple domain tasks and multiple key attention head index tables, the key attention head index table is obtained. After obtaining the mapping relationship between multiple domain tasks and multiple key attention head index tables, the subsequent training process can focus on optimizing these key attention heads, thereby improving the performance of the model on specific domain tasks.

[0125] S13. Based on the key attention head index table, load the target key attention head list corresponding to the target domain task.

[0126] Specifically, in the domain adaptation training of the large language model, based on the key attention head index table, selective fine-tuning can be performed for different domain tasks. i Extract the relevant attention head list H from the pre-built key attention head index table i The purpose of this step is to ensure that we focus only on those model parameter weights that are most critical to the task at hand.

[0127] S14. For the target domain task, perform key attention head training based on the target key attention head list, update the weight parameters of each key attention head in the target key attention head list, and freeze the weight parameters of other network layers.

[0128] Specifically, the core idea of ​​fine-tuning training is to use the index table M obtained above to update only the parameters of the key attention heads related to the current domain subtask during training, while freezing the weights of other network structures. This can reduce unnecessary computational overhead and maintain the stability of the model's general capabilities.

[0129] In some embodiments, the above step S14 (for the target domain task, performing key attention head training based on the target key attention head list, updating the weight parameters of each key attention head in the target key attention head list, and freezing the weight parameters of other network layers) can be implemented as follows:

[0130] By setting the value of the gradient tracking attribute corresponding to the target key attention head to true, setting the value of the gradient tracking attribute corresponding to other key attention heads to false, updating the weight parameters of each key attention head in the target key attention head list, and freezing the weight parameters of other network layers.

[0131] Specifically, for the weight matrix corresponding to each key attention head, by setting the value of the gradient tracking attribute corresponding to the target key attention head to true, these attention heads are allowed to update their parameters during back propagation. In the model, except for the key attention head, by setting the value of the gradient tracking attribute corresponding to other key attention heads to false, all other layers and parameters are frozen, that is, the gradients of these parts are not calculated. This strategy reduces unnecessary calculations and prevents irrelevant parts from interfering with the current task.

[0132] During the entire fine-tuning training process, forward propagation is first performed, which calculates the cross entropy loss function L by inputting the input data X into the model LLM. Then, the gradient of the loss function with respect to the updateable parameters is calculated through backpropagation. For each key attention head set H i The weight matrix w, using these gradients to update the weights w = w-η* where η is the learning rate.

[0133] The above process is continuously iterated to optimize the performance of the model on specific domain tasks. Since only a small number of parameters are updated, the training speed is significantly accelerated; freezing most of the network structure helps to maintain the consistency of the overall performance of the model and prevent the general ability of the model from degrading; focusing on optimizing components closely related to the task improves the performance of the model on specific domain tasks.

[0134] The present invention provides a method for fine-tuning a large domain language model. In the fine-tuning training stage, original instruction data of multiple domain tasks are obtained to construct a single-choice task data set, wherein the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: an original instruction title, and correct options and incorrect options corresponding to the original instruction title; data perturbation operations are performed on the original instruction titles in the single-choice task data set to identify key attention heads of each domain task and obtain a key attention head index table, wherein the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists; based on the key attention head index table, a target key attention head list corresponding to the target domain task is loaded; for the target domain task, key attention head training is performed based on the target key attention head list, weight parameters of each key attention head in the target key attention head list are updated, and weight parameters of other network layers are frozen. When fine-tuning instructions for training various domain subtasks based on the general model base, the data used for fine-tuning instructions is converted into a single-choice task suitable for key attention head recognition, and a mapping relationship between domain subtasks and key attention heads is constructed to locate the key attention heads that have the closest impact on the performance of the subtask, and fine-tune the attention heads corresponding to the subtasks, so as to improve the model's domain capabilities while avoiding a reduction in the model's general capabilities. In addition, during fine-tuning model training, the efficiency of domain model training is improved and the cost of domain model training is reduced by freezing the network structure that is not related to the training of each subtask or has an impact less than a certain threshold.

[0135] In some embodiments, reference Figure 3 As shown, a domain large language model fine-tuning training device 300 is provided, comprising:

[0136] The data set construction module 310 is used to obtain the original instruction data of multiple domain tasks and construct a single-choice task data set in the fine-tuning training stage; the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: the original instruction question, the correct option and the wrong option corresponding to the original instruction question;

[0137] An index table acquisition module 320 is used to perform a data perturbation operation on the original instruction questions in the single-choice task data set, identify the key attention heads of various domain tasks, and obtain a key attention head index table; the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists;

[0138] A list loading module 330 is used to load a target key attention head list corresponding to a target domain task based on the key attention head index table;

[0139] The parameter updating module 340 is used to perform key attention head training based on the target key attention head list for the target domain task, update the weight parameters of each key attention head in the target key attention head list, and freeze the weight parameters of other network layers.

[0140] As an optional implementation of the embodiment of the present disclosure, the data set construction module is specifically used to:

[0141] For each piece of original instruction data, according to a mapping function, extract the user instruction of each piece of original instruction data as the original instruction title of the single-choice task data;

[0142] Inputting the original instruction data into a preset large language model to obtain the correct option of the single-choice task data;

[0143] Inputting the original instruction title and the correct option into the preset large language model to obtain the wrong option of the single-choice task data;

[0144] Constructing single-choice task data based on the original instruction title, the correct options, and the incorrect options;

[0145] Based on multiple single-choice task data from tasks in multiple fields, a single-choice task dataset is constructed.

[0146] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module includes:

[0147] An initial probability acquisition unit, for each domain task, inputs the single-choice task data into the preset large language model, and obtains the initial probability that the preset large language model predicts that the original instruction title of each single-choice task data is the correct option;

[0148] A perturbation data acquisition unit, used for performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbation single-choice task data;

[0149] A target probability acquisition unit is used to input the disturbance single-choice task data into the disturbance preset large language model to obtain the target probability that the original instruction title of each disturbance single-choice task data predicted by the disturbance preset large language model is a correct option;

[0150] An attention head list acquisition unit, used to determine whether a target attention head in the perturbed preset large language model is a key attention head according to the initial probability and the target probability, so as to obtain a key attention head list for tasks in various fields;

[0151] The attention head index table acquisition unit is used to obtain the key attention head index table according to the mapping relationship between multiple domain tasks and multiple key attention head index tables.

[0152] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module further includes:

[0153] The perturbation model acquisition unit is used to replace the original activation value of the current attention head with the perturbation activation value for each attention head in the preset large language model, keep the original activation values ​​of other attention heads unchanged, and obtain the perturbation preset large language model.

[0154] As an optional implementation of the embodiment of the present disclosure, the attention head list acquisition unit is specifically used to:

[0155] For each domain task, after replacing the perturbation activation value of each attention head, the difference between the initial probability and the target probability is obtained;

[0156] According to the difference between the initial probability and the target probability, a list of key attention heads for tasks in various fields is obtained.

[0157] As an optional implementation of the embodiment of the present disclosure, the parameter updating module is specifically used to:

[0158] For the target domain task, key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen.

[0159] As an optional implementation of the embodiment of the present disclosure, the index table acquisition module is further used to:

[0160] The original instruction titles in the single-choice task data set are subjected to sequence permutation and typos replacement operations to obtain perturbed single-choice task data. The domain large language model fine-tuning training device provided by the present disclosure obtains original instruction data of multiple domain tasks in the fine-tuning training stage, and constructs a single-choice task data set, wherein the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: the original instruction title, the correct options and the wrong options corresponding to the original instruction title; the original instruction title in the single-choice task data set is subjected to data perturbation operations, the key attention heads of each domain task are identified, and a key attention head index table is obtained, wherein the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists; based on the key attention head index table, the target key attention head list corresponding to the target domain task is loaded; for the target domain task, the key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen. When fine-tuning instructions for training various domain subtasks based on the general model base, the data used for fine-tuning instructions is converted into a single-choice task suitable for key attention head recognition, and a mapping relationship between domain subtasks and key attention heads is constructed to locate the key attention heads that have the closest impact on the performance of the subtask, and fine-tune the attention heads corresponding to the subtasks, so as to improve the model's domain capabilities while avoiding a reduction in the model's general capabilities. In addition, during fine-tuning model training, the efficiency of domain model training is improved and the cost of domain model training is reduced by freezing the network structure that is not related to the training of each subtask or has an impact less than a certain threshold.

[0161] For the specific limitations of the domain large language model fine-tuning training device, please refer to the limitations of the domain large language model fine-tuning training method above, which will not be repeated here. Each module in the above-mentioned domain large language model fine-tuning training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor of the electronic device in the form of hardware, or can be stored in the processor of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0162] The present disclosure also provides an electronic device, Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 4As shown, the electronic device provided in this embodiment includes: a memory 41 and a processor 42, the memory 41 is used to store a computer program; the processor 42 is used to execute the steps executed in any embodiment of the fault identification method of the image acquisition device provided by the above method embodiment when calling the computer program. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a method for identifying a fault of an image acquisition device is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0163] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In some embodiments, the domain large language model fine-tuning training device provided by the present disclosure can be implemented in the form of a computer, and the computer program can be used in the following manner: Figure 4 The computer program composed of various program modules enables the processor to execute the steps of the fault identification method of the image acquisition device of the electronic device of each embodiment of the present disclosure described in this specification.

[0165] The embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the fault identification method for the image acquisition device provided by the above method embodiment is implemented.

[0166] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0167] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0168] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0170] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0171] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fine-tuning a domain large language model, characterized in that: The method comprises: In the fine-tuning training stage, original instruction data of multiple domain tasks are obtained to construct a single-choice task data set; the single-choice task data set includes multiple single-choice task data of multiple domain tasks, and each single-choice task data includes: an original instruction title, a correct option and an incorrect option corresponding to the original instruction title; Performing data perturbation operations on the original instruction questions in the single-choice task data set, identifying key attention heads for tasks in various fields, and obtaining a key attention head index table; the key attention head index table is used to represent a mapping relationship between multiple field tasks and multiple key attention head lists; Based on the key attention head index table, load the target key attention head list corresponding to the target domain task; For the target domain task, key attention head training is performed based on the target key attention head list, the weight parameters of each key attention head in the target key attention head list are updated, and the weight parameters of other network layers are frozen.

2. The method according to claim 1, characterized in that The method of obtaining the original instruction data of tasks in multiple fields and constructing a single-choice task dataset includes: For each piece of original instruction data, according to a mapping function, extract the user instruction of each piece of original instruction data as the original instruction title of the single-choice task data; Inputting the original instruction data into a preset large language model to obtain the correct option of the single-choice task data; Inputting the original instruction title and the correct option into the preset large language model to obtain the wrong option of the single-choice task data; Constructing single-choice task data based on the original instruction title, the correct options, and the incorrect options; Based on multiple single-choice task data from tasks in multiple fields, a single-choice task dataset is constructed.

3. The method according to claim 2, characterized in that The data perturbation operation is performed on the original instruction title in the single-choice task data set, the key attention heads of tasks in various fields are identified, and the key attention head index table is obtained, including: For each domain task, the single-choice task data is input into the preset large language model, and the initial probability that the original instruction title of each single-choice task data predicted by the preset large language model is the correct option is obtained; Performing a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbed single-choice task data; Inputting the disturbance single-choice task data into the disturbance preset large language model, and obtaining the target probability that the disturbance preset large language model predicts that the original instruction title of each disturbance single-choice task data is the correct option; According to the initial probability and the target probability, determining whether the target attention head in the perturbed preset large language model is a key attention head, so as to obtain a list of key attention heads for tasks in various fields; According to the mapping relationship between multiple domain tasks and multiple key attention head index tables, a key attention head index table is obtained.

4. The method according to claim 3, characterized in that Before inputting the disturbance single-choice task data into the disturbance preset large language model, the method further includes: For each attention head in the preset large language model, the original activation value of the current attention head is replaced with the perturbed activation value, and the original activation values ​​of other attention heads are kept unchanged to obtain the perturbed preset large language model.

5. The method according to claim 4, characterized in that The determining, based on the initial probability and the target probability, whether the target attention head in the perturbed preset large language model is a key attention head, so as to obtain a list of key attention heads for tasks in various fields, includes: For each domain task, after replacing the perturbation activation value of each attention head, the difference between the initial probability and the target probability is obtained; According to the difference between the initial probability and the target probability, a list of key attention heads for tasks in various fields is obtained.

6. The method according to claim 1, characterized in that The updating of the weight parameters of each key attention head in the target key attention head list and freezing the weight parameters of other network layers include: By setting the value of the gradient tracking attribute corresponding to the target key attention head to true, setting the value of the gradient tracking attribute corresponding to other key attention heads to false, updating the weight parameters of each key attention head in the target key attention head list, and freezing the weight parameters of other network layers.

7. The method according to claim 1, characterized in that The performing of a data perturbation operation on the original instruction questions in the single-choice task data set to obtain perturbed single-choice task data includes: The original instruction questions in the single-choice task data set are subjected to sequence permutation and typos replacement operations to obtain disturbed single-choice task data.

8. A domain large language model fine-tuning training device, characterized in that: include: The dataset construction module is used to obtain the original instruction data of tasks in multiple fields and construct a single-choice task dataset during the fine-tuning training phase; The single-choice task data set includes multiple single-choice task data of multiple field tasks, and each single-choice task data includes: an original instruction title, a correct option and an incorrect option corresponding to the original instruction title; An index table acquisition module is used to perform data perturbation operations on the original instruction questions in the single-choice task data set, identify the key attention heads of various domain tasks, and obtain a key attention head index table; the key attention head index table is used to represent the mapping relationship between multiple domain tasks and multiple key attention head lists; A list loading module, used for loading a target key attention head list corresponding to a target domain task based on the key attention head index table; The parameter updating module is used to perform key attention head training based on the target key attention head list for the target domain task, update the weight parameters of each key attention head in the target key attention head list, and freeze the weight parameters of other network layers.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the domain large language model fine-tuning training method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for fine-tuning a large domain language model as described in any one of claims 1 to 7 is implemented.

Images