Parameter fine tuning method and device for downstream task adaptation of large language model

Abstract

The invention relates to the technical field of natural language processing, in particular to a parameter fine tuning method and device for downstream task adaptation of a large language model.The method comprises the steps that parameter fine tuning is conducted on a target layer of a pre-trained large language model to obtain the large language model after fine tuning; calculating first output probability distribution of the last layer of the target layer and second output probability distribution of the last layer of the non-target layer in the large language model after fine tuning; and performing knowledge updating on the target downstream task according to the first output probability distribution and the second output probability distribution. Therefore, the problems that the parameter quantity of a large language model is large, the requirement for consumption of computing resources in the fine adjustment process is high, meanwhile, the training time is long, and the downstream task adaptation cost of the large model is high are solved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a parameter fine-tuning method and device for downstream task adaptation of a large language model. Background Art

[0002] Many studies have been devoted to addressing the adaptation challenges of pre-trained large language models (LLMs) in downstream tasks by focusing on fine-tuning techniques and the construction of training datasets. Among various fine-tuning techniques, PEFT (Parameter-Efficient Fine-Tuning) has attracted widespread attention from the community due to its ability to freeze most of the model parameters while producing impressive performance. Therefore, PEFT methods for LLMs have been proposed, such as a parameter-efficient transfer learning method that is implemented by using adapter modules, which allows very few trainable parameters to be added to each downstream task without sacrificing performance; a lightweight transfer learning method for natural language generation tasks that fine-tunes the model by optimizing a series of continuous task-specific vectors (called prefixes) without changing the parameters of the pre-trained language model. At the same time, in order to align the behavior of LLMs with human responses, fine-tuning LLMs usually requires manually annotated datasets, but this incurs significant costs. In contrast, some studies have constructed high-quality training datasets by maximizing the use of existing datasets. For example, we enrich training examples by reversing the input-output pairs of existing examples and pairing them with specially crafted task descriptions; we enhance the annotated dataset by adding human-written task descriptions to guide LLM to understand the task.

[0003] As mentioned above, existing large model downstream task adaptation methods often introduce high training costs or data costs, which not only requires high resource costs for training, but also takes a long time to train. Considering the large number of parameters in existing large language models, the above-mentioned training-side or data-side methods all involve updating a large number of parameters, resulting in inefficiency in downstream task adaptation. In addition, existing large model downstream task adaptation methods do not take into account the different parameter properties of large language models, and do not distinguish the roles of shallow and deep decoding of large language models. Therefore, they fail to make good use of the inherent characteristics of large language model parameters to reduce fine-tuning parameters, and cannot complete the adaptation of large language models in downstream tasks under resource-constrained conditions (such as insufficient GPU memory), and cannot meet the needs of scenarios with high training time requirements such as fast iteration. Summary of the invention

[0004] The present invention provides a parameter fine-tuning method and device for downstream task adaptation of a large language model, so as to solve the problems that due to the large number of parameters of the large language model, the fine-tuning process has high requirements on the consumption of computing resources, the training time is long, and the downstream task adaptation cost of the large model is high.

[0005] The first aspect of the present invention provides a parameter fine-tuning method for adapting a large language model to downstream tasks, comprising the following steps: fine-tuning the parameters of a target layer of a pre-trained large language model to obtain a fine-tuned large language model; calculating a first output probability distribution of the last layer of the target layer and a second output probability distribution of the last layer of a non-target layer in the fine-tuned large language model; and updating knowledge of a target downstream task according to the first output probability distribution and the second output probability distribution.

[0006] Optionally, fine-tuning parameters of a target layer of the pre-trained large language model to obtain a fine-tuned large language model includes:

[0007] Obtain the total number of layers of the pre-trained large language model;

[0008] Determining a boundary between a shallow decoding layer and a deep decoding layer of the pre-trained large language model according to the total number of layers, using the shallow decoding layer as the non-target layer and the deep decoding layer as the target layer;

[0009] Based on the limit, parameters of the non-target layer are frozen, and parameters of the target layer are fine-tuned to obtain a fine-tuned large language model.

[0010] Optionally, updating knowledge of a target downstream task according to the first output probability distribution and the second output probability distribution includes:

[0011] Performing inter-layer comparison decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution;

[0012] The knowledge of the target downstream task is updated according to the final probability distribution.

[0013] Optionally, the final probability distribution is expressed as:

[0014] P final (y|x)=P L (y|x,θ′)-P L-m (y|x,θ)+λP L (y|x,θ′)

[0015] Among them, P final (y|x) is the final probability distribution, P L(y|x, θ′) is the first output probability distribution, P L-m (y|x, θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

[0016] The second aspect of the present invention provides a parameter fine-tuning device for adapting downstream tasks of a large language model, including: a parameter fine-tuning module, used to fine-tune the parameters of the target layer of the pre-trained large language model to obtain a fine-tuned large language model; a probability calculation module, used to calculate the first output probability distribution of the last layer of the target layer and the second output probability distribution of the last layer of the non-target layer in the fine-tuned large language model; a knowledge updating module, used to update the knowledge of the target downstream task according to the first output probability distribution and the second output probability distribution.

[0017] Optionally, the parameter fine-tuning module includes:

[0018] An acquisition unit, used to acquire the total number of layers of the pre-trained large language model;

[0019] A determination unit, configured to determine a boundary between a shallow decoding layer and a deep decoding layer of the pre-trained large language model according to the total number of layers, and use the shallow decoding layer as the non-target layer and the deep decoding layer as the target layer;

[0020] The fine-tuning unit is used to freeze the parameters of the non-target layer based on the limit and fine-tune the parameters of the target layer to obtain a fine-tuned large language model.

[0021] Optionally, the knowledge updating module includes:

[0022] A decoding unit, configured to perform inter-layer comparison decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution;

[0023] An updating unit is used to update the knowledge of the target downstream task according to the final probability distribution.

[0024] Optionally, the final probability distribution is expressed as:

[0025] P final (y|x)=P L (y|x,θ′)-P L-m (y|x,θ)+λP L (y|x,θ′)

[0026] Among them, P final (y|x) is the final probability distribution, P L (y|x, θ′) is the first output probability distribution, P L-m(y|x, θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

[0027] An embodiment of the third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the parameter fine-tuning method for downstream task adaptation of a large language model as described in the above embodiment.

[0028] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above parameter fine-tuning method for downstream task adaptation of a large language model.

[0029] The parameter fine-tuning method and device for downstream task adaptation of a large language model proposed in an embodiment of the present invention only fine-tune some layers of the pre-trained large language model, which can not only reduce computational overhead, but also retain and utilize the grammatical knowledge mastered in the pre-training of the model to the greatest extent; an inter-layer comparative decoding method is used to solve the problem of weak performance caused by fine-tuning only some layers, so that the knowledge differences before and after fine-tuning and between different layers of the model can be simultaneously utilized to improve the model performance.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A flow chart of a parameter fine-tuning method for downstream task adaptation of a large language model provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of a specific execution of a parameter fine-tuning method for large language model downstream task adaptation provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of the structure of a parameter fine-tuning device for large language model downstream task adaptation provided by an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0037] The following describes a parameter fine-tuning method and apparatus for large language model downstream task adaptation according to an embodiment of the present invention with reference to the accompanying drawings.

[0038] Figure 1 A flowchart of a parameter fine-tuning method for downstream task adaptation of a large language model provided by an embodiment of the present invention.

[0039] like Figure 1 As shown, the parameter fine-tuning method for downstream task adaptation of a large language model includes the following steps:

[0040] In step S101, parameters of the target layer of the pre-trained large language model are fine-tuned to obtain a fine-tuned large language model.

[0041] In some embodiments, fine-tuning parameters of a target layer of a pre-trained large language model to obtain a fine-tuned large language model includes:

[0042] Get the total number of layers of the pre-trained large language model;

[0043] Determine the boundary between the shallow decoding layer and the deep decoding layer of the pre-trained large language model according to the total number of layers, use the shallow decoding layer as a non-target layer, and use the deep decoding layer as a target layer;

[0044] Based on the limit, the parameters of the non-target layer are frozen and the parameters of the target layer are fine-tuned to obtain a fine-tuned large language model.

[0045] like Figure 2 As shown in the figure, in the actual implementation process, in the field of natural language processing, large language models (LLMs) such as GPT, BLOOM, LLaMA, etc. have demonstrated excellent performance. In order to improve the performance of these models in downstream tasks, fine-tuning is a common strategy.

[0046] In the transformer network used by mainstream large models, the shallow decoding layer mainly processes grammatical information, and the deep decoding layer processes semantic information and calls the model's advanced knowledge. Since the pre-trained large language model already has the ability to understand and generate grammatically coherent text, the main purpose of fine-tuning the model on downstream tasks is to update the knowledge base, that is, to improve the semantic level. However, the existing traditional fine-tuning methods require adjusting the parameters of all layers, which is computationally expensive and time-consuming.

[0047] Therefore, an embodiment of the present invention proposes a method for fine-tuning some layers. First, the total number of layers of the pre-trained large language model is obtained, and the boundary between the shallow decoding layer and the deep decoding layer of the pre-trained large language model is determined according to the total number of layers. The shallow decoding layer is used as a non-target layer, and the parameters of the non-target layer are frozen. The deep decoding layer is used as the target layer, and the parameters of the target layer are fine-tuned to obtain the fine-tuned large language model.

[0048] Assume that a pre-trained large language model has a total of L layers. According to the L layers, the limit P = 1 / 4L is determined. After the limit P is the deep decoding layer of the pre-trained large language model. Assume that the deep decoding layer has a total of m layers.

[0049] The layer before the boundary P is the shallow decoding layer of the pre-trained large language model, which is the Lm layer.

[0050] Furthermore, the parameters of the Lm layers (i.e., the 1st to the Lmth layers) are frozen, and the last m layers are fine-tuned (i.e., the L-m+1th to the Lth layers) to obtain a fine-tuned large language model. By this method of partial layer fine-tuning, the parameter updates can be reduced by approximately (Lm) / L, thereby greatly reducing the computational cost.

[0051] The fine-tuned expression is:

[0052] θ′={θ Lm+1 ,θ Lm+2 , K, θ L}

[0053] Where θ is the parameter of the pre-trained large language model, and θ′ is the set of fine-tuned parameters.

[0054] In addition, since only the deep decoding layer of the pre-trained large language model is fine-tuned, the shallow decoding layer of the pre-trained large language model still retains the global grammatical knowledge during pre-training, which can make good use of the versatility of the pre-trained large language model.

[0055] In step S102, the first output probability distribution of the last layer of the target layer and the second output probability distribution of the last layer of the non-target layer in the fine-tuned large language model are calculated.

[0056] In step S103, knowledge of the target downstream task is updated according to the first output probability distribution and the second output probability distribution.

[0057] In some embodiments, updating knowledge of the target downstream task according to the first output probability distribution and the second output probability distribution includes:

[0058] Performing inter-layer comparative decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution;

[0059] The knowledge of the target downstream task is updated according to the final probability distribution.

[0060] like Figure 2 As shown, in the actual execution process, since the embodiment of the present invention only fine-tunes the parameters of the deep decoding layer of the pre-trained large language model, this method will bring about the problem of weaker performance compared to the full fine-tuning model. Therefore, when the embodiment of the present invention performs reasoning on the downstream task, it is assumed that the given input is x, and the first output probability distribution P of the last layer (i.e., the Lth layer) of the deep decoding layer after fine-tuning of the model is calculated at the same time. L (y|x, θ′) and the second output probability distribution P of the last layer (i.e., the Lm-th layer) of the shallow decoding layer without fine-tuning L-m (y|x,θ).

[0061] Furthermore, the inter-layer contrast decoding method is used to improve the model performance by using the knowledge differences before and after fine-tuning and between different layers of the model. That is, the final probability distribution is generated by comparing the output probability distributions of the two layers, and then the knowledge of the target downstream task is updated according to the final probability distribution, which makes up for the accuracy loss caused by fine-tuning some parameters compared to full fine-tuning. Among them, the expression of the final probability distribution is:

[0062] P final (y|x)=P L (y|x,θ′)-P L-m (y|x,θ)+λP L (y|x,θ′)

[0063] Where P final (y|x) is the final probability distribution, P L (y|x, θ′) is the first output probability distribution, P L-m (y|x, θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

[0064] In summary, the parameter fine-tuning method for large language model downstream task adaptation proposed in an embodiment of the present invention has the following beneficial effects:

[0065] (1) Since only the last few layers are fine-tuned, the number of parameter updates is greatly reduced, the computational cost is reduced, the fine-tuning time is saved, and the requirement for training video memory is low.

[0066] (2) The un-fine-tuned layers retain the global knowledge from pre-training, which helps the model to be more versatile in different tasks;

[0067] (3) Through inter-layer contrast decoding, the pre-trained knowledge and fine-tuned task-specific knowledge can be effectively combined to complete efficient knowledge transfer and improve model performance.

[0068] Next, a parameter fine-tuning device for large language model downstream task adaptation proposed according to an embodiment of the present invention is described with reference to the accompanying drawings.

[0069] Figure 3 It is a block diagram of a parameter fine-tuning device for large language model downstream task adaptation according to an embodiment of the present invention.

[0070] like Figure 3 As shown, the parameter fine-tuning device 30 for large language model downstream task adaptation includes: a parameter fine-tuning module 301, a probability calculation module 302 and a knowledge updating module 303.

[0071] Among them, the parameter fine-tuning module 301 is used to fine-tune the parameters of the target layer of the pre-trained large language model to obtain the fine-tuned large language model. The probability calculation module 302 is used to calculate the first output probability distribution of the last layer of the target layer and the second output probability distribution of the last layer of the non-target layer in the fine-tuned large language model. The knowledge update module 303 is used to update the knowledge of the target downstream task according to the first output probability distribution and the second output probability distribution. The device reduces the fine-tuning parameters to achieve the purpose of saving computing resources, reducing fine-tuning time, and reducing the downstream task adaptation cost of the large model.

[0072] In some embodiments, the parameter fine-tuning module 301 includes:

[0073] An acquisition unit is used to obtain the total number of layers of the pre-trained large language model;

[0074] A determination unit, used to determine the boundary between the shallow decoding layer and the deep decoding layer of the pre-trained large language model according to the total number of layers, and to use the shallow decoding layer as a non-target layer and the deep decoding layer as a target layer;

[0075] The fine-tuning unit is used to freeze the parameters of the non-target layer based on the limit and fine-tune the parameters of the target layer to obtain a fine-tuned large language model.

[0076] In some embodiments, the knowledge update module 303 includes:

[0077] A decoding unit, configured to perform inter-layer comparative decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution;

[0078] The updating unit is used to update the knowledge of the target downstream task according to the final probability distribution.

[0079] In some embodiments, the expression of the final probability distribution is:

[0080] P final (y|x)=P L (y|x,θ′)-PL-m (y|x,θ)+λP L (y|x,θ′)

[0081] Among them, P final (y|x) is the final probability distribution, P L (y|x, θ′) is the first output probability distribution, P L-m (y|x, θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

[0082] It should be noted that the aforementioned explanation of the embodiment of the parameter fine-tuning method for adapting downstream tasks of a large language model is also applicable to the parameter fine-tuning device for adapting downstream tasks of a large language model in this embodiment, and will not be repeated here.

[0083] In summary, the parameter fine-tuning device for large language model downstream task adaptation proposed in the embodiment of the present invention has the following beneficial effects:

[0084] (1) Since only the last few layers are fine-tuned, the number of parameter updates is greatly reduced, the computational cost is reduced, the fine-tuning time is saved, and the requirement for training video memory is low.

[0085] (2) The un-fine-tuned layers retain the global knowledge from pre-training, which helps the model to be more versatile in different tasks;

[0086] (3) Through inter-layer contrast decoding, the pre-trained knowledge and fine-tuned task-specific knowledge can be effectively combined to complete efficient knowledge transfer and improve model performance.

[0087] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0088] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0089] When the processor 402 executes the program, the parameter fine-tuning method for large language model downstream task adaptation provided in the above embodiment is implemented.

[0090] Furthermore, the electronic device further comprises:

[0091] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0092] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0093] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0094] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0095] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0096] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0097] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above parameter fine-tuning method for downstream task adaptation of a large language model.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0100] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0102] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0103] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0105] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A parameter fine-tuning method for downstream task adaptation of a large language model, characterized in that: The following steps are involved: Fine-tune the parameters of the target layer of the pre-trained large language model to obtain a fine-tuned large language model; Calculate the first output probability distribution of the last layer of the target layer and the second output probability distribution of the last layer of the non-target layer in the fine-tuned large language model; Knowledge of a target downstream task is updated according to the first output probability distribution and the second output probability distribution.

2. The parameter fine-tuning method for downstream task adaptation of a large language model according to claim 1, characterized in that: The step of fine-tuning parameters of the target layer of the pre-trained large language model to obtain a fine-tuned large language model includes: Obtain the total number of layers of the pre-trained large language model; Determining a boundary between a shallow decoding layer and a deep decoding layer of the pre-trained large language model according to the total number of layers, using the shallow decoding layer as the non-target layer and the deep decoding layer as the target layer; Based on the limit, parameters of the non-target layer are frozen, and parameters of the target layer are fine-tuned to obtain a fine-tuned large language model.

3. The parameter fine-tuning method for large language model downstream task adaptation according to claim 1, characterized in that: The updating of knowledge of the target downstream task according to the first output probability distribution and the second output probability distribution includes: Performing inter-layer comparison decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution; The knowledge of the target downstream task is updated according to the final probability distribution.

4. The parameter fine-tuning method for large language model downstream task adaptation according to claim 3, characterized in that: The expression of the final probability distribution is: P final (y|x)=P L (y|x,θ′)-P L-m (y|x,θ)+λP L (y|x,θ′) Among them, P final (y|x) is the final probability distribution, P L (y|x,θ′) is the first output probability distribution, P L-m (y|x,θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

5. A parameter fine-tuning device for downstream task adaptation of a large language model, characterized in that: include: A parameter fine-tuning module is used to fine-tune the parameters of the target layer of the pre-trained large language model to obtain a fine-tuned large language model; A probability calculation module, used to calculate the first output probability distribution of the last layer of the target layer and the second output probability distribution of the last layer of the non-target layer in the fine-tuned large language model; A knowledge updating module is used to update the knowledge of the target downstream task according to the first output probability distribution and the second output probability distribution.

6. The parameter fine-tuning device for large language model downstream task adaptation according to claim 5, characterized in that: The parameter fine-tuning module includes: An acquisition unit, used to acquire the total number of layers of the pre-trained large language model; A determination unit, configured to determine a boundary between a shallow decoding layer and a deep decoding layer of the pre-trained large language model according to the total number of layers, and use the shallow decoding layer as the non-target layer and the deep decoding layer as the target layer; The fine-tuning unit is used to freeze the parameters of the non-target layer based on the limit and fine-tune the parameters of the target layer to obtain a fine-tuned large language model.

7. The parameter fine-tuning device for large language model downstream task adaptation according to claim 5, characterized in that: The knowledge updating module comprises: A decoding unit, configured to perform inter-layer comparison decoding on the first output probability distribution and the second output probability distribution to generate a final probability distribution; An updating unit is used to update the knowledge of the target downstream task according to the final probability distribution.

8. The parameter fine-tuning device for large language model downstream task adaptation according to claim 7, characterized in that: The expression of the final probability distribution is: P final (y|x)=P L (y|x,θ′)-P L-m (y|x,θ)+λP L (y|x,θ′) Among them, P final (y|x) is the final probability distribution, P L (y|x,θ′) is the first output probability distribution, P L-m (y|x,θ) is the output probability distribution of the last layer without fine-tuning, and λ is the balancing coefficient.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the parameter fine-tuning method for downstream task adaptation of a large language model as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the parameter fine-tuning method for large language model downstream task adaptation as described in any one of claims 1 to 4.

Images