Method and device for fine-tuning pre-trained language model and computer readable storage medium

Abstract

The application provides a pre-training language model fine-tuning method, device and computer readable storage medium, and belongs to the technical field of natural language processing. The pre-training language model fine-tuning method comprises the following steps: obtaining a pre-trained pre-training language model, determining the gradient threshold of each encoding layer in the pre-training language model; training the pre-training language model, and calculating the gradient norm of any encoding layer before updating the model parameters of the encoding layer each time in the training process; and determining whether the model parameters of the encoding layer need to be updated this time according to whether the gradient norm of the encoding layer is less than the gradient threshold of the encoding layer. The application can achieve a better compromise in retaining the general language knowledge of the pre-training language model and training a model suitable for a downstream task, so that the model can retain more general language knowledge and improve the performance of the model.

Description

Technical Field

[0001] This invention relates to the field of pre-trained language model (PLM) technology in Natural Language Processing (NLP), and in particular to a method, apparatus and computer-readable storage medium for fine-tuning a pre-trained language model. Background Technology

[0002] Pre-trained models have long been used in image processing, while the concept of PLM (Plug-in Model) is considered to have been first proposed in 2015 and has gradually achieved superior performance in various tasks. Prior to this, NLP more commonly used word embedding techniques, embedding words into a multi-dimensional space and training them on a large corpus to capture specific relationships between words. Commonly used word vector training models include Word2vec and GloVe. Word embeddings can be used to initialize the first embedding layer of downstream models, along with other functional layers, to build the entire model. However, early word embedding methods did not retain the contextual information of each word, which had its limitations.

[0003] Pre-trained language models (PLMs) are trained using a large amount of text that appears in people's daily lives. This allows the model to learn the probability distribution of each word or character in these texts, thereby modeling a model that conforms to these text distributions. Furthermore, language models typically predict adjacent words or characters based on context, which varies depending on the training task. Therefore, the labels of the language model's corpus are its context, requiring no re-labeling. This allows for almost unlimited use of large-scale corpora to train language models, enabling them to learn rich semantic knowledge. The sheer size of the corpus gives pre-trained language models powerful capabilities, significantly improving performance on downstream tasks while reducing training difficulty. Current pre-trained language models include: Embeddings from Language Models (ELMo), Generative Pre-Training (GPT / GPT2), and Bidirectional Encoder Representations from Transformers (BERT).

[0004] After obtaining a pre-trained language model, the network structure for downstream tasks can be initialized using the pre-trained model parameters. This allows the linguistic knowledge learned during pre-training to be incorporated into the downstream tasks. To adapt the model for solving specific downstream tasks, fine-tuning of the network parameters and structural modifications are typically required. For example... Figure 1 As shown, the fine-tuning process for a pre-trained language model typically includes two stages: a first stage (warm-up) and a second stage (standard training). Standard training begins after warm-up and continues until a predetermined termination condition is met. Compared to standard training, warm-up training usually involves more significant adjustments to the pre-trained language model. These large adjustments to model parameters can lead to the loss (forgetting) of general language knowledge learned during pre-training. Therefore, finding a compromise between preserving general language knowledge and obtaining a model suitable for downstream tasks during the fine-tuning process becomes a pressing issue. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, apparatus and computer-readable storage medium for fine-tuning a pre-trained language model, which can achieve a good compromise between preserving the general language knowledge of the pre-trained language model and training to obtain a model suitable for downstream tasks.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide the following technical solutions:

[0007] Embodiments of the present invention provide a method for fine-tuning a pre-trained language model, comprising:

[0008] Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0009] The pre-trained language model is trained, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process.

[0010] Whether the model parameters of any coding layer need to be updated depends on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0011] Optionally, the method further includes:

[0012] If it is determined that the model parameters of any coding layer need to be updated, the model parameters of any coding layer are updated according to the gradient of the model parameters of any coding layer.

[0013] If it is determined that updating the model parameters of any coding layer is not necessary in this instance, then updating the model parameters of any coding layer is rejected.

[0014] Optionally, the gradient norm of any of the coding layers may be calculated in any of the following ways:

[0015] The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer.

[0016] The gradient norm of any coding layer is obtained by summing the absolute values ​​of each model parameter in any coding layer.

[0017] The maximum absolute value of each model parameter in any given coding layer is determined and used as the gradient norm of that coding layer.

[0018] Optionally, determining the gradient threshold for each encoding layer in the pre-trained language model includes:

[0019] The gradient threshold of each coding layer is calculated according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

[0020] Optionally, the encoding layer is numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

[0021] Optionally, the gradient norm is non-negative.

[0022] This invention also provides a fine-tuning device for a pre-trained language model, comprising:

[0023] A determination module is used to obtain a pre-trained language model and determine the gradient threshold of each coding layer in the pre-trained language model.

[0024] The computation module is used to train the pre-trained language model and, before updating the model parameters of any encoding layer each time during the training process, calculate the gradient norm of any encoding layer.

[0025] The judgment module is used to determine whether the model parameters of any coding layer need to be updated this time based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0026] Optionally, the fine-tuning device further includes:

[0027] An update module is configured to update the model parameters of any coding layer according to the gradient of the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer need to be updated; and to refuse to update the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer do not need to be updated.

[0028] Optionally, the calculation module is further configured to calculate the gradient norm of any coding layer in any of the following ways:

[0029] The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer.

[0030] The gradient norm of any coding layer is obtained by summing the absolute values ​​of each model parameter in any coding layer.

[0031] The maximum absolute value of each model parameter in any given coding layer is determined and used as the gradient norm of that coding layer.

[0032] Optionally, the determining module is further configured to calculate the gradient threshold of each coding layer according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

[0033] Optionally, the encoding layer is numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

[0034] This invention also provides a fine-tuning device for a pre-trained language model, comprising:

[0035] Processor; and

[0036] A memory, in which computer program instructions are stored,

[0037] When the computer program instructions are executed by the processor, the processor performs the following steps:

[0038] Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0039] The pre-trained language model is trained, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process.

[0040] Whether the model parameters of any coding layer need to be updated depends on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0041] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the pre-trained language model fine-tuning method as described above.

[0042] The embodiments of the present invention have the following beneficial effects: The embodiments of the present invention set a corresponding gradient threshold for each coding layer, and determine whether the model parameters of the coding layer need to be updated based on whether the gradient norm of the coding layer is less than the threshold, so as to avoid excessive adjustment of the model parameters. This achieves a good trade-off between preserving the general language knowledge of the pre-trained language model and training a model suitable for downstream tasks, so that the model can retain more general language knowledge and improve the performance of the model. Attached Figure Description

[0043] Figure 1 A schematic diagram illustrating the fine-tuning process of a pre-trained language model in the prior art;

[0044] Figure 2 This is a flowchart illustrating a method for fine-tuning a pre-trained language model according to an embodiment of the present invention.

[0045] Figure 3 This is another flowchart illustrating the fine-tuning method for a pre-trained language model according to an embodiment of the present invention.

[0046] Figure 4 A schematic diagram of the structure of a pre-trained language model provided in an embodiment of the present invention;

[0047] Figure 5 An example diagram of the threshold function provided in an embodiment of the present invention;

[0048] Figure 6 This is a structural block diagram of a fine-tuning device for a pre-trained language model according to an embodiment of the present invention;

[0049] Figure 7 This is another structural block diagram of the fine-tuning device for the pre-trained language model according to an embodiment of the present invention;

[0050] Figure 8 This is another structural block diagram of the fine-tuning device for the pre-trained language model according to an embodiment of the present invention. Detailed Implementation

[0051] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0052] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The terms “and / or” in the specification and claims indicate at least one of the connected objects.

[0053] The following description provides examples and is not intended to limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0054] As described in the background section, during the fine-tuning of a pre-trained language model, the model parameters need to be adjusted according to the downstream task. However, if the adjustment of the model parameters is too large, the general language knowledge learned by the model during the pre-training stage will be discarded or forgotten, thus affecting the model performance.

[0055] To achieve a better trade-off between preserving the general language knowledge of the pre-trained language model and training a model suitable for downstream tasks, enabling the model to retain more general language knowledge and thus improve model performance, this invention provides a fine-tuning method for a pre-trained language model, such as... Figure 2 As shown, it includes:

[0056] Step 201: Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0057] Here, the pre-trained language model is a model that has been trained in advance through a pre-training phase. This pre-trained language model is typically trained on a large amount of text, learning a wealth of general language knowledge. Specifically, the pre-trained language model can be an ELMo model, a GPT / GPT2 model, or a BERT model, etc., and this embodiment of the invention does not impose any specific limitations on it.

[0058] The pre-trained language model typically includes an input layer and an output layer, with multiple encoding layers usually existing between them. In step 201 of this embodiment, the gradient threshold of each encoding layer is determined for use in subsequent steps.

[0059] Step 202: Train the pre-trained language model, and before updating the model parameters of any encoding layer each time during the training process, calculate the gradient norm of any encoding layer.

[0060] When applying a pre-trained language model to a specific downstream task, it needs to be fine-tuned according to the downstream task. Specifically, the pre-trained language model can be trained using training samples to adjust its parameters. This fine-tuning process typically includes a warm-up training phase and a standard training phase. Compared to the standard training phase, the warm-up training phase usually involves more significant adjustments to the model parameters.

[0061] For example, during fine-tuning, the pre-trained language model is trained 10,000 times, with the first 2,000 iterations being warm-up training and the last 8,000 being standard training. Typically, the training samples for the two phases differ; for instance, the training samples used in the warm-up training phase are less complex than those used in the standard training phase. Specifically, the text length of the training samples in the warm-up training phase is shorter than that in the standard training phase. Furthermore, the learning rate of the model during the warm-up training phase can gradually increase as the training progresses.

[0062] Preferably, considering the significant adjustments to model parameters during the warm-up training phase, this embodiment of the invention can be applied to the warm-up training phase of the fine-tuning process. Of course, the method of this embodiment may also be executed during the standard training phase, and this invention does not specifically limit its application in this regard.

[0063] In step 202, the pre-trained language model can be trained using training samples from specific downstream tasks. During training, the model parameters are adjusted (fine-tuned). Since multiple iterations of training may be performed, the model parameters of the encoding layer usually need to be updated after each iteration. To avoid excessively large adjustments to the model parameters, this embodiment of the invention calculates the gradient norm of each encoding layer before adjusting its model parameters. The gradient norm characterizes the degree of gradient change of the model parameters of that encoding layer. Generally, the greater the degree of gradient change of the model parameters of the encoding layer, the larger the gradient norm. Here, the gradient norm is usually a non-negative value.

[0064] Step 203: Determine whether the model parameters of any coding layer need to be updated this time based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0065] In step 203 of this embodiment of the invention, by comparing the gradient norm of any coding layer with the gradient threshold of any coding layer, it is determined whether the model parameters of any coding layer need to be updated after the current iteration of training. Generally, if the gradient norm of any coding layer is less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer need to be updated; if the gradient norm of any coding layer is not less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer do not need to be updated.

[0066] Through the above steps, in the fine-tuning process of the pre-trained language model, before updating the model parameters of any encoding layer, the present invention uses a gradient threshold to determine whether the adjustment range of the model parameters of that encoding layer is too large. If it is too large, the adjustment of the model parameters of that encoding layer is rejected. This achieves a good trade-off between preserving the general language knowledge of the pre-trained language model and training a model suitable for downstream tasks, enabling the model to retain more general language knowledge and thus improving the model's performance.

[0067] like Figure 3 As shown in the embodiment of the present invention, another method for fine-tuning a pre-trained language model includes:

[0068] Step 301: Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0069] Step 302: Train the pre-trained language model, and before updating the model parameters of any encoding layer each time during the training process, calculate the gradient norm of any encoding layer.

[0070] Step 303: Determine whether the model parameters of any coding layer need to be updated this time based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0071] Steps 301 to 303 above and Figure 2 Steps 201 to 203 are similar and will not be repeated here to save space.

[0072] Step 304: If it is determined that the model parameters of any coding layer need to be updated this time, update the model parameters of any coding layer according to the gradient of the model parameters of any coding layer.

[0073] When it is determined that the model parameters need to be updated, this embodiment of the invention can update the model parameters based on the gradient of the model parameters in the encoding layer. After the update is completed, if the preset termination condition for fine-tuning training is met, the process ends; otherwise, the next round of iterative training continues.

[0074] Step 305: If it is determined that it is not necessary to update the model parameters of any coding layer in this instance, then the update of the model parameters of any coding layer is rejected.

[0075] If it is determined that the model parameters do not need to be updated, this embodiment of the invention will reject the update of the model parameters for this time. Then, it is determined whether the preset termination condition for fine-tuning training has been met. If so, the process ends; otherwise, the next round of iterative training continues.

[0076] Figure 4 A schematic diagram of a pre-trained language model is presented, including an input layer, an output layer, and multiple encoding layers located between the input and output layers. Encoding layers closer to the input layer typically learn more general language knowledge, such as grammatical structures and parts-of-speech features. Encoding layers closer to the output layer typically learn more knowledge relevant to specific downstream tasks. Therefore, encoding layers closer to the input layer have smaller gradient thresholds for their model parameters, while those closer to the output layer have larger gradient thresholds. This allows encoding layers closer to the input layer to retain more general language knowledge, while those closer to the output layer learn more task-specific knowledge, thus achieving a better trade-off between preserving the general language knowledge of the pre-trained language model and training a model suitable for downstream tasks.

[0077] In steps 201 and 301 above, the embodiments of the present invention can calculate the gradient threshold of each coding layer according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

[0078] Assuming the encoding layers are numbered sequentially from the input layer to the output layer of the pre-trained language model, for example, numbered encoding layer 1, encoding layer 2, ..., encoding layer n, and the output of the threshold function is positively correlated with the input of the threshold function, this ensures that the gradient threshold of the model parameters of the encoding layer closer to the input layer is smaller, and the gradient threshold of the model parameters of the encoding layer closer to the output layer is larger. Furthermore, the gradient norm is typically non-negative. Figure 5 An example of a threshold function is given, in which the gradient threshold output by the threshold function has a linear relationship with the input coding layer number, and the output increases as the input increases.

[0079] In steps 202 and 302 above, the gradient norm of the coding layer can be calculated in any of the following ways:

[0080] Method 1: Calculate the sum of squares of the gradients of each model parameter in any coding layer to obtain the gradient norm of any coding layer;

[0081] Method 2: Calculate the sum of the absolute values ​​of each model parameter in any coding layer to obtain the gradient norm of any coding layer;

[0082] Method 3: Determine the largest absolute value among all model parameters of any coding layer as the gradient norm of any coding layer.

[0083] Of course, embodiments of the present invention can also use other calculation methods to calculate the gradient norm. For example, the average of the absolute values ​​of each model parameter of any coding layer can be calculated as the gradient norm of any coding layer; or, the absolute values ​​of each model parameter of any coding layer can be weighted and summed according to preset parameter weights to obtain the gradient norm of any coding layer, and so on. These methods can all be applied to the present invention, and will not be described in detail here.

[0084] Based on the above-described fine-tuning method for pre-trained language models, this embodiment of the invention also provides a fine-tuning device for pre-trained language models, such as... Figure 6 As shown, it includes:

[0085] The determination module 61 is used to obtain a pre-trained language model and determine the gradient threshold of each coding layer in the pre-trained language model.

[0086] The calculation module 62 is used to train the pre-trained language model and calculate the gradient norm of any coding layer before updating the model parameters of any coding layer during the training process.

[0087] The judgment module 63 is used to determine whether the model parameters of any coding layer need to be updated this time based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0088] Through the above model, the fine-tuning device of this embodiment of the invention can achieve a good trade-off between preserving the general language knowledge of the pre-trained language model and training a model suitable for downstream tasks, so that the model can retain more general language knowledge and improve the performance of the model.

[0089] Optional, such as Figure 7 As shown, another fine-tuning device for a pre-trained language model provided by the present invention includes, in addition to, Figure 6 In addition to the modules in the document, it also includes:

[0090] The update module 64 is configured to update the model parameters of any coding layer according to the gradient of the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer need to be updated this time; and to refuse to update the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer do not need to be updated this time.

[0091] Optional, Figure 6 or Figure 7 The calculation module 62 is further configured to calculate the gradient norm of the coding layer in any of the following ways:

[0092] The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer.

[0093] The gradient norm of any coding layer is obtained by summing the absolute values ​​of each model parameter in any coding layer.

[0094] The maximum absolute value of each model parameter in any given coding layer is determined and used as the gradient norm of that coding layer.

[0095] Optional, Figure 6 or Figure 7 The determining module 61 is further configured to calculate the gradient threshold of each coding layer according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

[0096] Optionally, the encoding layer is numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

[0097] Optionally, the gradient norm is non-negative.

[0098] It should be noted that the device in this embodiment is the same as the one described above. Figure 2 or Figure 3 The devices corresponding to the methods shown are all applicable to the embodiments of the above-described devices, and can achieve the same technical effects. The devices provided by the embodiments of the present invention can implement all the method steps implemented in the above-described method embodiments and can achieve the same technical effects. Therefore, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail here.

[0099] Please refer to Figure 8 The present invention also provides a hardware structure block diagram of a fine-tuning device for a pre-trained language model, such as... Figure 8 As shown, the fine-tuning device 800 for the pre-trained language model includes:

[0100] Processor 802; and

[0101] Memory 804, wherein computer program instructions are stored.

[0102] When the computer program instructions are executed by the processor, the processor 802 performs the following steps:

[0103] Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0104] The pre-trained language model is trained, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process.

[0105] Whether the model parameters of any coding layer need to be updated depends on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0106] Furthermore, such as Figure 8 As shown, the fine-tuning device 800 for the pre-trained language model may also include a network interface 801, an input device 803, a hard disk 805, and a display device 806.

[0107] The various interfaces and devices described above can be interconnected via a bus architecture. The bus architecture can include any number of interconnected buses and bridges. Specifically, it connects one or more processors with computing capabilities, represented by processor 802, which may include a central processing unit (CPU) and / or a graphics processing unit (GPU), and various circuits of one or more memories, represented by memory 804. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. It is understood that the bus architecture is used to implement communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are well known in the art and will not be described in detail herein.

[0108] The network interface 801 can be connected to a network (such as the Internet, local area network, etc.), receive data (such as training samples and / or pre-trained language models) from the network, and save the received data to the hard disk 805.

[0109] The input device 803 can receive various instructions input by the operator and send them to the processor 802 for execution. The input device 803 may include a keyboard or a clicking device (e.g., a mouse, trackball, touchpad, or touchscreen).

[0110] The display device 806 can display the results obtained by the processor 802 executing instructions, such as displaying the fine-tuning progress of the pre-trained language model training.

[0111] The memory 804 is used to store programs and data necessary for the operation of the operating system, as well as intermediate results and other data during the calculation process of the processor 802.

[0112] It is understood that the memory 804 in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. The memory 804 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0113] In some implementations, memory 804 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operating system 8041 and application program 8042.

[0114] The operating system 8041 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 8042 includes various applications, such as a browser, used to implement various application functions. The program implementing the method of this embodiment can be included in the application program 8042.

[0115] The fine-tuning method for the pre-trained language model disclosed in the above embodiments of the present invention can be applied to, or implemented by, processor 802. Processor 802 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the fine-tuning method for the pre-trained language model can be completed by the integrated logic circuits in the hardware of processor 802 or by instructions in software form. Processor 802 may be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 804. Processor 802 reads the information in memory 804 and, in conjunction with its hardware, completes the steps of the above method.

[0116] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0117] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0118] Specifically, when the computer program is executed by the processor 802, it can also perform the following steps:

[0119] If it is determined that the model parameters of any coding layer need to be updated, the model parameters of any coding layer are updated according to the gradient of the model parameters of any coding layer.

[0120] If it is determined that updating the model parameters of any coding layer is not necessary in this instance, then updating the model parameters of any coding layer is rejected.

[0121] Specifically, when the computer program is executed by the processor 802, it can also perform the following steps:

[0122] The gradient norm of any coding layer is calculated in any of the following ways:

[0123] The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer.

[0124] The gradient norm of any coding layer is obtained by summing the absolute values ​​of each model parameter in any coding layer.

[0125] The maximum absolute value of each model parameter in any given coding layer is determined and used as the gradient norm of that coding layer.

[0126] Specifically, when the computer program is executed by the processor 802, it can also perform the following steps:

[0127] The gradient threshold of each coding layer is calculated according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

[0128] Specifically, the coding layers are numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

[0129] Specifically, the gradient norm is a non-negative value.

[0130] Understandably, in this embodiment of the invention, the computer program executed by the processor 802 can achieve the above-mentioned functions. Figure 2 or Figure 3The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0131] In some embodiments of the present invention, a computer-readable storage medium is also provided, on which a program is stored, which, when executed by a processor, performs the following steps:

[0132] Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model.

[0133] The pre-trained language model is trained, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process.

[0134] Whether the model parameters of any coding layer need to be updated depends on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer.

[0135] When executed by the processor, this program can implement all the methods of fine-tuning the pre-trained language model described above and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0136] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0138] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0140] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0141] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the fine-tuning method for the pre-trained language model described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, disks, or optical discs.

[0142] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for fine-tuning a pre-trained language model, characterized in that, include: Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model. The pre-trained language model is trained using training samples, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process; wherein, the training samples are text data; Whether the model parameters of any coding layer need to be updated is determined based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer; wherein, if the gradient norm of any coding layer is less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer need to be updated; if the gradient norm of any coding layer is not less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer do not need to be updated.

2. The fine-tuning method according to claim 1, characterized in that, Also includes: If it is determined that the model parameters of any coding layer need to be updated, the model parameters of any coding layer are updated according to the gradient of the model parameters of any coding layer. If it is determined that updating the model parameters of any coding layer is not necessary in this instance, then updating the model parameters of any coding layer is rejected.

3. The fine-tuning method according to claim 1, characterized in that, The gradient norm of any coding layer is calculated in any of the following ways: The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer. The gradient norm of any coding layer is obtained by summing the absolute values ​​of the gradients of each model parameter in any coding layer. The maximum absolute value of the gradient of each model parameter of any given coding layer is determined as the gradient norm of that coding layer.

4. The fine-tuning method according to claim 1, characterized in that, Determining the gradient threshold for each encoding layer in the pre-trained language model includes: The gradient threshold of each coding layer is calculated according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

5. The fine-tuning method for a pre-trained language model according to claim 4, characterized in that, The coding layers are numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

6. The fine-tuning method according to any one of claims 1 to 5, characterized in that, The gradient norm is non-negative.

7. A fine-tuning device for a pre-trained language model, characterized in that, include: A determination module is used to obtain a pre-trained language model and determine the gradient threshold of each coding layer in the pre-trained language model. The calculation module is used to train the pre-trained language model using training samples, and to calculate the gradient norm of any encoding layer before updating the model parameters of any encoding layer during the training process; wherein the training samples are text data; The determination module is used to determine whether the model parameters of any coding layer need to be updated in this instance based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer; wherein, if the gradient norm of any coding layer is less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer need to be updated in this instance; if the gradient norm of any coding layer is not less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer do not need to be updated in this instance.

8. The fine-tuning device according to claim 7, characterized in that, Also includes: An update module is configured to update the model parameters of any coding layer according to the gradient of the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer need to be updated; and to refuse to update the model parameters of any coding layer when the determination module determines that the model parameters of any coding layer do not need to be updated.

9. The fine-tuning device according to claim 7, characterized in that, The calculation module is further configured to calculate the gradient norm of any coding layer in any of the following ways: The gradient norm of any coding layer is obtained by calculating the sum of squares of the gradients of each model parameter of any coding layer. The gradient norm of any coding layer is obtained by summing the absolute values ​​of the gradients of each model parameter in any coding layer. The maximum absolute value of the gradient of each model parameter of any given coding layer is determined as the gradient norm of that coding layer.

10. The fine-tuning device according to claim 7, characterized in that, The determining module is further configured to calculate the gradient threshold of each coding layer according to a preset threshold function, wherein the input of the threshold function is the index of the coding layer, and the output of the threshold function is the gradient threshold of the coding layer.

11. The fine-tuning device according to claim 10, characterized in that, The coding layers are numbered sequentially from the input layer to the output layer of the pre-trained language model, and the output of the threshold function is positively correlated with the input of the threshold function.

12. A fine-tuning device for a pre-trained language model, comprising: processor; as well as A memory, in which computer program instructions are stored, When the computer program instructions are executed by the processor, the processor performs the following steps: Obtain a pre-trained language model and determine the gradient threshold of each encoding layer in the pre-trained language model. The pre-trained language model is trained using training samples, and the gradient norm of any encoding layer is calculated before each update of the model parameters of any encoding layer during the training process; wherein, the training samples are text data; Whether the model parameters of any coding layer need to be updated is determined based on whether the gradient norm of any coding layer is less than the gradient threshold of any coding layer; wherein, if the gradient norm of any coding layer is less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer need to be updated; if the gradient norm of any coding layer is not less than the gradient threshold of any coding layer, it is determined that the model parameters of any coding layer do not need to be updated.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the fine-tuning method for the pre-trained language model as described in any one of claims 1 to 6.

Images