Method and device for performing parameter fine adjustment on pre-trained large model

Abstract

A method and apparatus for performing parameter fine tuning on a pre-trained large model, the large model comprising N PEFT modules, the method comprising: obtaining a first training sample corresponding to a downstream task, the first training sample comprising a first text and a first tag; determining N importance scores of the N PEFT modules in the i updating stage according to the large model passing through the (i-1) updating stage; inputting a first text into the large model, and enabling the large model to output a first prediction text; according to the N importance scores of the N PEFT modules in the ith updating stage, determining a target PEFT module to be updated of the N PEFT modules; and updating parameters in the target PEFT module based on the first prediction text and the first tag.

Description

Technical Field

[0001] The embodiments of this specification belong to the field of computer technology, and more particularly, to a method and device for fine-tuning parameters of a pre-trained large model. Background Art

[0002] Usually, a large model that can be used to perform downstream tasks in specific technical scenarios can be obtained by fine-tuning the pre-trained large model. In order to save computing resources and storage resources, there are currently provided parameter-efficient fine tuning (PEFT) methods such as adapter-based methods, reparameterization-based methods, and prompt-based methods for fine-tuning pre-trained large models, aiming to minimize the number of trainable parameters.

[0003] The various PEFT methods in the aforementioned examples all require setting up multiple PEFT modules for the pre-trained large model, and completing the process of fine-tuning the pre-trained large model by updating the parameters of the multiple PEFT modules. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for fine-tuning parameters of a pre-trained large model.

[0005] In a first aspect, a method for fine-tuning parameters of a pre-trained large model is provided, wherein the large model includes N PEFT modules, and the method includes: obtaining a first training sample corresponding to a downstream task, wherein the first training sample includes a first text and a first label; determining N importance scores of the N PEFT modules in the i-th update stage based on the large model after passing the i-1th update stage; inputting the first text into the large model so that the large model outputs a first predicted text; determining a target PEFT module to be updated among the N PEFT modules based on the N importance scores of the N PEFT modules in the i-th update stage; and updating parameters in the target PEFT module based on the first predicted text and the first label.

[0006] In a second aspect, a device for fine-tuning parameters of a pre-trained large model is provided, wherein the large model includes N parameter efficient fine-tuning PEFT modules, and the device includes: a data acquisition unit, used to acquire a first training sample corresponding to a downstream task, wherein the first training sample includes a first text and a first label; an important new metric unit, used to determine N importance scores of the N PEFT modules in the i-th update stage according to the large model after passing the i-1-th update stage; a model calling unit, used to input the first text into the large model so that the large model outputs a first predicted text; a module selection unit, used to determine a target PEFT module to be updated among the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th update stage; and a parameter updating unit, used to update the parameters in the target PEFT module based on the first predicted text and the first label.

[0007] In a third aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores a computer program / instructions, and when the processor executes the computer program / instructions, the method described in the first aspect is implemented.

[0008] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed in a computing device, the computing device executes the method described in the first aspect.

[0009] In the technical solution provided in the embodiments of the present specification, the process of updating the parameters of the N PEFT modules included in the pre-trained large model may include multiple update stages, and each update stage needs to redetermine the N importance scores of the N PEFT modules relative to the downstream tasks based on the large model obtained by updating in the previous update stage; and a single update stage may include one or more rounds of update operations, and in each round of update operations, some target PEFT modules among the N PEFT modules can be dynamically selected and updated based on the N importance scores of the N PEFT modules in the corresponding update stage, without the need to update all the N PEFT modules, which is beneficial to saving computing resources and storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1A schematic diagram of a technical scenario of the technical solution provided in the embodiments of this specification;

[0012] Figure 2 A flowchart of a method for fine-tuning parameters of a pre-trained large model provided in an embodiment of this specification;

[0013] Figure 3 A schematic diagram of a process for obtaining the importance score of a PEFT module provided in an embodiment of this specification;

[0014] Figure 4 A schematic diagram of a device for fine-tuning parameters of a pre-trained large model provided in an embodiment of this specification. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0016] PEFT methods for fine-tuning parameters of pre-trained large models, such as adapter-based methods, re-parameterization-based methods, and prompt-based methods, all require first setting multiple PEFT modules for the pre-trained large model, and then completing the process of fine-tuning parameters of the pre-trained large model by updating the parameters of the multiple PEFT modules, thereby obtaining a target large model that can be used to perform downstream tasks. The pre-trained large model can be various general large language models (LLM), such as GPT (Generative Pre-trained Transformer).

[0017] Figure 1 A schematic diagram of a technical scenario of the technical solution provided in the embodiments of this specification.

[0018] Reference Figure 1As shown, the pre-trained large model may include N network layers, such as network layer 1 to network layer N; when the pre-trained large model needs to be fine-tuned in order to obtain a target large model for performing downstream tasks, the pre-trained large model may first be set with N PEFT modules including PEFT module 1 to PEFT module N. Then, the large model including N network layers and N PEFT modules may be PEFTed according to the training samples corresponding to the downstream tasks; here, in the process of forward propagation of the input text in the training samples of the large model, the output data of any nth network layer is recorded as The output data of the nth PEFT module is recorded as The input data of the n+1th PEFT module is recorded as It may be possible to use a preset operation rule such as addition operation processing and And get.

[0019] The foregoing Figure 1 The technical scenario is exemplary. Depending on the PEFT method used in the specific technical scenario, there may be other relationships between the network layer and the PEFT module in the large model. For example, the number of PEFT modules may be N-2, and the N-2 PEFT modules correspond to the 2nd to N-1th network layers; for example, using The PEFT module may not only be the n+1th PEFT module, but may also include other PEFT modules located after it; for example, It may be processed by other preset operation rules and And get.

[0020] The inventors have found that when updating parameters of multiple PEFT modules, it is usually necessary to update all parameters in the multiple PEFT modules, unnecessarily introduce trainable parameters into all PEFT modules, ignore the differences between different downstream tasks, and limit the performance and role of different PEFT modules in different downstream tasks. In some solutions that do not require full parameter updates for multiple PEFT modules, they are too dependent on human inspiration, and each round of parameter updates for multiple PEFT modules requires the parameters of some PEFT modules to be updated according to the user's specifications or randomly selected.

[0021] The embodiments of this specification provide a method and device for fine-tuning parameters of a pre-trained large model. The process of updating parameters of the N PEFT modules included in the pre-trained large model may include multiple update stages, each of which needs to re-determine the N importance scores of the N PEFT modules relative to the downstream tasks based on the large model updated in the previous update stage; a single update stage may include one or more rounds of update operations, and each round of update operations may dynamically select and update some target PEFT modules among the N PEFT modules based on the N importance scores of the N PEFT modules in the corresponding update stage, without the need to update all the N PEFT modules, which is conducive to saving computing resources and storage resources.

[0022] Combine the following Figure 1 The technical scenario shown exemplarily describes a method for fine-tuning parameters of a pre-trained model.

[0023] Figure 2 This is a flowchart of a method for fine-tuning parameters of a pre-trained large model provided in an embodiment of this specification.

[0024] As mentioned above, a pre-trained large model may contain multiple network layers and N PEFT modules associated with them.

[0025] Reference Figure 2 As shown, the method may include but is not limited to part or all of the following steps S201 to S209.

[0026] Step S201: Acquire a first training sample corresponding to a downstream task, where the first training sample includes a first text and a first label.

[0027] After obtaining the first training sample, the process of fine-tuning the parameters of the pre-trained large model using the first training sample can include multiple update stages. In any i-th update stage, the following step S203 can be first executed to determine the N importance scores of the N PEFT modules in the i-th update stage based on the large model after passing the i-1-th update stage.

[0028] In the process of fine-tuning the parameters of the large model, the N PEFT modules included in the large model need to perform forward propagation on the input text in the training sample. During the forward propagation process, if some of the N PEFT modules have been disturbed, it will affect the result of the forward propagation, thereby affecting the loss value of the large model on the training sample.

[0029] For the case where the PEFT module is disturbed, compared with the case where none of the N PEFT modules are disturbed: if the loss value changes greatly, it means that the importance of the disturbed PEFT module is higher, and if the loss value changes less, it means that the importance of the disturbed PEFT module is smaller. Therefore, the importance score of the selected PEFT module in the i-th update stage can be measured by selectively disturbing the PEFT module and observing the corresponding loss value.

[0030] In one possible implementation, a second sample corresponding to a downstream task and containing a second text and a second label can be obtained; then target operations are performed on the N PEFT modules respectively to obtain model loss values ​​corresponding to each of the N PEFT modules, and the target operations include: interfering with the PEFT module so that during the forward propagation process, the input data of other PEFT modules located after the PEFT module and directly connected to the PEFT module are calculated based on the scaling result of the output data of the PEFT module; the second text is input into the interfered large model so that the large model outputs a second predicted text; based on the second predicted text and the second label, the model loss value corresponding to the PEFT module is determined; and according to the model loss values ​​corresponding to each of the N PEFT modules, the importance scores of the N PEFT modules in the i-th update stage are determined.

[0031] Reference Figure 1 As shown, it is assumed here that the PEFT module located after the nth PEFT module and directly connected to it is the n+1th PEFT module. When interfering with the nth PEFT module, during the propagation of the second text in the second training sample by the large model: the output data of the nth PEFT module can be scaled according to the preset scaling factor β Perform scaling processing to obtain the corresponding scaling result Unlike the nth PEFT module which is not disturbed, the n+1th PEFT module No longer Rather

[0032] Shrinking or amplifying the output data of the nth PEFT module can affect the model loss value of the model on the second training sample. However, when the output data of the nth PEFT module is amplified, the impact on the result of the forward propagation is usually relatively large, and the stability is more uncontrollable. The preferred method is to shrink the output data of the nth PEFT module, that is, to suppress the output data of the nth PEFT module, that is, to set the value of β to a preset value less than 1, for example 0.25.

[0033] In a more specific example, the importance score of the nth PEFT module in the i-th update stage may be positively correlated with the model loss value of the nth PEFT module in the i-th update stage.

[0034] In another more specific example, based on the idea of ​​reinforcement learning, the importance increments of the N PEFT modules can be determined according to their corresponding model loss values ​​in the i-th update stage, and then the importance scores of the N PEFT modules in the i-1 update stage and the importance increments of the N PEFT modules can be determined.

[0035] In another possible implementation, based on the idea of ​​reinforcement learning, the N importance scores of N PEFT modules in the i-th update stage can be determined according to the large model after the i-1th update stage and the N importance scores of the N PEFT modules in the i-1th update stage. The N importance scores of the N PEFT modules in the first update stage can be N pre-set initial values ​​of the same size. When i is greater than 1, the N importance scores corresponding to the N PEFT modules can be reinforced through part or all of the following steps S2011 to S2021 to obtain the N importance scores of the N PEFT modules in the i-th update stage.

[0036] Step S2011: acquiring K sampling results according to the N importance scores of the N PEFT modules in the (i-1)th updating stage, wherein any k-th sampling result indicates the Q first PEFT modules to be interfered.

[0037] The number K of sampling results may be a preset value not less than 1.

[0038] The number Q of the first PEFT modules may be a preset value not less than 1.

[0039] First, N first intervals can be determined based on the N importance scores of the N PEFT modules in the i-1th update stage, and then K rounds of sampling operations can be performed on the N first intervals. Any k-th round of sampling operations includes randomly sampling the N first intervals to obtain N first sampling values, and determining the k-th sampling result based on the N first sampling values.

[0040] For any nth first interval: its lower limit value can be 0, and its upper limit value can be the importance score of the nth PEFT module in the i-1th update stage; or, the N importance scores of the N PEFT modules in the i-1th update stage can be normalized, and the upper limit value can be the normalized processing result corresponding to the nth PEFT module.

[0041] In any k-th round of sampling operation: when randomly sampling the n-th first interval, the sampling action conforms to the uniform distribution in the n-th first interval; after obtaining N first sampling values, Q PEFT modules corresponding to the Q first sampling values ​​can be selected in order from large to small as the Q first PEFT modules to be interfered. In this way, combined with the subsequent importance score update process, it is ensured that the PEFT modules with larger importance scores in the i-1th update stage have a higher probability of being updated in the i-th update stage; while for the PEFT modules with smaller importance scores in the i-1th update stage, their importance scores have a lower probability of being updated in the i-th update stage but may still be updated, which is conducive to better distinguishing the importance differences of different PEFT modules relative to downstream tasks after multiple update stages.

[0042] Alternatively, the K sampling results may be obtained by other means. For example, based on the N importance scores of the N PEFT modules in the i-1th update phase, multiple candidate PEFT modules with relatively large importance scores may be determined from the N PEFT modules, and then K rounds of random sampling may be performed on the determined multiple candidate PEFT modules, and each round of random sampling randomly selects Q first PEFT modules to be interfered from the multiple candidate PEFT modules.

[0043] Step S2013, interfere with the Q first PEFT modules indicated by the kth sampling result, so that during the forward propagation process, the input data of the second PEFT module located after any qth first PEFT module and directly connected to it is calculated based on the scaling result of the output data of the qth first PEFT module.

[0044] Step S2015, inputting the second text in the second training sample corresponding to the downstream task into the large model of the Q first PEFT modules that have been interfered with, so that the large model outputs the third predicted text and removes the interference to the Q first PEFT modules.

[0045] Step S2017, calculating the loss value according to the second label and the third predicted text in the second training sample.

[0046] Reference Figure 1As shown, it is assumed here that the qth first PEFT module indicated by the kth sampling result is the nth PEFT module among the N PEFT modules, and the second PEFT module located after and directly connected to the qth first PEFT module is the n+1th PEFT module among the N PEFT modules. Then, when we interfere with the nth PEFT module, the large model propagates the second text in the second training sample: the output data of the nth PEFT module can be scaled according to the preset scaling factor β Perform scaling processing to obtain the corresponding scaling result Correspondingly, unlike the case where the nth PEFT module is not disturbed, the input data of the n+1th PEFT module Will no longer be Rather

[0047] With reference to the foregoing description, the value of β may be set to a preset value less than 1, such as 0.25.

[0048] By executing the aforementioned steps S2013 and S2017 for the K sampling results, K loss values ​​corresponding to the K sampling results can be obtained. It can be understood that the same second training sample needs to be used in the process of executing the aforementioned step S2015 for the K sampling results respectively. After obtaining the K loss values ​​corresponding to the K sampling results, the following step S2019 can be executed to determine the K reward values ​​corresponding to the K sampling results according to the K loss values ​​corresponding to the K sampling results.

[0049] In a possible implementation, the K loss values ​​may be firstly indexed and reciprocally transformed in turn to obtain K transformation results; then, for any k-th sampling result, the reward value corresponding to the k-th sampling result may be calculated based on the k-th transformation result and the average of the K transformation results. In a more specific example, the reward value corresponding to the k-th sampling result may be calculated by the following formula 1:

[0050]

[0051] Among them, e is a natural constant, r k Represents the reward value corresponding to the k-th sampling result, L k Represents the loss value corresponding to the k-th sampling result, L h Represents the loss value corresponding to the h-th sampling result.

[0052] Or it is also possible to obtain K reward values ​​by other means. For example, the K loss values ​​can be sorted in order from small to large, and the preset reward value can be assigned to the corresponding sampling result according to the arrangement position of the loss value. Specifically, for the loss value arranged in the first place, the reward value of the sampling result corresponding to the loss value can be set to a first preset value less than 1, and the reward value of the sampling result corresponding to the loss value thereafter can be increased according to a predetermined step size, wherein for the loss value arranged at the end, the reward value of the sampling result corresponding to the loss value is the opposite of the first preset value.

[0053] Step S2021, determining the N importance scores of the N PEFT modules in the i-th update stage based on the K sampling results, the K reward values, and the N importance scores of the N PEFT modules in the i-1-th update stage.

[0054] For any nth PEFT module, the corresponding target sampling results can be first determined from the K sampling results, and the target sampling results indicate that the nth PEFT module is the first PEFT module to be disturbed; then, the importance score of the nth PEFT module in the i-1th update stage can be determined based on the importance score of the nth PEFT module in the i-1th update stage and the reward values ​​corresponding to the target sampling results.

[0055] For example, the reward value r corresponding to the kth sampling result can be calculated based on the preset learning rate μ k , calculate the importance increment μ*r corresponding to the k-th sampling result k ; For any nth PEFT module, the importance score of the nth PEFT module in the i-1th update stage and the importance increments corresponding to each target sampling result of the nth PEFT module can be summed to obtain the importance score of the nth PEFT module in the i-1th update stage.

[0056] The updating process of the importance score in the above example means that if a PEFT module is not selected as the first PEFT module to be disturbed in K rounds of sampling results, the importance score of the PEFT module in the i-th update stage will still be the importance score of the PEFT module in the i-1-th update stage and will not be updated.

[0057] The foregoing Figure 3The updating process of the importance scores shown is exemplary, or it is also possible to obtain the N importance scores of the N PEFT modules in the i-th update stage by other means. For example, when some of the N PEFT modules have been disturbed, the loss values ​​of the N PEFT modules on the second training sample are recorded as L1, and when none of the N PEFT modules have been disturbed, the loss values ​​of the N PEFT modules on the second training sample are recorded as L2. The importance increment corresponding to the i-th update stage can be determined according to the difference between the loss value L1 and the loss value L2; for the disturbed PEFT module, its importance score in the i-th update stage can be determined according to its own importance score in the i-1 update stage and the importance increment corresponding to the i-th update stage; for the PEFT module that has not been disturbed, its importance score in the i-th update stage is still the importance score in the i-1 update stage.

[0058] After obtaining the N importance scores of the N PEFT modules in the i-th update stage, the large model can be updated in one or more rounds in the i-th update stage in combination with the first training sample obtained. In the i-th update stage, a single update round performed on the large model may include but is not limited to part or all of the following steps S205 to S209.

[0059] Step S205: input a first text into the large model so that the large model outputs a first predicted text.

[0060] Step S207: determining the target PEFT module to be updated among the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th updating stage.

[0061] N second intervals corresponding to the N PEFT modules can be determined based on the N importance scores of the N PEFT modules in the i-th update stage, and the N second intervals can be randomly sampled to obtain N second sampling values, and one or more target PEFT modules for parameter update can be determined based on the N second sampling values.

[0062] It is assumed here that J rounds of update operations are required for N PEFT modules in the i-th update phase. In any j-th update operation in the J-th update operations, when randomly sampling any n-th second interval, the sampling action is evenly distributed in the n-th second interval. After obtaining N second sampling values, a predetermined number of PEFT modules corresponding to a predetermined number of second sampling values ​​can be selected in order from large to small as the target PEFT modules in the j-th update operation.

[0063] In this way, it is ensured that in the j-th round of update operations, the PEFT modules with larger importance scores have a greater probability of being determined as the target PEFT modules that require parameter update, while the PEFT modules with smaller importance scores still have the possibility of being determined as the target PEFT modules that require parameter update, thereby ensuring a balance between the exploration and utilization of the importance of the N PEFT modules in downstream tasks in the process of continuously updating the importance scores of the N PEFT modules.

[0064] In some embodiments, in any j-th update operation in the i-th phase, a PEFT module with a relatively large importance score can be directly selected as the target PEFT module to be updated. For example, multiple candidate PEFT modules with relatively large importance scores can be determined based on the N importance scores of the N PEFT modules in the i-th update phase, and then the target PEFT module to be updated can be randomly selected from the multiple determined candidate PEFT modules.

[0065] Finally, in step S209, the parameters in the target PEFT module are updated based on the first predicted text and the first label.

[0066] That is, in any j-round update operation in the i-th stage, the gradient information for parameter updating of the target PEFT module determined in the j-round update operation can be calculated based on the first predicted text, the first label and the corresponding loss function, and then the parameters of the target PEFT module determined in the j-round update operation can be updated based on the calculated gradient information.

[0067] Through the aforementioned steps S201 to S209, multiple rounds of updates of the N PEFT modules in the large model are completed in multiple update stages. When the convergence conditions are met, for example, the update rounds reach a preset threshold or the loss value of the large model on the first training sample meets the preset conditions, the parameter fine-tuning of the large model can be terminated. At this time, the large model containing multiple network layers and multiple PEFT modules can be used as the target large model for executing related downstream tasks.

[0068] Based on the same concept as the aforementioned method embodiment, the embodiment of this specification also provides a device 400 for fine-tuning parameters of a pre-trained large model, wherein the large model includes N PEFT modules, and the device 400 includes: a data acquisition unit 401, used to acquire a first training sample corresponding to a downstream task, wherein the first training sample includes a first text and a first label; an important new metric unit 403, used to determine the N importance scores of the N PEFT modules in the i-th update stage according to the large model after passing the i-1th update stage; a model calling unit 405, used to input the first text into the large model so that the large model outputs a first predicted text; a module selection unit 407, used to determine the target PEFT module to be updated of the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th update stage; a parameter updating unit 409, used to update the parameters in the target PEFT module based on the first predicted text and the first label.

[0069] A computer-readable storage medium is also provided in an embodiment of the present specification, on which a computer program / instruction is stored. When the computer program / instruction is executed in a computer, the computer is caused to execute the methods in the aforementioned embodiments.

[0070] A computing device is also provided in an embodiment of the present specification, including a memory and a processor, wherein the memory stores a computer program / instruction, and when the processor executes the computer program / instruction, the method in the aforementioned embodiments is implemented.

[0071] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0072] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0073] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0074] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0075] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0082] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0084] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0085] The above description is only an example of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims.

Claims

1. A method for fine-tuning parameters of a pre-trained large model, wherein the large model includes N parameter efficient fine-tuning PEFT modules, the method comprising: Acquire a first training sample corresponding to a downstream task, wherein the first training sample includes a first text and a first label; Determining N importance scores of the N PEFT modules in the i-th updating stage according to the large model after passing the i-1-th updating stage; Inputting the first text into the large model so that the large model outputs a first predicted text; Determining a target PEFT module to be updated among the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th update phase; Based on the first predicted text and the first label, parameters in the target PEFT module are updated.

2. The method according to claim 1, wherein determining the N importance scores of the N PEFT modules in the i-th updating stage according to the large model after passing the i-1-th updating stage comprises: Acquire a second training sample corresponding to the downstream task, where the second training sample includes a second text and a second label; Performing target operations on the N PEFT modules respectively to obtain model loss values ​​corresponding to the N PEFT modules respectively, wherein the target operations include: Interfering with the PEFT module so that, during the forward propagation process, the input data of other PEFT modules located after the PEFT module and directly connected to the PEFT module are calculated based on the scaling result of the scaling process on the output data of the PEFT module; Inputting the second text into the large model after interference, so that the large model outputs a second predicted text; Determine a model loss value corresponding to the PEFT module based on the second predicted text and the second label; According to the model loss values ​​corresponding to each of the N PEFT modules, the importance scores of the N PEFT modules in the i-th updating stage are determined.

3. The method according to claim 1, wherein determining N importance scores of the N PEFT modules in the i-th updating stage according to the large model after passing the i-1-th updating stage comprises: The N importance scores of the N PEFT modules in the i-th update stage are determined according to the large model after passing the i-1th update stage and the N importance scores of the N PEFT modules in the i-1th update stage.

4. The method according to claim 3, wherein determining the N importance scores of the N PEFT modules in the i-th update stage according to the large model after passing the i-1th update stage and the N importance scores of the N PEFT modules in the i-1th update stage comprises: According to the N importance scores of the N PEFT modules in the (i-1)th update phase, K sampling results are obtained, and any k-th sampling result indicates the Q first PEFT modules to be interfered; Interfering the Q first PEFT modules indicated by the k-th sampling result, so that in the forward propagation process of the large model, the input data of the second PEFT module located after and directly connected to any q-th first PEFT module is calculated based on the scaling result of the scaling processing of the output data of the q-th first PEFT module; Inputting the second text in the second training sample corresponding to the downstream task into the large model in which the Q first PEFT modules have been disturbed, so that the large model outputs a third predicted text; Remove interference to the Q first PEFT modules; Calculating a loss value according to the second label in the second training sample and the third predicted text; Determine K reward values ​​corresponding to the K sampling results according to the K loss values ​​corresponding to the K sampling results; Determine the N importance scores of the N PEFT modules in the i-th update stage according to the K sampling results, the K reward values, and the N importance scores of the N PEFT modules in the i-1-th update stage.

5. The method according to claim 4, wherein obtaining K sampling results according to the N importance scores of the N PEFT modules in the i-1th update phase comprises: Determining N first intervals according to the N importance scores of the N PEFT modules in the i-1th update phase; K rounds of sampling operations are performed on the N first intervals, and any k-th round of sampling operations includes randomly sampling the N first intervals to obtain N first sampling values, and determining a k-th sampling result according to the N first sampling values.

6. The method according to claim 4, wherein determining the K reward values ​​corresponding to the K sampling results according to the K loss values ​​corresponding to the K sampling results comprises: Performing exponential and reciprocal conversion on the K loss values ​​in sequence to obtain K conversion results; For any k-th sampling result, the reward value corresponding to the k-th sampling result is calculated according to the k-th conversion result and the average of the K conversion results.

7. The method according to claim 4, wherein determining the N importance scores of the N PEFT modules in the i-th update stage according to the K sampling results, the K reward values, and the N importance scores of the N PEFT modules in the i-1-th update stage comprises: For any n-th PEFT module, determining respective target sampling results corresponding thereto from the K sampling results, wherein the target sampling results indicate that the n-th PEFT module is the first PEFT module to be interfered with; The importance score of the nth PEFT module in the i-1th update stage is determined according to the importance score of the nth PEFT module in the i-1th update stage and the reward values ​​corresponding to each of the target sampling results.

8. The method according to any one of claims 1 to 7, wherein determining the target PEFT module to be updated of the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th update phase comprises: Randomly sample the N second intervals corresponding to the N PEFT modules to obtain N second sampling values, and determine the target PEFT module for parameter update based on the N second sampling values, and any nth second interval is determined based on the importance score of the nth PEFT module in the i-th update stage.

9. A device for fine-tuning parameters of a pre-trained large model, wherein the large model includes N parameter efficient fine-tuning PEFT modules, and the device includes: A data acquisition unit, configured to acquire a first training sample corresponding to a downstream task, wherein the first training sample includes a first text and a first label; An important new metric unit, used to determine N importance scores of the N PEFT modules in the i-th updating stage according to the large model after passing the i-1-th updating stage; A model calling unit, used for inputting the first text into the large model so that the large model outputs a first predicted text; a module selection unit, configured to determine a target PEFT module to be updated of the N PEFT modules according to the N importance scores of the N PEFT modules in the i-th update phase; A parameter updating unit is used to update the parameters in the target PEFT module based on the first predicted text and the first label.

10. A computing device, comprising a memory and a processor, wherein the memory stores a computer program / instruction, and when the processor executes the computer program / instruction, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program / instruction stored thereon, wherein when the computer program / instruction is executed in a computing device, the computing device executes the method according to any one of claims 1 to 8.

Images