Large model fine-tuning method, device, equipment, medium and program product
By reparameterizing the self-attention layer of the large model and introducing low-rank adapters, combined with fine-tuning of the linear layer and attention mechanism, the problems of slow fine-tuning speed and high memory requirements of the large model are solved, and efficient adaptation of the large model is achieved Ability to domain-specific tasks.
Patent Information
- Application Number
- CN202411141783.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The prior art is difficult to fine-tune large models comprehensively, quickly and efficiently, especially with challenges in slow training speed and high memory requirements.
By constructing a re-parameterization factor for the self-attention layer in the large model, combining static multi-head self-attention weights and low-rank adapter parameters, a fine-tuning module is built, and parameter adjustments are performed on the training data, combining the re-parameterization method of the linear layer. and fine-tuning of attention mechanism to optimize the representation of self-attention map.
It realizes comprehensive and efficient fine-tuning of large models, improves processing efficiency, and can optimize resource usage without sacrificing model performance and adapt to task requirements in specific fields.
Smart Images

Figure CN118673999B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large model fine-tuning method, device, equipment, medium and program product. Background Art
[0002] Large language and graphic models are trained on large amounts of text and image data obtained from the Internet. These data need to go through a careful cleaning process before training to ensure the quality of the data. Despite this, the performance of large models in certain vertical fields or specific scenarios is often unsatisfactory. To address this problem, it is usually necessary to further fine-tune the model to adapt to specific application scenarios. Fine-tuning a large model with a large number of parameters faces multiple challenges, including slow training speed and excessive training memory requirements. In response to these challenges, academia and industry are looking for ways to fine-tune large models comprehensively, quickly and efficiently. Summary of the invention
[0003] The present application provides a large model fine-tuning method, device, equipment, medium and program product to solve the defect in the prior art that it is difficult to comprehensively and efficiently perform large model fine-tuning.
[0004] In a first aspect, the present application provides a large model fine-tuning method, comprising:
[0005] For each self-attention layer in the large model, a reparameterization factor is constructed; the number of the reparameterization factors is multiple; the reparameterization factor is a multi-head weight;
[0006] Based on the re-parameterization factor, construct static multi-head self-attention weights;
[0007] Determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0008] Determining a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices;
[0009] Based on the first fine-tuning module and the second fine-tuning module, some parameters of the large model are adjusted on the training data.
[0010] Optionally, determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map includes:
[0011] Initializing the static multi-head self-attention weights by a random initialization method to obtain initialized static multi-head self-attention weights;
[0012] Truncating the initialized static multi-head self-attention weight to obtain a target weight;
[0013] Based on the target weight and the initial self-attention map, determine the first fine-tuning module.
[0014] Optionally, the target weight includes a first weight and a second weight;
[0015] The determining the first fine-tuning module based on the target weight and the initial self-attention map includes:
[0016] Performing an inner product operation on the first weight and the second weight to obtain a target self-attention map;
[0017] Perform an inner product operation on the target self-attention map and the value vector of the initial self-attention map to obtain an operation result;
[0018] The first fine-tuning module is determined based on the sum of the operation result and the initial self-attention map.
[0019] Optionally, the method further comprises:
[0020] Filter data sources based on fine-tuning goals;
[0021] Extracting data based on the data source and performing data cleaning to obtain cleaned data;
[0022] Performing data enhancement on the cleaned data to obtain augmented data;
[0023] The amplified data is annotated to obtain the training data.
[0024] Optionally, the number of the low-rank matrices is 2;
[0025] The determining of the second fine-tuning module based on the target feed-forward neural network layer and the low-rank adapter parameters comprises:
[0026] Multiply the two low-rank matrices to get the parameter space;
[0027] The second fine-tuning module is determined based on the parameter space and the weight matrix of the target feed-forward neural network layer.
[0028] Optionally, adjusting some parameters of the large model on the training data based on the first fine-tuning module and the second fine-tuning module includes:
[0029] The original parameters of the large model are fixed, and based on the first fine-tuning module and the second fine-tuning module, the re-parameterization factor and the low-rank adapter parameters are gradient updated by back propagation on the training data to obtain the fine-tuned large model.
[0030] In a second aspect, the present application also provides a large model fine-tuning device, comprising:
[0031] The first construction module is used to construct a reparameterization factor for each self-attention layer in the large model; the number of the reparameterization factors is multiple; and the reparameterization factor is a multi-head weight;
[0032] A second building block is used to construct static multi-head self-attention weights based on the re-parameterization factor;
[0033] A first determination module, configured to determine a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0034] A second determination module is used to determine a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include multiple low-rank matrices;
[0035] A fine-tuning module is used to adjust some parameters of the large model on the training data based on the first fine-tuning module and the second fine-tuning module.
[0036] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.
[0037] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.
[0038] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.
[0039] The large model fine-tuning method, device, equipment, medium and program product provided in the present application can combine the reparameterization of the linear layer and the fine-tuning of the attention mechanism by reparameterizing the self-attention map and introducing a low-rank adapter in the feedforward neural network layer, and then fine-tune the large model based on two fine-tuning modules, thereby optimizing the representation of the self-attention map, improving processing efficiency, and being able to comprehensively and efficiently fine-tune the large model. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0041] Figure 1 It is a flowchart of the large model fine-tuning method provided in this application;
[0042] Figure 2 is a schematic diagram of the reparameterization of the self-attention map provided in this application;
[0043] Figure 3 is a schematic diagram of a low-rank adapter provided by this application;
[0044] Figure 4 It is a structural schematic diagram of a large model fine-tuning device provided by the present application;
[0045] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0047] Figure 1 Schematic diagram of the process of fine-tuning a large model provided in the embodiment of the present application. Figure 1 The embodiment of the present application provides a large model fine-tuning method, the execution subject of which may be an electronic device, for example, a controller, and the method may include:
[0048] Step 110: for each self-attention layer in the large model, construct a reparameterization factor; the number of reparameterization factors is multiple; the reparameterization factor is a multi-head weight;
[0049] Step 120: construct static multi-head self-attention weights based on the re-parameterization factor;
[0050] Step 130: Determine a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0051] Step 140: determining a second fine-tuning module based on the target feedforward neural network layer and the low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices;
[0052] Step 150: Based on the first fine-tuning module and the second fine-tuning module, adjust some parameters of the large model on the training data.
[0053] Currently, research on efficient fine-tuning of large models mainly focuses on how to adjust only a small number of parameters to achieve efficient fine-tuning (PEFT). Low-Rank Adaptation (LoRA) technology can freeze most model parameters and introduce only some learnable low-rank parameters, thereby speeding up fine-tuning while maintaining low memory consumption. LoRA mainly reparameterizes the linear layers in the model, but lacks corresponding learning and fine-tuning for the attention mechanism in the processing model.
[0054] Based on this, this application proposes an efficient fine-tuning technology based on attention mechanism re-parameterization. This technology absorbs and learns knowledge in a specific field by adding some learnable parameters to the attention map generated by the attention mechanism. At the same time, combined with LoRA's re-parameterization of the linear layer, it is possible to achieve efficient re-parameterization of the entire Transformer unit. This method not only covers the linear layer, but also takes the fine-tuning of the attention mechanism into consideration, thereby constructing a comprehensive and efficient large model fine-tuning module.
[0055] The goal of reparameterization is to make the representation of the self-attention map and the original self-attention mechanism efficiently combined. For each input sequence group, the controller can be adjusted by introducing two low-rank factors and Reparameterization is performed, where and The dimension of is carefully designed to match the dimension of the self-attention map, thereby enhancing the self-attention mechanism rather than simply replacing it. It is worth noting that although each sequence group is processed independently, and The learning,model is able to capture and leverage shared domain knowledge across groups.
[0056] In step 110, the controller constructs a reparameterization factor for each self-attention layer in the large model. It can be understood that the number of reparameterization factors is multiple. Specifically, the reparameterization factor can be a multi-head weight.
[0057] In step 120, the controller may construct static multi-head self-attention weights based on the reparameterization factors to achieve a low-rank reparameterized representation of the initial self-attention map.
[0058] In step 130, the controller may superimpose the product of the static multi-head self-attention weights onto the initial self-attention map to obtain a first fine-tuning module.
[0059] In step 140, the controller further improves the adaptability and processing efficiency of the model to specific domain data by introducing a low-rank adapter on the feedforward neural network (FNN) layer. For each linear transformation in the FFN layer, LoRA is implemented by adding two low-rank matrices.
[0060] Step 150 involves fine-tuning the large model and its implementation in a specific application domain, with a core focus on two main fine-tuning parameters: the reparameterization factor and the low-rank adapter parameters of the feedforward neural network layer. The goal of this stage is to optimize the above two main fine-tuning parameters through a precise training process to significantly improve the performance of the model in a specific domain. Before the fine-tuning process begins, the controller first loads the pre-trained model and initializes the newly introduced parameters. During the fine-tuning training, the core parameters of the large model remain unchanged, and only the self-attention map reparameterization factor and the low-rank adapter parameters of the FFN layer are adjusted.
[0061] The large model fine-tuning method provided in the embodiment of the present application can combine the reparameterization of the linear layer and the fine-tuning of the attention mechanism, optimize the representation of the self-attention map, improve the processing efficiency, and be able to comprehensively and efficiently fine-tune the large model by reparameterizing the self-attention map and introducing a low-rank adapter in the feedforward neural network layer, and then fine-tuning the large model based on two fine-tuning modules.
[0062] In one embodiment, based on the static multi-head self-attention weights and the initial self-attention map, determining a first fine-tuning module includes:
[0063] Initializing the static multi-head self-attention weights by a random initialization method to obtain initialized static multi-head self-attention weights;
[0064] Truncating the initialized static multi-head self-attention weight to obtain a target weight;
[0065] Based on the target weight and the initial self-attention map, determine the first fine-tuning module.
[0066] Figure 2 is a schematic diagram of the re-parameterization of the self-attention map provided in this application. Figure 2As shown in Figure 1, the reparameterization of the self-attention map is to learn the domain knowledge by introducing a small number of learnable parameters on the attention map. The reparameterization of the self-attention map can be achieved by following the following steps:
[0067] (1) Introducing low-rank factors. By introducing reparameterization factors, for example, two multi-head low-rank weights and ( Shape , Shape ,in is the number of heads, is the maximum tag sequence length, is a low-rank dimension), constructs static multi-head self-attention weights to achieve a low-rank re-parameterized representation of the initial self-attention map.
[0068] (2) Parameter training and truncation. In actual training, in addition to and The main parameters of the network will be frozen except that it remains in a trainable state. The static multi-head self-attention weights are initialized by random initialization to obtain the initialized static multi-head self-attention weights. In order to adapt to the actual number of tags, the initialized static multi-head self-attention weights need to be appropriately truncated to adapt to the length of the input tag to obtain the target weight, that is, the truncated and .
[0069] (3) Generate a self-attention map and perform weighting. and With the initial self-attention map, the first fine-tuning module is determined.
[0070] The large model fine-tuning method provided in the embodiment of the present application re-parameterizes the self-attention map by introducing low-rank weights that match the dimensions in the initial self-attention map, thereby enhancing the self-attention mechanism rather than simply replacing it. This step not only optimizes the representation of the self-attention map and improves processing efficiency, but also makes up for the potential limitations of the original self-attention on specific tasks, and effectively simplifies and optimizes the complex model structure, thereby achieving optimal resource usage without sacrificing model performance.
[0071] In one embodiment, the target weight includes a first weight and a second weight;
[0072] Based on the target weight and the initial self-attention map, the first fine-tuning module is determined, including:
[0073] Perform an inner product operation on the first weight and the second weight to obtain the target self-attention map;
[0074] Perform an inner product operation on the target self-attention map and the value vector of the initial self-attention map to obtain the operation result;
[0075] Based on the sum of the operation result and the initial self-attention map, the first fine-tuning module is determined.
[0076] like Figure 2 As shown, the target weight is the truncated low-rank weight and . By truncating and Perform an inner product operation to generate a new self-attention map, namely the target self-attention map. This map is the same as the value vector of the initial self-attention map. An inner product operation is performed and then the result is added to the result of the initial self-attention map to produce the final output, thereby determining the first fine-tuning module.
[0077] The specific implementation is as follows:
[0078]
[0079] in, represents the output of the multi-head attention mechanism layer, represents the initial self-attention score, represents the value vector of the initial self-attention map, , represents the long-term low-rank weight, where is the number of heads, is the maximum sequence length, is a low-rank dimension, , represents the truncation function, which reduces the sequence length Truncated to actual number of markers .
[0080] parameter and All need to be truncated, and then the inner product is multiplied to get (in, is the number of heads, is the actual number of tags), and then the value vector An inner product operation is performed and further added to the original self-attention result to obtain the output.
[0081] The large model fine-tuning method provided in the embodiment of the present application re-parameterizes the self-attention map by introducing low-rank weights that match the dimensions in the initial self-attention map, thereby enhancing the self-attention mechanism rather than simply replacing it. This step not only optimizes the representation of the self-attention map and improves processing efficiency, but also makes up for the potential limitations of the original self-attention on specific tasks, and effectively simplifies and optimizes the complex model structure, thereby achieving optimal resource usage without sacrificing model performance.
[0082] In one embodiment, the large model fine-tuning method further includes:
[0083] Filter data sources based on fine-tuning goals;
[0084] Extract data based on the data source and clean the data to obtain cleaned data;
[0085] Perform data enhancement on the cleaned data to obtain augmented data;
[0086] The amplified data is labeled to obtain training data.
[0087] This application trains low-rank weights by constructing training data. The construction of training data mainly includes: data source identification and selection: First, identify and select appropriate data sources according to the fine-tuning goals. These data sources can be public data sets, internally collected data, or online content obtained through web crawler technology. For embodied intelligent applications, the data source should cover language or image information that is closely related to the application scenario. Data cleaning: After extracting data from the selected data source, perform data cleaning to remove irrelevant information, duplicate data, and noise. Data enhancement: In order to improve the generalization ability of the model, data enhancement technology can be used to generate more training data. Data annotation: For fine-tuning tasks that require supervised learning, the extracted data needs to be annotated. Data partitioning: Divide the preprocessed data into training sets, validation sets, and test sets for subsequent training.
[0088] The large model fine-tuning method provided in the embodiment of the present application constructs training set data for the re-parameterization process of the self-attention map through steps such as screening data sources, data cleaning, data enhancement, and data labeling, thereby providing data preparation for large model fine-tuning and ensuring comprehensive and efficient large model fine-tuning.
[0089] In one embodiment, the number of low-rank matrices is 2;
[0090] Based on the target feedforward neural network layer and the low-rank adapter parameters, a second fine-tuning module is determined, including:
[0091] Multiply the two low-rank matrices to get the parameter space;
[0092] A second fine-tuning module is determined based on the parameter space and the weight matrix of the target feed-forward neural network layer.
[0093] Figure 3 is a schematic diagram of a low-rank adapter provided by this application. Figure 3 As shown in Figure 1, to further improve the model's adaptability and processing efficiency for data in specific fields, the controller can introduce a low-rank adapter on the FFN layer. The low-rank adapter can be implemented as follows:
[0094] (1) Select the FFN layer as the target. It is decided to introduce a low-rank adapter on the model feedforward neural network layer to enhance the model's adaptability to specific domains.
[0095] (2) Introducing low-rank matrices. For each linear transformation in the FFN layer, two low-rank matrices are added and To implement the low-rank adaptation technique, the dimensions of these two matrices are chosen to be low-rank, meaning that their product has fewer parameters, thus reducing the increase in additional computation.
[0096] (3) Form an enhanced weight representation. and The product of creates an additional, learnable parameter space. This new parameter space is consistent with the original weight matrix of the FFN layer Combined, an enhanced weighted representation is formed.
[0097] The specific implementation of the above low-rank adapter is as follows:
[0098]
[0099] in, represents the enhanced weight representation, represents the weight of the original FFN, Represents the input of the current FFN layer, represents the newly added parameters of the low-rank adapter, , in, is the first dimension of the original weights, is a low-rank dimension, ,in, is a low-rank dimension, is the first dimension of the input, .
[0100] The large model fine-tuning method provided in the embodiment of the present application further improves the adaptability and processing efficiency of the model to data in specific fields while maintaining a balance between the number of parameters and computational complexity by introducing a low-rank adapter on the target feedforward neural network layer.
[0101] In one embodiment, based on the first fine-tuning module and the second fine-tuning module, adjusting some parameters of the large model on the training data includes:
[0102] The original parameters of the large model are fixed, and based on the first fine-tuning module and the second fine-tuning module, the re-parameterization factors and the low-rank adapter parameters are gradient updated by back propagation on the training data to obtain the fine-tuned large model.
[0103] After the self-attention map reparameterization and low-rank adapter are performed, fine-tuning training is required to ensure that the large model can still maintain superior performance in practical applications. This process aims to make the model well applicable to downstream tasks by making minor adjustments to the introduced parameters. In fine-tuning training, the controller first fixes the original parameters of the large model, and then performs gradient updates on the parameters of the self-attention map reparameterization and low-rank adapter through back-propagation to achieve fine-tuning of the model.
[0104] For simplicity, the cross entropy loss function is used to evaluate the accuracy of classification, and the calculation target is as follows:
[0105]
[0106] in, represents the cross entropy loss function, Indicates the total number of data. Indicates the total number of categories that need to be classified. represents the true label, The pseudo-labels that represent the model's classification can be optimized via gradient descent.
[0107] The large model fine-tuning method provided in the embodiment of the present application adjusts the introduced parameters (reparameterization factor and low-rank adapter parameters) so that the large model can better adapt to downstream tasks, which helps to improve the performance of the model in actual tasks. This method not only ensures the efficiency of training, but also helps to prevent overfitting, while maintaining the overall structure of the model, and improving the accuracy and generalization ability of the model on the target task.
[0108] The large model fine-tuning device provided in the present application is described below. The large model fine-tuning device described below and the large model fine-tuning method described above can be referenced to each other.
[0109] Figure 4 Schematic diagram of the structure of the large model fine-tuning device provided in the embodiment of the present application. Figure 4 The large model fine-tuning device provided in the embodiment of the present application may include:
[0110] The first construction module 410 is used to construct a re-parameterization factor for each self-attention layer in the large model; the number of the re-parameterization factors is multiple; and the re-parameterization factor is a multi-head weight;
[0111] A second construction module 420 is used to construct static multi-head self-attention weights based on the re-parameterization factor;
[0112] A first determination module 430, configured to determine a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0113] A second determination module 440 is used to determine a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include multiple low-rank matrices;
[0114] The fine-tuning module 450 is used to adjust some parameters of the large model on the training data based on the first fine-tuning module and the second fine-tuning module.
[0115] The large model fine-tuning device provided in the embodiment of the present application can combine the reparameterization method of the linear layer and the fine-tuning of the attention mechanism by reparameterizing the self-attention map, introducing a low-rank adapter in the feedforward neural network layer, and then fine-tuning the large model based on two fine-tuning modules, thereby optimizing the representation of the self-attention map, improving processing efficiency, and being able to comprehensively and efficiently fine-tune the large model.
[0116] Specifically, the above-mentioned large model fine-tuning device provided in the embodiment of the present application can implement all the method steps implemented by the method embodiment in which the execution subject is the controller, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0117] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communications interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the large model fine-tuning method, for example, including:
[0118] For each self-attention layer in the large model, a reparameterization factor is constructed; the number of the reparameterization factors is multiple; the reparameterization factor is a multi-head weight;
[0119] Based on the re-parameterization factor, construct static multi-head self-attention weights;
[0120] Determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0121] Determining a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices;
[0122] Based on the first fine-tuning module and the second fine-tuning module, some parameters of the large model are adjusted on the training data.
[0123] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0124] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the steps of the large model fine-tuning method provided by the above methods, for example, including:
[0125] For each self-attention layer in the large model, a reparameterization factor is constructed; the number of the reparameterization factors is multiple; the reparameterization factor is a multi-head weight;
[0126] Based on the re-parameterization factor, construct static multi-head self-attention weights;
[0127] Determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0128] Determining a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices;
[0129] Based on the first fine-tuning module and the second fine-tuning module, some parameters of the large model are adjusted on the training data.
[0130] In another aspect, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the steps of the large model fine-tuning method provided by the above methods, for example, including:
[0131] For each self-attention layer in the large model, a reparameterization factor is constructed; the number of the reparameterization factors is multiple; the reparameterization factor is a multi-head weight;
[0132] Based on the re-parameterization factor, construct static multi-head self-attention weights;
[0133] Determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map;
[0134] Determining a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices;
[0135] Based on the first fine-tuning module and the second fine-tuning module, some parameters of the large model are adjusted on the training data.
[0136] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.
[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0138] It should also be noted that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.
[0139] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0140] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.
[0141] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.
[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A large model fine-tuning method, It is characterized in that include: For each self-attention layer in the large model, construct a reparameterization factor; The number of the reparameterization factors is multiple; the reparameterization factors are multi-head weights; Based on the re-parameterization factor, construct static multi-head self-attention weights; Determining a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map; Determining a second fine-tuning module based on the target feedforward neural network layer and low-rank adapter parameters; the low-rank adapter parameters include a plurality of low-rank matrices; Based on the first fine-tuning module and the second fine-tuning module, some parameters of the large model are adjusted on the training data, wherein the training data is obtained by screening a data source, and the data source includes language or image information.
2. The large model fine-tuning method according to claim 1, It is characterized in that The determining of a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map includes: Initializing the static multi-head self-attention weights by a random initialization method to obtain initialized static multi-head self-attention weights; Truncating the initialized static multi-head self-attention weight to obtain a target weight; Based on the target weight and the initial self-attention map, determine the first fine-tuning module.
3. The large model fine-tuning method according to claim 2, It is characterized in that The target weight includes a first weight and a second weight; The determining the first fine-tuning module based on the target weight and the initial self-attention map includes: Performing an inner product operation on the first weight and the second weight to obtain a target self-attention map; Perform an inner product operation on the target self-attention map and the value vector of the initial self-attention map to obtain an operation result; The first fine-tuning module is determined based on the sum of the operation result and the initial self-attention map.
4. The large model fine-tuning method according to claim 1, It is characterized in that The method further comprises: Filter data sources based on fine-tuning goals; Extracting data based on the data source and performing data cleaning to obtain cleaned data; Performing data enhancement on the cleaned data to obtain augmented data; The amplified data is annotated to obtain the training data.
5. The large model fine-tuning method according to claim 1, It is characterized in that The number of the low-rank matrices is 2; The determining of the second fine-tuning module based on the target feed-forward neural network layer and the low-rank adapter parameters comprises: Multiply the two low-rank matrices to get the parameter space; The second fine-tuning module is determined based on the parameter space and the weight matrix of the target feed-forward neural network layer.
6. The large model fine-tuning method according to claim 1, It is characterized in that The adjusting of some parameters of the large model on the training data based on the first fine-tuning module and the second fine-tuning module includes: The original parameters of the large model are fixed, and based on the first fine-tuning module and the second fine-tuning module, the re-parameterization factor and the low-rank adapter parameters are gradient updated by back propagation on the training data to obtain the fine-tuned large model.
7. A large model fine-tuning device, It is characterized in that include: The first building block is used to construct a reparameterization factor for each self-attention layer in the large model; The number of the reparameterization factors is multiple; the reparameterization factors are multi-head weights; A second building block is used to construct static multi-head self-attention weights based on the re-parameterization factor; A first determination module, configured to determine a first fine-tuning module based on the static multi-head self-attention weights and the initial self-attention map; A second determination module, configured to determine a second fine-tuning module based on the target feed-forward neural network layer and the low-rank adapter parameters; The low-rank adapter parameters include a plurality of low-rank matrices; A fine-tuning module is used to adjust some parameters of the large model on training data based on the first fine-tuning module and the second fine-tuning module, wherein the training data is obtained by screening a data source, and the data source includes language or image information.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the large model fine-tuning method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the large model fine-tuning method according to any one of claims 1 to 6 is implemented.
10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the large model fine-tuning method according to any one of claims 1 to 6 is implemented.
Citation Information
Cited By
Basic model neuron granularity adaptation method for edge side data drift
CN119578472A
An edge-oriented side data drift-based basic model neuron granularity adaptation method
CN119578472B
