Efficient large language model fine tuning method and system, electronic equipment and readable storage medium
By dividing the input data into multiple copies for full connection and convolutional fusion, local and global information features are generated, and the dimensions are expanded to the consistent base model, the problem of low fine-tuning accuracy of large language models is solved, and efficient information feature fusion and Improved accuracy.
Patent Information
- Application Number
- CN202411310320.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-14
AI Technical Summary
The existing fine-tuning methods of large language models have low accuracy in specific tasks, resulting in the inability to effectively retain information features and affect the use effect.
By dividing the input data into multiple copies for full connection and convolutional fusion, local and global information features are generated, and the convolution kernel is used to expand the dimension to the base model consistent, efficient fusion of information features is achieved.
While reducing the amount of parameters, the model fine-tuning accuracy is significantly improved, ensuring the retention of information features and improving the use effect.
Smart Images
Figure CN119312862A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an efficient large language model fine-tuning method, system, electronic device and readable storage medium. Background Art
[0002] Against the backdrop of the continuous development of the social economy, information technology has also developed rapidly, and natural language technology has also been applied in this context. In recent years, large language models have been effectively applied. However, pre-trained large language models are usually trained on a large amount of general text data. Although they have mastered a wealth of language knowledge and structure, this knowledge is general and may not be directly applicable to specific tasks. Through fine-tuning, the model can perform better on specific tasks (such as sentiment analysis, question-answering systems, text generation, etc.). For example, the BERT model can be fine-tuned to optimize its performance in sentiment classification tasks, enabling it to more accurately predict the emotional tendency of the text. Therefore, fine-tuning of large models has become extremely important in vertical fields. The following is an example of the lora fine-tuning method:
[0003] In the prior art, the implementation steps are:
[0004] For the large model, the total weight W O ∈R n×m , no longer perform full parameter fine-tuning training, add residuals to the weights, and complete the optimization process by training ΔW:
[0005] W′=W O +ΔW
[0006] Where ΔW=AB,A∈R n×r ,B∈N r×m ,r<<n, r is the rank (Rank, lora_dim) of the parameter matrix ΔW. ΔW is composed of the product of two low-rank matrices through low-rank decomposition. Generally speaking, although the number of parameters of the base model is large, the intrinsic dimension (Intrinsic Dimension) corresponding to each subdivision task is not large, so setting a smaller value can obtain a lora model with a much smaller number of parameters than the base model, and achieve better results in some subdivision tasks. The structure of the lora model is shown below:
[0007]
[0008] When the LoRa model is used, r is often set very small to reduce the amount of parameter calculation. R is often set to 8, which causes the input to be encoded and compressed very small, resulting in low fine-tuning accuracy and failure to retain information features well, affecting the use effect. Summary of the invention
[0009] In order to solve the technical problems raised in the above background technology, an efficient large language model fine-tuning method is provided, which can ensure the fine-tuning accuracy of the model and thus improve the use effect.
[0010] In order to achieve the above object, the technical solution adopted by the present invention is:
[0011] An efficient large language model fine-tuning method includes the following steps:
[0012] S1, divide the input data into n equal parts to obtain n pieces of data, each piece of data has a dimension of v, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features;
[0013] S2, respectively fuse the n local information features and use the activation function to activate the output, so that the n local information features generate n global information features respectively;
[0014] S3, respectively splicing the global information features to the corresponding local information features to obtain n spliced information features, and obtaining a corresponding convolution kernel K according to the dimension of each spliced information feature;
[0015] S4, the n global information features are fused with the corresponding convolution kernel K by convolution, and the activation function is used to activate the output to obtain the fused information features;
[0016] S5, taking the fused information features outputted from step S4 as input data, and then repeating steps S1-S4, and in the process of calculating the convolution kernel K in step S3, expanding the feature dimension to a dimension consistent with the base model.
[0017] Furthermore, in step S1, assuming that the input data is x, the input data x is divided into n equal parts, that is:
[0018] x=[x 1 ,x 2 ,x 3 ,.....,x n ],x i ∈R v
[0019] where x i ∈R v Represents the i-th piece of data; R v Represents the set after the input data is divided into n equal parts;
[0020] Perform full connection inside each piece of data, and use the following activation function to activate the output to obtain the local information feature y ai :
[0021] y ai =σ(Wi x i +b i ),W i ∈R v′×v ,b i ∈R v′ ,y ai ∈R v′
[0022] Among them, σ(·) is the activation function; W i is the weight of local information fusion, b i is the weight offset value of local information fusion, R v ' is the set of local information fusion weight offset values; R v ' ×v is the set of weights for local information fusion.
[0023] The final output data is obtained, that is, the output local information feature is y a :
[0024] y a =[y a1 ,y a2 ,y a3 ,.....,y an ],y i ∈R n×v′ ,y a The same dimension as x, R n×v' It is the feature set output after local information fusion.
[0025] Furthermore, in step S2, the following formula is used to fully connect and fuse the n local information features and activate the output to obtain n global information features y ci :
[0026] y ci =σ(W i x i +b i ),W i ∈R 1×v ,b i ∈R 1 ,y ci ∈R 1 ,
[0027] Where σ(·) is the activation function, W i is the weight of global information fusion, b i is the weight offset value of global information fusion, R 1 is the set of features output by global feature fusion; R 1×v is the set of weights for global information fusion.
[0028] Further, in step S3, each splicing information feature is set to y di ,but:
[0029] y di =[x 1 ,x 2 ,......x n ,y c1 ,y c2 ,......y cn ],x i ∈R v ,y ci ∈R 1 ;
[0030] The convolution kernel K is obtained according to the dimension of the obtained splicing information features:
[0031] K=(v+n)*1*n*v.
[0032] Further, in step S4, the global information feature y is converted into c The corresponding convolution kernel K is fused using convolution and activated output to obtain the fused information feature z:
[0033] z=σ(Conv(y c ,K)), where σ(·) is the activation function.
[0034] Furthermore, in step S5, when executing step S3, the convolution kernel K′ is obtained by using the following formula, so as to expand the feature dimension to a dimension consistent with the base model:
[0035] The convolution formula used is as follows:
[0036] K′=(v+n)*1*n*v″, n*v″ is equal to the output dimension of the base model;
[0037] When executing step S4, the global information feature and the convolution kernel K′ are fused using convolution, and the output is activated to obtain the final output information feature z′.
[0038] Furthermore, the present invention provides an efficient large language model fine-tuning system, comprising:
[0039] The local information fusion module is used to divide the input data into n equal parts to obtain n pieces of data, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features;
[0040] A global information feature generation module, which is used to fuse n local information features respectively, so that the n local information features respectively generate n global information features;
[0041] A splicing information feature generation module, which is used to splice n global information features to corresponding local information features respectively to obtain n splicing information features, and obtain a corresponding convolution kernel K according to the dimension of each splicing information feature;
[0042] The local information and global information fusion module is used to fuse the n global information features with the corresponding convolution kernel K using convolution, and activate the output using the activation function;
[0043] The dimension expansion module is used to expand the feature dimension to the same dimension as the base model during the calculation of the convolution kernel K.
[0044] The present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described above when executing the computer program.
[0045] The present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the methods described are executed.
[0046] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:
[0047] The above-mentioned efficient large language model fine-tuning method first uses full connection to fuse local information to obtain global information features, and then uses convolution to fuse the global information. It greatly reduces the number of parameters without compressing the data, ensuring its fine-tuning accuracy, and can well retain information features while reducing the weight. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A flowchart of an efficient large language model fine-tuning method according to a preferred embodiment of the present invention;
[0049] Figure 2 The module diagram of an efficient large language model fine-tuning system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a component centered. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a component centered. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a component centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.
[0053] See also Figure 1 The first embodiment of the present invention provides an efficient large language model fine-tuning method, comprising the following steps:
[0054] S1, divide the input data into n equal parts to obtain n pieces of data, each piece of data has a dimension of v, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features.
[0055] Specifically, let the input data be x, and divide the input data x into n equal parts, that is:
[0056] x=[x 1 ,x 2 ,x 3 ,.....,x n ],x i ∈R v ;
[0057] where x i ∈R v represents the i-th input data; R v Represents the set after the input data is divided into n equal parts;
[0058] Perform full connection inside each piece of data, and use the following activation function to activate the output to obtain the local information feature y ai :
[0059] y ai =σ(W i x i +b i ),W i ∈R v′×v ,b i∈R v′ ,y ai ∈R v′
[0060] Among them, σ(·) is the activation function; W i is the weight of local information fusion, b i is the weight offset value of local information fusion, R v ' is the set of local information fusion weight offset values; R v ' ×v is the set of weights for local information fusion.
[0061] The final output data is obtained, that is, the output local information feature is y a :
[0062] y a =[y a1 ,y a2 ,y a3 ,.....,y an ],y i ∈R n×v′ , R n×v' is the feature set output after local information fusion; y a Same dimension as x.
[0063] In this embodiment, full connection is performed inside the local information features, which can make the local information fusion tighter and ensure that the dimension of the output data is the same as the dimension of the input data.
[0064] S2, respectively fuse the n local information features and use the activation function to activate the output, so that the n local information features generate n global information features respectively.
[0065] In this embodiment, in step S2, the following formula is used to fully connect and fuse the n local information features output in step S1 and activate them with an activation function to obtain n global information features y ci :
[0066] y ci =σ(W i x i +b i ),W i ∈R 1×v ,b i ∈R 1 ,y ci ∈R 1 ,
[0067] Where σ(·) is the activation function, W i is the weight of global information fusion, b i is the weight offset value of global information fusion, R1 is the set of features output by global feature fusion; R 1×v is the set of weights for global information fusion.
[0068] S3, respectively splicing the global information features to the corresponding local information features to obtain n spliced information features, and obtaining a corresponding convolution kernel K according to the dimension of each spliced information feature.
[0069] In this embodiment, in step S3, each splicing information feature is y di ,but:
[0070] y di =[x 1 ,x 2 ,......x n ,y c1 ,y c2 ,......y cn ],x i ∈R v ,y ci ∈R 1 ;
[0071] The convolution kernel K is obtained according to the dimension of the obtained splicing information features:
[0072] K=(v+n)*1*n*v.
[0073] S4, the n global information features are fused with the corresponding convolution kernel K using convolution, and the output is activated using the activation function to obtain the fused information feature z.
[0074] In this embodiment, in step S4, the global information feature y is converted into c The corresponding convolution kernel K is fused using convolution and activated output to obtain the fused information feature z:
[0075] z=σ(Conv(y c ,K)), where σ(·) is the activation function and Conv(·) is the convolution formula.
[0076] S5, taking the fused information features outputted from step S4 as input data, and then repeating steps S1-S4, and in the process of calculating the convolution kernel K in step S3, expanding the feature dimension to a dimension consistent with the base model.
[0077] In step S5, the fusion information feature z outputted from step S4 is used as input data, and steps S1-S4 are repeated, specifically:
[0078] In the process of executing step S1, the fusion information feature z is divided into n equal parts, each of which has a dimension of v, and then a full connection is performed inside each piece of data, and the activation function is used to activate the output to obtain n local information features z a ;
[0079] In the process of executing step S2, the n local information features are fused and the activation function is used to activate the output, so that the n local information features generate n global information features z respectively. c ;
[0080] In the process of executing step S3, the global information features are respectively spliced to the corresponding local information features to obtain n spliced information features, and the convolution kernel K′ is obtained by the following formula, so as to expand the feature dimension to the dimension consistent with the base model:
[0081] K′=(v+n)*1*n*v″, where n*v″ is equal to the base model output dimension.
[0082] In the process of executing step S4, the global information feature z is calculated using the following formula: c The convolution kernel K′ is fused using convolution and the output is activated to obtain the final output information feature z':
[0083] z′=σ(Conv(z c ,K′));
[0084] Where σ(·) is the activation function and Conv(·) is the convolution formula.
[0085] Please also see Figure 2 The second embodiment of the present invention provides an efficient language large model fine-tuning system, including:
[0086] The local information fusion module is used to divide the input data into n equal parts to obtain n pieces of data, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features;
[0087] A global information feature generation module, which is used to fuse n local information features respectively, so that the n local information features respectively generate n global information features;
[0088] A splicing information feature generation module, which is used to splice n global information features to corresponding local information features respectively to obtain n splicing information features, and obtain a corresponding convolution kernel K according to the dimension of each splicing information feature;
[0089] The local information and global information fusion module is used to fuse the n global information features with the corresponding convolution kernel K using convolution, and activate the output using the activation function;
[0090] The dimension expansion module is used to expand the feature dimension to the same dimension as the base model during the calculation of the convolution kernel K.
[0091] A third embodiment of the present invention provides an electronic device, including a memory and a processor, wherein:
[0092] A memory for storing instructions;
[0093] A processor is coupled to the memory, and the processor is configured to execute the method described in the first embodiment based on instructions stored in the memory.
[0094] The memory may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory may also be a memory array. The memory may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules. The processor may be a central processing unit CPU, or a GPU, or may be an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0095] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in the first embodiment is implemented.
[0096] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices or computer program products. Therefore, the present invention may adopt a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present invention may adopt the form of a computer program product of an embodiment on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A means for specifying functions in one or more blocks in a process or multiple processes and / or a block diagram.
[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 The functions specified in one or more blocks in a process or multiple processes and / or a block diagram.
[0099] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.
Claims
1. An efficient large language model fine-tuning method, characterized in that: The following steps are involved: S1, divide the input data into n equal parts to obtain n pieces of data, each piece of data has a dimension of v, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features; S2, respectively fuse the n local information features and use the activation function to activate the output, so that the n local information features generate n global information features respectively; S3, respectively splicing the global information features to the corresponding local information features to obtain n spliced information features, and obtaining a corresponding convolution kernel K according to the dimension of each spliced information feature; S4, the n global information features are fused with the corresponding convolution kernel K by convolution, and the activation function is used to activate the output to obtain the fused information features; S5, taking the fused information features outputted from step S4 as input data, and then repeating steps S1-S4, and in the process of calculating the convolution kernel K in step S3, expanding the feature dimension to a dimension consistent with the base model.
2. An efficient large language model fine-tuning method as claimed in claim 1, characterized in that: In step S1, let the input data be x, and divide the input data x into n equal parts, that is: x=[x1,x2,x3,.....,x n ],x i ∈R v where x i ∈R v Represents the i-th piece of data; R v Represents the set after the input data is divided into n equal parts; Perform full connection inside each piece of data, and use the following activation function to activate the output to obtain the local information feature y ai : y ai =σ(W i x i +b i ),W i ∈R v′×v ,b i ∈R v′ ,y ai ∈R v′ Among them, σ(·) is the activation function; W i is the weight of local information fusion, b i is the weight offset value of local information fusion, R v ' is the set of local information fusion weight offset values; R v ' ×v is the set of weights for local information fusion. The final output data is obtained, that is, the output local information feature is y a : y a =[y a1 ,y a2 ,y a3 ,.....,y an ],y i ∈R n×v′ ,y a The same dimension as x, R n×v ' is the feature set output after local information fusion.
3. An efficient large language model fine-tuning method as claimed in claim 1, characterized in that: In step S2, the following formula is used to fully connect and fuse n local information features and activate the output to obtain n global information features y ci : y ci =σ(W i x i +b i ),W i ∈R 1×v ,b i ∈R 1 ,y ci ∈R 1 , Where σ(·) is the activation function, W i is the weight of global information fusion, b i is the weight offset value of global information fusion, R 1 is the set of features output by global feature fusion; R 1×v is the set of weights for global information fusion.
4. An efficient large language model fine-tuning method as claimed in claim 1, characterized in that: In step S3, let each splicing information feature be y di ,but: and di =[x1,x2,......x n ,and c1 ,and c2 ,......and cn ],x i ∈R v ,and ci ∈R 1 ; The convolution kernel K is obtained according to the dimension of the obtained splicing information features: K=(v+n)*1*n*v.
5. An efficient large language model fine-tuning method as claimed in claim 1, characterized in that: In step S4, the global information feature y is converted into c The corresponding convolution kernel K is fused using convolution and activated output to obtain the fused information feature z: z=σ(Conv(y c ,K)), where σ(·) is the activation function.
6. An efficient large language model fine-tuning method as claimed in claim 1, characterized in that: In step S5, when executing step S3, the convolution kernel K′ is obtained using the following formula, thereby expanding the feature dimension to a dimension consistent with the base model: The convolution formula used is as follows: K′=(v+n)*1*n*v″, n*v″ is equal to the output dimension of the base model; When executing step S4, the global information feature and the convolution kernel K′ are fused using convolution, and the output is activated to obtain the final output information feature z′.
7. An efficient language large model fine-tuning system, characterized by: include: The local information fusion module is used to divide the input data into n equal parts to obtain n pieces of data, then perform full connection inside each piece of data, and use the activation function to activate the output to obtain n pieces of local information features; A global information feature generation module, which is used to fuse n local information features respectively, so that the n local information features respectively generate n global information features; A splicing information feature generation module, which is used to splice n global information features to corresponding local information features respectively to obtain n splicing information features, and obtain a corresponding convolution kernel K according to the dimension of each splicing information feature; The local information and global information fusion module is used to fuse the n global information features with the corresponding convolution kernel K using convolution, and activate the output using the activation function; The dimension expansion module is used to expand the feature dimension to the same dimension as the base model during the calculation of the convolution kernel K.
8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.
9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.