A large model fine-tuning method, system and application for increasing the proportion of key information

Abstract

The present invention discloses a large model fine-tuning method for increasing the proportion of key information and improving application effects, and the large model fine-tuning method comprises the following steps: step 1, processing general information and key information in the input text, and expanding the word list of the word segmenter; step 2, adjusting the embedding layer dimension of the large model, and initializing the weight value of the newly added token; step 3, using the word segmenter after the expanded word list to segment the fine-tuning data; step 4, adding the adjusted embedding layer to the decomposable module, and fine-tuning the large model; step 5, using the fine-tuned large model and the expanded word segmenter for reasoning. The present invention also discloses a large model fine-tuning system for implementing the above method, and corresponding applications, which have a wide range of application scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of large model fine-tuning optimization, and relates to a large model fine-tuning method, system and application that increases the proportion of key information. Background Art

[0002] Mainstream parameter efficient fine-tuning SFT technology, such as LoRA [1] This is done by adding a trainable low-rank decomposition matrix to the linear layer of the Transformer architecture, reducing the trainable parameters of downstream tasks and the space and computing power required for fine-tuning large models; the original large model is retrained using the fine-tuning dataset to improve the model's expressiveness on the fine-tuning data.

[0003] However, when processing fine-tuning data, the existing SFT technology generally uses a tokenizer to encode the input sentence into multiple tokens. Each token is represented by a corresponding integer id. When training a large model, an integer id sequence is used instead of the original character sequence for input. The output is also represented by an integer id sequence, and then the tokenizer is used to decode it into a user-readable character sequence. There is a problem with this method. It will unify all tokens in the character sequence, that is, they have the same weight. The model cannot extract key information from them and treats all inputs equally. For common question-and-answer tasks, this method can use the alignment module inside the large model to learn relevant knowledge. However, for more complex downstream tasks, such as matching correct answers with user answers, grading test papers, etc., the performance is not good in application areas that are strongly related to user input and weakly related to generated content. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a large model fine-tuning method, system and application for increasing the proportion of key information.

[0005] The present invention aims to increase the proportion of key information in the input text and improve the training effect on the model. First, the vocabulary of the original tokenizer is expanded. This operation can: uniformly process continuous general information into one token instead of multiple tokens, so as to reduce their weight when fine-tuning the large model; uniformly process continuous key information into one token, and different key information is represented by different token ids, so as to increase the accuracy of the model in predicting key information. After that, the fine-tuning method commonly used in large models is used for tuning, and the tuned model is obtained. Finally, the tokenizer with an expanded vocabulary is used for character serialization during reasoning, including encoding and decoding, to obtain the prediction results of the model on a specific task. The present invention belongs to a data preprocessing method, does not involve the adjustment of training parameters or model architecture, and is more convenient and efficient to deploy and apply.

[0006] The present invention provides a large model fine-tuning method for increasing the proportion of key information and improving application effect, and the large model fine-tuning method comprises the following steps:

[0007] Step 1: Process the general information and key information in the input text to expand the vocabulary of the tokenizer;

[0008] The general information refers to the fields in the output data that have no practical meaning and are used as the content template for model generation. In the processing of the present invention, it is hoped that the weight of the general information part will be reduced; the key information refers to the information field that contains substantial content information, substantially affects the subsequent reasoning results, and needs to be focused on and processed by the model in the fine-tuning data. In the processing of the present invention, it is necessary to increase the weight of the key information part;

[0009] In the present invention, the general information that appears continuously is uniformly processed into a token, and the key information that appears continuously is uniformly processed into a token, and different key information is represented by different token ids;

[0010] In step 1, processing the input text and expanding the vocabulary of the tokenizer further comprises the following steps:

[0011] Step 1.1: Count the general information and key information in the input text;

[0012] Step 1.2, obtain a list of general information and key information respectively;

[0013] Step 1.3, merging the lists of the general information and the key information as a new vocabulary;

[0014] Step 1.4: Add the new vocabulary obtained by merging in step 1.3 to the word segmenter.

[0015] Step 2: Adjust the embedding layer dimension of the large model and initialize the weight value of the newly added token;

[0016] In step 2, before and after the adjustment, the embedding layer dimension of the large model is converted from [old_vocab_size, embed_dim] to [new_vocab_size, embed_dim], where old_vocab_size represents the size of the vocabulary before expansion, new_vocab_size represents the size of the vocabulary after expansion, and embed_dim represents the embedding layer dimension;

[0017] The weight of the newly added token is initialized by the subword average method. The newly added token will be segmented according to the existing vocabulary to generate multiple subwords, including the following steps:

[0018] Step 2.1. Encode each newly added token using the word segmenter before and after the word list expansion, and obtain the subword id list of the new token and the id of the new token in the new word list, respectively. The id value in the new word list is the index value of the new token in the expanded embedding layer;

[0019] Step 2.2. For the index value in the subword id list, find the corresponding weight value of each subword in the embedding layer of the original model;

[0020] Step 2.3. Sum the weight values ​​of all subwords obtained in step 2.2 and calculate the average by dimension as the initial weight value of the newly added token;

[0021] Step 2.4. Re-assign the corresponding weight value in the embedding layer of the new model using the id value of the new token obtained in step 2.1. The new value is the mean sub-word weight obtained in step 2.3.

[0022] Step 3: Use the word segmenter with the expanded vocabulary to segment the fine-tuning data;

[0023] In step three, according to the technical solution of the present invention, the fine-tuned data is segmented, the general information that appears continuously is uniformly processed into a token, the key information that appears continuously is also uniformly processed into a token, and the fine-tuned data is segmented, which can significantly increase the proportion of key information and also reduce the list length of the output token id.

[0024] Step 4: Retrain the adjusted embedding layer, add it to the decomposable module, and fine-tune the large model;

[0025] Decomposable module: Low-rank decomposition technology is widely used when fine-tuning large models to reduce the amount of trainable parameters, thereby improving model training efficiency. For example, when fine-tuning, it is necessary to adjust the parameters of a matrix of dimension M*N, but the number of parameters is too large, so it is low-rank decomposed into two sub-matrices of dimension M*R and dimension R*N, which can significantly reduce the number of parameters and thus reduce the cost of fine-tuning. Adding a decomposable module means that the corresponding matrix parameters must first be low-rank decomposed and then fine-tuned. Since the size of the vocabulary has changed in steps 1-3, the weight dimension of the model Embedding has also changed, and the parameters need to be retrained. Therefore, it is necessary to add it to the decomposable module, perform low-rank decomposition on the matrix to reduce the amount of training parameters, and then enter the fine-tuning process until the new model is obtained after the fine-tuning is completed.

[0026] Step 5: Use the fine-tuned large model and the expanded word segmenter for reasoning: Get the fine-tuned new model, and use the word segmenter with an expanded vocabulary to segment the data during reasoning.

[0027] The present invention also provides a large model fine-tuning system for realizing the large model fine-tuning method for increasing the proportion of key information, the system comprising: a text preprocessing and word segmenter expansion module, an embedding layer adjustment and initialization module, a data segmentation module, an embedding layer and decomposable module integration module, and a reasoning and application module;

[0028] 1) Text preprocessing and word segmentation extension module

[0029] Function: Identify and process general and key information in the input text, and expand the vocabulary of the tokenizer.

[0030] Implementation steps: Corresponding to step 1 in the fine-tuning method. This module receives the original input text, identifies and distinguishes general information from key information, and then expands the vocabulary of the existing word segmenter to enable it to handle new or unseen words.

[0031] 2) Embedding layer adjustment and initialization module

[0032] Function: Adjust the embedding layer dimension of the large model and initialize the weight value for the newly added token.

[0033] Implementation steps: Corresponding to step 2. This module adjusts the model's embedding layer according to the expanded vocabulary to ensure that it can accommodate new tokens. After that, it generates initial weights for these new tokens so that they can effectively participate in model training in subsequent fine-tuning.

[0034] 3) Data segmentation module

[0035] Function: Use the extended tokenizer to segment the fine-tuning data and generate a token sequence suitable for the input model.

[0036] Implementation steps: Corresponding to step 3. This module receives the preprocessed fine-tuning data and uses the extended tokenizer to segment the text to ensure that all key information can be correctly represented and passed to the model.

[0037] 4) Embedding layer and decomposable module integration module

[0038] Function: Integrate the adjusted embedding layer with the decomposable module to efficiently fine-tune large models.

[0039] Implementation steps: Corresponding to step 4. This module further reduces the training parameters of the embedding layer by integrating modules such as low-rank decomposition, reduces the computing resources and storage space required during fine-tuning, and performs model fine-tuning.

[0040] 5) Reasoning and application module

[0041] Function: Use the fine-tuned large model and the expanded word segmenter for inference and output the model prediction results.

[0042] Implementation steps: Corresponding to step 5. This module uses the fine-tuned model to process new input data, uses the expanded word segmenter to ensure that key information is fully utilized, and generates high-quality output.

[0043] The present invention also provides the above-mentioned large model fine-tuning method, or the application of the above-mentioned large model fine-tuning system in scenarios such as improving the prediction effect of the large model, calculating the accuracy of user answers, marking user test papers, and analyzing private special data.

[0044] The present invention also provides a hardware system for implementing the above-mentioned large model fine-tuning method, and the hardware system includes: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned method is implemented.

[0045] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above method is implemented.

[0046] The beneficial effects of the present invention include: the prior art uses a trained tokenizer to uniformly encode all characters in the input data and convert them into an integer ID sequence for representation. Each character has the same weight, and it is impossible to effectively distinguish between key information and general information. The present invention focuses on alleviating the above problems. First, the vocabulary of the tokenizer is expanded, and a token is uniformly used to represent general information to reduce their weights during model training; in addition, different key information is represented by different tokens to improve the model's prediction ability for key information. The present invention focuses on more complex downstream tasks, such as answer matching, test paper scoring, etc. The fine-tuning data of these tasks has a feature, namely: the tokens of general information account for a large proportion, and the tokens of key information account for a small proportion, which causes the model to mainly learn general information, while the prediction accuracy of important information is low. By adding a data preprocessing step, the proportion of key information in fine-tuning is artificially increased, thereby improving the model's understanding and expression capabilities for complex tasks, thereby improving the application effect of downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 It is a flow chart for implementing the large model fine-tuning method of the present invention for increasing the proportion of key information.

[0049] Figure 2 It is a diagram of the tokenizer vocabulary expansion method of the present invention.

[0050] Figure 3 It is a weight initialization strategy diagram of the new vocabulary of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0052] The explanations of the relevant English terms involved in the present invention are as follows:

[0053] Tokenizer: a word segmenter that divides Chinese sentences into sequences of multiple characters, words, phrases, etc.

[0054] Token: field. Each field is a part of a Chinese sentence. It is the smallest unit read by the model and corresponds to a specific index number in the tokenizer.

[0055] token id: field index number, an integer number is used to refer to a specific field

[0056] The current mainstream large model fine-tuning strategy is based on the fine-tuning data set, and uses the tokenizer tool to unify all input tokens. There is a problem with this method, that is, all tokens have the same weight, and the key information model will be ignored, so it cannot be well applied to downstream tasks. The present invention aims to optimize the above-mentioned problem, by highlighting the proportion of key information, so that the large model can better learn the important part of the knowledge, strengthen the model's predictive ability for key information, and thus improve the application effect of the large model in specific fields. The advantage of the present invention is that the model performance is improved only by modifying the preprocessing stage data, without involving training parameters, model architecture and other parts, which makes it more convenient and efficient to deploy applications.

[0057] The present invention provides a large model fine-tuning method for increasing the proportion of key information and improving application effect, and the large model fine-tuning method comprises the following steps:

[0058] Step 1: Process the general information and key information in the input text to expand the vocabulary of the tokenizer;

[0059] Step 2: Adjust the embedding layer dimension of the large model and initialize the weight value of the newly added token;

[0060] Step 3: Use the word segmenter with the expanded vocabulary to segment the fine-tuning data;

[0061] Step 4: Retrain the adjusted embedding layer parameters, add them to the decomposable modules, and fine-tune the large model;

[0062] Step 5: Use the fine-tuned large model and the expanded word segmenter for inference.

[0063] Example

[0064] In this embodiment, the downstream task of "matching the correct answer and the user answer" is taken as an example to explain the implementation mode of the present invention in detail.

[0065] 1. Task purpose: Calculate the accuracy of the user's answers by matching the correct answers with the user's answers. For example, if the correct answer to this question is A and the user's answer is B, then the question is wrong; if the answer to this question is C and the user's answer is C, then the question is correct. Similarly, count the number of questions the user answered correctly and calculate the accuracy rate. The following is an example of model fine-tuning:

[0066] Input: "The correct answer is: C, B, A, A, D, BD, ABD, ACD. The user's answer is: A, B, D, A, B, AB, ABD, BCD. Calculate the accuracy rate."

[0067] Output: "The correct answer to question 1 is: C, the user's answer is: A, the answer is wrong;

[0068] The correct answer to question 2 is: B. The user's answer is: B. The answer is correct.

[0069] The correct answer to question 3 is: A, the user's answer is: D, the answer is wrong;

[0070] The correct answer to question 4 is: A. The user's answer is: A. The answer is correct.

[0071] The correct answer to question 5 is: D, the user's answer is: B, the answer is wrong;

[0072] The correct answer to question 6 is: BD, the user's answer is: AB, the answer is wrong;

[0073] The correct answer to question 7 is: ABD, the user's answer is: ABD, the answer is correct;

[0074] The correct answer to question 8 is: ACD, the user's answer is: BCD, the answer is wrong;

[0075] Number of correct answers: 3, total number of questions: 8, user accuracy rate: 37.5%"

[0076] 2. Data processing method in the prior art: Use the trained tokenizer to pre-process the fine-tuning data. For the above example, the segmented character sequence is represented as:

[0077] Input: [59568,7041,9739,59626,106,59608,105,59616,105,59603,105,59603,105,59614,105,7925,105,4442,59614,105,2596,59614,101,4576,9739,59626,106,59603,105,59616,105,59614,105,59603,105,59616,105,4442,105,4442,59614,105,4088,59614,101,4804,7041,60130]

[0078] Output: [59568,19107,78,7041,9739,59626,106,59608,101,4576,9739,59626,106,59603,101,8992,8755,59816,144,19107,79,7041,9739,59626,106,59616,101,4576,9739,59626,106,59616,101,8992,7041,59816,144,19107,80,7041,9739,59626, 106,59603,101,4576,9739,59626,106,59614,101,8992,8755,59816,144,19107,81,7041,9739,59626,106,59603,101,4576,9739,59626,106,59603,101,8992,7041,59816,144,19107,82,7041,9739,59626,106,59614,101,4576,9739,59626,106,5 9616,101,8992,8755,59816,144,19107,83,7041,9739,59626,106,7925,101,4576,9739,59626,106,4442,101,8992,8755,59816,144,19107,84,7041,9739,59626,106,4442,59614,101,4576,9739,59626,106,4442,59614,101,8992,7041,59816,14 4,19107,85,7041,9739,59626,106,2596,59614,101,4576,9739,59626,106,4088,59614,101,8992,8755,59816,144,8992,7041,59650,59843,106,80,101,19107,59965,59650,59843,106,85,101,4576,7041,60130,59626,106,80,84,98,82,59677]

[0079] Each integer represents the token id of a character in a sentence. The integer sequence will then be input into the large model for parameter fine-tuning to complete the adaptation and optimization of downstream tasks.

[0080] 3. Problem analysis: When matching the correct answer with the user answer, the model is expected to focus on the correct options and the user options, and whether the two are the same, and to perform statistics and calculations based on this information; when fine-tuning the model, for the above examples, since the output data contains too many meaningless fields such as "The correct answer is:" and "The user answer is:", these fields serve as templates for the model to generate content, and their proportion is much higher than the proportion of the option fields (key information), thus affecting the model's learning process for the ability to match the correct options and user options.

[0081] 4. Large model fine-tuning strategy to increase the proportion of key information: The general information in the fine-tuning data is uniformly replaced with a token to reduce the weight of the general information during model fine-tuning; the key information that determines the output results of the model is also uniformly replaced with a token to improve the accuracy of the model's prediction of this part of information.

[0082] Therefore, the large model fine-tuning method of increasing the proportion of key information and improving the application effect in the present invention is specifically implemented as follows in this embodiment:

[0083] Step 1: Expand the tokenizer vocabulary

[0084] (1) First, general information and key information need to be collected.

[0085] 1) Taking the above model fine-tuning examples as a reference, by analyzing the data composition, it can be seen that this type of data is used to match the correct answer with the user's answer and calculate the accuracy of the answer. The output template of this type of task is relatively unified, so that the large model can better understand the user's intention and give the required output. This type of data can be used as pre-training data to cultivate the matching ability of the model, and can also be used as fine-tuning data to support the performance of the model on downstream tasks.

[0086] 2) By analyzing the input and output statements, you can manually define general information and key information:

[0087] A. Key information: Fields indicating correct options and user options, including: A, B, C, D, AB, AC, AD, BC, BD, CD, ABC, ABD, ACD, BCD, ABCD. The above are all possible option results.

[0088] B. General information: indicates the text content of the generated template, including: correct answer is, user answer is, calculated accuracy, question, wrong answer, correct answer, number of correct answers, total number of questions, user accuracy is, the above are all the template content

[0089] (2) Then, expand the tokenizer vocabulary

[0090] 1) Tokenizer Description: Tokenizer is a word segmenter. It is used to convert a character sequence into a token ID sequence during encoding and to convert a token ID sequence into a character sequence during decoding. It has the following characteristics:

[0091] A.tokenizer.decode(tokenizer.encode(String))=String / / For any string, use a tokenizer to first encode it into a token id, and then use the same tokenizer to decode it to obtain the original string, that is, there is no information loss.

[0092] B. Segmentation first and then mapping: "Today is a good weather" -> "Today is a good weather"

[0093] C.tokenizer.encode("Today is a good day") = [3569,8023,59706,9468] / / Using the tokenizer to encode, the string will be converted into an integer sequence

[0094] D.tokenizer.decode([3569,8023,59706,9468]) = "Today is a good day" / / Using the tokenizer to decode, the integer sequence will be converted into a string

[0095] 2) Expand the vocabulary code:

[0096] keys_info_list = ['A','B','C','D','AB','AC','AD','BC','BD','CD','ABC','ABD','ACD','BCD','ABCD'] / / This list is a key information list, containing important information related to user options and standard answers

[0097] norm_info_list = ['Correct answer is', 'User answer is', 'Calculate correct rate', 'Topic', 'Wrong answer', 'Correct answer', 'Number of correct answers', 'Total number of topics', 'User correct rate is'] / / This list is a general information list, including output templates and other related information

[0098] new_vocab_list = keys_info_list + norm_info_list / / The new vocabulary list is formed by merging the general information list and the key information list

[0099] SPECIAL_TOKEN = {"additional_special_tokens": new_vocab_list} / / Add the new vocabulary list to the dictionary of the tokenizer

[0100] tokenizer.add_special_tokens(SPECIAL_TOKEN) / / Merge the new vocabulary into the existing tokenizer

[0101] Step 2: Adjust the model Embedding layer dimensions and initialize the weight values ​​of the new tokens

[0102] Due to the addition of new tokens, the dimensions of the Embedding layer of the original model will change accordingly, and need to be converted from [old_vocab_size, embed_dim] to [new_vocab_size, embed_dim], where old_vocab_size represents the size of the vocabulary before expansion, new_vocab_size represents the size of the vocabulary after expansion, and embed_dim represents the dimension of the embedding layer. Use the following code to implement this function:

[0103] model.resize_token_embeddings(len(tokenizer))

[0104] The code will initialize all newly added words uniformly according to the standard normal distribution. This operation will cause the weight distribution of the newly added words to be significantly different from the weight distribution of the existing words, which will affect the convergence effect of the model during fine-tuning. In view of the above problems, the present invention adopts the subword mean method to initialize the weights.

[0105] (1) Detailed description: The original unexpanded tokenizer contains sub-word representations of smaller granularity. For example, the general information "total number of questions" in the above text can be divided into: question, total, number, number, a total of four sub-words, and the corresponding token ids are: 19107, 59965, 59650, 59843. The method adopted by the present invention is as follows:

[0106] A. For each newly added token (including: "total number of questions", "calculated accuracy", "question number", etc.), use the extended tokenizer to split it to obtain the id of the new token; then use the original tokenizer to split it to obtain the subword list of the new token

[0107] B. For the subwords in the subword list after segmentation by the original tokenizer (for example, the newly added token is "total number of topics", and the original tokenizer is used to segment the words, we can get: ["topic", "total", "items", "number"] subword list with a length of 4), find the corresponding weight value of each subword in the Embedding layer

[0108] C. Add the above weight values ​​by dimension (the weight value of the entire subword list is obtained by adding the dimensions, for example, [a,b,c]+[x,y,z]=[a+x,b+y,c+z]), and calculate the average by dimension, and use it as the initial weight value of the newly added token to participate in the training

[0109] (2) Weight initialization code:

[0110] new_token_id = new_tokenizer.encode(new_token) / / Use the extended tokenizer to encode the newly added token and obtain the index value corresponding to the token

[0111] token_ids = ori_tokenizer.encode(new_token) / / Use the original tokenizer to encode the newly added token and obtain the subword id list of the token

[0112] embed_data = torch.zeros(model.weight.data.shape[1])(model.weight.data.shape[1] is used to obtain the dimension of the weight, torch.zeros() is used to generate an initial matrix with all elements being 0 according to the specified dimension) / / Define the numerical type of the weight of the new token and initialize it for subsequent assignment

[0113] for itemin token_ids:

[0114] embed_data+=model.weight.data[item] / / Add the weight values ​​of the subwords corresponding to the new token according to the dimension

[0115] model.weight.data[new_token_id] = embed_data / len(token_ids) / / The weight values ​​obtained in the previous step are averaged according to the dimension as the initial weight of the new token

[0116] Among them, new_token refers to any new token in the new vocabulary; new_tokenizer refers to the expanded tokenizer; ori_tokenizer refers to the original tokenizer; model refers to the large model to be trained.

[0117] Step 3: Use the tokenizer with the expanded vocabulary to segment the fine-tuning data

[0118] The model fine-tuning examples above are segmented, and the segmentation results of the tokenizer after expanding the vocabulary are as follows:

[0119] Input: [64004,106,59608,105,59616,105,59603,105,59603,105,59614,105,7925,105,64000,105,64001,101,64005,106,59603,105,59616,105,59614,105,59603,105,59616,105,4442,105,64000,105,64002,101,64006]

[0120] Output: [19107,78,64004,106,59608,101,64005,106,59603,101,64007,59816,144,19107,79,64004,106,59616,101,64005,106,59616,101,64008,59816,144,19107,80,640 04,106,59603,101,64005,106,59614,101,64007,59816,144,19107,81,64004,106,59603,101,64005,106,59603,101,64008,59816,144,19107,82,64004,106,59614,101 ,64005,106,59616,101,64007,59816,144,19107,83,64004,106,7925,101,64005,106,4442,101,64007,59816,144,19107,84,64004,106,64000,101,64005,106,64000,1 01,64008,59816,144,19107,85,64004,106,64001,101,64005,106,64002,101,64007,59816,144,64009,106,80,101,64010,106,85,101,64011,106,80,84,98,82,59677]

[0121] The following table compares the segmentation effects of the two tokenizers:

[0122] Tokenizer Type Enter the number of tokens Output token number The proportion of key information The proportion of general information original 48 173 53.23％ 46.77％ expansion 37 119 56.69％ 43.31％

[0123] Key information is: user answer and correct answer options; general information is: other characters except key information.

[0124] From the information in the table, we can see that the use of the tokenizer with expanded vocabulary for data segmentation has increased the proportion of key information from 53.23% to 56.69%, and the length of the token id list after segmentation has decreased significantly from 221 to 156. Therefore, more key information can be focused on in subsequent model training, thereby improving the performance of the model on downstream tasks.

[0125] Step 4: Add the Embedding layer to the decomposable module and then fine-tune the model

[0126] Step 2 adjusts the dimension of the model's Embedding layer and initializes the weights of new words, so this layer needs to be trained again during fine-tuning. This can be achieved by adding it to the decomposable module.

[0127] Step 5: Use the tokenizer with the expanded vocabulary to segment the test data, and then use the fine-tuned model to infer the data to evaluate the effect of the large model on the test data;

[0128] This step is a regular step for model inference, except that the expanded tokenizer is used to segment the text.

[0129] 5. Large model training parameters and test results based on fine-tuning of the strategy of the present invention

[0130] 1) The following table compares the segmentation effects of the two tokenizers on the training data set:

[0131] Tokenizer Type Enter the number of tokens Output token number The proportion of key information The proportion of general information original 18971 73793 61.84％ 38.16％ expansion 15024 51482 67.87％ 32.13％

[0132] From the information in the table, we can see that the use of the tokenizer with expanded vocabulary for data segmentation has increased the proportion of key information from 61.84% to 67.87%, and the total length of the token id list after segmentation has decreased significantly from 92764 to 66506.

[0133] 2) In a specific implementation scenario, the model category, data scale, and training parameters used are as follows:

[0134] (1) Large model structure: The Yi-6B-Chat model is used for training, which contains 600 million parameters.

[0135] (2) Training data size: 128 fine-tuning data pairs

[0136] (3) Test data scale: 100 test data pairs

[0137] (4) Model training parameters:

[0138] a. Batch data size: 4

[0139] b. Maximum sentence length: 1024

[0140] c. Initial learning rate: 8e-5

[0141] d. Number of training rounds: 48

[0142] e.LORA parameters: rank = 32, lora_alpha = 32

[0143] f. Model preservation strategy: take the last model as the final model

[0144] 3) Test results of large models fine-tuned based on different strategies

[0145] A. Test results of a large model fine-tuned based on common strategies

[0146] Tokenizer Type Vocabulary size Model structure Percentage of trainable parameters Reasoning time Model Accuracy original 64000 Yi-6B-Chat 1.18％ 203.02s 89％

[0147] B. Test results of a large model fine-tuned based on the strategy of the present invention

[0148] Tokenizer Type Vocabulary size Model structure Percentage of trainable parameters Reasoning time Model Accuracy expansion 64012 Yi-6B-Chat 1.18％ 203.14s 94％

[0149] 6. Summary

[0150] From the above table data, it can be seen that the large model fine-tuning strategy proposed by the present invention to increase the proportion of key information is effective, the model reasoning time is basically the same as the baseline model, and the accuracy rate is increased from 89% to 94%, which is in line with the expected effect. Considering the application scenario of the present invention, it can be easily deployed in the field of large model fine-tuning to improve the performance of large models in specific tasks.

[0151] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0155] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0156] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0157] References

[0158] [1]Hu EJ,Shen Y,Wallis P,et al.Lora:Low-rank adaptation of largelanguage models[J].arXiv preprint arXiv:2106.09685,2021.

[0159] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A large model fine-tuning method for increasing the proportion of key information, characterized in that: The large model fine-tuning method comprises the following steps: Step 1: Process the general information and key information in the input text to expand the vocabulary of the tokenizer; In step 1, the general information refers to the fields in the output data that have no practical meaning and serve as the content template for model generation; the key information refers to the information fields in the fine-tuning data that the model needs to focus on and process; The general information that appears continuously is uniformly processed into one token, and the key information that appears continuously is uniformly processed into one token. Different key information is represented by different token ids. Step 2: Adjust the embedding layer dimension of the large model and initialize the weight value of the newly added token; In step 2, the embedding layer dimension in the model is converted from [old_vocab_size, embed_dim] to [new_vocab_size, embed_dim], where old_vocab_size represents the size of the vocabulary before expansion, new_vocab_size represents the size of the vocabulary after expansion, and embed_dim represents the embedding layer dimension; Initialize the weight of the newly added token by the subword averaging method. The newly added token will be segmented according to the existing vocabulary to generate multiple subwords, including the following steps: Step 2.

1. Encode each newly added token using the word segmenter before and after the vocabulary expansion, and obtain the subword id list of the new token and the id of the new token in the new vocabulary, respectively. The id value in the new vocabulary is the index value of the new token in the expanded embedding layer; Step 2.

2. For the index value in the subword id list, find the corresponding weight value of each subword in the embedding layer of the original model; Step 2.

3. Sum the weight values ​​of all subwords obtained in step 2.2 and calculate the average by dimension as the initial weight value of the newly added token; Step 2.

4. Re-assign the corresponding weight value in the embedding layer of the new model using the id value of the new token obtained in step 2.

1. The new value is the mean of the subword weights obtained in step 2.

3. Step 3: Use the word segmenter with the expanded vocabulary to segment the fine-tuning data; Step 4: Add the adjusted embedding layer to the decomposable module and fine-tune the large model; Step 5: Use the fine-tuned large model and the expanded word segmenter for inference.

2. The large model fine-tuning method according to claim 1, characterized in that: In step 1, processing the input text and expanding the vocabulary of the tokenizer further includes the following steps: Step 1.

1. Count the general information and key information in the input text; Step 1.

2. Obtain a list of general information and key information respectively; Step 1.

3. Merge the general information list and the key information list as a new vocabulary; Step 1.

4. Add the new vocabulary obtained by merging in step 1.3 to the word segmenter.

3. The large model fine-tuning method according to claim 1, characterized in that: In step three, the general information that appears continuously is uniformly processed into a token, and the key information that appears continuously is also uniformly processed into a token. The fine-tuned data is segmented to increase the proportion of key information and reduce the length of the token id list.

4. The large model fine-tuning method according to claim 1, characterized in that: In step 4, the embedding layer with adjusted dimensions is retrained and added to the decomposable module to reduce the number of model training parameters, and then the fine-tuning process is entered until a new model is obtained after the fine-tuning is completed; and / or, In step 5, a new fine-tuned model is obtained, and the word segmenter with an expanded vocabulary is used to segment the data during inference.

5. A large model fine-tuning system, characterized in that: The system can implement the large model fine-tuning method as described in any one of claims 1 to 4, comprising: a text preprocessing and word segmenter expansion module, an embedding layer adjustment and initialization module, a data segmentation module, an embedding layer and decomposable module integration module, and an inference and application module; The text preprocessing and word segmenter expansion module identifies and processes general information and key information in the input text and expands the word segmenter vocabulary; The embedding layer adjustment and initialization module adjusts the embedding layer dimension of the large model and initializes the weight value for the newly added token; The data segmentation module uses the extended word segmenter to segment the fine-tuning data and generate a token sequence suitable for the input model; The embedding layer and decomposable module integration module integrates the adjusted embedding layer and the decomposable module to perform fine-tuning of the large model; The reasoning and application module uses the fine-tuned large model and the expanded word segmenter to perform reasoning and output the model prediction results.

6. A hardware system for implementing the method according to any one of claims 1 to 4, characterized in that: The hardware system comprises: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images