Contextual learning improvement method for large language models based on parameter parsing

Abstract

The invention relates to the technical field of natural language processing, and discloses a method for improving context learning of a large language model based on parameter parsing, comprising: forming a verification prompt prefix from a task example sample and a verification context through a template of a specific task and inputting the prefix into a large language model to obtain a predicted verification context; dividing the large language model into group representations, observing the degree of performance degradation of the large language model on a specific task after being disturbed through an evaluation index, and establishing an association between the specific task and the group representation; improving the ability corresponding to the group representation through a representation improvement module; forming a test prompt prefix from a task example sample and a test context through a template of a specific task, and inputting the test prompt prefix into a large language model introduced with the representation improvement module to obtain a predicted test context; locating the group representation associated with the cognitive ability of the large language model through a parameter parsing technology, and improving the performance of the large language model on the specific task.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a large language model context learning and improvement method based on parameter parsing. Background Art

[0002] In recent years, large language models of increasing scale have continued to emerge. As the size of the model increases, large language models begin to show some new capabilities that do not exist or are not obvious in small-scale models, such as multi-step arithmetic, logical reasoning, contextual understanding, etc. As these high-order cognitive abilities grow, in downstream tasks of natural language processing, traditional methods based on fine-tuning the entire large language model are gradually abandoned due to the increasing high training costs. As one of the alternative methods, contextual learning only requires a few task examples as prompts, and can use the contextual understanding ability of the large language model to complete the adaptation to specific downstream tasks, so it has become a new hot research direction.

[0003] Although contextual learning can call upon the original high-order cognitive ability of the large language model and the prompts of several task examples to initially complete downstream tasks, all model parameters of the large language model are not optimized for specific tasks, that is, the high-order cognitive ability required for specific tasks has not been strengthened, and there is still a lot of room for improvement. Efficient fine-tuning methods or representation fine-tuning methods provide solutions for further improving specific high-order cognitive abilities, but due to the lack of interpretability of model parameters, the optimized group parameters or group representations are often specified through heuristic strategies, lacking guidance on the correlation between specific high-order cognitive abilities and group parameters or representations, resulting in a mismatch between the optimization object and the optimization goal, and thus a decrease in optimization performance. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a large language model context learning improvement method based on parameter analysis to improve the performance of the large language model in context learning. The present invention proposes two innovative ideas. First, in context learning, the group representation associated with the specific cognitive ability of the large language model is located through parameter analysis technology, which enhances the interpretability of the context learning of the large language model and lays the foundation for further improving the specific cognitive ability of the large language model; second, a representation improvement module is introduced on the group representation associated with the specific ability, and the representation is optimized by a small amount of fine-tuning of the representation improvement module while maintaining the parameters of the large language model to improve the specific cognitive ability of the large language model. In addition, context learning has a wide range of application scenarios due to its simplicity, high efficiency and strong generalization. The context learning improvement method proposed in the present invention only optimizes specific group representations and maintains the above advantages of context learning, so it has good application prospects.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for improving contextual learning of a large language model based on parameter parsing, specifically comprising:

[0007] Construct a training set and a validation set. Each training sample in the training set includes the training And the corresponding training context label ; Each validation sample in the validation set includes the validation above And the corresponding verification tags below;

[0008] Locate the group representation corresponding to the cognitive ability of the large language model for a specific task through parameter parsing technology: Based on the task example samples sampled from the training samples and the verification prompt prefix constructed above ,Will Input the large language model to obtain the original prediction results of the large language model; divide the large language model into group representations, observe the performance degradation of the large language model on specific tasks after being disturbed through evaluation indicators, and establish the association between specific tasks and group representations; introduce a representation enhancement module for each group representation h to optimize the cognitive ability of the group representation for specific tasks; train the representation enhancement module through the training set;

[0009] Task example samples and test above Through task-specific templates Composition test prompt prefix ,Will Input into the large language model that introduces the representation enhancement module to obtain the test context The prediction test below .

[0010] Further, the task example sample sampled from the training sample and the verification context construct a verification prompt prefix ,Will Input the large language model to obtain the original prediction results of the large language model; divide the large language model into group representations, observe the performance degradation of the large language model on specific tasks after being disturbed through evaluation indicators, and establish the association between specific tasks and group representations, including:

[0011] Sampling from the training set training samples as task example samples, and compare the task example samples with the templates for the specific tasks verified above. Composition verification prompt prefix ;Will Input into the large language model and get the verification above The corresponding predictions are verified below ; Divide the group representation of the large language model: divide the input position of the large language model by the number of task example samples as the segmentation granularity, At each level, the Transformer block of each layer is used as the segmentation granularity to obtain group representations; for each group representation, add Gaussian noise as interference, and observe the performance degradation of the large language model on a specific task after the interference through evaluation indicators; select the group representations h corresponding to the top H performance degradations with the highest values ​​for a specific task, and establish an association between the specific task and the group representation.

[0012] Furthermore, the degree of performance degradation of the large language model on a specific task after being disturbed is observed through evaluation indicators, specifically including:

[0013] The perplexity is used as the evaluation indicator; after interfering with the i-th group representation, the performance of the large language model on a specific task decreases for:

[0014] ;

[0015] in, is the original perplexity of the large language model, To interfere with the i-th group representation The perplexity of the post-large language model.

[0016] Further, the task example sample and the template for verifying the specific task above Composition verification prompt prefix , including:

[0017] ;

[0018] Represents string concatenation, represents the context of the D-th task example sample, represents the context label of the Dth task example sample, To verify the above, Represents a placeholder.

[0019] Further, the task example sample and the test above Through task-specific templates Composition test prompt prefix , including:

[0020] ;

[0021] Represents string concatenation, represents the context of the D-th task example sample, represents the context label of the Dth task example sample, Represents a placeholder.

[0022] Furthermore, the specific task is a sentiment classification task, the training context, the verification context, and the test context represent sentiment texts; the training context labels, the verification context labels, the predicted verification context, and the predicted test context represent categories of sentiment texts; the group representation h represents an intermediate representation output by each layer of the Transformer block of the large language model after the verification prompt prefix in the sentiment classification task is input into the large language model.

[0023] Furthermore, the specific task is a mathematical reasoning task, the training context, verification context, and test context represent the mathematical reasoning context; the training context label, verification context label, predicted verification context, and predicted test context represent the mathematical reasoning context corresponding to the mathematical reasoning context; the group representation h represents the intermediate representation output by each layer of the Transformer block of the large language model after the verification prompt prefix in the mathematical reasoning task is input into the large language model.

[0024] Furthermore, a representation enhancement module is introduced for each group representation h to optimize the cognitive ability of the group representation for a specific task, specifically including:

[0025] ;

[0026] in, It is a characterization enhancement module The trainable parameters of Represents the matrix that maps group representation from the general semantic space to the subspace of cognitive ability for specific tasks, which is also the representation improvement module The trainable parameters of Indicates transpose.

[0027] Compared with the prior art, the beneficial technical effects of the present invention are:

[0028] The present invention includes two parts: modeling the relationship between group representation and cognitive ability based on parameter parsing technology, and a lightweight representation improvement module. First, in contextual learning, the group representation associated with the specific cognitive ability of the large language model is located through parameter parsing technology, which makes the internal working mechanism of the large language model more transparent, and lays a solid foundation for further optimizing and improving the specific cognitive ability of the large language model; in addition, for the group representation associated with the cognitive ability of a specific task, the introduced lightweight representation improvement module can significantly optimize the specific representation of the large language model through a small amount of fine-tuning, thereby improving the performance of the large language model on specific cognitive tasks. Secondly, the lightweight representation improvement module maintains the overall parameter structure of the large language model while optimizing the group representation. This means that with a small amount of fine-tuning, not only the training overhead is reduced, but also the overall performance and structure of the large language model will not be negatively affected, thereby maintaining the stability and efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0030] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0031] Specific tasks, such as common sense reasoning, mathematical reasoning, instruction following, natural language understanding, etc., can usually be formalized as follows: given a set of samples , the sample set includes the training set and the test set By learning the mapping relationship between input and output under a specific task, we can get the predicted test context corresponding to the test context .

[0032] Contextual learning can usually be formalized as: sampling from the training set training samples as task example samples, through the template of the specific task Task example samples and test above Convert to a string with the same format and concatenate to form the test prompt prefix , input to the large language model:

[0033] ;

[0034] Large language model based on Continue writing, and finally parse the continued content to get the predicted test content of the previous test content. .

[0035] The context learning improvement method proposed in the present invention is as follows Figure 1As shown in Figure 1, it includes the following two parts: (1) locating the group representation corresponding to the cognitive ability for a specific task through parameter parsing technology; (2) the lightweight representation improvement module further improves the cognitive ability corresponding to the group representation through a small amount of fine-tuning. It should be pointed out that the data used in both parameter parsing and training and reasoning of the representation improvement module are the input and output forms of contextual learning.

[0036] (1) Positioning group representation through parameter analysis technology

[0037] In the present invention, the correlation between different group representations and specific high-level cognitive abilities in contextual learning is obtained through parameter parsing technology, laying the foundation for subsequent ability improvement. Specifically, just like in contextual learning, the present invention combines several task example samples and verification of the above specific task templates. Composition verification prompt prefix Then, the present invention inputs it into the large language model to obtain the original prediction of the large language model. These original predictions will be used as the benchmark performance for subsequent evaluation. The present invention evaluates the performance of the original predictions through evaluation indicators (such as the perplexity of the correct output). The lower the perplexity, the better the prediction performance of the large language model. The next step is to determine the detection target, that is, the division of group representation. The present invention uses the task example sample as the segmentation granularity (the task example sample has a total of 100 samples) at the input position of the large language model. For example, if "{task example sample 1} {task example sample 2} {task example sample 3} {verify the above}" is used as the input of the large language model, the segmentation granularity at the input position of the large language model is 3.

[0038] At the level of the large language model, each layer of Transformer blocks is used as the segmentation granularity (the large language model has layer), get A group representation is a set of intermediate representations of a large language model. For example, when a text is input into a large language model, many intermediate representations will be obtained after the parameter calculation in the large language model. If the length of the text after word segmentation is 150, the word unit representation dimension is 768, and the large language model has 12 layers of Transformer blocks, then after each layer of Transformer blocks, Intermediate representation of size, a total of If we want to divide the intermediate representation of size into two parts according to the number of task examples and the number of layers of the large language model, group representations, then when the task examples are of equal length, each group representation contains An intermediate representation of size.

[0039] For each group representation, the present invention adds Gaussian noise corresponding to the magnitude of the group representation as interference. In this way, the prediction changes of the large language model under interference can be observed, so as to understand the impact of different group representations on specific tasks. The prediction of the large language model after interference is obtained and its performance is evaluated. The evaluation indicators are the same as the original predictions, which are still perplexity and other evaluation indicators. Finally, the degree of performance degradation of interfering with the group representation on a specific task can be obtained. The evaluation indicator takes perplexity as an example:

[0040] ;

[0041] in, is the original perplexity of the large language model, To interfere with the i-th group representation The perplexity of the large language model. The lower the perplexity, the better the performance. At this point, the performance degradation on a specific task can be seen as the importance of the group representation for the higher-order cognitive abilities required for the specific task, thus establishing a correlation between group representation and specific higher-order abilities (e.g. Figure 1 ), laying the foundation for subsequent characterization improvement.

[0042] (2) Characterization enhancement module

[0043] In view of the association between the group representation and the specific high-order cognitive ability established above, the group representation with a strong correlation with the cognitive ability is selected, and the group representation is further optimized through the representation enhancement module to improve the performance of the large language model. Specifically, the present invention introduces a lightweight representation enhancement module for each group representation, which aims to optimize the representation of specific high-order cognitive abilities while minimizing the impact on the overall performance of the large language model:

[0044] ;

[0045] in For the selected group representation, It is a characterization enhancement module The trainable parameters of In this function, the matrix R maps the group representation from the general semantic space to a subspace of specific high-order cognitive abilities and performs Similarly, the training set on a specific task is reconstructed in the form of input and output of contextual learning, and the representation enhancement module is trained so that it can effectively improve the high-order cognitive ability of group representation. The training process is to adjust the trainable parameters of the representation enhancement module to achieve the best performance on a specific task.

[0046] After training is completed, the representation enhancement module can be applied at specific locations and levels of the large language model during the reasoning process to further enhance the specific high-order cognitive capabilities of the large language model and achieve higher performance on specific tasks.

[0047] In summary, the present invention locates group representations related to specific high-order cognitive abilities through parameter analysis, and further introduces a lightweight representation enhancement module to optimize the representation, significantly improving the cognitive ability of the large language model in contextual learning while maintaining general capabilities. In addition, the lightweight characteristics of the representation enhancement module of the present invention significantly reduce the training cost, making the large language model more efficient and practical in practical applications.

[0048] Embodiment 1

[0049] When the task is a sentiment classification task: the sample consisting of sentiment text x and the category label y of the sentiment text is given in the form of (x, y).

[0050] The training samples are: ("This movie is powerful, easy to understand, and interesting", "positive").

[0051] The validation samples are: (“no content, only crude jokes”, “negative”).

[0052] The sentiment text of the test sample is: "I like this movie very much, the picture is beautiful". The goal of the sentiment classification task is to predict the category label of the sentiment text: "positive".

[0053] The template for the sentiment classification task is “review: {x} sentiment: {y}”.

[0054] The following is a simple example of a task example and the verification of the above text to illustrate the input and output of the large language model in the sentiment classification task:

[0055] The input of the large language model is: "Comment: {This movie is powerful, easy to understand, and interesting} Sentiment: {Positive} Comment: {No connotation, only crude jokes} Sentiment: ". The output of the large language model is the predicted verification text.

[0056] Then the group representation h refers to the intermediate representation obtained after each layer of Transformer block corresponding to the following task example sample "Comment: {This movie is powerful, easy to understand, and interesting.} Sentiment: {Positive}".

[0057] Embodiment 2

[0058] When the task is a mathematical reasoning task: a sample consisting of a mathematical reasoning context x and a mathematical reasoning context y is given in the form of (x, y).

[0059] The training samples are: ("What is 5 plus 3?", "8").

[0060] The verification samples are: ("How many times is 20 greater than 4?", "5").

[0061] The mathematical reasoning context of the test sample is: "What is 100 minus 30?" The purpose of the mathematical reasoning task is to predict the mathematical reasoning context of the mathematical reasoning context: "70".

[0062] The template for the mathematical reasoning task is “Question: {x} Answer: {y}”.

[0063] The following is a simple example of a task example and the verification of the above text to illustrate the input and output of the large language model in the mathematical reasoning task:

[0064] The input of the large language model is: "Question: {What is 5 plus 3?} Answer: {8} Question: {How many times is 20 of 4?} Answer: ", and the output of the large language model is the following predicted verification.

[0065] Then the group representation h refers to the intermediate representation obtained after each layer of Transformer block corresponding to the following task example sample "Question: {5 plus 3 equals what?} Answer: {8}".

[0066] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0067] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A large language model context learning and improvement method based on parameter parsing, characterized in that: Specifically include: Construct a training set and a validation set. Each training sample in the training set includes the training And the corresponding training context label ; Each validation sample in the validation set includes the validation above And the corresponding verification tags below; Locate the group representation corresponding to the cognitive ability of the large language model for a specific task through parameter parsing technology: Based on the task example samples sampled from the training samples and the verification prompt prefix constructed above ,Will Input the large language model to obtain the original prediction results of the large language model; divide the large language model into group representations, observe the performance degradation of the large language model on specific tasks after being disturbed through evaluation indicators, and establish the association between specific tasks and group representations, including: Sampling from the training set training samples as task example samples, and compare the task example samples with the templates for the specific tasks verified above. Composition verification prompt prefix ;Will Input into the large language model and get the verification above The corresponding predictions are verified below ; Divide the group representation of the large language model: divide the input position of the large language model by the number of task example samples as the segmentation granularity, At each level, the Transformer block of each layer is used as the granularity for segmentation, and we get group representations; for each group representation, add Gaussian noise as interference, and observe the performance degradation of the large language model on a specific task after the interference through evaluation indicators; select the group representations h corresponding to the top H performance degradations with the highest values ​​for a specific task, and establish the association between the specific task and the group representation; A representation enhancement module is introduced for each group representation h to optimize the cognitive ability of the group representation for specific tasks, including: ； in, It is a characterization enhancement module The trainable parameters of Represents the matrix that maps group representation from the general semantic space to the subspace of cognitive ability for specific tasks, which is also the representation improvement module The trainable parameters of represents transpose; Train the representation enhancement module through the training set; Task example samples and test above Through task-specific templates Composition test prompt prefix ,Will Input into the large language model that introduces the representation enhancement module to obtain the test context The prediction test below .

2. The method for improving contextual learning of a large language model based on parameter parsing according to claim 1 is characterized in that: The evaluation indicators used to observe the performance degradation of the large language model on a specific task after being disturbed include: The perplexity is used as the evaluation indicator; after interfering with the i-th group representation, the performance of the large language model on a specific task decreases for: ； in, is the original perplexity of the large language model, To interfere with the i-th group representation The perplexity of the post-large language model.

3. The method for improving contextual learning of a large language model based on parameter parsing according to claim 1 is characterized in that: The task examples and verifications described above are done through task-specific templates Composition verification prompt prefix , specifically including: ； Represents string concatenation, represents the context of the D-th task example sample, represents the context label of the Dth task example sample, To verify the above, Represents a placeholder.

4. The method for improving contextual learning of a large language model based on parameter parsing according to claim 1 is characterized in that: The task examples and tests described above Through task-specific templates Composition test prompt prefix , specifically including: ； Represents string concatenation, represents the context of the D-th task example sample, represents the context label of the Dth task example sample, Represents a placeholder.

5. The method for improving contextual learning of a large language model based on parameter parsing according to claim 1 is characterized in that: The specific task is a sentiment classification task, the training context, verification context, and test context represent sentiment texts; the training context labels, verification context labels, predicted verification contexts, and predicted test contexts represent categories of sentiment texts; the group representation h represents an intermediate representation output by each layer of the Transformer block of the large language model after the verification prompt prefix in the sentiment classification task is input into the large language model.

6. The method for improving contextual learning of a large language model based on parameter parsing according to claim 1, characterized in that: The specific task is a mathematical reasoning task, the training context, verification context, and test context represent the mathematical reasoning context; the training context label, verification context label, predicted verification context, and predicted test context represent the mathematical reasoning context corresponding to the mathematical reasoning context; the group representation h represents the intermediate representation output by each layer of the Transformer block of the large language model after the verification prompt prefix in the mathematical reasoning task is input into the large language model.

Images