Language model prompt learning-based common sense reasoning method

Abstract

The embodiment of the invention discloses a common sense reasoning method based on language model prompt learning. The method comprises the steps of obtaining a training data set and a pre-training language model; wherein the training data set comprises question and answer pair data corresponding to at least one batch; for the current training batch, based on the current question and answer pair data of the current training batch, a predetermined current prompt vector and a pre-training language model, determining a current loss value corresponding to the current prompt vector and a current gradient corresponding to the current loss value; determining a next prompt vector corresponding to the question and answer pair data of the next training batch based on the current gradient and a predetermined preset embedding vector, and repeatedly executing a loss value determination operation based on the next prompt vector until the newest loss value meets a preset training target to obtain a newest prompt vector; and determining response information corresponding to the to-be-answered question stem based on the latest prompt vector, the to-be-answered question stem and the to-be-migrated reasoning model.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of natural language processing and large language model prompt learning, and in particular to a common sense reasoning method based on language model prompt learning. Background Art

[0002] As large-scale pre-trained language models have demonstrated good reasoning capabilities and considerable knowledge, prompt learning, as a new transfer learning paradigm, has begun to be widely used in various reasoning tasks. It generalizes pre-trained language models to various reasoning tasks by attaching prompts before the task text.

[0003] In the prior art, discrete prompt learning technology is usually used to determine the prompts required by the pre-trained language model, and the prompts are transferred to different common sense reasoning language models to complete different types of reasoning tasks. However, the current discrete prompt learning technology has the problem of slow convergence. Summary of the invention

[0004] The embodiment of the present invention provides a common sense reasoning method based on language model prompt learning to achieve the purpose of improving the efficiency of determining prompt vectors and response information.

[0005] According to one aspect of the present invention, a common sense reasoning method based on language model prompt learning is provided, comprising:

[0006] Obtaining a training data set and a pre-trained language model; wherein the training data set includes at least one batch of question-answer pair data;

[0007] For a current training batch, based on current question-answer pair data of the current training batch, a predetermined current prompt vector, and the pre-trained language model, determine a current loss value corresponding to the current prompt vector and a current gradient corresponding to the current loss value;

[0008] Based on the current gradient and the predetermined preset embedding vector, determine the next prompt vector corresponding to the question-answer pair data of the next training batch, so as to repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, thereby obtaining the latest prompt vector; wherein the preset embedding vector is a discrete prompt vector obtained by the pre-trained language model;

[0009] Based on the latest prompt vector, the question stem to be answered and the reasoning model to be transferred, the response information corresponding to the question stem to be answered is determined.

[0010] The technical solution of the embodiment of the present invention is to obtain a training data set and a pre-trained language model; wherein the training data set includes question-answer pair data corresponding to at least one batch; for the current training batch, based on the current question-answer pair data of the current training batch, the predetermined current prompt vector and the pre-trained language model, determine the current loss value corresponding to the current prompt vector and the current gradient corresponding to the current loss value; based on the current gradient and the predetermined preset embedding vector, determine the next prompt vector corresponding to the question-answer pair data of the next training batch, so as to repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, and obtain the latest prompt vector; wherein the preset embedding vector is a discrete prompt vector obtained by the pre-trained language model; thereby obtaining the latest prompt vector by updating the current gradient-constrained prompt vector; and determining the response information corresponding to the question stem to be answered based on the latest prompt vector, the question stem to be answered and the inference model to be transferred. The technical solution of this embodiment solves the problem of slow convergence of discrete prompt learning by updating the gradient-constrained prompt vector, and improves the efficiency of determining the prompt vector and the response information.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 is a flowchart of a common sense reasoning method based on language model prompt learning according to an embodiment of the present invention;

[0014] Figure 2 It is a flowchart of another common sense reasoning method based on language model prompt learning provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "etc." and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0017] Figure 1 This is a flowchart of a common sense reasoning method based on language model prompt learning provided according to an embodiment of the present invention. This embodiment is applicable to the case where a prompt vector is determined and an attempt is made to reason through the prompt vector and the language model. The method can be performed by a common sense reasoning device based on language model prompt learning, and the common sense reasoning device based on language model prompt learning can be implemented in the form of hardware and / or software.

[0018] like Figure 1 As shown, the method of this embodiment may specifically include:

[0019] S110, obtaining a training data set and a pre-trained language model.

[0020] The training data set includes at least one batch of question-answer pair data; each batch contains at least one question-answer pair data. The question-answer pair data includes a question stem, relevant facts, and a true answer expressed in natural language. Exemplarily, the question stem may be a judgment type question, the relevant facts are the true description of the question stem, and the true answer is the correct answer to the question stem. For example, the true answer includes yes or no. The pre-trained language model may be a pre-trained generative language model for common sense reasoning. Common sense reasoning can be performed by combining the pre-trained language model with prompts expressed in natural language.

[0021] In this embodiment, a pre-built common sense reasoning dataset can be obtained, and the common sense reasoning dataset is used as a training dataset for training prompts. In addition, a pre-trained language model for common sense reasoning is obtained, and the prompt vector is initialized based on the embedding layer dimension of the pre-trained language model to obtain an initialized prompt vector. Specifically, the initialization process can set the number of prompt vectors as a hyperparameter of the pre-trained language model, and set the dimension of the prompt vector as the embedding layer dimension of the pre-trained language model.

[0022] S120. For the current training batch, based on the current question-answer pair data of the current training batch, a predetermined current prompt vector and a pre-trained language model, determine a current loss value corresponding to the current prompt vector and a current gradient corresponding to the current loss value.

[0023] In a specific implementation, the question-answer pair data in the training data set can be divided into multiple training batches, and each training batch can be traversed to determine the prompt vectors corresponding to the prompts used for migration to different common sense reasoning models through the question-answer pair data in different training batches.

[0024] Specifically, the currently traversed batch can be used as the current training batch, and the question-answer pair data in the current training batch can be used as the current question-answer pair data. The current prompt vector is the prompt vector used in the current training batch. Exemplarily, if the current training batch is the first training batch, the current prompt vector can be the initialization prompt vector. If the current training batch is not the first training batch, the current prompt vector can be the prompt vector obtained by training the question-answer pair data of the previous training batch.

[0025] Optionally, the current question-answer pair data includes the current question stem, at least one current relevant fact and the current true answer; based on the current question-answer pair data of the current training batch, a predetermined current prompt vector and a pre-trained language model, determine the current loss value corresponding to the current prompt vector, including: for each current question-answer pair data, based on the current question stem, the current prompt vector, the current relevant facts and the pre-trained language model in the current question-answer pair data, determine the current predicted answer corresponding to the current question stem; based on the current predicted answer and the current true answer corresponding to the current predicted answer, determine the current loss value.

[0026] Specifically, the current input vector can be generated based on the current question stem and the current prompt vector, and the current input vector can be input into the pre-trained language model to obtain the current predicted answer that matches the current relevant facts. The current loss value corresponding to the current prompt vector is determined by comparing the current true answer corresponding to each current question stem with the current predicted answer.

[0027] It should be noted that the calculation formula of the current loss value L is:

[0028] L=(1-λ)L1+L2

[0029] Among them, λ is a hyperparameter that controls the fluency loss, L1 is the task loss, and L2 is the fluency loss. The task loss consists of cross entropy loss and format loss.

[0030] L1=L3+λ format L4

[0031] Among them, λ formatTo control the hyperparameters of format loss, L3 is the cross entropy loss and L4 is the format loss. The calculation formula of L4 is:

[0032]

[0033] Wherein, C represents the number of subsequences that meet the format requirements output by the pre-trained language model, and ∈ represents a positive number; illustratively, ∈ is less than 0.1.

[0034] In this embodiment, the calculation formula of the fluency loss L2 is:

[0035] L2=-logP θ (X)

[0036] Among them, L2 is the fluency loss, P θ (X) is the probability of predicting the current true answer X from the current predicted answer obtained by the pre-trained language model.

[0037] This embodiment quickly and effectively determines the current loss value through the current predicted answer and the current true answer output by the pre-trained language model.

[0038] Optionally, based on the current stem, current prompt vector, current relevant facts and pre-trained language model in the current question and answer pair data, a current predicted answer corresponding to the current stem is determined, including: determining a current input vector based on the current stem and current prompt vector in the current question and answer pair data; inputting the current input vector into the pre-trained language model, and outputting current predicted facts corresponding to current relevant facts, so that the pre-trained language model obtains a current predicted answer corresponding to the current stem based on the current predicted facts.

[0039] Specifically, the current question stem can be marked by a marker corresponding to the pre-trained language model to obtain a marker vector corresponding to the current question stem, the marker vector is concatenated with the current prompt vector, the concatenated vector is used as the current input vector, and the current input vector is input into the pre-trained language model. The pre-trained language model refers to the number of current relevant facts and outputs the current predicted facts consistent with the number. In addition, the pre-trained language model combines the current predicted facts and the current input vector to obtain the current predicted answer corresponding to the current question stem.

[0040] This embodiment uses a pre-trained language model to first propose the current prediction facts so as to answer based on the current prediction facts, which is beneficial to improving the accuracy and explainability of the current prediction answer.

[0041] S130. Based on the current gradient and a predetermined preset embedding vector, determine the next prompt vector corresponding to the question-answer pair data of the next training batch, and repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, thereby obtaining the latest prompt vector.

[0042] The preset embedding vector is a discrete prompt vector obtained by the pre-trained language model. That is, a real embedding vector obtained by pre-training corresponding to the pre-trained language model. The number of preset embedding vectors is multiple. The next prompt vector is the prompt vector used when determining the loss value through the question-answer pair data of the next training batch.

[0043] In a specific implementation, a preset embedding vector can be determined from each preset embedding vector based on the current gradient as the next prompt vector. Specifically, the preset embedding vector closest to the current prompt vector in the reverse direction of the current gradient can be determined from each preset embedding vector as the next prompt vector. When the next training batch is traversed as the current training batch, the loss value corresponding to the next training batch can be determined again in the above-mentioned manner of determining the current loss value, and it is judged whether the loss value corresponding to the next training batch meets the preset training target. If so, the next prompt vector is used as the latest prompt vector; if not, other training batches are traversed until the latest loss value obtained last time meets the preset training target, and the prompt vector corresponding to the latest loss value that meets the preset training target is used as the latest prompt vector.

[0044] Optionally, the preset training goal may include convergence of loss values ​​obtained for each training batch; and / or, the current number of training times reaches a preset maximum number of training times threshold.

[0045] S140. Based on the latest prompt vector, the question stem to be answered and the reasoning model to be transferred, determine the response information corresponding to the question stem to be answered.

[0046] In a specific implementation, the latest prompt vector can be used as a trained prompt vector to be migrated to other language reasoning models, so that different language reasoning models can use the latest trained prompt vector to perform different common sense reasoning tasks.

[0047] Optionally, based on the latest prompt vector, the question stem to be answered and the reasoning model to be transferred, the response information corresponding to the question stem to be answered is determined, including: splicing the latest prompt corresponding to the latest prompt vector with the question stem to be answered to obtain a spliced ​​text; inputting the spliced ​​text into the marker corresponding to the reasoning model to be transferred to obtain a splicing vector corresponding to the spliced ​​text; inputting the splicing vector into the reasoning model to be transferred to obtain the response information corresponding to the question stem to be answered.

[0048] The reasoning model to be transferred can be a generative language model, and the stem to be answered is the stem of the actual question. Exemplarily, the stem to be answered can be a judgment type question. It should be noted that different reasoning models to be transferred can be selected for different reasoning tasks and used in combination with the latest prompt vector, so as to accurately determine the response information of the stem to be answered.

[0049] In a specific implementation, based on the marker corresponding to the pre-trained language model, the latest prompt corresponding to the latest prompt vector is determined. The latest prompt can be represented by text. In order to obtain accurate response information, the text obtained by splicing the latest prompt with the question stem to be answered is used as the spliced ​​text, and the spliced ​​text is marked by the marker corresponding to the reasoning model to be transferred to obtain the splicing vector. The splicing vector is input into the reasoning model to be transferred, and the information output by the reasoning model to be transferred is used as the response information corresponding to the question stem to be answered, so that the reasoning model to be transferred, in combination with the latest prompt, obtains the response information used to reflect the reasoning result of the question stem to be answered. Exemplarily, the response information can be yes or no.

[0050] This embodiment combines the latest prompts obtained through training with the reasoning model to be transferred to infer the results of the question to be answered, thereby improving the flexibility of executing reasoning tasks.

[0051] The technical solution of the embodiment of the present invention is to obtain a training data set and a pre-trained language model; wherein the training data set includes question-answer pair data corresponding to at least one batch; for the current training batch, based on the current question-answer pair data of the current training batch, the predetermined current prompt vector and the pre-trained language model, determine the current loss value corresponding to the current prompt vector and the current gradient corresponding to the current loss value; based on the current gradient and the predetermined preset embedding vector, determine the next prompt vector corresponding to the question-answer pair data of the next training batch, so as to repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, and obtain the latest prompt vector; wherein the preset embedding vector is a discrete prompt vector obtained by the pre-trained language model; thereby obtaining the latest prompt vector by updating the current gradient-constrained prompt vector; and determining the response information corresponding to the question stem to be answered based on the latest prompt vector, the question stem to be answered and the inference model to be transferred. The technical solution of this embodiment solves the problem of slow convergence of discrete prompt learning by updating the gradient-constrained prompt vector, and improves the efficiency of determining the prompt vector and the response information.

[0052] Figure 2It is a flowchart of another common sense reasoning method based on language model prompt learning provided in accordance with an embodiment of the present invention. Based on the above embodiment, this embodiment determines an optimized prompt vector corresponding to the current prompt vector based on the current gradient and learning rate optimization algorithm; determines the preset embedding vector corresponding to the optimized prompt vector as the target embedding vector, and determines the target embedding vector as the next prompt vector corresponding to the question-answer pair data of the next training batch. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 2 As shown, the method includes:

[0053] S210: Obtain a training data set and a pre-trained language model.

[0054] The training data set includes at least one batch of question-answer pair data.

[0055] S220. For the current training batch, based on the current question-answer pair data of the current training batch, a predetermined current prompt vector and a pre-trained language model, determine a current loss value corresponding to the current prompt vector and a current gradient corresponding to the current loss value.

[0056] S230. Based on the current gradient and learning rate optimization algorithm, determine the optimized prompt vector corresponding to the current prompt vector; determine the preset embedding vector corresponding to the optimized prompt vector as the target embedding vector, and determine the target embedding vector as the next prompt vector corresponding to the question-answer pair data of the next training batch, so as to repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, thereby obtaining the latest prompt vector.

[0057] The optimized hint vector is a vector obtained by adjusting the current hint vector in the opposite direction of the current gradient.

[0058] In a specific implementation, based on the current gradient and learning rate optimization algorithm, an optimized hint vector corresponding to the current hint vector is determined, and the formula for determining the optimized hint vector is as follows:

[0059]

[0060] Among them, P' represents the optimization prompt vector, P represents the current prompt vector, and γ represents the pre-set learning rate of model training. Represents the current gradient. Exemplarily, the learning rate optimization algorithm may be an adaptive learning rate optimization algorithm (AdaFactor).

[0061] In this embodiment, a preset embedding vector mapped to the optimization hint vector may be determined from the preset embedding vectors as the target embedding vector. Specifically, the implementation method of determining the preset embedding vector corresponding to the optimization hint vector as the target embedding vector may be: sampling each preset embedding vector to obtain at least one sampled embedding vector; and determining the sampled embedding vector corresponding to the optimization hint vector as the target embedding vector based on the direction vector that is the same as the current gradient direction.

[0062] It should be noted that in order to avoid the situation where the line connecting the target embedding vector and the optimization hint vector is perpendicular to the current gradient but too far apart, the preset embedding vector can be sampled first to divide the range and ensure that a reasonable target embedding vector is obtained.

[0063] Optionally, the preset embedding vectors may be sampled using a top-k method, and the sampled preset embedding vectors are referred to as sampled embedding vectors. The specific process of top-k sampling is as follows: from the vector set composed of preset embedding vectors, the first k preset embedding vectors whose directions are closest to the direction of the optimization hint vector are sampled as sampled embedding vectors. Where k is a positive integer.

[0064] In this embodiment, the target embedding vector may be determined as follows:

[0065]

[0066] Where e' represents the target embedding vector; represents the optimization hint vector, e v Represents the sampled embedding vector, E' represents the n-dimensional Euclidean space composed of the sampled embedding vector; where n is the dimension of the sampled embedding vector. Represents the direction vector with the same direction as the current gradient; Indicates that v Unit vectors with the same direction, Representation and Unit vectors with the same direction. argmax is the maximum independent variable point set function.

[0067] In this embodiment, the determined sampled embedding vector corresponding to the optimization hint vector is used as the target embedding vector, so that through the sampling process, it is avoided that the line connecting the target embedding vector and the optimization hint vector is perpendicular to the current gradient but too far away.

[0068] S240. Based on the latest prompt vector, the question stem to be answered and the reasoning model to be transferred, determine the response information corresponding to the question stem to be answered.

[0069] This embodiment uses a gradient-based method to learn discrete prompts, so that the prompts have the advantages of natural language comprehensibility and transferability, while making the prompt acquisition process fully automated, reducing the use of manual labor. In addition, the use of gradient constraints to update discrete prompts overcomes the shortcomings of existing discrete prompt learning, such as slow convergence and poor results.

[0070] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0071] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A common sense reasoning method based on language model prompt learning, characterized in that: include: Obtaining a training data set and a pre-trained language model; wherein the training data set includes at least one batch of question-answer pair data; For a current training batch, based on current question-answer pair data of the current training batch, a predetermined current prompt vector, and the pre-trained language model, determine a current loss value corresponding to the current prompt vector and a current gradient corresponding to the current loss value; Based on the current gradient and the predetermined preset embedding vector, determine the next prompt vector corresponding to the question-answer pair data of the next training batch, so as to repeatedly perform the loss value determination operation based on the next prompt vector until the latest loss value meets the preset training target, thereby obtaining the latest prompt vector; wherein the preset embedding vector is a discrete prompt vector obtained by the pre-trained language model; Based on the latest prompt vector, the question stem to be answered and the reasoning model to be transferred, the response information corresponding to the question stem to be answered is determined.

2. The method according to claim 1, characterized in that The determining, based on the current gradient and a predetermined preset embedding vector, a next prompt vector corresponding to the question-answer pair data of the next training batch includes: Determining an optimized hint vector corresponding to the current hint vector based on the current gradient and learning rate optimization algorithm; A preset embedding vector corresponding to the optimized prompt vector is determined as a target embedding vector, and the target embedding vector is determined as a next prompt vector corresponding to question-answer pair data of a next training batch.

3. The method according to claim 2, characterized in that The step of determining a preset embedding vector corresponding to the optimization hint vector as the target embedding vector comprises: Performing sampling processing on each of the preset embedding vectors to obtain at least one sampled embedding vector; Based on a direction vector that is the same as the current gradient direction, a sample embedding vector corresponding to the optimization hint vector is determined as the target embedding vector.

4. The method according to claim 1, characterized in that: The step of determining response information corresponding to the question stem to be answered based on the latest prompt vector, the question stem to be answered, and the reasoning model to be transferred includes: splicing the latest prompt corresponding to the latest prompt vector with the question stem to be answered to obtain a spliced ​​text; Inputting the concatenated text into a marker corresponding to the inference model to be transferred to obtain a concatenated vector corresponding to the concatenated text; The concatenated vector is input into the reasoning model to be transferred to obtain the response information corresponding to the question to be answered.

5. The method according to claim 1, characterized in that The current question-answer pair data includes a current question stem, at least one current relevant fact, and a current true answer; The determining, based on the current question-answer pair data of the current training batch, a predetermined current prompt vector and the pre-trained language model, a current loss value corresponding to the current prompt vector comprises: For each of the current question-answer pair data, based on the current question stem, the current prompt vector, the current relevant facts and the pre-trained language model in the current question-answer pair data, determine a current predicted answer corresponding to the current question stem; The current loss value is determined based on the current predicted answer and a current true answer corresponding to the current predicted answer.

6. The method according to claim 5, characterized in that The determining, based on the current question stem, the current prompt vector, the current relevant facts and the pre-trained language model in the current question-answer pair data, a current predicted answer corresponding to the current question stem comprises: Determine a current input vector based on the current question stem and the current prompt vector in the current question-answer pair data; The current input vector is input into the pre-trained language model, and the current predicted fact corresponding to the current relevant fact is output, so that the pre-trained language model obtains the current predicted answer corresponding to the current question stem based on the current predicted fact.

Images