Training device, training method, and program

Abstract

A training device according to one aspect of the present disclosure comprises: an input unit that inputs, as training data, first text for additionally training a large-scale language model; a first calculation unit that, on the basis of a first parameter of the large-scale language model, a second parameter added to the first parameter, and the first text, calculates a first generation probability representing the generation probability of second text generated by the large-scale language model; a second calculation unit that, on the basis of the first parameter and the first text, calculates a second generation probability representing the generation probability of the second text; a computation unit that, on the basis of the first generation probability and the second generation probability, computes the value of a predetermined loss function; and an update unit that, on the basis of the value of the loss function, updates the second parameter.

Description

Learning device, learning method, and program

[0001] The present disclosure relates to a learning device, a learning method, and a program.

[0002] Low-Rank Adaptation of Large Language Models (LoRA) is known as a training method for large language models (LLMs) (Non-Patent Document 1). LoRA is a method for efficiently incrementally training LLMs by fixing model parameters of the LLMs and then training parameters represented by a low-rank matrix.

[0003] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, "LoRA: Low-Rank Adaptation of Large Language Models", arXiv:2106.09685 [cs.CL].

[0004] However, there are cases where learning using LoRA does not provide sufficient accuracy for certain tasks (problems).

[0005] The present disclosure has been made in consideration of the above points, and aims to realize additional learning that can acquire a large-scale language model with high accuracy.

[0006] A learning device according to one aspect of the present disclosure includes an input unit that inputs a first text as learning data for additional learning of a large-scale language model; a first calculation unit that calculates a first generation probability representing the generation probability of a second text generated by the large-scale language model based on a first parameter of the large-scale language model, a second parameter added to the first parameter, and the first text; a second calculation unit that calculates a second generation probability representing the generation probability of the second text based on the first parameter and the first text; a calculation unit that calculates the value of a predetermined loss function based on the first generation probability and the second generation probability; and an update unit that updates the second parameter based on the value of the loss function.

[0007] It is possible to realize additional learning that can acquire a large-scale language model with high accuracy.

[0008] Fig. 1 is a diagram showing an example of the hardware configuration of a language processing device during learning; Fig. 2 is a diagram showing an example of the functional configuration of a language processing device during learning; Fig. 3 is a flowchart showing an example of a learning process according to the present embodiment; Fig. 4 is a diagram showing an example of the functional configuration of a language processing device during inference; Fig. 5 is a flowchart showing an example of an inference process according to the present embodiment.

[0009] An embodiment of the present invention will be described in detail below with reference to the drawings. In the following embodiment, a language processing device 10 will be described that acquires an LLM capable of generating sentences with high accuracy even for a specific task through additional learning, and then generates sentences using the LLM. Note that additional learning refers to additional learning of a pre-trained LLM. Additional learning allows the LLM to acquire new knowledge (including cases where existing knowledge is edited, overwritten, updated, or deleted). Additional learning is sometimes called, for example, "fine tuning." For simplicity, additional learning will also be simply referred to as "learning" hereinafter.

[0010] Here, the language processing device 10 according to the present embodiment operates during a "learning" period, during which the LLM is learned using given learning data, and during an "inference" period, during which a sentence corresponding to a given query (e.g., a question) is generated using the trained LLM. Note that the language processing device 10 during learning may be called, for example, a "learning device" or a "training device." Furthermore, the language processing device 10 during inference may be called, for example, an "inference device."

[0011] [During Learning] The following describes the language processing device 10 during learning. Here, texts expressing new knowledge in natural language sentences are provided as learning data to the language processing device 10 during learning.

[0012] <Preparation> First, prepare symbols, terms, etc. For each text, let L denote the number of tokens that make up the text. A token is a partial text that serves as a processing unit when processing text. Examples of tokens that can be used include words, phrases, and subwords.

[0013] Also, V is a vocabulary set, which is a set in which each token is assigned an integer value from 1 to |V|.

[0014] Furthermore, for each text, let i be the index that indicates the position of the token that constitutes the text. Let x be the index that indicates the position of the token that constitutes the text. ｉ Let ∈V.

[0015] The parameters of the LLM are called "model parameters" and are represented by a matrix W. The parameters used when additionally training the LLM using LoRA (Non-Patent Document 1) (i.e., parameters represented by a low-rank matrix) are called "LoRA parameters" and are represented by matrices A and B. As with Non-Patent Document 1, the LoRA parameters are initialized such that matrix A is normal distribution and matrix B is initialized to 0. The model parameters of the LLM are composed of multiple matrices W, but for simplicity, these multiple matrices W will be collectively referred to simply as "W" below.

[0016] Note that W∈R ｄ×ｋThen, B∈R ｄ×ｒ , A∈R ｒ×ｋ (where r<d, r<k). By learning the LoRA parameters A and B, the model parameters of the LLM after learning become W←W+BA.

[0017] <Example of Hardware Configuration of Language Processing Device 10 During Learning> An example of the hardware configuration of the language processing device 10 during learning will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the language processing device 10 during learning.

[0018] 1, the language processing device 10 during learning includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0019] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the language processing device 10 does not necessarily have to include at least one of the input device 101 and the display device 102, for example.

[0020] The external I / F 103 is an interface with an external device such as a recording medium 103a. The language processing device 10 can read from and write to the recording medium 103a via the external I / F 103. Examples of the recording medium 103a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0021] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a storage device (storage device) such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is an arithmetic device such as a central processing unit (CPU) or a graphics processing unit (GPU).

[0022] The language processing device 10 during learning can realize the learning process described below by having the hardware configuration shown in Fig. 1. Note that the hardware configuration shown in Fig. 1 is an example, and the hardware configuration of the language processing device 10 during learning is not limited to this. The language processing device 10 during learning may have, for example, multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0023] <Example of functional configuration of language processing device 10 during learning> An example of the functional configuration of the language processing device 10 during learning will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the functional configuration of the language processing device 10 during learning.

[0024] As shown in FIG. 2 , the language processing device 10 during training includes an input unit 201, a language encoding unit 202, a loss calculation unit 203, a parameter update unit 204, and a termination condition determination unit 205. Each of these units is realized, for example, by a process in which one or more programs installed in the language processing device 10 are executed by the processor 108 or the like. The language processing device 10 during training also includes a training data storage unit 301 and a parameter storage unit 302. Each of these storage units is realized, for example, by a storage area of ​​the auxiliary storage device 107 or the like. Note that, for example, at least one of the training data storage unit 301 and the parameter storage unit 302 may be realized by a storage area of ​​a storage device (e.g., a storage device included in a database server) communicatively connected to the language processing device 10.

[0025] The input unit 201 inputs text stored as learning data in the learning data storage unit 301 .

[0026] The language encoding unit 202 calculates a first text generation probability by the encoding layer included in the LLM using the model parameter W stored in the parameter storage unit 302, the LoRA parameters A and B stored in the parameter storage unit 302, and the text input by the input unit 201. That is, the language encoding unit 202 calculates a first text generation probability by the encoding layer included in the LLM from the text input by the input unit 201 using W+BA as the model parameter.

[0027] Furthermore, the language encoding unit 202 calculates the second text generation probability by the encoding layer included in the LLM using the model parameter W stored in the parameter storage unit 302 and the text input by the input unit 201. That is, the language encoding unit 202 calculates the second text generation probability by the encoding layer included in the LLM from the text input by the input unit 201 using only the model parameter W without using the LoRA parameters A and B.

[0028] The coding layer included in the LLM is a layer that sequentially inputs tokens included in a text, encodes them into a coded sequence, and calculates the probability of generating the i-th token from the coded sequence based on the i-1th tokens. The coding layer includes a machine learning model such as a neural network. For example, a Transformer (Reference 1) is used as the coding layer of the LLM. The coding layer may also be called, for example, an "encoder," an "encoder," or a "decoder." However, it goes without saying that the Transformer is just one example of a coding layer and is not limited to this.

[0029] Hereinafter, the first text generation probability is defined as p １ｉ (i=1,...,L), the second text generation probability is p ２ｉ (i=1, ..., L). The first text generation probability p １ｉ and p ２ｉ is a |V|-dimensional vector that represents the distribution of the probability that a token t∈V is generated as the i-th token when the i-1th tokens contained in the text are given. Hereinafter, the first text generation probability p １ｉ The t-th element of p １ｉ，ｔ , the second text generation probability p ２ｉ The t-th element of p ２ｉ，ｔ Let's say.

[0030] The loss calculation unit 203 calculates the first text generation probability p １ｉ and the second text generation probability p ２ｉ The value of the loss function Loss shown in the following equation (1) is calculated using

[0031] Here, q(p ２ｉ , x ｉ ) ｔ is a function q(p ２ｉ , x ｉ ) and is defined by the following equation (2). ｉ is the i-th token (correct token) included in the text input by the input unit 201.

[0032] In this way, the loss function Loss is the function q(p ２ｉ , x ｉ ) the first text generation probability p １ｉ Therefore, the first text generation probability p １ｉ function q(p ２ｉ , x ｉ ) output.

[0033] On the other hand, the token generated by the LLM greedily without using the LoRA parameters A and B has a second text generation probability p ２ｉ The token that maximizes ｍａｘ ) is expressed as t ｍａｘ = x ｉ If so, even without LoRA parameters A and B, LLM can obtain token x ｉ Since we can generate token x ｉ It can be said that learning about q(p ２ｉ , x ｉ ) ｔ By the definition of t ｍａｘ = x ｉ When t∈V, q(p ２ｉ , x ｉ ) ｔ = p ２ｉ，ｔ Therefore, at this time, the first text generation probability p １ｉ is the second text generation probability p ２ｉ Learning is carried out to approach

[0034] Therefore, by using the loss function Loss shown in the above formula (1), the token x ｉ If learning about is not required, learning is performed to keep the output of the original LLM, and as a result, the token x that needs additional learning is ｉ This allows LLMs to effectively obtain information about

[0035] In addition, in existing learning methods for LLM (including existing learning by LoRA), the tokens constituting the text given as learning data are treated equally, and parameters are learned using cross entropy error or the like. For this reason, it can be said that it is not possible to consider which information about a token is additional information required for LLM, and this is thought to be one of the reasons why sufficient accuracy cannot be achieved in certain tasks. In contrast, by using the loss function Loss shown in the above formula (1), it is possible to learn the parameters of the token x ｉ If learning about is not required, learning is performed to keep the output of the original LLM, so the token x that needs additional learning ｉ The LLM can effectively acquire information about the LoRA parameters A and B. Since the LoRA parameters A and B are low-rank matrices, the number of parameters that can be learned is limited, while the token x that needs to be additionally learned is ｉ It becomes possible to efficiently learn information about.

[0036] Parameter update unit 204 updates LoRA parameters A and B stored in parameter storage unit 302 using the loss function value calculated by loss calculation unit 203. Note that the LoRA parameters A and B may be updated using a method similar to that used in existing LoRA learning.

[0037] Furthermore, when termination condition determination section 205 determines that the termination condition is satisfied, parameter update section 204 integrates LoRA parameters A and B into model parameter W. That is, parameter update section 204 updates model parameter W by W←W+BA.

[0038] The termination condition determination unit 205 determines whether a predetermined termination condition is satisfied, such as when the LoRA parameters A and B have been updated a predetermined number of times or more, when the values ​​of the LoRA parameters A and B have converged, or when the value of the loss function Loss has converged.

[0039] The training data storage unit 301 stores the training data provided to it. The training data is text representing natural language sentences.

[0040] The parameter storage unit 302 stores the model parameter W and the LoRA parameters A and B. The model parameter W is a learnable parameter of the coding layer included in the LLM.

[0041] <Example of Learning Process> An example of the learning process according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the learning process according to this embodiment.

[0042] The input unit 201 inputs text stored as learning data in the learning data storage unit 301 (step S101). For example, the input unit 201 may input learning data randomly from the learning data storage unit 301, or may input learning data that has not yet been input from the learning data stored in the learning data storage unit 301. Hereinafter, the text input in this step is referred to as (x １ ...x Ｌ ) where x ｉ is the i-th token (i-th correct token).

[0043] The language encoding unit 202 uses W+BA as a model parameter to encode the text (x １ ...x Ｌ ) from the coding layer included in the LLM to obtain the first text generation probability p １ｉ (i = 1, ..., L) is calculated (step S102). When W + BA is used as a model parameter, ΔWx = BAx is scaled with respect to the input x using the parameter α, as in the existing LoRA (see Non-Patent Document 1 for details).

[0044] The language encoding unit 202 uses the model parameter W to encode the text (x １ ...x Ｌ ) to obtain the second text generation probability p ２ｉ (i=1, . . . , L) is calculated (step S103).

[0045] The loss calculation unit 203 calculates the first text generation probability p １ｉ(i=1, . . . , L), and the second text generation probability p calculated in step S103 above. ２ｉ (i=1, . . . , L), the value of the loss function Loss shown in the above formula (1) is calculated (step S104). ２ｉ , x ｉ ) uses a function where t∈V-th element is defined by the above equation (2).

[0046] The parameter update unit 204 uses the value of the loss function Loss calculated in step S104 to update the LoRA parameters A and B stored in the parameter storage unit 302 (step S105). That is, the parameter update unit 204 uses an existing optimization technique to update the LoRA parameters A and B by an existing parameter update method so as to minimize the value of the loss function Loss.

[0047] The termination condition determination unit 205 determines whether a predetermined termination condition is satisfied (step S106).

[0048] If it is not determined in step S106 that the predetermined termination condition is satisfied, the language processing device 10 returns to step S101, whereby steps S101 to S106 are executed again.

[0049] On the other hand, if it is determined in step S106 that the predetermined termination condition is satisfied, the parameter update unit 204 integrates the LoRA parameters A and B into the model parameter W (step S107). That is, the parameter update unit 204 updates the model parameter W by W←W+BA. This results in the LLM after learning.

[0050] [During inference] The following describes the language processing device 10 during inference. Here, during inference, text expressing a question or the like in natural language sentences is provided as a query to the language processing device 10. Note that, hereinafter, the model parameters W of the LLM after learning will be referred to as "trained model parameters W." Furthermore, hereinafter, differences from the learning period will be mainly described, and descriptions of components that are the same as those during learning will be omitted.

[0051] <Example of Functional Configuration of Language Processing Device 10 During Inference> An example of the functional configuration of the language processing device 10 during inference will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the functional configuration of the language processing device 10 during inference.

[0052] As shown in Figure 4, the language processing device 10 at the time of inference has an input unit 201, a language encoding unit 202, a text generation unit 206, and an output unit 207. Each of these units is realized, for example, by a process in which one or more programs installed in the language processing device 10 are executed by the processor 108 or the like. The language processing device 10 at the time of inference also has a learned parameter storage unit 303. The learned parameter storage unit 303 is realized, for example, by a storage area of ​​the auxiliary storage device 107 or the like. Note that, for example, the learned parameter storage unit 303 may be realized by a storage area of ​​a storage device (e.g., a storage device included in a database server) communicatively connected to the language processing device 10.

[0053] The input unit 201 receives text given as a query.

[0054] The language encoding unit 202 calculates the text generation probability by the coding layer included in the LLM using the trained model parameters W stored in the trained parameter storage unit 303 and the text input by the input unit 201. That is, the language encoding unit 202 calculates the text generation probability by the coding layer included in the LLM from the text input by the input unit 201 using the trained model parameters W. Hereinafter, the text generation probability will be referred to as p ｉ The text generation probability p ｉ is a |V|-dimensional vector that represents the distribution of the probability that a token tεV is generated as the i-th token when tokens up to the i−1-th token are given.

[0055] The text generator 206 uses the text generation probability p ｉ For example, the text generator 206 generates the i-th token using the text generation probability p ｉThe i-th token is sampled from the vocabulary set V according to the probability distribution represented by (i) where V is the number of tokens in the vocabulary set V. This results in a text composed of each token (hereinafter also referred to as "generated text").

[0056] The output unit 207 outputs the generated text composed of the tokens generated by the text generation unit 206 to a predetermined output destination. The predetermined output destination is not limited to a specific output destination, but examples thereof include the display device 102 such as a display, a storage area such as the auxiliary storage device 107, a program, other devices or equipment connected in a communicable manner, etc.

[0057] The learned parameter storage unit 303 stores the learned model parameters W. Note that the learned model parameters W are learned parameters of the coding layer included in the LLM.

[0058] <Example of Inference Processing> An example of the inference processing according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the inference processing according to this embodiment.

[0059] The input unit 201 inputs a text given as a query (step S201). A specific example of the text given as a query is, for example, "Who is the president of Company C?". Hereinafter, the text input in this step is referred to as (x １ ...x Ｌ' ) where x ｉ is the i-th token.

[0060] The language encoding unit 202 uses the trained model parameters W stored in the trained parameter storage unit 303 and the text input in step S201 to generate a text generation probability p ｉ That is, the language encoding unit 202 calculates the text generation probability p ∈ V, which represents the distribution of the probability that a token t∈V will be generated as the i-th token when tokens up to i-1 are given by the encoding layer included in the LLM, using the trained model parameter W. ｉ where i≧L′+1.

[0061] The text generation unit 206 uses the text generation probability p ｉ The i-th token is generated using y (step S203). Hereinafter, the token generated in this step is referred to as y ｉ Let's say.

[0062] The above steps S202 and S203 are repeatedly executed until a token representing the end of a sentence is generated in step S203. Ｌ'＋１ ...y Ｌ A specific example of generated text is "It's Mr. D" when the query is "Who is the president of Company C?". Ｌ is a token that represents the end of a sentence.

[0063] The output unit 207 outputs the generated text made up of the tokens generated in step S203 to a predetermined output destination (step S204).

[0064] <Experimental Example> Below, we will explain the results of an experiment that used a dataset called Counterfact (Reference 2) to compare the language processing device 10 according to this embodiment with LoRA, an existing method. Counterfact is a typical dataset used in knowledge compilation. In Counterfact, o' is prepared for a knowledge triple (s, r, o). s represents subject, r represents relation, and o represents object, respectively.

[0065] In this case, knowledge triple (s, r, o) was assumed to be existing knowledge and compiled into (s, r, o'), and learning was performed using the text converted from (s, r, o') as training data. Here, an example of the text converted from (s, r, o') is "The president of Company C is D" when (s, r, o') = (Company C, President, D).

[0066] The evaluation was carried out using the following three evaluation indices (ability to memorize, ability to generalize, and ability to distinguish from neighboring items) in a problem of guessing o'.

[0067] - Memorization ability Memorization ability is the ability to correctly guess o' from the text used in the study. For example, in response to the query "What goes in X in the following sentence? The president of Company C is X," it is the ability to correctly guess "It's D."

[0068] Generalization ability is the ability to correctly guess o' from another text converted from (s, r, o'). For example, in response to the query "What goes in for X in the following sentence? X is the president of Company C," it is the ability to correctly guess "It's D."

[0069] Ability to distinguish from neighbors Ability to distinguish from neighbors refers to the ability to correctly guess o'' from the text converted from (s'', r, o''), which is a neighbor of (s, r, o'). For example, in response to the query "Please answer what goes in X in the following sentence. The president of C' Company is X.", when the president of C' Company is D', it refers to the ability to correctly guess "It is D'."

[0070] The model parameters were learned using 21,919 pieces of data included in Counterfact as learning data, and the problem of estimating o' was evaluated using the above three evaluation indices. The results are shown in Table 1 below.

[0071] In Table 1, LoRA indicates the existing method, and Proposed indicates the proposed method (the inference process executed by the language processing device 10 according to this embodiment). In both the existing method and Proposed, the LoRA parameters A and B are fixed as a matrix with rank = 16, and the learning rate for updating the LoRA parameters A and B is 10 －３ , 5 x 10 －４ , 10 －４ Furthermore, when calculating (W + ΔW)x for input x, the parameter α used for the scale of ΔWx = BAx was chosen to be the best out of 16, 32, and 64.

[0072] As shown in Table 1 above, the proposed method receives better evaluations on average than existing methods. In particular, the proposed method receives better evaluations than existing methods in terms of its ability to distinguish between neighboring words. This is thought to be because existing methods are strongly affected by knowledge editing, as they do not distinguish between conversions that do not change meaning (e.g., "The president of Company C is D." to "D is the president of Company C.") and conversions that do change meaning (e.g., "The president of Company C is D." to "The president of Company C' is D'."). In contrast, the proposed method can efficiently embed only the necessary information into the parameters, which is thought to keep the impact of knowledge editing within an appropriate range.

[0073] Therefore, in tasks that require the ability to distinguish between neighboring words, the language processing device 10 according to this embodiment is expected to achieve particularly high performance compared to existing methods.

[0074] <Modifications> Modifications of the above embodiment will be described below. Note that it is possible to combine multiple of the following modifications as long as they do not contradict each other.

[0075] Modification 1: The function q(p ２ｉ , x ｉ ) ｔ The definition of t is an example. ｍａｘ = x ｉ When (i.e., p ２ｉ，ｔ The token t that maximizes ｍａｘ Ga x ｉ , for all t∈V, q(p ２ｉ , x ｉ ) ｔ = p ２ｉ，ｔ Any function can be used as long as it has a function such that the t-th element is a function such that

[0076] Variation 2 In the above embodiment, it is assumed that the language processing device 10 at the time of learning and the language processing device 10 at the time of inference are the same device, but the language processing device 10 at the time of learning and the language processing device 10 at the time of inference may be different devices.

[0077] Modification 3 In the above embodiment, LoRA is used as one of the efficient learning methods using parameter addition, but an efficient learning method using parameter addition other than LoRA may also be used. Efficient learning methods using parameter addition, including LoRA, are collectively called PEFT (Parameter-efficient-fine-tuning) (Reference 3). In other words, the above embodiment can be similarly applied to PEFT other than LoRA.

[0078] <Summary> As described above, in the language processing device 10 according to this embodiment, when additionally learning an LLM using LoRA, the LoRA parameters A and B are learned based on the text generation probability when the LoRA parameters A and B are used and the text generation probability when the LoRA parameters A and B are not used. This makes it possible to efficiently embed only the necessary information into the LoRA parameters A and B, and to keep the impact of knowledge editing within an appropriate range.

[0079] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0080] [References] Reference 1: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, "Attention Is All You Need", arXiv:1706.03762 [cs.CL]. Reference 2: Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov, "Locating and Editing Factual Associations in GPT", arXiv:2202.05262 [cs.CL]. Reference 3: https: / / github.com / huggingface / peft

[0081] REFERENCE SIGNS LIST 10 Language processing device 101 Input device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage device 108 Processor 109 Bus 201 Input unit 202 Language encoding unit 203 Loss calculation unit 204 Parameter update unit 205 Termination condition determination unit 206 Text generation unit 207 Output unit 301 Learning data storage unit 302 Parameter storage unit 303 Learned parameter storage unit

Claims

1. A learning device comprising: an input unit that inputs a first text as training data for additional training of a large-scale language model; a first calculation unit that calculates a first generation probability representing the generation probability of a second text generated by the large-scale language model based on a first parameter of the large-scale language model, a second parameter to be added to the first parameter, and the first text; a second calculation unit that calculates a second generation probability representing the generation probability of the second text based on the first parameter and the first text; a calculation unit that calculates the value of a predetermined loss function based on the first generation probability and the second generation probability; and an update unit that updates the second parameter based on the value of the loss function.

2. The learning device described in claim 1, wherein the first text is composed of correct answer partial text for each predetermined processing unit, the first generation probability and the second generation probability are generation probabilities of partial text for each predetermined processing unit of the second text, and the calculation unit uses a function that outputs the second generation probability when a correct answer partial text is generated using the second generation probability for each processing unit of the second text, and calculates the KL-divergence value between the function and the first generation probability as the value of the loss function.

3. Let L be the total number of processing units contained in the first text, and x be the correct partial text of the i-th processing unit. ｉ , the first generation probability and the second generation probability representing that the t-th type of partial text is generated in the i-th processing unit are respectively expressed as p １ｉ，ｔ and p ２ｉ，ｔ , p ２ｉ，ｔ Let p be the vector whose t-th element is ２ｉ , the t-th element of the output of the function is q(p ２ｉ , x ｉ ) ｔ Then, the loss function is q(p ２ｉ , x ｉ ) ｔ logp １ｉ，ｔ 3. The learning device according to claim 2, wherein the sum of i and t is multiplied by −1 / L.

4. The t-th element q(p ２ｉ , x ｉ ) ｔ is p ２ｉ，ｔ Take the maximum value of t = t ｍａｘ Ga x ｉ If it matches, the maximum value is taken, and t=t ｍａｘ Ga x ｉ If it does not match, ２ｉ，ｔ × (1 - the maximum value) / (1 - (t = x ｉ p ２ｉ，ｔ 4. The learning device according to claim 3, wherein 5. A learning device according to any one of claims 1 to 4, wherein the second parameter is represented by the product of two matrices of lower rank than the matrix representing the first parameter.

6. A learning method performed by a computer, comprising: an input step of inputting a first text as training data for additional training of a large-scale language model; a first calculation step of calculating a first generation probability representing the generation probability of a second text generated by the large-scale language model based on a first parameter of the large-scale language model, a second parameter added to the first parameter, and the first text; a second calculation step of calculating a second generation probability representing the generation probability of the second text based on the first parameter and the first text; a calculation step of calculating the value of a predetermined loss function based on the first generation probability and the second generation probability; and an update step of updating the second parameter based on the value of the loss function.

7. A program that causes a computer to execute the following steps: an input step of inputting a first text as training data for additional training of a large-scale language model; a first calculation step of calculating a first generation probability representing the generation probability of a second text generated by the large-scale language model based on a first parameter of the large-scale language model, a second parameter added to the first parameter, and the first text; a second calculation step of calculating a second generation probability representing the generation probability of the second text based on the first parameter and the first text; a calculation step of calculating the value of a predetermined loss function based on the first generation probability and the second generation probability; and an update step of updating the second parameter based on the value of the loss function.

Images