Language model generation device and language model generation method

Abstract

To achieve learning of a language model of region specialization without requiring a large-scale calculation resource.SOLUTION: An small-scale language model (SLM) learning part 10 for learning an SLM added to a learned large-scale language model (LLM) using a sentence data set not limited to a specific region includes a first model learning part 11 for performing learning for region specialization using a sentence data set limited to the specific region so that a sentence output from the SLM reflects knowledge of the specific region, and learns the SLM having a scale smaller than the versatilely learned LLM for region specialization, provides a structure enabling inference using the learned SLM together with the LLM, and thereby achieves learning of a language model of region specialization without requiring a large-scale calculation resource.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a language model generating device and a language model generating method, and in particular to a technique for generating a small language model to be added to a large language model that has already been trained using a text data set that is not limited to a specific domain. [Background technology]

[0002] Conventionally, systems have been provided that use large-scale language models (LLMs) to perform natural language processing such as question-answering. LLMs are natural language processing models trained using large amounts of text data, and take sentences as input and output sentences. When LLMs are applied to a system that performs question-answering, when a question is input to the LLM, the LLM outputs an answer.

[0003] In recent years, the use of LLM has progressed, but to train a large language model, expensive GPU computing resources of more than hundreds of units are required. Therefore, the focus has been on developing a "general-purpose model" that can be used for anything by learning a wide range of data. On the other hand, a "specialized model" that is trained based on sentences in a specific domain is expected to perform better in that specific domain, and there are examples of attempts to do this (e.g., Bloomberg GPT). "Domain specialization," which learns sentences in a specific domain and embeds that knowledge, is called Knowledge-Injection. Non-Patent Document 1 discloses a method for additional learning of this domain specialization. However, both models require huge investments of hundreds of millions of yen, and have not yet been widely adopted.

[0004] One method to reduce the amount of calculations is to start from a large-scale pre-trained model (Pre-Trained Language Model: PLM) that has been trained for general purposes, and then fine-tune it by adding a small model. A method called LoRA (Low-Rank Adaptation) has also been developed to further reduce the amount of calculations and memory consumption. However, these methods have mainly been used for "task specialization," such as changing the output format of PLMs, such as outputting colloquial language instead of formal language, or outputting clear conversational language instead of long sentences. [Prior art documents] [Patent documents]

[0005] [Non-Patent Document 1] “Plug-and-Play Knowledge Injection for Pre-trained Language Models” (ACL Rolling Review - June 2022, 2022.6.6 (modified:2023.5.6)<URL:https: / / openreview.net / forum?id=f4RClsuGaHS> Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in view of the above circumstances, and has an object to make it possible to realize learning of a domain-specific language model without requiring large-scale computational resources. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention trains a small-scale language model to be added to an already-trained large-scale language model using a sentence data set that is not limited to a specific domain, and trains for domain specialization using a sentence data set that is limited to a specific domain so that output sentences from the small-scale language model reflect knowledge of the specific domain, and trains for task specialization related to expression form adjustment using a sentence data set consisting of input / output pairs of sentences so that output sentences from the small-scale language model have a desired expression form. Effect of the Invention

[0008] According to the present invention configured as described above, a small-scale language model smaller than a large-scale language model trained generically using a text data set not limited to a specific domain is trained for domain specialization, and a configuration is provided in which the trained small-scale language model can be used together with the large-scale language model for inference, so that learning of a domain-specific language model can be realized without requiring large-scale computational resources. Furthermore, in the present invention, training for task specialization related to morphological adjustment is also performed on the small-scale language model, so that learning of domain-specific and task-specific language models can be realized without requiring large-scale computational resources. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of a functional configuration of a language model generating device according to an embodiment of the present invention; [Diagram 2] FIG. 11 is a block diagram showing another example of the functional configuration of the language model generation device according to the present embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of text data generated by a data set generating unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an example of a functional configuration of a language model generation device 1 according to this embodiment. The language model generation device 1 of this embodiment includes a small-scale language model learning unit 10 as a functional configuration. Hereinafter, the small-scale language model learning unit 10 is abbreviated as "SLM learning unit 10" (SLM: Small Language Models). The SLM learning unit 10 includes a first model learning unit 11, a second model learning unit 12, and a model integration unit 13 as specific functional configurations.

[0011] The functional blocks 11 to 13 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, the functional blocks 11 to 13 are realized by the operation of a program stored in a storage medium such as a RAM, ROM, hard disk, or semiconductor memory under the control of a microcomputer configured with a CPU, RAM, ROM, etc. Instead of or in addition to the CPU, a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), DSP, etc. may be used.

[0012] Moreover, the language model generating device 1 of this embodiment is connected to storage units 100-104 as storage media. The LLM storage unit 100 stores a large-scale language model (LLM) that has been trained using a text data set that is not limited to a specific domain. This LLM can be trained using known techniques. Since the training of the LLM is not the subject of this embodiment, a detailed explanation will be omitted. The SLM training unit 10 of this embodiment uses this generically trained LLM as a pre-trained model (PLM) and performs training by applying perturbations to the intermediate output of the PLM.

[0013] The specific domain data storage unit 101 stores a text data set limited to a specific domain. For example, the specific domain data storage unit 101 stores a plurality of text data recording specialized texts in a specific domain. As an example, text data collected from various resources such as books, websites, papers, and patent documents can be used. The specific domain may be one or more. The language model generation device 1 of this embodiment generates an SLM specialized for each specific domain by performing learning by the SLM learning unit 10 for each specific domain.

[0014] Generally, text data collected from books, websites, papers, patent documents, etc. may contain non-character data such as blanks, line breaks, or HTML tags. Therefore, text data from which such noise data has been removed in advance may be stored in the specific area data storage unit 101. For example, noise data can be removed by a programming method such as regular expressions. Alternatively, noise data may be removed by using an LLM or SLM created for noise data removal, or a well-known NLP (natural language processing) method.

[0015] Also, new sentence data may be generated based on sentence data collected from the various resources described above, and this new sentence data may also be stored in the specific domain data storage unit 101. For example, by using an LLM (which may be stored in the LLM storage unit 100 or may be other), sentences that summarize the original sentences read from the specific domain data storage unit 101, other sentences with the same meaning as the original sentences, or sentence questions that test new knowledge from the original sentences, etc., and adding and storing these sentence data in the specific domain data storage unit 101, it is possible to pad the sentence data for learning. Note that only the original sentences may be stored in the specific domain data storage unit 101, and new sentence data may be generated when the SLM learning unit 10 performs learning.

[0016] The input / output pair data storage unit 102 stores a sentence data set as teacher data consisting of a pair of an input sentence and an output sentence. For example, the input / output pair data storage unit 102 stores a sentence data set consisting of a pair of an output sentence written in a desired expression form and an input sentence written in an expression form other than the desired expression form. As an example, a plurality of sentence data sets consisting of a pair of an input sentence written in a formal style and an output sentence written in a colloquial style (desired expression form) are stored. Alternatively, a plurality of sentence data sets consisting of a pair of an input sentence with a large number of characters and an output sentence with a small number of characters (desired expression form) that summarizes the input sentence may be stored.

[0017] The sentence data set consisting of these input / output pair sentences may be created manually or automatically generated by a computer. In the case of automatic generation, for example, it is possible to generate a sentence data set by obtaining input / output pairs from dialogue history data that records past dialogues between a user and an LLM. In this case, the output sentences of the automatically generated input / output pair sentences are set to be sentences consisting of a desired expression form.

[0018] The SLM learning unit 10 of the language model generating device 1 learns a small language model (SLM) that is a language model to be added to the LLM stored in the LLM storage unit 100 and is smaller in scale than the LLM, using a sentence dataset limited to a specific domain stored in the specific domain data storage unit 101 and a sentence dataset consisting of input / output pair sentences stored in the input / output pair data storage unit 102. In this embodiment, LoRA is used as an example of an SLM to reduce the amount of calculation and memory consumption. LoRA is an SLM configured to be able to calculate differential parameters for fine tuning without updating the original parameters of the base LLM by adopting a mechanism for inserting a learnable rank decomposition matrix (parameters) into a layer of a language model called Transformer.

[0019] Here, the SLM learning unit 10 performs learning for domain specialization using a sentence data set limited to a specific domain so that output sentences from the SLM (LoRA) reflect knowledge of the specific domain, and also performs learning for task specialization related to expression form adjustment using a sentence data set consisting of input / output pair sentences so that output sentences from the SLM (LoRA) have a desired expression form. In this embodiment, the SLM learning unit 10 performs additional learning for task specialization after performing learning for domain specialization, and stores the SLMs generated by each learning in the high-rank model storage unit 103 and the low-rank model storage unit 104, respectively.

[0020] Specifically, the SLM learning unit 10 performs two-stage learning using a first model learning unit 11 and a second model learning unit 12. The first model learning unit 11 performs self-supervised learning for domain specialization using a sentence data set limited to a specific domain stored in the specific domain data storage unit 101, thereby generating a small-scale domain-specific SLM. In the field of language models, self-supervised learning is generally a learning method in which a sentence is given to the language model to be learned and the model predicts the next word to appear or the word to fill the blank. The first model learning unit 11 performs this type of self-supervised learning.

[0021] At this time, the first model learning unit 11 sets the rank R of LoRA to a large value. Setting the rank R to a large value means setting a rank larger than the rank R (for example, R=2 to 16) generally set when learning LoRA, which has been conventionally used for task specialization. By doing so, it is possible to make it easier for LoRA to learn domain-specific knowledge. For example, R>16 is sufficient, but empirically, if the rank R is set in the range of R>64, the degree of knowledge learning will be large, and the larger it is, the greater the knowledge learning ability will be. However, if the rank R is set too large, the scale of the model will become large, and the amount of calculation and memory consumption will increase, so it needs to be adjusted to an appropriate value. Hereinafter, LoRA learned with the rank R set to a large value will be referred to as "high rank LoRA". Note that the numerical value of the rank R given here should be adjusted according to the type of data and model, and is not limited to the above value.

[0022] By using the LLM stored in the LLM storage unit 100 as the PLM and combining it with a domain-specific SLM trained by the first model learning unit 11, it is possible to construct an LLM with specialized knowledge in a specific domain. However, this LLM is good at answering the following words one after another when a certain sentence is given, but it has not been trained to output sentences in a specific form of expression. For example, it has not been constructed to receive a human question as input and output an appropriate answer in a natural conversational format.

[0023] In contrast, the second model learning unit 12 performs supervised learning for task specialization using a sentence dataset consisting of sentences of input / output pairs stored in the input / output pair data storage unit 102, thereby generating a task-specific SLM smaller in scale than the domain-specific SLM generated by the first model learning unit 11. In learning by the second model learning unit 12, in order to further reduce the amount of calculation and memory consumption, the rank R of LoRA is set smaller than that of High rank LoRA. For example, R≦16 is possible. Hereinafter, LoRA learned with a small rank R is referred to as "Low rank LoRA". Note that the numerical value of rank R given here should be adjusted according to the type of data and model, and is not limited to the above value.

[0024] As described above, the first model learning unit 11 constructs a domain-specific high-rank LoRA with a large rank R as a fine-tuning model for the LLM (PLM) stored in the LLM storage unit 100. The second model learning unit 12 constructs a task-specific low-rank LoRA with a small rank R as a fine-tuning model for the combination of PLM and high-rank LoRA. Therefore, three modules appear: the PLM stored in the LLM storage unit 100, the high-rank LoRA stored in the high-rank model storage unit 103, and the low-rank LoRA stored in the low-rank model storage unit 104. These modules can be regarded as a matrix of parameter groups, and therefore can be arbitrarily added and integrated.

[0025] As shown in FIG. 1(a), for example, the model integration unit 13 integrates the domain-specific high-rank LoRA generated by the first model learning unit 11 and stored in the high-rank model storage unit 103 with the task-specific low-rank LoRA generated by the second model learning unit 12 and stored in the low-rank model storage unit 104 (it does not integrate the PLM in the LLM storage unit 100). Hereinafter, the LoRA generated by integrating the domain-specific high-rank LoRA and the task-specific low-rank LoRA is referred to as the "domain- and task-specific LoRA". When the domain- and task-specific LoRA is generated, matrix calculations are performed in the PLM and the domain- and task-specific LoRA during inference, and the results of the calculations are added together to obtain an output. By generating domain- and task-specific LoRA for each of multiple specific domains, it is possible to quickly switch between LoRAs with specialized knowledge for inference, which has the advantage of efficient use of computational resources.

[0026] 1(b), the model integration unit 13 may integrate the LLM (PLM) stored in the LLM storage unit 100 with at least one of the domain-specific high-rank LoRA generated by the first model learning unit 11 and stored in the high-rank model storage unit 103 and the task-specific low-rank LoRA generated by the second model learning unit 12 and stored in the low-rank model storage unit 104. That is, the PLM may be integrated with the high-rank LoRA, the PLM may be integrated with the low-rank LoRA, or the PLM may be integrated with the high-rank LoRA and the low-rank LoRA.

[0027] For example, if a PLM, a high-rank LoRA, and a low-rank LoRA are integrated to construct a single LLM, inference can be performed with minimal computational effort and memory consumption, since only matrix calculations are required in the LLM. However, in this case, the two LoRAs cannot be removed from the LLM, making it difficult to dynamically switch areas of expertise during inference.

[0028] In addition, when PLM and high-rank LoRA are integrated while low-rank LoRA is separated, task-specific low-rank LoRA can be generated for each of multiple expression forms, allowing inference to be performed by quickly switching between the desired expression forms in a specific specialized field. For example, in order to utilize the property of low-rank LoRA that adjusts the expression form of output text for the purpose of "preserving the taste of conversation for each user," it is possible to learn low-rank LoRA for each user and separate and store low-rank LoRA so that it can be switched appropriately according to the user during inference.

[0029] Although the configuration including the model integration unit 13 has been described here, the model integration unit 13 may be omitted, and the PLM, the high rank LoRA, and the low rank LoRA may be kept separately. In this case, by generating a domain-specific high rank LoRA for each of a plurality of specific domains and a task-specific low rank LoRA for each of a plurality of expression forms, it is possible to perform inference by quickly switching between combinations of a desired specific domain and a desired expression form.

[0030] Also, in the above embodiment, it has been described that a sentence dataset consisting of input / output pair sentences to be used as supervised data in the supervised learning of the second model learning unit 12 may be automatically generated. In this regard, as shown in Fig. 2, a dataset generating unit 14 may be further provided, and input / output pair sentence data may be generated from sentence data in a specific domain used by the first model learning unit 11 in the self-supervised learning, using a method such as LLM or NLP.

[0031] The dataset generation unit 14 generates a sentence dataset consisting of sentences of input / output pairs reflecting knowledge of the specific domain, using a sentence dataset limited to the specific domain used when the first model learning unit 11 generates a domain-specific SLM (high rank LoRA). Figure 3 is a diagram showing an example of generating sentence data consisting of sentences of input / output pairs using sentence data of a specific domain. Figure 3 shows a state in which three sentences of input / output pairs are generated from sentences of one specific domain.

[0032] In the configuration of Figure 2, the second model learning unit 2 performs supervised learning for task specialization using a sentence dataset generated by the dataset generation unit 14 in addition to the sentence dataset stored in the input / output pair data storage unit 102.

[0033] As described in detail above, in this embodiment, when training an SLM to be added to a large-scale PLM that has already been trained using a sentence data set that is not limited to a specific domain, domain-specific training is performed using a sentence data set that is limited to a specific domain so that the output sentences from the SLM reflect knowledge of the specific domain, and task-specific training related to morphology adjustment is performed using a sentence data set consisting of input / output pairs of sentences so that the output sentences from the SLM have the desired morphology.

[0034] According to the present embodiment configured as described above, a smaller SLM is trained for domain specialization than a large-scale PLM trained generically using a text data set not limited to a specific domain, and a configuration is provided in which the trained SLM can be used together with the PLM for inference, so that learning of a domain-specific language model can be realized without requiring large-scale computational resources. Furthermore, in this embodiment, the SLM is also trained for task specialization related to morphological adjustment, so that learning of domain-specific and task-specific language models can be realized without requiring large-scale computational resources.

[0035] In the above embodiment, an example has been described in which the domain-specific SLM is configured with a high rank LoRA and the task-specific SLM is configured with a low rank LoRA, but the relationship between high and low may be reversed. That is, the domain-specific SLM may be configured with a low rank LoRA and the task-specific SLM with a high rank LoRA. For example, the rank R of the domain-specific LoRA may be set to a value in the 100s, and the rank R of the task-specific LoRA may be set slightly higher than the rank R of the domain-specific LoRA.

[0036] In the above embodiment, an example has been described in which learning for domain specialization is performed using a sentence data set limited to a specific domain, and then additional learning for task specialization is performed using a sentence data set consisting of sentences in input / output pairs, but this is not limiting. For example, learning for domain specialization and learning for task specialization may be performed simultaneously. In this case, the rank R of the domain-specific LoRA and the rank R of the task-specific LoRA are set to the same value, and the sentence data set limited to a specific domain and the sentence data set consisting of sentences in input / output pairs are mixed together to perform learning.

[0037] In the above embodiment, an example using LoRA has been described as an example of SLM, but the present invention is not limited to this. For example, QLoRA derived from LoRA may be used, or a method different from LoRA, such as one classified as Adapter, may be used. These methods are collectively called PEFT (Parameter-Efficient Fine-Tuning). If the language model is configured to be capable of calculating differential parameters without updating the original parameters of the base LLM, it is possible to use a fine-tuning method that does not use LoRA.

[0038] In the above embodiment, the learning for domain specialization is performed by self-supervised learning, and the learning for task specialization is performed by supervised learning. However, the present invention is not limited to this. For example, both types of learning may be performed by supervised learning.

[0039] In addition, the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited thereby. In other words, the present invention can be implemented in various forms without departing from the gist or main characteristics thereof. [Explanation of symbols]

[0040] 1. Language model generator 10 SLM Learning Section 11 First model learning section 12 Second Model Learning Section 13 Model Integration Department 14 Dataset Generation Section

Claims

1. a small-scale language model training unit that trains a small-scale language model that is added to a large-scale language model trained using a text data set that is not limited to a specific domain, the small-scale language model being smaller in scale than the large-scale language model; The small-scale language model learning unit performs learning for domain specialization using a sentence data set limited to the specific domain so that output sentences from the small-scale language model reflect knowledge of the specific domain, and performs learning for task specialization related to expression form adjustment using a sentence data set consisting of input / output pair sentences so that output sentences from the small-scale language model have a desired expression form. A language model generating device comprising:

2. 2. The language model generating device according to claim 1, wherein the small-scale language model training unit performs training for the domain specialization using a sentence data set limited to the specific domain, and then performs additional training for the task specialization using a sentence data set consisting of sentences of the input / output pairs.

3. 3. The language model generating device according to claim 1, wherein the learning for the domain specialization is performed by self-supervised learning, and the learning for the task specialization is performed by supervised learning.

4. The small-scale language model training unit a first model learning unit that performs self-supervised learning for the domain specialization using a text data set limited to the specific domain to generate a domain-specific small-scale language model smaller in scale than the large-scale language model; and a second model learning unit that performs supervised learning for the task specialization using a sentence data set consisting of the sentences of the input / output pairs to generate a task-specific small-scale language model smaller in scale than the domain-specific small-scale language model.

2. The language model generating device according to claim 1.

5. 5. The language model generating device according to claim 4, further comprising a model integration unit that integrates the domain-specific small-scale language model generated by the first model learning unit and the task-specific small-scale language model generated by the second model learning unit.

6. 5. The language model generation device according to claim 4, further comprising a model integration unit that integrates the large-scale language model with at least one of the domain-specific small-scale language model generated by the first model learning unit and the task-specific small-scale language model generated by the second model learning unit.

7. a data set generation unit that generates a sentence data set consisting of sentences of input / output pairs reflecting knowledge of the specific domain, using a sentence data set limited to the specific domain used when the first model learning unit generates the domain-specific small-scale language model; The second model learning unit performs supervised learning for the task specialization using the sentence dataset generated by the dataset generation unit.

5. The language model generating device according to claim 4.

8. The small-scale language model is a model whose rank can be set, and the rank of the task-specific small-scale language model is set lower than the rank of the domain-specific small-scale language model. The language model generation device according to any one of claims 4 to 7,

9. The language model generation device according to any one of claims 4 to 7, characterized in that the first model learning unit learns LoRA, the rank of which is set to any value greater than 64, as the domain-specific small-scale language model.

10. 1. A language model generation device for generating a small-scale language model which is added to a large-scale language model trained using a text data set not limited to a specific domain, the small-scale language model being smaller than the large-scale language model, A language model generating device comprising a small-scale language model training unit that uses a sentence data set limited to the specific domain to train LoRA with a rank set to any value greater than 64 so that output sentences from the small-scale language model reflect knowledge of the specific domain.

11. A language model generation method for generating a small-scale language model, which is added to a large-scale language model trained using a text data set not limited to a specific domain, and which is smaller in scale than the large-scale language model, comprising: A language model generating method, characterized in that a small-scale language model learning unit of a computer performs learning for domain specialization using a sentence data set limited to a specific domain so that output sentences from the small-scale language model reflect knowledge of the specific domain, and performs learning for task specialization related to expression form adjustment using a sentence data set consisting of input / output pair sentences so that output sentences from the small-scale language model have a desired expression form.

Images