Method and system for generating learning data for large-scale language model

Abstract

To provide a method and a system for generating learning data for a large-scale language model.SOLUTION: A learning data generation method includes the steps of: receiving input of a list of a context and using a generation type artificial intelligence model for each context to generate an inquiry; using the inquiry, a correct answer context corresponding to the inquiry and the generation type artificial intelligence model to generate an answer for the inquiry and generating a data set including the inquiry, the correct answer context and the answer; using the inquiry included in the data set to select at least one negative context from the list of the context and generating an enhancement data set including the correct answer context, the at least one negative context, the inquiry and the answer; and using the enhancement data set to generate a prompt and combining the answer with an index of the correct answer context to generate learning data.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The following description relates to a method and system for generating training data for large-scale language models. [Background technology]

[0002] Large language models (LLMs) are a form of artificial intelligence trained on large sets of text data to generate human-like responses to natural language input. They are language models composed of artificial neural networks with many parameters (usually billions or more weights). These LLMs are often called self-supervised learning or Using semi-self-supervised learning, it can be trained on large amounts of unlabeled text.

[0003] On the other hand, compared to OpenAI's ChatGPT, DAVINCI, GPT-4, etc., open source LLMs have very poor ability to follow general instructions. To improve this poor ability, learning from open source LLMs is required. In this regard, Retrieval-Augmented Generation (RAG) is a methodology for more effectively utilizing LLMs, which can combine the advantages of pre-trained LLMs with external knowledge. For example, the basic idea of RAG is to provide a method for generating a set of instructions for a specific question or topic. When generating responses, the AI model references external knowledge in real time to provide more accurate and richer information. This reduces hallucination. To implement RAG, you first create a knowledge base. If there is a question, the sentence is also converted into an embedding, the most similar paragraph is searched for in the vector database, and the searched paragraph and the question are both input to the LLM as prompts. Then an answer is generated.

[0004] In order to train the LLM to perform the citation function, one of the core functions of RAG, thousands or tens of thousands of training data are required. However, due to the nature of training data for generative models, it is extremely difficult to create such data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Korean Patent Registration No. 10-2551531 Summary of the Invention [Problem to be solved by the invention]

[0006] A method and system for generating training data for large-scale language models is provided. [Means for solving the problem]

[0007] A method for generating training data using a computer device including at least one processor, comprising: receiving, by the at least one processor, a list of contexts; and generating a question for each context using a generative artificial intelligence model; generating an answer to the question using the question, a correct answer context corresponding to the question, and the generative artificial intelligence model, and generating a dataset including the question, the correct answer context, and the answer, using the at least one processor; selecting, by the at least one processor, at least one negative context from the list of contexts using a question included in the dataset, and generating an augmented dataset including the correct context, the at least one negative context, the question, and the answer; and The present invention provides a training data generation method including the steps of generating prompts using the augmented data set and combining the answers with an index of the correct answer context to generate training data.

[0008] According to one embodiment, generating the augmented data set comprises extracting the context using a sparse retriever or a dense retriever. selecting the at least one negative context from a list of negative contexts.

[0009] According to another embodiment, the step of selecting the at least one negative context comprises selecting a context from the list of contexts excluding the correct context as the at least one negative context.

[0010] According to yet another embodiment, the step of generating training data includes a method of generating prompts in Retrieval-Augmented Generation (RAG) using the augmented data set. The method is characterized by including a step of generating a prompt with an expression.

[0011] According to another embodiment, a dataset including the questions, the correct answer contexts, and the answers is used to generate the augmented dataset through a human verification process.

[0012] According to yet another embodiment, the training data generation method includes generating a large language model (LLM) by using the generated training data by the at least one processor. The method may further include fine-tuning the citation instructions of the citation model (Models, LLM).

[0013] A computer program recorded on a computer-readable recording medium is provided for coupling with a computer device to cause the computer device to execute the method.

[0014] A computer-readable recording medium is provided that stores a program for executing the method on a computer device.

[0015] a computer device including at least one processor implemented to execute computer-readable instructions, the at least one processor receiving as input a list of contexts; generating a question for each context using a generative artificial intelligence model; generating the question, a correct answer context corresponding to the question, and an answer to the question using the generative artificial intelligence model; generating a dataset including the question, the correct answer context, and the answer; selecting at least one negative context from the list of contexts using a question included in the dataset to generate an augmented dataset including the correct answer context, the at least one negative context, the question, and the answer; generating a prompt using the augmented dataset; and generating training data by combining the answer with an index of the correct answer context. [Effects of the Invention]

[0016] According to an embodiment of the present invention, a method and system for generating training data for a large-scale language model is provided. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating an example computing device according to one embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of a schematic configuration of a training data generation system according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart illustrating an example of a training data generation method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0019] A training data generation system according to an embodiment of the present invention is implemented by at least one computer device. In this case, a computer program according to an embodiment of the present invention is installed and launched in the computer device implementing the training data generation system, and the computer device can perform the training data generation method according to an embodiment of the present invention under the control of the launched computer program. The computer program may be recorded on a computer-readable recording medium so as to be combined with the computer device to execute the training data generation method on the computer.

[0020] FIG. 1 is a diagram showing an example of a network environment according to an embodiment of the present invention. The network environment in FIG. 1 includes a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170). This is an example for explaining the invention. The number of electronic devices and the number of servers are not limited to those shown in FIG.

[0021] The electronic devices (110, 120, 130, 140) may be fixed terminals or mobile terminals implemented as computer systems. For example, examples of the electronic devices (110, 120, 130, 140) include smartphones, mobile phones, navigation systems, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, and AR (Augmented Reality) devices. For example, in FIG. 1, a smartphone is used as an example of the electronic device (110). Although a wireless configuration is shown, in an embodiment of the present invention, the electronic device (110) is wireless in nature. It refers to one of various physical computer systems that can communicate with other electronic devices (120, 130, 140) and / or servers (150, 160) through a network (170) using a wired or wired communication method. It can be tasted.

[0022] The communication method is not limited, and the network (170) may include any communication network (e.g., a mobile communication network). In addition to communication methods utilizing wired internet, wireless internet, broadcasting networks, satellite networks, etc., it also includes short-distance wireless communication between devices. For example, the network (170) includes a PAN (Personal Area Network), The network (170) may include one or more of any network such as a LAN (Local Area Network), a CAN (Campus Area Network), a MAN (Metropolitan Area Network), a WAN (Wide Area Network), a BBN (Broadband Network), the Internet, etc. The network (170) may also include a bus network, a star network, a ring network, a mesh network, etc. The network topology may include any one or more of the following, but is not limited to: a network, a star-bus network, a tree or hierarchical network.

[0023] Each server (150, 160) communicates with multiple electronic devices (110, 120, 130, 140) through a network (170) and provides instructions, codes, files, content, services, etc. The server 150 may be implemented on a single computer or multiple computers. For example, the server 150 may be A first sub-system is provided to a plurality of electronic devices (110, 120, 130, 140) connected through a network (170). The server (160) is also a system that provides a second service to a plurality of electronic devices (110, 120, 130, 140) connected through a network (170). As a specific example, the server (150) is installed in the plurality of electronic devices (110, 120, 130, 140) and starts up. Through an application as a computer program executed by the application, the application may provide a service (e.g., a search service) targeted by the application as a first service to the plurality of electronic devices (110, 120, 130, 140). A service for distributing files for installing and launching applications to multiple electronic devices (110, 120, 130, 140) may be provided as a second service.

[0024] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. The electronic devices (110, 120, 130, 140) or each server (150, 160) may be implemented by a computer device (200) shown in FIG.

[0025] Such a computer device (200) may include a memory (210), a processor (220), a communication interface (230), and an input / output interface (240), as shown in FIG. The memory (210) is a computer-readable recording medium, such as a RAM (random access memory), a ROM (read-only memory), or a disk drive. This includes non-volatile mass storage devices (permanent storage devices), such as ROM and disk drives. The non-volatile mass storage device is a permanent storage device that is separate from the memory (210). The computer device 200 may include an operating system and at least one program code. These software components may be read from a computer-readable recording medium separate from the memory 210. The program can be loaded into the library (210). The storage medium may include a floppy disk, diskette, tape, DVD / CD-ROM drive, memory card, etc. In another embodiment, the software components may not be stored on a computer-readable storage medium, but may be loaded into memory 210 through communication interface 230. For example, the software components may be stored in a file received over network 170. The computer program installed on the computer device (200 ) can be loaded into memory (210).

[0026] The processor (220) performs basic arithmetic, logic, and input / output operations to The computer is configured to process instructions of a computer program, the instructions being stored in memory (210) or a communication The instructions may be provided to the processor 220 via the interface 230. For example, the processor 220 may be configured to execute the received instructions based on program code stored in a storage device, such as the memory 210.

[0027] The communication interface (230) communicates with the computer device (200) through the network (170). ) provides a function for the computer device (200) to communicate with other devices (such as the aforementioned storage device). For example, the processor (220) of the computer device (200) may communicate with the memory (210) stored in a storage device. Requests, commands, data, files, etc. generated based on the program code can be transmitted to other devices through the network 170 under the control of the communication interface 230. Conversely, signals, commands, data, files, etc. from other devices can be transmitted through the network 170. The signals, instructions, data, etc. received through the communication interface 230 of the computer device 200 are transferred to the processor 220 or memory 210, and files, etc., may be stored in a storage medium (the aforementioned permanent storage device) that the computer device 200 may further include.

[0028] The input / output interface 240 is an interface means with the input / output device 250. For example, the input device may be a microphone, keyboard, or mouse, and the output device may be a desktop computer. In another example, the input / output interface (240) may include devices such as a display, a speaker, etc. The input / output device 250 may be implemented as a device integrated with the computer system 200, such as a touch screen, which combines input and output functions.

[0029] Also, in another embodiment, the computing device (200) may include fewer components than those shown in FIG. However, it is not necessary to explicitly show the components that are typically described in technical literature. For example, the computer system 200 may include the input / output devices described above. The device 250 may be implemented to include at least a portion of the device 250, or may further include other components such as a transceiver, a database, etc.

[0030] Large Language Models (LLMs), which are provided as open source, can be trained through pre-training and Supervised Fine Tuning (SFT). Here, pre-learning includes learning by a task of generating the entire document by looking at a part of the document content, and SFT includes learning by a task of generating content according to given instructions. In this case, the instructions themselves are not learned through SFT. However, the ability to follow instructions can be learned during the SFT process.

[0031] On the other hand, in Retrieval-Augmented Generation (RAG), a methodology for making more effective use of the LLM, citation instructions that clearly indicate the source of the answer are extremely important. Table 1 below shows examples of citation prompts.

[0032] [Table 1]

[0033] The underlined parts in Table 1 are examples of citation instructions.

[0034] FIG. 3 is a diagram showing an example of a schematic aspect of a training data generation system in one embodiment of the present invention. The training data generation system (300) according to this embodiment is the same as the computer described in FIG. The system is implemented by a computer device (200).

[0035] When raw data (310) is given, the raw data (310) is divided into appropriate units and called a context. In this case, a list of contexts (320) can be input to the training data generation system (300). The system (300) can generate training data based on an input context list (320).

[0036] The training data generation system (300) can use a generative artificial intelligence model (330) to generate questions for each context included in the list of contexts (320). The function model (330) may be a well-known model such as GPT-4. Then, the training data generation system (300) generates the generated questions and their corresponding contexts. Furthermore, a generative artificial intelligence model (330) can be used to generate answers corresponding to the questions. This allows data (context, questions, answers) to be generated. These data are generated mechanically through the generative artificial intelligence model (330), so a sufficient amount of data is generated. This data is named silver data.

[0037] The silver data can be verified by a person (e.g., a verifier (340)). The verified data is named gold data.

[0038] The training data generation system (300) can then store the list of contexts (320) in an appropriate storage (e.g., storage (350)). The system (300) can use the gold data questions to select contexts relevant to the questions from the list of contexts (320). In this process, the training data generation system (300) can select n (n is a natural number) contexts as negative contexts, excluding the context (correct context) that generated the original question and answer. This allows us to generate data of (correct context, n negative contexts, question, answer). We call this data augmented gold data.

[0039] In this case, the training data generation system (300) can generate prompts from the augmented gold data using a conventional RAG prompt generation method, and generate training data by combining answers with indexes of correct answer contexts.

[0040] Table 2 below shows examples of prompts generated using the augmented gold data (augmented gold prompts).

[0041] [Table 2]

[0042] In Table 2, the answer is underlined, and the part added to follow the citation instructions (part (ID: 2)) is bolded. Here, (ID: 2) means the index of the corresponding context, and explains how the index of the correct context was combined with the answer mentioned above.

[0043] The enhanced gold data generated through the learning data generation system (300) is used as the learning data. When using it as data to fine-tune the LLM, you can train it to correctly understand citation prompts and follow citation instructions.

[0044] 4 is a flowchart showing an example of a training data generation method according to an embodiment of the present invention. The training data generation method according to this embodiment is implemented by the training data generation system (300) described above. The computer system 200 may be implemented by a processor 220 that executes instructions based on the operating system code or at least one computer program code contained in the memory 210. Here, the processor (220) can control the computer device (200) to perform steps (410 to 440) included in the method of Figure 4 according to control instructions provided by the code stored in the computer device (200).

[0045] In step 410, the computer device 200 receives a list of contexts and generates questions for each context using a generative artificial intelligence model. As mentioned above, the generative artificial intelligence model may be a commercially available generative artificial intelligence model such as GPT-4.

[0046] In step (420), the computer device (200) generates an answer to the question using the question, the correct answer context corresponding to the question, and the generative artificial intelligence model, and can generate a dataset including the question, the correct answer context, and the answer. The generated dataset corresponds to the silver data described above, and the silver data can undergo a human verification process. As already mentioned, the verified silver data corresponds to gold data, and can be verified as described in step (430). The resulting dataset is equivalent to gold data that has undergone a human verification process.

[0047] In step 430, the computer device 200 can use the questions included in the dataset to select at least one negative context from the list of contexts and generate an augmented dataset that includes the correct context, the at least one negative context, the question, and the answer. The computer device (200) can select at least one negative context from the list of contexts using a retriever or a dense retriever. At least one negative context can be selected from the list of contexts, excluding the correct context.

[0048] In step 440, the computer device 200 can generate prompts using the augmented data set and combine the answers with the index of the correct answer context to generate training data. For example, the computer device 200 can generate prompts using the augmented data set using a prompt generation method in RAG.

[0049] According to an embodiment, the computer device (200) uses the generated training data to perform large-scale The language model can be fine-tuned for direct citation instructions. In another embodiment, the fine-tuning of the large-scale language model can be performed separately from the computer system (200). For example, the computer device 200 may be implemented through a separate physical electronic device. Physical electronic devices communicate via a network (e.g., network (170)) and The training data generated by the computer device 200 can be transmitted to another physical electronic device.

[0050] Thus, according to an embodiment of the present invention, a method and system for generating training data for LLM is provided. We can provide stems.

[0051] The above-described systems or devices may be implemented using hardware components or a combination of hardware and software components. For example, the devices and components described in the embodiments may include, for example, a processor, a controller, an ALU (arithmetic logic unit), ), digital signal processor, microcomputer, FPGA (Field Programmable Gate Array), PLU (Programmable Logic Unit), microprocessor, or Any other device that responds, i.e., one or more general-purpose or special-purpose computers The processor may be implemented using a processor-based processor. The processor may execute an operating system (OS) and one or more software applications that run on the OS. The processor may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processor may be described; however, those skilled in the art will appreciate that a processor may include multiple processing elements and / or multiple types of processing elements. For example, a processor may include multiple processors or one processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0052] Software includes computer programs, codes, instructions, or a combination of one or more of these, configured to operate a processing device as desired, or capable of independently or in combination providing instructions to the processing device. The software and / or data may be embodied in any type of mechanical component, physical device, virtual device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across networked computer systems, stored or executed in a distributed manner. The software and data may also be stored on one or more computer-readable recording media.

[0053] The methods according to the embodiments may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The medium may continuously record a computer-executable program or temporarily store it for execution or download. The medium may also be various recording or storage means combined with one or more pieces of hardware. It is not limited to media directly connected to a specific computer system, but may also be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROMs, RAMs, and flash memories, all of which may be devices capable of recording program instructions. Other examples of media include recording media or storage media managed by app stores that distribute applications, websites that provide or distribute various software, servers, and the like. Examples of program instructions include not only machine language code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

[0054] Although the above embodiments have been described only by way of limited examples and drawings, those skilled in the art will appreciate that various changes and modifications may be made from the above description. For example, the techniques described may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or substituted with other components or equivalents, and still obtain suitable results.

[0055] Therefore, other examples, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. A method for generating training data by a computer device including at least one processor, comprising: (1) receiving, by the at least one processor, a list of contexts as input and generating a question for each context using a generative artificial intelligence model; (2) generating an answer to the question using the question, a correct answer context corresponding to the question, and the generative artificial intelligence model, by the at least one processor, to generate a dataset including the question, the correct answer context, and the answer; (3) using the at least one processor to select at least one negative context from the list of contexts using a question included in the dataset, and generating an augmented dataset including the correct context, the at least one negative context, the question, and the answer; and (4) generating, by the at least one processor, prompts using the augmented data set and combining the answers with the index of the correct answer context to generate training data. A training data generation method including:

2. In claim 1, generating the augmented data set comprises: A method for generating training data, comprising the step of selecting the at least one negative context from the list of contexts using a sparse retriever or a dense retriever.

3. In claim 2, The step of selecting at least one negative context comprises: A learning data generation method, comprising selecting, from the context list, a context excluding the correct context as the at least one negative context.

4. In claim 1, The step of generating training data includes: Using the augmented dataset, the process is performed in RAG (Retrieval-Augmented Generation).

10. A method for generating training data, comprising the step of generating a prompt in a manner similar to that of generating a prompt.

5. In claim 1, A training data generation method, characterized in that a dataset including the question, the correct answer context, and the answer is used to generate the augmented dataset through a human verification process.

6. In claim 1, The method for generating training data further comprising the step of fine-tuning, by the at least one processor, a large-scale language model for citation instructions using the generated training data.

7. A computer program recorded on a computer-readable recording medium for causing a computer device to execute the method of any one of claims 1 to 6.

8. At least one program implemented to execute readable instructions in a computer system also including one processor, receiving, by the at least one processor, a list of contexts as input, and generating a question for each context using a generative artificial intelligence model; generating an answer to the question using the question, a correct answer context corresponding to the question, and the generative artificial intelligence model, and generating a dataset including the question, the correct answer context, and the answer; selecting at least one negative context from the list of contexts using a question included in the dataset, and generating an augmented dataset including the correct context, the at least one negative context, the question, and the answer; generating prompts using the augmented data set and combining the answers with the index of the correct answer context to generate training data; A computer device comprising:

9. In claim 8, by the at least one processor to generate the augmented data set; selecting the at least one negative context from the list of contexts using a sparse retriever or a dense retriever; A computer device comprising:

10. In claim 9, selecting the at least one negative context by the at least one processor; selecting a context, excluding the correct context, from the list of contexts as the at least one negative context; A computer device comprising:

Images