Reference data generation device, reference data generation method, and program

Abstract

To provide a technique for generating reference data to be referred to by a generation model in order to create prompt data with high answer accuracy for the generation model.SOLUTION: According to an aspect of the present disclosure, a reference data generation device includes an acquisition unit that acquires document data used for inputting a question sentence to a generation model, and a reference data generation unit that generates a plurality of pieces of divided data obtained by dividing the document data into input units of the question sentence so as to maintain a document structure of the document data.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to a reference data generation device, a reference data generation method, and a program. [Background technology]

[0002] With the recent advances in deep learning, machine learning technology has been applied to a wide range of technical fields. For example, in the technical field of natural language processing, progress has been made in research and development of machine learning models used in information extraction, information retrieval, summary generation, machine translation, speech recognition, dialogue processing, etc. Natural language processing using generative models, such as the recently released ChatGPT (registered trademark), has attracted attention.

[0003] In dialogue processing using generative models, several approaches have been proposed to obtain more appropriate answers from a generative model in response to a question. One such approach is Retrieval-Augmented Generation (RAG). RAG extracts information related to the question from an external reference database that holds accurate information, and then creates prompt data to be input into the generative model from the extracted related information and the question. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] In-Context Retrieval-Augmented Language Models(https: / / arxiv.org / abs / 2302.00083) Summary of the Invention [Problem to be solved by the invention]

[0005] When creating prompt data to be input into a generative model using such a reference database, it is thought that the accuracy of the output results from the generative model for prompt data that uses reference data extracted from the reference database may vary depending on what data is stored in the reference database.

[0006] An object of the present disclosure is to provide a technology for generating reference data that is referenced by a generative model in order to create prompt data that provides high response accuracy for the generative model. [Means for solving the problem]

[0007] One aspect of the present disclosure relates to a reference data generation device having an acquisition unit that acquires document data used to input a question sentence into a generation model, and a split data generation unit that generates split data by dividing the document data into multiple pieces based on the input unit of the question sentence. [Effects of the Invention]

[0008] According to the present disclosure, a technique can be provided for generating reference data that is referenced by a generative model in order to create prompt data that provides high response accuracy for the generative model. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an interaction process for a generative model using a reference database according to one embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram showing an example of chunk creation of document data according to the prior art. [Figure 3] FIG. 3 is a diagram showing an example of chunk creation of document data according to the prior art. [Figure 4] FIG. 4 is a diagram illustrating document data according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a diagram showing an example of generation of divided data by the conventional technique of the present disclosure. [Figure 6]FIG. 6 is a diagram illustrating an example of generation of divided data according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a block diagram illustrating a hardware configuration of a reference data generation device according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a block diagram illustrating a functional configuration of a reference data generating device according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a block diagram illustrating an example of generating divided data according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating divided data according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart illustrating a reference data generation process according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] In the following embodiment, a reference data generation device that realizes a process for generating reference data used in a generative model is disclosed.

[0012] [Summary of the Disclosure] In the interactive processing for a generative model according to an embodiment described below, prompt data is generated using information stored in a reference database. For example, as shown in Fig. 1, when a user using the interactive processing for a generative model 60 operates a user device 20 to provide question data for the generative model 60 to an interactive processing program 80, the interactive processing program 80 sends a search query to the reference database 40 to extract information related to the question data.

[0013] The reference database 40 stores document data provided by the user in advance in data units (hereinafter referred to as chunks) divided from the document data. In existing RAGs, the document data is divided into document page units, as shown in Figure 2, and each divided page is used as a chunk to be input into prompt data. Alternatively, the document data is divided into document character units, as shown in Figure 3, and each divided document part is used as a chunk to be input into prompt data.

[0014] Such chunk division methods can affect the search accuracy of RAG and the response accuracy of the generative model 60. Meanwhile, the chunks based on page units or character counts described above are created without considering the semantic content of the documents contained in the chunks. For example, dividing document data by page units or character counts can result in documents with different semantic content being included in a single chunk. Alternatively, a document with a certain set of semantic content can be separated into adjacent chunks. In this case, even if chunks containing words related to the question data obtained from the user are detected and the detected chunks are included in the prompt data, the generative model 60 may not be able to generate response data that matches the question data.

[0015] In the following embodiment, document data is divided into groups of semantic content, and chunks consisting of coherent semantic content are generated. That is, when creating prompt data in response to question data from the user device 20, the reference data generation device 100 receives document data referenced by the generative model 60. Upon receiving the document data, the reference data generation device 100 detects the document structure of the received document data and generates reference data (chunks) as text data that retains the detected document structure.

[0016] For example, suppose document data such as that shown in FIG. 4 is given. In conventional chunk creation, as described above, reference data is generated in units of pages or characters, regardless of the semantic coherence of the content included in the document data. Furthermore, tabular content T1 included in the document data is typically converted into text data T1' with a collapsed table structure, as shown in FIG. 5, and included in the reference data. In this way, when tabular content T1' with a collapsed table structure is included in the reference data as text data, the generative model 60 may not be able to properly interpret the reference data and may not be able to generate appropriate answer data to the question data from the user device 20.

[0017] In this embodiment, for document data such as that shown in Fig. 4, the reference data generation device 100 detects the document structure of the document data as shown in Fig. 6. That is, the reference data generation device 100 detects content parts C1 to C3 that are considered to have a coherent meaning in the document data, and generates each piece of reference data (chunks) by dividing the content parts C1 to C3 in the document data.

[0018] For example, the reference data generation device 100 can extract headings, etc. from document data and extract content parts with common meanings based on the extracted headings, etc. In the illustrated document data, "Beverage business," "Beverage division," "(2) Overview of financial status for the current period," and "(3) Overview of cash flow for the current period" are extracted as headings.

[0019] In the illustrated document data, the headings "Beverage Business" and "Beverage Division" are associated with the text "Raw material prices and logistics costs have become...", and the heading and text are identified as a content part C1 with coherent meaning.

[0020] Next, the heading "(2) Overview of financial status for the current fiscal year" is associated with the text "Assets at the end of the current consolidated fiscal year were...", and the heading and text are identified as a content part C2 with a coherent meaning.

[0021] Furthermore, the heading "(3) Overview of cash flows for the current period" is associated with the text "At the end of the current consolidated fiscal year... is as follows," the table of cash flow indicators, and the text "(Note) Equity adequacy ratio...," and these headings, tables, and text are identified as content part C3, which has a coherent meaning. Furthermore, in content part C3, the table of cash flow indicators is extracted as a table. Here, in order to maintain the structure of the tabular content, the table of cash flow indicators is converted into text data in CVS format, JSON format, or the like using the Markdown method, and associated with content part C3.

[0022] In this way, when content portions having coherent meanings are identified, the reference data generation device 100 divides the document data into each content portion and generates reference data from each content portion.

[0023] Here, the reference data generation device 100 may be realized by a computing device such as a server, a personal computer (PC), a smartphone, or a tablet, and may have a hardware configuration such as that shown in Fig. 7. That is, the reference data generation device 100 has a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0024] The programs or instructions that realize the various functions and processes of the reference data generation device 100 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium to the storage device 102 or the memory device 103 via the drive device 101. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0025] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used to execute the programs or instructions.

[0026] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or instruction is activated, reads and stores the program, instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0027] The processor 104 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the reference data generation device 100 in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the memory device 103.

[0028] The user interface (UI) device 105 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a touch panel, and realizes an interface between a user and the reference data generation device 100. For example, a user operates the reference data generation device 100 by operating a GUI (Graphical User Interface) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0029] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with external devices, the Internet, a LAN (Local Area Network), a cellular network, or other communication networks.

[0030] However, the above-described hardware configuration is merely an example, and the reference data generation device 100 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0031] [Reference data generator] Next, a reference data generation device 100 according to an embodiment of the present disclosure will be described. FIG. 8 is a block diagram showing a functional configuration of the reference data generation device 100 according to an embodiment of the present disclosure. As shown in FIG. 8, the reference data generation device 100 includes an acquisition unit 110 and a divided data generation unit 120. For example, one or more functional units of the acquisition unit 110 and the divided data generation unit 120 may be realized by one or more processors 104 executing one or more programs or instructions.

[0032] The acquisition unit 110 acquires document data used to input a question to the generative model 60. For example, a user who uses the generative model 60 operates the user device 20 to send document data to the reference data generation device 100 that the user wants the generative model 60 to refer to. Here, the document data is not necessarily plain text data that can be directly input to the generative model 60, but may more generally be composed of data in various formats such as PDF (Portable Document Format), WORD, Excel, and images.

[0033] For example, the user device 20 may provide the reference data generation device 100 with document data having a document structure as shown in Fig. 4. The acquisition unit 110 acquires the document data from the user device 20 and passes the acquired document data to the divided data generation unit 120.

[0034] The divided data generation unit 120 generates divided data by dividing the document data into a plurality of pieces for each input unit of a question sentence. Specifically, the divided data generation unit 120 detects the document structure of the document data provided by the user device 20, and generates divided data from the document data based on the detected document structure. Particularly in this embodiment, the divided data generation unit 120 generates divided data by dividing the document data based on the semantic content of the document data.

[0035] Specifically, the divided data generation unit 120 detects the document structure of the content in the document data and identifies content portions in the document data that have coherent meanings based on the detected document structure. For example, based on the document structure detected for the document data, the divided data generation unit 120 divides the document data into content portion C4 consisting of text #1 and image #2, content portion C5 consisting of text #3 and table #4, and content portion C6 consisting of table #5 and text #6, as shown in Figure 9. Each of content portions C4 to C6 is converted into text-based divided data so as to maintain the document structure.

[0036] In one embodiment, the divided data generation unit 120 may detect the document structure of the acquired document data according to a predetermined rule regarding the layout of character strings. For example, the divided data generation unit 120 may detect headings and hierarchical structures (e.g., chapters, sections, etc.) based on the layout of character strings included in the content of the document data (e.g., character string size, font, thickness, placement, wording, etc.), and detect the document structure of the content according to a predetermined rule regarding the detected headings and hierarchical structures. For example, the predetermined rule is not limited to this, and may include determining content parts with coherent meanings according to the lowest level in the hierarchical structure of the content in the document data, and using the determined content parts as constituent units of the divided data.

[0037] The divided data generation unit 120 performs layout recognition on the document data using any known layout recognition tool, etc., extracts headings, tables, text, images, etc. from the document data, and converts identified content portions into text based on the extracted headings, tables, text, images, etc. For example, for tables, the divided data generation unit 120 uses any known table structure extraction tool, etc., to convert the detected table into text data in CVS format or JSON format.

[0038] 6, the divided data generation unit 120 detects headings such as "Beverage Business," "Beverage Division," "(2) Overview of Financial Condition for the Current Fiscal Year," and "(3) Overview of Cash Flow for the Current Fiscal Year" according to the character string patterns and layout in the document data. The divided data generation unit 120 then identifies the content associated with each detected heading and generates divided data from content parts C1 to C3 consisting of each heading and the corresponding content.

[0039] Note that such document structure detection is not limited to being performed using the rule base described above, and may be performed using, for example, a machine learning model. In one embodiment, the segmented data generation unit 120 may detect the document structure of the acquired document data using a document structure detection model trained with training data of document data and document structures. For example, the document structure detection model may be any machine learning model trained to detect semantically coherent content parts from the content of the document data upon receiving the document data.

[0040] When the document structure of the document data is detected in this manner, the divided data generation unit 120 converts each detected content portion into text data to generate divided data. If the content of the document data has a hierarchical structure, the divided data generation unit 120 may indicate the hierarchy of the divided data in the hierarchical structure in the divided data.

[0041] Furthermore, when a content portion includes tabular content, the divided data generation unit 120 may include text data that maintains the table structure of the document data in the divided data so as to preserve the table structure. For example, for document data such as that shown in FIG. 6, the divided data generation unit 120 converts tabular content T1 into Markdown content T1" as shown in FIG. 10, and generates divided data corresponding to the content portion C3 as text data that maintains the table structure.

[0042] 10, the segment data may be configured to include not only a heading and associated text data but also metadata related to the content portion C3. For example, in the illustrated example, the metadata is composed of “id”: “8”, “title”: “dnp_kessan_000”, “filename”: “dnp_kessan_000.pdf”, “headline”: “1. Overview of Business Results, etc.”, and “content”: “1. Overview of Business Results, etc. (3) Overview of Cash Flows for the Current Fiscal Year ¥n.” Here, “headline”: “1. Overview of Business Results, etc.” and “content”: “1. Overview of Business Results, etc. (3) Overview of Cash Flows for the Current Fiscal Year ¥n” relate to the hierarchical level of the content portion C3 in the document data, and in particular, “1. Overview of Business Results, etc.” indicates a higher hierarchical level of the content portion C3.

[0043] In this way, the divided data generation unit 120 may include header information in the divided data. In this case, it is possible to indicate, as metadata, the position of the divided data in the hierarchical structure of the entire content of the document data, which may contribute to the generation of more appropriate answer data by the generative model 60.

[0044] According to the reference data generation device 100 of this embodiment, segmented data with coherent semantics is generated from document data provided by the user device 20, and the generated segmented data is stored as chunks in the reference database 40. By using such chunks with coherent semantics as prompt data, it may be possible to obtain answer data with higher answer accuracy from the generative model 60.

[0045] [Reference data generation process] Next, a reference data generation process according to an embodiment of the present disclosure will be described. The reference data generation process can be realized by the reference data generation device 100, more specifically, by the processor 103 of the reference data generation device 100 executing a program stored in the memory device 102. Fig. 11 is a flowchart showing the reference data generation process according to an embodiment of the present disclosure.

[0046] 11, in step S101, the reference data generation device 100 acquires document data to be used for inputting a question to the generative model 60. Typically, the document data is data that the user desires to use in interactive processing with the generative model 60, and is data that serves as the basis for chunks embedded in prompt data when performing interactive processing with the generative model 60. The document data is not necessarily plain text data that can be directly input to the generative model 60, and more generally may be composed of data in various formats such as PDF, WORD, Excel, and images.

[0047] In step S102, the reference data generation device 100 detects the document structure of the document data. Specifically, the reference data generation device 100 extracts content such as headings, tables, text, and images from the document data, and identifies content portions with coherent meanings from the extracted headings, tables, text, and images. For example, the reference data generation device 100 performs layout recognition on the document data using a known layout recognition tool, extracts headings, tables, text, and images from the document data, and converts the identified content portions based on the extracted headings, tables, text, and images into text. For tables, the reference data generation device 100 converts the tables into text data in CVS format or JSON format using Markdown, using a known table structure extraction tool.

[0048] In step S103, the reference data generation device 100 generates segmented data for creating prompt data from the document data so as to preserve the document structure. Specifically, the reference data generation device 100 generates segmented data as text data from each content portion with coherent semantics while maintaining the hierarchical structure and tables of the content in the document data. For example, the hierarchical structure can be preserved in the segmented data by adding metadata indicating the hierarchy of the content portion within the entire document data to the segmented data. Furthermore, the tables can be preserved in the segmented data by converting tabular content included in the document data into text data in CVS format or JSON format.

[0049] According to the reference data generation process of this embodiment, segmented data with coherent semantics is generated from document data provided by the user device 20, and the generated segmented data is stored as chunks in the reference database 40. By using such chunks with coherent semantics as prompt data, it may be possible to obtain answer data with higher answer accuracy from the generative model 60.

[0050] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims. [Explanation of symbols]

[0051] 20 User Device 40 Reference Databases 60 Generative Models 80 Interactive Programs 100 Reference data generating device 110 Acquisition Department 120 Split data generation unit

Claims

1. an acquisition unit that acquires document data used to input a question to the generative model; a divided data generation unit that generates divided data by dividing the document data into a plurality of pieces in units of input of a question sentence; A reference data generating device comprising:

2. 2. The reference data generating device according to claim 1, wherein the divided data generating unit generates the divided data by dividing the document data based on a set of semantic contents in the document data.

3. The reference data generating device according to claim 1 , wherein the divided data generating unit detects the document structure of the acquired document data by using a document structure detection model trained with training data of document data and document structure.

4. 2. The reference data generating device according to claim 1, wherein the divided data generating unit detects the document structure of the acquired document data in accordance with a predetermined rule regarding the layout of character strings.

5. 2. The reference data generating device according to claim 1, wherein the reference data generating unit includes text data that maintains the table structure of the document data in the divided data so as to maintain the table structure.

6. The reference data generating device according to claim 1 , wherein the divided data generating unit includes header information in the divided data.

7. Obtaining document data to be used as input for a question to a generative model; generating divided data by dividing the document data into a plurality of data pieces in units of input of a question sentence; A reference data generation method, wherein the above is executed by one or more computers.

8. Obtaining document data to be used as input for a question to a generative model; generating divided data by dividing the document data into a plurality of data pieces in units of input of a question sentence; A program that causes one or more computers to execute the following.

Images