Extraction device, extraction method, and extraction program

Abstract

It makes it easy to use content including text and images as input for chatbot systems that utilize RAG. [Solution] In the extraction device (10), a separation unit (15b) separates content to be processed into sentence objects and image objects and acquires position information for each of the sentence objects and image objects. A sentence extraction unit (15c) extracts text data from the sentence objects. An image extraction unit (15d) extracts text data from the image objects. A generation unit (15e) generates structured sentences including text data arranged in order according to the position information for each of the sentence objects and image objects.

Description

[Technical Field]

[0001] The present invention relates to an extraction device, an extraction method, and an extraction program. [Background technology]

[0002] Conventionally, chatbot system construction technology that utilizes Retrieval-Augmented Generation (RAG) is known, using in-house content such as service usage and business manuals as input (see Non-Patent Documents 1 and 2). With this technology, text data that can be used with the RAG index service is extracted from PDFs of in-house content and registered in an index. At this time, characters are read and extracted using technology such as OCR. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “What is Azure AI Document Intelligence?”, [online], August 2024, AZURE AI Document Intelligence, [Retrieved August 20, 2024], Internet <https: / / learn.microsoft.com / ja-jp / azure / ai-services / document-intelligence / overview?view=doc-intel-4.0.0> [Non-patent document 2] “In-house Chat and Document Search for Enterprises,” [online], June 2023, Azure OpenAI Service Reference Architecture, Chapter 5, [Retrieved August 20, 2024], Internet<https: / / www.microsoft.com / ja-jp / events / azurebase / contents / > Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technologies, it can be difficult to use internal content as input for chatbot systems that utilize RAG. For example, technologies such as OCR cannot preserve the structure of text and images in content that combines text and images, and may extract text that loses its original meaning. This requires manual correction to ensure the chatbot's response accuracy.

[0005] The present invention has been made in consideration of the above, and aims to make it easy to use content including text and images as input for a chatbot system that utilizes RAG. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the extraction device of the present invention is characterized by having a separation unit that separates the content to be processed into sentence objects and image objects and acquires position information of each of the sentence objects and image objects, a sentence extraction unit that extracts text data from the sentence objects, an image extraction unit that extracts text data from the image objects, and a generation unit that generates structured sentences including the text data arranged in an order according to the position information of each of the sentence objects and image objects. [Effects of the Invention]

[0007] According to the present invention, it becomes possible to easily use content including text and images as input for a chatbot system that utilizes RAG. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an outline of the processing of the extraction device. [Figure 2] FIG. 2 is a schematic diagram illustrating the general configuration of the extraction device. [Figure 3]FIG. 3 is a diagram for explaining the processing of the acquisition unit. [Figure 4] FIG. 4 is a flowchart illustrating an example of the extraction processing procedure. [Figure 5] FIG. 5 is a diagram for explaining the effect of the extraction process. [Figure 6] FIG. 6 is a diagram for explaining the effect of the extraction process. [Figure 7] FIG. 7 illustrates an example of a computer that executes an extraction program. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0010] [Outline of the extraction device] Figure 1 is a diagram for explaining the outline of the processing of the extraction device. Figure 1 illustrates an outline of a chatbot system. As illustrated in Figure 1, when a user inputs a question from a business portal (see (1)), OpenAI, which is called by the chatbot API, generates a search query (see (2) → (3)), and searches AISearch documents using the generated search query (see (4)). OpenAI then summarizes the found documents to generate an answer (see (5)), and the generated answer is returned to the user (see (6)).

[0011] The user's input and the chatbot AI's responses are stored in the business portal's local database (business portal database), and employees can check the chat history via the operator portal to monitor and evaluate the chatbot (see (7)).

[0012] In such a chatbot system, as a preliminary step, search documents (indexes) are created from PDFs of business portals and services and website manuals, and then registered in AISearch (see (0)).

[0013] Such indexes have traditionally been created by converting PDFs into text using OCR. However, with content that combines text and images, OCR cannot preserve the structure of the text and images, and the extracted text may lose its original meaning. For example, the OCR function may only read the text in figures or images, or may include unnecessary text such as copyright or tables of contents, or the reading order may be incorrect, or extra spaces or line breaks may be included. As a result, creating an index that maintains the accuracy of chatbot responses requires manual correction of text and table data. This makes it difficult to operate a chatbot system in a timely and continuous manner.

[0014] Therefore, when creating an index from content that combines text and images, the extraction device of this embodiment improves character reading accuracy by separating the text and images and extracting the text data individually.The extraction device also retains positional information of the text and images in the content, outputs the extraction results in a structured representation format, and uses them as an index for a chatbot system that utilizes RAG.

[0015] This enables the extraction device to extract text data from content containing sentences and images while maintaining its structure. Therefore, the extraction device significantly reduces the manual correction work required when creating indexes containing specialized knowledge as input to maintain the accuracy of responses in chatbot systems that utilize RAG, enabling continuous operation of the chatbot system.

[0016] [Extraction device configuration] Fig. 2 is a schematic diagram illustrating the general configuration of an extraction device 10. As illustrated in Fig. 2, the extraction device 10 is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0017] The input unit 11 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to input operations by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, or the like.

[0018] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between an external device via a network and the control unit 15. For example, the communication control unit 13 controls communication between the control unit 15 and a sensor that outputs time-series data to be processed in the extraction process described later, a management device that manages the time-series data, or the like.

[0019] The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance the processing program that operates the extraction device 10, data used during execution of the processing program, and the like, or temporarily stores the data each time processing is performed. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0020] In this embodiment, for example, content 14a used in the extraction process described below is pre-stored in the storage unit 14. Here, the content refers to content including text objects and image objects such as figures and tables, and is, for example, in-house content such as service specifications and business manuals.

[0021] The control unit 15 is realized using a CPU (Central Processing Unit) or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a separation unit 15b, a sentence extraction unit 15c, an image extraction unit 15d, and a generation unit 15e, as exemplified in FIG. 2. Note that each or some of these functional units may be implemented in different hardware. For example, the generation unit 15e may be implemented in a device separate from the other functional units. The control unit 15 may also include other functional units.

[0022] The acquisition unit 15a acquires content to be processed. Here, FIG. 3 is a diagram for explaining the processing of the acquisition unit. For example, as illustrated in FIG. 3, the acquisition unit 15a acquires the content to be processed from a system user via a reception API, or via a reception API and a Web UI (User Interface) for operating the reception API. The acquisition unit 15a receives a content file, for example, by POST of an HTTP request. Furthermore, as described above, the content to be processed is content that includes text objects and image objects such as figures and tables, and is, for example, in-house content such as service specifications and business manuals.

[0023] The acquisition unit 15a stores the received content in the storage unit 14. At that time, the acquisition unit 15a sets the processing status to "processing." Furthermore, as illustrated in FIG. 3, the acquisition unit 15a sends a reception ID that can uniquely identify the request to the WebUI. Note that the acquisition unit 15a may immediately transfer the acquired content to the subsequent separation unit 15b without storing it in the storage unit 14.

[0024] Returning to the explanation of FIG. 2, the separation unit 15b separates the content to be processed into text objects and image objects, and acquires position information for each of the text objects and image objects. Specifically, the separation unit 15b acquires the content to be processed from the content 14a in the storage unit 14, and separates it by object type included in the file. If the content includes text and images, it is separated into text objects and image objects. At this time, the separation unit 15b extracts and saves position information for each object.

[0025] The sentence extraction unit 15c extracts text data from the sentence objects. Specifically, the sentence extraction unit 15c repeats the process of extracting text data from the separated sentence objects. The sentence extraction unit 15c also extracts sentence structures such as chapter divisions and emphasis expressions such as red text and underlined parts, and stores them in association with the extracted text data.

[0026] Image extraction unit 15d extracts text data from the image objects. Specifically, image extraction unit 15d repeats the process of extracting text data from the separated image objects.

[0027] As a preprocessing step, the image extraction unit 15d extracts text data from an image object that has been subjected to high-accuracy processing using a GAN (Generative Adversarial Network) algorithm. For example, the image extraction unit 15d extracts text data from the image object that has been high-accuracy processed using the GAN algorithm using a multimodal LLM (Large Scale Language Model).

[0028] Furthermore, the image extraction unit 15d extracts text data of explanatory sentences within the image object. That is, the image extraction unit 15d extracts text data of the context that explains the content of the image object. This extracts information that is effective as input information to be utilized in the RAG.

[0029] Furthermore, the image extraction unit 15d extracts information representing the relationship between each piece of text data. The information representing the relationship between each piece of text data is, for example, arrows, rows and columns in a table, or connections. The image extraction unit 15d stores the information representing the relationship between each piece of extracted text data in association with the extracted text data. This allows the time sequence and positional relationship between the text data included in the image object to be stored.

[0030] The image extraction unit 15d also excludes headers, footers, and logos from the extracted text data. For example, the image extraction unit 15d excludes the headers, footers, and logos by excluding text data at a predetermined position within an image object from the LLM processing target or processing results. This eliminates information that is unnecessary as input information to be used in the RAG.

[0031] The generation unit 15e generates a structured sentence including text data arranged in an order according to the position information of each of the sentence objects and the image objects. Specifically, the generation unit 15e arranges the text data extracted by the sentence extraction unit 15c and the image extraction unit 15d according to the position information of the sentence objects and the position information of the image objects. The generation unit 15e also arranges the text data in the sentence objects according to the chapter structure of the sentence. The generation unit 15e also arranges the text data in the image objects according to information indicating the relationship between each piece of text data.

[0032] Then, the generating unit 15e generates a structured sentence in a markup language, including a sentence structure such as the order of each text data, emphasis expressions, etc. The generating unit 15e also outputs the generated structured sentence as an index of the RAG.

[0033] [Extraction process procedure] Next, an example of extraction processing by the extraction device 10 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart illustrating an example of the extraction processing procedure. The flowchart in Fig. 4 starts, for example, when an input is made to instruct the start of the extraction processing.

[0034] First, the separator 15b acquires the content to be processed (step S1) For example, the separator 15b acquires the content to be processed from the content 14a in the storage unit 14 acquired by the acquirer 15a.

[0035] Next, the separation unit 15b separates the content to be processed into text objects and image objects, and at that time, the separation unit 15b extracts and stores the position information of each of the text objects and image objects (step S2).

[0036] Next, the sentence extraction unit 15c extracts text data from the separated sentence objects (step S3). The sentence extraction unit 15c also extracts sentence structures such as chapter divisions and emphasis expressions such as red text and underlined parts, and stores them in association with the extracted text data.

[0037] Furthermore, image extraction unit 15d extracts text data from the separated image objects. Image extraction unit 15d also extracts information representing the relationship between each piece of text data (step S4). Image extraction unit 15d associates the extracted text data with the information representing the relationship between each piece of text data and stores them.

[0038] Then, the generation unit 15e generates a structured sentence including text data arranged in an order according to the position information of the sentence objects and the image objects (step S5). Specifically, the generation unit 15e arranges the text data extracted by the sentence extraction unit 15c and the image extraction unit 15d according to the position information of the sentence objects and the position information of the image objects. The generation unit 15e also arranges the text data in the sentence objects according to the chapter structure of the sentence. The generation unit 15e also arranges the text data in the image objects according to information indicating the relationship between each piece of text data.

[0039] Then, the generation unit 15e generates a structured sentence in a markup language that maintains the sentence structure, such as the order of each text data and emphasis expression. The generation unit 15e also outputs the generated structured sentence as an index of the RAG. This completes the series of extraction processes.

[0040] [effect] As described above, in the extraction device 10 of the above embodiment, the separation unit 15b separates the content to be processed into sentence objects and image objects and acquires position information for each of the sentence objects and image objects. The sentence extraction unit 15c extracts text data from the sentence objects. The image extraction unit 15d extracts text data from the image objects. The generation unit 15e generates structured sentences including text data arranged in order according to the position information for each of the sentence objects and image objects.

[0041] This makes it possible to extract text data from content containing text and images while maintaining its structure. Therefore, when creating indexes containing specialized knowledge as input to maintain the accuracy of responses in chatbot systems that utilize RAG, manual correction work can be significantly reduced, enabling continuous operation of the chatbot system. In this way, content containing text and images, such as internal content, can be easily used as input for chatbot systems that utilize RAG.

[0042] Furthermore, the image extraction unit 15d extracts text data from the image object that has been subjected to high-precision processing using the GAN algorithm, thereby enabling highly accurate extraction of text data from the image.

[0043] Furthermore, the image extracting unit 15d extracts text data of explanatory text from image objects, thereby making it possible to extract information that is effective as input information to be utilized in the RAG.

[0044] Furthermore, the image extracting unit 15d further extracts information indicating the relationship between each piece of text data, thereby making it possible to maintain the time sequence and positional relationship between the text data included in the image object.

[0045] Furthermore, the image extraction unit 15d excludes the header, footer, and logo from the extracted text data, thereby making it possible to exclude information that is unnecessary as input information to be utilized in the RAG.

[0046] 5 and 6 are diagrams for explaining the effect of the extraction process. First, Fig. 5(a) shows an example of the service specification for "ABC IP Voice." Also, Fig. 5(b) shows an example of structured sentences generated by the extraction process from the content shown in Fig. 5(a).

[0047] As shown in FIG. 5(b), the extraction process of the extraction device 10 of this embodiment makes it possible to extract text data from content containing sentences and images while maintaining the structure. For example, for "specific number notification" shown in area A, the text data of the explanatory text is extracted without mixing the explanatory text with "0120 / 0570." In addition, the bold frame of "I want to forward my call" is maintained as it is.

[0048] FIG. 6 illustrates the results of an extraction process for content that has been refined using a GAN algorithm. FIG. 6(a) illustrates an image containing blurred text. FIG. 6(b) illustrates structured text generated by the extraction process from the image shown in FIG. 6(a). As illustrated in FIG. 6(b), text data is extracted with high accuracy from the image containing blurred text through the extraction process.

[0049] In this way, the extraction process makes it easy to use content containing text and images as input for chatbot systems that utilize RAG.

[0050] [System configuration, etc.] The components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU or GPU (Graphics Processing Unit) and a program analyzed and executed by the CPU or GPU, or can be realized as hardware using wired logic.

[0051] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0052] [program] It is also possible to create a program in which the processing performed by the extraction device described in the above embodiment is written in a language executable by a computer. For example, it is also possible to create a program in which the processing performed by the extraction device 10 according to the embodiment is written in a language executable by a computer. In this case, the same effects as those of the above embodiment can be obtained by having a computer execute the program. Furthermore, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer to realize processing similar to that of the above embodiment.

[0053] 7 is a diagram showing an example of a computer that executes an extraction program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0054] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052. The video adapter 1060 is connected to, for example, a display 1061.

[0055] Here, the hard disk drive 1031 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. The various pieces of information described in the above embodiments are stored in the hard disk drive 1031 or memory 1010, for example.

[0056] The extraction program is stored in the hard disk drive 1031 as a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the program module 1093 in which each process executed by the extraction device 10 described in the above embodiment is written is stored in the hard disk drive 1031.

[0057] Furthermore, data used for information processing by the extraction program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-described procedures.

[0058] The program module 1093 and program data 1094 related to the extraction program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041. Alternatively, the program module 1093 and program data 1094 related to the extraction program may be stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0059] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0060] 10 Extraction device 11 Input section 12 Output section 13 Communication control section 14 Storage section 14a Content 15 Control Unit 15a Acquisition part 15b Separation part 15c Sentence extraction part 15d Image extraction section 15e Generator

Claims

1. a separation unit that separates the content to be processed into text objects and image objects and acquires position information of each of the text objects and the image objects; a sentence extraction unit that extracts text data from a sentence object; an image extractor for extracting text data from image objects; a generation unit that generates structured text including the text data arranged in an order according to the position information of each of the text objects and the image objects; An extraction device having:

2. The extraction device according to claim 1 , wherein the image extraction unit extracts text data from the image object that has been subjected to high-precision processing using a GAN (Generative Adversarial Network) algorithm.

3. 2. The extraction device according to claim 1, wherein the image extraction unit extracts text data of an explanatory sentence in the image object.

4. 2. The extraction device according to claim 1, wherein the image extraction unit further extracts information indicating a relationship between the text data.

5. 2. The extraction device according to claim 1, wherein the image extraction unit excludes a header, a footer, and a logo from the extracted text data.

6. An extraction method performed by an extraction device, a separation step of separating the content to be processed into text objects and image objects and acquiring position information of each of the text objects and the image objects; a sentence extraction step of extracting text data from the sentence object; an image extraction step of extracting text data from the image object; a generating step of generating structured text including the text data arranged in an order according to the position information of each of the text objects and the image objects; An extraction method comprising:

7. a separation step of separating the content to be processed into text objects and image objects and acquiring position information of each of the text objects and the image objects; a sentence extraction step of extracting text data from the sentence object; an image extraction step of extracting text data from the image object; a generating step of generating structured text including the text data arranged in an order according to the position information of each of the text objects and the image objects; An extraction program to run on a computer.

Images