Information processing device, information processing method, and information processing program

Abstract

Provided are an information processing device, an information processing method, and an information processing program for generating QA data that is sufficiently comprehensive, reduces the possibility of hallucination occurring, and can reduce costs. [Solution] The system comprises an acquisition unit that acquires a target document, a first generation unit that generates text information, attribute information, and coordinate information for each of one or more components in the target document, a division unit that divides the target document into desired structural units based on the generated text information, attribute information, and coordinate information to generate divided sentences, an extraction unit that extracts at least one or more character strings contained in each of the divided sentences, a second generation unit that uses a large-scale language model to generate QA data for each of at least one or more divided sentences, based on each of at least one or more divided sentences, which includes question and answer sentences containing character strings, and an output unit that outputs the QA data.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, a technique has been disclosed for generating questions and their answers based on the contents of a target document (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-034276 Summary of the Invention [Means for solving the problem]

[0004] An information processing device according to one embodiment of the present invention includes an acquisition unit that acquires a target document, a first generation unit that generates text information, attribute information, and coordinate information for each of one or more components in the target document, a division unit that divides the target document into desired structural units based on the generated text information, attribute information, and coordinate information to generate divided sentences, an extraction unit that extracts at least one or more character strings contained in each of at least one or more of the divided sentences, a second generation unit that generates QA data for each of the at least one or more divided sentences using a large-scale language model, based on each of the at least one or more divided sentences, which includes question and answer sentences containing character strings, and an output unit that outputs the QA data.

[0005] In the information processing device according to one embodiment of the present invention, the constituent units divided by the dividing section may be a set of character strings describing a specific topic.

[0006] In an information processing device according to one embodiment of the present invention, the attribute information of each set of character strings written about a specific topic may be linked to attribute information of any of chapter, section, title, or subtitle.

[0007] In an information processing device according to one embodiment of the present invention, the constituent units divided by the dividing unit may be constituent units in which at least one or more character strings in the target document are divided into at least one or more character strings whose attribute information includes a title.

[0008] In the information processing device according to one embodiment of the present invention, at least one character string extracted by the extraction unit may be a character string related to a specific topic in a segmented sentence.

[0009] An information processing method according to one embodiment of the present invention includes a computer that executes an acquisition step of acquiring a target document, a first generation step of generating text information, attribute information, and coordinate information for each of one or more components in the target document, a division step of dividing the target document into desired structural units based on the generated text information, attribute information, and coordinate information to generate divided sentences, an extraction step of extracting at least one or more character strings contained in each of at least one or more divided sentences, a second generation step of generating QA data for each of the at least one or more divided sentences using a large-scale language model, the QA data including question and answer sentences containing the character strings based on each of the at least one or more divided sentences, and an output step of outputting the QA data.

[0010] An information processing program according to one embodiment of the present invention causes a computer to implement an acquisition function for acquiring a target document, a first generation function for generating text information, attribute information, and coordinate information for each of one or more components in the target document, a division function for dividing the target document into desired structural units based on the generated text information, attribute information, and coordinate information to generate divided sentences, an extraction function for extracting at least one or more character strings contained in each of at least one or more divided sentences, a second generation function for generating QA data containing question and answer sentences containing character strings for each of the at least one or more divided sentences based on each of the at least one or more divided sentences using a large-scale language model, and an output function for outputting the QA data. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration and functional units of an information processing device according to an embodiment of the present invention. [Figure 2] Figure 2 is an example of a target document. [Figure 3] Figure 3 shows an example of segmented sentences generated by the segmentation unit. [Figure 4] FIG. 4 shows an example of a character string having the "sentence" attribute, which is included in each constituent unit of a segmented sentence. [Figure 5] FIG. 5 shows an example of a character string extracted by the extraction unit. [Figure 6] FIG. 6 shows an example of a question and answer sentence generated by the second generation unit. [Figure 7] FIG. 7 shows an example of dividing a target document into its constituent elements. [Figure 8] FIG. 8 is a flowchart illustrating an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings relating to the embodiment, the same or similar parts are designated by the same or similar reference numerals. Of course, there are also parts whose relationships differ between the drawings.

[0013] Furthermore, the embodiments are merely examples of devices and methods for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not limit the configuration of each component to those described below. The technical idea of ​​the present invention can be modified in various ways within the technical scope defined by the claims.

[0014] Methods for generating questions and answers based on the content of a target document include manually creating the questions and answers, using a computer to generate structured data from the target document and then using a large language model (LLM) to generate questions and answers, and using a computer to generate questions and answers directly from images of the target document using a vision language model (VLM). Manual question and answer generation methods can be difficult to cover the entire content of a large document. When generating structured data from a target document and then using an LLM to generate questions and answers, the accuracy of the generated questions and answers is affected by the accuracy of the generated structured data, and the accuracy of the structured data decreases with the accuracy of the structured data. When generating questions and answers directly from a target document using a VLM, the VLM cannot accurately recognize small characters, figures, and other elements in the target document, resulting in a lack of comprehensiveness in the generated questions and answers. To address these issues, according to one embodiment of the present invention, OCR is used to divide the target document into desired structural units, character strings contained in each structural unit are extracted, and then questions and answers are generated using LLM. Compared to methods of generating questions and answers using LLM directly from the target document, this method makes it possible to generate questions and answers that are more comprehensive than questions and answers that are related to small characters, figures, charts, etc. in the target document, without missing any of them.

[0015] <Configuration> An example of the configuration and functional units of an information processing device 10 according to this embodiment is shown in Fig. 1. The information processing device 10 shown in Fig. 1 includes a CPU 101 for executing various calculations, a ROM 102 for storing processing programs, a RAM 103 for storing data and the like, a storage unit 104 for storing various data and calculation results and the like, an I / O (input / output interface) 105, a display unit 106, an input unit 107, and the like.

[0016] The I / O 105 is an interface, buffer, etc. for communication (transmission and reception).

[0017] The information processing device 10 according to this embodiment may also be connected to an input keyboard, mouse, and the like.

[0018] The information processing device 10 is any of various electronic computers (computational resources) such as a mobile terminal, a personal computer (PC), a mainframe, a workstation, and a cloud computing system.

[0019] 1 also shows functional units within the CPU 101. When each functional unit of the CPU 101 is realized by software, the CPU 101 realizes it by executing instructions of a program, which is software that realizes each function. In detail, the CPU 101 includes an acquisition unit 108, a first generation unit 109, a division unit 110, an extraction unit 111, a second generation unit 112, an output unit 113, and the like.

[0020] The acquisition unit 108 acquires the target document.

[0021] The first generating unit 109 generates text information, attribute information, and coordinate information of at least one or more constituent elements in the target document using optical character recognition (OCR).

[0022] The dividing unit 110 divides the target document into desired structural units based on the text information, attribute information, and coordinate information, to generate divided sentences.

[0023] The extraction unit 111 extracts at least one character string included in each structural unit of the divided sentence.

[0024] The second generation unit 112 generates Q&A data in which question and answer sentences including character strings are written using a large-scale language model.

[0025] The output unit 113 outputs the QA data.

[0026] The operation of each functional unit in the CPU 101 will be described in detail below.

[0027] The target document acquired by the acquisition unit 108 is a document containing character strings, images, numbers, graphs, tables, etc., and may be, for example, a report, a calculation sheet, a formula, a data file, etc. FIG. 2 shows an example of a target document acquired by the acquisition unit 108. The target document 20 shown in FIG. 2 is composed of a title 21 written as "Cyborg Research," a plurality of character strings 22 linked to the title 21 and describing an outline of the content of the target document 20, a figure 23 linked to the title 21 and describing an outline of the content of the target document 20, a subtitle 24 written as "Research Contents," a plurality of character strings 25 linked to the subtitle 24 and describing the content of the "Research Contents," a plurality of character strings 26 itemizing the content of the "Research Contents," a subtitle 27 written as "Technical Features," a plurality of character strings 28 linked to the subtitle 27 and describing the content of the "Technical Features," and a plurality of character strings 29 linked to the subtitle 27 and describing the content of the "Technical Features." FIG. 23 is made up of a plurality of drawings 231 and a plurality of character strings 232 that are linked to the plurality of drawings 231 and explain the plurality of drawings 231.

[0028] The acquiring unit 108 acquires the target document from an external device (not shown), for example, via the I / O 105. The external device may be, for example, a server or a user terminal. The user terminal is a terminal used by a user of the information processing device 10, and may be, for example, a desktop, a laptop, a tablet, or a smartphone. Furthermore, for example, when the target document is recorded in an external memory (not shown) and the external memory is connected to an interface of the information processing device 10, the acquiring unit 108 may acquire the target document from the external memory. Furthermore, when text information (text file) containing the target document is available and the target document is selected (when an area in which the target document is written is specified) via the input unit 121 and the user terminal, the acquiring unit 108 may acquire the selected target document (the specified area is the target document).

[0029] The first generating unit 109 structures the target document, that is, generates text information, attribute information, and coordinate information of at least one or more constituent elements in the target document.

[0030] In this embodiment, at least one or more components in a target document refers to elements described in the target document, such as character strings, images, numbers, graphs, tables, etc. For example, the document shown in Fig. 2 is composed of a title 21, multiple character strings 22, a figure 23, a subtitle 24, multiple character strings 25, multiple character strings 26, a subtitle 27, multiple character strings 28, multiple character strings 29, etc.

[0031] The text information generated by the first generation unit 109 is the character string contained in the component. For example, a component does not necessarily contain a character string, such as when the component is a graph that does not contain a character string, or when the component is a logo or illustration. When a component does not contain a character string, the first generation unit 109 generates attribute information and coordinate information. When a predetermined number of components, among at least one or more components in the target document, do not contain a character string, the first generation unit 109 generates text information, attribute information, and coordinate information of the component for the component that contains a character string, and further generates attribute information and coordinate information of the component for the component that does not contain a character string, and then generates structured data based on the generated text information, attribute information, and coordinate information.

[0032] The attribute information generated by the first generation unit 109 is the type used to classify components, and for example, the attribute of title 21 is title, the attribute of string 22 is "a sentence linked to title 21 that explains the outline of the contents of target document 20", the attribute of Figure 23 is "a summary diagram of the contents of target document 20 and a sentence explaining it, linked to title 21", the attribute of subtitle 24 is "subtitle", the attribute of string 25 is "a sentence linked to subtitle 24 that explains the contents of "research content", the attribute of string 26 is "a sentence in bullet points of the contents of "research content", linked to subtitle 24", the attribute of subtitle 27 is "subtitle", the attribute of string 28 is "a sentence linked to subtitle 27 that explains the contents of "technical features", and the attribute of string 29 is "a sentence in bullet points of the contents of "technical features", linked to subtitle 27". The attribute of the multiple figures 231 that make up Figure 23 is "an overview diagram of the contents of the target document 20 linked to the title 21," and the attribute of the multiple character strings 232 is "a sentence explaining the overview diagram of the contents of the target document 20 linked to the title 21."

[0033] The coordinate information generated by the first generation unit 109 is information that specifies the position of a component within the target document 20, and may include, for example, information such as coordinates indicating the position within the paper surface of the target document 20, the shape of the component, and the vertical and horizontal lengths of the component.

[0034] The segmentation unit 110 segments the target document 20 into desired structural units based on the text information, attribute information, and coordinate information, to generate at least one or more segmented sentences. Here, the structural units may be a set of character strings describing a specific topic. The set of character strings describing a specific topic may be linked to attribute information such as chapter, section, title, or subtitle in the set of character strings describing a specific topic. Alternatively, the segmentation unit 110 may segment at least one or more character strings in the target document 20 into at least one or more character strings whose attribute information includes a title, to generate at least one or more segmented sentences. The segmentation unit 110 analyzes topic changes in the target document 20 through the attribute information, and segments the target document 20 into each description of a specific topic, i.e., into each desired structural unit, to generate segmented sentences.

[0035] 3 shows segmented sentences generated by segmentation unit 110 by segmenting target document 20 shown in FIG. 2 into desired structural units based on text information, attribute information, and coordinate information. In the example shown in FIG. 3, target document 20 is segmented into segmented sentences 31, 32, and 33. Segmented sentence 31 includes title 21, character string 22, and figure 23. Segmented sentence 32 includes subtitle 24, character string 25, and character string 26. Segmented sentence 33 includes subtitle 27, character string 28, and character string 29.

[0036] The components of segmented sentence 31, that is, title 21, character string 22, and figure 23, all have attributes linked to the title or title 21, and explain an outline of the contents of target document 20. That is, segmented sentence 31 is a collection of character strings describing the specific topic, "Outline of the contents of target document 20." The components of segmented sentence 32, that is, subtitle 24, character string 25, and character string 26, all have attributes linked to the subtitle or subtitle 24, and explain the contents of "research content." Segmented sentence 31 is a collection of character strings describing the specific topic, "Contents of 'research content'." The components of segmented sentence 33, that is, subtitle 27, character string 28, and character string 29, all have attributes linked to the subtitle or subtitle 27, and explain the contents of "technical features." Segmented sentence 31 is a collection of character strings describing the specific topic, "Contents of 'technical features'."

[0037] The extraction unit 111 extracts at least one character string related to a specific topic in each segmented sentence from at least one character string having the attribute of "document" included in each constituent unit of the segmented sentences 31, 32, and 33. The at least one character string related to a specific topic in the segmented sentence is a character string corresponding to a keyword explaining the specific topic described in the segmented sentence.

[0038] As an example, a case will be described in which the extraction unit 111 extracts at least one or more character strings related to a specific topic in the segmented sentence 31 from at least one or more character strings having the "sentence" attribute that are included in each constituent unit of the segmented sentence 31. FIG. 4 shows at least one or more character strings having the "sentence" attribute that are included in each constituent unit of the segmented sentence 31. In FIG. 4, character strings 41, 42, 43, 44, 45, 46, and 47 are shown as the at least one or more character strings having the "sentence" attribute. The extraction unit 111 extracts at least one or more character strings related to the specific topic in the segmented sentence 31 from character strings 41 to 47. The specific topic in the segmented sentence 31 is "an outline of the contents of the target document 20," and the extraction unit 111 extracts multiple character strings shown in FIG. 5 as keywords that explain the specific topic. The multiple character strings shown in FIG. 5 are extracted not only from character string 22 consisting of only character strings, but also from multiple character strings 232 in FIG. 23.

[0039] The second generation unit 112 uses a large-scale language model to generate QA data for at least one or more segmented sentences generated by the segmentation unit 110, based on the content of each segmented sentence. The QA data includes question and answer sentences containing at least one or more character strings extracted by the extraction unit 111. The second generation unit 112 may generate QA data for a segmented sentence specified by a user input, for example, among the at least one or more segmented sentences, or may generate QA data for all of the at least one or more segmented sentences for each segmented sentence. FIG. 6 shows an example of question and answer sentences generated by the second generation unit 112. The question and answer sentences shown in FIG. 6 are questions and answers related to keywords describing the specific topic shown in FIG. 4.

[0040] The output unit 113 outputs the QA data to, for example, the display unit 106 or the like.

[0041] As described above, the information processing device according to this embodiment uses OCR to divide a target document into desired structural units to generate divided sentences, extracts character strings contained in each divided sentence, and then generates question and answer answers using VLM, thereby making it possible to generate highly comprehensive question and answer answers.

[0042] For comparison with the information processing device according to this embodiment, a case where a target document is not divided into desired structural units will be described with reference to Fig. 7. Fig. 7 shows a case where target document 20 shown in Fig. 2 is divided into structural elements rather than into desired structural elements. In Fig. 7, target document 20 is divided into structural elements including title 21, multiple character strings 22, a figure 23, subtitle 24, multiple character strings 25, multiple character strings 26, subtitle 27, multiple character strings 28, and multiple character strings 29.

[0043] 7, if the extraction unit 111 extracts at least one character string related to a specific topic for each component, and the second generation unit 112 generates Q&A data containing question and answer sentences that include at least one character string extracted by the extraction unit 111, the result obtained in the case shown in FIG. 7 will be highly comprehensive because the range of subjects for which the second generation unit 112 generates Q&A data is narrow, but the amount of information required to generate Q&A data is insufficient, increasing the possibility of hallucination. Furthermore, the amount of calculation required to generate Q&A data for the entire target document 20 increases, thereby increasing costs.

[0044] On the other hand, if the target document 20 shown in Figure 2 is not divided, and at least one character string related to a specific topic is extracted from the entire target document 20 by the extraction unit 111, and QA data is generated by the second generation unit 112, only one calculation is required to generate QA data for the entire target document 20.However, since the range of objects for which the second generation unit 112 generates QA data is wide, the comprehensiveness is low, for example, characters in figures and tables, small characters, etc. are not extracted.Furthermore, although the amount of information for generating QA data is sufficient, the amount of text is excessively large, increasing the possibility of hallucination occurring.

[0045] According to the information processing device of this embodiment, the target document is divided into desired structural units to generate divided sentences, at least one character string related to a specific topic is extracted from the divided documents by the extraction unit 111, and QA data is generated by the second generation unit 112, so that comprehensiveness is sufficient and the possibility of hallucination is reduced. Furthermore, the amount of calculation required to generate QA data for the entire target document 20 can be reduced, thereby reducing costs.

[0046] The information processing method according to this embodiment will be described with reference to the flowchart of FIG.

[0047] In step S801, the acquisition unit 108 acquires a target document (acquisition step).

[0048] In step S802, text information, attribute information of the component, and coordinate information of the component for at least one component in the target document are generated (first generation step).

[0049] In step S803, the dividing unit 110 divides the target document into desired structural units based on the text information, attribute information, and coordinate information of each of one or more components, to generate divided sentences (dividing step).

[0050] In step S804, the extraction unit 111 extracts at least one character string included in each structural unit of the divided sentence (extraction step).

[0051] In step S805, the second generation unit 112 generates Q&A data in which question and answer sentences including the character string are written, using a large-scale language model (second generation step).

[0052] In step S806, the output unit 113 outputs the QA data (output step).

[0053] As mentioned above, the present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description.

[0054] The programs of the embodiments of the present disclosure may be provided in a state stored in a storage medium readable by an information processing device. The storage medium may store the programs in a "non-transitory tangible medium." The programs include, for example, software programs and control programs. When the functional units of the information processing device are realized by software, the information processing device functions as an acquisition unit 108, a first generation unit 109, a division unit 110, an extraction unit 111, a second generation unit 112, and an output unit 113 by the processor executing the programs loaded into the memory.

[0055] The storage medium may, where appropriate, include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs), application specific ICs (ASICs), etc.), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable storage media, or any suitable combination of two or more of these. The storage medium may, where appropriate, be volatile, non-volatile, or a combination of volatile and non-volatile.

[0056] Furthermore, the program of the present disclosure may be provided to an information processing device via any transmission medium (such as a communication network or broadcast waves) capable of transmitting the program.

[0057] Furthermore, each embodiment of the present disclosure may be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission. Note that the program of the present disclosure may be implemented using, for example, a scripting language such as JavaScript (registered trademark) or Python (registered trademark), C language, Go language, Swift (registered trademark), Koltin (registered trademark), Java (registered trademark), or the like.

[0058] According to each aspect of the present disclosure described above, by providing an information processing device capable of generating comprehensive question and answer formats, it is possible to contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation." [Explanation of symbols]

[0059] 10. Information processing equipment 101 CPU 102 ROM 103 RAM 104 Storage section 105 I / O (input / output interface) 106 Display section 107 Input section 108 Acquisition Department 109 1st generation part 110 Split section 111 Extraction part 112 Second generation part 113 Output section 20 Target Documents 21 titles 22 Strings 23, 231 Fig. 24, 27 Subtitles 25, 26, 28, 29, 232, 41, 42, 43, 44, 45, 46, 47 Strings 31, 32, 33 divided sentences

Claims

1. an acquisition unit that acquires a target document; a first generating unit that generates text information, attribute information including any one of a chapter, a section, a title, and a subtitle, and coordinate information for each of one or more components in the target document; a division unit that determines, based on the generated text information, attribute information, and coordinate information, that the components linked to the same chapter, section, title, or subtitle describe the same specific topic, thereby dividing the target document into sections each describing the same specific topic, thereby generating at least one or more divided sentences; an extraction unit that extracts at least one character string included in each of at least one of the segmented sentences; a second generation unit that generates, for each of at least one of the segmented sentences, Q&A data including question and answer sentences each including the character string, based on the at least one of the segmented sentences using a large-scale language model; an output unit that outputs the QA data; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the dividing unit divides the target document into groups of character strings that describe the specific topic.

3. The information processing device according to claim 2, characterized in that the attribute information of each of the sets of character strings written about the specific topic is linked to attribute information of any of chapter, section, title, or subtitle.

4. 2 . The information processing apparatus according to claim 1 , wherein the dividing unit divides at least one character string in the target document for each of the at least one character strings whose attribute information includes a title.

5. The computer an acquisition step of acquiring a target document; a first generation step of generating text information, attribute information including any one of a chapter, a section, a title, and a subtitle, and coordinate information for each of one or more components in the target document; a division step of determining, based on the generated text information, attribute information, and coordinate information, that the components linked to the same chapter, section, title, or subtitle describe the same specific topic, thereby dividing the target document into sections each describing the same specific topic, thereby generating at least one or more divided sentences; an extraction step of extracting at least one character string included in each of at least one of the segmented sentences; a second generation step of generating, for each of at least one or more of the segmented sentences, Q&A data including question and answer sentences containing the character string based on the at least one or more of the segmented sentences using a large-scale language model; an output step of outputting the QA data; An information processing method comprising:

6. On the computer, an acquisition function for acquiring the target document; a first generating function that generates text information, attribute information including any one of a chapter, a section, a title, and a subtitle, and coordinate information for each of one or more components in the target document; a division function that determines, based on the generated text information, attribute information, and coordinate information, that the components linked to the same chapter, section, title, or subtitle describe the same specific topic, thereby dividing the target document into sections each describing the same specific topic, thereby generating at least one or more divided sentences; an extraction function for extracting at least one character string included in each of at least one divided sentence; a second generation function that generates, for each of at least one of the segmented sentences, Q&A data including question and answer sentences containing the character string based on the at least one of the segmented sentences using a large-scale language model; an output function for outputting the QA data; An information processing program that makes this possible.

Images