Information processing device, information processing method, and information processing program
By dividing documents into constituent units and using LLMs to extract and generate questions and answers, the method addresses the limitations of existing techniques, achieving comprehensive and cost-effective QA generation.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- SOFTBANK CORPORATION
- Filing Date
- 2024-12-23
- Publication Date
- 2026-07-03
AI Technical Summary
Existing methods for generating questions and answers from target documents face challenges such as incomplete coverage when manual creation is used, reduced accuracy due to low-quality structured data, and lack of comprehensiveness when using Vision Language Models (VLMs).
The method involves using Optical Character Recognition (OCR) to divide documents into desired constituent units, extracting relevant strings, and then generating questions and answers using a Large Language Model (LLM) to ensure comprehensive coverage and accuracy.
This approach generates highly comprehensive questions and answers while reducing the likelihood of hallucinations and computational costs, ensuring thorough extraction of text from various document components.
Smart Images
Figure 2026111096000001_ABST
Abstract
Description
Technical Field
[0003]
[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.
Background Art
[0002] Conventionally, a technique for generating questions and their answers based on the content of a target document has been disclosed (see, for example, Patent Document 1).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Means for Solving the Problems
[0004] An information processing apparatus according to an embodiment of the present invention includes an acquisition unit that acquires a target document, a first generation unit that generates text information, attribute information, and coordinate information for each of one or more components in the target document, a division unit that divides the target document into desired constituent units and generates divided sentences, an extraction unit that extracts at least one string included in each of at least one of the divided sentences, a second generation unit that generates QA data in which a question and an answer sentence including a string are described for each of at least one of the at least one divided sentences based on a large language model, and an output unit that outputs the QA data. [[ID=In an information processing device according to one embodiment of the present invention, a set of strings describing a specific topic may have attribute information of each string described in the set of strings describing the specific topic associated with one of the attribute information of a chapter, section, title, or subtitle.
[0007] In an information processing device according to one embodiment of the present invention, the component unit to be divided by the division unit may be a component unit that divides at least one string in the target document for each string in which the attribute information includes the title.
[0008] In an information processing device according to one embodiment of the present invention, at least one string extracted by the extraction unit may be a string relating to a specific topic in the segmented text.
[0009] An information processing method according to one embodiment of the present invention involves a computer performing an acquisition step of acquiring a target document; a first generation step of generating text information, attribute information, and coordinate information for each of one or more components within the target document; a division step of dividing the target document into desired constituent units based on the generated text information, attribute information, and coordinate information to generate divided documents; an extraction step of extracting at least one string contained in each of at least one divided documents; a second generation step of generating QA data containing question and answer sentences with strings for each of at least one divided documents based on each of at least one divided documents using a large-scale language model; and an output step of outputting the QA data.
[0010] An information processing program according to one embodiment of the present invention provides a computer with an acquisition function for acquiring a target document, a first generation function for generating text information, attribute information, and coordinate information for each of one or more components within the target document, a division function for dividing the target document into desired constituent units based on the generated text information, attribute information, and coordinate information to generate divided documents, an extraction function for extracting at least one string contained in each of at least one divided documents, a second generation function for generating QA data containing question and answer sentences with strings for each of at least one divided documents based on each of at least one divided documents using a large-scale language model, and an output function for outputting the QA data. [Brief explanation of the drawing]
[0011] [Figure 1] Figure 1 is a block diagram showing an example of the configuration and functional parts of an information processing device according to one embodiment of the present invention. [Figure 2] Figure 2 shows an example of a target document. [Figure 3] Figure 3 shows an example of a segmented text generated by the segmentation unit. [Figure 4] Figure 4 shows an example of a string containing the "text" attribute within each constituent unit of a segmented text. [Figure 5] Figure 5 shows an example of a string extracted by the extraction unit. [Figure 6] Figure 6 shows an example of a question and answer text generated by the second generation unit. [Figure 7] Figure 7 shows an example of dividing the target document into its constituent elements. [Figure 8] Figure 8 is a flowchart illustrating an information processing method according to one embodiment. [Modes for carrying out the invention]
[0012] Next, embodiments of the present invention will be described with reference to the drawings. In the drawings of the embodiments, identical or similar parts are denoted by the same or similar reference numerals. It goes without saying that there are parts in the drawings that have different relationships to each other.
[0013] Furthermore, the embodiments are illustrative of apparatus and methods for realizing the technical concept of the present invention, and the technical concept of the present invention does not limit the configuration of each component to those described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims described in the patent claims.
[0014] There are several methods for generating questions and answers based on the content of a target document: manually creating questions and answers; using a computer to generate structured data from the target document and then using a Large Language Model (LLM) to generate questions and answers from the generated structured data; and using a computer to directly generate questions and answers from images of the target document using a Vision Language Model (VLM). With the manual question and answer creation method, it is difficult to cover all the content if the amount of content in the target document is large. With the method of generating questions and answers using an LLM after generating structured data from the target document, the accuracy of the generated questions and answers is affected by the accuracy of the generated structured data; if the accuracy of the structured data is low, the accuracy of the questions and answers will also decrease. With the method of generating questions and answers directly using a VLM from the target document, there is a problem that the generated questions and answers lack comprehensiveness because small characters and diagrams in the target document cannot be accurately recognized. To address these problems, according to one embodiment of the present invention, by using OCR to divide the target document into desired constituent units, extracting the strings contained in each constituent unit, and then generating question and answer using LLM, it is possible to generate a highly comprehensive question and answer without missing any questions and answers related to small characters, figures, etc., within the target document, compared to a method of generating question and answer directly from the target document using LLM.
[0015] <Structure> FIG. 1 shows an example of the configuration and functional units of the information processing apparatus 10 according to the present embodiment. The information processing apparatus 10 shown in FIG. 1 includes a CPU 101 for executing various operations, a ROM 102 for storing processing programs, a RAM 103 for storing data, etc., a storage unit 104 for storing various data and operation results, etc., an I / O (input / output interface) 105, a display unit 106, an input unit 107, and the like.
[0016] The I / O 105 is an interface for communication (transmission and reception), a buffer, etc.
[0017] In the information processing apparatus 10 according to the present embodiment, other input keyboards, mice, etc. may be connected.
[0018] The information processing apparatus 10 is various electronic computers (computing resources) such as a mobile terminal, a personal computer (PC), a mainframe, a workstation, a cloud computing system, etc.
[0019] Furthermore, the block diagram of FIG. 1 shows the functional units in the CPU 101. When each functional unit of the CPU 101 is realized by software, the CPU 101 is realized by executing the instructions of a program that is software for realizing each function. Specifically, it includes an acquisition unit 108, a first generation unit 109, a division unit 110, an extraction unit 111, a second generation unit 112, an output unit 113, etc.
[0020] The acquisition unit 108 acquires a target document.
[0021] The first generation unit 109 uses optical character recognition (OCR) to generate text information, attribute information of at least one or more components in the target document, and coordinate information of the components.
[0022] The division unit 110 divides the target document into desired constituent units based on the text information, attribute information, and coordinate information to generate divided sentences.
[0023] The extraction unit 111 extracts at least one string from each constituent unit of the divided text.
[0024] The second generation unit 112 generates QA data containing question and answer sentences that include strings, using a large-scale language model.
[0025] The output unit 113 outputs QA data.
[0026] The operation of each functional unit within the CPU 101 is described in detail below.
[0027] The target documents acquired by the acquisition unit 108 are documents containing strings of text, images, numbers, graphs, tables, etc., and may include, for example, forms, calculation sheets, mathematical formulas, data files, etc. Figure 2 shows an example of a target document acquired by the acquisition unit 108. The target document 20 shown in Figure 2 consists of a title 21 that reads "Cyborg Research", multiple strings 22 linked to the title 21 that describe an overview of the contents of the target document 20, a figure 23 linked to the title 21 that describes an overview of the contents of the target document 20, a subtitle 24 that reads "Research Content", multiple strings 25 linked to the subtitle 24 that describe the contents of "Research Content", multiple strings 26 that list the contents of "Research Content", a subtitle 27 that reads "Technical Features", multiple strings 28 linked to the subtitle 27 that describe the contents of "Technical Features", multiple strings 29 linked to the subtitle 27 that list the contents of "Technical Features", etc. Figure 23 consists of multiple Figures 231 and multiple strings 232 that describe the multiple Figures 231, which are associated with the multiple Figures 231.
[0028] The acquisition unit 108 acquires the target document from an external device (not shown) via, for example, I / O 105. The external device may be, for example, a server and a user terminal. The user terminal is a terminal used by a user of the information processing device 10, and may be, for example, a desktop, laptop, tablet, and smartphone. The acquisition unit 108 may also acquire the target document from an external memory (not shown) if the target document is recorded in that external memory and that external memory is connected to the interface of the information processing device 10. Furthermore, if there is document information (document file) containing the target document, and the target document is selected via the input unit 121 and the user terminal, etc. (if an area in which the target document is written is specified), the acquisition unit 108 may acquire the selected target document (using the specified area as the target document).
[0029] The first generation unit 109 structures the target document, that is, it generates text information, attribute information of the components, and coordinate information of the components for at least one component within the target document.
[0030] In this embodiment, "at least one component within the target document" refers to elements contained within the target document, such as strings of text, images, numbers, graphs, tables, etc. For example, the document shown in Figure 2 consists of a title 21, multiple strings of text 22, a figure 23, a subtitle 24, multiple strings of text 25, multiple strings of text 26, a subtitle 27, multiple strings of text 28, multiple strings of text 29, and so on.
[0031] The text information generated by the first generation unit 109 is the string contained in the constituent elements. For example, constituent elements do not necessarily contain strings, such as graphs that do not contain strings, or logos and illustrations. If a constituent element does not contain a string, the first generation unit 109 generates attribute information and coordinate information. If a predetermined number of constituent elements among at least one or more elements in the target document do not contain strings, the first generation unit 109 generates text information, attribute information, and coordinate information for constituent elements that contain strings, and then generates attribute information and coordinate information for constituent elements that do not contain strings, and then generates structured data based on the generated text information, attribute information, and coordinate information.
[0032] The attribute information generated by the first generation unit 109 is the type used to classify the constituent elements. For example, the attribute of title 21 is "title," the attribute of string 22 is "a text describing the outline of the content of the target document 20, associated with title 21," the attribute of figure 23 is "a diagram describing the outline of the content of the target document 20, associated with title 21, and a text explaining it," the attribute of subtitle 24 is "subtitle," the attribute of string 25 is "a text describing the content of "research content," associated with subtitle 24," the attribute of string 26 is "a bulleted list of the content of "research content," associated with subtitle 24," the attribute of subtitle 27 is "subtitle," the attribute of string 28 is "a text describing the content of "technical features," associated with subtitle 27," and the attribute of string 29 is "a bulleted list of the content of "technical features," associated with subtitle 27." The attributes of the multiple Figures 231 that make up Figure 23 are "summary diagrams of the content of the target document 20 linked to Title 21," and the attributes of the multiple strings 232 are "texts that explain the summary diagrams of the content of the target document 20 linked to Title 21."
[0033] The coordinate information generated by the first generation unit 109 is information that specifies the position of a component within the target document 20, and may include, for example, coordinates indicating the position of the target document 20 on the page, the shape of the component, and information such as its vertical and horizontal lengths.
[0034] The splitting unit 110 divides the target document 20 into desired constituent units based on text information, attribute information, and coordinate information to generate at least one divided text. Here, a constituent unit may be a set of strings describing a specific topic. The attribute information of each string in the set of strings describing a specific topic may be linked to attribute information such as chapter, section, title, or subtitle. Alternatively, the splitting unit 110 may divide at least one string in the target document 20 for each string whose attribute information includes a title, thereby generating at least one divided text. The splitting unit 110 analyzes topic shifts within the target document 20 through the attribute information and divides the target document 20 for each description of a specific topic, i.e., for each desired constituent unit, to generate divided text.
[0035] Figure 3 shows the divided text generated by the division unit 110, which divides the target document 20 shown in Figure 2 into desired constituent units based on text information, attribute information, and coordinate information. In the example shown in Figure 3, the target document 20 is divided into divided texts 31, 32, and 33. Divided text 31 includes the title 21, string 22, and figure 23. Divided text 32 includes the subtitle 24, string 25, and string 26. Divided text 33 includes the subtitle 27, string 28, and string 29.
[0036] The components of segmented text 31, namely title 21, string 22, and figure 23, all have attributes linked to the title or title 21, and describe an overview of the content of the target document 20. In other words, segmented text 31 is a set of strings that describe the specific topic of "an overview of the content of the target document 20." The components of segmented text 32, namely subtitle 24, string 25, and string 26, all have attributes linked to the subtitle or subtitle 24, and describe the content of "research content." Segmented text 31 is a set of strings that describe the specific topic of "the content of 'research content'." The components of segmented text 33, namely subtitle 27, string 28, and string 29, all have attributes linked to the subtitle or subtitle 27, and describe the content of "technical features." Segmented text 31 is a set of strings that describe the specific topic of "the content of 'technical features'."
[0037] The extraction unit 111 extracts at least one string relating to a specific topic in each segmented text from among the at least one string having the attribute of "document" contained in each constituent unit of the segmented texts 31, 32, and 33. The at least one string relating to a specific topic in a segmented text is a string that corresponds to a keyword that describes the specific topic described in the segmented text.
[0038] As an example, we will explain the case in which the extraction unit 111 extracts at least one string related to a specific topic in the divided text 31 from among at least one string having the "text" attribute contained in each constituent unit of the divided text 31. Figure 4 shows at least one string having the "text" attribute contained in each constituent unit of the divided text 31. In Figure 4, strings 41, 42, 43, 44, 45, 46, and 47 are shown as at least one string having the "text" attribute. The extraction unit 111 extracts at least one string related to a specific topic in the divided text 31 from among strings 41 to 47. The specific topic in the divided text 31 is "summary of the content of the target document 20," and as keywords that explain the specific topic, the extraction unit 111 extracts the multiple strings shown in Figure 5. The multiple strings shown in Figure 5 are extracted not only from string 22, which consists only of strings, but also from the multiple strings 232 in Figure 23.
[0039] The second generation unit 112 generates QA data for each of the at least one segmented sentences generated by the segmentation unit 110, using a large-scale language model. This QA data contains question and answer sentences that include at least one string extracted by the extraction unit 111, based on the content of each segmented sentence. The second generation unit 112 may generate QA data for, for example, a segmented sentence specified by user input from among the at least one segmented sentences, or it may generate QA data for all segmented sentences from among the at least one segmented sentences, for each segmented sentence. Figure 6 shows an example of question and answer sentences generated by the second generation unit 112. The question and answer sentences shown in Figure 6 are questions and answers concerning keywords that describe a specific topic shown in Figure 4.
[0040] The output unit 113 outputs the QA data to, for example, the display unit 106.
[0041] As described above, according to the information processing device of this embodiment, a target document can be divided into desired constituent units using OCR to generate divided texts, and after extracting the strings contained in each divided text, a question and answer can be generated using VLM, thereby generating a highly comprehensive question and answer.
[0042] To compare with the information processing apparatus according to this embodiment, a case in which the target document is not divided into desired constituent units will be described with reference to Figure 7. Figure 7 shows a case in which the target document 20 shown in Figure 2 is divided into constituent units rather than into desired constituent units. In Figure 7, the target document 20 is divided into constituent units of a title 21, multiple strings 22, a figure 23, a subtitle 24, multiple strings 25, multiple strings 26, a subtitle 27, multiple strings 28, and multiple strings 29.
[0043] As shown in Figure 7, if the extraction unit 111 extracts at least one string related to a specific topic from each component, and the second generation unit 112 generates QA data containing question and answer sentences that include at least one string extracted by the extraction unit 111, the range that the second generation unit 112 targets for generating QA data is narrow. As a result, the obtained result in the case shown in Figure 7 is highly comprehensive, but the amount of information needed to generate the QA data is insufficient, increasing the possibility of hallucination. Furthermore, the computational amount required to generate QA data for the entire target document 20 increases, and therefore the cost increases.
[0044] On the other hand, if the target document 20 shown in Figure 2 is not divided, and at least one string related to a specific topic is extracted from the entire target document 20 by the extraction unit 111, and QA data is generated by the second generation unit 112, the calculation to generate QA data for the entire target document 20 is performed only once. However, because the range that the second generation unit 112 targets for generating QA data is wide, the comprehensiveness is reduced, as characters in figures and tables, small characters, etc., are not extracted. Furthermore, although the amount of information for generating QA data is sufficient, the amount of text is excessively large, increasing the possibility of hallucination.
[0045] According to the information processing device of this embodiment, the target document is divided into desired constituent units to generate divided documents, the extraction unit 111 extracts at least one string related to a specific topic from the divided documents, and the second generation unit 112 generates QA data. As a result, comprehensiveness is sufficient and the possibility of hallucination is reduced. Furthermore, the computational amount required to generate QA data for the entire target document 20 can be reduced, and therefore costs can be reduced.
[0046] The information processing method according to this embodiment will be explained with reference to the flowchart in Figure 8.
[0047] In step S801, the acquisition unit 108 acquires the target document (acquisition step).
[0048] In step S802, text information, attribute information of the component, and coordinate information of the component are generated for at least one component within the target document (first generation step).
[0049] In step S803, the division unit 110 divides the target document into desired constituent units based on the text information, attribute information, and coordinate information of one or more constituent units to generate divided documents (division step).
[0050] In step S804, the extraction unit 111 extracts at least one string from each constituent unit of the divided text (extraction step).
[0051] In step S805, the second generation unit 112 generates QA data containing question and answer sentences that include the string, using a large-scale language model (second generation step).
[0052] In step S806, the output unit 113 outputs QA data (output step).
[0053] As stated above, the present invention naturally includes various embodiments and the like that are not described herein. Therefore, the technical scope of the present invention is determined solely by the inventive features relating to the claims that are reasonable based on the above description.
[0054] The programs of each embodiment of this disclosure may be provided stored in a storage medium readable by the information processing device. The storage medium is a “non-temporary tangible medium” capable of storing programs. The programs include, for example, software programs and control programs. When each functional unit of the information processing device is implemented by software, the information processing device functions as an acquisition unit 108, a first generation unit 109, a splitting unit 110, an extraction unit 111, a second generation unit 112, and an output unit 113 by the processor executing a program loaded into memory.
[0055] The storage medium may, where appropriate, include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tape, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable storage medium, or two or more suitable combinations thereof. The storage medium may, where appropriate, be volatile, non-volatile, or a combination of volatile and non-volatile.
[0056] Furthermore, the program of this disclosure may be provided to an information processing device via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves).
[0057] Furthermore, each embodiment of this disclosure may also be realized in the form of a data signal embedded in a carrier wave, where the program is embodied by electronic transmission. The programs of this disclosure may be implemented using, for example, scripting languages such as JavaScript® and Python®, or languages such as C, Go, Swift®, Koltin®, and Java®.
[0058] According to each aspect of this disclosure described above, by providing an information processing device capable of generating highly comprehensive question and answer responses, it is possible to contribute to achieving Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation." [Explanation of Symbols]
[0059] 10 Information Processing Devices 101 CPU 102 ROM 103 RAM 104 Storage section 105 I / O (Input / Output Interface) 106 Display section 107 Input section 108 Acquisition Department 109 1st generation part 110 Split section 111 Extraction part 112 Second generation part 113 Output section 20 Target Documents 21 Titles 22 strings Figures 23, 231 24, 27 Subtitles 25, 26, 28, 29, 232, 41, 42, 43, 44, 45, 46, 47 string 31, 32, 33 divided sentences
Claims
1. The acquisition unit acquires the target document, A first generation unit generates text information, attribute information, and coordinate information for each of one or more components within the target document. A division unit that divides the target document into desired constituent units and generates at least one divided document based on the generated text information, attribute information, and coordinate information, An extraction unit that extracts at least one string from each of the at least one of the divided sentences, A second generation unit generates QA data containing question and answer sentences that include the string, based on at least one of the segmented sentences, using a large-scale language model. The output unit that outputs the aforementioned QA data and An information processing device characterized by comprising:
2. The information processing apparatus according to claim 1, characterized in that the constituent unit to be divided by the division unit is a set of strings describing a specific topic.
3. The information processing apparatus according to claim 2, characterized in that the set of strings describing the aforementioned specific topic is associated with attribute information of a chapter, section, title, or subtitle.
4. The information processing apparatus according to claim 1, characterized in that the constituent unit to be divided by the division unit is a constituent unit that divides the target document into at least one string for each of the at least one strings whose attribute information includes a title.
5. The information processing apparatus according to claim 2, characterized in that the at least one string extracted by the extraction unit is a string relating to the specific topic in the divided text.
6. Computers The acquisition step to obtain the target document, A first generation step in which text information, attribute information, and coordinate information are generated for each of one or more components within the target document, A splitting step of dividing the target document into desired constituent units based on the generated text information, attribute information, and coordinate information to generate at least one divided document, Extraction step of extracting at least one string from each of at least one of the divided sentences, A second generation step involves using a large-scale language model to generate QA data in which question and answer sentences containing the string are described, based on at least one of the segmented sentences, The output step of outputting the aforementioned QA data and An information processing method characterized by performing the following.
7. On the computer, The acquisition function retrieves the target document, A first generation function that generates text information, attribute information, and coordinate information for each of one or more components within the aforementioned target document, A splitting function that divides the target document into desired constituent units and generates at least one divided document based on the generated text information, attribute information, and coordinate information, An extraction function that extracts at least one string from each of at least one segmented sentences, A second generation function that uses a large-scale language model to generate QA data containing question and answer sentences that include the string, based on at least one of the segmented sentences, for each of the at least one of the segmented sentences, Output function for outputting the aforementioned QA data An information processing program that makes this possible.