Information processing device, information processing method, and program

Abstract

An information processing device and method for improving search expansion generation (RAG) accuracy are provided. [Solution] The information processing device includes a document data acquisition unit that acquires document data, a document processing unit that performs layout analysis of the document data to identify multiple areas included in the document data and recognizes the contents of the multiple areas to extract multiple elements, a structured data generation unit that generates structured data by structuring the multiple elements, a graph data generation unit that generates graph data in which the multiple elements are multiple nodes based on the relationships between the multiple elements included in the structured data, a determination unit that determines a suggested correction node from the multiple nodes included in the graph data and a suggested correction for the suggested correction node, a display control unit that controls the display of the graph data and the suggested correction for the suggested correction node, and a correction unit that corrects a portion of the structured data that corresponds to the suggested correction node based on a user instruction for the suggested correction.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 describes an information processing device that outputs a structured character string. [Prior art document] [Patent Documents] [Patent Document 1] Patent No. 7588260 Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided an information processing device. The information processing device may include a document data acquisition unit that acquires document data. The information processing device may include a document processing unit that analyzes the layout of the document data to identify multiple areas included in the document data and recognizes the contents of the multiple areas to extract multiple elements. The information processing device may include a structured data generation unit that generates structured data by structuring the multiple elements. The information processing device may include a graph data generation unit that generates graph data in which the multiple elements included in the structured data are multiple nodes based on relationships between the multiple elements. The information processing device may include a determination unit that determines a suggested correction node from the multiple nodes included in the graph data and a correction proposal for the suggested correction node. The information processing device may include a display control unit that controls display of the graph data and the correction proposal for the suggested correction node. The information processing device may include a correction unit that corrects a portion of the structured data corresponding to the suggested correction node based on a user instruction for the correction proposal.

[0004] In the information processing device, the determination unit may determine at least one of the plurality of nodes, a node to be connected to another node, a node to be integrated with another node, and a node to be split, as the proposed modification node, and for the node to be connected to the other node, determine a modification proposal indicating the node to be connected, for the node to be integrated with the other node, determine a modification proposal indicating the node to be integrated, and for the node to be split, determine a modification proposal indicating the plurality of nodes after split.

[0005] In any of the information processing devices, the determination unit may determine, as the proposed correction node, a node among the plurality of nodes whose corresponding element content should be corrected; the display control unit may display, as the proposed correction, an area of ​​the document data from which an element corresponding to the proposed correction node has been extracted; and when the correction unit receives a correction instruction from the user after the display control unit has displayed the area of ​​the document data, it may correct the portion of the structured data corresponding to the proposed correction node in accordance with the correction instruction.

[0006] In any of the information processing devices, the structured data generation unit may generate the structured data in which metadata related to the content of the element is assigned to each of the plurality of elements, the determination unit may determine a node among the plurality of nodes, which has the metadata assigned to a corresponding element, as the proposed correction node, and may determine the proposed correction of the metadata as a proposed correction for the proposed correction node, the display control unit may display the proposed correction of the metadata as a proposed correction for the proposed correction node, and the correction unit may correct the metadata assigned to the element of the structured data corresponding to the proposed correction node based on an instruction by the user regarding the proposed correction.

[0007] Any of the information processing devices may include a RAG functional unit that searches the structured data generated by the structured data generation unit for related information related to an input prompt, inputs the search results and the prompt into a large-scale language model, and outputs an answer to the prompt based on the output from the large-scale language model; and an incorrect answer cause identification unit that inputs a prompt associated with a model answer into the RAG functional unit, compares the answer output from the RAG functional unit with the model answer, and, if it is determined to be an incorrect answer, identifies the cause of the incorrect answer, and the determination unit may determine the correction suggestion node and the correction suggestion based on the cause of the incorrect answer identified by the incorrect answer cause identification unit.

[0008] In any of the information processing devices, when the cause of the incorrect answer identified by the incorrect answer cause identification unit is that the information required to correctly answer the prompt is divided into two or more elements, the determination unit may determine two or more nodes corresponding to the two or more elements as the correction suggestion nodes, and determine a correction suggestion that combines the two or more nodes.

[0009] In any of the information processing devices, if the cause of the incorrect answer identified by the incorrect answer cause identification unit is a mixture of information for correctly answering the prompt and other information within a single element, the determination unit may determine the node corresponding to the element as the correction suggestion node, and determine correction suggestions indicating multiple nodes after dividing the node.

[0010] In any of the information processing devices, if the cause of the incorrect answer identified by the incorrect answer cause identification unit is a transcription error for an area of ​​the document data from which an element containing the related information is extracted, the determination unit may determine a node corresponding to the element as the correction suggestion node, and determine a correction suggestion that includes the element and the area of ​​the document data from which the element was extracted.

[0011] In any of the information processing devices, if the cause of the incorrect answer identified by the incorrect answer cause identification unit is that related information related to the prompt is included in the structured data but could not be found by a search, the determination unit may determine a node among the multiple elements corresponding to an element that includes the related information as a correction suggestion node, and determine a correction suggestion for metadata attached to the element as a correction suggestion for the correction suggestion node.

[0012] In any of the information processing devices, if the cause of the incorrect answer identified by the incorrect answer cause identification unit is that related information related to the prompt is included in the structured data but could not be found by a search, the determination unit may determine a node among the multiple elements corresponding to an element that includes the related information as a correction suggestion node, and determine a correction suggestion that suggests adding the information included in the prompt to the metadata assigned to the element.

[0013] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include a document data acquisition step of acquiring document data. The information processing method may include a document processing step of analyzing the layout of the document data to identify multiple regions included in the document data and recognizing the contents of the multiple regions to extract multiple elements. The information processing method may include a structured data generation step of generating structured data by structuring the multiple elements. The information processing method may include a graph data generation step of generating graph data in which the multiple elements included in the structured data form multiple nodes based on relationships between the multiple elements. The information processing method may include a determination step of determining a suggested correction node from the multiple nodes included in the graph data and a correction suggestion for the suggested correction node. The information processing method may include a display control step of controlling the display of the graph data and the correction suggestion for the suggested correction node. The information processing method may include a correction step of correcting a portion of the structured data corresponding to the suggested correction node based on a user instruction for the correction suggestion.

[0014] According to one embodiment of the present invention, there is provided a program for causing a computer to execute the information processing method.

[0015] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 illustrates an example of an information processing device 100. [Figure 2] FIG. 4 is an explanatory diagram for explaining structured data 400. [Figure 3] FIG. 10 is an explanatory diagram for explaining an example of a cause of an incorrect answer. [Figure 4] 1 shows an example of a functional configuration of an information processing device 100. [Figure 5] 10 shows a schematic example of a display of graph data 500. [Figure 6] 6 shows a schematic example of a display of a correction suggestion menu 600. [Figure 7] 6A and 6B show a schematic example of a display of an element modification proposal 620. [Figure 8] 6 shows a schematic example of a display of transcription correction suggestions 630. [Figure 9] 6A and 6B show an example of a layout correction screen 640. [Figure 10] 6A and 6B show an example of a layout correction screen 640. [Figure 11] 10A and 10B show schematic examples of corrections on the layout correction screen 640. [Figure 12] 10A and 10B show schematic examples of corrections on the layout correction screen 640. [Figure 13] 6A and 6B show an example of a layout correction screen 650. [Figure 14] 6 shows a schematic example of a display of metadata revision suggestions 660. [Figure 15] 10A and 10B show an example of processing by the recognition result handling unit 121. [Figure 16] An example of the hardware configuration of a computer 1200 that functions as the information processing device 100 is shown in schematic form. DETAILED DESCRIPTION OF THE INVENTION

[0017] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0018] Instead of fine-tuning generative AI (artificial intelligence) and having it retrain itself on new information, a method called Retrieval Augmented Generation (RAG) combines a separate database (DB) containing proprietary data such as internal documents with a large-scale language model that receives the search results and generates easy-to-read text responses. Traditionally, preparing structured data (e.g., MarkDown format data) referenced by a large-scale language model in RAG requires transcribing text from images and charts in documents. For example, Microsoft's SaaS (Software as a Service) OCR (Optical Character Recognition) service, Document Intelligence, can be used to convert document data into text and automatically structure it. However, because automatically structured element data is written in MarkDown format, such as paragraph-by-paragraph, it is difficult for users to grasp the relationships and structure of the entire element data. Furthermore, it is difficult for users to directly modify and organize element data, and the element data is divided into pages, which is not an optimal structural unit for AI. The automatically structured element data is input to the RAG as AI input data. The information processing device 100 according to this embodiment enables, for example, during the verification and development stage of the RAG, the user to improve the accuracy of the RAG by simply accepting corrections suggested by the system, without having to directly correct the element data themselves.

[0019] FIG. 1 schematically illustrates an example of an information processing device 100. The information processing device 100 is capable of communicating with a communication terminal 20 via a network 80. The information processing device 100 may be a server device, a personal computer, a smartphone, a tablet terminal, or the like. The communication terminal 20 is used by a user 22. The communication terminal 20 may be a personal computer, a smartphone, a tablet terminal, or the like. The network 80 may include a cloud. The network 80 may include the Internet. The network 80 may include a mobile communication network. The network 80 may include a LAN (Local Area Network).

[0020] The information processing device 100 may be capable of executing a RAG function using document data 300. For example, as illustrated in FIG. 2 , the information processing device 100 performs a layout analysis of the document data 300 to identify multiple regions 320 included in the document data 300, recognizes the contents of the multiple regions 320, extracts elements from each of the multiple regions 320, and structures the extracted multiple elements to generate structured data 400. The information processing device 100 may generate one piece of structured data 400 from one piece of document data 300, or may generate one piece of structured data 400 from multiple pieces of document data 300. The information processing device 100 may assign metadata to each of the multiple elements included in the structured data 400.

[0021] For example, the information processing device 100 receives a prompt input from the communication terminal 20, searches for related information related to the prompt from structured data 400, inputs the search results and the prompt into an LLM (Large Language Model) 200, generates an answer to the prompt based on the output from the LLM 200, and outputs the answer to the communication terminal 20. The information processing device 100 may include its own LLM 200. Alternatively, the information processing device 100 may not include its own LLM 200 but may use an external LLM 200. For example, the information processing device 100 uses the LLM 200 via the network 80. Examples of the external LLM 200 include ChatGPT, Gemini, and Claude (registered trademark), but are not limited to these, and any existing LLM may be used.

[0022] For example, when the prompt is a question, the information processing device 100 outputs an answer to the question. If the information for answering the question is contained in one or more original document data 300, then in theory, an appropriate answer to the question should be able to be provided. However, this may not be possible due to various factors, and the information processing device 100 may provide an incorrect answer to the question or may not be able to provide an answer to the question at all. The cause of this problem may be a defect in the structured data 400.

[0023] For example, one cause of this is a failure to analyze the layout of the document data 300. For example, the information required for the answer may have been split into multiple elements. In conventional structuring, information is split on a page-by-page basis, and as shown in FIG. 3, when certain information spans multiple pages 310, it is split and managed, which may not be an easy-to-search information split. Another cause is the opposite, where different information is inappropriately included in a single element.

[0024] For example, the cause may be an error in transcribing the document data 300. Specifically, this may be a failure in OCR or a failure in structuring figures and tables.

[0025] For example, when metadata is configured to be assigned to multiple elements, the metadata may be inappropriate, such as a lack of metadata needed to narrow down the search, or incorrect metadata that does not correspond to the content of the elements.

[0026] To solve this problem and enable the information processing device 100 to provide an appropriate response to the prompt, for example, a prompt associated with a model answer may be input into the information processing device 100, and if the answer output from the information processing device 100 is compared with the model answer and the answer is found to be incorrect, the user 102 may identify the cause and correct the structured data 400. However, it is often difficult for the user 102 to identify the cause of the incorrect answer and correct the structured data 400. This is because the structured data 400 is generally not in a user-friendly format, such as Markdown, making it difficult for the user 102 to grasp the overall relationship and structure of multiple elements. To address this issue, the user 102 may view the original document data 300, identify the portion to be corrected, and correct the corresponding portion in the structured data 400. However, it is difficult to identify the portion in the structured data 400 that corresponds to a specific portion of the document data 300. Furthermore, even if it becomes necessary to divide or combine elements, it is difficult for the user 102 to determine which elements should be divided and how, or which elements should be combined.

[0027] The information processing device 100 according to this embodiment has a function of supporting the user 102 in correcting the structured data 400 .

[0028] 4 schematically illustrates an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 110, a registration unit 112, a display control unit 114, a document data acquisition unit 116, a document processing unit 118, a correction unit 120, a recognition result correspondence unit 121, a structured data generation unit 122, a graph data generation unit 124, a determination unit 126, a RAG function unit 130, an error cause identification unit 132, and a model update unit 136. Note that it is not essential for the information processing device 100 to include all of these units.

[0029] The registration unit 112 registers various types of data. The registration unit 112 may register data input by the user 102. The data registered by the registration unit 112 is stored in the storage unit 110. For example, the registration unit 112 registers model Q&A data in which multiple data sets each including a prompt and a model answer to the prompt are registered.

[0030] The display control unit 114 controls the display of various data. For example, the display control unit 114 displays various data on a display provided in the information processing device 100. For example, the display control unit 114 transmits various data to a communication terminal or the like used by the user 102 and causes the data to be displayed.

[0031] The document data acquisition unit 116 acquires document data 300. The document data 300 is, for example, a PDF (Portable Document Format) file. The format of the document data 300 is not limited to this and may be any existing format. The document data 300 may be image data of a document. For example, the document data acquisition unit 116 acquires one or more pieces of document data 300 prepared by the user 102. The document data acquisition unit 116 stores the acquired one or more pieces of document data 300 in the storage unit 110. The display control unit 114 may display the one or more pieces of document data 300 acquired by the document data acquisition unit 116.

[0032] The document processing unit 118 processes the document data 300 acquired by the document data acquisition unit 116. The document processing unit 118 may perform a layout analysis of the document data 300 to identify multiple regions 320 included in the document data 300, recognize the contents of the multiple regions 320, and extract multiple elements. The document processing unit 118 may use a known rule-based application, an AI application, or the like to perform a layout analysis of the document data 300 to identify multiple regions 320 included in the document data 300. The document processing unit 118 may use a known rule-based application, an AI application, or the like to recognize the contents of the multiple regions 320 and extract multiple elements. For example, the document processing unit 118 may extract character data included in the region 320, perform character recognition and transcription of character images included in the region 320, or analyze diagrams and tables included in the region 320 to extract the contents of the diagrams and tables. The document processing unit 118 may identify parent-child relationships between multiple elements in the multiple regions 320. The document processor 118 may use known rule-based applications, AI applications, or the like to identify parent-child relationships between the elements in the regions 320. The document processor 118 may output a confidence level of the recognition result for each of the regions 320.

[0033] For example, the document processing unit 118 uses a document recognition AI that receives document data as input and outputs the results of identifying multiple regions included in the document data, the results of recognizing the contents of the multiple regions, and the parent-child relationships of the multiple regions. The document processing unit 118 may input document data 300 acquired by the document data acquisition unit 116 to the document recognition AI, thereby identifying multiple regions 320 included in the document data 300 and recognizing the contents of the multiple regions 320. The document recognition AI may further output the reliability of the recognition results for each of multiple elements that are the recognition results of the multiple regions 320. The document recognition AI may be an AI generated by machine learning that receives document data as input and outputs the results of identifying multiple regions included in the document data, the results of recognizing the contents of the multiple regions, and the parent-child relationships of the multiple regions. The document recognition AI may also be a generation AI.

[0034] The display control unit 114 may display the document data 300 and a plurality of area objects indicating the ranges of a plurality of areas 320 included in the document data 300. The display control unit 114 may display the plurality of area objects superimposed on the document data 300. The area object may be, for example, an object that surrounds the area 320. The area object may be a so-called BBox (Bounding Box).

[0035] The display control unit 114 may further display the recognition result for at least one of the multiple areas 320. For example, the display control unit 114 displays the recognition result by the document processing unit 118 for only the area 320 selected by the user 102 among the multiple areas 320. For example, the display control unit 114 displays the recognition results for all of the multiple areas 320. As a specific example, the display control unit 114 may display the multiple elements of the multiple areas 320 in a tree structure according to the parent-child relationships between the multiple elements.

[0036] The correction unit 120 corrects the processing result of the document data 300 by the document processing unit 118 based on an instruction from the user 102 .

[0037] For example, the correction unit 120 corrects the recognition result of a region selected by the user 102 from among multiple regions of the document data 300 displayed by the display control unit 114, based on an instruction from the user 102. The correction unit 120 corrects, for example, characters included in the recognition result, based on an instruction from the user 102. For example, the display control unit 114 displays multiple region objects superimposed on the document data 300, and when the user 102 selects a region for which the recognition result is to be confirmed, displays only the recognition result of that region. The user 102 checks the content of that region of the document data 300 and the displayed recognition result, and corrects the recognition result as necessary. This allows the user 102 to clearly understand the recognition result corresponding to the region of the document data 300 and consider whether correction is necessary.

[0038] For example, the correction unit 120 corrects a region object selected by the user 102 from among multiple region objects in the document data 300 displayed by the display control unit 114, based on an instruction from the user 102. For example, the correction unit 120 changes the range of the region indicated by the region object selected by the user 102, based on an instruction from the user 102. If the user 102 checks the region object displayed by the display control unit 114 and discovers a failure in the layout analysis by the document processing unit 118, the user 102 changes the shape and size of the region object to correct the erroneous range indicated by the region object. When the correction unit 120 corrects the region object, the document processing unit 118 recognizes the content of the region indicated by the region object whose range has been changed by the correction unit 120, and extracts elements. This makes it easier for the user 102 to understand the results of the layout analysis by the document processing unit 118 and correct any errors.

[0039] For example, the correction unit 120 corrects the parent-child relationships of multiple elements in accordance with a correction instruction from the user 102 regarding the parent-child relationships of multiple elements displayed in a tree structure by the display control unit 114. The user 102 browses the multiple elements displayed in the tree structure, checks the parent-child relationships recognized among the multiple elements, and if a recognition error is found, issues a correction instruction to correct the parent-child relationships. This makes it easier for the user 102 to understand the recognition results of the parent-child relationships of multiple elements by the document processing unit 118 and to correct the error.

[0040] The display control unit 114 may control the display of the document data 300 and the plurality of area objects so as to reflect the reliability of the recognition results of each of the plurality of areas 320 by the document processing unit 118.

[0041] For example, the display control unit 114 controls the display so that each reliability is superimposed on each of the multiple regions 320. The display control unit 114 may display a numerical value representing the reliability as the reliability, or may display the reliability level. The reliability level may be two levels, low and high, or three levels, low, medium, and high, or four or more levels.

[0042] For example, the display control unit 114 may perform control to identify and display the regions 320 in which the reliability of the recognition result is lower than a predetermined threshold among the multiple regions 320. As a specific example, the display control unit 114 places a character string indicating low reliability only in the regions 320 in which the reliability of the recognition result is lower than a predetermined threshold among the multiple regions 320. As another specific example, the display control unit 114 performs control to highlight the regions 320 in which the reliability of the recognition result is lower than a predetermined threshold among the multiple regions 320.

[0043] This allows the user 102 to identify which of the multiple areas 320 needs to be checked in more detail, which contributes to making the checking by the user 102 more efficient and preventing oversights.

[0044] The recognition result handling unit 121 executes processing for a recognition result with a low reliability among the recognition results of the plurality of regions 320 by the document processing unit 118.

[0045] For example, the recognition result correspondence unit 121 generates a correction proposal for a recognition result with low reliability. For example, when the document processing unit 118 outputs the reliability of the recognition result for the region 320, the recognition result correspondence unit 121 compares the reliability with a predetermined threshold. The threshold may be arbitrarily settable or may be changeable after being set. When the recognition result correspondence unit 121 determines that the reliability of the recognition result is lower than the threshold, the recognition result correspondence unit 121 generates a correction proposal for the recognition result. Then, the display control unit 114 displays the correction proposal generated by the recognition result correspondence unit 121. The user 102 may confirm the displayed correction proposal and, if determined to be correct, may approve the correction proposal. In response to the user 102's approval of the correction proposal, the correction unit 120 corrects the recognition result in accordance with the correction proposal. In this way, when a recognition result with low reliability is actually incorrect, the user 102 can correctly correct the recognition result simply by approving the correction proposal. If the user 102 determines that the correction proposal is incorrect, the user 102 may correct the content of the correction proposal and then approve it, and the correction unit 120 may correct the recognition result in accordance with the correction proposal amended by the user 102. As a result, for example, if only a part of the correction proposal is incorrect, the user 102 can correct the recognition result by correcting only that part. If the user 102 determines that the correction proposal is incorrect, the user 102 may correct the recognition result to be corrected. When the correction unit 120 receives a correction instruction from the user 102 regarding the recognition result to be corrected in the correction proposal, the correction unit 120 corrects the recognition result in accordance with the correction instruction. As a result, if the user 102 notices an error in the target recognition result through the correction proposal but the content of the correction proposal is incorrect, it is possible to reliably make the correction in line with the user 102's intention, even though it requires the user 102 to make the correction.

[0046] The recognition result correspondence unit 121 may generate a correction suggestion using an AI model capable of generating a correction suggestion (sometimes referred to as a correction suggestion capable model). The correction suggestion capable model may be a generation AI. The correction suggestion capable model may be a document recognition AI used by the document processing unit 118. The correction suggestion capable model may be the LLM200. The correction suggestion capable model may be a model other than the document recognition AI and the LLM200.

[0047] For example, the recognition result correspondence unit 121 inputs the document data 300, the recognition result of the document data 300 by the document processing unit 118, area information indicating one or more areas 320 among the multiple areas 320 whose reliability is lower than a threshold, and an instruction to create a correction suggestion for the recognition result of the one or more areas 320 to the correction suggestion capable model, and obtains the correction suggestion output from the correction suggestion capable model. The recognition result of the document data 300 may include identification results of the multiple areas. The recognition result of the document data 300 may include recognition results of the contents of the multiple areas.

[0048] The recognition result correspondence unit 121 may further input information other than the above to the revision suggestable model. For example, the recognition result correspondence unit 121 further inputs the reliability of each of the one or more regions 320 to the revision suggestable model. This makes it possible to cause the revision suggestable model to output a revision suggestion with a smaller amount of revision for a recognition result with a higher reliability, and to output a revision suggestion with a larger amount of revision for a recognition result with a lower reliability, thereby making it possible to make revision suggestions according to the reliability.

[0049] For example, the recognition result correspondence unit 121 further inputs correction history data including multiple past correction histories of the recognition results of the region 320 of the document data 300 into the correction suggestable model. The correction history data may include a history of past corrections of the recognition results by humans, a history of accepted corrections among correction suggestions provided by the information processing device 100 to humans, and a history of accepted corrections among correction suggestions provided by other devices to humans. The correction history data may be registered in advance and stored in the storage unit 110. By further inputting the correction history data into the correction suggestable model, it is possible to cause the correction suggestable model to output correction suggestions that reflect past correction records, thereby reducing the possibility of making erroneous correction suggestions and increasing the possibility of making appropriate correction suggestions.

[0050] The revision history included in the revision history data may be associated with the category of the region 320 to be revised. Examples of the region 320 categories include, but are not limited to, titles, paragraphs, figures, tables, etc. The revision history included in the revision history data may be associated with the cause of the revision. Examples of the cause of the revision include, but are not limited to, false detection of a logo, misalignment of a bounding box, failure to interpret a merged cell structure in a table, OCR errors, and reversal or incorrect order of parent-child relationships in titles, etc. This makes it possible to cause the revision suggestion model to output revision suggestions based on the revision history data corresponding to the category of the region 320 to be revised, or to output revision suggestions based on the revision history corresponding to the cause of the revision in the region 320 to be revised, thereby contributing to improving the accuracy of the revision suggestions.

[0051] When the correction unit 120 corrects the recognition result in accordance with the correction proposal in response to the user 102's approval of the correction proposal, the recognition result correspondence unit 121 may add the correction history to the correction history data. Similarly, when the correction unit 120 corrects the recognition result in accordance with the correction proposal amended by the user 102, or when the correction unit 120 corrects the recognition result in accordance with an instruction from the user 102, the recognition result correspondence unit 121 may add the correction history to the correction history data. This makes it possible to enrich the correction record included in the correction history data and continuously improve quality.

[0052] For example, the recognition result correspondence unit 121 identifies the cause of a recognition result with low reliability. For example, when the document processing unit 118 outputs the reliability of the recognition result of the area 320, the recognition result correspondence unit 121 compares the reliability with a predetermined threshold. The threshold may be set arbitrarily and may be changeable after being set. When the recognition result correspondence unit 121 determines that the reliability of the recognition result is lower than the threshold, the recognition result correspondence unit 121 identifies the cause of the low reliability. Then, the display control unit 114 displays the cause identified by the recognition result correspondence unit 121. When the correction unit 120 receives an instruction from the user 102 to correct the recognition result, the correction unit 120 corrects the recognition result in accordance with the correction instruction from the user 102.

[0053] The recognition result correspondence unit 121 may identify the cause of low reliability using an AI model (sometimes referred to as an analyzable model) that can analyze the cause. The analyzable model may be a generative AI. The analyzable model may be a document recognition AI used by the document processing unit 118. The analyzable model may be the LLM200. The analyzable model may be a model other than the document recognition AI and the LLM200.

[0054] For example, the recognition result correspondence unit 121 inputs document data 300, the recognition result of the document data 300 by the document processing unit 118, area information indicating one or more areas 320 among the multiple areas 320 whose reliability is lower than a threshold, and instructions to analyze the cause of the low reliability of the recognition result of the one or more areas 320 and output the analysis results into the analyzable model, and obtains the analysis results output from the analyzable model.

[0055] The recognition result correspondence unit 121 may further input information other than the above to the analyzable model. For example, the recognition result correspondence unit 121 further inputs the reliability of each of the one or more regions 320 to the analyzable model. For example, the recognition result correspondence unit 121 further inputs correction history data to the analyzable model. When the correction unit 120 corrects the recognition result in accordance with a correction instruction from the user 102, the recognition result correspondence unit 121 may add the correction history to the correction history data.

[0056] If the document processing unit 118 is configured to output, in addition to the reliability of the recognition result, the cause of the reliability when the reliability of the recognition result is low, the recognition result correspondence unit 121 may identify the cause by obtaining the cause output by the document processing unit 118.

[0057] The recognition result correspondence unit 121 may execute both generating correction suggestions for recognition results with low reliability and identifying the causes of the low reliability. In this case, the correction suggestable model may also serve as an analyzable model and may perform both generating correction suggestions and outputting analysis results. The display control unit 114 may display both the correction suggestions generated by the recognition result correspondence unit 121 and the causes identified by the recognition result correspondence unit 121.

[0058] The structured data generation unit 122 generates structured data 400 corresponding to the document data 300 by structuring the multiple elements extracted by the document processing unit 118. The structured data generation unit 122 may generate one piece of structured data 400 from one piece of document data 300, or may generate one piece of structured data 400 from multiple pieces of document data 300. When correction is made by the correction unit 120, the structured data generation unit 122 may generate structured data 400 corresponding to the document data 300 based on the multiple elements extracted by the document processing unit 118 and the correction content made by the correction unit 120. The structured data generation unit 122 may generate structured data 400 by structuring the multiple elements after correcting the elements corrected by the correction unit 120 among the multiple elements extracted by the document processing unit 118. For example, for the area indicated by the area object whose range has been changed by the modification unit 120, the structured data generation unit 122 generates structured data 400 using the recognition result by the document processing unit 118 after the modification.

[0059] The structured data generation unit 122 may generate structured data 400 by assigning metadata related to the content of each of a plurality of elements. For example, the structured data generation unit 122 assigns words extracted from the elements or summaries of the content of the elements to the elements as metadata. The structured data generation unit 122 may extract words from elements or generate summaries of elements on a rule-based basis, or may use AI such as a generation AI to extract words from elements or generate summaries of elements.

[0060] When the display control unit 114 displays a plurality of area objects superimposed on the document data 300, if the user instructs to exclude any of the plurality of areas 320 indicated by the plurality of area objects, the structured data generation unit 122 may exclude the elements of the area 320 from the structured data 400. In this way, if an area in the document data 300 is recognized as an area but the user 102 does not want to include it in the structured data 400, the area can be excluded from the structured data 400.

[0061] When the display control unit 114 displays multiple area objects superimposed on the document data 300, and the user 102 specifies an area of ​​the document data 300 where no area object is superimposed, the document processing unit 118 may recognize the content of the area 320 and extract an element, and the structured data generation unit 122 may include the element in the structured data 400. This makes it possible to include information about the area in the structured data 400 when the document data 300 contains an area that was not automatically recognized as an area but that the user 102 wants to include in the structured data 400.

[0062] The graph data generation unit 124 generates graph data in which a plurality of elements are represented as a plurality of nodes based on the relationships between a plurality of elements included in the structured data 400 generated by the structured data generation unit 122. For example, the graph data generation unit 124 connects the nodes of elements determined to be related with each other by edges. As a specific example, the graph data generation unit 124 connects the nodes of elements in a parent-child relationship with edges, or connects the nodes of elements whose content similarity is higher than a predetermined threshold with edges.

[0063] The determining unit 126 determines a proposed correction node from among the multiple nodes included in the graph data generated by the graph data generating unit 124, and a proposed correction for the proposed correction node. The display control unit 114 may display the graph data and the proposed correction for the proposed correction node determined by the determining unit 126. The correcting unit 120 may correct a portion of the structured data 400 that corresponds to the proposed correction node, based on an instruction by the user 102 in response to the proposed correction.

[0064] For example, the determination unit 126 determines a node among multiple nodes that should be connected to other nodes as a modification counterproposal node, and determines a modification proposal indicating the node to be connected. As a specific example, the determination unit 126 determines a modification proposal including a modification proposal node, a node to be connected to the modification proposal node, and content proposing connecting these nodes. The display control unit 114 displays the graph data and the modification proposal for the modification proposal node, and in response to the user 102's approval of the modification proposal, the modification unit 120 may connect the modification proposal node and the node to be connected to the modification proposal node as proposed, and modify the relationships between elements corresponding to these nodes in the structured data 400. This allows the user 102 to modify the connection relationships between nodes, i.e., the relationships between elements, simply by approving the displayed modification proposal, thereby improving convenience for the user 102.

[0065] For example, the determination unit 126 determines a node among multiple nodes that should be merged with other nodes as a proposed revision node, and determines a revision proposal indicating the node to be merged. As a specific example, the determination unit 126 determines a revision proposal including the proposed revision node, a node to be merged with the proposed revision node, and content proposing merging these nodes. The display control unit 114 displays the graph data and the revision proposal for the proposed revision node, and in response to the user 102's approval of the revision proposal, the correction unit 120 may merge the proposed revision node with the node to be merged with the proposed revision node as proposed, and merge elements corresponding to these nodes in the structured data 400. This allows the user 102 to merge elements that should have been treated as multiple elements into a single element simply by approving the displayed revision proposal, thereby improving convenience for the user 102.

[0066] For example, the determination unit 126 determines a node to be split from among the multiple nodes as a proposed revision node and determines a revision proposal indicating the multiple nodes after the split. As a specific example, the determination unit 126 determines a revision proposal including the proposed revision node and content proposing splitting the proposed revision node into the multiple nodes after the split. The display control unit 114 displays the graph data and the revision proposal for the proposed revision node, and in response to the user 102's approval of the revision proposal, the correction unit 120 may split the proposed revision node as proposed and divide the element corresponding to the proposed revision node in the structured data 400 into multiple elements. This allows the user 102 to split an element that should have been split into multiple elements but has been treated as a single element into multiple elements simply by approving the displayed revision proposal, thereby improving convenience for the user 102.

[0067] For example, the determining unit 126 determines, from among the multiple nodes, a node whose corresponding element content should be corrected as a correction-proposed node. The display control unit 114 displays, as a correction proposal, an area of ​​the document data 300 from which an element corresponding to the correction-proposed node has been extracted. After the area of ​​the document data 300 has been displayed by the display control unit 114, the correcting unit 120 may, upon receiving a correction instruction from the user 102, correct the portion of the structured data 400 corresponding to the correction-proposed node in accordance with the correction instruction. This allows the user 102 to correct the element corresponding to the correction-proposed node after checking the content of the document data 300 displayed as the correction proposal, thereby improving convenience for the user 102.

[0068] For example, the determining unit 126 determines, among the multiple nodes, a node for which metadata assigned to a corresponding element should be modified as a modification proposal node, and determines the proposed modification of the metadata as a modification proposal for the modification proposal node. For example, the determining unit 126 determines, among the multiple nodes, a node for which metadata should be added to a corresponding element as a modification proposal node, and determines the metadata proposed to be added as a modification proposal for the modification proposal node. For example, the determining unit 126 determines, among the multiple nodes, a node for which metadata assigned to a corresponding element should be deleted as a modification proposal node, and determines the metadata proposed to be deleted as a modification proposal for the modification proposal node. For example, the determining unit 126 determines, among the multiple nodes, a node for which the content of the metadata assigned to a corresponding element should be changed as a modification proposal node, and determines the content of the change as a modification proposal for the modification proposal node. The display control unit 114 may display the proposed modification of the metadata as a modification proposal for the modification proposal node, and the modifying unit 120 may modify the metadata assigned to an element corresponding to the modification proposal node in the structured data 400, based on an instruction from the user 102 regarding the modification proposal. This allows the user 102 to optimize the metadata assigned to the element simply by accepting the displayed correction suggestion, thereby improving convenience for the user 102.

[0069] The RAG function unit 130 searches for relevant information related to the input prompt from the structured data 400 generated by the structured data generation unit 122, inputs the search results and the prompt to the LLM 200, and outputs an answer to the prompt based on the output from the LLM 200.

[0070] The incorrect answer cause identifying unit 132 identifies the cause of an incorrect answer when the RAG function unit 130 gives an incorrect answer. For example, the incorrect answer cause identifying unit 132 inputs a prompt included in model QA data stored in the storage unit 110 to the RAG function unit 130, compares the answer output from the RAG function unit 130 with the model answer corresponding to the prompt, and identifies the cause of the incorrect answer when it determines that the answer is incorrect. When the incorrect answer cause identifying unit 132 identifies the cause of the incorrect answer, the determining unit 126 may determine a correction suggestion node and a correction suggestion based on the cause of the incorrect answer.

[0071] Examples of causes of incorrect answers identified by the incorrect answer cause identification unit 132 include, but are not limited to, missing information, transcription errors, scattered information, mixed information, and incomplete metadata. Missing information may be caused by a layout analysis failure that prevents information from being correctly acquired from the document data 300. Dispersed information may be caused by information required to correctly respond to a prompt being split into two or more elements. Mixed information may be caused by information required to correctly respond to a prompt being mixed with other information within a single element. Incomplete metadata may be caused by missing or incorrect metadata required to narrow down the search.

[0072] For example, if the cause of the incorrect answer identified by the incorrect answer cause identification unit 132 is that the information required to correctly answer the prompt is divided into two or more elements, the determination unit 126 determines two or more nodes corresponding to the two or more elements as suggested correction nodes and determines a correction proposal to integrate the two or more nodes. If the suggested correction is approved by the user 102, the correction unit 120 integrates the two or more nodes and makes the elements corresponding to the two or more nodes in the structured data 400 into a single element.

[0073] For example, if the cause of the incorrect answer identified by the incorrect answer cause identification unit 132 is that information for correctly answering the prompt and other information are mixed within one element, the determination unit 126 determines the node corresponding to the element as a correction proposal node and determines a correction proposal indicating multiple nodes after dividing the node. If the correction proposal is approved by the user 102, the correction unit 120 divides the correction proposal node and divides the element in the structured data 400 that corresponds to the correction proposal node.

[0074] For example, if the cause of an incorrect answer identified by the incorrect answer cause identification unit 132 is a transcription error for an area of ​​document data from which an element containing related information is extracted, the determination unit 126 determines a node corresponding to the element as a correction suggestion node and determines a correction suggestion that includes the element and the area of ​​document data 300 from which the element was extracted. The display control unit 114 displays the correction suggestion. The user 102 compares the element included in the correction suggestion with the area of ​​document data 300 and instructs correction as necessary. The correction unit 120 corrects the element in structured data 400 in accordance with the correction instruction from the user 102.

[0075] For example, if the cause of the incorrect answer identified by the incorrect answer cause identification unit 132 is that related information related to the prompt is included in the structured data 400 but could not be found by a search, the determination unit 126 determines a node corresponding to an element including the related information among the multiple elements as a correction suggestion node, and determines a correction suggestion for the metadata assigned to the element as a correction suggestion for the correction suggestion node. If the correction suggestion is approved by the user 102, the correction unit 120 corrects the metadata assigned to the element in the structured data 400.

[0076] For example, if the cause of the incorrect answer identified by the incorrect answer cause identification unit 132 is that related information related to the prompt is included in the structured data 400 but could not be found by a search, the determination unit 126 determines a node corresponding to an element among the multiple elements that includes the related information as a correction suggestion node, and determines a correction suggestion that suggests adding the information included in the prompt to the metadata assigned to the element. If the correction suggestion is approved by the user 102, the correction unit 120 adds the information included in the prompt to the metadata assigned to the element in the structured data 400.

[0077] If the cause of the incorrect answer identified by the incorrect answer cause identification unit 132 is that the information required to correctly answer the prompt is divided into two or more elements, that one element contains a mixture of information required to correctly answer the prompt and other information, or that the transcription is incorrect for an area from which an element containing related information has been extracted, the display control unit 114 may display a correction screen to accept corrections from the user 102.

[0078] As an example, the display control unit 114 superimposes multiple area objects on the document data 300 and displays multiple areas 320 in a selectable manner. When the user 102 selects an area 320, the display control unit 114 displays only the element corresponding to the selected area 320. The user 102 compares the content of the area 320 in the document data 300 with the content of the displayed element and instructs the user 102 to modify the content of the element as necessary. The correction unit 120 modifies the element in the structured data 400 in accordance with the modification instruction from the user 102. When the correction unit 120 receives an instruction from the user 102 to change the range of the area 320 indicated by the area object, the correction unit 120 modifies the range of the area 320 indicated by the area object in accordance with the modification instruction, and the document processing unit 118 recognizes the area 320 within the range indicated by the modified area object and extracts the element.

[0079] As another example, the display control unit 114 displays document data 300 in which multiple area objects are superimposed and multiple elements arranged in a tree structure in a selectable manner. When the user 102 selects an area 320 in the document data 300, the display control unit 114 may highlight an element in the tree structure corresponding to the area 320. When an element in the tree structure is selected, the display control unit 114 may highlight an area 320 in the document data 300 corresponding to the element. The user 102 checks the contents of the document data 300 and the contents of the multiple elements, and, if necessary, instructs correction of the content of the element or correction of the parent-child relationship of the element. The correction unit 120 corrects the element in the structured data 400 in accordance with the instruction from the user 102 to correct the content of the element. The correction unit 120 corrects the relationship between the element and other elements in the structured data 400 in accordance with the instruction from the user 102 to correct the parent-child relationship of the element.

[0080] When the content of the structured data 400 is modified by the modifying unit 120, the graph data generating unit 124 reflects the modification of the structured data 400 in the graph data.

[0081] For example, the incorrect answer cause identification unit 132 inputs a prompt included in the model QA data to the RAG function unit 130, and if the answer output from the RAG function unit 130 is an incorrect answer, it identifies the cause of the incorrect answer using the prompt input to the RAG function unit 130, the answer output by the RAG function unit 130, the model answer to the prompt, and an element containing related information searched for by the RAG function unit 130 from the structured data 400. As a specific example, the incorrect answer cause identification unit 132 uses an incorrect answer cause identification AI that receives as input the prompt, the answer by the RAG function unit 130, the model answer to the prompt, and an element containing related information searched for by the RAG function unit 130 from the structured data 400, and outputs the cause of the incorrect answer. The incorrect answer cause identification unit 132 may generate the incorrect answer cause identification AI by supervised learning. That is, the incorrect answer cause identification unit 132 generates an incorrect answer cause identification AI by performing machine learning using multiple data sets including a prompt, an answer by the RAG functional unit 130, a model answer to the prompt, an element containing related information searched for by the RAG functional unit 130 from the structured data 400, and the cause of the incorrect answer. The incorrect answer cause identification unit 132 inputs a prompt to the RAG functional unit 130, and when the answer output from the RAG functional unit 130 is an incorrect answer, it inputs the prompt input to the RAG functional unit 130, the answer output by the RAG functional unit 130, the model answer to the prompt, and the element containing related information searched for by the RAG functional unit 130 from the structured data 400 to the incorrect answer cause identification AI, thereby identifying the cause of the incorrect answer.

[0082] Instead of model QA data, the incorrect answer cause identifying unit 132 may use model element data in which multiple data sets each including a prompt and an element (sometimes referred to as a model element) included in the structured data 400 that should be referenced to provide a correct answer to the prompt are registered. The model element data may be registered by the registration unit 112 and stored in the storage unit 110 after the structured data 400 is generated. For example, the incorrect answer cause identifying unit 132 inputs a prompt included in the model element data to the RAG function unit 130, and if the answer output from the RAG function unit 130 is an incorrect answer, identifies the cause of the incorrect answer using the prompt input to the RAG function unit 130, the answer output by the RAG function unit 130, the model element for the prompt, and an element containing related information searched for by the RAG function unit 130 from the structured data 400. As a specific example, the incorrect answer cause identification unit 132 uses an incorrect answer cause identification AI that receives as input a prompt, an answer by the RAG function unit 130, a model element for the prompt, and an element containing related information searched for by the RAG function unit 130 from the structured data 400, and outputs the cause of an incorrect answer. The incorrect answer cause identification unit 132 may generate the incorrect answer cause identification AI by supervised learning. That is, the incorrect answer cause identification unit 132 generates the incorrect answer cause identification AI by performing machine learning using multiple data sets that include the prompt, the answer by the RAG function unit 130, a model element for the prompt, an element containing related information searched for by the RAG function unit 130 from the structured data 400, and the cause of an incorrect answer. The incorrect answer cause identification unit 132 inputs a prompt to the RAG function unit 130, and if the answer output from the RAG function unit 130 is incorrect, identifies the cause of the incorrect answer by inputting the prompt input to the RAG function unit 130, the answer output by the RAG function unit 130, the model element for the prompt, and the element containing related information searched for by the RAG function unit 130 from the structured data 400 into the incorrect answer cause identification AI.

[0083] For example, the incorrect answer cause identification unit 132 may input a prompt to the RAG function unit 130, and when the answer output from the RAG function unit 130 is an incorrect answer, first identify the incorrect answer pattern and then identify the cause of the incorrect answer based on the incorrect answer pattern.

[0084] Examples of error patterns identified by the error cause identification unit 132 include, but are not limited to, search failure, semantic inconsistency, misinterpretation of context, failure to integrate information, failure to divide information, and hallucination.

[0085] As a specific example, the incorrect answer cause identification unit 132 identifies the incorrect answer pattern as a search failure when the answer output from the RAG function unit 130 is an answer indicating that the information could not be found, such as "No such information exists."

[0086] As a specific example, the incorrect answer cause identification unit 132 identifies an incorrect answer pattern as a mismatch in meaning when it is not possible to identify information to answer the question due to differences in expression between the question and the structured data 400, such as when the question "What are the camera capabilities of the latest smartphones?" cannot be answered because the structured data 400 uses the expression "imaging functions of mobile devices."

[0087] As a specific example, the incorrect answer cause identification unit 132 identifies the incorrect answer pattern as a misinterpretation of the context when the information required to answer the question is obtained but the interpretation fails, such as when related information about product A is obtained in response to the question, "What is the release date of product A?", but the release date of another product is answered.

[0088] As a specific example, the incorrect answer cause identification unit 132 identifies the incorrect answer pattern as a failure to integrate information when a comprehensive answer cannot be given due to the information being divided, such as when the participant list is divided into multiple parts and it is not possible to list everyone in response to the question, "Who are the participants in project X?"

[0089] As a specific example, the incorrect answer cause identifying unit 132 identifies the incorrect answer pattern as a failure to divide information when an incorrect answer is given due to an inappropriate mixture of information, such as when a list of participants in Project X and a list of participants in other projects are mixed in a certain element in response to the question "Who are the participants in Project X?", and participants other than those in Project X are also included in the answer.

[0090] As a specific example, the incorrect answer cause identification unit 132 identifies the incorrect answer pattern as hallucination when the information necessary to answer the question is obtained but a false answer is given, such as when the relevant sales information is obtained but a fictitious sales amount is answered in response to the question, ``What were Company Y's sales last year?''

[0091] For example, if the error pattern is a search failure, the error cause identifying unit 132 may identify the cause of the error as a missing information. For example, if the error pattern is a semantic inconsistency, the error cause identifying unit 132 may identify the cause of the error as a metadata defect. For example, if the error pattern is a context misinterpretation, the error cause identifying unit 132 may identify the cause of the error as a mixture of information. For example, if the error pattern is a failure to integrate information, the error cause identifying unit 132 may identify the cause of the error as a dispersion of information. For example, if the error pattern is a failure to divide information, the error cause identifying unit 132 may identify the cause of the error as a mixture of information. For example, if the error pattern is hallucination, the error cause identifying unit 132 may identify the cause of the error as a dispersion of information or a mixture of information.

[0092] Before the RAG function unit 130 starts operating the RAG function, the information processing device 100 may modify the structured data 400 using model QA data and model element data prepared by the user 102. Then, after modifying the structured data 400, the information processing device 100 may start operating the RAG function by the RAG function unit 130.

[0093] After the information processing device 100 starts operation, if the RAG function unit 130 incorrectly answers a prompt input from the communication terminal 20, the information processing device 100 may acquire a correct answer from the user 22 or 102 and correct the structured data 400 using the correct answer. For example, the incorrect answer cause identification unit 132 identifies the cause of the incorrect answer using the prompt input to the RAG function unit 130, the answer output by the RAG function unit 130, the correct answer to the prompt, and an element containing related information searched for by the RAG function unit 130 from the structured data 400. The incorrect answer cause identification unit 132 may identify the cause of the incorrect answer using the correct answer in a manner similar to that when using a model answer. The determination unit 126 may determine a suggested correction node and a suggested correction based on the cause of the incorrect answer determined by the incorrect answer cause identification unit 132. The display control unit 114 may display the graph data 500 and the suggested correction for the suggested correction node determined by the determination unit 126. The correction unit 120 may correct the part of the structured data 400 that corresponds to the correction suggestion node, based on an instruction from the user 102 in response to the correction suggestion.

[0094] After the information processing device 100 starts operation, if the RAG function unit 130 incorrectly answers a prompt input from the communication terminal 20, the information processing device 100 may acquire from the user 102 elements to be referenced in order to correctly answer the prompt, and may correct the structured data 400 using the elements. For example, the incorrect answer cause identification unit 132 may identify the cause of the incorrect answer using the prompt input to the RAG function unit 130, the answer output by the RAG function unit 130, the elements to be referenced in response to the prompt, and elements containing related information searched for by the RAG function unit 130 from the structured data 400. The incorrect answer cause identification unit 132 may identify the cause of the incorrect answer using the acquired elements in a manner similar to that when using model elements. The determination unit 126 may determine a suggested correction node and a suggested correction based on the cause of the incorrect answer determined by the incorrect answer cause identification unit 132. The display control unit 114 may display the graph data 500 and the suggested correction for the suggested correction node determined by the determination unit 126. The correction unit 120 may correct the part of the structured data 400 that corresponds to the correction suggestion node, based on an instruction from the user 102 in response to the correction suggestion.

[0095] These make it possible for the RAG function unit 130 to correct the structured data 400 so that, if it is unable to correctly respond to an input prompt after starting operation of the RAG function using the structured data 400, it is possible to correctly respond to the prompt. By continuously making such corrections, it is possible to contribute to the construction of structured data 400 that can output correct answers to all prompts.

[0096] When the information processing device 100 corrects the structured data 400, the information processing device 100 may accumulate knowledge of the correction and improve its function so that the structured data 400 generated by the structured data generation unit 122 does not need to be corrected in the first place. For example, when the correction unit 120 makes a correction, the correction content is stored in the storage unit 110. Then, the model update unit 136 updates the AI ​​used by the document processing unit 118 based on the multiple correction content stored in the storage unit 110. For example, the model update unit 136 updates the document recognition AI based on the multiple correction content stored in the storage unit 110.

[0097] For example, when a character included in an element is corrected by the correction unit 120 based on instructions from the user 102, the model update unit 136 updates the document recognition AI so that the recognition result by the document recognition AI for the character before correction becomes the corrected character.

[0098] For example, when the recognition result of a diagram included in document data 300 is corrected by correction unit 120 based on an instruction from user 102, model update unit 136 updates the document recognition AI so that the recognition result by the document recognition AI for the same diagram becomes the corrected recognition result. As a specific example, when at least one of the graph type (line graph, bar graph, pie chart, band graph, etc.), graph color coding, scale unit, and scale interval of a graph included in document data 300 is corrected by correction unit 120, model update unit 136 updates the document recognition AI so that these are in the same state in the recognition result of the graph by document recognition AI.

[0099] For example, when the range of an area indicated by an area object is corrected by the correction unit 120 based on instructions from the user 102, the model update unit 136 updates the document recognition AI so that the layout analysis results by the document recognition AI for similar areas are the same as those for the corrected area.

[0100] For example, when the parent-child relationships of multiple elements displayed in a tree structure are corrected by the correction unit 120 based on instructions from the user 102, the model update unit 136 updates the document recognition AI so that the recognition results by the document recognition AI for similar parent-child relationships are the same as the corrected parent-child relationships.

[0101] For example, when the connection relationship of nodes 502 in graph data 500 corresponding to structured data 400 of certain document data 300 is corrected by the correction unit 120 based on a user's instruction or in accordance with the user's approval of a correction proposal, the model update unit 136 updates the document recognition AI so that the recognition result of the parent-child relationship by the document recognition AI for similar document data 300 is the same as the corrected parent-child relationship.

[0102] For example, when a node 502 in graph data 500 corresponding to structured data 400 of certain document data 300 is split by the correction unit 120 based on a user instruction or in accordance with the user's approval of a correction proposal, the model update unit 136 updates the document recognition AI so that, for similar document data 300, in the recognition results by the document recognition AI, the element corresponding to the node 502 becomes the same as the multiple elements corresponding to the split node 502.

[0103] For example, when multiple nodes 502 in graph data 500 corresponding to structured data 400 of certain document data 300 are integrated by the correction unit 120 based on a user's instruction or in accordance with the user's approval of a correction proposal, the model update unit 136 updates the document recognition AI so that, for similar document data 300, the multiple elements corresponding to the multiple nodes 502 become a single element in the recognition results by the document recognition AI.

[0104] For example, when metadata assigned to an element of structured data 400 of certain document data 300 is corrected by the correction unit 120 based on a user's instruction or in accordance with the user's approval of a correction proposal, the model update unit 136 updates the document recognition AI so that the metadata assigned to the element by the document recognition AI for similar document data 300 becomes the corrected metadata.

[0105] As a result, the structured data 400 generated by the structured data generating unit 122 can be structured data 400 that does not need to be modified later, which can contribute to improving the RAG function of the information processing device 100.

[0106] 5 shows a schematic display example of graph data 500 generated by graph data generation unit 124. Graph data 500 includes a plurality of nodes 502 corresponding to a plurality of elements included in structured data 400, and edges 504 connecting related nodes 502. A character string representing the corresponding element is placed in each of the plurality of nodes 502.

[0107] 5, the display control unit 114 displays graph data 500 and a sidebar 510. The display control unit 114 arranges, in the sidebar 510, a list of a plurality of elements corresponding to a plurality of nodes 502 included in the graph data 500.

[0108] In response to a user 102 selecting an element in the sidebar 510, the display control unit 114 displays information about the element and highlights the node 502 corresponding to the element and the nodes 502 related to the node 502 in the graph data 500. The element information includes metadata corresponding to the element, the reliability of the recognition result of the element, and a menu for displaying the document data 300 corresponding to the element. FIG. 5 illustrates a state in which the element "GGGG" is selected.

[0109] In response to a node 502 in the graph data 500 being selected by the user 102, the display control unit 114 may highlight the node 502 and nodes 502 related to the selected node 502, and may display information related to the element in the sidebar 510 that corresponds to the selected node 502.

[0110] 5 to the user 102, the information processing device 100 can allow the user 102 to easily understand what elements have been extracted and how the elements have been structured from one or more pieces of document data 300 that the user 102 has prepared. In addition, the user 102 can easily understand, for each element, elements determined to be related, assigned metadata, the reliability of the recognition result, and the like.

[0111] 6 schematically illustrates a display example of a correction suggestion menu 600. When one correction suggestion has been determined by the determination unit 126, the correction suggestion menu 600 may include the single correction suggestion. When multiple correction suggestions have been determined by the determination unit 126, the correction suggestion menu 600 may include the multiple correction suggestions and a collective approval menu 610 for collectively approving the multiple correction suggestions. Here, an example is shown in which the determination unit 126 has determined three correction suggestions: an element correction suggestion, a transcription correction suggestion, and a metadata correction suggestion. The display control unit 114 displays the correction suggestion menu 600 including an element correction menu 612 corresponding to the element correction suggestion, a transcription correction menu 614 corresponding to the transcription correction suggestion, and a metadata correction menu 616 corresponding to the metadata correction suggestion.

[0112] In addition to the correction suggestion menu 600, the display control unit 114 may arrange and display objects indicating that the correction suggestion node is a correction suggestion node and the type of correction suggestion for each correction suggestion node in the graph data 500. In the example shown in Fig. 6, an element correction object 522 is arranged in a correction suggestion node corresponding to an element correction, a transcription correction object 524 is arranged in a correction suggestion node corresponding to a transcription correction, and a metadata correction object 526 is arranged in a correction suggestion node corresponding to a metadata correction. This allows the user 102 to easily understand which of the multiple nodes 502 is a correction suggestion node and the type of correction suggestion.

[0113] When the batch approval menu 610 is selected by the user 102, the correction unit 120 may execute corrections in accordance with the correction suggestions. This reduces the time and effort required for the approval work by the user 102, and improves the convenience for the user 102.

[0114] When the user 102 selects one of the menus from which each of a plurality of correction suggestions can be selected, the display control unit 114 may execute display control according to the selected menu. When the element correction menu 612 is selected, the display control unit 114 may display element correction suggestions 620. When the transcription correction menu 614 is selected, the display control unit 114 may display transcription correction suggestions 630. When the metadata correction menu 616 is selected, the display control unit 114 may display metadata correction suggestions 660.

[0115] 7 schematically illustrates an example of the display of an element modification proposal 620. Here, the element modification proposal 620 is illustrated when the determination unit 126 determines that the node 502 containing "HHHH" is the modification proposal node and that the node 502 containing "AAAAAA" is the node to be joined to the modification proposal node. In the example illustrated in FIG. 7, the element modification proposal 620 includes the input prompt and the answer output by the RAG functional unit 130 when the RAG functional unit 130 gives an incorrect answer, the node 502 to be joined to the modification proposal node, content proposing to join these nodes 502, an approve button 622, and a reject button 624.

[0116] The user 102 views the element modification proposal 620, and if the user 102 decides that the modification is acceptable, the user selects the approve button 622. If the user rejects the modification, the user selects the reject button 624. When the approve button 622 is selected, the modification unit 120 merges the proposed modification node with the node 502 that should be joined to the proposed modification node, as proposed, and combines the two elements into a single element in the structured data 400. When the reject button 624 is selected, the display control unit 114 ends the display of the element modification proposal 620.

[0117] This allows the user 102 to accomplish the same thing by simply selecting the approval button 622, thereby reducing the burden of modifying the structured data 400, compared to when the user 102 discovers multiple nodes 502 to be integrated, identifies multiple elements in the structured data 400 that correspond to those multiple nodes 502, and modifies the multiple elements to combine them into a single element.

[0118] FIG. 8 schematically illustrates a display example of the transcription correction proposal 630. Here, the transcription correction proposal 630 is illustrated when the determination unit 126 determines that the node 502 containing "EEEE" is the correction proposal node whose corresponding element content should be corrected. In the example illustrated in FIG. 8, the transcription correction proposal 630 includes the input prompt and the answer output by the RAG function unit 130 when the RAG function unit 130 gives an incorrect answer, document data 300 from which the element corresponding to the correction proposal node has been extracted, and a layout correction screen transition button 632. When the layout correction screen transition button 632 is selected, the display control unit 114 displays a layout correction screen.

[0119] Fig. 9 schematically shows an example of a display of a layout modification screen 640. The layout modification screen 640 includes document data 300 and a plurality of area objects 642 that indicate the ranges of a plurality of areas 320 included in the document data 300. In Fig. 9, an object drawn with a dotted line surrounding an area is shown as an example of the area object 642, but the form of the area object 642 is not limited to this.

[0120] The display control unit 114 highlights the area object 642 of the selected area 320 out of the multiple areas 320 in the document data 300, and displays an edit menu 644 that displays an editable version of the element corresponding to the selected area 320, i.e., the recognition result of the content of the selected area 320. When the content of an element is edited by the user 102 in the edit menu 644, the correction unit 120 edits the content of the element in the structured data 400 in accordance with the edited content.

[0121] 10 is a schematic diagram illustrating a display example of a layout correction screen 640. As illustrated in FIG. 10, the display control unit 114 may display the document data 300 and the plurality of region objects 642 in a manner that reflects the reliability of the recognition results of each of the plurality of regions 320 by the document processing unit 118. In the example illustrated in FIG. 10, "Reliability: Low" is superimposed only on the regions 320 among the plurality of regions 320 for which the reliability of the recognition result is lower than a predetermined threshold. By the display control unit 114 performing the display as illustrated in FIG. 10, the user 102 can identify the regions 320 among the plurality of regions 320 that need to be checked in more detail, which can contribute to improving the efficiency of the user 102's checking and preventing oversights.

[0122] FIG. 11 schematically illustrates an example of a modification on the layout modification screen 640. Here, an example is shown in which the user 102 has received an instruction to resize a region object 642 in one of the regions 320 in the document data 300. The modification unit 120 modifies the region object 642 in accordance with the instruction. The document processing unit 118 recognizes the content of the region 320 indicated by the region object 642 whose range has been changed by the modification unit 120, and extracts the element. In the example shown in FIG. 11, an element that originally contained four lines of text is changed to an element that contains five lines of text. This allows the user 102 to easily modify a portion that should be recognized as a region 320 but is not included within that region 320.

[0123] 12 schematically shows an example of a correction on the layout correction screen 640. Here, an example is shown in which the user 102 has specified an area in the document data 300 on which an area object 642 is superimposed. The document processing unit 118 recognizes the content of the area 320 and extracts the element, and the correction unit 120 includes the element in the structured data 400. As a result, if there is an area in the document data 300 that was not automatically recognized as an area but that the user 102 wants to include in the structured data 400, it is possible to include information about that area in the structured data 400.

[0124] 13 schematically shows an example of a display of a layout modification screen 650. The layout modification screen 650 includes document data 300, a plurality of area objects 642 indicating the ranges of a plurality of areas 320 included in the document data 300, and a tree display 652 showing a plurality of elements of the plurality of areas 320 in a tree structure according to the parent-child relationships between the plurality of elements.

[0125] The contents and parent-child relationships of multiple elements in tree view 652 are editable. Correction unit 120 corrects the contents of an element in structured data 400 in accordance with an instruction from user 102 to correct the content of that element. Correction unit 120 corrects the relationships between elements in structured data 400 in accordance with an instruction from user 102 to correct the parent-child relationships of the elements.

[0126] 14 schematically shows an example of a display of a metadata revision proposal 660. Here, the metadata revision proposal 660 is shown in the case where the determining unit 126 determines that the node 502 containing "FFFF" is the revision proposal node for revising the metadata assigned to the element corresponding to the node 502. In the example shown in FIG. 14, the metadata revision proposal 660 includes the input prompt and the answer output by the RAG functional unit 130 when the RAG functional unit 130 gives an incorrect answer, a proposal to correct the metadata assigned to the element corresponding to the revision proposal node, an approve button 662, and a reject button 664.

[0127] The user 102 views the metadata modification proposal 660, and if they decide that the modification is acceptable, they select the approve button 662. If they reject the modification, they select the reject button 664. If the approve button 662 is selected, the modification unit 120 modifies the metadata of the element corresponding to the modification proposal node as proposed. If the reject button 624 is selected, the display control unit 114 ends the display of the metadata modification proposal 660.

[0128] 15 schematically illustrates an example of processing by the recognition result correspondence unit 121. Here, a case will be described in which the recognition result correspondence unit 121 uses the correction history data 700 to generate correction suggestions for recognition results with a reliability lower than a predetermined threshold, among the recognition results of a plurality of regions 320 by the document processing unit 118, and to identify the cause of the low reliability.

[0129] The correction history data 700 includes a plurality of correction histories. In this example, the correction history data 700 includes, for each of the plurality of correction histories, a before-correction 702 and an after-correction 704, as well as a category 706, a cause 708, and a correction policy 710.

[0130] The recognition result handling unit 121 acquires the document data 300, the recognition results 350 of the multiple regions 320 of the document data 300 obtained by the document processing unit 118, region information 360 indicating one or more regions 320 among the multiple regions 320 whose reliability is lower than a threshold, and correction history data 700, and inputs these, along with an instruction to analyze the cause of the low reliability of the recognition results of the one or more regions 320 and output the analysis results, and an instruction to create a correction proposal for the recognition results of the one or more regions 320, to the correction suggestion capable model, and acquires the cause and correction proposal output from the correction suggestion capable model. The display control unit 114 may display the cause and correction proposal. The correction unit 120 may correct the recognition result in accordance with the correction proposal in response to the user 102's approval of the correction proposal. If the user 102 has revised and approved the correction proposal, the correction unit 120 may correct the recognition result in accordance with the correction proposal revised by the user 102. When the correction unit 120 receives a correction instruction from the user 102 for the recognition result to be corrected in the correction proposal, the correction unit 120 may correct the recognition result in accordance with the correction instruction from the user 102.

[0131] When the correction unit 120 corrects the recognition result in accordance with a correction suggestion in response to the user 102 approving the correction suggestion, when the correction unit 120 corrects the recognition result in accordance with a correction suggestion amended by the user 102, or when the correction unit 120 corrects the recognition result in accordance with a correction instruction from the user 102, the recognition result response unit 121 may add the correction history to the correction history data 700.

[0132] 16 schematically illustrates an example of the hardware configuration of a computer 1200 functioning as the information processing device 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of an apparatus according to the present embodiment, or can cause the computer 1200 to execute operations associated with the apparatus according to the present embodiment or one or more "units," and / or can cause the computer 1200 to execute a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0133] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0134] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0135] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0136] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0137] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0138] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0139] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0140] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0141] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0142] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0143] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0144] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0145] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or another programmable data processing device, or a programmable circuit, either locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, so that the processor of the programmable data processing device, such as a computer, or the programmable circuit executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computers. In a distributed computing system, multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0146] Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0147] By using the invention according to this embodiment, it is possible to contribute, for example, to optimizing the structured data used in RAG, thereby contributing to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0148] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0149] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0150] 20 communication terminal, 22 user, 80 network, 100 information processing device, 102 user, 110 memory unit, 112 registration unit, 114 display control unit, 116 document data acquisition unit, 118 document processing unit, 120 correction unit, 121 recognition result correspondence unit, 122 structured data generation unit, 124 graph data generation unit, 126 determination unit, 130 RAG function unit, 132 error cause identification unit, 136 model update unit, 200 LLM, 300 document data, 310 page, 320 region, 350 recognition result, 360 region information, 400 structured data, 500 graph data, 502 node, 504 edge, 510 sidebar, 522 element correction object, 524 transcription correction object, 526 metadata correction object, 600 correction suggestion menu, 610 Bulk approval menu, 612 Element correction menu, 614 Transcription correction menu, 616 Metadata correction menu, 620 Element correction proposal, 622 Approval button, 624 Reject button, 630 Transcription correction proposal, 632 Layout correction screen transition button, 640 Layout correction screen, 642 Area object, 644 Edit menu, 650 Layout correction screen, 652 Tree view, 660 Metadata correction proposal, 662 Approval button, 664 Reject button, 700 Correction history data, 702 Before correction, 704 After correction, 706 Category, 708 Cause, 710 Correction policy, 1200 Computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 Graphics controller, 1218 Display device, 1220 Input / output controller, 1222 Communication interface, 1224 Storage device, 1230 ROM, 1240 I / O chip

Claims

1. a document data acquisition unit that acquires document data; a document processing unit that performs a layout analysis of the document data to identify a plurality of areas included in the document data, recognizes the contents of the plurality of areas, and extracts a plurality of elements; a structured data generation unit that generates structured data by structuring the plurality of elements; a graph data generation unit that generates graph data in which the plurality of elements are a plurality of nodes based on the relationships between the plurality of elements included in the structured data; a determination unit that determines a proposed modification node from among the plurality of nodes included in the graph data and a proposed modification for the proposed modification node; a display control unit that controls display of the graph data and the correction suggestion for the correction suggestion node; a correction unit that corrects a portion of the structured data corresponding to the correction suggestion node based on a user's instruction for the correction suggestion; Equipped with the structured data generation unit generates the structured data by adding metadata related to the content of each of the plurality of elements to the structured data; the determination unit determines, among the plurality of nodes, a node for which the metadata assigned to a corresponding element should be modified as the modification-proposed node, and determines a modification proposal for the metadata as a modification proposal for the modification-proposed node; the display control unit displays the metadata correction suggestion as a correction suggestion for the correction suggestion node; the correction unit corrects the metadata assigned to an element of the structured data corresponding to the correction suggestion node based on an instruction from the user with respect to the correction suggestion; Information processing device.

2. A document data acquisition unit that acquires document data; a document processing unit that performs a layout analysis of the document data to identify a plurality of areas included in the document data, recognizes the contents of the plurality of areas, and extracts a plurality of elements; a structured data generation unit that generates structured data by structuring the plurality of elements; a graph data generation unit that generates graph data in which the plurality of elements are a plurality of nodes based on the relationships between the plurality of elements included in the structured data; a RAG function unit that searches the structured data generated by the structured data generation unit for related information related to an input prompt, inputs the search results and the prompt into a large-scale language model, and outputs an answer to the prompt based on an output from the large-scale language model; an incorrect answer cause identification unit that inputs a prompt associated with a model answer to the RAG function unit, compares the answer output from the RAG function unit with the model answer, and, if it is determined that the answer is incorrect, identifies the cause of the incorrect answer; a determination unit that determines a proposed correction node from among the plurality of nodes included in the graph data and a proposed correction for the proposed correction node based on the cause of the incorrect answer identified by the incorrect answer cause identification unit; a display control unit that controls display of the graph data and the correction suggestion for the correction suggestion node; a correction unit that corrects a portion of the structured data corresponding to the correction suggestion node based on a user's instruction for the correction suggestion; An information processing device comprising:

3. 3. The information processing device according to claim 2, wherein the determination unit determines, as the proposed modification node, at least one of a node to be connected to another node, a node to be integrated with another node, and a node to be split, and for the node to be connected to the other node, determines a modification proposal indicating the node to be connected, for the node to be integrated with the other node, determines a modification proposal indicating the node to be integrated, and for the node to be split, determines a modification proposal indicating multiple nodes after split.

4. the determination unit determines, as the modification-proposed node, a node among the plurality of nodes whose corresponding element content should be modified; the display control unit displays, as the revision suggestion, an area of ​​the document data from which an element corresponding to the revision suggestion node is extracted; 3. The information processing device according to claim 2, wherein, when a correction instruction is received from the user after the display control unit has displayed the area of ​​the document data, the correction unit corrects a portion of the structured data corresponding to the suggested correction node in accordance with the correction instruction.

5. 3. The information processing device according to claim 2, wherein when the cause of the incorrect answer identified by the incorrect answer cause identification unit is that the information required to correctly answer the prompt is divided into two or more elements, the determination unit determines two or more nodes corresponding to the two or more elements as the correction suggestion nodes and determines a correction suggestion that combines the two or more nodes.

6. 3. The information processing device of claim 2, wherein when the cause of the incorrect answer identified by the incorrect answer cause identification unit is a mixture of information for correctly answering the prompt and other information within a single element, the determination unit determines a node corresponding to the element as the correction suggestion node and determines a correction suggestion indicating multiple nodes after dividing the node.

7. 3. The information processing device of claim 2, wherein when the cause of the incorrect answer identified by the incorrect answer cause identification unit is a transcription error for an area of ​​the document data from which an element containing the related information is extracted, the determination unit determines a node corresponding to the element as the correction suggestion node and determines a correction suggestion that includes the element and the area of ​​the document data from which the element is extracted.

8. 3. The information processing device of claim 2, wherein when the cause of the incorrect answer identified by the incorrect answer cause identification unit is that related information related to the prompt is included in the structured data but could not be found by a search, the determination unit determines a node among the multiple elements corresponding to an element that includes the related information as a correction suggestion node, and determines a correction suggestion for metadata attached to the element as a correction suggestion for the correction suggestion node.

9. 9. The information processing device of claim 8, wherein when the cause of the incorrect answer identified by the incorrect answer cause identification unit is that related information related to the prompt is included in the structured data but could not be found by a search, the determination unit determines a node among the multiple elements corresponding to an element that includes the related information as a correction suggestion node, and determines a correction suggestion that suggests adding the information included in the prompt to metadata assigned to the element.

10. 1. A computer-implemented information processing method, comprising: a document data acquisition stage for acquiring document data; a document processing step of identifying a plurality of regions included in the document data by performing a layout analysis of the document data, and extracting a plurality of elements by recognizing the contents of the plurality of regions; a structured data generation step of generating structured data by structuring the plurality of elements; a graph data generation step of generating graph data in which the plurality of elements are a plurality of nodes based on the relationships between the plurality of elements included in the structured data; a determination step of determining a proposed modification node from among the plurality of nodes included in the graph data and a proposed modification for the proposed modification node; a display control step of controlling the display of the graph data and the modification suggestions for the modification suggestion nodes; a correction step of correcting a portion of the structured data corresponding to the correction suggestion node based on a user's instruction for the correction suggestion; Equipped with the structured data generating step generates the structured data by adding metadata related to the contents of the elements to each of the plurality of elements; The determining step determines, among the plurality of nodes, a node for which the metadata assigned to a corresponding element should be modified as the modification proposal node, and determines a modification proposal for the metadata as a modification proposal for the modification proposal node; The display control step displays the metadata revision suggestions as revision suggestions for the revision suggestion node, the modification step modifies the metadata assigned to an element of the structured data corresponding to the modification suggestion node based on an instruction by the user regarding the modification suggestion; Information processing methods.

11. An information processing method implemented by a computer, comprising: a document data acquisition stage for acquiring document data; a document processing step of identifying a plurality of regions included in the document data by performing a layout analysis of the document data, and extracting a plurality of elements by recognizing the contents of the plurality of regions; a structured data generation step of generating structured data by structuring the plurality of elements; a graph data generation step of generating graph data in which the plurality of elements are a plurality of nodes based on the relationships between the plurality of elements included in the structured data; an incorrect answer cause identification step of searching the structured data generated in the structured data generation step for related information related to the input prompt, inputting the search results and the prompt into a large-scale language model, inputting a prompt associated with a model answer into a RAG function unit that outputs an answer to the prompt based on an output from the large-scale language model, comparing the answer output from the RAG function unit with the model answer, and identifying the cause of the incorrect answer if it is determined to be an incorrect answer; a determination step of determining a proposed correction node from among the plurality of nodes included in the graph data and a proposed correction for the proposed correction node based on the cause of the incorrect answer identified in the cause identification step; a display control step of controlling the display of the graph data and the modification suggestions for the modification suggestion nodes; a correction step of correcting a portion of the structured data corresponding to the correction suggestion node based on a user's instruction for the correction suggestion; An information processing method comprising:

12. A program for causing a computer to execute the information processing method according to claim 10 or 11.

Images