Information processing apparatus, information processing method, program, and recording medium

Abstract

The present application provides an information processing device, having: a unit that obtains a first document; a parsing unit that extracts content to be confirmed from the first document to generate a task for each of the content; a search unit that searches a second document for a condition related to the content to be confirmed in the task; a processing unit that sequentially inputs the task generated by the parsing unit to a learned model to cause the learned model to answer whether the content to be confirmed in the input task meets the condition searched from the second document; and an output unit that outputs the processing result of the processing unit, the parsing unit generates the task for each of the content by inputting the first document and a prompt including an instruction for extracting the content to be confirmed to generate the task and a learning sample to be provided to the learned model to the learned model.

Description

Technical Field

[0001] This invention relates to information processing apparatus, information processing method, program, and recording medium. Background Technology

[0002] In the past, document comparisons were done manually, through visual inspection. However, in recent years, with the large volume of electronic documents processed, computers have been used for efficient work. But, when reviewing a large number of documents, human error is more likely. Furthermore, in comparing electronic documents using computers, documents with standardized formats are easier to compare than those with inconsistent formats. For example, it is unsuitable for tasks such as verifying whether specifications or other documents with potentially inconsistent formats conform to standards.

[0003] Patent document 1 proposes the following technology: For each benchmark content of the standard benchmark, calculate the identity score by calculating the semantic similarity of multiple provisions of the organization's internal procedures, associate the provision with the highest identity score, and calculate the consistency score between the standard benchmark and the organization's internal procedures by averaging the identity scores of each benchmark content and its associated provision. The technology then displays each benchmark content, the provision associated with the benchmark content, and the consistency score to the user.

[0004] Patent document 2 proposes the following technique: decompose a query document containing multiple data types into elements of different data types, perform a similarity search on the elements of the decomposed data types, and find documents similar to the query document.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2020-86737

[0008] Patent Document 2: Japanese Patent Publication No. 2002-537604 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] The present invention aims to provide a structure that allows for determining whether the content of an object document meets required conditions using an information processing device such as a computer, without limiting the format. The objective of this invention is to provide an information processing device, information processing method, program, and recording medium that can efficiently perform the verification process of whether the content of an object document meets required conditions.

[0011] Methods for solving problems

[0012] The information processing apparatus of the present invention is characterized by comprising: an acquisition unit that acquires a first document; a parsing unit that extracts content to be confirmed from the first document and generates a task for each of the contents; a search unit that searches a second document for conditions related to the content to be confirmed in the task; a processing unit that sequentially inputs the tasks generated by the parsing unit into a learned model, causing the learned model to answer whether the content to be confirmed in the input task meets the conditions searched from the second document; and an output unit that outputs the processing result of the processing unit, wherein the parsing unit generates the task for each of the contents by inputting the first document and prompts including instructions for extracting the content to be confirmed to generate the task and learning samples to be provided to the learned model into the learned model.

[0013] Invention Effects

[0014] According to the present invention, an information processing apparatus, information processing method, program, and recording medium can be provided to efficiently perform the verification operation of whether the content of an object document meets the required conditions. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating an example of the hardware structure of an information processing device.

[0016] Figure 2 This is a diagram illustrating an example of the functional structure of the information processing apparatus according to the first embodiment.

[0017] Figure 3 This is a diagram illustrating a processing example of the information processing apparatus according to the first embodiment.

[0018] Figure 4 This is a diagram illustrating an example of the segmentation of a specification sheet.

[0019] Figure 5A This is a diagram illustrating an example of task decomposition.

[0020] Figure 5B This is a diagram illustrating an example of task decomposition.

[0021] Figure 5C This is a diagram illustrating an example of task decomposition.

[0022] Figure 6 This is a diagram illustrating an example of prompts in task breakdown.

[0023] Figure 7A This is a diagram illustrating an example of task decomposition.

[0024] Figure 7B This is a diagram illustrating an example of task decomposition.

[0025] Figure 7C This is a diagram illustrating an example of task decomposition.

[0026] Figure 7D This is a diagram illustrating an example of task decomposition.

[0027] Figure 8 This is a diagram illustrating an example of prompts in task breakdown.

[0028] Figure 9 This is a diagram illustrating an example of prompts in the extraction of prerequisite information.

[0029] Figure 10 This is a diagram illustrating an example of a standard document database.

[0030] Figure 11 This is a diagram illustrating an example of using a search tool to perform a search.

[0031] Figure 12 This is a diagram illustrating an example of a control treatment.

[0032] Figure 13 This is a diagram illustrating an example of a control treatment.

[0033] Figure 14 This is a diagram illustrating an example of a control treatment.

[0034] Figure 15 This is a diagram illustrating an example of a control treatment.

[0035] Figure 16A This is a diagram illustrating an example of a control treatment.

[0036] Figure 16B This is a diagram illustrating an example of a control treatment.

[0037] Figure 17A This is a diagram illustrating an example of a control treatment.

[0038] Figure 17B This is a diagram illustrating an example of a control treatment.

[0039] Figure 18 This is a diagram illustrating an example of a control treatment.

[0040] Figure 19 This is a diagram illustrating an example of a control treatment.

[0041] Figure 20A This is a diagram showing an example of the output of the processing result.

[0042] Figure 20B This is a diagram showing an example of the output of the processing result.

[0043] Figure 21 This is a diagram illustrating an example of the functional structure of the information processing apparatus according to the second embodiment.

[0044] Figure 22 This is a diagram illustrating a processing example of the information processing apparatus according to the second embodiment.

[0045] Figure 23A This is an example of a prompt used to illustrate attribute information extraction.

[0046] Figure 23B This is a diagram illustrating an example of attribute information extraction.

[0047] Figure 24A This is a diagram illustrating an example of hints in the construction of a rule index.

[0048] Figure 24B This is a diagram illustrating an example of rule-based index construction.

[0049] Figure 25 This is a diagram illustrating the processing in the second embodiment.

[0050] Figure 26A This is a diagram illustrating an example of a sample related to a control treatment.

[0051] Figure 26B This is a diagram illustrating an example of a sample related to a control treatment.

[0052] Figure 26C This is a diagram illustrating an example of a sample related to a control treatment.

[0053] Figure 26D This is a diagram illustrating an example of a sample related to a control treatment. Detailed Implementation

[0054] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, in the following description, the term "document" also includes electronic documents (digital documents) composed of digital data.

[0055] <First Implementation Method>

[0056] Figure 1 This diagram illustrates an example of the hardware structure of an information processing apparatus 100 according to an embodiment of the present invention. The information processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, an output device 105, an input device 106, and a network I / F 107. The CPU 101, ROM 102, RAM 103, auxiliary storage device 104, output device 105, input device 106, and network I / F 107 are communicatively connected via a system bus 108.

[0057] The CPU (Central Processing Unit) 101 is the central arithmetic unit that controls various operations of the information processing device 100. For example, the CPU 101 can also control the operation of the entire information processing device 100. The ROM (Read Only Memory) 102 stores control programs, boot programs, etc., that can be executed by the CPU 101. The RAM (Random Access Memory) 103 is the main memory of the CPU 101, and is used as a working area or a temporary storage area for expanding various programs.

[0058] The auxiliary storage device 104 stores various data, programs, etc. The auxiliary storage device 104 is implemented by a storage device that can temporarily or continuously store various data, such as a non-volatile memory represented by HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0059] Output device 105 is a device that outputs various information to prompt the user with various information. For example, output device 105 is implemented by a display device such as a monitor. Output device 105 prompts the user with information by displaying various display information. As another example, output device 105 may also be implemented by an audio output device that outputs voice, electronic sounds, etc. In this case, output device 105 prompts the user with information by outputting voice, electronic sounds, etc. In addition, the device used as output device 105 may be appropriately changed depending on the medium used to prompt the user with information.

[0060] Input device 106 is used to receive various instructions from the user. For example, input device 106 includes input devices such as a mouse, keyboard, and touch panel. As another example, input device 106 may include a sound collection device such as a microphone to collect the user's speech. In this case, by performing various analytical processes such as audio analysis and natural language processing on the collected speech, the content displayed by the speech is recognized as an instruction from the user. Furthermore, the device used as input device 106 can be appropriately changed depending on the method of recognizing the user's instruction. Additionally, various devices can be used as input device 106.

[0061] The Network I / F107 is used for communication with external devices via a network. Devices using the Network I / F107 can be appropriately modified depending on the type of communication path or the communication method used.

[0062] The CPU 101 expands the program stored in the ROM 102 or auxiliary storage device 104 into the RAM 103 and executes the program, thereby realizing the various functions and processes of the information processing device described later. The program of the information processing device 100 can be provided to the information processing device 100 via a recording medium such as a CD-ROM, or downloaded via a network. When the program of the information processing device 100 is provided via a recording medium, the program recorded on the recording medium is installed in the auxiliary storage device 104 by placing the recording medium on a predetermined drive device.

[0063] Figure 2 This is a diagram illustrating an example of the functional structure of the information processing apparatus 100 according to the first embodiment. The information processing apparatus 100 includes an acquisition unit 201, an analysis unit 202, a database (DB) generation unit 203, an information extraction unit 204, a conformity determination unit 205, an output unit 209, and a storage unit 210.

[0064] In this embodiment, the information processing apparatus 100 utilizes a learned model 211 generated through machine learning (deep learning) and stored in the storage unit 210 to perform part of the functionality. Hereinafter, the learned model 211 will be described as a Large Language Model (LLM). A Large Language Model (LLM) is a language model constructed using large amounts of text data (large-scale corpora, etc.) and deep learning techniques. When input text data representing instructions, commands, etc., called prompts, it infers based on the prompts, generates text data corresponding to the input prompts, and outputs it. Furthermore, it is not limited to a Large Language Model (LLM); for example, a Large Multimodal Model (LMM) capable of corresponding to text data can also be applied as the learned model 211.

[0065] The acquisition unit 201 acquires documents related to the conformity verification operation. The acquisition unit 201 acquires a first document (the document of the verification object) confirming whether the required conditions are met, and a second document indicating the conditions of the verification object corresponding to the first document. The acquisition unit 201 is an example of an acquisition unit. Examples of the first document include specifications related to products or services, contracts related to contract content, proposals, etc. Examples of the second document include specifications or regulations indicating the rules used as a basis, or RFPs (Request for Proposals). However, the examples of the first and second documents are just examples and are not limited to these.

[0066] The analysis unit 202 analyzes the first document acquired by the acquisition unit 201, extracts content from the first document to determine whether it meets the required conditions, and generates a comparison task (hereinafter, also simply referred to as a "task") for each extracted content. The analysis unit 202 is an example of a parsing unit. The analysis unit 202 generates a task related to the verification operation of the first document by inputting the first document into the learned model 211 and providing a prompt containing instructions to extract the content to be verified and output the task. The generated comparison task (task) is stored in the storage unit 210 as task data 213. In addition, the analysis unit 202 is not limited to using the learned model 211 to generate the comparison task (task). For example, the analysis unit 202 may also generate the comparison task (task) by extracting the content to be verified from a document (first document) with a predetermined format through algorithm-based mechanical processing.

[0067] DB generation unit 203 generates a document-related database (retriever) based on the documents obtained by acquisition unit 201. The database (retriever) generated by DB generation unit 203 is stored as a document DB (retriever) in storage unit 210. That is, DB generation unit 203 generates a first document DB (retriever) 215 based on the first document obtained by acquisition unit 201, and generates a second document DB (retriever) 216 based on the second document obtained by acquisition unit 201. These document DBs (retrievers) 215 and 216 are DBs (retrievers) used as search tools when performing content-related conformity determinations for documents using the learned model 211. Document DBs (retrievers) 215 and 216 are, for example, full-text search engines, performing searches based on BM25. DB generation unit 203 is an example of a generation unit.

[0068] Information extraction unit 204 extracts predetermined information from the first document obtained by acquisition unit 201. Information extraction unit 204 is an example of an information extraction unit. In the first embodiment, information extraction unit 204 extracts prerequisite information from the first document obtained by acquisition unit 201. Here, prerequisite information is information generalized through the first document, such as information related to a reference (rule) in the first document. Information extraction unit 204 inputs the first document and a prompt containing an instruction to extract information generalized through the first document into the learned model 211, thereby extracting prerequisite information related to the first document. The extracted prerequisite information is stored in storage unit 210 as prerequisite information 214, and is provided to the learned model 211 as prompt information during the conformity determination process related to the document content using the learned model 211.

[0069] The conformity determination unit 205 performs conformity determination related to the first document as the object of verification. Based on the task related to the first document stored as task data 213 in the storage unit 210, the conformity determination unit 205 determines whether the content of the first document to be verified in the task conforms to the required conditions. Specifically, the conformity determination unit 205 determines whether the content of the first document shown in the task conforms to the conditions shown in the second document by inputting the task related to the first document and prerequisite information as prompts into the learned model 211. Furthermore, if the task verified by the conformity determination unit 205 in the conformity determination is not a single task that can be verified in the comparison processing unit 206 (described later), but rather consists of multiple tasks containing multiple contents, the task is divided into verifiable tasks. The conformity determination unit 205 includes a comparison processing unit 206, an information search unit 207, and a code execution unit 208.

[0070] The comparison processing unit 206 performs comparison processing between the first document and the second document to determine whether the content of the first document meets the required conditions. The comparison processing unit 206 is an example of a processing unit. The comparison processing unit 206 infers whether the content of the first document, as indicated by the task, meets the conditions indicated by the second document by inputting a prompt containing task and prerequisite information related to the first document into the learned model 211. Furthermore, the comparison processing unit 206 outputs a search request or code (program) for performing calculations to the information search unit 207 or the code execution unit 208. The search request is generated by inputting task and prerequisite information related to the first document as prompts into the learned model 211.

[0071] Information search unit 207 searches for the first document and the second document based on a search request from comparison processing unit 206 and outputs the search results to comparison processing unit 206. Information search unit 207 is an example of a search unit. Information search unit 207 accepts a search query input from comparison processing unit 206, performs search processing on the first document DB215 and the second document DB216 using learned model 211, and returns the search results corresponding to the input search query to comparison processing unit 206. However, information search unit 207 is not limited to using learned model 211 for search processing and outputting search results. For example, information search unit 207 may also accept a search query input from comparison processing unit 206, perform rule-based search processing on the first document DB215 and the second document DB216, and return the obtained search results to comparison processing unit 206. For example, as described later... Figure 11 In the search process shown, the search result 1106 obtained by performing search process 1104 can also be directly output as the final search result 1108.

[0072] The code execution unit 208 executes the code (program) and outputs the execution result. The code execution unit 208 executes the code (program) input from the comparison processing unit 206 and returns the execution result to the comparison processing unit 206.

[0073] The output unit 209 outputs the result of the compliance determination related to the first document performed by the compliance determination unit 205, that is, the determination result of whether the content of the first document extracted as a task related to the first document meets the required conditions. The output unit 209 may output the determination result of whether the content of the first document meets the required conditions and the reasons for the determination result together.

[0074] The storage unit 210 stores various data used during processing in the information processing device 100. For example, the storage unit 210 stores a learned model 211, a setting file 212, task data 213, prerequisite information 214, a first document DB 215, and a second document DB 216. The learned model 211 is a machine learning (deep learning) model used in various inferences performed in the information processing device 100. The setting file 212 is a file storing prompts to be input into the learned model 211. The setting file 212 may also store learning samples input as prompts. Multiple learning samples can be prepared in the setting file 212, etc., and selected and provided to the learned model 211 according to the function to be implemented. The task data 213 is a task related to the confirmation operation generated by the analysis unit 202, and the prerequisite information 214 is prerequisite information extracted by the information extraction unit 204. The first document DB 215 is a DB for first documents generated by the DB generation unit 203, and the second document DB 216 is a DB for second documents generated by the DB generation unit 203.

[0075] Reference Figure 3 The processing in the information processing apparatus 100 of the first embodiment will be described. Figure 3 This is a diagram illustrating a processing example of the information processing apparatus 100 in the first embodiment. Hereinafter, as an example, we will describe a case where the first document to be verified is the product specification 301, and the second document representing the conditions of the required verification object is a standard document 302 related to the product standard, and we verify whether the contents recorded in the specification 301 conform to the standard of the standard document 302.

[0076] The information processing device 100 acquires the specification 301 as the first document. Furthermore, the information processing device 100 acquires the standard document 302 as the second document.

[0077] (Handling 303 and 305: Segmentation of specifications and task decomposition)

[0078] The information processing device 100 performs a specification segmentation process 303 on the acquired specification 301, dividing the specification 301 into multiple documents 304 that become part of the specification. In the specification segmentation process 303, the information processing device 100 uses algorithm-based mechanical processing to segment the specification 301 into multiple documents, making each segmented document a size that can be processed by the learned model 211. For example, the information processing device 100 segments the text data of the specification 301 read by an OCR or PDF parser, etc., according to a predetermined pattern such as listing numbers one by one, dividing the specification 301 into multiple documents 304. Figure 4 In the example of specification document segmentation shown, the information processing device 100 lists the specification document 401 into text data 402-405 for each item number (item) based on a numbered list. The tables in the specification document are, for example, converted to CSV format.

[0079] In addition, the information processing device 100 performs task decomposition processing 305 on the segmented multiple documents 304 to generate a comparison task (task) 306 related to the content verification of the specification 301. In the task decomposition processing 305, the information processing device 100 generates the task 306 related to the content verification of the specification 301 by inputting the segmented documents 304 into the learned model 211 and a prompt containing instructions to extract the content that needs to be verified to conform to the standard and output the task.

[0080] Reference Figures 5A to 5C , Figure 6 , Figures 7A to 7D as well as Figure 8 This section provides an example of task decomposition in task decomposition processing 305.

[0081] Reference Figures 5A to 5C as well as Figure 6 An example of generating a task to confirm quantitative matters is provided. Figure 5A This indicates that the chemical composition of the product, as described in specification 301, includes the values ​​of carbon (C), manganese (Mn), silicon (Si), and phosphorus (P) for each product. Regarding this... Figure 5A The data 304 shown is the segmented data. Figure 5B The data shown. The information processing device 100 transmits the data via... Figure 5B The data shown and Figure 6 The prompt 600 shown is input into the learned model 211, which breaks down the content that needs to be confirmed into tasks, such as... Figure 5C Generate the task as shown.

[0082] like Figure 6As shown, prompt 600 includes prompt 601, which comprehensively outputs values ​​that need to be verified to conform to the standard because the recorded values ​​are related to the standard, and instructs that the items to be checked should be included in verifiable small tasks as much as possible. Furthermore, prompt 600 may also include learning samples 602 for tabular data, learning samples 603 for cases where no information to be extracted is available, etc. By using such prompts, such as... Figure 5C As shown, the granularity of the information (task) to be extracted can be controlled, decomposed into smaller tasks that can be verified in the comparison processing described later, and tasks are generated to confirm whether the values ​​described in specification 301 conform to the standard. Furthermore, by including the illustrated learning samples in prompt 600, the desired form of task output can be obtained using the characteristics of a high-precision large-scale language model that returns output adapted to the learning samples.

[0083] Next, refer to Figures 7A to 7D as well as Figure 8 An example of generating a task (comparison task) for confirming qualitative matters is provided. Figure 7A This refers to the specifications in specification 301, which detail product-related quality management standards, product identification and traceability, and surface finishing. The information processing device 100 processes information related to... Figure 7A The data after the segmentation of the quality management benchmark shown. Figure 8 The prompt 800 inputs into the learned model 211, breaking down the content that needs to be confirmed into tasks, such as... Figure 7B The task is generated as shown (comparison task). Similarly, for the learned model 211, the input is used to generate the task based on... Figure 7A The data shown is segmented for product identification and traceability. Figure 8 The prompt 800 shown is as follows: Figure 7C The task is generated as shown (comparison task), by inputting information about... Figure 7A The data shown are the segmented data after surface finishing. Figure 8 The prompt 800 shown is as follows: Figure 7D Generate the task as shown (compare with the task).

[0084] like Figure 8 As shown, prompt 800 includes prompt 801, whose output requires verification of consistency with the standard's documentation due to its relevance to the standard, and instructs that the items to be checked should be incorporated into verifiable mini-tasks as much as possible. Furthermore, prompt 800 may include learning samples 802 and 803 related to the extraction of qualitative items, learning samples in cases where no information to be extracted exists, etc. By using such prompts, such as... Figures 7B to 7DAs shown, the granularity of the information (task) to be extracted can be controlled, decomposed into smaller tasks that can be verified in the comparison processing described later, and tasks are generated to confirm whether the content described in specification 301 conforms to the standard. Furthermore, by including the illustrated learning samples in prompt 800, the characteristics of a high-precision large-scale language model that returns output adapted to the learning samples can be used to obtain the task output in the desired format.

[0085] (Process 307: Prerequisite Information Extraction)

[0086] For the obtained specification 301, the information processing device 100 performs a prerequisite information extraction process 307 to extract prerequisite information 308, which is information applicable through the specification 301. In the prerequisite information extraction process 307, the information processing device 100 inputs the specification 301 and a prompt containing an instruction to extract information applicable through the specification to the learned model 211, thereby generating prerequisite information 308 related to the specification 301. In this example, such as... Figure 9 As shown, by inputting a prompt 900 to instruct the extraction of the standards and product types explicitly stated in the specification from the specification 301, the referenced standard information and the product types stated in the specification are extracted as prerequisite information 308.

[0087] (Handling 309 and 311: DB generation)

[0088] The information processing apparatus 100 performs a specification retriever (DB) generation process 309 based on the acquired specification 301, generating a specification DB 310. Furthermore, the information processing apparatus 100 performs a standard document retriever (DB) generation process 311 based on the acquired standard document 302, generating a standard document DB 312. The specification DB 310 is equivalent to... Figure 2 The first document DB (retriever) 215 shown is equivalent to the standard document DB (retriever) 312. Figure 2 The second document DB (retriever) 216 is shown. In the DB generation processes 309 and 311, the information processing device 100 generates a database by processing the specification 301 and standard document 302 into a searchable form through algorithm-based mechanical processing.

[0089] For example, such as Figure 10As shown in one example, the information processing device 100 generates a record for each page of the standard document 302, containing information such as filenames like "IronStandard2023," document names like "steel plate standard," body text like "1. First, this document is...", and page numbers like "p1," thus forming the standard document DB (retriever) 312. For example, by including page numbers in the records, when searching using the learned model 211 described later, it is possible to respond to different pages such as referring to the next page or the previous page. The specification DB (retriever) 310 can be constructed in the same way. Furthermore, the information contained in each record is not limited to the above example and may include other information. In addition, while a record is generated for each page of the document, this is just one example; records may also be generated for each other constituent unit.

[0090] Here, the standard document DB (retriever) 312 can be generated in advance or when verifying whether the content of specification 301 conforms to the standard. For example, if a standard document DB (retriever) 312 is generated and maintained for repeated use or long-term use, it is not necessary to generate it every time, which can reduce processing. In addition, if it is a standard document that is not allowed to be stored for a long time, generating the standard document DB (retriever) 312 when verifying whether the content of specification 301 conforms to the standard can prevent information leakage and other occurrences.

[0091] (Process 320: Meets the criteria)

[0092] For each task generated in task decomposition processing 305, information processing device 100 uses learned model 211 to perform compliance judgment processing 320, determining whether the content recorded in specification 301 conforms to the standard in standard document 302. Tasks 306 generated by task decomposition processing 305 are stored in a queue in task list 321. In compliance judgment processing 320, for each task 322 taken sequentially from task list 321, information processing device 100 uses learned model 211 to perform comparison processing 323, outputting processing result 340. Comparison processing 323 continues until tasks in task list 321 disappear.

[0093] In the comparison processing 323, the learned model 211 simulates the workflow of a person comparing the content of a document, using tools to collect the information required for comparison, and makes meaningful inferences based on the collected information to obtain the processing result. In addition, when making inferences, the learned model 211 learns (such as Few-shot learning) samples of workflows in a few or most specific domains to make it conform to the domain of the workflow.

[0094] In this embodiment, the contrast processing 323 mimics a framework called ReAct (Reasoning and Acting) to process and output the processing result. That is, in contrast processing 323, mimicking the ReAct framework, text is generated for each stage of Thought, Action, and Observation for the Question. After one or more cycles of Thought→Action→Observation, the Final Answer is derived.

[0095] In this example, Task 322 is equivalent to Question. In Thought, a plan from the current process is considered to achieve the goal using the learned model 211. In Action, the tool used to execute the plan is selected using the learned model 211. Then, Observation is used to observe the execution result of the tool after input for execution. Based on the execution result of the tool obtained through this Observation, a plan is considered using Thought; if there is a plan, it proceeds to Action; if there is a conclusion, a Final Answer is generated.

[0096] In this embodiment, the tools used in the comparison process 323 for the learned model 211 include a task segmentation tool 324, a specification search tool 326, a standard search tool 329, and a code execution tool 332. The task segmentation tool 324 is implemented by the conformity determination unit 205, the specification search tool 326 and the standard search tool 329 are implemented by the information search unit 207, and the code execution tool 332 is implemented by the code execution unit 208.

[0097] The task segmentation tool 324 has the function of appending the tasks 325 segmented by the comparison process 323 to the task list 321. As described later, if the task 322 taken from the task list 321 and input into the comparison process 323 is not a single verifiable task, but rather consists of multiple tasks containing multiple contents, then in the comparison process 323, the task 322 is segmented into verifiable tasks 325. The task segmentation tool 324 then appends the segmented tasks to the task list 321.

[0098] The specification search tool 326 has the function of searching for detailed information required for the verification of task 322 from specification 301. Specification search tool 326 accepts specification search queries 327, searches specification DB (Retriever) 310, and returns the search results as specification information 328. Furthermore, specification search tool 326 has the function of extracting relevant parts from the search results of specification DB (Retriever) 310 and providing a summary, as well as the function of attaching explanations to the extracted content. Through this search result summary function, for example, the number of characters can be reduced to provide the search results as suggestions to the learned model 211.

[0099] The standard search tool 329 has the function of searching the standard document 302 for detailed information on the standard corresponding to the content described in the specification 301. The standard document search tool 329 accepts a standard document search query 330, searches the standard document DB (Retriever) 312, and returns the search results as standard information 331. Furthermore, the standard document search tool 329 has the function of extracting relevant parts from the search results in the standard document DB (Retriever) 312 and summarizing them, as well as adding explanations to the extracted content. Through this search result summarization function, for example, the number of characters can be reduced to provide the search results as suggestions to the learned model 211.

[0100] The code execution tool 332 has the function of executing the code (program) generated by the learned model 211 for performing numerical comparisons and other operations (including unit conversions, arithmetic operations, etc.). The code execution tool 332 accepts the code (program) 333 generated by the learned model 211, executes the code (program), and returns the execution result 334.

[0101] Here, we will explain the searches based on search tools 326 and 329. The following explanation will use a search based on standard search tool 329 as an example, but the same applies to searches based on specification tool 326. Figure 11 This is a diagram used to illustrate an example of searching using the standard document search tool 329.

[0102] When searching for standard documents, the information processing device 100 generates a search intent 1101 and a search query 1103 based on the task to be verified and prerequisite information. Figure 11 In the example shown, the information processing device 100 generates a search intent 1101 containing "want to know the fatigue strength of IS360B-SSB" and a search query 1103 containing "IronStandard 2023", "steel plate", "IS360B-SSB", and "fatigue strength".

[0103] Next, the information processing device 100 performs search processing 1104 using the generated search query 1103. In search processing 1104, rule-based search processing is performed on the standard document data 1105 using search query 1103, and search results 1106 corresponding to search query 1103 are output.

[0104] Here, in the contrast processing described later, the search results based on search tools 326 and 329 are appended to the prompts for inference based on the learned model 211. For example, when information about a specific row in a multi-row table is needed, if no explanation is provided and only the relevant part is extracted, information such as the title is lost, and only information listing words is extracted. Even if only such information is input into the learned model 211 as a search result, it is difficult to interpret within the learned model 211, and therefore sometimes incorrect inferences and plans related to the contrast processing are established, resulting in inferences lacking planning. This is unlikely to occur when extracting information from documents that are typically written in articles, but it is possible when extracting information from documents with layouts, such as specifications and standard documents, due to the extensive use of charts and graphs.

[0105] Therefore, in this embodiment, the information processing device 100 performs search result summarization processing 1107 on the search result 1106 obtained through search processing 1104, summarizing the search results according to the search intent 1101, and outputs the processing result as the final search result 1108. The search result summarization processing 1107 is performed using the learned model 211. In the search result summarization processing 1107, for the search result 1106 obtained through search processing 1104, the corresponding part (the part associated with the search intent 1101) is extracted and a description of the extracted content is provided. For example, regarding the search intent 1101, if object information is obtained, the corresponding part and description are output; if object information is not obtained but information such as a table to be referenced is obtained, that information and description are output; if no object information is obtained, "NO_OUTPUT" and a description are output. Furthermore, the description is generated, for example, considering articles recorded on one or more pages where the information was obtained. For example, in... Figure 11 In the example shown, the first search result 1106 contains information related to the fatigue strength of IS360A, IS360B and IS340, but the information processing device 100 extracts only the information related to the fatigue strength of IS360B from the first search result 1106 according to the search intention 1101 and outputs it together with the description as the final search result 1108.

[0106] In this way, by performing search result summary processing 1107 on the search results 1106 of search tools 326 and 329 and outputting the final search results 1108, the search results and their explanatory information are provided to the learned model 211 in the comparison processing, thereby enabling the input of appropriate information into the learned model 211 and improving the accuracy related to inference and planning.

[0107] The following examples illustrate the control treatment. Figures 12-19 In the reasoning process of the contrast processing shown, Question is the task of confirming the object (input information of the contrast processing), Thought, Action, Action Input, and Final Answer are reasoning based on the learned model 211, and Observation is based on the execution results of each tool.

[0108] As described above, in this embodiment, when making inference in the control process, the learned model 211 is made to learn (such as through few-shot learning) samples of the workflow related to the control process, so that it conforms to the domain related to the control process. For example, by providing the learned model 211 with samples containing, Figures 26A-26D Learning is based on cues from the samples shown. Inferences are made during control treatments, combined with... Figures 26A-26D Elements 2600A to 2600D, as shown, have a task appended to their end as a question for inference. Figure 26A In this context, rule 2601 relates to the control treatment, and rule 2602 contains examples that are compared against the search criteria and samples related to the accompanying numerical comparisons. Figure 26B In the sample, 2603 is an example that is compared with the search criteria and specifications, and is related to the control examples with accompanying numerical comparisons. Figure 26C In the sample, 2604 is a case study related to examples of using appropriate information in multi-stage searches. Additionally, in... Figure 26D In the example, 2606 is a sample related to handling failures, and 2607 is a sample related to dividing a task consisting of multiple tasks into a single task. By providing hints containing such samples to the learned model 211, task-related processing can be performed in an appropriate flow corresponding to the task.

[0109] Figure 12 This is a diagram illustrating an example of a control treatment. Figure 12 This illustrates the inference process of comparing and verifying a search criterion for quantitative matters. Figure 12In the example shown, as indicated in Question, task 322 was entered to confirm whether the applicable size "10m" of product model IS380A (-SSP) meets the IronStandard's limitations. Additionally, information on standard candidates extracted from specification 301 as prerequisite information 308 and information on the type of steel were entered as supplementary information.

[0110] By inputting information including the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 performs inference based on the information input as prompts through the learned model 211. Through this inference, it plans to investigate the applicable size (Thought) of the corresponding product in the standard document 302, selects the standard search tool 329 (Search Standard) as the tool (Action) to execute it, and generates information (independent variables) (Action Input) to be input into the standard search tool 329. Here, standard_no, title, and sentence are information input into the search tool as search queries, and question is information input into the search tool as search intent.

[0111] Based on the information recorded in the Action Input, the standard document DB312 is searched using the standard search tool 329, and the search results of the standard search tool 329, as the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). In the standard search tool 329, the search result summarization processing described above is performed; for example, the final search result, consisting of the relevant parts extracted from the search results for the standard document 302 and a description of the extracted content, is output as the search result of the standard search tool 329. (The same applies to other inference processes.)

[0112] Subsequently, by adding the execution result of the tool recorded in the Observation (the search result of the standard document) as a prompt, the information processing device 100 performs an inference based on the learned model 211 based on the information input as a prompt. Through this inference, it is planned to verify whether the applicable size limit (Thought) of the product, inferred from the record of the standard document 302, is met by code (program), select the code execution tool 332 (Run Code) as the tool (Action) to execute this content, and generate code (Action Input) to be executed by the code execution tool 332.

[0113] The code (program) recorded in the Action Input is executed by the code execution tool 332, and the execution result (True) of the code execution tool 332 is returned to the comparison processing unit 206 (Observation). By appending the execution result of the tool (the execution result of the code) recorded in the Observation as a prompt, the information processing device 100 performs inference based on the information input as a prompt, based on the learned model 211. Based on the code-based verification result, it is determined that the applicable size of the product model IS380A meets the limitation (Thought) of the applicable size inferred from the standard document 302. As the comparison processing result, various information related to confirmation and a confirmation result of OK (suitable) are generated (Final Answer).

[0114] Figure 13 This is a diagram illustrating an example of a control treatment. Figure 13 This illustrates the inference process of comparing and verifying a search criterion for quantitative matters. Figure 13 In the example shown, as in Question, a comparison process is performed for task 322 to confirm whether the record of the applicable size "9m" for product model IS360B(-SSS) meets the IronStandard's limitation.

[0115] exist Figure 13 In the example shown, with Figure 12 Similarly, in the example shown, inference based on the learned model 211 is performed, processed using the standard search tool 329 and the code execution tool 332, and the execution result (False) based on the code execution tool 332 is returned to the comparison processing unit 206. By adding the execution result of the code execution tool 332 as a prompt, the information processing device 100 performs inference based on the information input as a prompt. Based on this inference and the code-based verification result, it is determined that the applicable size of product model IS360B does not meet the limitation (Thought) of the applicable size inferred from the description in standard document 302. As a comparison processing result, various information related to confirmation and a final answer indicating that the confirmation result is NG (Not compliant) are generated.

[0116] Figure 14 This is a diagram illustrating an example of a control treatment. Figure 14 This illustrates the reasoning process for comparing and verifying qualitative search criteria. Figure 14In the example shown, as indicated in Question, task 322 was entered to confirm whether the surface pretreatment on the standard is pickling. Additionally, information on the standard candidate and steel type extracted from specification 301 as prerequisite information 308 was entered as supplementary information.

[0117] By inputting information including the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 makes inferences based on the information input as prompts, using the learned model 211. Based on this inference, it plans to investigate whether the surface pretreatment in the standard document 302 is a pickling process (Thought), selects the standard search tool 329 (SearchStandard) as the tool (Action) to perform this inference, and generates information (independent variables) (Action Input) to be input into the standard search tool 329.

[0118] Based on the information recorded in the Action Input, the standard document DB312 is searched using the standard search tool 329, and the search results of the standard search tool 329, as the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). By adding the tool's execution result (the search results of the standard document) recorded in the Observation as a prompt, the information processing device 100 performs an inference based on the learned model 211 based on the information input as a prompt. Through this inference, it is determined that the surface pretreatment inferred from the record in the standard document 302 is acid washing and conforms to the standard (Thought). As the result of the comparison processing, various information related to confirmation and a confirmation result OK (conforms) are generated as a final answer.

[0119] Figure 15 This is a diagram illustrating an example of a control treatment. Figure 15 This illustrates the reasoning process for comparing and verifying qualitative search criteria. Figure 15 In the example shown, as in Question, a comparison process is performed for task 322 to confirm whether the final finish of the surface on the standard is a brush finish.

[0120] exist Figure 15 In the example shown, with Figure 14Similarly, in the example shown, inference based on the learned model 211 is performed, processed using the standard search tool 329, and the execution result (search result) based on the standard search tool 329 is returned to the comparison processing unit 206. By adding the execution result of the standard search tool 329 as a prompt, the information processing device 100 performs inference based on the information input as a prompt, based on the learned model 211. Through this inference, it is determined that the final finishing of the surface predicted according to the description of the standard document 302 is not brush finishing and does not meet the standard (Thought). As a result of the comparison processing, various information related to confirmation and a confirmation result of NG (not conforming) (FinalAnswer) are generated.

[0121] Figure 16A and Figure 16B This is a diagram illustrating an example of a control treatment. In Figure 16A and Figure 16B The example illustrates the reasoning process for comparing and verifying quantitative matters against search criteria and specifications. In this example, such as... Figure 16A As shown in the Question section, task 322 asks whether the fatigue strength of IS380A (-SSP) "280≤F≤290" meets the IronStandard limit. Additionally, information on the standard candidate and steel type extracted from specification 301 as prerequisite information 308 is input as supplementary information.

[0122] By inputting information including the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 performs inferences based on the information input as prompts, using the learned model 211. Based on this inference, it plans to investigate the fatigue strength (Thought) of the corresponding product in the standard document 302, selects the standard search tool 329 (Search Standard) as the tool (Action) to perform this inference, and generates information (independent variables) (Action Input) to be input into the standard search tool 329.

[0123] Based on the information recorded in Action Input, the standard document DB312 is searched using the standard search tool 329, and the search results of the standard search tool 329, as the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). By adding the result of the tool's execution (the search results of the standard document) recorded in the Observation as a prompt, the information processing device 100 performs inferences based on the information input as a prompt using the learned model 211. Through this inference, whether the fatigue strength of the corresponding product predicted according to the description in the standard document 302 depends on the length of the product, therefore, it is planned to investigate the length of the product (Thought) from the specification 301, and the specification document search tool 326 (Search Design Document) is selected as the tool (Action) to perform this inference, generating information (independent variables) (Action Input) to be input to the specification document search tool 326.

[0124] Based on the information recorded in the Action Input, the specification DB310 is searched using the specification search tool 326, such as... Figure 16B As shown, the search results of the specification search tool 326, which is the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). By adding the execution result of the tool recorded in the Observation (the search results of the specification) as a prompt, the information processing device 100 makes an inference based on the information input as a prompt, based on the learned model 211. Based on this inference, it is planned to verify by code (program) whether the description of fatigue strength in specification 301 meets the fatigue strength limit (Thought) of the corresponding product inferred from the description in standard document 302, select code execution tool 332 (Run Code) as the tool (Action) to perform this processing, and generate code (Action Input) to be executed by code execution tool 332.

[0125] The code (program) recorded in the Action Input is executed by the code execution tool 332, and the execution result (True) of the code execution tool 332 is returned to the comparison processing unit 206 (Observation). By appending the execution result of the tool (the execution result of the code) recorded in the Observation as a prompt, the information processing device 100 performs inference based on the learned model 211 based on the information input as a prompt. Based on the code-based verification result, it is determined that the fatigue strength of IS380A meets the fatigue strength limit (Thought) inferred from the standard document 302. As the comparison processing result, various information related to the confirmation and a confirmation result OK (compliant) answer (Final Answer) are generated.

[0126] exist Figure 16A as well as Figure 16B The example shown uses the task of confirming quantitative matters, but the same approach is taken for the task of confirming qualitative matters.

[0127] Figure 17A and Figure 17B This is a diagram illustrating an example of a control treatment. In Figure 17A and Figure 17B In this example, for quantitative matters, an inference process is illustrated regarding the use of appropriate information for comparative processing and confirmation in a multi-stage search. In this example, such as... Figure 17A As shown in the Question section, task 322 asks whether the carbon value of "0.20%" in the chemical composition of IS380A (-SSP) meets the Iron Standard's limit. Additionally, information on the standard candidate extracted from specification 301 as prerequisite information 308 and the steel type are input as supplementary information.

[0128] By inputting information including the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 makes inferences based on the information input as prompts, using the learned model 211. Based on this inference, it plans to investigate the carbon value in the chemical composition of the corresponding product in the standard document 302 (Thought), selects the standard search tool 329 (Search Standard) as the tool (Action) to execute it, and generates information (independent variables) (Action Input) to be input into the standard search tool 329.

[0129] Based on the information recorded in Action Input, the standard document DB312 is searched using the standard search tool 329, and the search results of the standard search tool 329, as the execution result of the tool, are returned to the comparison processing unit 206 (Observation). By adding the execution result of the tool (the search results of the standard document) recorded in the Observation as a prompt, the information processing device 100 performs inference of the learned model 211 based on the information input as a prompt. Through this inference, it is not possible to directly obtain the carbon value in the chemical composition of the product from the standard document 302, but it is inferred from the record in the standard document 302 that the chemical composition is recorded in Table 2. Therefore, it is planned to search for the chemical composition (Thought) in Table 2 of the standard document 302, and the standard search tool 329 (Search Standard) is selected as the tool (Action) to execute it, and information (independent variable) (Action Input) to be input to the standard search tool 329 is generated.

[0130] Based on the information recorded in the Action Input, the specification DB310 is searched using the standard search tool 329, such as... Figure 17B As shown, the search results of the standard document search tool 329, which is the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). By adding the execution result of the tool (the search results of the standard document) recorded in the Observation as a prompt, the information processing device 100 performs the inference of the learned model 211 based on the information input as a prompt. Through this inference, it is planned to verify whether the carbon value limit (Thought) in the chemical composition of the product, which is inferred from the record of the standard document 302, is met by code (program). The code execution tool 332 (Run Code) is selected as the tool (Action) to execute it, and code (Action Input) to be executed by the code execution tool 332 is generated.

[0131] The code (program) recorded in the Action Input is executed by the code execution tool 332, and the execution result (True) of the code execution tool 332 is returned to the comparison processing unit 206 (Observation). By appending the execution result of the tool (the execution result of the code) recorded in the Observation as a prompt, the information processing device 100 performs an inference based on the learned model 211 based on the information input as a prompt. Based on the code-based verification result, it is determined that the carbon value in the chemical composition of IS380A meets the limit (Thought) of the carbon value in the chemical composition predicted according to the description of the standard document 302. As the comparison processing result, various information related to the confirmation and a final answer indicating that the confirmation result is OK are generated.

[0132] exist Figure 17A as well as Figure 17B The example shown uses the task of confirming quantitative matters, but the same approach is taken for the task of confirming qualitative matters.

[0133] Figure 18 This is a diagram illustrating an example of a control treatment. In Figure 18 The diagram illustrates the inference process for decomposing an input task containing multiple tasks into individual tasks. Figure 18 In the example shown, as indicated in Question, task 322 is entered to confirm whether the fatigue strengths "270≤F≤280" and "260≤F≤308" of IS380A (-SSA) and IS380B (-SSS) meet the IronStandard limits. Additionally, information on standard candidates and steel types extracted from specification 301 as prerequisite information 308 is entered as supplementary information.

[0134] By inputting information including the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 performs inference based on the information input as prompts using the learned model 211. Through this inference, task decomposition is required; therefore, it is planned to decompose the tasks and add them to the task list (Thought), select the task segmentation tool 324 (Add Task) as the tool (Action) to execute it, and generate information (Action Input) to be input into the task segmentation tool 324.

[0135] Based on the information recorded in the Action Input, the task segmentation tool 324 adds the decomposed tasks to the task list 321, and returns the processing result of the task segmentation tool 324, which is the execution result of the tool, to the comparison processing unit 206 (Observation). For example, the task of confirming whether the fatigue strength "270≤F≤280" and "260≤F≤308" of IS380A (-SSA) and IS380B (-SSS) meet the Iron Standard limit is decomposed into the tasks of confirming whether the fatigue strength "270≤F≤280" of IS380A (-SSA) and IS380B (-SSS) meets the Iron Standard limit and whether the fatigue strength "260≤F≤308" of IS380B (-SSS) meets the Iron Standard limit, and is added to the task list 321.

[0136] By adding the execution results of the tool recorded in the Observation as a prompt, the information processing device 100 makes inferences based on the information input as a prompt, using the learned model 211. Through this inference, it is determined that the addition of the task decomposed by the task segmentation tool 324 to the task list 321 is a normal completion (Thought), and as a comparison processing result, a final answer indicating skip is generated.

[0137] Figure 19 This is a diagram illustrating an example of a control treatment. Figure 19 The inference process for a case study of suppressing hallucinations when the search fails is illustrated. Figure 19 In the example shown, as indicated in Question, task 322 is entered to confirm whether the resistance of IS380A (-SSA) "1.0×10^-7 or more, 1.5×10^-7 Ω·m or less" meets the IronStandard limit. Additionally, information on the standard candidate and steel type extracted from datasheet 301 as prerequisite information 308 is entered as supplementary information.

[0138] By inputting information containing the tasks and additional information recorded in these Questions as prompts into the learned model 211, the information processing device 100 makes inferences based on the information input as prompts, using the learned model 211. Through this inference, it plans to investigate the resistance (Thought) of the corresponding product in the standard document 302, selects the standard search tool 329 (SearchStandard) as the tool (Action) to execute it, and generates information (independent variables) (Action Input) to be input into the standard search tool 329.

[0139] Based on the information recorded in the Action Input, the standard document DB312 is searched using the standard search tool 329, and the search results of the standard search tool 329, as the result of the tool's execution, are returned to the comparison processing unit 206 (Observation). By appending the result of the tool's execution (the search results of the standard document) recorded in the Observation as a prompt, the information processing device 100 performs inference based on the information input as a prompt, using the learned model 211. Through this inference, if information about the product's resistance cannot be obtained in the standard document 302, it is determined that it is impossible to confirm whether the resistance recorded in the specification 301 meets the standard's limitations (Thought). As a result of the comparison processing, various information related to confirmation and a final answer indicating that the confirmation result is NG (not compliant) are generated. In this way, when object information cannot be obtained, "NO_OUTPUT" is output, clearly indicating that object information cannot be obtained, thereby suppressing the occurrence of illusions.

[0140] Figure 20A and Figure 20B This is a diagram illustrating an example of the output of the processing result in this embodiment. Regarding the processing result in the information processing apparatus 100 described above, for example... Figure 20A List 2001 or Figure 20B As shown in List 2011, the verification content and results are displayed in a summary for each verification task. Figure 20A List 2001 and Figure 20B The list 2011 shown displays all the tasks that have been verified, but it is also possible to show only the tasks that were verified as NG (not deemed to be compliant with the specification).

[0141] Alternatively, a task with an NG search result can be selected from the tasks displayed in the list based on user operations on the input device 106, for example, Figure 20A The detailed report shown in 2002 Figure 20B As shown in the detailed report 2012, it displays the basis for the verification result being NG. In this example, detailed reports 2002 and 2012 consist of a problem section and a verification process section, and can be generated based on inference processes such as those in comparison processing. Additionally, a GUI such as a button to display a detailed report for tasks with NG search results can be set up, and the detailed report is displayed based on the actions performed on that GUI.

[0142] Figure 20A The detailed report shown in 2002 is related to Figure 13 The report shows a detailed description of the task related to the inference process. Figure 20B The detailed report shown in 2012 is related to Figure 19The report provides a detailed overview of the inference process. For example, the question section is generated based on the Final Answer during the inference process, and the validated process section is generated based on the Thought, Action Input, and Observation from the inference process. Furthermore, the validated process section may generate queries based on the Action Input from the inference process, search results based on the Observation from the inference process, and analysis based on the Thought from the inference process.

[0143] Furthermore, the example above shows a scenario where the list of results and the detailed report indicating the basis for the validation result being NG are displayed separately, but it is also possible to display the list of results and the detailed report together. Additionally, the information shown in the detailed report is not limited to the example above; for example, it may also include information related to the tools used during validation.

[0144] According to the first embodiment, the information processing device 100 extracts the content to be verified from a first document (e.g., a specification) and generates a task for each content. The generated tasks are then sequentially input into a learned model 211, which answers whether the content to be verified in the task meets the conditions searched from a second document (e.g., a standard document), and outputs the processing result. This makes the verification process of whether the content of the first document, as the object of verification, meets the required conditions shown in the second document more efficient.

[0145] Furthermore, hallucinations typically occur in the generation of knowledge outside of domains that have not been learned in the already learned model. However, in this embodiment, the hallucination can be suppressed by using external tools (search tools 326, 329) to search for information during verification, appending the obtained search results to the prompt, and providing them to the already learned model for comparison processing.

[0146] <Second Implementation Method>

[0147] The first embodiment shows an example where a second document representing the conditions for confirmation can be determined based on the confirmed first document; however, it is sometimes difficult to determine the second document based on the first document. In the second embodiment, an example of a situation where it is difficult to determine the second document representing the conditions for confirmation based on the confirmed first document will be described. The hardware structure of the information processing apparatus 2100 in the second embodiment is similar to... Figure 1 The hardware structure of the information processing device 100 in the first embodiment shown is the same, so the description is omitted.

[0148] Figure 21This is a diagram illustrating an example of the functional structure of the information processing apparatus 2100 according to the second embodiment. Figure 21 In the middle, for those with Figure 2 Components with the same function shown are labeled with the same symbols, and repeated descriptions are omitted. The information processing apparatus 2100 includes an acquisition unit 201, an analysis unit 202, a database generation unit 203, an information extraction unit 204, a conformity determination unit 205, an output unit 209, a storage unit 210, and a candidate selection unit 2101. Furthermore, the conformity determination unit 205 includes a comparison processing unit 206, an information search unit 207, and a code execution unit 208.

[0149] In the information processing apparatus 2100, similarly to the information processing apparatus 100 in the first embodiment, a portion of the functionality is achieved using a learned model 211 generated through machine learning (deep learning) and stored in the storage unit 210. The learned model 211 is, for example, a large-scale language model (LLM) or a large-scale multimodal model (LMM) capable of corresponding to text data.

[0150] In the second embodiment, the analysis unit 202 generates a comparison task (task) related to the verification of the content of the first document, similar to the first embodiment. Furthermore, the analysis unit 202 parses one or more second documents obtained by the acquisition unit 201, extracts information representing the features of the benchmarks (rules) recorded in each second document, and constructs a rule index associated with the second document. The analysis unit 202 extracts information representing the benchmark (rule) features from each second document by inputting the second document into the learned model 211 and providing prompts including instructions to extract and output information representing the benchmark (rule) features. The information representing the extracted benchmark (rule) features is associated with the extracted second document and stored in the storage unit 210 as a rule index 2103 for the second document. Alternatively, the second document can be divided into multiple documents according to a predetermined pattern, such as chapters, and input into the learned model 211.

[0151] In the second embodiment, the information extraction unit 204 extracts attribute information from the first document acquired by the acquisition unit 201. Here, the attribute information is information representing the characteristics of the first document, such as information that helps determine a reference (rule) in the first document. The information extraction unit 204 extracts attribute information related to the first document by inputting the first document into the learned model 211 and providing a prompt containing instructions to extract information representing the characteristics of the first document. The extracted attribute information is stored as attribute information 2102 in the storage unit 210. The information extraction unit 204 divides the first document into multiple documents, for example, based on chapters, and inputs them into the learned model 211. Therefore, attribute information is extracted for each of the multiple documents divided based on chapters, etc.

[0152] The candidate selection unit 2101 selects a second document (candidate document) from one or more second documents to confirm the content of the first document based on the attribute information 2102 extracted from the first document by the information extraction unit 204 and the rule index 2103 of the second document constructed by the analysis unit 202. The candidate selection unit 2101 is an example of a selection unit. The candidate selection unit 2101 selects a second document to confirm the content of the first document based on the similarity between the features of the first document and the features of the second document. For example, the candidate selection unit 2101 searches the rule index 2103 of the second document for each attribute information 2102 extracted from the first document, and selects the second document with the highest score (feature similarity) as a candidate document based on the total score (feature similarity) related to the search results obtained for each attribute information 2102 for each second document. Furthermore, the candidate selection unit 2101 is not limited to a single second document; for example, it may select multiple second documents whose score (feature similarity) exceeds a threshold. Regarding the second document selected by the candidate selection unit 2101, the DB generation unit 203 generates a database (retriever) related to the document and stores it in the storage unit 210 as the second document DB (retriever) 216.

[0153] Reference Figure 22 This explains the processing in the information processing apparatus 2100 of the second embodiment. Figure 22 This is a diagram illustrating a processing example of the information processing apparatus 2100 according to the second embodiment. Hereinafter, as an example, we will describe the case where the first document is set as the product specification 2201, the second document is set as the standard document 2202 related to the product standard, and we check whether the contents recorded in the specification 2201 conform to the standard of the standard document 2202.

[0154] The information processing device 2100 acquires the specification 2201 as a first document. Furthermore, the information processing device 2100 acquires the standard document 2202 as one or more second documents.

[0155] (Handling 2203 and 2205: Specifications segmentation and task decomposition)

[0156] The information processing device 2100 performs a specification segmentation process 2203 on the obtained specification 2201, dividing the specification 2201 into multiple documents 2204 that become part of the specification. In the specification segmentation process 2203, similar to the specification segmentation process 303 in the first embodiment, the information processing device 2100 divides the specification 2201 into multiple documents through algorithm-based mechanical processing, so that each of the segmented documents is of a size that can be processed by the learned model 211.

[0157] Furthermore, the information processing device 2100 performs task decomposition processing 2205 on the segmented multiple documents 2204 to generate a comparison task (task) 2206 related to the content verification of the specification 2201. In the task decomposition processing 2205, similarly to the task decomposition processing 305 in the first embodiment, the information processing device 2100 inputs the segmented documents 2204 and a prompt containing instructions to extract the content that needs to be verified to meet the standards and output the task, thereby generating the task 2206 related to the content verification of the specification 2201.

[0158] (Process 2207: Attribute Information Extraction)

[0159] The information processing device 2100 performs attribute information extraction processing 2207 on the acquired specification 2201, extracting attribute information representing features of the first document that helps determine the standards referenced in the first document. In attribute information extraction processing 2207, the information processing device 2100 generates attribute information 2208 related to the specification 2201 by inputting the learned model 211 with a prompt containing instructions to extract information representing features in the specification 2201, which is divided into multiple documents based on chapters, etc. Figure 23A As shown, prompt 2300, through input instructions, extracts information representing the characteristics of the product described in the specification from specification 2201. This extraction helps determine, for example, the characteristics of the product described in the specification. Figure 23B Information such as the standards referenced in the specification sheet is used as attribute information 2208.

[0160] (Process 2209: DB generation for the specification)

[0161] Information processing device 2100 performs specification DB generation processing 2209 based on the obtained specification 2201, generating specification DB (retriever) 2210. Specification DB (retriever) 2210 is equivalent to Figure 21 The first document DB (retriever) 215 is shown. The information processing device 2100 generates a database by processing the specification 2201 into a searchable form through algorithm-based mechanical processing.

[0162] (Process 2211: Rule Index Construction)

[0163] For the obtained standard document 2202, the information processing device 2100 performs a rule index construction process 2211 for the standard document, extracting information representing the features of the standards recorded in each standard document, and constructing a rule index 2212 for the standard document. In the rule index construction process 2211, the information processing device 2100 extracts information representing the features of the standards from each standard document by inputting a standard document segmented into multiple documents based on chapters, etc., and a prompt containing instructions to extract and output information representing the features of the standards from the learned model 211. For example, the information processing device 2100 combines the segmented standard documents and... Figure 24A The prompt 2400 is input into the learned model 211, thereby extracting data from the standard document, such as... Figure 24B The information processing device 2100 constructs a rule index 2212 of the standard document by associating the extracted information representing the characteristics of the standard with the standard document. Figure 24A The prompts 2400 shown include prompt 2401 indicating that information representing the features of the standard should be briefly output from the standard document, and learning samples 2402, 2403 for the learned model 211 to learn.

[0164] (Process 2213: Extraction of candidate documents from standard documents)

[0165] Information processing device 2100 performs candidate document extraction processing 2213 for standard documents, selecting standard document 2202 as a candidate document for confirming the content of specification 2201. In the candidate document extraction processing 2213, information processing device 2100 searches the rule index 2212 of standard documents based on attribute information 2208 extracted in attribute information extraction processing 2207, selecting standard document 2202 with a high score (feature similarity) related to the search results as a candidate document for confirming the content of specification 2201. Furthermore, information processing device 2100 generates a standard document DB (retriever) 2214 based on the selected candidate document (the selected standard document 2202). The standard document DB (retriever) 2214 is equivalent to... Figure 21 The second document shown is DB (retriever) 216.

[0166] (Process 2220: Meets the criteria)

[0167] For each task generated in the task decomposition process 2205, the information processing device 2100, similarly to the first embodiment, uses the learned model 211 to perform a compliance determination process 2220 for each task, determining whether the content recorded in the specification 2201 conforms to the standard document 2202. In the compliance determination process 220, for each task 2222 sequentially retrieved from the task list 2221, which stores the tasks 2206 generated by the task decomposition process 2205 in a queue, the information processing device 2100 uses the learned model 211 to perform a comparison process 2223, outputting a processing result 2240. The comparison process 2223 continues until the tasks in the task list 2221 disappear.

[0168] In the comparison processing 2223, similarly to the first embodiment, the learned model 211 simulates the workflow of a person comparing the content of a document, using tools to collect information required for comparison, and making meaningful inferences based on the collected information to obtain the processing result. In this embodiment, the comparison processing 2223 also mimics a framework called ReAct and outputs the processing result.

[0169] In this embodiment, the tools used in the learned model 211 during the comparison process 2223 include a task segmentation tool 2224, a specification search tool 2226, a standard search tool 2229, and a code execution tool 2232. The task segmentation tool 2224 is implemented by the conformity determination unit 205, the specification search tool 2226 and the standard search tool 2229 are implemented by the information search unit 207, and the code execution tool 2232 is implemented by the code execution unit 208. The task segmentation tool 2224, specification search tool 2226, standard search tool 2229, and code execution tool 2232 are the same as those in the first embodiment, and therefore their descriptions are omitted.

[0170] like Figure 25 As shown, the information processing device 2100 performs extraction processing 2501 on the specification 2201 to extract content that needs to be confirmed to meet the standard, and generates a comparison task (task) 2502. The information processing device 2100 performs extraction processing 2503 on the specification 2201 to extract attribute information representing features, and generates attribute information 2504 related to the specification 2201.

[0171] Furthermore, the information processing device 2100 performs extraction processing 2505 on the standard document 2202 to extract information representing the characteristics of the standard, thereby constructing a rule index 2506 for the standard document. In the rule index 2506 of the standard document, information representing the characteristics of the standard (conditional clauses) is associated with the standard document (the rules followed) that records them.

[0172] The information processing device 2100 performs a search process 2507 based on the attribute information 2504 and the rule index 2506 for the standard document, and selects candidate documents 2508 for the standard document to confirm the content of the specification 2201 based on the similarity of features obtained from the search results. Then, for the generated task 2502, the information processing device 2100 uses the learned model 211 to perform a conformity determination process 2509 for each task, determining whether the content recorded in the specification 2201 meets the criteria of the standard document 2202 selected as the candidate document 2508, and outputs the processing result 2240. In addition, if the search results of the standard document in the conformity determination process 2509 require reference to other criteria not included in the candidate document 2508, the search can be performed on other documents (e.g., all standard documents) instead of the selected candidate document 2508.

[0173] For example, when it's difficult to determine a second document based on a first document used for confirmation, consider directly inputting the original text into a search engine for keyword search or embedding search. Keyword search, however, can easily result in noisy search results because it doesn't consider meaning. Furthermore, embedding search aims to yield similar results that fit the semantics / context, but sometimes it's difficult to understand why a particular search result was selected.

[0174] In contrast, in this embodiment, the information processing device 2100 selects a second document for content verification of the first document based on the similarity between the features of the first document obtained from the search results and the features of the second document. This search result uses a rule index related to the second document, constructed using attribute information extracted from the first document and feature information representing the benchmark (rules) extracted from the second document. By selecting the second document used for content verification of the first document in this way, the chances of selecting a second document unrelated to the search results are reduced, and highly relevant second documents can be selected as search results. Furthermore, by using domain-specific language (technical terms) in the prompts provided to the learned model, domain-specific areas can be addressed, enabling professional searches.

[0175] Furthermore, according to the second embodiment, even when it is difficult to determine the second document for content verification based on the first document as the object of verification, it is possible to select a second document that corresponds to the characteristics of the first document, thereby making the verification operation of whether the content of the first document meets the required conditions more efficient.

[0176] Furthermore, in the first and second embodiments described above, the learned model 211 is configured to be stored in the storage unit 210 of the information processing device 100 (2100), but it may also be configured to be equipped with other information processing devices (server devices, etc.) capable of communicating with the information processing device 100 (2100) via a network. In this case, the information processing device 100 (2100) can send input to the learned model 211 to other information processing devices equipped with the learned model 211 via a network I / F 107, etc., and receive processing results for the input.

[0177] In addition, in the first and second embodiments described above, the learned model 211 is given a prompt containing learning samples to learn (Few-shot learning), but the learned model can also be fine-tuned in conjunction with various functions.

[0178] <Other Implementation Methods>

[0179] This invention provides a program that implements one or more functions of the above embodiments to a system or device via a network or various recording media. Furthermore, the program can be read and executed by one or more processors in the computer of the system or device. Alternatively, it can be implemented using a circuit (e.g., an ASIC) that implements one or more functions.

[0180] Furthermore, the above embodiments are merely examples illustrating specific implementations of the present invention, and the scope of the invention should not be limited by them. That is, the present invention can be implemented in various forms without departing from its technical concept or its main features.

Claims

1. An information processing device, characterized in that, have: The unit retrieves the first document; The parsing unit extracts the content to be verified from the first document and generates the task of each of the contents; The search unit searches the second document for conditions related to the content to be confirmed in the task. The processing unit sequentially inputs the tasks generated by the parsing unit into the learned model, so that the learned model answers whether the content to be confirmed in the input task meets the conditions searched from the second document; as well as The output unit outputs the processing result of the processing unit. The parsing unit generates a task for each of the contents by inputting the first document, along with instructions for extracting the content to be verified to generate the task and prompts for providing learning samples to the learned model, into the learned model.

2. The information processing device according to claim 1, characterized in that, The parsing unit has: A generation unit that generates a task based on the content extracted from the first document; as well as A segmentation unit, which, when the task generated by the generation unit contains multiple of the aforementioned contents, divides the task into multiple tasks.

3. The information processing apparatus according to claim 1 or 2, characterized in that, The parsing unit divides the first document into multiple documents and inputs them into the learned model.

4. The information processing apparatus according to claim 3, characterized in that, The parsing unit divides the first document into the plurality of documents according to a predetermined pattern.

5. The information processing apparatus according to any one of claims 1 to 4, characterized in that, The learned model is a large-scale language model.

6. The information processing apparatus according to any one of claims 1 to 4, characterized in that, The learned model is a large-scale multimodal model.

7. The information processing apparatus according to any one of claims 1 to 6, characterized in that, The search unit inputs the content to be confirmed in the task into the learned model to search for conditions related to the content to be confirmed in the task.

8. An information processing method executed by an information processing device, characterized in that, The information processing method includes the following steps: Obtain the process flow and the first document; The parsing process involves extracting the content to be verified from the first document and generating a task for each of the contents; The search process involves searching the second document for conditions related to the content to be verified in the task. The processing step involves sequentially inputting the tasks generated in the parsing step into the learned model, so that the learned model can answer whether the content to be confirmed in the input tasks meets the conditions searched from the second document. as well as The output process outputs the processing results from the aforementioned processing steps. In the parsing process, a task for each of the contents is generated by inputting the first document, along with instructions for extracting the content to be verified to generate the task and prompts for providing learning samples to the learned model, into the learned model.

9. A program, characterized in that, The computer of the information processing device performs the following steps: The steps to obtain the first document are: The parsing step involves extracting the content to be confirmed from the first document and generating a task for each of the contents. The search step involves searching the second document for conditions related to the content to be verified in the task. The processing step involves sequentially inputting the tasks generated in the parsing step into the learned model, so that the learned model can answer whether the content to be confirmed in the input tasks meets the conditions searched from the second document. as well as Output steps: Output the processing results of the processing steps. In the parsing step, a task for each of the contents is generated by inputting the first document, along with instructions for extracting the content to be verified to generate the task and prompts for providing learning samples to the learned model, into the learned model.

10. A computer-readable recording medium containing a program, characterized in that, The computer of the information processing device performs the following steps: The steps to obtain the first document are: The parsing step involves extracting the content to be confirmed from the first document and generating a task for each of the contents. The search step involves searching the second document for conditions related to the content to be verified in the task. The processing step involves sequentially inputting the tasks generated in the parsing step into the learned model, so that the learned model can answer whether the content to be confirmed in the input tasks meets the conditions searched from the second document. as well as Output steps: Output the processing results of the processing steps. In the parsing step, a task for each of the contents is generated by inputting the first document, along with instructions for extracting the content to be verified to generate the task and prompts for providing learning samples to the learned model, into the learned model.

Images