Information processing device, information processing method, and program

Abstract

An information processing device is provided that can improve the efficiency of a confirmation task as to whether the contents of a target document conform to required conditions. [Solution] The information processing device has an acquisition means for acquiring a first document, an analysis means for extracting content to be confirmed from the first document and generating a task for each of the content, a search means for searching a second document for conditions related to the content to be confirmed in the task, a processing means for sequentially inputting the tasks generated by the analysis means into a trained model and causing the trained model to respond as to whether the content to be confirmed in the inputted task complies with the conditions searched from the second document, and an output means for outputting the processing result by the processing means, and the analysis means generates a task for each content by inputting the first document and a prompt including an instruction to extract the content to be confirmed and generate a task, and a learning sample to be given to the trained model, into the trained model.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, documents were compared visually by humans, but in recent years, as a large number of electronic documents have come to be handled, the task has come to be performed efficiently using computers. However, when a large number of documents are checked by humans, human error is likely to occur, and while it is easy to compare documents in a standardized format when comparing electronic documents using a computer, it is not easy to compare documents in an unstandardized format. For example, it was not suitable for use in tasks such as checking whether the contents of specifications, etc., which do not necessarily have a standardized format, comply with standards, etc.

[0003] Patent Document 1 proposes a technology that calculates the semantic identity of each of the multiple regulation contents of an organizational regulation for each standard content of a specification standard, calculates an identity score for each of the regulation contents with the highest identity score, averages the identity scores between each standard content and its linked regulation content to calculate a consistency score between the specification standard and the organizational regulation, and displays to the user the consistency score between each standard content and the regulation content linked to it.

[0004] Patent document 2 proposes a technology that decomposes a query document containing multiple data types into elements of different data types, performs a data type similarity search for the decomposed data type elements, and finds documents similar to the query document. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-86737 A [Patent Document 2] Special Publication No. 2002-537604 Summary of the Invention [Problem to be solved by the invention]

[0006] There is a need for a mechanism that can use an information processing device such as a computer to determine whether the contents of a target document conform to required conditions without being limited by its format. The present invention aims to provide an information processing device, an information processing method, and a program that can efficiently check whether the contents of a target document conform to required conditions. [Means for solving the problem]

[0007] The information processing device according to the present invention includes an acquisition means for acquiring a first document, an analysis means for extracting contents to be confirmed from the first document and generating a task for each of the contents, a search means for searching a second document for conditions related to the contents to be confirmed in the task, and a search means for searching the tasks generated by the analysis means. and conditions regarding the contents to be confirmed in the task searched for by the search means. the first document and a prompt including an instruction to extract the content to be confirmed and generate the task, and a learning sample to be given to the trained model, to generate a task for each of the contents. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that can improve the efficiency of the task of checking whether the contents of a target document conform to required conditions. [Brief description of the drawings]

[0009] [Figure 1]FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device according to the first embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of processing performed by the information processing device according to the first embodiment. [Figure 4] FIG. 13 is a diagram illustrating an example of division of a specification. [Diagram 5] FIG. 10 is a diagram illustrating an example of task decomposition. [Figure 6] FIG. 13 is a diagram illustrating an example of a prompt in task decomposition. [Figure 7] FIG. 10 is a diagram illustrating an example of task decomposition. [Figure 8] FIG. 13 is a diagram illustrating an example of a prompt in task decomposition. [Figure 9] FIG. 13 is a diagram illustrating an example of a prompt for extracting premise information. [Figure 10] FIG. 10 is a diagram illustrating an example of a standards document DB. [Figure 11] FIG. 13 is a diagram illustrating an example of a search using a search tool. [Figure 12] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 13] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 14] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 15] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 16A] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 16B] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 17A] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 17B] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 18] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 19] FIG. 13 is a diagram illustrating an example of matching processing. [Figure 20] FIG. 13 is a diagram illustrating an example of an output of a processing result. [Figure 21]FIG. 11 is a diagram illustrating an example of a functional configuration of an information processing device according to a second embodiment. [Figure 22] FIG. 11 is a diagram illustrating an example of processing by an information processing device according to a second embodiment. [Diagram 23] FIG. 11 is a diagram illustrating an example of attribute information extraction. [Figure 24] FIG. 13 is a diagram illustrating an example of rule index construction. [Diagram 25] FIG. 11 is a diagram illustrating a process according to the second embodiment. [Figure 26A] FIG. 13 is a diagram for explaining a sample example relating to matching processing. [Figure 26B] FIG. 13 is a diagram for explaining a sample example relating to matching processing. [Figure 26C] FIG. 13 is a diagram for explaining a sample example relating to matching processing. [Figure 26D] FIG. 13 is a diagram for explaining a sample example relating to matching processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, when a "document" is referred to, it is assumed that an electronic document (digital document) made up of digital data is also included.

[0011] <First embodiment> 1 is a diagram showing an example of a hardware configuration of an information processing device 100 according to an embodiment of the present invention. The information processing device 100 has a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, an output device 105, an input device 106, and a network I / F 107. The CPU 101, the ROM 102, the RAM 103, the auxiliary storage device 104, the output device 105, the input device 106, and the network I / F 107 are communicatively connected via a system bus 108.

[0012] The CPU (Central Processing Unit) 101 is a central processing unit that controls various operations of the information processing device 100. For example, the CPU 101 may control the operation of the entire information processing device 100. The ROM (Read Only Memory) 102 stores a control program, a boot program, etc. that can be executed by the CPU 101. The RAM (Random Access Memory) 103 is the main storage memory of the CPU 101, and is used as a work area or a temporary storage area for expanding various programs.

[0013] The auxiliary storage device 104 stores various data, various programs, etc. The auxiliary storage device 104 is realized by a storage device capable of temporarily or persistently storing various data, such as a non-volatile memory represented by an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0014] The output device 105 is a device that outputs various types of information, and is used to present various types of information to a user. For example, the output device 105 is realized by a display device such as a display. The output device 105 presents information to a user by displaying various types of display information. As another example, the output device 105 may be realized by an audio output device that outputs sounds such as voice and electronic sounds. In this case, the output device 105 presents information to a user by outputting sounds such as voice and electronic sounds. In addition, the device applied as the output device 105 may be appropriately changed depending on a medium used to present information to a user.

[0015] The input device 106 is used to receive various instructions from a user. For example, the input device 106 includes input devices such as a mouse, a keyboard, and a touch panel. As another example, the input device 106 may include a sound collection device such as a microphone, and collect voice spoken by the user. In this case, various analysis processes such as acoustic analysis and natural language processing are performed on the collected voice, and the content indicated by the voice is recognized as an instruction from the user. Furthermore, the device applied as the input device 106 may be appropriately changed depending on the method for recognizing an instruction from the user. Furthermore, multiple types of devices may be applied as the input device 106.

[0016] The network I / F 107 is used for communication with an external device, etc. via a network. The device applied as the network I / F 107 may be changed as appropriate depending on the type of communication path and the communication method applied.

[0017] The CPU 101 loads a program stored in the ROM 102 or the auxiliary storage device 104 into the RAM 103 and executes the program, thereby implementing various functions and processes of the information processing device described below.

[0018] 2 is a diagram showing an example of a functional configuration of the information processing device 100 in the first embodiment. The information processing device 100 includes an acquisition unit 201, an analysis unit 202, a database (DB) creation unit 203, an information extraction unit 204, a conformity determination unit 205, an output unit 209, and a storage unit 210.

[0019] Here, in the information processing device 100 in this embodiment, a part of the function is realized using a trained model 211 generated by machine learning (deep learning) and stored in the storage unit 210. In the following, the trained model 211 will be described as a large-scale language model (LLM). The large-scale language model (LLM) is a language model constructed using a large amount of text data (such as a large-scale corpus) and deep learning technology, and when text data called a prompt indicating an instruction or command is input, inference is performed based on the prompt, and text data corresponding to the input prompt is generated and output. Note that the trained model 211 is not limited to the large-scale language model (LLM), and for example, a large-scale multimodal model (LMM) capable of handling text data may be applied.

[0020] The acquisition unit 201 acquires documents related to the compatibility check work. The acquisition unit 201 acquires a first document (document to be checked) to be checked for conformity with required conditions, and a second document indicating the conditions to be checked corresponding to the first document. The acquisition unit 201 is an example of an acquisition means. Examples of the first document include specifications for products or services, contracts regarding the contents of contracts, proposals, etc. Examples of the second document include documents such as specifications and regulations indicating rules to be used as standards, and RFPs (Request for Proposal). Note that the examples of the first document and the second document described above are merely examples, and are not limited thereto.

[0021] The analysis unit 202 analyzes the first document acquired by the acquisition unit 201, extracts contents to be confirmed whether or not the contents of the first document conform to required conditions, and generates a matching task (hereinafter, also simply referred to as a "task") for each extracted content. The analysis unit 202 is an example of an analysis means. The analysis unit 202 generates a task related to the confirmation work of the first document by inputting the first document and a prompt including an instruction to extract contents to be confirmed and output a task to the trained model 211. The generated matching task (task) is stored as task data 213 in the storage unit 210. Note that the analysis unit 202 is not limited to generating a matching task (task) using the trained model 211. For example, the analysis unit 202 may extract contents to be confirmed from a document (first document) having a predetermined format by mechanical processing using an algorithm, and generate a matching task (task).

[0022] The DB creation unit 203 creates a database (retriever) related to documents based on the documents acquired by the acquisition unit 201. The database (retriever) created by the DB creation unit 203 is stored as a document DB (retriever) in the storage unit 210. That is, the DB creation unit 203 creates a first document DB (retriever) 215 based on a first document acquired by the acquisition unit 201, and creates a second document DB (retriever) 216 based on a second document acquired by the acquisition unit 201. These document DBs (retriever) 215 and 216 are DBs (retrievers) as search tools used when performing conformity determination regarding the contents of documents using the trained model 211. The document DBs (retriever) 215 and 216 are, for example, full-text search engines, and searches are performed using BM25. The DB creation unit 203 is an example of a creation means.

[0023] The information extraction unit 204 extracts predetermined information from the first document acquired by the acquisition unit 201. The information extraction unit 204 is an example of an information extraction means. In the first embodiment, the information extraction unit 204 extracts premise information from the first document acquired by the acquisition unit 201. Here, the premise information is information common throughout the first document, for example, information related to a standard (rule) referred to in the first document. The information extraction unit 204 inputs the first document and a prompt including an instruction to extract common information throughout the first document to the trained model 211, thereby extracting premise information related to the first document. The extracted premise information is stored in the storage unit 210 as premise information 214, and is given to the trained model 211 as hint information in a conformity determination process related to the contents of the document using the trained model 211.

[0024] The conformance determination unit 205 performs conformance determination on the first document to be confirmed. Based on the task related to the first document stored in the storage unit 210 as task data 213, the conformance determination unit 205 confirms whether the contents of the first document to be confirmed in the task conform to the required conditions. Specifically, the conformance determination unit 205 inputs the task related to the first document and premise information as a prompt to the trained model 211 to determine whether the contents of the first document indicated in the task conform to the conditions indicated in the second document. In addition, when the task to be verified in the conformance determination is not one task that can be verified by the matching processing unit 206 described later, but is configured to include multiple tasks, i.e., multiple contents, the conformance determination unit 205 divides the task into verifiable tasks. The conformance determination unit 205 has a matching processing unit 206, an information search unit 207, and a code execution unit 208.

[0025] The matching processor 206 performs matching processing between the first document and the second document, and determines whether the contents of the first document meet the required conditions. The matching processor 206 is an example of a processing means. The matching processor 206 inputs a prompt including a task and premise information related to the first document to the trained model 211, thereby inferring whether the contents of the first document indicated in the task meets the conditions indicated in the second document. In addition, the matching processor 206 outputs a search request, a code (program) for performing calculation processing, etc., generated by inputting the task and premise information related to the first document as a prompt to the trained model 211, to the information search unit 207 and the code execution unit 208.

[0026] The information search unit 207 searches the first document and the second document in response to a search request from the matching processing unit 206, and outputs the search result to the matching processing unit 206. The information search unit 207 is an example of a search means. The information search unit 207 receives a search query input from the matching processing unit 206, performs search processing on the first document DB 215 and the second document DB 216 using the trained model 211, and returns the search result according to the input search query to the matching processing unit 206. Note that the information search unit 207 is not limited to performing search processing using the trained model 211 and outputting the search result. For example, the information search unit 207 may receive a search query input from the matching processing unit 206, perform rule-based search processing on the first document DB 215 and the second document DB 216, and return the acquired search result to the matching processing unit 206. For example, in the search processing shown in FIG. 11, which will be described later, the search results 1106 obtained by the search processing 1104 may be output as the final search results 1108 as they are.

[0027] The code execution unit 208 executes the code (program) and outputs the execution result. The code execution unit 208 executes the code (program) input from the matching unit 206 and returns the execution result to the matching unit 206.

[0028] Output unit 209 outputs the result of the conformance determination made by conformance determination unit 205 regarding the first document, i.e., the determination result as to whether the content of the first document extracted as a task related to the first document conforms to the required conditions. Output unit 209 may output the reason for the determination result together with the determination result as to whether the content of the first document conforms to the required conditions.

[0029] The storage unit 210 stores various data and the like used when performing processing in the information processing device 100. The storage unit 210 stores, for example, a trained model 211, a setting file 212, task data 213, premise information 214, a first document DB 215, and a second document DB 216. The trained model 211 is a model on which machine learning (deep learning) has been performed, which is used for various inferences performed in the information processing device 100. The setting file 212 is a file in which a prompt to be input to the trained model 211 and the like are stored. The setting file 212 may also store a training sample to be input as a prompt. A plurality of training samples may be prepared in the setting file 212 and the like, and selected according to a function to be realized and provided to the trained model 211. The task data 213 is a task related to a confirmation work generated by the analysis unit 202, and the premise information 214 is premise information extracted by the information extraction unit 204. The first document DB 215 is a DB for the first document created by the DB creation unit 203, and the second document DB 216 is a DB for the second document created by the DB creation unit 203.

[0030] Processing in the information processing device 100 in the first embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining an example of processing in the information processing device 100 in the first embodiment. In the following, as an example, a case will be described in which a first document to be checked is a product specification 301, a second document indicating the required conditions to be checked is a standard document 302 relating to the product standard, and whether or not the contents described in the specification document 301 conform to the standard of the standard document 302 is checked.

[0031] The information processing device 100 acquires a specification 301 as a first document, and also acquires a standards document 302 as a second document.

[0032] (Process 303, 305: Division of specifications and task decomposition) The information processing device 100 performs a specification division process 303 on the acquired specification 301, and divides the entire specification 301 into a plurality of documents 304 that become parts of the specification. In the specification division process 303, the information processing device 100 divides the specification 301 into a plurality of documents by mechanical processing using an algorithm so that each document after division has a size that can be processed by the trained model 211. For example, the information processing device 100 divides the text data of the specification 301 read by an OCR or a PDF parser or the like in a predetermined pattern based on numerical bullet points or the like, and divides the specification 301 into a plurality of documents 304. In the example of dividing a specification shown in FIG. 4, the information processing device 100 divides the specification 401 into text data 402 to 405 for each item number (item) based on numerical bullet points. Tables in the specification are converted into data in CSV format, for example.

[0033] Furthermore, the information processing device 100 performs a task decomposition process 305 on the multiple documents 304 after division, and generates a matching task (task) 306 related to checking the contents of the specification 301. In the task decomposition process 305, the information processing device 100 generates a task 306 related to checking the contents of the specification 301 by inputting the divided documents 304 and a prompt including an instruction to extract contents that need to be checked for compliance with the standard and output a task to the trained model 211.

[0034] An example of task decomposition in the task decomposition process 305 will be described with reference to FIGS. An example of generating a task for checking quantitative matters will be described with reference to Figs. 5 and 6. Fig. 5(A) shows a description in a specification 301, in which the values ​​of carbon (C), manganese (Mn), silicon (Si), and phosphorus (P) are described for each product as chemical components in the product. Data 304 after division of the description shown in Fig. 5(A) is the data shown in Fig. 5(B). The information processing device 100 inputs the data shown in Fig. 5(B) and the prompt 600 shown in Fig. 6 to the trained model 211, thereby breaking down the contents that need to be confirmed into tasks, and generates the tasks as shown in Fig. 5(C).

[0035] As shown in FIG. 6, the prompt 600 includes a prompt 601 that instructs the user to comprehensively output values ​​that are related to the standard and therefore need to check whether the description of the standard matches the described values, and break down the items to be checked into as small tasks as possible that can be verified. The prompt 600 may also include a learning sample 602 for tabular data and a learning sample 603 for when there is no information to be extracted. By using such a prompt, as shown in FIG. 5(C), it is possible to control the granularity of the information (task) to be extracted, break it down into small tasks that can be verified by a matching process described later, and generate a task that checks whether the value described in the specification 301 complies with the standard. In addition, by including a learning sample such as the one shown in the example in the prompt 600, it is possible to obtain the output of the task in a desired format by utilizing the characteristics of a large-scale language model that has a high accuracy of returning an output adapted to the learning sample.

[0036] Next, an example of generating a task (matching task) for checking qualitative matters will be described with reference to FIG. 7 and FIG. 8. FIG. 7(A) shows the description in the specification 301, which describes the quality control standard for the product, product identification and traceability, and surface finishing. The information processing device 100 inputs the divided data for the quality control standard shown in FIG. 7(A) and the prompt 800 shown in FIG. 8 to the trained model 211, thereby decomposing the contents to be checked into tasks, and generates a task (matching task) as shown in FIG. 7(B). Similarly, the trained model 211 inputs the divided data for the product identification and traceability shown in FIG. 7(A) and the prompt 800 shown in FIG. 8 to generate a task (matching task) as shown in FIG. 7(C), and inputs the divided data for the surface finishing shown in FIG. 7(A) and the prompt 800 shown in FIG. 8 to generate a task (matching task) as shown in FIG. 7(D).

[0037] As shown in FIG. 8, the prompt 800 includes a prompt 801 that outputs the contents that are related to the standard and therefore need to be confirmed to match the description of the standard, and instructs breaking down the items to be checked into as small tasks that can be verified as much as possible. The prompt 800 may also include learning samples 802 and 803 related to the extraction of qualitative items, learning samples when there is no information to be extracted, and the like. By using such a prompt, as shown in FIG. 7(B) to FIG. 7(D), it is possible to control the granularity of the information (task) to be extracted, break it down into small tasks that can be verified by a matching process described later, and generate a task that checks whether the contents described in the specification 301 conform to the standard. In addition, by including a learning sample such as the one shown as an example in the prompt 800, it is possible to obtain the output of the task in a desired format by utilizing the characteristics of a large-scale language model that has a high accuracy of returning an output adapted to the learning sample.

[0038] (Process 307: Extract background information) The information processing device 100 performs premise information extraction processing 307 on the acquired specification 301, and extracts premise information 308, which is common information, through the specification 301. In the premise information extraction processing 307, the information processing device 100 inputs the specification 301 and a prompt including an instruction to extract common information through the specification to the trained model 211, thereby generating premise information 308 related to the specification 301. In this example, as shown in FIG. 9 , by inputting a prompt 900 instructing to extract the standard and product type specified in the specification from the specification 301, information on the standard referenced in the specification and the product type described in the specification are extracted as premise information 308.

[0039] (Process 309, 311: DB creation) The information processing device 100 performs DB creation processing 309 for specifications based on the acquired specifications 301, and creates a specification DB (retriever) 310. The information processing device 100 also performs DB creation processing 311 for standards documents based on the acquired standards document 302, and creates a standards document DB (retriever) 312. The specification DB (retriever) 310 corresponds to the first document DB (retriever) 215 shown in Fig. 2, and the standards document DB (retriever) 312 corresponds to the second document DB (retriever) 216 shown in Fig. 2. In the DB creation processing 309, 311, the information processing device 100 processes the specifications 301 and the standards document 302 into a searchable state by mechanical processing using an algorithm, and creates a database.

[0040] For example, as shown in FIG. 10, the information processing device 100 creates a record including information on a file name such as "IronStandard2023", a document name such as "Steel Plate Standard", a text such as "1. Introduction This document is...", and a page number such as "p1" for each page of the standard document 302 to configure the standard document DB (retriever) 312. For example, by including a page number in the record, it becomes possible to refer to a different page such as the next page or the previous page when performing a search using the trained model 211 described later. The specification DB (retriever) 310 may be configured in the same manner. Note that the information held by each record is not limited to the above-mentioned example, and other information may be included. In addition, a record is created for each page of the document, but this is an example, and a record may be created for each other structural unit.

[0041] Here, the standards document DB (retriever) 312 may be created in advance, or may be created when checking whether the contents of the specification 301 conform to the standard. For example, if a standards document DB (retriever) 312 for standards documents that are used repeatedly or over a long period of time is created and stored, it is not necessary to create it each time, and processing can be reduced. In addition, for standards documents that cannot be permanently stored, the occurrence of information leakage, etc. can be suppressed by creating the standards document DB (retriever) 312 when checking whether the contents of the specification 301 conform to the standard.

[0042] (Process 320: Compliance determination) For the tasks generated in the task decomposition process 305, the information processing device 100 performs conformance determination process 320 for each task using the trained model 211 to determine whether or not the contents described in the specification 301 conform to the standards in the standard document 302. In the task list 321, the tasks 306 generated by the task decomposition process 305 are stored in a queue format. In the conformance determination process 320, the information processing device 100 performs matching process 323 using the trained model 211 for tasks 322 taken out one by one in order from the task list 321, and outputs a processing result 340. The matching process 323 is performed until there are no more tasks in the task list 321.

[0043] In the matching process 323, the trained model 211 is made to simulate a human workflow when matching document contents, a tool is used to collect information necessary for matching, and a processing result is obtained by performing semantic inference based on the collected information, etc. In addition, when making the inference, a small number or a large number of workflow samples of a specific domain are trained (few-shot learning, etc.) in the trained model 211 to adapt it to the workflow domain.

[0044] The matching process 323 in this embodiment performs processing according to a framework called ReAct (Reasoning and Acting) and outputs the processing result. That is, in the matching process 323, following the ReAct framework, text is generated for each phase of Thought, Action, and Observation for a Question, and a Final Answer is derived by going through this Thought → Action → Observation flow once or multiple times.

[0045] In this example, Task 322 corresponds to Question. In Thought, a plan from the current process to achieve the goal is thought up using the trained model 211, and in Action, a tool for executing the plan is selected using the trained model 211. Then, input is made to execute the tool, and the execution result of the executed tool is observed in Observation. Based on the execution result of the tool obtained in Observation, a plan is thought up in Thought, and if there is a plan, it proceeds to Action, and if a conclusion has been reached, a Final Answer is generated.

[0046] In this embodiment, the tools used by the trained model 211 in the matching process 323 include a task division tool 324, a specification search tool 326, a standards search tool 329, and a code execution tool 332. The task division tool 324 is realized by the conformance determination unit 205, the specification search tool 326 and the standards search tool 329 are realized by the information search unit 207, and the code execution tool 332 is realized by the code execution unit 208.

[0047] The task division tool 324 has a function of adding the tasks 325 divided by the matching process 323 to the task list 321. As will be described later, if the task 322 taken out of the task list 321 and input to the matching process 323 is not a single verifiable task but is composed of multiple tasks, i.e., multiple contents, the matching process 323 divides the task 322 into verifiable tasks 325. The task division tool 324 adds these multiple divided tasks to the task list 321.

[0048] The specification search tool 326 has a function of searching the specification 301 for detailed information required for verifying the task 322. The specification search tool 326 accepts a specification search query 327, searches the specification DB (Retriever) 310, and returns the information obtained by the search as specification information 328. The specification search tool 326 also has a function of extracting and summarizing relevant parts from the search results of the specification DB (Retriever) 310, and a function of adding an explanation of the extracted contents. This function of summarizing the search results makes it possible to reduce the number of characters so that the search results can be given to the trained model 211 as a prompt, for example.

[0049] The standards search tool 329 has a function of searching the standards document 302 for details of standards corresponding to the contents described in the specification 301. The standards document search tool 329 accepts a search query 330 for standards documents, searches the standards document DB (Retriever) 312, and returns the information obtained by the search as standards information 331. The standards document search tool 329 also has a function of extracting and summarizing relevant parts from the search results of the standards document DB (Retriever) 312, and a function of adding an explanation of the extracted contents. This function of summarizing the search results makes it possible, for example, to reduce the number of characters so that the search results can be given to the trained model 211 as a prompt.

[0050] The code execution tool 332 has a function of executing code (program) generated by the trained model 211 for performing calculations such as numerical comparison (including unit conversion and arithmetic operations). The code execution tool 332 accepts code (program) 333 generated by the trained model 211, executes the code (program), and returns an execution result 334.

[0051] Here, a search using the search tools 326 and 329 will be described. A search using the standards search tool 329 will be described below as an example, but the search using the specifications search tool 326 is similar. FIG. 11 is a diagram for explaining an example of a search using the standards search tool 329.

[0052] When searching for standards, the information processing device 100 generates a search intention 1101 and a search query 1103 based on the task to be confirmed and the premise information. In the example shown in Fig. 11, the information processing device 100 generates a search intention 1101 of "I want to know the fatigue strength of IS360B-SSB" and a search query 1103 including "IronStandard2023", "steel plate", "IS360B-SSB", and "fatigue strength".

[0053] Next, the information processing device 100 executes a search process 1104 using the generated search query 1103. In the search process 1104, a rule-based search process is performed on standard document data 1105 using the search query 1103, and a search result 1106 corresponding to the search query 1103 is output.

[0054] Here, in the matching process described later, the search results by the search tools 326 and 329 are added to the prompt and used for inference by the trained model 211. For example, when information on a specific row is required in a table spanning multiple rows, if there is no explanation and only the relevant part is extracted, the header information and the like are lost and only information listing words is extracted. Even if only such information is input to the trained model 211 as the search result, it is difficult for the trained model 211 to interpret it, so that it may lead to incorrect inferences or plans regarding the matching process, and the inference may be poorly planned. This is unlikely to occur in information extraction from a document containing only normal text, but it may occur in information extraction from documents with layouts such as specifications and standard documents because charts and the like are frequently used.

[0055] Therefore, in this embodiment, the information processing device 100 executes a search result summarization process 1107 for summarizing the search results 1106 obtained by the search process 1104 based on the search intention 1101, and outputs the processing result as a final search result 1108. The search result summarization process 1107 is performed using the trained model 211. In the search result summarization process 1107, for the search results 1106 obtained by the search process 1104, the relevant part (part related to the search intention 1101) is extracted and an explanation of the extracted content is provided. For example, when target information is obtained with respect to the search intention 1101, the relevant part and explanation are output, when the target information is not obtained but reference information such as a table to be referenced is obtained, the information and explanation are output, and when the target information is not obtained, "NO_OUTPUT" and an explanation are output. Note that the explanation is generated, for example, taking into consideration sentences or the like described on one or more pages from which the information is obtained. For example, in the example shown in FIG. 11, the first search result 1106 includes information regarding the fatigue strength of IS360A, IS360B, and IS340, but the information processing device 100 extracts only information regarding the fatigue strength of IS360B from the first search result 1106 in accordance with the search intent 1101 and outputs it as the final search result 1108 together with an explanation.

[0056] In this way, by executing search result summary process 1107 on search results 1106 from search tools 326, 329 and outputting final search results 1108, the search results and their explanatory information can be provided to the trained model 211 in the matching process, making it possible to input appropriate information into the trained model 211, thereby improving the accuracy of inference and planning.

[0057] An example of matching processing will be described below. In the inference process of matching processing shown in Fig. 12 to Fig. 19, Question is a task to be confirmed (input information for matching processing), Thought, Action, Action Input, and Final Answer are inferences based on the trained model 211, and Observation is an execution result by each tool.

[0058] As described above, in this embodiment, when making inference in matching processing, the trained model 211 is trained (few-shot training, etc.) with a sample of a workflow related to matching processing, and adapted to the domain related to matching processing. For example, a prompt including a sample as shown in FIG. 26A to FIG. 26D is given to the trained model 211 to make learning. When making inference in matching processing, the elements 2600A to 2600D shown in FIG. 26A to FIG. 26D are combined, and a task is added to the end as a question to make inference. In FIG. 26A, 2601 is a rule related to matching processing, and 2602 is a sample related to an example of searching and matching standards and an example of matching involving numerical comparison. In FIG. 26B, 2603 is a sample related to an example of searching and matching standards and specifications and an example of matching involving numerical comparison, and in FIG. 26C, 2604 is a sample related to an example of matching using appropriate information in a multi-stage search. 26D, 2606 is a sample relating to an example of handling when a search fails, and 2607 is a sample relating to an example of dividing a task consisting of multiple tasks into single tasks. By providing a prompt including such a sample to the trained model 211, processing related to the task can be executed in an appropriate flow according to the task.

[0059] Fig. 12 is a diagram for explaining an example of matching processing. Fig. 12 shows an inference process for an example of searching for standards and performing matching processing to confirm quantitative items. In the example shown in Fig. 12, as shown in Question, a task 322 is input to confirm "Does the description of the applicable size "10m" of the product model number IS380A(-SSP) meet the restrictions of the Iron Standard?" In addition, information on the standard candidates and the type of steel material extracted as premise information 308 from the specification 301 is input as additional information.

[0060] By inputting information including the task and additional information described in the Question as a prompt into the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as a prompt. Through this inference, it plans to investigate the applicable size of the relevant product in the standard document 302 (Thought), selects the standard search tool 329 (SearchStandard) as a tool to execute this (Action), and generates information (arguments) to be input into the standard search tool 329 (Action Input). Here, standard_no, title, and sentence are information to be input into the search tool as a search query, and question is information to be input into the search tool as the search intent.

[0061] The standards search tool 329 searches the standards document DB 312 based on the information written in the Action Input, and the search results by the standards search tool 329, which are the execution results of the tool, are returned to the matching processing unit 206 (Observation). The standards search tool 329 performs summary processing of the search results as described above, and outputs the final search results by the standards search tool 329, which are composed of, for example, related parts extracted from the search results for the standards document 302 and an explanation of the extracted contents (the same applies to other inference processes).

[0062] Next, by inputting the execution result of the tool described in Observation (search result of the standard document) as an additional prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Through this inference, it plans (Thought) to verify with the code (program) whether the applicable size limit of the relevant product inferred from the description of the standard document 302 is satisfied, selects the code execution tool 332 (RunCode) as the tool to execute it (Action), and generates code to be executed by the code execution tool 332 (Action Input).

[0063] The code (program) written in the Action Input is executed by the code execution tool 332, and the execution result (True) by the code execution tool 332, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool (execution result of the code) written in the Observation as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. From this inference, it is determined from the verification result by the code that the applicable size of the product model number IS380A satisfies the applicable size restriction estimated from the description in the standard document 302 (Thought), and an answer indicating that various information related to the confirmation and the confirmation result are OK (compliant) is generated as a matching processing result (Final Answer).

[0064] Fig. 13 is a diagram for explaining an example of matching processing. Fig. 13 shows an inference process for an example of searching for standards and performing matching processing to confirm quantitative items. In the example shown in Fig. 13, as shown in Question, matching processing is performed for task 322, which confirms, "Does the description of the applicable size "9m" of the product model number IS360B(-SSS) meet the restrictions of the Iron Standard?"

[0065] 13, in the same manner as in the example shown in FIG. 12, inference is performed using the trained model 211, processing is performed using the standards search tool 329 and the code execution tool 332, and the execution result (False) by the code execution tool 332 is returned to the matching processing unit 206. By additionally inputting the execution result by the code execution tool 332 as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. From this inference, it is determined from the verification result by the code that the applicable size of the product model number IS360B does not satisfy the applicable size restriction estimated from the description in the standards document 302 (Thought), and various information regarding the confirmation and an answer indicating that the confirmation result is NG (not compliant) are generated as a matching processing result (Final Answer).

[0066] Fig. 14 is a diagram for explaining an example of matching processing. Fig. 14 shows an inference process for an example of searching for standards and performing matching processing to confirm qualitative matters. In the example shown in Fig. 14, as shown in Question, a task 322 is input to confirm "Is the surface pretreatment in the standard a pickling process?". In addition, information on standard candidates and steel type extracted as premise information 308 from the specification 301 is input as additional information.

[0067] By inputting information including the task and additional information described in the Question as a prompt to the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Through this inference, it plans to investigate whether the surface pretreatment in the standard document 302 is a pickling process (Thought), selects the standard search tool 329 (SearchStandard) as a tool to execute this (Action), and generates information (arguments) to be input to the standard search tool 329 (Action Input).

[0068] The standards search tool 329 searches the standards document DB 312 based on the information described in the Action Input, and the search results by the standards search tool 329, which are the execution results of the tool, are returned to the matching processing unit 206 (Observation). By additionally inputting the execution results of the tool described in Observation (search results of the standards document) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Through this inference, it is determined that the surface pretreatment inferred from the description in the standards document 302 is pickling processing and conforms to the standards (Thought), and an answer indicating that various information related to the confirmation and the confirmation result are OK (conforms) is generated as a matching processing result (Final Answer).

[0069] Fig. 15 is a diagram for explaining an example of matching processing. Fig. 15 shows an inference process for an example of searching standards and performing matching processing to confirm qualitative matters. In the example shown in Fig. 15, matching processing is performed for task 322, which confirms, as shown in the Question, "Is the final finish of the surface according to the standard a brush finish?"

[0070] 15, in the same manner as in the example shown in FIG. 14, inference is performed using the trained model 211, processing is performed using the standards search tool 329, and the execution result (search result) of the standards search tool 329 is returned to the matching processing unit 206. By additionally inputting the execution result of the standards search tool 329 as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Through this inference, it is determined that the final finish of the surface inferred from the description in the standards document 302 is not a brush finish and does not conform to the standard (Thought), and various information regarding the confirmation and an answer indicating that the confirmation result is NG (not conforming) are generated as a matching processing result (Final Answer).

[0071] 16A and 16B are diagrams for explaining an example of matching processing. FIGS. 16A and 16B show an inference process for an example of searching standards and specifications for quantitative matters and performing matching processing to confirm the results. In this example, as shown in Question in FIG. 16A, a task 322 is input to confirm, "Does the description of the fatigue strength of IS380A(-SSP) "280≦F≦290" satisfy the IronStandard limit?" In addition, information on the standard candidates and the type of steel material extracted as premise information 308 from the specification 301 is input as additional information.

[0072] By inputting information including the tasks and additional information described in the Questions as prompts into the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as prompts. Through this inference, it plans to investigate the fatigue strength of the relevant product in the standard document 302 (Thought), selects the standard search tool 329 (SearchStandard) as a tool to execute this (Action), and generates information (arguments) to be input into the standard search tool 329 (Action Input).

[0073] The standards search tool 329 searches the standards document DB 312 based on the information described in the Action Input, and the search results by the standards search tool 329, which are the execution results of the tool, are returned to the matching processing unit 206 (Observation). By additionally inputting the execution results of the tool described in Observation (search results of the standards document) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Because this inference indicates that whether or not the fatigue strength of the relevant product, which is estimated from the description in the standards document 302, is satisfied depends on the length of the product, it is planned to check the length of the product from the specification 301 (Thought), the specification search tool 326 (SearchDesignDocument) is selected as the tool to execute this (Action), and information (arguments) to be input to the specification search tool 326 is generated (Action Input).

[0074] The specification DB 310 is searched by the specification search tool 326 based on the information described in the Action Input, and as shown in Fig. 16B, the search result by the specification search tool 326, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool described in Observation (specification search result) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Through this inference, it is planned (Thought) to verify with code (program) whether the description of fatigue strength in the specification 301 satisfies the limit of fatigue strength of the corresponding product inferred from the description in the standard document 302, and the code execution tool 332 (RunCode) is selected as the tool to execute it (Action), and code to be executed by the code execution tool 332 is generated (Action Input).

[0075] The code (program) described in the Action Input is executed by the code execution tool 332, and the execution result (True) by the code execution tool 332, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool (execution result of the code) described in the Observation as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. From this inference, it is determined from the verification result by the code that the fatigue strength of IS380A satisfies the fatigue strength limit estimated from the description in the standard document 302 (Thought), and an answer indicating that various information related to the confirmation and the confirmation result are OK (compliant) is generated as a matching processing result (Final Answer). In the example shown in FIG. 16A and FIG. 16B, a task for checking quantitative items is exemplified, but the same processing is performed for a task for checking qualitative items.

[0076] 17A and 17B are diagrams for explaining an example of matching processing. In FIG. 17A and FIG. 17B, an inference process is shown for an example of performing matching processing and checking quantitative matters using appropriate information in a multi-stage search. In this example, as shown in Question in FIG. 17A, a task 322 is input to check "Does the carbon value "0.20%" in the chemical composition of IS380A(-SSP) meet the IronStandard limit?". In addition, information on the standard candidates and the type of steel material extracted as premise information 308 from the specification 301 is input as additional information.

[0077] By inputting information including the tasks and additional information described in the Questions as prompts to the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as prompts. Through this inference, the information processing device 100 plans to check the carbon value in the chemical components of the relevant product in the standard document 302 (Thought), selects the standard search tool 329 (SearchStandard) as a tool to execute this (Action), and generates information (arguments) to be input to the standard search tool 329 (Action Input).

[0078] The standard search tool 329 searches the standard document DB 312 based on the information described in the Action Input, and the search result by the standard search tool 329, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool described in Observation (search result of the standard document) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. Although the carbon value in the chemical composition of the corresponding product was not directly obtained from the standard document 302 by this inference, it is inferred from the description in the standard document 302 that the chemical composition is described in Table 2, so it is planned to search for the chemical composition in Table 2 of the standard document 302 (Thought), the standard search tool 329 (SearchStandard) is selected as a tool to execute it (Action), and information (arguments) to be input to the standard search tool 329 is generated (Action Input).

[0079] The specifications DB 310 is searched by the standards search tool 329 based on the information described in the Action Input, and as shown in Fig. 17B, the search result by the standards document search tool 329, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool described in Observation (search result of the standards document) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. With this inference, it is planned (Thought) to verify with code (program) whether the carbon value limit in the chemical composition of the relevant product inferred from the description in the standards document 302 is satisfied, the code execution tool 332 (RunCode) is selected as the tool to execute it (Action), and code to be executed by the code execution tool 332 is generated (Action Input).

[0080] The code (program) described in the Action Input is executed by the code execution tool 332, and the execution result (True) by the code execution tool 332, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool described in Observation (execution result of the code) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. From this inference, it is determined from the verification result by the code that the carbon value in the chemical composition of IS380A satisfies the restriction on the carbon value in the chemical composition estimated from the description in the standard document 302 (Thought), and an answer indicating that various information related to the confirmation and the confirmation result are OK (compliant) is generated as a matching processing result (Final Answer). In the example shown in FIG. 17A and FIG. 17B, a task for checking quantitative items is exemplified, but the same processing is performed for a task for checking qualitative items.

[0081] Fig. 18 is a diagram for explaining an example of matching processing. Fig. 18 shows an inference process for an example in which an input task including multiple tasks is decomposed into single tasks. In the example shown in Fig. 18, as shown in Question, a task 322 is input to confirm whether "the fatigue strengths of IS380A(-SSA) and IS380B(-SSS), '270≦F≦280' and '260≦F≦308', satisfy the limits of the Iron Standard." In addition, information on the candidate standards and the type of steel extracted as premise information 308 from the specification 301 is input as additional information.

[0082] By inputting information including the tasks and additional information described in the Questions as prompts into the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as prompts. Because this inference requires task decomposition, the information processing device 100 plans to decompose the task and add it to the task list (Thought), selects the task division tool 324 (AddTask) as a tool to execute this (Action), and generates information to be input to the task division tool 324 (Action Input).

[0083] Based on the information written in the Action Input, the task division tool 324 adds each of the tasks after decomposition to the task list 321, and the processing result by the task division tool 324, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). For example, the task of checking "whether the fatigue strengths "270≦F≦280" and "260≦F≦308" of IS380A(-SSA) and IS380B(-SSS) satisfy the limits of the Iron Standard" shown in this example is decomposed into tasks of checking "whether the fatigue strength "270≦F≦280" of IS380A(-SSA) satisfies the limits of the Iron Standard" and "whether the fatigue strength "260≦F≦308" of IS380B(-SSS) satisfies the limits of the Iron Standard", respectively, and added to the task list 321.

[0084] By inputting the execution result of the tool described in Observation as an additional prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. This inference determines that the addition of the decomposed task to the task list 321 by the task division tool 324 has been completed normally (Thought), and generates an answer indicating skip as a matching process result (Final Answer).

[0085] Fig. 19 is a diagram for explaining an example of matching processing. Fig. 19 shows an inference process for an example of processing to suppress hallucination when a search fails. In the example shown in Fig. 19, as shown in Question, a task 322 is input to confirm whether the electrical resistance of IS380A(-SSA) "1.0 x 10^-7 or more and 1.5 x 10^-7 Ω·m or less" satisfies the restrictions of the Iron Standard. In addition, information on the standard candidates and the type of steel material extracted as premise information 308 from the specification 301 is input as additional information.

[0086] By inputting information including the tasks and additional information described in the Questions as prompts to the trained model 211, the information processing device 100 performs inference using the trained model 211 based on the information input as prompts. Through this inference, it plans to check the electrical resistance of the relevant product in the standard document 302 (Thought), selects the standard search tool 329 (SearchStandard) as a tool to execute this (Action), and generates information (arguments) to be input to the standard search tool 329 (Action Input).

[0087] The standard search tool 329 searches the standard document DB 312 based on the information described in the Action Input, and the search result by the standard search tool 329, which is the execution result of the tool, is returned to the matching processing unit 206 (Observation). By additionally inputting the execution result of the tool described in Observation (search result of the standard document) as a prompt, the information processing device 100 performs inference using the trained model 211 based on the information input as the prompt. By this inference, it is determined that information on the electrical resistance of the corresponding product cannot be obtained from the standard document 302, and it is not possible to confirm whether the description of the electrical resistance in the specification 301 satisfies the restrictions of the standard (Thought), and an answer indicating that various information related to the confirmation and the confirmation result are NG (not conforming) is generated as a result of the matching process (Final Answer). In this way, when the target information cannot be obtained, "NO_OUTPUT" is output, and by explicitly indicating that the target information could not be obtained, the occurrence of hallucination can be suppressed.

[0088] Fig. 20(A) and Fig. 20(B) are diagrams showing examples of output of processing results in this embodiment. The processing results in the information processing device 100 described above are displayed as a list of verification contents and verification results for each verified task, for example, as in list 2001 shown in Fig. 20(A) and list 2011 shown in Fig. 20(B). In list 2001 shown in Fig. 20(A) and list 2011 shown in Fig. 20(B), all tasks that have been verified are displayed, but it is also possible to display only tasks for which the verification result is NG (it was not determined that the description in the specification complies with the standard).

[0089] In addition, when a task with a search result of NG is selected from among the tasks displayed in the list by a user operation on the input device 106, the reason why the verification result is NG may be displayed, for example, as in detailed report 2002 shown in FIG. 20(A) or detailed report 2012 shown in FIG. 20(B). In this example, detailed reports 2002 and 2012 are composed of a problem part and a verified process part, and can be generated based on an inference process in a matching process. Note that a GUI such as a button for instructing the display of a detailed report for a task with a search result of NG may be provided, and the detailed report may be displayed in response to an operation on the GUI.

[0090] A detailed report 2002 shown in Fig. 20(A) is a detailed report on a task whose inference process is shown in Fig. 13, and a detailed report 2012 shown in Fig. 20(B) is a detailed report on a task whose inference process is shown in Fig. 19. For example, in the detailed report, the problem part is generated based on the Final Answer in the inference process, and the verified process part is generated based on the Thought, Action Input, Observation, etc. in the inference process. Also, in the verified process part, for example, a query is generated based on the Action Input in the inference process, a search result is generated based on the Observation in the inference process, and a consideration is generated based on the Thought in the inference process.

[0091] In the above example, the list of results and the detailed report showing the reason why the verification result was NG are displayed separately, but the list of results and the detailed report may be displayed together. Also, the information shown in the detailed report is not limited to the above example, and may show information about the tools used when performing the verification.

[0092] According to the first embodiment, the information processing device 100 extracts contents to be checked from a first document (e.g., a specification), generates a task for each content, sequentially inputs the generated tasks to the trained model 211, and causes the trained model 211 to respond as to whether or not the contents to be checked in the tasks conform to conditions retrieved from a second document (e.g., a standard document), and outputs the results as a processing result. This makes it possible to efficiently check whether or not the contents of the first document to be checked conform to the required conditions indicated in the second document.

[0093] Generally, hallucination occurs in generation that requires knowledge outside the domain that has not been learned in the trained model. In this embodiment, however, when performing confirmation, information is searched for using an external tool (search tools 326, 329), and the obtained search results are added to the prompt and given to the trained model that performs matching processing, thereby making it possible to prevent the occurrence of hallucination.

[0094] <Second embodiment> In the first embodiment, an example was shown in which a second document indicating the conditions to be confirmed can be identified from a first document to be confirmed, but there are cases in which it is difficult to identify a second document from a first document. In the second embodiment, an example in which it is difficult to identify a second document indicating the conditions to be confirmed from a first document to be confirmed will be described. The hardware configuration of the information processing device 2100 in the second embodiment is similar to the hardware configuration of the information processing device 100 in the first embodiment shown in FIG. 1, and therefore a description thereof will be omitted.

[0095] Fig. 21 is a diagram showing an example of a functional configuration of an information processing device 2100 in the second embodiment. In Fig. 21, components having the same functions as those shown in Fig. 2 are given the same reference numerals, and duplicated explanations will be omitted. The information processing device 2100 has an acquisition unit 201, an analysis unit 202, a DB creation unit 203, an information extraction unit 204, a conformance determination unit 205, an output unit 209, a storage unit 210, and a candidate selection unit 2101. In addition, the conformance determination unit 205 has a matching processing unit 206, an information search unit 207, and a code execution unit 208.

[0096] In the information processing device 2100, like the information processing device 100 in the first embodiment, a part of the functions is realized using a trained model 211 that is generated by machine learning (deep learning) and stored in the storage unit 210. The trained model 211 is, for example, a large-scale language model (LLM) or a large-scale multimodal model (LMM) that can handle text data.

[0097] In the second embodiment, the analysis unit 202 generates a matching task (task) related to the confirmation work of the contents of the first document in the same manner as in the first embodiment. In addition, the analysis unit 202 analyzes one or more second documents acquired by the acquisition unit 201, extracts information representing the characteristics of the criteria (rules) described in each second document, and constructs a rule index related to the second document. The analysis unit 202 inputs the second document and a prompt including an instruction to extract and output the information representing the characteristics of the criteria (rules) to the trained model 211, thereby extracting information representing the characteristics of the criteria (rules) from each second document. The extracted information representing the characteristics of the criteria (rules) is linked to the second document from which it was extracted, and is stored in the storage unit 210 as a rule index 2103 of the second document. Note that the second document may be divided into multiple documents in a predetermined pattern based on chapters or the like, and input to the trained model 211.

[0098] In the second embodiment, the information extraction unit 204 extracts attribute information from the first document acquired by the acquisition unit 201. Here, the attribute information is information indicating the characteristics of the first document, for example, information that contributes to identifying a standard (rule) referred to in the first document. The information extraction unit 204 extracts attribute information related to the first document by inputting the first document and a prompt including an instruction to extract information indicating the characteristics of the first document to the trained model 211. The extracted attribute information is stored as attribute information 2102 in the storage unit 210. The information extraction unit 204 divides the first document into a plurality of documents based on, for example, chapters, and inputs the documents to the trained model 211. Therefore, attribute information is extracted for each of the plurality of documents divided based on chapters.

[0099] The candidate selection unit 2101 selects a second document (candidate document) to be used for confirming the contents of the first document from one or more second documents based on the attribute information 2102 extracted from the first document by the information extraction unit 204 and the rule index 2103 of the second document constructed by the analysis unit 202. The candidate selection unit 2101 is an example of a selection means. The candidate selection unit 2101 selects a second document to be used for confirming the contents of the first document based on the similarity between the features of the first document and the features of the second document. For example, the candidate selection unit 2101 searches the rule index 2103 of the second document for each attribute information 2102 extracted from the first document, and adds up the scores (similarity of features) related to the search results obtained for each attribute information 2102 for each second document, and selects a second document with a high score as a candidate document. Note that the candidate selection unit 2101 is not limited to selecting one second document, and may select, for example, multiple second documents whose scores (similarity of features) exceed a threshold value. For the second document selected by the candidate selection unit 2101 , a database (retriever) relating to the document is created by the DB creation unit 203 , and is stored in the storage unit 210 as a second document DB (retriever) 216 .

[0100] Processing in the information processing device 2100 in the second embodiment will be described with reference to Fig. 22. Fig. 22 is a diagram for explaining an example of processing in the information processing device 2100 in the second embodiment. In the following, as an example, a case will be described in which a first document is a product specification 2201, a second document is a standard document 2202 relating to the product standard, and it is confirmed whether the contents described in the specification document 2201 conform to the standard of the standard document 2202.

[0101] The information processing device 2100 acquires a specification 2201 as a first document. The information processing device 2100 also acquires one or more standards documents 2202 as a second document.

[0102] (Process 2203, 2205: Division of specifications and task decomposition) The information processing device 2100 performs specification division processing 2203 on the acquired specification 2201, and divides the entire specification 2201 into multiple documents 2204 that become parts of the specification. In the specification division processing 2203, similar to the specification division processing 303 in the first embodiment, the information processing device 2100 divides the specification 2201 into multiple documents by mechanical processing using an algorithm so that each document after division has a size that can be processed by the trained model 211.

[0103] Moreover, the information processing device 2100 performs a task decomposition process 2205 on the multiple documents 2204 after division, and generates a matching task (task) 2206 related to checking the contents of the specification 2201. In the task decomposition process 2205, similarly to the task decomposition process 305 in the first embodiment, the information processing device 2100 inputs the divided document 2204 and a prompt including an instruction to extract contents that need to be checked for compliance with the standard and output a task to the trained model 211, thereby generating a task 2206 related to checking the contents of the specification 2201.

[0104] (Process 2207: Extract attribute information) The information processing device 2100 performs attribute information extraction processing 2207 on the acquired specification 2201, and extracts attribute information indicating the features of the first document that contribute to identifying the standard referenced in the first document. In the attribute information extraction processing 2207, the information processing device 2100 inputs the specification 2201 divided into multiple documents based on chapters, etc., and a prompt including an instruction to extract information indicating features in the specification 2201 to the trained model 211, thereby generating attribute information 2208 regarding the specification 2201. In this example, by inputting a prompt 2300 instructing to extract information indicating the features of the product described in the specification as shown in FIG. 23(A) from the specification 2201, information that contributes to identifying the standard referenced in the specification as shown in FIG. 23(B) is extracted as attribute information 2208.

[0105] (Process 2209: Create DB of specifications) The information processing device 2100 performs DB creation processing 2209 for the specifications based on the acquired specifications 2201, and creates a specification DB (retriever) 2210. The specification DB (retriever) 2210 corresponds to the first document DB (retriever) 215 shown in Fig. 21. The information processing device 2100 processes the specifications 2201 into a searchable state by mechanical processing using an algorithm, and creates a database.

[0106] (Process 2211: Rule index construction) The information processing device 2100 performs a rule index construction process 2211 for the standard document for the acquired standard document 2202, extracts information representing the characteristics of the standard described in each standard document, and constructs a rule index 2212 for the standard document. In the rule index construction process 2211 for the standard document, the information processing device 2100 inputs, to the trained model 211, the standard document divided into a plurality of documents based on chapters or the like, and a prompt including an instruction to extract and output information representing the characteristics of the standard, thereby extracting information representing the characteristics of the standard from each standard document. For example, the information processing device 2100 inputs the divided standard document and a prompt 2400 shown in FIG. 24(A) to the trained model 211, thereby extracting information representing the characteristics of the standard as shown in FIG. 24(B) from the standard document. The information processing device 2100 links the extracted information representing the characteristics of the standard to the standard document to construct a rule index 2212 for the standard document. Prompt 2400 shown in FIG. 24(A) includes prompt 2401 that instructs the user to concisely output information that represents the characteristics of the standard from the standard document, and learning samples 2402 and 2403 to be trained by trained model 211.

[0107] (Process 2213: Extraction of candidate documents for standard documents) The information processing device 2100 performs candidate document extraction processing 2213 for standard documents to select standard documents 2202 as candidate documents to be used for confirming the contents of the specifications 2201. In the candidate document extraction processing 2213 for standard documents, the information processing device 2100 searches the rule index 2212 for standard documents based on the attribute information 2208 extracted in the attribute information extraction processing 2207, and selects standard documents 2202 with high scores (similarity of features) related to the search results as candidate documents to be used for confirming the contents of the specifications 2201. The information processing device 2100 also creates a standard document DB (retriever) 2214 based on the selected candidate documents (selected standard documents 2202). The standard document DB (retriever) 2214 corresponds to the second document DB (retriever) 216 shown in FIG. 21.

[0108] (Process 2220: Conformity determination) For the tasks generated by the task decomposition process 2205, the information processing device 2100 performs conformance determination process 2220 for each task using the trained model 211 in the same manner as in the first embodiment, and determines whether or not the contents described in the specification 2201 conform to the standard of the standard document 2202. In the conformance determination process 220, the information processing device 2100 performs matching process 2223 using the trained model 211 for tasks 2222 that are sequentially extracted one by one from a task list 2221 in which the tasks 2206 generated by the task decomposition process 2205 are stored in a queue format, and outputs a processing result 2240. The matching process 2223 is performed until there are no more tasks in the task list 2221.

[0109] In the matching process 2223, as in the first embodiment, the trained model 211 is made to simulate the human workflow when matching the contents of documents, a tool is used to collect information necessary for matching, and semantic inference is performed based on the collected information, etc., to obtain a processing result. In this embodiment, the matching process 2223 also performs processing following a framework called ReAct, and outputs the processing result.

[0110] In this embodiment, the tools used by the trained model 211 in the matching process 2223 include a task division tool 2224, a specification search tool 2226, a standards search tool 2229, and a code execution tool 2232. The task division tool 2224 is realized by the conformity determination unit 205, the specification search tool 2226 and the standards search tool 2229 are realized by the information search unit 207, and the code execution tool 2232 is realized by the code execution unit 208. The task division tool 2224, the specification search tool 2226, the standards search tool 2229, and the code execution tool 2232 are similar to the task division tool 324, the specification search tool 326, the standards search tool 329, and the code execution tool 332 in the first embodiment, respectively, and therefore description thereof will be omitted.

[0111] 25, the information processing device 2100 performs extraction processing 2501 to extract content that needs to be checked for conformity with standards in the specification 2201, and generates a matching task (task) 2502. The information processing device 2100 performs extraction processing 2503 to extract attribute information indicating characteristics in the specification 2201, and generates attribute information 2504 related to the specification 2201.

[0112] Furthermore, the information processing device 2100 performs extraction processing 2505 for extracting information representing the characteristics of the standard from the standard document 2202, and constructs a standard document rule index 2506. In the standard document rule index 2506, the information representing the characteristics of the standard (conditional phrase) is associated with the standard document in which it is described (the rule to be followed).

[0113] The information processing device 2100 performs a search process 2507 on a rule index 2506 of the standard document based on the attribute information 2504, and selects a candidate document 2508 of the standard document used to confirm the contents of the specification 2201 based on the similarity of features obtained from the search result. Then, the information processing device 2100 performs a conformance determination process 2509 for each task on the generated task 2502 using the trained model 211, and determines whether the contents described in the specification 2201 conform to the standard of the standard document 2202 selected as the candidate document 2508, and outputs a processing result 2240. Note that, if it is necessary to refer to another standard not included in the candidate document 2508 as a result of the search for the standard document in this conformance determination process 2509, the search may be performed on other documents (for example, all standard documents) in addition to the selected candidate document 2508.

[0114] For example, if it is difficult to identify the second document from the first document to be confirmed, it is possible to enter the original text directly into a search engine and perform a keyword search or embedded search. Keyword searches tend to produce noisy search results because they do not take semantics into account. Embedded searches are expected to produce similar results based on meaning and context, but it can be difficult to understand why a particular search result was selected.

[0115] In contrast, in this embodiment, the information processing device 2100 selects a second document to be used for content confirmation of the first document based on the similarity between the features of the first document and the features of the second document obtained from a search result using attribute information extracted from the first document and a rule index related to the second document constructed by extracting information representing the features of a criterion (rule) from the second document. By selecting the second document to be used for content confirmation of the first document in this manner, the selection of an unrelated second document as a search result is reduced, and it becomes possible to select a second document that is highly relevant as a search result. In addition, by using words (technical terms) specialized for a specific domain as a prompt to be given to the trained model, it is possible to deal with domain-specific areas and perform specialized searches.

[0116] In addition, according to the second embodiment, even if it is difficult to identify the second document to be used for content confirmation from the first document to be confirmed, it is possible to select the second document according to the characteristics of the first document, thereby making it possible to efficiently confirm whether the content of the first document complies with the required conditions.

[0117] In the first and second embodiments described above, the trained model 211 is configured to be stored in the storage unit 210 of the information processing device 100 (2100), but may be configured to be provided in another information processing device (server device, etc.) that can communicate with the information processing device 100 (2100) via a network. In this case, the information processing device 100 (2100) may transmit an input to the trained model 211 via the network I / F 107 or the like to the other information processing device that has the trained model 211, and receive a processing result for the input.

[0118] In addition, in the first and second embodiments described above, a prompt including a learning sample is given to train the trained model 211 (few-shot training), but a trained model that has been fine-tuned to suit each function may also be used.

[0119] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0120] 100, 2100 Information processing device 101 CPU 102 ROM 103 RAM 104 Auxiliary storage 105 Output Device 106 Input Devices 107 Network Interface 201 Acquisition Department 202 Analysis Department 203 DB creation department 204 Information extraction section 205 Conformity Judgment Department 206 Matching Processing Section 207 Information Search Department 208 Code Execution Unit 209 Output section 210 Storage section 211 Trained Models 212 Configuration File 213 Task Data 214 Prerequisite information 215 First Document DB (Retriever) 216 Second Document DB (Retriever) 2101 Candidate Selection Department 2102 Attribute information 2103 Rule Index

Claims

1. An acquisition means for acquiring a first document; an analysis means for extracting contents to be confirmed from the first document and generating a task for each of the contents; a search means for searching a second document for conditions related to the contents to be confirmed in the task; a processing means for sequentially inputting conditions related to the contents to be confirmed in the tasks generated by the analysis means and the tasks searched for by the search means into a trained model, and causing the trained model to respond as to whether or not the contents to be confirmed in the input tasks conform to the conditions searched for in the second document; output means for outputting a processing result by the processing means; The information processing device is characterized in that the analysis means generates a task for each of the contents by inputting the first document, an instruction to extract the content to be confirmed and generate the task, and a prompt including a learning sample to be given to the trained model into a trained model.

2. The analysis means a generating means for generating a task for each of the contents extracted from the first document; 2. The information processing apparatus according to claim 1, further comprising a dividing means for dividing a task generated by said generating means into a plurality of tasks when the task includes a plurality of said contents.

3. The information processing apparatus according to claim 1 , wherein the analysis means divides the first document into a plurality of documents and inputs the documents into the trained model.

4. 4. The information processing apparatus according to claim 3, wherein the analyzing means divides the first document into the plurality of documents according to a predetermined pattern.

5. The information processing device according to claim 1 , wherein the trained model is a large-scale language model.

6. The information processing device according to claim 1 , wherein the trained model is a large-scale multi-modal model.

7. The information processing device according to claim 1 , characterized in that the search means searches for conditions related to the content to be confirmed in the task by inputting the content to be confirmed in the task into a trained model.

8. An information processing method executed by an information processing device, acquiring a first document; an analysis step of extracting contents to be confirmed from the first document and generating a task for each of the contents; a searching step of searching a second document for conditions related to the contents to be confirmed in the task; a processing step of sequentially inputting conditions related to the contents to be confirmed in the tasks generated in the analysis step and the tasks searched for in the search step into a trained model, and causing the trained model to respond as to whether or not the contents to be confirmed in the input tasks conform to the conditions searched for in the second document; An output step of outputting a processing result in the processing step, The information processing method is characterized in that, in the analysis step, a task for each of the contents is generated by inputting into a trained model the first document, an instruction to extract the content to be confirmed and generate the task, and a prompt including a learning sample to be given to the trained model.

9. A computer comprising: An acquisition step of acquiring a first document; an analysis step of extracting contents to be confirmed from the first document and generating a task for each of the contents; a searching step of searching a second document for conditions related to the contents to be confirmed in the task; a processing step of sequentially inputting conditions related to the contents to be confirmed in the task generated in the analysis step and the task searched in the search step into a trained model, and causing the trained model to respond as to whether or not the contents to be confirmed in the input task conform to the conditions searched from the second document; and an output step of outputting a processing result of the processing step. The program, characterized in that in the analysis step, a task for each of the contents is generated by inputting into a trained model a prompt including the first document, instructions for extracting the content to be confirmed and generating the task, and a learning sample to be given to the trained model.

Images