Information processing device, information processing method, and information processing program

Abstract

An information processing device (100) is provided with an output evaluation unit (140). The output evaluation unit (140) uses an evaluator (150) to evaluate a first generation result generated by a machine learning model on the basis of a first input, thereby generating a first evaluation result indicating whether or not the first generation result contains a contradiction with respect to external information. Furthermore, in the case where the first evaluation result indicates that the first generation result contains a contradiction with respect to the external information, when the machine learning model has generated a second generation result on the basis of a second input that is generated on the basis of the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, the output evaluation unit (140) uses the evaluator (150) to evaluate the second generation result, thereby generating a second evaluation result indicating whether or not the contradiction corresponding to the first evaluation result has been resolved in the second generation result.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] Generative AI, such as Large Language Models (LLMs) such as ChatGPT (registered trademark) and image generation AI (Artificial Intelligence) such as Stable Diffusion, has been gaining popularity in recent years. Generative AI may output content containing errors that constitute hallucination. Hallucination is content that differs from objective facts or content that does not match the input. Hallucination hinders the use of generative AI, particularly in use cases where mistakes cannot be tolerated or where fact-checking is time-consuming. Therefore, technology is needed to suppress and detect the occurrence of hallucination.

[0003] Non-Patent Document 1 discloses a technique for improving the reliability of sentences output by a generative AI. In this technique, the quality of sentences generated by an LLM is evaluated by another LLM using various external tools such as web searches and code interpreters (Process 1). The evaluation results and the original input are then provided to the LLM again (Process 2) with the aim of obtaining output with problematic portions corrected. Processes 1 and 2 are repeated until the quality evaluation results by the other LLM meet a predetermined standard, or a predetermined number of times, to obtain generated sentences with relatively high reliability.

[0004] Z. Gou et al. , “CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiqueing”, arXiv preprint arXiv:2305.11738,2023

[0005] According to the conventional technology, when repeating corrections, it is not clear what error was pointed out immediately before, and if the error pointed out immediately before is not corrected, there is a possibility that the same error will need to be pointed out repeatedly, which is inefficient. The present disclosure aims to reduce the risk of repeatedly pointing out the same error in a technology that resolves inconsistencies in external information contained in the results generated by the generation AI by correcting the input to the generation AI.

[0006] The information processing device according to the present disclosure includes an output evaluation unit that uses an evaluator to evaluate a first generation result generated by a machine learning model based on a first input, thereby generating a first evaluation result indicating whether the first generation result includes a contradiction with external information; and when the first evaluation result indicates that the first generation result includes a contradiction with the external information, when a second generation result is generated by the machine learning model based on a second input generated based on the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, evaluates the second generation result using the evaluator, thereby generating a second evaluation result indicating whether the contradiction corresponding to the first evaluation result has been resolved in the second generation result.

[0007] According to the present disclosure, when a first generation result generated by a machine learning model contains a contradiction with external information, a second generation result is generated by the machine learning model for the purpose of resolving the contradiction corresponding to the first evaluation result. The output evaluation unit uses an evaluator to evaluate whether the contradiction has been resolved in the second generation result. Here, the machine learning model may be a generative AI. Therefore, according to the present disclosure, in a technique for resolving a contradiction with external information included in a generation result by the generative AI by correcting an input to the generative AI, the risk of repeatedly pointing out the same error can be reduced.

[0008] FIG. 1 is a diagram showing an example of the configuration of an information processing device 100 according to embodiment 1. FIG. 2 is a diagram showing an example of the hardware configuration of the information processing device 100 according to embodiment 1. FIG. 3 is a flowchart showing the operation of the information processing device 100 according to embodiment 1. FIG. 4 is a diagram showing an example of the configuration of an output evaluation unit 140 according to a modification of embodiment 1. FIG. 5 is a diagram showing an example of the hardware configuration of the information processing device 100 according to a modification of embodiment 1. FIG. 6 is a diagram showing an example of the configuration of an information processing device 100 according to embodiment 2. FIG. 7 is a diagram showing an example of the configuration of an information processing device 100 according to embodiment 3. FIG. 8 is a diagram explaining the processing of the information processing device 100 according to embodiment 3. FIG. 9 is a diagram showing an example of the configuration of an information processing device 100 according to embodiment 4. FIG. 10 is a flowchart showing the operation of the information processing device 100 according to embodiment 4.

[0009] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. The description of elements given the same reference numerals will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, "unit" may be read as "circuit," "step," "procedure," "process," or "circuitry" as appropriate.

[0010] First Embodiment Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0011] *** Description of Configuration *** Fig. 1 shows an example configuration of an information processing device 100 according to this embodiment. As shown in Fig. 1, the information processing device 100 includes an input unit 110, an output generation unit 120, an output evaluation unit 140, and an output unit 160. The information processing device 100 also stores a generative AI (Artificial Intelligence) 130 and an evaluator 150. In this embodiment, the target of evaluation and correction is not limited to the output of an LLM, but may be the output of text, images, audio, or other modalities, or the output of a generative AI that handles multiple of these simultaneously.

[0012] The input unit 110 accepts input from the user to the generation AI 130 and passes the accepted input to the output generation unit 120. Specific examples of the input include a prompt that indicates the content of the sentence or image to be generated, data indicating the sentence to be subjected to summarization processing, or a combination of the prompt and data.

[0013] The output generation unit 120 receives input for the generation AI 130 from the input unit 110, inputs the received input to the generation AI 130, and obtains an output corresponding to the input from the generation AI 130. The output generation unit 120 passes the input received from the input unit 110 and the output obtained from the generation AI 130 to the output evaluation unit 140. The output of the generation AI 130 is the generation result, which is content. Furthermore, when the output generation unit 120 receives a correction instruction from the output evaluation unit 140, it corrects the input for the generation AI 130 based on the correction instruction. In this case, the output generation unit 120 corrects or adds a prompt, as a specific example. Furthermore, the output generation unit 120 may obtain a corrected generation result by inputting the original input received from the input unit 110, the previous generation result, the evaluation result received from the output evaluation unit 140, and a prompt instructing the generation AI 130 to make corrections based on the evaluation result to the generation AI 130. The evaluation results and correction instructions may be input using text prompts, images, audio, or other modalities, or a combination of these.

[0014] The generation AI 130 is an example of a machine learning model. The generation AI 130 generates content based on input and returns the generated content to the output generation unit 120. Specific examples of the content include text, images, and audio, or a combination of these. Specific examples of the generation AI 130 include a general LLM (large-scale language model) or an image generation AI model (such as stable diffusion).

[0015] The output evaluation unit 140 generates a first evaluation result by evaluating the first generation result using the evaluator 150. The first generation result is a generation result generated by a machine learning model based on a first input. The first evaluation result indicates whether the first generation result contains a contradiction with external information. External information is information that exists outside the generation result. Here, in addition to information indicated by information sources such as websites or literature, the content of input to the machine learning model, such as the content of the first input, also constitutes external information. When the first evaluation result indicates that the first generation result contains a contradiction with external information, and a second generation result is generated by the machine learning model based on a second input, the output evaluation unit 140 generates a second evaluation result by evaluating the second generation result using the evaluator 150. The second input is an input generated based on the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result. The second evaluation result indicates whether the contradiction corresponding to the first evaluation result has been resolved in the second generation result. Furthermore, when the second evaluation result indicates that the contradiction corresponding to the first evaluation result has not been resolved in the second generation result, and the third generation result is generated by the machine learning model based on the third input, the output evaluation unit 140 uses the evaluator 150 to evaluate whether the contradiction corresponding to the first evaluation result has been resolved in the third generation result. The third input is an input generated based on the first input and the second evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result. The machine learning model is not limited to the generation AI 130, but may be another AI or an inference model that does not fall under AI. As a specific example, the output evaluation unit 140 receives the input and the generation result from the output generation unit 120 and evaluates the received generation result using the evaluator 150. In this case, the output evaluation unit 140 may process the generation result, input the processed generation result to the evaluator 150, and receive the evaluation result from the evaluator 150. Specific examples of processing the generation result include combining the input to the generation AI 130 and the generation result, or converting the data format of the generation result.The evaluation results include, for example, an evaluation result of whether the generated result contains a hallucination and an evaluation result of whether the generated result has been corrected in accordance with the previous evaluation result. A hallucination corresponds to a contradiction in external information. Furthermore, the output evaluation unit 140 determines whether to use the current generated result as the final output or to correct the current generated result based on the evaluation result received from the evaluator 150. If the current generated result is the final output, the output evaluation unit 140 passes the current generated result to the output unit 160. If the output evaluation unit 140 determines that the current generated result should be corrected, it issues a correction instruction regarding the generated result to the output generation unit 120. At this time, the output evaluation unit 140 may pass the evaluation result to the output generation unit 120. When re-evaluating the corrected generated result, the output evaluation unit 140 first evaluates whether the points pointed out in the previous evaluation result have been appropriately corrected. At this time, the evaluator 150 may use the same method or model as before, or a lighter-weight method or model.

[0016] The evaluator 150 evaluates whether the generated result contains an error corresponding to hallucination based on the input received from the output evaluation unit 140, and returns the evaluation result to the output evaluation unit 140. The evaluation result may be a binary value indicating the presence or absence of an error, a numerical value or character string indicating the type of error, a sentence specifically explaining the content of the error, a numerical value indicating the location of the error, or image data. The evaluation result may also be a result of predicting the presence or absence of an error. The evaluator 150 may be implemented using an LLM-based machine learning model as in conventional technology, a rule-based method that provides more stable results, or a method that compares with an external information source such as a knowledge graph. The evaluator 150 may be composed of two evaluators: a first evaluator and a second evaluator. The first evaluator evaluates the entire generated result. The second evaluator evaluates whether the generated result has been appropriately corrected corresponding to the previous evaluation result, if the generated result has been corrected one or more times.

[0017] The output unit 160 receives the generated results and outputs the received generated results.

[0018] 2 shows an example of the hardware configuration of the information processing device 100 according to this embodiment. The information processing device 100 is made up of a computer. The information processing device 100 may be made up of multiple computers.

[0019] As shown in the figure, the information processing device 100 is a computer including hardware such as a processor 11, a memory 12, an auxiliary storage device 13, an input / output IF (Interface) 14, and a communication device 15. These pieces of hardware are connected as appropriate via signal lines 19.

[0020] The processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The information processing device 100 may include multiple processors that replace the processor 11. The multiple processors share the role of the processor 11.

[0021] The memory 12 is typically a volatile storage device, specifically a random access memory (RAM). The memory 12 is also called a primary storage device or a main memory. Data stored in the memory 12 is saved in the secondary storage device 13 as needed.

[0022] The auxiliary storage device 13 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 13 is loaded into the memory 12 as needed. The memory 12 and the auxiliary storage device 13 may be configured integrally.

[0023] The input / output IF 14 is a port to which an input device and an output device are connected. Specific examples of the input / output IF 14 include a USB (Universal Serial Bus) terminal. Specific examples of the input device include a keyboard and a mouse. Specific examples of the output device include a display.

[0024] The communication device 15 is a receiver and a transmitter, and is specifically a communication chip or a NIC (Network Interface Card).

[0025] Each unit of the information processing device 100 may use the input / output IF 14 and the communication device 15 as appropriate when communicating with other devices.

[0026] The auxiliary storage device 13 stores an information processing program. The information processing program is a program that causes a computer to realize the functions of each unit included in the information processing device 100. The information processing program is loaded into the memory 12 and executed by the processor 11. The functions of each unit included in the information processing device 100 are realized by software.

[0027] Data used when executing an information processing program and data obtained by executing the information processing program are stored in a storage device as appropriate. Each part of the information processing device 100 uses a storage device as appropriate. Specific examples of the storage device include at least one of the memory 12, the auxiliary storage device 13, a register in the processor 11, and a cache memory in the processor 11. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer. The functions of the memory 12 and the auxiliary storage device 13 may be realized by other storage devices.

[0028] The information processing program may be recorded on a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The information processing program may be provided as a program product. The processing of the generation AI 130 may be executed by an external server or the like.

[0029] ***Description of Operation*** The operational procedure of the information processing device 100 corresponds to an information processing method. Also, the program that realizes the operation of the information processing device 100 corresponds to an information processing program.

[0030] 3 is a flowchart showing an example of the operation of the information processing device 100. The operation of the information processing device 100 will be described with reference to FIG.

[0031] (Step S1 ) The input unit 110 receives an input from a user and passes the received input to the output generation unit 120 .

[0032] (Step S2) The output generation unit 120 processes the input as appropriate and inputs it to the generation AI 130. The input in this step corresponds to the first input.

[0033] (Step S3) The generation AI 130 generates content corresponding to the input and returns the generated content to the output generation unit 120. The content generated by the generation AI 130 in this step is called the "current generation result."

[0034] (Step S4) If the current generation result is a generation result in which corrections based on correction instructions corresponding to the previous evaluation result have been performed a predetermined number of times, the output evaluation unit 140 proceeds to step S10 without evaluating the current generation result. Otherwise, the output evaluation unit 140 proceeds to step S5. Here, it is assumed that the predetermined number of times is set in advance.

[0035] (Step S5) If the current generation result is a generation result that has been modified based on the modification instruction, the output evaluation unit 140 proceeds to step S6. Otherwise, the output evaluation unit 140 proceeds to step S8.

[0036] (Step S6) The output evaluation unit 140 evaluates the current generation result based on the evaluator 150. In this step, the output evaluation unit 140 may generate a second evaluation result using the evaluator 150. Based on the evaluation result of the current generation result, the output evaluation unit 140 determines whether or not the current generation result has been appropriately corrected based on the correction instruction corresponding to the previous evaluation result.

[0037] (Step S7) If the current generation result has been appropriately corrected based on the correction instruction corresponding to the previous evaluation result, the output evaluation unit 140 proceeds to step S8. Otherwise, the output evaluation unit 140 proceeds to step S11.

[0038] (Step S8) The output evaluation unit 140 processes the current generation result as appropriate and then inputs it to the evaluator 150. In this step, the output evaluation unit 140 generates a first evaluation result using the evaluator 150. The evaluator 150 predicts the result of evaluating whether or not the current generation result contains an error based on the input received from the output evaluation unit 140, and returns the predicted result to the output evaluation unit 140 as the evaluation result corresponding to the current generation result.

[0039] (Step S9) The output evaluation unit 140 determines whether or not to use the current generation result as the final generation result based on the evaluation result. If the current generation result is to be used as the final generation result, the output evaluation unit 140 proceeds to step S10. Otherwise, the output evaluation unit 140 proceeds to step S12.

[0040] (Step S10) The output evaluation unit 140 passes the current generation result as the final generation result to the output unit 160. The output unit 160 outputs the received generation result.

[0041] (Step S11) If it is determined that the current generation result has not been appropriately corrected based on the correction instructions corresponding to the previous evaluation result, the output evaluation unit 140 passes the most recent evaluation result to the output generation unit 120. The output generation unit 120 inputs the original input and an input based on the evaluation result to the generation AI 130, with the aim of making corrections based on the correction instructions corresponding to the previous evaluation result. At this time, the output generation unit 120 inputs at least one of the following to the generation AI 130, as a specific example: the "current generation result," the "input at the time of current generation (the previous generation result, the evaluation result, and a prompt instructing correction)," and "a notice that the corrections based on the correction instructions corresponding to the previous evaluation result have not been appropriately made, and an instruction to make new corrections based on the correction instructions corresponding to the previous evaluation result." The input in this step corresponds to the second input or the third input.

[0042] (Step S12) The output evaluation unit 140 issues a correction instruction corresponding to the current generation result to the output generation unit 120 based on the immediately preceding evaluation result. This correction instruction corresponds to a "correction instruction corresponding to the previous evaluation result." When correcting the current generation result, the output generation unit 120 inputs to the generation AI 130, as a specific example, the original input received from the input unit 110, the current generation result, the evaluation result received from the output evaluation unit 140, and a prompt instructing correction based on the evaluation result.

[0043] ***Explanation of the Effects of First Embodiment*** According to the prior art, only text information is targeted, and halocation of modal data other than text, such as images and audio, is not supported. On the other hand, according to this embodiment, it is possible to correct not only the results of LLM generation, but also the results of all generation AI.

[0044] Furthermore, in conventional technology, output evaluation and correction are performed using an LLM, so the reliability of the generated text is heavily dependent on the performance of the LLM used. However, since training an LLM requires sufficient computational resources, power, and time, it is difficult to improve the accuracy of evaluation using conventional technology. On the other hand, according to this embodiment, the evaluator 150 that evaluates the generated results is not limited to an LLM. Furthermore, by separating the generation AI 130 from the evaluator 150, it is relatively easy to improve accuracy and apply domains through additional training of the evaluator 150. Therefore, according to this embodiment, the reliability of the output of the generation AI can be improved without relying on the performance of the LLM. Furthermore, according to this embodiment, an appropriate evaluator 150 can be selected depending on the modality of the target generation AI 130.

[0045] 4 shows an example of the configuration of the output evaluation unit 140 according to this modification. The output evaluation unit 140 includes an evaluation processing unit 141, an information acquisition unit 142, and an external tool cooperation unit 143.

[0046] The evaluation processing unit 141 passes the current generation result received from the output generation unit 120 to each of the information acquisition unit 142 and the external tool cooperation unit 143. The evaluation processing unit 141 also evaluates the current generation result based on the result of evaluating the current generation result using the evaluator 150, the information acquired from the information acquisition unit 142, and the processing result received from the external tool cooperation unit 143.

[0047] The information acquisition unit 142 acquires, as necessary, information to be used for evaluating the current generation result from the external database 170, and passes the acquired information to the evaluation processing unit 141. The external database 170 consists of data used when determining whether or not there is a contradiction in the content included in the generation result.

[0048] The external tool cooperation unit 143 accesses the external tool 180, performs the necessary processing for evaluating the current generated result, and passes the processing result to the evaluation processing unit 141. The external tool 180 is a tool that has a function of evaluating inconsistencies in the content included in the generated result. Specific examples of the external tool 180 are a web search system, a code interpreter, or a calculator program.

[0049] 5 shows an example of the hardware configuration of an information processing device 100 according to this modification. The information processing device 100 includes a processing circuit 18 instead of the processor 11, the processor 11 and memory 12, the processor 11 and auxiliary storage device 13, or the processor 11, memory 12, and auxiliary storage device 13. The processing circuit 18 is hardware that realizes at least a portion of the components included in the information processing device 100. The processing circuit 18 may be dedicated hardware, or may be a processor that executes a program stored in the memory 12.

[0050] When the processing circuitry 18 is dedicated hardware, the processing circuitry 18 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The information processing device 100 may include multiple processing circuits that replace the processing circuitry 18. The multiple processing circuits share the role of the processing circuitry 18.

[0051] In the information processing device 100, some of the functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0052] The processing circuitry 18 is realized by, for example, hardware, software, firmware, or a combination of these. The processor 11, memory 12, auxiliary storage device 13, and processing circuitry 18 are collectively referred to as "processing circuitry." In other words, the functions of the functional components of the information processing device 100 are realized by the processing circuitry. Information processing devices 100 according to other embodiments may also have a configuration similar to that of this modified example.

[0053] Second Embodiment The following mainly describes the differences from the above-described embodiment with reference to the drawings.

[0054] ***Description of Configuration*** Fig. 6 shows an example configuration of an information processing device 100 according to this embodiment. Compared to embodiment 1, the information processing device 100 further includes a correction result evaluation unit 210, a database update unit 220, a storage unit 240, and an evaluator update unit 250. The information processing device 100 also stores an external database 230. Note that the information processing device 100 does not necessarily have to include the database update unit 220 and does not necessarily have to store the external database 230.

[0055] When outputting the final generated result, the output evaluation unit 140 according to this embodiment passes all or part of the evaluation results and the revision history up until the final generated result is determined to the output unit 160 .

[0056] The evaluator 150 according to this embodiment evaluates the results of generation by the generation AI 130 based on an external database 230 that indicates external information.

[0057] The output section 160 according to this embodiment outputs the final generation result together with the evaluation result and the revision history received from the output evaluation section 140. The revision history includes one or more revision results.

[0058] The correction result evaluation unit 210 accepts an evaluation result for the evaluation result by the evaluator 150 as a first user evaluation result. Furthermore, the correction result evaluation unit 210 accepts an evaluation result for data corresponding to the evaluation result by the evaluator 150, among the data indicated by the external database 230, as a second user evaluation result. As a specific example, the correction result evaluation unit 210 accepts, from the user, an evaluation of each evaluation result and each correction result output by the output unit 160 as a user evaluation result, and records the accepted user evaluation result in the storage unit 240. The user evaluation result is a collective term for the first user evaluation result and the second user evaluation result. As a specific example, the user evaluation result is data indicating the correctness or incorrectness of each evaluation result and each correction result using a binary value. Instead of accepting an evaluation from the user, the correction result evaluation unit 210 may automatically evaluate each evaluation result and each correction result.

[0059] The database update unit 220 updates the external database 230 based on the second user evaluation result. As a specific example, the database update unit 220 updates the contents of the external database 230 as needed based on the evaluation result input by the user in the correction result evaluation unit 210 and the evaluation result on the correction result.

[0060] The external database 230 is similar to the external database 170. The external database 230 may be similar to the external tool 180.

[0061] The storage unit 240 stores the user evaluation results.

[0062] The evaluator update unit 250 updates the evaluator 150 based on the first user evaluation result. As a specific example, the evaluator update unit 250 appropriately updates the evaluator 150 based on the user evaluation results accumulated in the storage unit 240. As a specific example, the updating of the evaluator 150 is realized by fine-tuning or the like. If the evaluator 150 is realized by a neural network, the evaluator update unit 250 may update the evaluator 150 when the amount of accumulated user evaluation results in the storage unit 240 reaches or exceeds a certain level. If the evaluator 150 is realized by a rule base or a knowledge graph, the evaluator update unit 250 may update the evaluator 150 every time a user evaluation result is added to the storage unit 240.

[0063] ***Explanation of Operation*** The operation of the information processing device 100 according to this embodiment is basically the same as the operation of the information processing device 100 according to embodiment 1. The following mainly describes the differences from embodiment 1.

[0064] (Step S10) The output evaluation unit 140 passes the final generation result as well as the evaluation results and correction history up to the determination of the final generation result to the output unit 160. The output unit 160 presents the generation result as well as each evaluation result and each correction history included in the correction history to the user. The user evaluates each evaluation result and each correction result presented via the correction result evaluation unit 210. The correction result evaluation unit 210 stores data indicating the user's evaluations and comments, etc. in the storage unit 240. The database update unit 220 appropriately updates the external database 230 based on the user's evaluations and comments, etc. The evaluator update unit 250 appropriately updates the evaluator 150 based on the user's evaluations and comments, etc. stored in the storage unit 240.

[0065] ***Explanation of Effect of Second Embodiment*** In the prior art, when an erroneous evaluation or correction is made, the model used for the evaluation or the external tool, etc., is not updated, and therefore there is a high possibility that a similar error will occur again in the future. On the other hand, according to this embodiment, by using the external database 230 and appropriately updating the external database 230 and the evaluator 150 based on the evaluation by the user, it is possible to improve the accuracy of the evaluation by the evaluator 150 and the identification of inconsistencies, and reduce the possibility of making a similar error.

[0066] Third Embodiment Hereinafter, differences from the above-described embodiments will be mainly described with reference to the drawings.

[0067] ***Description of Configuration*** Fig. 7 shows an example of the configuration of an information processing device 100 according to this embodiment. Compared to embodiment 2, the information processing device 100 includes a knowledge graph update unit 310 instead of the database update unit 220, and a knowledge graph 320 instead of the external database 230. In this embodiment, the external information is the knowledge graph 320.

[0068] The evaluator 150 according to this embodiment evaluates the results generated by the generation AI 130 based on the knowledge graph 320. As a specific example, the evaluator 150 is a model that determines whether or not there is a contradiction in the current generated result with the premise knowledge graph 320, and if it determines that there is a contradiction, presents which facts indicated in the knowledge graph 320 are in conflict with the current generated result. The contradiction in the text with respect to the facts indicated in the knowledge graph 320 is presented to the user in the form of a graph or a triplet, for example.

[0069] When correcting the generation result based on the evaluation result, the output evaluation unit 140 according to this embodiment specifically reflects, in the input to the generation AI 130, any inconsistencies contained in the current generation result that are inconsistent with a certain part on the knowledge graph 320. In this embodiment, when the first evaluation result indicates that the first generation result contains an inconsistency with external information, the first evaluation result indicates, from the knowledge indicated by the knowledge graph 320, the knowledge that corresponds to the inconsistency corresponding to the first generation result.

[0070] The correction result evaluation unit 210 according to this embodiment receives, as a third user evaluation result, the evaluation result for the knowledge corresponding to the evaluation result by the evaluator 150, from among the knowledge indicated by the knowledge graph 320.

[0071] The knowledge graph update unit 310 updates the knowledge graph 320 based on the third user evaluation result. The third user evaluation result is a subordinate concept of the user evaluation result. As a specific example, the knowledge graph update unit 310 updates the contents of the knowledge graph 320 as necessary based on the evaluation result input by the user in the correction result evaluation unit 210 and the evaluation result of the correction result.

[0072] Knowledge graph 320 is similar to external database 230 and consists of data that represents a knowledge graph.

[0073] ***Description of Operation*** The operation of the information processing device 100 according to this embodiment corresponds to the operation of the information processing device 100 according to embodiment 2, with the database update unit 220 replaced with the knowledge graph update unit 310 and the external database 230 replaced with the knowledge graph 320.

[0074] 8 and 9 are diagrams illustrating an example of the processing of the information processing device 100 when generating a sentence using the generation AI 130. In this example, the evaluator 150 is a model that identifies parts of the sentence that contradict the facts indicated by the knowledge graph 320 and the contradictory facts on the knowledge graph 320.

[0075] An example of a process for correcting the generated results based on the knowledge graph 320 will be described using FIG. 8 . First, after a sentence is generated by the generation AI 130, the output evaluation unit 140 evaluates, based on the evaluator 150 and the knowledge graph 320, that the part of the generated sentence, "A1 is equipped with Y," is inconsistent with the facts of <Product A1, equipped, Z> shown in the knowledge graph 320, and passes the evaluation result to the output generation unit 120. Next, the output generation unit 120 creates a prompt based on the evaluation result received from the output evaluation unit 140 and inputs the created prompt to the generation AI 130. FIG. 8 shows an example of a prompt created based on the initial input, the evaluation result, and the facts shown in the knowledge graph 320. This prompt corresponds to the second input or the third input.

[0076] An example of the process of updating the knowledge graph 320 will be described with reference to Figure 9. First, the output unit 160 presents to the user the facts indicated by the knowledge graph 320 that were used in the correction, and the correction history, along with the generation result. Next, the user evaluates the presented facts and correction history via the correction result evaluation unit 210. Next, the knowledge graph update unit 310 updates the knowledge graph 320 based on the user's evaluation, comments, etc.

[0077] ***Explanation of the Effects of Embodiment 3*** As described above, according to this embodiment, the generation results are corrected based on the knowledge graph 320, and the knowledge graph 320 is updated as appropriate, thereby improving the accuracy of content correction by the generation AI 130. Furthermore, according to this embodiment, when presenting correction content to the user, contradictions in the text relative to the facts on the knowledge graph 320 are presented in the form of a graph or a triplet, for example. Therefore, according to this embodiment, the history of corrections can be presented in a form that is easy for humans to understand, making it relatively easy for the user to evaluate.

[0078] Fourth Embodiment Hereinafter, differences from the above-described embodiments will be mainly described with reference to the drawings.

[0079] ***Description of Configuration*** Fig. 10 shows an example of the configuration of an information processing device 100 according to this embodiment. Compared to the first embodiment, the information processing device 100 further includes a prompt change unit 410.

[0080] When the first evaluation result indicates that the first generation result contains a contradiction with external information, the prompt modification unit 410 changes the prompt indicated by the second input to a correction prompt when the generation of a generation result by the generation AI 130 aimed at resolving the contradiction corresponding to the first evaluation result has been repeated a reference number of times or more. In this embodiment, the second input indicates a prompt. The second generation result is a generation result generated using the correction prompt. The reference number of times of correction may be determined in any manner. As a specific example, when a loop of evaluation and correction of the current generation result has been repeated a reference number of times or more, the prompt modification unit 410 changes the content of the prompt that instructs the correction and is provided to the generation AI 130 along with the original input and evaluation result during correction. When changing the content of the prompt, the prompt modification unit 410 may accept manual modification, may select a prompt randomly or sequentially from a set of predefined prompts, or may use the same or different machine learning model to generate the prompt.

[0081] ***Explanation of Operation*** Fig. 11 is a flowchart showing an example of the operation of the information processing device 100. The operation of the information processing device 100 will be described with reference to Fig. 11 .

[0082] (Step S41) The output generation unit 120 checks whether the correction process (steps S3 to S7, step S41, step S41, and step S11) has been repeated a predetermined number of times or more since the first correction or since the last time the correction instruction prompt was changed. If the correction process has been repeated a predetermined number of times or more, the output generation unit 120 proceeds to step S42. Otherwise, the output generation unit 120 proceeds to step S11. Note that in step S7, the output evaluation unit 140 proceeds to step S41 instead of step S11.

[0083] (Step S42) The prompt change unit 410 changes the content of the prompt that instructs modification of the generated result.

[0084] ***Explanation of the Effects of the Fourth Embodiment*** As described above, according to this embodiment, the prompt content is changed when the generated result is corrected a certain number of times or more, thereby preventing the same mistake from being repeated multiple times. Therefore, according to this embodiment, generated content of relatively high quality can be obtained efficiently.

[0085] ***Other Embodiments*** The above-described embodiments can be freely combined, or any of the components of each embodiment can be modified, or any of the components can be omitted from each embodiment. Furthermore, the embodiments are not limited to those shown in embodiments 1 to 4, and various modifications are possible as needed. The procedures described using flowcharts, etc., can be modified as appropriate.

[0086] 11 Processor, 12 Memory, 13 Auxiliary storage device, 14 Input / output IF, 15 Communication device, 18 Processing circuit, 19 Signal line, 100 Information processing device, 110 Input unit, 120 Output generation unit, 130 Generation AI, 140 Output evaluation unit, 141 Evaluation processing unit, 142 Information acquisition unit, 143 External tool collaboration unit, 150 Evaluator, 160 Output unit, 170 External database, 180 External tool, 210 Correction result evaluation unit, 220 Database update unit, 230 External database, 240 Memory unit, 250 Evaluator update unit, 310 Knowledge graph update unit, 320 Knowledge graph, 410 Prompt change unit.

Claims

1. An information processing device comprising: an output evaluation unit that uses an evaluator to evaluate a first generation result generated by a machine learning model based on a first input, thereby generating a first evaluation result indicating whether the first generation result includes a contradiction with external information; and when the first evaluation result indicates that the first generation result includes a contradiction with the external information, when a second generation result is generated by the machine learning model based on a second input generated based on the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, evaluates the second generation result using the evaluator, thereby generating a second evaluation result indicating whether the contradiction corresponding to the first evaluation result has been resolved in the second generation result.

2. The information processing device of claim 1, wherein when the second evaluation result indicates that the contradiction corresponding to the first evaluation result has not been resolved in the second generation result, and a third generation result is generated by the machine learning model based on a third input generated based on the first input and the second evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, the output evaluation unit uses the evaluator to evaluate whether the contradiction corresponding to the first evaluation result has been resolved in the third generation result.

3. The information processing device according to claim 1 or 2, further comprising: a correction result evaluation unit that accepts an evaluation result for the evaluation result by the evaluator as a first user evaluation result; and an evaluator update unit that updates the evaluator based on the first user evaluation result.

4. The information processing device described in any one of claims 1 to 3, wherein the evaluator evaluates the results generated by the machine learning model based on an external database indicating the external information, and the information processing device further comprises: a correction result evaluation unit that accepts, as a second user evaluation result, an evaluation result for data indicated by the external database that corresponds to the evaluation result by the evaluator; and a database update unit that updates the external database based on the second user evaluation result.

5. An information processing device described in any one of claims 1 to 3, wherein the external information is a knowledge graph, the evaluator evaluates the results generated by the machine learning model based on the knowledge graph, and when the first evaluation result indicates that the first generation result contains a contradiction with the external information, the first evaluation result indicates knowledge indicated by the knowledge graph that corresponds to the contradiction corresponding to the first generation result.

6. The information processing device according to claim 5, further comprising: a correction result evaluation unit that accepts, as a third user evaluation result, an evaluation result for knowledge indicated by the knowledge graph that corresponds to the evaluation result by the evaluator; and a knowledge graph update unit that updates the knowledge graph based on the third user evaluation result.

7. The second input indicates a prompt, and the information processing device further comprises: a prompt change unit that changes the prompt indicated by the second input to a correction prompt when, in a case where the first evaluation result indicates that the first generation result contains a contradiction with the external information, generation of a generation result by the machine learning model aimed at resolving the contradiction corresponding to the first evaluation result is repeated a correction standard number of times or more, and the second generation result is a generation result generated using the correction prompt. An information processing device as described in any one of claims 1 to 6.

8. The information processing device according to any one of claims 1 to 7, wherein the machine learning model is generative AI (Artificial Intelligence).

9. An information processing method in which a computer uses an evaluator to evaluate a first generation result generated by a machine learning model based on a first input, thereby generating a first evaluation result indicating whether the first generation result includes a contradiction with external information; and when the first evaluation result indicates that the first generation result includes a contradiction with the external information, when a second generation result is generated by the machine learning model based on a second input generated based on the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, the computer uses the evaluator to evaluate the second generation result, thereby generating a second evaluation result indicating whether the contradiction corresponding to the first evaluation result has been resolved in the second generation result.

10. An information processing program that causes an information processing device that is a computer to execute an output evaluation process that generates a first evaluation result indicating whether or not the first generation result contains a contradiction with external information by using an evaluator to evaluate a first generation result generated by a machine learning model based on a first input, and when the first evaluation result indicates that the first generation result contains a contradiction with the external information, when a second generation result is generated by the machine learning model based on a second input generated based on the first input and the first evaluation result for the purpose of resolving the contradiction corresponding to the first evaluation result, by evaluating the second generation result using the evaluator to generate a second evaluation result indicating whether or not the contradiction corresponding to the first evaluation result has been resolved in the second generation result.

Images