Method, program, and information processing device for performing processing to improve the accuracy of answers given by generation AI

Abstract

To provide a method, a program, and an information processing device that can improve the accuracy of answers generated by a generation AI in response to user inquiries. [Solution] The method of the present invention is a method in which an information processing device 1 performs processing to improve the accuracy of answers generated by a generation AI in response to a user's inquiry, wherein the information processing device 1 is accessible to a storage device 2, and the storage device 2 stores information on example answers used by the generation AI 6 to generate answers, and includes a step of accepting an evaluation of the answer from the user (S202), a step of displaying the evaluated answer on the screen (S204), a step of accepting corrections to the answer displayed on the screen (S205), and a step of linking the corrected answer to the example inquiry and storing it in the storage device 2 as example answer information (S206).

Description

[Technical Field]

[0001] The present invention relates to a method, a program, and an information processing device for performing processing to improve the accuracy of answers given by generation AI (Artificial Intelligence). [Background technology]

[0002] Patent Document 1 discloses a method for supporting conversations with users using a chatbot that improves answer accuracy. In this method, a control unit of a server terminal receives a question from a user terminal of a user, generates an answer to the question, receives an instruction to ask a follow-up question to the answer, and generates an answer different from the original answer in response to the instruction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-148084 Summary of the Invention [Problem to be solved by the invention]

[0004] When a generation AI generates an answer to a user's inquiry, it is not always an accurate answer. For this reason, it is important to be able to provide feedback on the user's reaction to the answer.

[0005] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a method, program, and information processing device that can improve the accuracy of answers generated by a generation AI in response to user inquiries. [Means for solving the problem]

[0006] A method according to a first aspect of the present invention is a method in which an information processing device performs processing to improve the accuracy of answers generated by a generation AI in response to a user's inquiry, the information processing device being accessible to a storage device, the storage device storing information on example answers used by the generation AI to generate answers, and the method having the steps of accepting an evaluation of the answer from a user, displaying the evaluated answer on a screen, accepting corrections to the answer displayed on the screen, and linking the corrected answer to the example inquiry and storing it in the storage device as example answer information.

[0007] A program according to a second aspect of the present invention is a program including instructions for causing an information processing device to perform processing to improve the accuracy of answers generated by a generation AI in response to a user's inquiry, wherein in the processing to improve the accuracy of the answer, the information processing device is capable of accessing a storage device, and the storage device stores information on example answers used by the generation AI to generate an answer, and the program includes each of the steps of a method having the following steps: accepting an evaluation of the answer from a user; displaying the evaluated answer on a screen; accepting corrections to the answer displayed on the screen; and linking the corrected answer to the example inquiry and storing it in the storage device as example answer information.

[0008] An information processing device according to a third aspect of the present invention is an information processing device that performs processing to improve the accuracy of answers generated by a generation AI in response to a user's inquiry, and has a processing unit and a memory unit that stores instructions for causing the processing unit to perform processing, and in the processing performed by the processing unit in accordance with the instructions, the information processing device is able to access a storage device, and the storage device stores information on example answers used by the generation AI to generate an answer, and includes each step of a method having the steps of accepting an evaluation of the answer from a user, displaying the evaluated answer on a screen, accepting corrections to the answer displayed on the screen, and linking the corrected answer to the example inquiry and storing it in the storage device as example answer information.

[0009] An information processing device according to a fourth aspect of the present invention is an information processing device that performs a process to improve the accuracy of answers generated by a generation AI in response to a user's inquiry, the information processing device being able to access a storage device, the storage device storing information on example answers used by the generation AI to generate answers, and having means for performing each step of a method including the steps of accepting an evaluation of the answer from a user, displaying the evaluated answer on a screen, accepting corrections to the answer displayed on the screen, and linking the corrected answer to the example inquiry and storing it in the storage device as information on the example answer. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a method, a program, and an information processing device that can improve the accuracy of answers generated by a generation AI in response to user inquiries. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a system to which an information processing device 1 according to this embodiment is applied. [Figure 2] FIG. 2 is a diagram illustrating an example of processing in a system including the information processing device 1. As shown in FIG. [Figure 3] FIG. 3 is a diagram illustrating an example of the process of correcting an answer by the information processing device 1. In FIG. [Figure 4] FIG. 4 is a diagram illustrating another example of the process of correcting an answer by the information processing device 1. In FIG. [Figure 5] FIG. 5 is a diagram showing an example of a screen display for evaluating and correcting an answer. [Figure 6] FIG. 6 is a diagram showing an example of a screen display for evaluating and correcting an answer. [Figure 7] FIG. 7 is a diagram showing an example of a screen display for evaluating and correcting an answer. [Figure 8] FIG. 8 is a diagram showing an example of a screen display for evaluating and correcting an answer. [Figure 9]FIG. 9 is a diagram showing an example of a screen display for evaluating and correcting an answer. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same components will be designated by the same reference numerals, and the description of components that have already been described will be omitted as appropriate.

[0013] FIG. 1 is a diagram showing an example of the configuration of a system to which an information processing device 1 according to this embodiment is applied. The system shown in the example of FIG. 1 includes an information processing device 1, a terminal device 3, a cloud server 4, a search engine 5, and a generating AI 6, all of which are capable of communicating via a communication network 9 such as the Internet. Here, the information processing device 1 is an example of an information processing device of the present invention. The storage device 2 is an example of a storage device of the present invention. The terminal device 3 is an example of a terminal device of the present invention. The generation AI 6 is an example of a generation AI of the present invention.

[0014] In the system shown in Figure 1, the generation AI 6 generates an answer to an inquiry from a user operating the terminal device 3 and responds to the terminal device 3, and the generation AI 6 obtains an evaluation from the user regarding the answer and provides feedback to correct the answer based on the evaluation.

[0015] (Terminal device 3) The terminal device 3 is a device operated by a user to make inquiries, ask questions, search for information, etc., and is a device equipped with information communication functions, such as a personal computer, tablet, smartphone, etc. In the example of the system shown in Figure 1, there is one terminal device 3, but the system may have multiple terminal devices 3 corresponding to multiple users.

[0016] The terminal device 3 includes, for example, a communication unit, a storage unit, and a processing unit similar to the communication unit 11, the storage unit 12, and the processing unit 13 of the information processing device 1 described later. The terminal device 3 may also include an input unit (such as a touch panel, a mouse, or a keyboard) for inputting user instructions to the processing unit, a display unit (such as a liquid crystal display or an organic EL display) for displaying an image corresponding to the image data generated in the processing unit, and a speaker for outputting sound.

[0017] (Cloud Server 4) The cloud server 4 is a server for providing cloud services and is connected to the communication network 9. The cloud server 4 executes processing in response to requests from the terminal device 3 and the information processing device 1, and responds by providing information, etc. For example, in order to digitize workflows to improve business efficiency, the cloud server 4 provides various services in response to requests from the terminal device 3 operated by employees or customers. The cloud server 4 may also respond to requests from the information processing device 1, or may respond to requests from the terminal device 3 via the information processing device 1.

[0018] (Search Engine 5) The search engine 5 is an information search system that uses search keys (keywords, key images, etc.) and is connected to the communication network 9. The search engine 5 performs similarity searches and filtering by, for example, vector searches using a vector database, to perform highly accurate searches.

[0019] (Generation AI6) Generative AI6 is a system configured to mimic human comprehension by training an artificial neural network with a large number of parameters (e.g., billions to hundreds of billions) on large amounts of data. One example of generative AI6 is a large-scale language model. A large-scale language model can perform a wide range of natural language processing tasks, such as understanding sentence patterns and context, answering questions, generating sentences, and translating, according to given prompts. An example of a large-scale language model is the GPT series (such as Chat GPT) operated by OpenAI, a US company.

[0020] (Information processing device 1) The information processing device 1 is a device that performs processing to improve the accuracy of answers generated by the generation AI 6 in response to user inquiries. The information processing device 1 searches for answers to user questions using, for example, RAG (Retrieval Augmented Generation), instructs the generation AI 6 to generate answers, and acquires the answers. The information processing device 1 is configured to include one or more computers connected to a communication network 9. In the example of FIG. 1, the information processing device 1 has a communication unit 11, a storage unit 12, and a processing unit 13.

[0021] The communication unit 11 communicates with other devices (such as a server connected to a communication network) via a communication network such as the Internet. The communication unit 11 includes a device (such as a network interface card) that communicates in accordance with a predetermined communication standard such as Ethernet (registered trademark) or wireless LAN.

[0022] The storage unit 12 stores one or more programs 121 including instructions to be executed by the processing unit 13, data temporarily saved during processing by the processing unit 13, data used in processing by the processing unit 13, data obtained as a result of processing by the processing unit 13, etc. The storage unit 12 may include, for example, a main storage device (RAM, ROM, etc.) and an auxiliary storage device (flash memory, SSD, hard disk, memory card, optical disk, etc.). The storage unit 12 may be composed of one storage device or multiple storage devices. When the storage unit 12 is composed of multiple storage devices, each storage device is connected to the processing unit 13 via a computer bus or any other communication means.

[0023] The processing unit 13 controls the overall operation of the information processing device 1 and executes predetermined information processing. The processing unit 13 includes, for example, one or more processors (such as a central processing unit (CPU), a micro-processing unit (MPU), or a digital signal processor (DSP)) that perform processing according to instructions in one or more programs 121 stored in the storage unit 12. The processing unit 13 operates as a computer by the one or more processors executing instructions in one or more programs 121 stored in the storage unit 12.

[0024] The processing unit 13 may include one or more dedicated hardware (such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)) configured to realize a specific function. In this case, the processing unit 13 may execute all of the processes described in this embodiment on a computer, or may execute at least some of the processes on dedicated hardware.

[0025] The program 121 may be recorded, for example, on a computer-readable recording medium (such as an optical disc, a memory card, a USB memory, or other non-transitory tangible medium). The processing unit 13 may read at least a portion of the one or more programs 121 recorded on such a recording medium using a recording medium reading device (such as an optical disc device) or an interface device (such as a USB interface), not shown, and write the read at least a portion of the one or more programs 121 to the storage unit 12. Alternatively, the processing unit 13 may download at least a portion of the one or more programs 121 from another device connected to the communication network 9 via the communication unit 11 and write the downloaded program to the storage unit 12. The one or more programs 121 include instructions that cause the processing unit 13 to perform at least a portion of the processing according to this embodiment, which will be described later.

[0026] (Storage device 2) The storage device 2 stores various information used in the processing of the information processing device 1. The information processing device 1 and the storage device 2 can communicate with each other via any communication path (LAN, dedicated line network, Internet, etc.). For example, the storage device 2 may be included in a file server, database server, cloud server, etc. that accepts access from multiple devices, or may be a dedicated storage device that is accessible only to the information processing device 1. The storage device 2 may also be incorporated as a storage means of the information processing device 1. In the example of FIG. 1, the storage device 2 stores a knowledge database 21 and a feedback database 22. In the following description, database may be abbreviated to "DB."

[0027] The knowledge DB21 is a database of knowledge, experience, and know-how of employees of a company or organization. The knowledge DB21 includes information on FAQs (Frequently Asked Questions). FAQ information is used, for example, in a question and answer system on a company's homepage, and is information that processes frequently asked questions from users in the system and encourages them to solve their own problems. The knowledge DB21 may include FAQ information for internal use in addition to FAQ information for general customers.

[0028] The feedback DB 22 includes information on evaluations sent by users. For example, it includes information on user evaluations (accurate, inaccurate, good, average, bad, etc.) of answers generated by the generation AI in response to user inquiries. The information on user evaluations included in the feedback DB 22 includes information linked to the user ID, date and time, the user evaluation, and the answer that was the subject of the evaluation.

[0029] Here, the operation of the system having the above-described configuration shown in FIG. 1 will be described. Fig. 2 is a diagram illustrating an example of processing in a system including the information processing device 1. Fig. 2 shows an example of processing in which a user operating the terminal device 3 inputs a question (inquiry), and an answer to the question is generated by the generation AI 6 and sent to the user.

[0030] The terminal device 3 accesses the cloud server via the communication network 9 to receive predetermined processing and information (e.g., knowledge) from the cloud service (step S101). After accessing the cloud server 4, the user inputs a question into an input field displayed on the display unit of the terminal device 3 (step S102). The question input by the user is provided to the information processing device 1.

[0031] The information processing device 1 acquires the user's question provided from the terminal device 3 (step S103). The information processing device 1 transmits the user's question sent from the terminal device 3 to the search engine 5 via the communication network 9 (step S104).

[0032] The search engine 5 acquires the user's question sent from the information processing device 1 (step S105) and searches for an answer to the acquired question (step S106). For example, the search engine 5 performs a vector search using the knowledge DB 21 based on the user's question, and searches for an answer to the question by performing a similarity search or filtering. There may be multiple answers obtained by the search. The search engine 5 transmits the search results to the generation AI 6 (step S107).

[0033] The generation AI 6 acquires the search results of the search engine 5 (step S108) and generates an answer based on the acquired search results (step S109). For example, the generation AI 6 performs natural language processing on the search results of the search engine 5 using a large-scale language model, which is an example of the generation AI 6, to generate an answer. If multiple search results are sent from the search engine 5 to the generation AI 6, the generation AI 6 performs natural language processing based on the multiple search results to generate an answer. The generation AI 6 transmits the generated answer to the information processing device 1 via the communication network 9 (step S110).

[0034] The information processing device 1 acquires the answer transmitted from the generation AI 6 (step S111) and registers the answer in the knowledge DB 21 (step S112). The answer is registered in the knowledge DB 21 in association with the question from the user. The information processing device 1 transmits the answer acquired from the generation AI 6 to the terminal device 3 (step S113).

[0035] The terminal device 3 receives the answer from the generation AI 6 sent from the information processing device 1 and displays it on the display unit (step S114). Through this series of processes, the answer from the generation AI 6 to the question from the user is displayed on the terminal device 3 and is also registered in the knowledge DB 21. Therefore, every time a question is asked by the user, the answer to that question is accumulated as knowledge in the knowledge DB 21.

[0036] FIG. 3 is a diagram illustrating an example of the process of correcting an answer by the information processing device 1. In FIG. FIG. 3 shows an example of processing when an answer to a user's question is displayed on the terminal device 3 and then the user evaluates the answer.

[0037] With the answer to the user's question displayed on the display unit of the terminal device 3 (step S114), the user inputs an evaluation of the answer (step S201). The evaluation may be made by selecting preset buttons such as "good," "bad," "average," "correct," "inaccurate," "more details," "answer contains outdated information," or "answer is harmful information," or by inputting an evaluation comment in text. The evaluation input by the user is transmitted from the terminal device 3 to the information processing device 1 via the communication network 9.

[0038] The information processing device 1 acquires the user's evaluation sent from the terminal device 3 (step S202) and registers it in the feedback DB 22 (step S203). The information processing device 1 displays the answer that has received the acquired evaluation on the display unit (step S204). As a result, the display unit of the information processing device 1 displays the acquired evaluation, the answer that received the evaluation (the answer generated by the generation AI 6), and the question that was the basis for the answer (the user's inquiry). When the evaluated answer is displayed on the screen, the example answer that the generation AI 6 used to generate the answer, i.e., the example answer (another example answer) that is an example answer to the question searched by the search engine 5 and that the generation AI 6 used to generate the answer, may also be displayed.

[0039] The information processing device 1 examines and edits the displayed answer (step S205). The examination and editing of the answer is performed by a knowledge administrator. The information processing device 1 registers the edited answer in the knowledge DB 21 (step S206). Through this series of processes, the answer generated by the generation AI 6 based on the evaluation sent by the user is examined and registered in the knowledge DB 21 as a highly accurate answer. Therefore, the evaluation of the answer from the user is reflected in the answer, and the accuracy of the answers stored in the knowledge DB 21 is improved.

[0040] FIG. 4 is a diagram illustrating another example of the process of correcting an answer by the information processing device 1. In FIG. FIG. 4 shows another example of processing when an answer to a user's question is displayed on the terminal device 3 and then the user evaluates the answer.

[0041] With the answer to the user's question displayed on the display unit of the terminal device 3 (step S114), the user inputs an evaluation of the answer (step S301). The evaluation may be made by selecting preset buttons such as "good," "bad," "average," "correct," "inaccurate," "more details," "answer contains outdated information," or "answer is harmful information," or by inputting a comment in text. The evaluation input by the user is transmitted from the terminal device 3 to the information processing device 1 via the communication network 9.

[0042] The information processing device 1 acquires the user's evaluation sent from the terminal device 3 (step S302) and registers it in the feedback DB 22 (step S303). The information processing device 1 displays the answer that received the acquired evaluation on the display unit (step S304). As a result, the display unit of the information processing device 1 displays the acquired evaluation, the answer that received the evaluation (the answer generated by the generation AI 6), and the question that was the basis for the answer (the user's inquiry). Next, the information processing device 1 sends an instruction to the generation AI 6 to generate a correction candidate (step S305).

[0043] The generation AI 6 receives the generation instruction sent from the information processing device 1 (step S306) and generates a correction candidate (step S307). That is, the generation AI 6 generates another example answer as a correction candidate based on the acquired evaluation, the answer that received the evaluation (the answer before correction generated by the generation AI 6), and the question (user inquiry) that was the basis for the answer. For example, if the user's evaluation is "The answer contains old information," the generation AI 6 acquires information newer than the acquisition date and time of the original information that generated the answer before correction, and generates a new example answer using the new information. The generation AI 6 sends the generated correction candidate to the information processing device 1 (step S308).

[0044] The information processing device 1 acquires the correction candidates sent from the generation AI 6 (step S309) and examines and edits the answer based on the correction candidates (step S310). The examination and editing (correction) of the answer may be performed by a knowledge manager while referring to new answer candidates generated by the generation AI 6, or the new answer generated by the generation AI 6 may be automatically determined as the corrected answer. Furthermore, when accepting an answer edit, correction examples obtained by inputting different prompts into a different generation AI 6 or the same generation AI 6 may be displayed on the screen so that the corrector can select from them. Furthermore, past correction history for the evaluated answer may be displayed, or correction examples based on the correction history may be displayed on the screen. This allows accurate corrections to be made by referring to information that provides hints for correction, such as correction candidates from the generation AI 6 and past correction history.

[0045] The information processing device 1 registers the edited answer in the knowledge DB 21 (step S311). Through this series of processes, the answer generated by the generation AI 6 is scrutinized based on the evaluation sent by the user, and is registered as a highly accurate answer in the knowledge DB 21. Therefore, the evaluation of the answer from the user is reflected in the answer, and the accuracy of the answers stored in the knowledge DB 21 is improved.

[0046] (Screen display example) 5 to 9 are diagrams showing examples of screen displays for evaluating and correcting answers. FIG. 5(a) is a diagram showing an example of a screen displayed when a user evaluates an answer. The example screen shown in FIG. 5(a) displays the answer of the generated AI 6 to the user's question, as well as a field for inputting an evaluation of this answer. In the field for inputting the evaluation, an evaluation example button B1 is displayed, as shown in FIG. 5(b), and a text box TB for inputting a comment is displayed. The user can freely indicate their evaluation of the obtained answer by selecting button B1 or entering a comment in text box TB. After inputting the evaluation, when the user presses send button B2, the input evaluation is sent to the information processing device 1 and registered in the feedback DB 22.

[0047] Figure 6 is an example of a hallucination list and management screen display. A knowledge administrator using the information processing device 1 operates the information processing device 1 to refer to the hallucination list and management screen, which shows examples of user evaluations, and processes user feedback. The hallucination list displays, for each hallucination, the ID assigned to the user's question, the date and time the user's evaluation was made (report date), the user ID, the content of the answer, and the status. The administrator operates the operation buttons on this list screen to examine and revise the answer based on the evaluation. For example, for an answer with a status of "Not addressed," the administrator selects the "Details" button B3 and revise the answer as necessary. Furthermore, for answers with a status of "No response required," the administrator determines that no particular revision is required.

[0048] FIG. 7 is an example of a screen display for reviewing and correcting an answer. FIG. 8 is an enlarged view of the correction input field shown in FIG. 7. When reviewing and correcting a rated answer, for example, by selecting an operation button in the list display shown in FIG. 6, a correction screen (see FIG. 7) for the answer for which the operation button was selected is displayed. This screen displays the content of the user's question, the answer before correction, and the corrected answer. The administrator refers to the answer that has been reviewed by the user (the answer before correction), reviews it to see if it is an appropriate answer to the user's question, and enters the corrected answer in the answer input text box TB in the correction input field shown in FIG. 8.

[0049] An approval request button B4 may be provided to obtain approval from others for the revised content. When the approval request button B3 is selected, the answer is allowed to be revised only if the revised content has been approved by others. This allows the content of the revision to be double-checked.

[0050] Figure 9 is another example of a screen display for reviewing and correcting an answer. When reviewing and correcting a rated answer, for example, by selecting an operation button in the list display shown in Figure 6, a correction screen (see Figure 8) for the answer for which that operation button was selected is displayed. This screen displays the answer that has been rated by the user, the question on which it was based, and the date and time of the evaluation (the date and time of feedback), as well as an input field for correcting the answer. The administrator refers to the answer that has been rated by the user (the answer before correction), reviews whether it is an appropriate answer to the user's question, and enters the corrected answer in the text box TB for answer input.

[0051] Additionally, the correction input field includes a button B5 for displaying correction suggestions by the generation AI6. Selecting this button displays correction suggestions by the generation AI6 in the AI ​​correction suggestion display field A next to the answer input text box TB. The administrator can refer to the correction suggestions by the generation AI6 and adopt them as the corrected answer, or make further adjustments to create the corrected answer. Also, when generating correction suggestions by the generation AI6, different correction suggestions may be generated and displayed by different generation AI6s, or by the same generation AI6. This allows the administrator to select the appropriate one from multiple correction suggestions by the generation AI6. Once the corrections are complete, pressing the Save Corrections button B6 registers the corrected answer in the knowledge DB21.

[0052] As described above, according to this embodiment, it is possible to provide a method, a program, and an information processing device that can improve the accuracy of answers generated by a generation AI in response to user inquiries.

[0053] It should be noted that the present invention is not limited to the above-described embodiment, and includes various variations. Those skilled in the art can make various modifications and substitutions to the components of the above-described embodiment within the technical scope of the present invention or its equivalents.

[0054] Furthermore, some of the processes executed by the above-described information processing device 1 may be executed by another device (such as a server). In this case, it can be said that the processes of the information processing device according to this embodiment are executed by a plurality of computers including the processing unit 13 of the information processing device 1 and processing units (such as a processor) of the other device.

[0055] The following additionally describes the contents of the claims as originally filed in this application. [1] A method for performing a process in an information processing device to improve the accuracy of an answer generated by a generation AI (Artificial Intelligence) in response to a user inquiry, comprising: the information processing device is capable of accessing a storage device; The storage device stores information about example answers that the generation AI uses to generate the answer, receiving an evaluation from the user regarding the answer; displaying the evaluated answer on a screen; a step of accepting corrections to the answer displayed on the screen; and linking the corrected answer to the example inquiry and storing it in the storage device as information about the example answer. method. [2] The step of accepting a correction to the answer includes accepting approval for the corrected answer; the step of storing the corrected answer in the storage device includes storing the approved corrected answer in the storage device. [1] The method described in [1]. [3] The step of displaying the evaluated answer on a screen includes displaying the example answer used by the generation AI to generate the answer on the screen. [1] The method described in [1]. [4] The step of displaying the evaluated answer on a screen includes generating the example answer used by the generation AI to generate the answer based on the content of the evaluation. [1] The method described in [1]. [5] The evaluation includes at least one of the following evaluations: the answer is incorrect, the answer contains outdated information, and the answer contains harmful information. [1] The method described in [1]. [6] the step of accepting a correction of the answer includes displaying on the screen the answer before the correction and an input box for inputting the corrected answer; [1] The method described in [1]. [7] The step of accepting a correction to the answer includes displaying, on the screen, candidates for corrections to the answer by the generation AI. [1] The method described in [1]. [8] The step of accepting corrections to the answer includes displaying a screen that allows the corrector to select correction examples obtained by inputting different prompts to a different generation AI or the same generation AI. [1] The method described in [1]. [9] The step of accepting corrections to the answer includes displaying the correction examples based on a past correction history. [8] The method described in [8].

[10] The step of displaying the evaluated answers on a screen includes displaying a list of the evaluated answers and indicating a response status for each of the answers as a status. [1] The method described in [1].

[11] A program including instructions for causing an information processing device to perform a process for improving the accuracy of answers generated by a generation AI in response to user inquiries, The process for improving the accuracy of the answer includes the steps of any one of the methods described in [1] to

[10] . program.

[12] An information processing device that performs a process of improving the accuracy of an answer generated by a generation AI in response to a user's inquiry, a processing unit; a storage unit that stores instructions for causing the processing unit to perform processing; The processing performed by the processing unit in accordance with the instructions includes each step of the method described in any one of [1] to

[10] . Information processing device.

[13] An information processing device that performs a process of improving the accuracy of an answer generated by a generation AI in response to a user's inquiry, [1] to

[10] , comprising a means for performing each step of the method described in any one of [1] to

[10] , Information processing device. [Explanation of symbols]

[0056] 1...information processing device, 2...storage device, 3...terminal device, 4...cloud server, 5...search engine, 6...generative AI, 9...communication network, 11...communication unit, 12...storage unit, 13...processing unit, 21...knowledge database, 22...feedback database, 121...program, knowledge database (DB)...21, 22...feedback database (DB), A...AI revision proposal display field, B1...button, B2...send button, B3...button, B4...approval request button, B5...button, B6...button, TB...text box

Claims

1. A method for performing a process in an information processing device to improve the accuracy of an answer generated by a generation AI (Artificial Intelligence) in response to a user inquiry, comprising: the information processing device is capable of accessing a storage device; The storage device stores information on example answers used by the generation AI to generate the answer, receiving an evaluation from the user regarding the answer; A step of displaying on a screen the answer example used by the generation AI to generate the answer based on the content of the evaluation; a step of correcting the answer displayed on the screen and accepting approval of the corrected answer; and storing the approved corrected answer in the storage device as information on the example answer in association with the example inquiry, The method, wherein the step of accepting corrections to the answer includes displaying candidate correction examples for the answer on the screen by the generation AI.

2. The step of accepting corrections to the answer includes displaying a screen that allows the corrector to select correction examples obtained by inputting different prompts to a different generating AI or the same generating AI. The method of claim 1.

3. The step of accepting corrections to the answer includes displaying the correction examples based on a past correction history. The method of claim 2.

4. The step of displaying the evaluated answers on a screen includes displaying a list of the evaluated answers and indicating a response status for each of the answers as a status. The method of claim 3.

5. A program including instructions for causing an information processing device to perform a process for improving the accuracy of an answer generated by a generation AI in response to a user's inquiry, The process of improving the accuracy of the answer includes the steps of the method according to any one of claims 1 to 4. program.

6. An information processing device that performs a process of improving the accuracy of an answer generated by a generation AI in response to a user's inquiry, a processing unit; a storage unit that stores instructions for causing the processing unit to perform processing; The processing performed by the processing unit in accordance with the instructions includes each step of the method described in any one of claims 1 to 4. Information processing device.

7. An information processing device that performs a process of improving the accuracy of an answer generated by a generation AI in response to a user's inquiry, A method comprising: Information processing device.

Images