Method, program, and information processing device for improving the accuracy of responses generated by AI.

Abstract

To provide a method, program, and information processing device that can improve the accuracy of responses generated by a generation AI in response to user inquiries. [Solution] The present invention relates to a method in which an information processing device 1 performs a process to improve the accuracy of the answers generated by a generation AI in response to a user inquiry, wherein the information processing device 1 is able to access a storage device 2, and the storage device 2 stores information of example answers used by the generation AI 6 to generate answers, and includes the steps of receiving evaluations of the answers from the user (S202), displaying the evaluated answers on a screen (S204), receiving corrections to the answers displayed on the screen (S205), and storing the corrected answers in the storage device 2 as information of example answers, linked to the inquiry examples (S206).

Description

Technical Field

[0006] , , ,

[0005] , ,

[0001] The present invention relates to a method, a program, and an information processing apparatus for performing a process of improving the accuracy of answers by generative AI (Artificial Intelligence).

Background Art

[0002] Patent Document 1 discloses a method for assisting a conversation between a user and a chatbot that improves answer accuracy. In this method, a control unit of a server terminal receives a question from a user's user terminal, generates an answer to the question, receives an instruction to ask a follow-up question to the answer, and generates an answer different from the answer in response to the instruction.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a generative AI generates an answer to a user's inquiry, it is not always an appropriate answer. Therefore, it is important to be able to feedback the user's reaction to the answer.

[0005] The present invention has been made in view of such circumstances, and its object is to provide a method, a program, and an information processing apparatus capable of improving the accuracy of answers generated by a generative AI for a user's inquiry.

Means for Solving the Problems

[0007] A program according to a second aspect of the present invention is a program that includes an instruction to cause an information processing device to perform a process to improve the accuracy of the answers generated by a generation AI in response to a user inquiry, wherein the information processing device has access to a storage device, the storage device stores information of example answers used by the generation AI to generate answers, and the program includes each of the steps of a method that includes receiving evaluations of answers from the user, displaying the evaluated answers on a screen, receiving corrections to the answers displayed on the screen, and storing the corrected answers in the storage device as information of example answers, linked to the inquiry examples.

[0008] An information processing device according to a third aspect of the present invention is an information processing device that processes to improve the accuracy of answers generated by a generation AI in response to a user inquiry, and comprises a processing unit and a storage unit that stores instructions for the processing unit to perform processing, wherein in the processing performed by the processing unit in accordance with the instructions, the information processing device is able to access a storage device, the storage device stores information of example answers used by the generation AI to generate answers, and the information processing device includes each step of a method comprising: receiving evaluations of answers from the user; displaying the evaluated answers on a screen; receiving corrections to the answers displayed on the screen; and storing the corrected answers in the storage device as information of example answers, linked to the inquiry examples.

[0009] An information processing device according to a fourth aspect of the present invention is an information processing device that processes to improve the accuracy of answers generated by a generation AI in response to a user inquiry, wherein the information processing device is accessible to a storage device, the storage device stores information of example answers used by the generation AI to generate answers, and the information processing device is equipped with means for performing each step of a method comprising: receiving evaluations of answers from the user; displaying the evaluated answers on a screen; receiving corrections to the answers displayed on the screen; and storing the corrected answers in the storage device as example answer information linked to the inquiry example. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a method, program, and information processing device that can improve the accuracy of responses generated by a generation AI in response to user inquiries. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the configuration of a system to which the information processing device 1 according to this embodiment is applied. [Figure 2] Figure 2 illustrates an example of processing in a system including the information processing device 1. [Figure 3] Figure 3 illustrates an example of the process by which the information processing device 1 modifies the answer. [Figure 4] Figure 4 illustrates another example of the process by which the information processing device 1 modifies the answer. [Figure 5] Figure 5 shows an example of the screen display during the evaluation and correction of responses. [Figure 6] Figure 6 shows an example of the screen display during the evaluation and correction of responses. [Figure 7] Figure 7 shows an example of the screen display during the evaluation and correction of responses. [Figure 8] Figure 8 shows an example of the screen display during the evaluation and correction of responses. [Figure 9]FIG. 9 is a diagram showing an example of a screen display in the evaluation and correction of answers.

Embodiment for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described based on the drawings. In the following description, the same members are denoted by the same reference numerals, and the description of the members once described will be omitted as appropriate.

[0013] FIG. 1 is a diagram showing an example of the configuration of a system to which the information processing apparatus 1 according to the present embodiment is applied. The system shown in the example of FIG. 1 includes an information processing apparatus 1, a terminal device 3, a cloud server 4, a search engine 5, and a generation AI 6 that can communicate via a communication network 9 such as the Internet. Here, the information processing apparatus 1 is an example of the information processing apparatus of the present invention. The storage device 2 is an example of the storage device of the present invention. The terminal device 3 is an example of the terminal device of the present invention. The generation AI 6 is an example of the generation AI of the present invention.

[0014] In the system shown in FIG. 1, a process is performed in which the generation AI 6 generates an answer in response to an inquiry from a user operating the terminal device 3 and responds to the terminal device 3, and a process is performed in which an evaluation from the user with respect to the answer of the generation AI 6 is obtained, and feedback for correcting the answer based on the evaluation is performed.

[0015] (Terminal device 3) The terminal device 3 is a device operated by a user to make inquiries, ask questions, search for information, etc., and is a device equipped with an information communication function such as a personal computer, a tablet, or a smartphone. In the example of the system shown in FIG. 1, there is one terminal device 3, but the system may have a plurality of terminal devices 3 corresponding to a plurality of users.

[0016] The terminal device 3 includes, for example, a communication unit, a storage unit, and a processing unit similar to the communication unit 11, the storage unit 12, and the processing unit 13 of the information processing device 1 described later. The terminal device 3 may also include an input unit (such as a touch panel, a mouse, a keyboard, etc.) for inputting a user's instruction to the processing unit, a display unit (such as a liquid crystal display, an organic EL display, etc.) for displaying an image corresponding to the video data generated by the processing unit, a speaker for outputting sound, and the like.

[0017] (Cloud server 4) The cloud server 4 is a server for providing cloud services and is connected to the communication network 9. The cloud server 4 executes processing in response to requests from the terminal device 3 or the information processing device 1 and provides responses such as information. For example, for the digitization of workflows to improve business efficiency, the cloud server 4 provides various services in response to requests from terminal devices 3 operated by employees or customers. The cloud server 4 may also respond to requests from the information processing device 1 or requests from the terminal device 3 via the information processing device 1.

[0018] (Search engine 5) The search engine 5 is an information search system using search keys (such as keywords or key images) and is connected to the communication network 9. The search engine 5 performs similarity search and filtering by vector search using, for example, a vector database, and conducts highly accurate searches.

[0019] (Generative AI 6) The generative AI 6 is a system configured to mimic human understanding ability by training a large amount of data in an artificial neural network having a large number of parameters (for example, billions to hundreds of billions). An example of the generative AI 6 is a large language model. The large language model can perform a wide range of natural language processing tasks such as grasping the pattern and context of a sentence, responding to questions, generating sentences, and translation according to a given prompt. As the large language model, for example, the GPT series (such as Chat GPT) operated by OpenAI in the United States can be used.

[0020] (Information Processing Device 1) Information processing device 1 is a device that performs processing to improve the accuracy of the answers generated by generation AI 6 in response to user inquiries. Information processing device 1 performs tasks such as searching for answers to user questions using RAG (Retrieval Augmented Generation), instructing generation AI 6 to generate answers, and retrieving answers. Information processing device 1 is composed of one or more computers connected to a communication network 9. In the example in Figure 1, information processing device 1 has a communication unit 11, a storage unit 12, and a processing unit 13.

[0021] The communication unit 11 communicates with other devices (such as servers connected to the communication network) via a communication network such as the Internet. The communication unit 11 includes devices (such as network interface cards) that communicate in accordance with a predetermined communication standard, such as Ethernet (registered trademark) or wireless LAN.

[0022] The storage unit 12 stores one or more programs 121 containing instructions executed by the processing unit 13, data temporarily stored during processing by the processing unit 13, data used in processing by the processing unit 13, and data obtained as a result of processing by the processing unit 13. The storage unit 12 may include, for example, a main memory (RAM, ROM, etc.) and an auxiliary memory (flash memory, SSD, hard disk, memory card, optical disc, etc.). The storage unit 12 may consist of one memory device or multiple memory devices. If the storage unit 12 consists of multiple memory devices, each memory device is connected to the processing unit 13 via a computer bus or other arbitrary communication means.

[0023] The processing unit 13 comprehensively controls the overall operation of the information processing device 1 and executes predetermined information processing. The processing unit 13 includes, for example, one or more processors (such as a CPU (central processing unit), MPU (micro-processing unit), or DSP (digital signal processor)) that process according to the instructions of one or more programs 121 stored in the memory unit 12. The processing unit 13 operates as a computer by having one or more processors execute the instructions of one or more programs 121 stored in the memory unit 12.

[0024] The processing unit 13 may include one or more dedicated hardware components (such as an ASIC (application-specific integrated circuit) or an FPGA (field-programmable gate array)) configured to perform specific functions. In this case, the processing unit 13 may perform all the processing described in this embodiment on a computer, or at least some of the processing may be performed on dedicated hardware.

[0025] The program 121 may be recorded on a computer-readable recording medium (optical disc, memory card, USB memory, or other non-temporary tangible medium), for example. The processing unit 13 may read at least a portion of one or more programs 121 recorded on such a recording medium using a recording medium reading device (such as an optical disc drive) or interface device (such as a USB interface), and write it to the storage unit 12. Alternatively, the processing unit 13 may download at least a portion of one or more programs 121 from another device connected to the communication network 9 using the communication unit 11 and write it to the storage unit 12. One or more programs 121 include instructions that cause the processing unit 13 to perform at least a portion of the processing according to this embodiment, which will be described later.

[0026] (Storage device 2) The storage device 2 stores various types of information used in the processing of the information processing device 1. The information processing device 1 and the storage device 2 can communicate with each other via any communication channel (LAN, dedicated network, internet, etc.). For example, the storage device 2 may be included in a file server, database server, cloud server, etc. that accepts access from multiple devices, or it may be a dedicated storage device that can only be accessed by the information processing device 1. Alternatively, the storage device 2 may be incorporated as a storage means of the information processing device 1. In the example in Figure 1, the storage device 2 stores a knowledge database 21 and a feedback database 22. In the following description, databases may be abbreviated as "DB".

[0027] Knowledge DB21 is a database of the knowledge, experience, and know-how of employees and other personnel belonging to a company or organization. Knowledge DB21 includes FAQ (Frequently Asked Questions) information. FAQ information is used, for example, in question-and-answer systems on a company's website, and is information that processes frequently asked questions from users and encourages self-resolution. Knowledge DB21 may include FAQ information provided for general customers as well as FAQ information used internally within the company.

[0028] The Feedback DB22 contains evaluation information submitted by users. For example, it includes user evaluations (accurate, inaccurate, good, average, bad, etc.) of answers generated by the AI ​​in response to user inquiries. The user evaluation information included in the Feedback DB22 includes the user ID, date and time, the user's evaluation, and information linked to the answer that was the subject of that evaluation.

[0029] Here, we will explain the operation of the system shown in Figure 1, which has the configuration described above. Figure 2 illustrates an example of processing in a system including the information processing device 1. Figure 2 shows an example of processing in which a user operating the terminal device 3 inputs a question (inquiry), and the generation AI 6 generates an answer to that question and responds to the user.

[0030] Terminal device 3 accesses the cloud server via the communication network 9 to receive predetermined processing and information (e.g., knowledge) from the cloud service (step S101). After accessing the cloud server 4, the user enters a question into the input field displayed on the display unit of terminal device 3 (step S102). The question entered by the user is provided to the information processing device 1.

[0031] The information processing device 1 obtains the user's question provided by the terminal device 3 (step S103). The information processing device 1 transmits the user's question sent from the terminal device 3 to the search engine 5 via the communication network 9 (step S104).

[0032] The search engine 5 obtains the user's question sent from the information processing device 1 (step S105) and searches for an answer to the obtained question (step S106). For example, the search engine 5 performs a vector search using the knowledge DB 21 based on the user's question, and searches for an answer to the question by performing similarity searches and filtering. There may be multiple answers obtained by the search. The search engine 5 sends the search results to the generation AI 6 (step S107).

[0033] The generating AI 6 obtains the search results from the search engine 5 (step S108) and generates an answer based on the obtained search results (step S109). For example, a large-scale language model, which is an example of the generating AI 6, performs natural language processing on the search results from the search engine 5 and generates an answer. If multiple search results are sent to the generating AI 6 from the search engine 5, it performs natural language processing on the multiple search results and generates an answer. The generating AI 6 transmits the generated answer to the information processing device 1 via the communication network 9 (step S110).

[0034] The information processing device 1 obtains the answer sent from the generating AI 6 (step S111) and registers the answer in the knowledge database 21 (step S112). The answer is registered in the knowledge database 21, linked to the question from the user. The information processing device 1 sends the answer obtained from the generating AI 6 to the terminal device 3 (step S113).

[0035] Terminal device 3 receives the response from generating AI 6 transmitted from information processing device 1 and displays it on the display unit (step S114). Through this series of processes, the response from generating AI 6 to the user's question is displayed on terminal device 3 and registered in knowledge DB 21. Therefore, each time a user asks a question, the answer to that question is accumulated as knowledge in knowledge DB 21.

[0036] Figure 3 illustrates an example of the process by which the information processing device 1 modifies the answer. Figure 3 shows an example of the process that occurs when the answer to the user's question is displayed on terminal device 3, and the user then evaluates that answer.

[0037] With the answer to the user's question displayed on the display unit of terminal device 3 (step S114), the user enters an evaluation of the answer (step S201). The evaluation may be made by selecting pre-set buttons such as "Good," "Bad," "Average," "Correct," "Inaccurate," "More detail," "Answer contains outdated information," or "Answer contains harmful information," or by entering an evaluation comment as text. The evaluation entered by the user is transmitted from terminal device 3 to information processing device 1 via the communication network 9.

[0038] The information processing device 1 acquires the user's evaluation transmitted from the terminal device 3 (step S202) and registers it in the feedback DB 22 (step S203). The information processing device 1 displays the acquired evaluation-receiving response on the display unit (step S204). As a result, the display unit of the information processing device 1 displays the acquired evaluation, the evaluation-receiving response (the response generated by the generation AI 6), and the question (the user's inquiry) that the response is based on. When displaying the evaluated response on the screen, the generation AI 6 may also display example answers used to generate the response, i.e., example answers to questions searched by the search engine 5, which the generation AI 6 used to generate the response (another example answer).

[0039] The information processing device 1 scrutinizes and edits the displayed answer (step S205). The scrutinization and editing of the answer is performed by the knowledge administrator. The information processing device 1 registers the edited answer in the knowledge DB 21 (step S206). Through this series of processes, the answer generated by the generation AI 6 based on the evaluation sent by the user is scrutinized and registered in the knowledge DB 21 as a highly accurate answer. Therefore, the evaluation of the answer from the user is reflected in the answer, and the accuracy of the answers accumulated in the knowledge DB 21 improves.

[0040] Figure 4 illustrates another example of the process by which the information processing device 1 modifies the answer. Figure 4 shows another example of the process that occurs when the answer to the user's question is displayed on terminal device 3, and the user then evaluates that answer.

[0041] With the answer to the user's question displayed on the terminal device 3's display (step S114), the user enters an evaluation of the answer (step S301). The evaluation may be made by selecting pre-set buttons such as "Good," "Bad," "Average," "Correct," "Inaccurate," "More detail," "Answer contains outdated information," or "Answer contains harmful information," or by entering a comment as text. The evaluation entered by the user is transmitted from the terminal device 3 to the information processing device 1 via the communication network 9.

[0042] The information processing device 1 acquires the user's evaluation sent from the terminal device 3 (step S302) and registers it in the feedback DB 22 (step S303). The information processing device 1 displays the acquired evaluation and the resulting response on the display unit (step S304). As a result, the display unit of the information processing device 1 displays the acquired evaluation, the response that received that evaluation (the response generated by the generation AI 6), and the question (the user's inquiry) that the response was based on. Next, the information processing device 1 sends an instruction to the generation AI 6 to generate a candidate for correction (step S305).

[0043] The generating AI 6 receives a generation instruction from the information processing device 1 (step S306) and generates a candidate for correction (step S307). That is, the generating AI 6 generates an alternative answer example as a candidate for correction based on the acquired evaluation, the answer that received that evaluation (the answer before correction generated by the generating AI 6), and the question that was the basis for that answer (the user's inquiry). For example, if the user's evaluation is "The answer contains outdated information," the generating AI 6 acquires information newer than the acquisition date and time of the original information that generated the answer before correction, and generates a new example answer using the new information. The generating AI 6 sends the generated candidate for correction to the information processing device 1 (step S308).

[0044] The information processing device 1 acquires correction candidates sent from the generation AI 6 (step S309) and scrutinizes and edits the answer based on the correction candidates (step S310). The scrutinization and editing (correction) of the answer may be performed by the knowledge manager while referring to the new answer candidates generated by the generation AI 6, or the new answer generated by the generation AI 6 may be automatically determined as the corrected answer. When accepting an edit for an answer, the system may also display correction examples obtained from different generation AI 6s, or by entering different prompts into the same generation AI 6, so that the corrector can select from them. In addition, the system may display the past correction history for the evaluated answer, or display correction examples based on the correction history. This allows for highly accurate corrections to be made to the answer while referring to information that serves as a hint for correction, such as correction candidates from the generation AI 6 and past correction history.

[0045] The information processing device 1 registers the edited answer in the knowledge database 21 (step S311). Through this series of processes, the answer generated by the generation AI 6 based on the evaluation sent by the user is reviewed and registered in the knowledge database 21 as a highly accurate answer. Therefore, the evaluation of the answer from the user is reflected in the answer, and the accuracy of the answers accumulated in the knowledge database 21 improves.

[0046] (Screen display example) Figures 5 to 9 show examples of screen displays for evaluating and correcting responses. Figure 5(a) shows an example of a screen displayed when a user evaluates an answer. The example screen shown in Figure 5(a) displays the answer generated by the AI ​​6 to the user's question, along with a field for entering an evaluation of this answer. As shown in Figure 5(b), the evaluation input field displays an evaluation example button B1, as well as a text box TB for entering comments. The user can freely express their evaluation of the obtained answer by selecting button B1 or entering comments in text box TB. After entering the evaluation, when the user presses the submit button B2, the entered evaluation is sent to the information processing device 1 and registered in the feedback DB 22.

[0047] Figure 6 shows an example of the hallucination list and management screen display. Knowledge administrators using the information processing device 1 operate the information processing device 1 to refer to the hallucination list and management screen, which is an example of user evaluations, and process user feedback. The hallucination list displays the ID assigned to the user's question, the date and time the evaluation was made by the user (report date), the user ID, the content of the answer, and the status for each hallucination. Administrators use the operation buttons on this list screen to review and correct answers based on evaluations. For example, for answers with the status "Not yet addressed," they select the "Details" button B3 and correct the answer as necessary. For answers with the status "No action required," they determine that no particular correction is needed.

[0048] Figure 7 shows an example of a screen display for reviewing and correcting answers. Figure 8 is a magnified view of the correction input field shown in Figure 7. When reviewing and correcting an evaluated answer, for example, selecting an operation button in the list display shown in Figure 6 will display a correction screen for the selected answer (see Figure 7). This screen displays the content of the user's question, the answer before correction, and the corrected answer. The administrator refers to the answer evaluated by the user (the answer before correction), reviews whether it is an appropriate answer to the user's question, and enters the corrected answer in the answer input text box TB in the correction input field shown in Figure 8.

[0049] A button B4 may be provided to request approval from others for the revised content. If the approval request button B3 is selected, the response will only be allowed to be revised after approval from others. This ensures a double-check of the revised content.

[0050] Figure 9 shows another example of a screen display for reviewing and correcting answers. When reviewing and correcting an evaluated answer, for example, selecting an operation button in the list view shown in Figure 6 will display a correction screen for the selected answer (see Figure 8). This screen displays the answer evaluated by the user, the original question, and the evaluation date and time (feedback date and time), as well as an input field for correcting the answer. The administrator refers to the answer evaluated by the user (the answer before correction), reviews whether it is an appropriate answer to the user's question, and enters the corrected answer in the answer input text box TB.

[0051] Furthermore, the input field for corrections has a button B5 to display correction suggestions from the generating AI6. By selecting this button, correction suggestions from the generating AI6 are displayed in the AI ​​correction suggestion display field A, which is located next to the text box TB for answer input. The administrator can refer to the example corrections from the generating AI6 and adopt them as the corrected answer, or make further adjustments to create the corrected answer. In addition, when generating correction suggestions with the generating AI6, it is possible to have different generating AI6s, or the same generating AI6 generate and display different correction suggestions. This allows the administrator to select the most suitable one from multiple correction suggestions from the generating AI6. Once the corrections are complete, pressing the save correction button B6 registers the corrected answer in the knowledge DB21.

[0052] As described above, this embodiment makes it possible to provide a method, program, and information processing device that can improve the accuracy of the responses generated by the AI ​​in response to user inquiries.

[0053] It should be noted that the present invention is not limited to the embodiments described above, but includes various variations. Those skilled in the art can make various changes and substitutions to the components of the embodiments described above within the technical scope of the present invention or its equivalents.

[0054] Furthermore, some of the processing performed by the information processing device 1 described above may be performed by other devices (such as servers). In this case, it can be said that the processing as an information processing device according to this embodiment is performed by multiple computers, including the processing unit 13 of the information processing device 1 and the processing units (such as processors) of other devices.

[0055] The following is an addendum to the claims as originally filed for this application. [1] A method for improving the accuracy of responses generated by artificial intelligence (AI) in response to user inquiries using an information processing device, The aforementioned information processing device is capable of accessing the storage device, The memory device stores information of example answers that the generating AI uses to generate the answers. A step of receiving evaluations from the user regarding the aforementioned response, The steps include displaying the aforementioned response, which has received the aforementioned evaluation, on the screen, A step of accepting corrections to the answer displayed on the aforementioned screen, The system includes the step of associating the modified response with the example query and storing it in the storage device as information of the example query, method. [2] The step of accepting revisions to the aforementioned response includes accepting approval for the revised response. The step of storing the modified response in the storage device includes storing the approved modified response in the storage device. The method described in [1]. [3] The step of displaying the evaluated response on the screen includes displaying the example response used by the generating AI to generate the response on the screen. The method described in [1]. [4] The step of displaying the evaluated response on the screen includes generating the example response used by the generating AI for generating the response, based on the content of the evaluation. The method described in [1]. [5] The aforementioned evaluation includes at least one of the following evaluations: the answer is incorrect, the answer contains outdated information, or the answer contains harmful information. The method described in [1]. [6] The step of accepting corrections to the aforementioned answer includes displaying the original answer and an input box for entering the corrected answer on the screen. The method described in [1]. [7] The step of accepting corrections to the aforementioned answer includes displaying candidate correction examples of the aforementioned answer on the screen using the generating AI. The method described in [1]. [8] The step of accepting corrections to the aforementioned response includes displaying a screen in which the corrector can select from corrected examples obtained by inputting different prompts into the same generation AI, or into different prompts into the same generation AI. The method described in [1]. [9] The step of accepting corrections to the above response includes displaying examples of the above corrections based on past correction history. The method described in [8].

[10] The step of displaying the evaluated responses on the screen includes displaying a list of the evaluated responses and indicating the status of the response for each response as a status, The method described in [1].

[11] A program that includes instructions to cause an information processing device to perform a process to improve the accuracy of the answers generated by a generation AI in response to user inquiries, The process for improving the accuracy of the above answer includes each step of the method described in any one of [1] to

[10] , program.

[12] An information processing device that processes the accuracy of the answers generated by a generation AI in response to user inquiries, Processing unit and It has a storage unit that stores commands to cause the processing unit to perform processing, The processing performed by the processing unit in accordance with the instruction includes each step of the method described in any one of [1] to

[10] , Information processing device.

[13] An information processing device that processes the accuracy of the answers generated by a generation AI in response to user inquiries, The means for performing each step of the method described in any one of [1] to

[10] , Information processing device. [Explanation of Symbols]

[0056] 1...Information processing device, 2...Storage device, 3...Terminal device, 4...Cloud server, 5...Search engine, 6...Generative AI, 9...Communication network, 11...Communication unit, 12...Storage unit, 13...Processing unit, 21...Knowledge database, 22...Feedback database, 121...Program, Knowledge database (DB)...21, 22...Feedback database (DB), A...AI revision suggestion display field, B1...Button, B2...Send button, B3...Button, B4...Approval request button, B5...Button, B6...Button, TB...Text box

Claims

1. A method for improving the accuracy of responses generated by artificial intelligence (AI) in response to user inquiries using an information processing device, The aforementioned information processing device is capable of accessing the storage device, The storage device stores information of example answers that the generating AI uses to generate the answers. A step of receiving evaluations from the user regarding the aforementioned response, The steps include displaying the aforementioned response, which has received the aforementioned evaluation, on the screen, A step of accepting corrections to the answer displayed on the aforementioned screen, The system includes the step of associating the modified response with the example query and storing it in the storage device as information of the example query, method.

2. The step of accepting revisions to the aforementioned response includes accepting approval for the revised response. The step of storing the modified response in the storage device includes storing the approved modified response in the storage device. The method according to claim 1.

3. The step of displaying the evaluated response on the screen includes displaying the example response used by the generating AI for generating the response on the screen. The method according to claim 2.

4. The step of displaying the evaluated response on the screen includes generating the example response used by the generating AI for generating the response, based on the content of the evaluation. The method according to claim 3.

5. The aforementioned evaluation includes at least one of the following evaluations: the answer is incorrect, the answer contains outdated information, or the answer contains harmful information. The method according to claim 4.

6. The step of accepting corrections to the aforementioned answer includes displaying the original answer and an input box for entering the corrected answer on the screen. The method according to claim 5.

7. The step of accepting corrections to the aforementioned answer includes displaying candidate correction examples of the answer on the screen using the generating AI. The method according to claim 6.

8. The step of accepting corrections to the aforementioned response includes displaying a screen in which the corrector can select from corrected examples obtained by inputting different prompts into the same generation AI, or by inputting different prompts into the same generation AI. The method according to claim 7.

9. The step of accepting corrections to the above response includes displaying examples of the above corrections based on past correction history. The method according to claim 8.

10. The step of displaying the evaluated responses on the screen includes displaying a list of the evaluated responses and indicating the status of the response for each response as a status, The method according to claim 9.

11. A program that includes instructions to cause an information processing device to perform a process to improve the accuracy of the answers generated by the AI ​​in response to user inquiries, The process for improving the accuracy of the aforementioned answer includes each step of the method described in any one of claims 1 to 10. program.

12. An information processing device that processes the responses generated by a generation AI in response to user inquiries, Processing unit and It has a storage unit that stores commands to cause the processing unit to perform processing, The processing performed by the processing unit in accordance with the instruction includes each step of the method described in any one of claims 1 to 10. Information processing device.

13. An information processing device that processes the accuracy of the responses generated by the AI ​​in response to user inquiries, The method comprises means for performing each step of the method described in any one of claims 1 to 10. Information processing device.

Images