Interactive device, program, and method for generating response sentence to user

Abstract

To provide an interactive device and the like that create a response sentence obtained by rewriting a sentence with factual errors.SOLUTION: An interactive device that creates a response sentence following an interaction history with a user includes: draft response sentence generation means that transmits, to a first database, an instruction sentence instructing creation of a draft response sentence from the interaction history with the user, and receives the draft response sentence; strategy maintenance means that extracts element sentences from the draft response sentence, creates a strategy sentence inherent in each element sentence, transmits, to the first database, an instruction sentence instructing creation of a strategy query for a second database from the element sentences and the strategy sentences, receives the strategy query, transmits the strategy query to the second database, and receives related documents; and response sentence generation means that transmits, to the first database, an instruction sentence instructing creation of the response sentence following the interaction history by citing the strategy sentences and the related documents, and receives the response sentence.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for a dialogue device that generates a response sentence according to a dialogue history with a user. [Background technology]

[0002] Dialogue devices are expected to use AI (Artificial Intelligence) to respond to user utterances with natural-sounding responses, thereby increasing the user's willingness to engage in dialogue. On the other hand, dialogue devices also need to strengthen their knowledge in the responses they provide to users. In this case, it is important that the responses generated by the dialogue device in response to user utterances do not contain any knowledge or factual errors.

[0003] There is a conventional pipeline technology called WikiChat (registered trademark) (see, for example, Non-Patent Document 1). According to this technology, references to factual information are extracted in units called "claims" from response sentences generated by large language models. Each extracted claim is verified while obtaining related information from a corpus, and if it is incorrect, it is deleted.

[0004] There is also a technology that checks for points where factual errors are likely to occur while referring to probability values ​​when generating tokens from a large-scale language model (see, for example, Non-Patent Document 2). With this technology, information related to points with low probability values ​​(uncertainty) is obtained from the web or a corpus, and verification and correction are performed. For example, if the response draft sentence "Rick Mahler was born in Jupiter, Florida," contains a factual error about Rick Mahler being born in Jupiter, Florida, verification will correct it to "Rick Mahler was born in Austin, Texas."

[0005] There is a document proofreading device technology that checks whether factual statements in a text document are accurate by checking the accuracy of the statements against factual data in an existing database, and detects and corrects errors in the text document (see, for example, Patent Document 1). According to this technology, the document proofreading device compares a factual data database that stores records related to specific items with the factual data from the input text, and corrects expressions in the text that are inconsistent. The records are organized in a specific structure, such as organization names, job titles, and personal names. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 11-167576 [Non-patent literature]

[0007] [Non-Patent Document 1] Sina Semnani, Violet Yao, Heidi Zhang, and Monica Lam, 2023, WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia, In Findings of EMNLP, pages 2387-2413. [online], [Retrieved June 1, 2024], Internet<URL:https: / / aclanthology.org / 2023.findings-emnlp.157.pdf> [Non-patent document 2] Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu, 2023, A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation, arXiv. [online], [Retrieved June 1, 2024], Internet<URL:https: / / arxiv.org / pdf / 2307.03987> Summary of the Invention [Problem to be solved by the invention]

[0008] A dialogue device can respond to a user's utterance with a response sentence that contains a lot of knowledge. In response to this, the user generally recognizes that the response sentence generated by the dialogue device is correct. However, the knowledge contained in the response sentence may contain factual errors, making it difficult to determine whether or not the response sentence is correct.

[0009] According to Non-Patent Document 1, when a factual error is detected, the relevant part is simply deleted. In this case, the large-scale language model's ability to perform the task (task-oriented dialogue) that it is intended to accomplish for the user is reduced. For example, suppose a large-scale language model is trying to respond to a user with a touching story about a girl who was saved by a charity organization in a task to persuade the user to make a donation. If this is not true, the description will simply be deleted, resulting in the loss of effective persuasive material in the dialogue (losing its strategic function for the task).

[0010] Furthermore, according to Non-Patent Document 2, when searching the web or a corpus for a passage where a factual error has been detected, further incorrect information may be retrieved. This occurs when the factual data does not contain any records related to it. For example, if a record for "Rick Mahler" is not included and a similar record for "Rick Mahorn" is retrieved, the information will be corrected to match the information for Rick Mahorn, and the topic will also be changed. In other words, the information will be rewritten with information that is out of context.

[0011] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a dialogue device, program and method for creating a response sentence by rewriting a sentence containing a factual error. [Means for solving the problem]

[0012] According to the present invention, there is provided a dialogue device that uses a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, to create a response sentence following a dialogue history with a user, the dialogue device comprising: a response draft generation means for transmitting an instruction to a first database to generate a response draft from a dialogue history with a user, and for receiving the response draft; a strategy maintenance means for extracting element sentences from the answer draft sentences, creating an inherent strategy sentence for each element sentence, transmitting an instruction sentence to the first database instructing the creation of a strategy query for the second database from the element sentences and the strategy sentence, receiving the strategy query, and transmitting the strategy query to the second database and receiving related documents; a response sentence generating means for transmitting an instruction sentence to the first database, instructing the user to create a response sentence following the dialogue history by citing the strategy sentence and related documents, and for receiving the response sentence; The present invention is characterized by having the following.

[0013] According to another embodiment of the interactive device of the present invention, The means of maintaining the strategy are: a strategy sentence extraction means for extracting element sentences from the response plan sentence, transmitting an instruction sentence instructing to create a strategy sentence inherent in each element sentence to a first database, and receiving the strategy sentence for each element sentence; a related document search means for transmitting to the first database an instruction statement instructing the creation of a strategic query for the second database from the element sentence and the strategic sentence, receiving the strategic query, and transmitting the strategic query to the second database and receiving related documents; It is also preferred that the

[0014] According to another embodiment of the interactive device of the present invention, The first database may be a large language model or a rule-based database.

[0015] According to another embodiment of the interactive device of the present invention, The second database may be a search database that accumulates documents that are correct facts, or an information corpus for natural language processing that accumulates text data that are correct facts.

[0016] According to another embodiment of the interactive device of the present invention, The means of maintaining the strategy are: The system further comprises a fact verification means for determining whether the element sentences extracted by the strategic sentence extraction means are valid, unclear, or invalid using the first database and the second database, The related document search means processes only element sentences for which the judgment result is unclear or invalid. It is also preferable.

[0017] According to another embodiment of the interactive device of the present invention, The means of verifying facts are transmitting an instruction statement instructing a first database to create a claim statement from the element statement, and receiving the claim statement; sending instructions to the first database for each assertion to generate a search query for the second database; and receiving the search query; submitting the search query to a second database and receiving relevant documents; For each relevant document, sending instructions to the first database citing the search query to extract relevant information, and receiving the relevant information; For each assertion, a directive citing related information and instructing the assertion to be judged as valid, unclear, or invalid is transmitted to the first database, and the judgment result is received. It is also preferable.

[0018] According to another embodiment of the interactive device of the present invention, a query generation means for transmitting to the first database an instruction statement instructing the first database to generate a query for the second database from a dialogue history with the user, and for receiving the query; a query-related document search means for sending the query to a second database and receiving query-related documents; and The response sentence generating means transmits an instruction sentence to the first database, instructing the first database to generate a response sentence from the dialogue history, citing the strategy sentence and the question-related document, and receives the response sentence. It is also preferable.

[0019] According to the present invention, there is provided a program for causing a computer to function to create a response sentence following a dialogue history with a user, using a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, the program comprising: a response draft generation means for transmitting an instruction to a first database to generate a response draft from a dialogue history with a user, and for receiving the response draft; a strategy maintenance means for extracting element sentences from the answer draft sentences, creating an inherent strategy sentence for each element sentence, transmitting an instruction sentence to the first database instructing the creation of a strategy query for the second database from the element sentences and the strategy sentence, receiving the strategy query, and transmitting the strategy query to the second database and receiving related documents; a response sentence generating means for transmitting an instruction sentence to the first database, instructing the user to create a response sentence following the dialogue history by citing the strategy sentence and related documents, and for receiving the response sentence; The present invention is characterized by the fact that it makes a computer function by using the above-mentioned method.

[0020] According to the present invention, there is provided a method for a device that uses a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, to create a response sentence following a dialogue history with a user, the method comprising: The device is a first step of transmitting an instruction to a first database to generate a response draft from a dialogue history with a user, and receiving the response draft; a second step of extracting element sentences from the answer draft sentence, creating an inherent strategy sentence for each element sentence, sending an instruction sentence to the first database instructing the first database to create a strategy query for a second database from the element sentences and the strategy sentence, receiving the strategy query, and sending the strategy query to the second database and receiving factual related documents; a third step of sending an instruction to the first database, citing the strategy statement and related documents and instructing the first database to create a response statement following the dialogue history, and receiving the response statement; The present invention is characterized by carrying out the following. [Effects of the Invention]

[0021] According to the dialogue device, program and method of the present invention, it is possible to create a response sentence by rewriting a sentence containing a factual error. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating the configuration of a dialogue system and an example dialogue including a factual error. [Figure 2] 1 is a diagram showing a basic functional configuration of a dialogue device according to the present invention; [Figure 3] 10 is an example of a prompt from the response draft generator of the present invention. [Figure 4]10 is an example of a prompt of the strategy statement extraction unit in the strategy maintenance unit of the present invention. [Figure 5] 10 is an example of a prompt for the related document search unit in the strategy maintenance unit of the present invention. [Figure 6] 10 is a diagram showing an example of a search performed by a related document search unit in the strategy maintenance unit of the present invention. [Figure 7] 10 is an example of a prompt from the response sentence generation unit in the present invention. [Figure 8] 2 is an interactive example of the present invention with factual errors redrawn relative to FIG. 1. [Figure 9] FIG. 2 is a second functional configuration diagram of the dialogue device according to the present invention. [Figure 10] 1 is a first example prompt and search example of a fact verification unit in a strategy maintenance unit of the present invention. [Figure 11] 10 is a second example prompt of the fact verification unit in the strategy maintenance unit of the present invention. [Figure 12] 10 is a third example prompt of the fact verification unit in the strategy maintenance unit of the present invention. [Figure 13] 10 shows example prompts and search results for the related document search section following the fact verification section of the present invention. [Figure 14] 14 shows examples of prompts from the response sentence generation unit in response to FIGS. 10 to 13. [Figure 15] FIG. 3 is a third functional configuration diagram of the dialogue device according to the present invention. [Figure 16] 1 shows an example of a prompt from a query generation unit and an example of a search from a question-related document search unit in the present invention. [Figure 17] 17 is an example of a prompt from the response sentence generator in response to FIG. 16. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0024] FIG. 1 is an explanatory diagram of the configuration of a dialogue system and an example dialogue that includes a factual error.

[0025] The dialogue system of FIG. 1 includes a dialogue device 1, a terminal 2, and a large-scale language model 3 (first database). The terminal 2 transmits the user's utterance to the dialogue device 1. The dialogue device 1 creates a response sentence that will result in a natural dialogue in response to the utterance sentence, and returns the response sentence to the terminal 2. At this time, the dialogue device 1 creates the response sentence using a database such as a large-scale language model 3, for example. Then, the terminal 2 returns the response sentence to the user by voice or text.

[0026] The terminal 2 may have, as a user interface, a speech recognition function for the user's speech and a speech synthesis function for generating a response to the user. The speech recognition function converts the user's speech captured by the microphone of the terminal 2 into a text-based speech. The speech synthesis function converts the response generated by the dialogue device 1 into a voice signal. Of course, the voice recognition function and the voice synthesis function may be installed in the terminal 2 or in the dialogue device 1.

[0027] The large-scale language model 3 may be an existing generative AI as the first database. Specific examples of large-scale language models3 include ChatGPT-xx (registered trademark) from OpenAI (registered trademark), Gemini (registered trademark) from Google (registered trademark), and Claude (registered trademark) from Antheropic (registered trademark). These models use Transformer to improve inference accuracy for tasks such as text generation and language understanding. They are built by training a deep neural network with more than billions of parameters using a large dataset, and are capable of generating natural-sounding sentences. In the embodiments of the present invention, the first database is described as a large-scale language model, but it can also be a rule-based database, i.e., a database that inputs instructions and outputs answers.

[0028] According to the example dialogue in Figure 1, the dialogue is as follows: -------------------------------------------------------------------------------- User: Hello. Agent: Hello! Would you consider donating to Save the Children? Your donation can make a big impact on the lives of children in need. User: Please tell us about Save the Children's activities. Agent: Save the Children works to save children all over the world. For example, thanks to our work, a young girl named Maria was saved from poverty and is now able to attend school. Would you consider making a donation? --------------------------------------------------------------------------------

[0029] Here, when a large-scale language model (external database) is referenced as a response sentence for a dialogue with a user, there is a possibility that factual errors (hallucination) may be included. According to the embodiment described below, in the example dialogue of FIG. 1, for example, the statement "For example, thanks to our activities, a young girl named Maria was saved from poverty and is now able to attend school" is explained as being a factual error.

[0030] FIG. 2 is a diagram showing the basic functional configuration of the dialogue device according to the present invention.

[0031] 2, the dialogue device 1 uses a first database (e.g., a large-scale language model) for inputting instructions and outputting answers, and a second database for inputting queries and outputting related documents, to create a "response sentence" following the dialogue history with the user. At this time, the dialogue device 1, as an agent for the user, generates a response sentence that suppresses factual errors, taking into account a "task." A task is intended to be a goal that the agent should achieve.

[0032] Specifically, the second database may be a search database that accumulates documents that serve as correct facts, or a natural language processing information corpus that accumulates text data that serve as correct facts. The search database may be Google (registered trademark), a common search site. Furthermore, the natural language processing information corpus may be a large-scale corpus constructed from Wikipedia (registered trademark), for example. In other words, the second database is a publicly available dataset that has been manually compiled and is assumed to be factually correct. The dialogue device 1 of FIG. 2 has an information corpus database 100 as a second database.

[0033] 2, the dialogue device 1 includes a user utterance receiving unit 10, a response draft generation unit 11, a strategy maintenance unit 12, and a response sentence generation unit 13. These functional components are realized by executing a program that causes a computer installed in the device to function. The processing flow of these functional components can also be understood as a dialogue method.

[0034] [User utterance receiving unit 10] The user utterance receiving unit 10 receives a user utterance from the terminal 2. The received utterance is output to the response draft generating unit 11.

[0035] [Response draft generation unit 11] The response draft generation unit 11 transmits a prompt (instruction sentence) instructing the large-scale language model 3 to create a "response draft" from the dialogue history with the user, and receives the "response draft."

[0036] FIG. 3 is an example of a prompt for the response draft generator in the present invention.

[0037] (Prompt sent from response draft generator 11) -------------------------------------------------------------------------------- You are an agent who wants to convince users to donate to Save the Children. Create a response according to the dialogue history. #Dialogue history User: Hello. Agent: Hello! Would you consider donating to Save the Children? Your donation can make a big impact on the lives of children in need. User: Please tell us about Save the Children's activities. -------------------------------------------------------------------------------- Here, "You are an agent whose job is to persuade users to donate to Save the Children" represents a "task" for the agent. The agent requests a response that is optimal for this task from the large-scale language model 3.

[0038] (Response draft returned by large-scale language model 3) -------------------------------------------------------------------------------- Save the Children works to save children all over the world. For example, thanks to our work, a young girl named Maria was saved from poverty and is now able to attend school. Would you consider making a donation? -------------------------------------------------------------------------------- This response draft contains a "strategy" for accomplishing the task of "an agent that persuades users to donate to Save the Children."

[0039] [Strategic Maintenance Division 12] The strategy maintenance unit 12 extracts "element sentences" from the response draft sentences, creates "strategy sentences" inherent in each element sentence, and sends an instruction statement to the large-scale language model 3 (first database) instructing it to create a "strategy query" for the information corpus database 100 (second database) from the element sentences and strategy sentences. In response to this, the strategy maintenance unit 12 receives the "strategy query" from the large-scale language model 3, transmits the strategy query to the information corpus database 100, and receives the "related documents."

[0040] Here, the strategy maintenance unit 12 is configured as a strategy statement extraction unit 121 and a related document search unit 122 as follows.

[0041] (Strategy statement extraction unit 121) The strategy sentence extraction unit 121 extracts "element sentences" from the response proposal sentences, sends a prompt (instruction sentence) to the large-scale language model 3 instructing it to create a "strategy sentence" inherent in each element sentence, and receives the "strategy sentence" for each element sentence.

[0042] FIG. 4 shows an example of a prompt of the strategy statement extraction unit in the strategy maintenance unit of the present invention.

[0043] (Prompt sent from the strategy statement extraction unit 121) -------------------------------------------------------------------------------- Please refer to the dialogue history and provide the "element sentences" of the proposed response and the "strategy" as an agent. ========== #Dialogue history ... #Response draft ... -------------------------------------------------------------------------------- Note that a prompt may be written in JSON format to break down a response plan into one or more element sentences (Parts) and assign a strategy label to each element sentence.

[0044] (Element sentences and strategy sentences returned by large-scale language model 3) -------------------------------------------------------------------------------- Element 1: "Save the Children works to save children all over the world." Strategy 1: “Share your organization's reach and impact” Element 2: "For example, Maria, a young girl, was saved from poverty through our work and is now able to attend school." Strategy 2: “Telling your impact story” Element 3: "Would you consider making a donation?" Strategy 3: “Ask for donations” -------------------------------------------------------------------------------- Here, the strategy statement is the intention, or "strategy," that describes the element statements to accomplish the task of "an agent that persuades users to donate to Save the Children."

[0045] (Related document search unit 122) The related document search unit 122 sends a prompt to the large-scale language model 3, instructing it to create a "strategic query" for the information corpus database 100 from the element sentences and the strategic sentences. The strategic query means a search key. In response to this, the related document search unit 122 receives the "strategic query" from the large-scale language model 3, and also transmits the strategic query to the information corpus database 100 and receives the "related documents."

[0046] FIG. 5 is an example of a prompt for the related document search unit in the strategy maintenance unit of the present invention.

[0047] (Prompt sent from related document search unit 122) -------------------------------------------------------------------------------- Create a strategy query to search the information corpus database using the element sentences and strategy sentences below. ========= Element 1: Strategy 1: Element 2: Strategy 2: .... --------------------------------------------------------------------------------

[0048] (Strategy query answered by large-scale language model 3) -------------------------------------------------------------------------------- Strategic Query 1: Save the Children organization Strategic Query 2: Inspiring stories of children saved by Save the Children Strategic Query 3: The effect of donating to Save the Children --------------------------------------------------------------------------------

[0049] FIG. 6 shows an example of a search performed by the related document search unit in the strategy maintenance unit of the present invention.

[0050] The related document search unit 122 sends each strategic query to the information corpus database 100 to search for related documents such as the following: (Strategic Query 1: Save the Children organization) (Related Documents 1) -------------------------------------------------------------------------------- Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries and continues to work towards a world where "children's rights" are realized. -------------------------------------------------------------------------------- (Strategic Query 2: Inspiring stories of children saved by Save the Children) (Related Documents 2) -------------------------------------------------------------------------------- One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. -------------------------------------------------------------------------------- (Strategic Query 3: The effect of donating to Save the Children) (Related Documents 3) -------------------------------------------------------------------------------- Your donation will help many children... --------------------------------------------------------------------------------

[0051] It is also preferable that the information corpus database 100 records the search key (strategy query) and its related documents from the related document search unit 122. For example, the information corpus database 100 may flag related documents that are required for maintaining the strategy during a dialogue with the user. By feeding back this information to the operator, it can be useful for building and updating the information corpus database 100.

[0052] [Response sentence generation unit 13] The response sentence generation unit 13 quotes the "strategy sentence" and "related documents" and sends a prompt to the large-scale language model 3 instructing it to create a "response sentence" that follows the dialogue history, and receives the "response sentence."

[0053] FIG. 7 shows an example of a prompt from the response sentence generator in the present invention.

[0054] According to FIG. 7, the response sentence generator 13 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- Based on the relevant documentation from the strategy, follow the dialogue history and create a response that will persuade them to donate to Save the Children. ======= #Dialogue history ... #strategy Strategy 1: Strategy 2: Strategy 3: #Strategy 1 Related Document 1: ... #Strategy 2 Related Document 2: ... #Strategy 3 Related Document 3: ... --------------------------------------------------------------------------------

[0055] It is also a good idea to include a word limit at the end of the prompt, such as "The response should be approximately 300 words." This is to prevent the generated response from being excessively long compared to the original response. In particular, the prompt may cite long related documents, and if the response is inferred from these documents, the response will tend to be excessively long.

[0056] According to FIG. 7, the large-scale language model 3 returns, for example, the following response sentence: -------------------------------------------------------------------------------- Save the Children works to save children in over 120 countries. One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. Your donation can help many children. --------------------------------------------------------------------------------

[0057] Figure 8 is an example of a dialogue of the present invention in which factual errors have been rewritten from Figure 1. When comparing Figure 1 and Figure 8, it is worth noting that the descriptions in [ ] have been rewritten. -------------------------------------------------------------------------------- ... User: Please tell us about Save the Children's activities. Agent: Save the Children works to save children [in about 120 countries]. One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. Your donation can help save many children. --------------------------------------------------------------------------------

[0058] FIG. 9 is a second functional configuration diagram of the dialogue device according to the present invention.

[0059] 9 further includes a fact verification unit 123 in the strategy maintenance unit 12, as compared with the dialogue device 1 in FIG. The fact verification unit 123 uses the large-scale language model 3 (first database) and the information corpus database 100 (second database) to determine whether the element sentences extracted from the response draft sentences by the strategy sentence extraction unit 121 (strategy maintenance unit 12) are valid, unclear, or invalid. Subsequently, the related document search unit 122 processes only the element sentences for which the judgment result is unclear or invalid.

[0060] [Fact-Verification Section 123] The fact verification unit 123 executes the following five steps. (S1) A prompt instructing the creation of an "assertion sentence" from an "element sentence" based on a proposed response sentence is sent to the large-scale language model 3, and the "assertion sentence" is received. (S2) For each assertion sentence, a prompt instructing the creation of a "search query" for the information corpus database 100 is sent to the large-scale language model 3, and the "search query" is received. (S3) The search query is sent to the information corpus database 100, and "related documents" are received. (S4) For each relevant document, a prompt citing the search query and instructing the large-scale language model 3 to extract relevant information is sent to the large-scale language model 3, and the "related information" is received. (S5) For each assertion, a prompt citing related information and instructing the large-scale language model 3 to judge whether the assertion is valid, unclear, or invalid is sent to the large-scale language model 3, and the judgment result is received. Each step is described in detail below.

[0061] (S1) The fact verification unit 123 sends a prompt to the large-scale language model 3 instructing it to create an "argument sentence" from an "element sentence" based on a response draft sentence, and receives the "argument sentence."

[0062] FIG. 10 is a first example prompt and search of the fact verification unit in the strategy maintenance unit of the present invention.

[0063] According to S1 in FIG. 10, the fact verification unit 123 sends the following prompt to the large-scale language model 3, for example: -------------------------------------------------------------------------------- Extract the assertion from the following sentences. ================ Element 1: Element 2: Element 3: -------------------------------------------------------------------------------- In response to this, the fact verification unit 123 receives, for example, the following assertion: -------------------------------------------------------------------------------- Claim 1: Save the Children's activities are aimed at saving children around the world. Claim 2: Save the Children helped a young girl named Maria escape poverty and get her to school. --------------------------------------------------------------------------------

[0064] (S2) The fact verification unit 123 sends a prompt to the large-scale language model 3 for each assertion sentence, instructing it to create a "search query" for the information corpus database 100, and receives the "search query."

[0065] According to S2 in FIG. 10, the fact verification unit 123 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- Generate a search query against the information corpus database needed to verify the following assertion: ================ Claim 1: Save the Children's activities are aimed at saving children around the world. Claim 2: Save the Children helped a young girl named Maria escape poverty and get her to school. -------------------------------------------------------------------------------- In response to this, the fact verification unit 123 receives, for example, the following search query: -------------------------------------------------------------------------------- Search Query 1: Save the Children's activities Search Query 2: The story of Maria, a girl saved by Save the Children --------------------------------------------------------------------------------

[0066] (S3) The fact verification unit 123 sends the search query to the information corpus database 100 and receives the "related documents."

[0067] According to S3 in FIG. 10, the fact verification unit 123 sends the following search query to the information corpus database 100 and receives related documents. (Search query 1: Save the Children's activities) (Related Documents 1) -------------------------------------------------------------------------------- Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries and continues to work towards a world where "children's rights" are realized. -------------------------------------------------------------------------------- (Search query 2: The story of a girl named Maria who was saved by Save the Children) (Related Documents 2) -------------------------------------------------------------------------------- ...(some loosely related document)... --------------------------------------------------------------------------------

[0068] (S4) The fact verification unit 123 transmits a prompt to the large-scale language model 3 for each related document, citing the search query, instructing it to extract related information, and receives the "related information."

[0069] FIG. 11 is a second example prompt of the fact verification section of the strategy maintenance section of the present invention.

[0070] According to S4 in FIG. 11, the fact verification unit 123 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- Please extract relevant information based on your search query from the related documents below in bullet points. ================= #Related Document 1: Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries and continues to work towards a world where "children's rights" are realized. #Related Document 2: ...(some loosely related document)... Search Query 1: Save the Children's activities Search Query 2: The story of Maria, a girl saved by Save the Children -------------------------------------------------------------------------------- In response to this, the fact verification unit 123 receives, for example, the following related information: -------------------------------------------------------------------------------- #Related information 1: - Save the Children works in approximately 120 countries. - Save the Children's mission is to create a world where children's rights are realized. #Related information 2: - Save the Children saves many boys and girls. --------------------------------------------------------------------------------

[0071] (S5) For each assertion, the fact verification unit 123 cites related information and sends a prompt to the large-scale language model 3 instructing it to determine whether the assertion is valid, unclear, or invalid, and receives the determination result.

[0072] FIG. 12 is a third example prompt of the fact verification unit in the strategy maintenance unit of the present invention.

[0073] According to S5 in FIG. 12, the fact verification unit 123 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- Please determine whether the following statements are valid or unclear based on the relevant information. ======================== #Claim 1: Save the Children's activities are aimed at saving children all over the world. #Related information 1: - Save the Children works in approximately 120 countries. - Save the Children's mission is to create a world where children's rights are realized. #Claim 2: Save the Children saved a young girl named Maria from poverty and enabled her to go to school. #Related information 2: - Save the Children saves many boys and girls. --------------------------------------------------------------------------------

[0074] In response to this, the fact verification unit 123 receives the following true / false judgment: -------------------------------------------------------------------------------- Judgment 1: Based on the information that Save the Children operates in approximately 120 countries, it is safe to say that they are active all over the world. Also, aiming for a world where "children's rights" are realized can be said to be saving children, so the truth of this statement is "true." Outcome 2: Although Save the Children has saved many boys and girls, there is insufficient information about the story of the girl named Maria, so the true / false outcome for this statement is "insufficient information." --------------------------------------------------------------------------------

[0075] FIG. 13 shows an example prompt and search for the related document search section following the fact verification section of the present invention. 13, the related document search unit 122 following the fact verification unit 123 sends the following prompt to the large-scale language model 3. Here, processing is performed only on element sentences that are determined to be unknown or invalid by the fact verification unit 123. (Prompt sent from related document search unit 122) -------------------------------------------------------------------------------- Create a strategy query to search the information corpus database using the element sentences and strategy sentences below. ========= Element 2: "For example, Maria, a young girl, was saved from poverty through our work and is now able to attend school." Strategy 2: “Tell your impact story” --------------------------------------------------------------------------------

[0076] (Strategy query answered by large-scale language model 3) -------------------------------------------------------------------------------- Strategic Query 2: Inspiring stories of children saved by Save the Children --------------------------------------------------------------------------------

[0077] The related document search unit 122 sends the strategic query to the information corpus database 100 to search for related documents such as the following: (Strategic Query 2: Inspiring stories of children saved by Save the Children) (Related Documents 2) -------------------------------------------------------------------------------- One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. --------------------------------------------------------------------------------

[0078] FIG. 14 shows examples of prompts from the response sentence generator in response to FIGS.

[0079] According to FIG. 14, the response sentence generator 13 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- Refer to the dialogue history and strategy and, based on the relevant documents in the strategy, write a response that will persuade them to donate to Save the Children. ======= #Dialogue history ... #strategy Strategy 2: Tell your impact story Related documents for #Strategy2: ... --------------------------------------------------------------------------------

[0080] In response to this, the response sentence generation unit 13 receives, for example, the following response sentence from the large-scale language model 3. -------------------------------------------------------------------------------- Save the Children works to save children. One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. Would you consider making a donation? --------------------------------------------------------------------------------

[0081] 9 to 14, by having the fact verification unit 123, the response sentence is rewritten based on the related documents that are facts only for the element sentences related to the assertion sentences that are judged to be "invalid or unclear." On the other hand, the response sentence is not rewritten for the element sentences related to the assertion sentences that are judged to be "valid."

[0082] If the related document search unit 122 following the fact verification unit 123 is unable to search the information corpus database 100 for related documents corresponding to the strategy query, the related document search unit 122 may discard the strategy.

[0083] FIG. 15 is a diagram showing a third functional configuration of the dialogue device according to the present invention.

[0084] 15 further includes a question query generation unit 14 and a question-related document search unit 15, as compared with FIG.

[0085] [Query Generation Unit 14] The query generation unit 14 transmits to the first database an instruction statement instructing the first database to generate a query for the information corpus database 100 based on the dialogue history with the user, and receives the query.

[0086] FIG. 16 shows an example of a prompt from the query generation unit and an example of a search from the question-related document search unit in the present invention.

[0087] According to FIG. 16, the query generation unit 14 sends the following prompt to the large-scale language model 3: -------------------------------------------------------------------------------- From the dialogue history, create a question query against the information corpus database about the user's question. ======== #Dialogue history ... -------------------------------------------------------------------------------- In response to this, the query generation unit 14 receives the following query from the large term model 3: -------------------------------------------------------------------------------- Query: What does Save the Children do? --------------------------------------------------------------------------------

[0088] [Question-related document search section 15] The question-related document search unit 15 transmits a question query to the information corpus database 100 and receives question-related documents.

[0089] According to FIG. 16, the question-related document search unit 15 transmits a question query to the information corpus database 100 and receives question-related documents. (Query: What is Save the Children doing?) -------------------------------------------------------------------------------- (Question related documents)······Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries and continues to work towards a world where "children's rights" are realized.······ --------------------------------------------------------------------------------

[0090] FIG. 17 shows an example of a prompt from the response sentence generator in response to FIG.

[0091] According to FIG. 17, the response sentence generation unit 13 transmits to the large-scale language model 3 a prompt instructing it to create a response sentence from the dialogue history, citing the strategy sentence and the question-related document. -------------------------------------------------------------------------------- Refer to the dialogue history, strategy and question query and create a response that will persuade the person to donate to Save the Children based on the question-related documents. ======= #Dialogue history User: Hello. Agent: Hello! Would you consider donating to Save the Children? Your donation can make a huge impact on the lives of children in need. User: Please tell us about Save the Children's activities. #strategy Strategy 2: Tell your impact story #Strategy 2 Related Document 2: One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. #Question query: Save the Children's Activities #Question related documents: Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries and continues to work towards a world where "children's rights" are realized. ... --------------------------------------------------------------------------------

[0092] In response to this, the response sentence generation unit 13 receives the following response sentence from the large term model 3. It should be noted that the description in [ ] has been rewritten by the question-related document. -------------------------------------------------------------------------------- Save the Children is an international NGO founded by Grantine Jebb in 1919. Since its founding, Save the Children has supported children in approximately 120 countries, working to create a world where children's rights are realized. One of the most moving stories is that of Maha and Maya. Maha, 10, and her sister Maya, 16, lived in extreme poverty and had to drop out of school to help their family. They were injured by a landmine while collecting firewood in Yemen. Maha lost her left eye and her right hand, and Maya suffered shrapnel injuries. Thanks to Save the Children's education program, she was able to return to school. Would you consider making a donation? --------------------------------------------------------------------------------

[0093] The technology for improving answer accuracy by combining external information retrieval when generating text using a large-scale language model is generally called RAG (Retrieval-Augmented Generation). By providing a RAG sequence in addition to the stage of generating a proposed answer, it is possible to generate a question query from a question included in a user's utterance. This allows: It can answer the user's questions as appropriately as possible, and if the search information cannot be obtained, it can reply with something like "Sorry, I don't know."

[0094] As explained above in detail, the dialogue device, program and method of the present invention make it possible to create a response sentence by rewriting a sentence containing a factual error. According to the prior art, even if a response draft sentence containing a factual error is created, it is possible to rewrite the sentence into an accurate sentence while maintaining a strategy similar to that of the original sentence, without simply deleting the sentence, thereby preventing the user from losing the ability to accomplish the task (the persuasiveness of the dialogue) as much as possible.

[0095] Furthermore, this will enable, for example, "a dialogue system free from factual errors," which will contribute to Goal 8 of the United Nations-led Sustainable Development Goals (SDGs), "Promote inclusive and sustainable economic growth, employment and decent work for all."

[0096] With respect to the various embodiments of the present invention described above, various changes, modifications, and omissions that fall within the scope of the technical spirit and aspects of the present invention may be easily made by those skilled in the art. The above description is merely illustrative and is not intended to be limiting in any way. The present invention is limited only by the claims and their equivalents. [Explanation of symbols]

[0097] 1. Interactive device 100 Information Corpus Database 10 User utterance receiving unit 11 Response draft generation unit 12 Strategic Sustainment Department 121 Strategy Statement Extraction Unit 122 Related Document Search Unit 123 Fact-Verification Department 13 Response sentence generation unit 14. Query Generation 15. Question-related document search section 2. Devices 3 Large-scale language models

Claims

1. 1. A dialogue device that uses a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, to create a response sentence following a dialogue history with a user, the dialogue device comprising: a response draft generation means for transmitting an instruction to a first database instructing the creation of a response draft from a dialogue history with a user, and for receiving the response draft; a strategy maintenance means for extracting element sentences from the answer draft sentences, creating an inherent strategy sentence for each element sentence, transmitting an instruction sentence to the first database instructing the second database to create a strategy query from the element sentences and the strategy sentence, receiving the strategy query, and transmitting the strategy query to the second database and receiving related documents; a response sentence generating means for transmitting an instruction sentence to the first database, instructing the user to create a response sentence following the dialogue history by citing the strategy sentence and related documents, and for receiving the response sentence; An interactive device comprising:

2. The means of maintaining the strategy are: a strategy sentence extraction means for extracting element sentences from the response plan sentence, transmitting an instruction sentence instructing to create a strategy sentence inherent in each element sentence to a first database, and receiving the strategy sentence for each element sentence; a related document search means for transmitting to the first database an instruction statement instructing the creation of a strategic query for the second database from the element sentence and the strategic sentence, receiving the strategic query, and transmitting the strategic query to the second database and receiving related documents; 2. The interactive device according to claim 1, further comprising:

3. The first database is a large language model or rule-based database.

2. The interactive device according to claim 1.

4. The second database is a search database that stores documents that are facts, or a natural language processing information corpus that stores text data that are facts.

2. The interactive device according to claim 1.

5. The means of maintaining the strategy are: The system further comprises a fact verification means for determining whether the element sentences extracted by the strategic sentence extraction means are valid, unclear, or invalid using the first database and the second database, The related document search means processes only element sentences for which the judgment result is unclear or invalid.

3. The interactive device according to claim 2.

6. The means of verifying facts are transmitting an instruction statement instructing the creation of a claim statement from the element statement to a first database, and receiving the claim statement; sending instructions to the first database for each assertion to generate a search query for the second database; and receiving the search query; sending the search query to a second database and receiving relevant documents; For each relevant document, sending a directive to the first database citing the search query to extract relevant information, and receiving the relevant information; For each assertion, a command citing related information and instructing the assertion to be judged as valid, unclear, or invalid is transmitted to the first database, and the judgment result is received.

6. The interactive device according to claim 5.

7. a query generating means for transmitting to the first database an instruction statement instructing the first database to generate a query for the second database from a dialogue history with the user, and for receiving the query; a query-related document search means for sending the query to a second database and receiving query-related documents; and The response sentence generating means transmits an instruction sentence to the first database, instructing the first database to generate a response sentence from the dialogue history, citing the strategy sentence and the question-related document, and receives the response sentence.

7. An interactive device according to claim 1.

8. A program that causes a computer to function to create a response sentence following a dialogue history with a user, using a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, the program comprising: a response draft generation means for transmitting an instruction to a first database instructing the creation of a response draft from a dialogue history with a user, and for receiving the response draft; a strategy maintenance means for extracting element sentences from the answer draft sentences, creating an inherent strategy sentence for each element sentence, transmitting an instruction sentence to the first database instructing the second database to create a strategy query from the element sentences and the strategy sentence, receiving the strategy query, and transmitting the strategy query to the second database and receiving related documents; a response sentence generating means for transmitting an instruction sentence to the first database, instructing the user to create a response sentence following the dialogue history by citing the strategy sentence and related documents, and for receiving the response sentence; A program that causes a computer to function.

9. 1. A method for creating a response sentence following a dialogue history with a user, using a first database for inputting an instruction and outputting a response, and a second database for inputting a query and outputting a related document, comprising: The device is a first step of transmitting an instruction to a first database to generate a response draft from a dialogue history with a user, and receiving the response draft; a second step of extracting element sentences from the answer draft sentence, creating an inherent strategy sentence for each element sentence, sending an instruction sentence to the first database instructing the creation of a strategy query for a second database from the element sentences and the strategy sentence to receive the strategy query, and sending the strategy query to the second database to receive factual related documents; a third step of sending an instruction statement to the first database, citing the strategy statement and related documents and instructing the first database to create a response statement following the dialogue history, and receiving the response statement; 2. A method for device interaction, comprising:

Images