Information processing device and information processing method

Abstract

To provide an information processing device and an information processing method capable of providing a user with a highly precise reply matching with the situation of an individual user while accomplishing a prompt and tentative reply thereto.SOLUTION: An information processing device 100 includes: a receiving unit 10 that accepts a user question from a user; a retrieval unit 20 that retrieves, in a storing unit 60, a reply to the accepted user question by the receiving unit 10 so as to obtain an obtained reply; an evaluating unit 30 that evaluates an obtained reply which is obtained by the retrieval unit 20; an output unit 80 which outputs, as an actual reply to the user question, the obtained reply when the evaluation result for the obtained reply by the evaluating unit 30 is good, and which outputs a tentative reply when the evaluation result is not good; and a requesting unit 50 that causes a generating unit 300 to generate the actual reply to the user question when the evaluation result for the obtained reply by the evaluating unit 30 is not good. The output unit 80 outputs the actual reply generated by the generating unit 300.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method that accept a user question from a user and provide a real answer to the user question. [Background technology]

[0002] Conventionally, a user inputs a question using a chatbot or the like and receives an answer to the question from a server. For example, Patent Document 1 provides a question generation device including a question acquisition unit that acquires a question, and a question generation unit that generates a new question different from the acquired question based on a result of applying a machine learning model to one or more words that make up the acquired question. Patent Document 1 also proposes that questions to be used for testing a chatbot or the like can be efficiently generated. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-124315 Summary of the Invention [Problem to be solved by the invention]

[0004] The registered FAQs used in Patent Document 1 are combination data of questions and answers that are registered in advance in the chatbot, and the questions and answers are stored in advance in a memory unit.

[0005] The present invention provides an information processing device and an information processing method that can provide a user with a quick provisional answer and an answer that is highly accurate and suited to the user's individual situation. [Means for solving the problem]

[0006] The information processing device according to the present invention comprises: a reception unit that receives user questions from users; a search unit that searches a storage unit for an answer corresponding to the user question received by the reception unit and acquires the acquired answer; an evaluation unit that evaluates the answers obtained by the search unit; an output unit that outputs the acquired answer as a real answer to the user question if the evaluation result of the evaluation unit is good, and outputs a provisional answer if the evaluation result is bad; a request unit that, when the evaluation result of the acquired answer by the evaluation unit is bad, causes a generation unit to create a real answer to the user question; Equipped with The output unit may output the final answer generated by the generation unit.

[0007] In the information processing device according to the present invention, The output unit may output the final answer generated by the generation unit with a time lag after outputting the provisional answer.

[0008] In the information processing device according to the present invention, the storage unit has a first storage unit and a second storage unit, the search unit searches a first storage unit for an answer corresponding to the user question and acquires the acquired answer; The request unit may search a second storage unit for content corresponding to the user question, and cause the generation unit to generate the real answer using the content obtained from the second storage unit.

[0009] In the information processing device according to the present invention, The request unit may generate a plurality of final answer candidates using the content acquired from the second storage unit and a plurality of prompts or parameters.

[0010] In the information processing device according to the present invention, the request unit causes the generation unit to generate a plurality of real answer candidates; The evaluation unit may evaluate the plurality of real answer candidates.

[0011] The information processing device according to the present invention comprises: a synthesis unit that generates a new real answer candidate by combining two or more of the plurality of real answer candidates; The evaluation unit may evaluate the final answer candidate generated by the synthesis unit.

[0012] In the information processing device according to the present invention, When the evaluation by the evaluation unit is low, the request unit may cause the generation unit to evaluate the factors of the low-rated real answer candidate.

[0013] In the information processing device according to the present invention, If the evaluation unit gives a high evaluation to the final answer candidate, the final answer candidate may be output as a final answer and may be stored in the storage unit as an answer to the user question.

[0014] In the information processing device according to the present invention, outputting the acquired answer as a real answer to the user question when the evaluation by the evaluation unit for the acquired answer searched by the search unit is equal to or greater than a first threshold value; If there is an acquired answer whose evaluation by the evaluation unit for the acquired answer searched by the search unit is less than the first threshold but greater than or equal to the second threshold, the output unit may output the acquired answer as a provisional answer.

[0015] The information processing method according to the present invention comprises: a step of accepting a user question from a user by a accepting unit; a step of searching a storage unit for an answer corresponding to the user question accepted by the accepting unit and acquiring the acquired answer by a searching unit; a step of evaluating the obtained answer obtained by the search unit by an evaluation unit; an output unit outputting the acquired answer as a real answer to the user question when the evaluation result of the acquired answer by the evaluation unit is good, and outputting a provisional answer when the evaluation result is bad; a step of causing a request unit to generate a real answer to the user question using a generation unit when the evaluation result of the acquired answer by the evaluation unit is bad; outputting, by the output unit, the real answer generated by the generation unit; may also be provided. [Effects of the Invention]

[0016] In the present invention, when a mode is adopted in which a provisional answer is output when the answer stored in the memory unit as an answer to a user question is inadequate, and a permanent answer to the user question is created by the generation unit, an information processing device and information processing method are provided that can quickly provide a provisional answer while providing the user with an answer that is tailored to the user's individual situation with high accuracy compared to the conventional mode in which an answer is provided based on information stored in the memory unit. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram showing an example of an information flow in an information processing system according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram showing existing questions and existing answers in one-to-one correspondence, which are used in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] The information processing device 100 of this embodiment may be composed of a single device or multiple devices. This embodiment also provides an information processing method using the information processing device 100, a program (server program) installed to generate the information processing device 100, and a storage medium such as a USB or DVD on which the program is stored. This embodiment also provides a program (user program) installed in the user terminal 200 and a storage medium such as a USB or DVD on which the program is stored. The information processing device 100 of this embodiment may be a server capable of communicating with multiple user terminals 200, and this embodiment may also be available in a cloud environment. The user terminal 200 may be a smartphone, tablet terminal, PC, or the like. The user terminal 200 has an operation unit 210 that accepts operations from a user and a display unit 220 that displays various information, such as a final answer and a provisional answer, output from the output unit 80 (described later). When the user terminal is a smartphone or tablet terminal, the operation and display unit realizes the functions of the operation unit 210 and the display unit 220.

[0019] As an example, as shown in FIG. 1, the information processing device 100 of this embodiment includes a reception unit 10 (see "(1)" in FIG. 2) that receives a user question input from a user via an operation unit 210 of a user terminal 200, a search unit 20 that searches a storage unit 60 for an answer corresponding to the user question received by the reception unit 10, and an evaluation unit 30 that evaluates an acquired answer searched and acquired by the search unit 20. The storage unit 60 may store a plurality of existing questions and existing answers corresponding to each existing question in a one-to-one correspondence, and an acquired answer may be acquired using a result of comparing the user question with the existing question (see FIG. 3). Note that the reception of a user question from a user and the answer to the user question may be performed by a chatbot.

[0020] The search unit 20 may use a user question input from the user terminal 200 as input data, perform a vector search on a storage unit 60 (first storage unit 61, described later) such as a cache database shown in FIG. 2, and acquire the top N evaluation results from the evaluation unit 30. The search unit 20 may perform a search after performing preprocessing such as deleting unnecessary parts of questions such as standard greetings. The search unit 20 may also use vectors created using a sentence-by-sentence vectorization model to calculate COS similarity (cosine similarity) with existing questions stored in the storage unit 60. When creating vectors, a provided API (Application Programming Interface), LaBSE, or the like may be used.

[0021] The evaluation unit 30 may calculate, for each of the acquired one or more existing questions, 1) the semantic similarity with the user question, 2) the semantic inclusion relationship with the user question (however, if the user question contains multiple questions, the user question ⊃ the existing questions), and 3) the similarity of proper nouns with the user question, and calculate a score by taking a weighted average of these. The "weight" in the weighted average may be input by the user via the user terminal 200 and changed as appropriate, or may be input by the administrator via the administrator terminal 200a and changed as appropriate. The administrator terminal 200a may be a smartphone, a tablet terminal, a personal computer, or the like. The administrator terminal 200a has an operation unit 210a that accepts operations from the administrator and a display unit 220a that displays various information.

[0022] The evaluation unit 30 may perform evaluation based on the similarity between one or more acquired existing questions and the user's question. However, in this embodiment, if there is no existing question with a high similarity, the generation unit 300 generates a real answer candidate, so it is not necessary to select an existing question with a high similarity. In this regard, a more appropriate evaluation can be performed by using the above 1) to 3) to calculate the score. Regarding "1) semantic similarity with the user's question," the similarity between the existing question and the user's question may be evaluated using a vector. Regarding "2) semantic inclusion relationship with the user's question," the meaning of the existing question may be compared with the meaning of the user's question, and the extent to which the existing questions are covered by the user's question may be evaluated using an existing model such as a large-scale language model (LLM). Regarding "3) proper noun similarity with the user's question," the evaluation unit 30 may extract proper nouns from the user's question and the existing question, and count the number of matching proper nouns. As an example of a method for evaluating "1) the semantic similarity with the user's question" and "2) the semantic inclusion relationship with the user's question," two sentences for learning are first vector-represented, and then a semantic relationship model that has been trained on the semantic similarity and the semantic inclusion relationship is prepared by machine learning. Then, a vectorized existing question and a vectorized user's question may be input as input data to the prepared semantic relationship model, and the semantic similarity and the inclusion relationship between the two may be evaluated.

[0023] The information processing device 100 may have an output unit 80 that outputs the acquired answer as a final answer to the user question when the evaluation unit 30 evaluates the acquired answer as a good answer, for example, equal to or greater than a first threshold, and outputs the acquired answer as a provisional answer or a notification that an answer will be provided later or at a later date when the evaluation unit 30 evaluates the acquired answer as a bad answer, for example, less than the first threshold, and a request unit 50 that causes the generation unit 300 to create a final answer to the user question when the evaluation unit 30 evaluates the acquired answer as a bad answer, less than the first threshold. For the flow of this process, see "(2)" and "(3)" in Fig. 2.

[0024] More specifically, if the evaluation unit 30 obtains an existing question with an evaluation value equal to or greater than the first threshold, the output unit 80 may output the existing answer corresponding to the existing question as a permanent answer to the user's question. If multiple existing questions have evaluation values equal to or greater than the first threshold, the output unit 80 may output the existing answer (corresponding to the obtained answer) to the existing question with the highest evaluation value. However, the present invention is not limited to this configuration, and a predetermined number (e.g., the top three) of existing answers to existing questions with evaluation values equal to or greater than the first threshold may be output. Furthermore, if there is an existing question with an evaluation value equal to or greater than the first threshold, the request unit 50 may not cause the generation unit 300 to create a permanent answer to the user's question, as described below. By adopting such a configuration, it is possible to prevent duplicate questions and their answers from being stored in the storage unit 60. The generation unit 300 is an external device separate from the information processing device 100, and may utilize a commonly available generation AI (e.g., ChatGPT). However, the present invention is not limited to this embodiment, and the generation unit 300 may be included in the information processing device 100 (see the generation unit 300 indicated by the dotted line in FIG. 1).

[0025] If the evaluation by the evaluation unit 30 indicates that an existing question that is equal to or greater than the first threshold has not been acquired, the request unit 50 may have the generation unit 300 create a permanent answer to the user question (see "(3)" in FIG. 2). In this case, the output unit 80 may not propose a tentative answer to the user terminal 200, but may output a tentative answer indicating that the answer will be provided later or at a later date. The output of the existing answer corresponding to the existing question and the request by the generation unit 300 to create a permanent answer may be performed asynchronously.

[0026] If there is an existing question for which the evaluation result (e.g., evaluation score) by the evaluation unit 30 is less than the first threshold but equal to or greater than the second threshold (first threshold > second threshold), the output unit 80 may output an existing answer to the existing question as a provisional answer to the user. If multiple existing questions have evaluation values equal to or greater than the second threshold, the output unit 80 may output an existing answer to the existing question with the highest evaluation value as a provisional answer. However, this is not limited to such an embodiment, and a predetermined number (e.g., the top three) of existing answers to existing questions with evaluation values equal to or greater than the second threshold may be output as provisional answers. When this embodiment is adopted, the output unit 80 may output the existing answer as a provisional answer to the user terminal 200 and a message indicating that a permanent answer will be provided later or at a later date. Furthermore, if no existing question with an evaluation value equal to or greater than the second threshold is obtained, the output unit 80 may output, to the user terminal 200, only a message indicating that a reply will be provided later or at a later date without providing a provisional answer.

[0027] When the generation unit 300 generates the final answer, the output unit 80 may output the final answer to the user terminal 200. The user question and user identification information such as a user ID may be associated in the storage unit 60, and the final answer corresponding to the user question may be output by email from the output unit 80 to the user terminal 200 associated with the user identification information. Note that information about the user may be stored in either the first storage unit 61 or the second storage unit 62, or may be stored in another area of the storage unit 60 different from the first storage unit 61 and the second storage unit 62.

[0028] If there is an existing question for which the evaluation result by the evaluation unit 30 is less than the first threshold but equal to or greater than the second threshold, the answer generated as the final answer may be compared with the existing answer provisionally notified to the user. If the evaluation unit 30 determines that new information has been added, the final answer may be notified to the user. If it determines that new information has not been added, the output unit 80 may notify the user that the existing question provisionally notified is the final answer, or may automatically send a message to an operator requesting a response. The presence of added information may be determined using a large-scale language model (LLM). For example, two training sentences may be vector-represented in advance, and an addition judgment model that learns the relationship between semantic additions may be prepared by machine learning. Then, vectorized versions of the existing answer and vectorized versions of the final answer candidate may be input as input data to the prepared addition judgment model, and the semantic addition of the final answer candidate to the existing answer may be evaluated. In this case, a predetermined threshold may be prepared, and if the semantic addition score is equal to or greater than the predetermined threshold, it may be determined that added information has been added.

[0029] When the above-described embodiment is adopted, an answer is prepared primarily based on the information stored in the storage unit 60, but the content of the answer is evaluated, and if it is satisfactory, the answer is presented as a final answer. On the other hand, if the information stored in the storage unit 60 is evaluated as insufficient (poor), a provisional answer is output, and the request unit 50 causes the generation unit 300 to create a final answer to the user's question. This allows the provisional answer to be quickly presented to the user, while providing an answer to the user's question that is highly accurate and tailored to the user's individual situation, thereby making it possible to provide an answer to a user's question that includes content not anticipated in existing questions.

[0030] The output unit 80 may output the final answer generated by the generation unit 300 with a time lag after outputting the tentative answer. It may take time for the generation unit 300 to generate a final answer and properly evaluate it. On the other hand, some kind of response is required for the user question input by the user. Therefore, when this mode is adopted, a tentative answer is first presented to the user, and then a formal final answer is provided at a later time. This is advantageous for the user in that, after first receiving the tentative answer, they can later receive a highly accurate final answer.

[0031] The storage unit 60 may include a first storage unit 61 and a second storage unit 62. The search unit 20 may search the first storage unit 61 for an answer corresponding to a user question, and the request unit 50 may search the second storage unit 62 for content corresponding to the user question. The request unit 50 may then cause the generation unit 300 to generate a real answer using the content searched for in the second storage unit 62. The first storage unit 61 may store existing questions and existing answers in a one-to-one association (see FIG. 3). On the other hand, the second storage unit 62 may store a business manual, past response records, etc. (see "(3)" in FIG. 2). More specifically, the second storage unit 62 may store various information related to business, such as a business manual, past response records, etc., instead of storing existing questions and existing answers in a one-to-one association.

[0032] The request unit 50 may use a user question input from the user terminal 200 as input data, perform a vector search on the second storage unit 62, such as a business manual and response record database, to obtain the top M results, and score the obtained top M results using a large-scale language model (LLM) from the perspective of question and answer. The request unit 50 may perform preprocessing, such as removing unnecessary parts from questions such as standard greetings, before performing the search. The division unit 45 may chunk (divide information) the information stored in the second storage unit 62, such as a business manual and response record database. In this case, chunking may be performed, for example, by chapter. The information divided by the division unit 45 may be stored in the second storage unit 62 separately from the information before division. The search unit 20, rather than the request unit 50, may search the second storage unit 62 for content corresponding to the user question.

[0033] The evaluation unit 30 may calculate 1) the semantic similarity with the user question, 2) the semantic inclusion relationship with the user question, and 3) the similarity of proper nouns with the user question for each piece of input information for generation acquired by chunking, and extract several to several tens of pieces of input information for generation that are ranked highest in terms of 1) the semantic similarity with the user question, 2) the semantic inclusion relationship with the user question, and 3) the similarity of proper nouns with the user question. The evaluation here is for the purpose of usability as information for creating a prompt for the generation unit 300. For this reason, there is little need for weighted averaging, and weighted averaging may not be performed for 1) to 3). However, a mode in which weighted averaging is performed for 1) to 3) and evaluation is also possible. The evaluation unit 30 may also perform evaluation using only the similarity of the input information for generation to the user question.

[0034] The request unit 50 may generate one or more permanent answer candidates using one or more pieces of input information for generation acquired from the second storage unit 62 and one or more prompts stored in the storage unit 60. While the present embodiment will be described primarily with reference to an embodiment in which multiple prompts are used to generate multiple permanent answer candidates, multiple permanent answer candidates may be generated by changing parameters. The term "prompt" in the present embodiment may be appropriately replaced with "parameter" or "prompt and parameter." For example, multiple permanent answer candidates may be generated by changing the "parameter" rather than the "prompt." Alternatively, multiple permanent answer candidates may be generated by changing both the "prompt" and the "parameter." For example, if "temperature" is used as one of the parameters used in the prompt, multiple permanent answer candidates may be generated by using different values for "temperature."

[0035] As an example, the request unit 50 may generate multiple final answer candidates using multiple pieces of input information for generation and multiple prompts acquired from the second storage unit 62. When such an embodiment is adopted, multiple final answer candidates can be obtained. This increases the likelihood of obtaining highly accurate content as the final answer. Specifically, the request unit 50 may use multiple pieces of input information for generation having a certain score or higher (e.g., three, five, seven, etc.) and multiple prompts (templates for generation, including answer plans and sequential execution). This allows the generation unit 300 to generate dozens of answer patterns. By increasing the number of pieces of information and prompts used, a large number of final answer candidates can be obtained. Prompts generated with reference to existing questions and existing answers may be stored in the storage unit 60, and prompts read from the storage unit 60 may be used. Regarding prompts, those input by a user from the user terminal 200 or those input by an administrator from the administrator terminal 200a may be stored in the storage unit 60. Furthermore, the user may input a prompt via the user terminal 200, or the prompt to be used may be determined by the user selecting from pre-prepared prompts. An example of a prompt is "Please create an answer to the question 'XX' using '△△····△△' as a reference."

[0036] When the generation unit 300 generates a plurality of main answer candidates, the evaluation unit 30 may evaluate the plurality of main answer candidates. When this mode is adopted, it is advantageous in that the evaluation result by the evaluation unit 30 can be used to select an appropriate main answer from the plurality of main answer candidates.

[0037] As an example, each of the real answer candidates generated by the generation unit 300 may be scored from the perspectives of 1) the validity of the answer to the question (whether there are any omissions or omissions), 2) the fluency of the description, and 3) the possibility of asking a follow-up question based on past correspondence records, and a weighted average may be calculated. The "weight" in the weighted average may be input by the user at the user terminal 200 and changed as needed, or may be input by the administrator at the administrator terminal 200a and changed as needed.

[0038] Regarding "1) the validity of the question and answer," a business manual may be used to check for omissions. Furthermore, similar to existing text proofreading tasks, a large-scale language model (LLM) may be used to detect omissions. For example, two training sentences may be vector-represented in advance, and a semantic inclusion model that learns semantic inclusion may be prepared by machine learning. Then, a vectorized portion of the business manual (chunked portion) used to generate the actual answer candidate and a vectorized portion of the actual answer candidate may be input as input data to the prepared semantic inclusion model, and the semantic inclusion between them may be evaluated. Regarding "2) the fluency of the description," an LLM (large-scale language model) and a fluency judgment model (a conventional existing fluency judgment model may also be used) stored in advance in the storage unit 60 may be used for evaluation. Regarding "3) the possibility of a re-ask based on past response records," an LLM (large-scale language model) or a re-ask possibility judgment model stored in advance in the storage unit 60 may be used for evaluation. The re-ask possibility judgment model may be machine-learned using past response records. More specifically, answers to users and whether or not follow-up questions are asked in response to those answers may be used as learning data from past response records, and a machine-learned model may be prepared in advance as a follow-up question possibility determination model and stored in the storage unit 60. If the answer accuracy is determined to be low through the above process, the real answer candidates may be generated again, or these processes may be repeated until the answer accuracy is improved to a certain extent.

[0039] Note that if the generation unit 300 generates an answer to each user question, it will take time to obtain the answer. On the other hand, in a mode in which a user question is answered using only the information stored in the storage unit 60, an appropriate answer to the user question cannot be provided unless appropriate answer candidates are stored in the storage unit 60. Furthermore, when the system is switched, the appropriate answer to the user question cannot be provided due to a limited number of answer candidates. In this regard, as in the present embodiment, a provisional answer is quickly presented to the user, and the generation unit 300 generates a final answer candidate using information acquired from the second storage unit 62, such as a business manual and response record database, thereby enabling a highly accurate final answer to the user question to be provided. When generating the final answer candidate, in addition to the information stored in the second storage unit 62, such as a business manual and response record database, existing questions stored in the first storage unit 61, such as existing FAQ data, may be referenced. Furthermore, when providing the generated answer, in addition to the answer itself, information stored in the second storage unit 62 used in the answer (e.g., file contents and file metadata) may be provided as source information.

[0040] A synthesis unit 40 may be provided that generates a new final answer candidate by combining two or more of the multiple final answer candidates. Adopting such an embodiment using the synthesis unit 40 is advantageous in that it may be possible to generate a final answer candidate that is highly accurate and tailored to the user's individual circumstances while utilizing previously obtained final answer candidates. For example, if there are 20 final answer candidates, these may be combined in pairs or in groups of three. The evaluation unit 30 may evaluate the final answer candidate generated by the synthesis unit 40 (a final answer candidate generated by combining multiple final answer candidates). For example, if there are omissions or omissions and the "1) validity of the question answer" is low, the omissions or omissions may be identified, and several patterns may be combined from dozens of patterns of final answer candidates created, to see if the evaluation by the evaluation unit 30 improves. Adopting such an embodiment is expected to provide a satisfactory answer to the user's question without omissions or omissions.

[0041] If the evaluation unit 30 provides a low rating, the request unit 50 may have the generation unit 300 evaluate the factors behind the low-rated final answer candidate. This configuration is beneficial in that it allows the factors behind the low-rated final answer candidate to be identified and can lead to future improvements. For example, if an omission or omission is identified, the generation unit 300 may determine whether the issue should be internally confirmed within the organization or whether the user should be asked. An example of a prompt might be, "We prepared 'BBB' as an answer to the question 'AAA,' but it was determined that there was an omission or omission. Using 'CC...CC' as a reference, the reason for the low rating may be an omission or an ambiguous question. Please explain why the rating was low."

[0042] If the cause of the low evaluation is a question from the user (for example, an ambiguous question from the user), the output unit 80 outputs information prompting the user to input, and the reception unit 10 receives input of supplementary information from the user via the user terminal 200. Also, if there is insufficient information in the business manual or the like, the person in charge of the department in charge can handle it. A binary classification model may be used to determine whether an item should be confirmed internally or with the user, and in this case, an LLM (large-scale language model) may be used.

[0043] If an existing question with an evaluation value equal to or greater than the second threshold cannot be obtained and the evaluation unit 30 evaluates the generated answer poorly (for example, if the score by the evaluation unit 30 is less than the third threshold), the output unit 80 may notify an operator, who may then take action (see "(4)" in FIG. 2). If such an embodiment is adopted, human intervention will be performed only when the evaluation unit 30 evaluates the answer poorly, which is expected to reduce labor costs and provide a user with a highly satisfying answer.

[0044] When an operator responds, if there is a reply from the user (see "(5)" in Figure 2), the operator may continue to respond, but if complementary information is received from the user, the request unit 50 may automatically supplement each piece of input information for generation with the complementary information, and then request the generation unit 300 to create the actual answer, and the series of steps described above may be performed.

[0045] If the generation unit 300 determines that the matter requires internal confirmation, the output unit 80 may automatically send a notice to that effect by internal email, for example. In this case, the receiving unit 90 may receive a reply from the department in charge of the business in response to the automatically sent email, and the storage unit 60 may store the reply.

[0046] Furthermore, if the generation unit 300 determines that the question should be asked to the user, the output unit 80 may transmit the content to the operator, who may then take over. In this case, for example, the output unit 80 may automatically transmit a list of questions to be asked to the operator, and receive feedback of the answers (user answers) from the operator, which may then be stored in the storage unit 60. Then, the generation unit 300 may generate real answer candidates again using the user answers.

[0047] When the evaluation unit 30 highly evaluates the final answer candidate (for example, when the score by the evaluation unit 30 is equal to or greater than a third threshold), the output unit 80 may output the final answer candidate as a final answer and store it in the storage unit 60 (typically, the first storage unit 61) as an answer to the user question. When this mode is adopted, highly evaluated final answers can be stored in the storage unit 60, and final answers that can be used to immediately answer future questions from users can be prepared in advance. When storing the question and the final answer in the storage unit 60, the adjustment unit 70 may adjust the final answer by converting colloquial language to written language or deleting names, etc., before storing them in the storage unit 60. The adjustment unit 70 may shape the final answer using a large-scale language model (LLM).

[0048] The reception unit 10, search unit 20, evaluation unit 30, synthesis unit 40, division unit 45, request unit 50, adjustment unit 70, etc. may be realized by a single unit (control unit) or by different units. The functions of multiple "units" may be integrated, and for example, the functions of the search unit 20, evaluation unit 30, synthesis unit 40, and division unit 45 may be realized by a single unit. Furthermore, the reception unit 10, search unit 20, evaluation unit 30, synthesis unit 40, division unit 45, request unit 50, adjustment unit 70, etc. may be realized by the functions of a processor or by a circuit configuration.

[0049] The above description of the embodiment and the disclosure of the drawings are merely examples for explaining the invention described in the claims, and the invention described in the claims is not limited by the above description of the embodiment or the disclosure of the drawings. [Explanation of symbols]

[0050] 10 Reception 20 Search Section 30 Evaluation Department 40 Synthesis section 50 Request Department 60 Storage section 61 First memory section 62 Second memory section 80 Output section 100 Information processing device 200 user terminals 300 Generation part

Claims

1. a reception unit that receives user questions from users; a search unit that searches a storage unit for an answer corresponding to the user question received by the reception unit and acquires the acquired answer; an output unit that outputs the acquired answer as a real answer to the user question if the evaluation result of the acquired answer acquired by the search unit is good, and outputs a provisional answer if the evaluation result is bad; a requesting unit that, when an evaluation result of the acquired answer is poor, causes a generating unit to generate a real answer to the user question; Equipped with The output unit is an information processing device that, when the evaluation result of the obtained answer is poor, outputs the provisional answer to the user terminal, and then outputs the actual answer generated by the generation unit to the user terminal with a time difference from the provisional answer.

2. the storage unit has a first storage unit and a second storage unit, the search unit searches a first storage unit for an answer corresponding to the user question and acquires the acquired answer; The information processing device according to claim 1 , wherein the request unit searches a second storage unit for content corresponding to the user question, and causes the generation unit to generate a real answer using the content obtained from the second storage unit.

3. The information processing device according to claim 2 , wherein the request unit generates a plurality of real answer candidates using the content acquired from the second storage unit and a plurality of prompts or parameters.

4. the request unit causes the generation unit to generate a plurality of real answer candidates; The information processing device according to claim 1 , further comprising an evaluation unit that evaluates the plurality of real answer candidates.

5. The request unit causes the generation unit to generate a plurality of real answer candidates, The information processing device according to claim 1 , further comprising a synthesis unit that generates a new final answer candidate by combining two or more of the plurality of final answer candidates.

6. The information processing device according to claim 4 , wherein when the evaluation unit has given a low evaluation of the actual answer candidate, the request unit causes the generation unit to evaluate factors behind the low evaluation of the actual answer candidate.

7. The information processing device according to claim 4 , wherein when the evaluation unit evaluates the real answer candidate highly, the real answer candidate is output as a real answer and stored in a storage unit as an answer to the user question.

8. An evaluation unit that evaluates the obtained answer obtained by the search unit, outputting the acquired answer as a real answer to the user question when the evaluation by the evaluation unit for the acquired answer searched by the search unit is equal to or greater than a first threshold value; 3. The information processing device according to claim 1, wherein when there is an acquired answer whose evaluation by the evaluation unit for the acquired answer searched by the search unit is less than a first threshold but greater than a second threshold, the output unit outputs the acquired answer as a provisional answer.

9. a step of accepting a user question from a user by a accepting unit; a step of searching a storage unit for an answer corresponding to the user question accepted by the accepting unit and acquiring the acquired answer by a searching unit; an output unit outputting the acquired answer as a real answer to the user question when the evaluation result of the acquired answer acquired by the search unit is good, and outputting a provisional answer when the evaluation result is bad; a step of causing a request unit to generate a real answer to the user question using a generation unit when an evaluation result of the answer obtained by the search unit is poor; an output step in which, when the evaluation result of the acquired answer is poor, the output unit outputs the provisional answer to the user terminal, and then outputs the final answer generated by the generation unit to the user terminal with a time difference from the provisional answer; An information processing method comprising:

Images