Information processing device, method, and program

Abstract

The present invention provides an information processing device that generates information useful for responding to users in a simple and appropriate manner, in accordance with a search extension generation method. [Solution] An information processing device 20 that takes over the history of question-and-answer sessions between questions provided to a user who desires an interaction and the answers obtained from the user, and generates information related to the interaction with the user, is characterized by performing a summary generation process 2 that generates a summary of the history, and a similarity search process 3 that searches for similar items from a plurality of interaction realization information pre-constructed as a database. Furthermore, an interaction text generation process 4 is performed to generate an interaction text for realizing the interaction from the search results of the similar items.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, method, and program that generate information related to responding to users using retrieval-augmented generation.

Background Art

[0002] The increase in labor costs for operators and employees due to customer support in companies and responses to internal inquiries has become an issue. Although QA (Question and Answer)-based chatbots have been introduced, many existing chatbots only answer based on pre-prepared QA scenarios, and for questions that were not anticipated in advance, humans need to respond individually, and there was a limit to reducing labor costs.

[0003] On the other hand, with the emergence and development of large-scale language models (hereinafter abbreviated as LLM) in recent years, there is an expectation for a system that can generate response sentences similar to those of an operator for such unanticipated questions to provide automatic responses or assist in creating responses from an operator.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] An existing approach to generating answers using LLM is called Search Augmented Generation (RAG), which has been disclosed in Non-Patent Document 1, etc. While LLM is highly versatile due to being trained on vast amounts of data, applying LLM directly to specific fields (such as customer support or internal inquiries) where the vast amount of data used for training does not sufficiently contain the necessary insights may not yield appropriate results. In fact, it can sometimes produce false answers, a known problem with LLM known as hallucination. In contrast, the RAG method allows LLM to be easily specialized for specific fields.

[0007] A prior art example that specifically applies the RAG method to responding to inquiries is Patent Document 1, which extracts search keywords from the text of the user's inquiry, searches a database such as manuals and past response history, and generates an answer by inputting the results into the LLM prompt.

[0008] However, methods that simply extract keywords from the inquiry content, such as those described in Patent Document 1, have drawbacks, including the fact that they do not always match the appropriate manual, and that they may refer to past responses with different detailed circumstances.

[0009] In other words, even if keywords matched manuals or past responses, the context of the keywords in those manuals or responses might differ from the context of the keywords in the inquiry. For these reasons, simple keyword matching did not always guarantee an appropriate response.

[0010] For example, if a specific smartphone model name (let's call it "Model α") is extracted as a keyword, and the user is trying to purchase that smartphone in the context of the inquiry, but is currently using a different model (let's call it "Model β"), the system might simply refer to the user manual for Model α, which matches the keyword, and generate an answer based on how to operate Model α. Since the user is currently using Model β, the answer based on how to operate Model α will be irrelevant.

[0011] In view of the problems of the above-mentioned prior art, the present invention aims to provide an information processing device, method, and program that generate information useful for responding to users in a simple and appropriate manner in accordance with a search extension generation method. [Means for solving the problem]

[0012] To achieve the above objective, the present invention provides an information processing device, method, and program that take a history of question-and-answer sessions between questions provided to a user who desires to receive a response and answers obtained from the user, and generates information relating to the response to the user, characterized by performing a summary generation process that generates a summary of the history, and a similarity search process that searches for similar summaries from a plurality of response realization information pre-constructed as a database. [Effects of the Invention]

[0013] According to the present invention, by searching using a summary of the question-and-answer history rather than keywords, it is possible to generate information that is simple and appropriate for responding to users in accordance with the search extension generation method. [Brief explanation of the drawing]

[0014] [Figure 1] This is a diagram illustrating the configuration of an information processing system according to one embodiment. [Figure 2] This is a functional block diagram of an information processing device according to one embodiment. [Figure 3] This is a diagram showing the processing of the LLM section. [Figure 4] This is a diagram showing a configuration example of the case DB according to an embodiment. [Figure 5] This is a flowchart of the operation of the information processing apparatus according to an embodiment. [Figure 6] This is a diagram showing the schematic flow of questions and answers in the fixed response section on the upper side and a schematic example of the obtained fixed response history on the lower side. [Figure 7] This is a diagram showing a schematic example of the processing of the summary generation section. [Figure 8] This is a diagram showing a schematic example of the processing in the similar case search section. [Figure 9] This is a diagram showing a schematic example of the input prompt (and output prompt) for screening useful ones in the similar case search section in the case of searching the response case DB. [Figure 10] This is a diagram showing a schematic example of the input prompt (and output prompt) for screening useful ones in the similar case search section in the case of searching the manual DB. [Figure 11] This is a diagram showing a schematic example of the processing in the response sentence generation section. [Figure 12] This is a diagram showing a hardware configuration example in a general computer.

Modes for Carrying Out the Invention

[0015] FIG. 1 is a configuration diagram of an information processing system according to an embodiment. The information processing system 100 includes a terminal 10 used by a user and an information processing apparatus 20, and these 10 and 20 can communicate with each other via a network NW with an arbitrary configuration such as the Internet and / or a local area network. The terminal 10 may be configured as an arbitrary computer device such as a personal computer, a smartphone, or a tablet, receives text input from a user via an arbitrary input device such as a keyboard, a mouse, or a touch panel, and transmits the text to the information processing apparatus 20. The information processing apparatus 20 generates a response text for the text received from the user and transmits it to the terminal 10. By displaying the response text on a display device such as a display at the terminal 10, the user can know the response content for his / her input text.

[0016] By performing the transmission and reception of the text between the terminal 10 and the information processing apparatus 20 one or more times, the user can receive the response service he / she desires. On the terminal 10, the reception of text input from the user in the response service and the display of the result responded by the information processing apparatus 20 to the user can be realized by an arbitrary existing technology such as a web browser.

[0017] The information processing apparatus 20 may be configured as a single computer device as schematically shown in FIG. 1, or may be configured as two or more computer devices that can communicate with each other via the network NW in a form that分担 its roles described below.

[0018] Also, as a modification, the terminal 10 in the configuration of FIG. 1 may be omitted, and the user may directly operate an input device such as a keyboard provided in the information processing apparatus 20 (for example, a stand-alone computer device) so that the response content can be directly obtained on a display device such as a display provided in the information processing apparatus 20.

[0019] It should be noted that there is an unclear expression "分担 its roles" in the translation of . It might be a misspelling or an unclear term in the original text. If possible, it is recommended to check and correct it for a more accurate translation.As described above, text input from the user is received by terminal 10 or directly by information processing device 20, and is input to information processing device 20. The text of the response is also displayed on terminal 10 or on information processing device 20, so that the user can see it. In the following explanation, references to the points where input and output between the user and the information processing device 20 take place will be omitted, and the details of the process by which information processing device 20 responds to such user input will be explained.

[0020] Furthermore, the user's text input to the information processing device 20 may be done by the user freely creating text data as a document, or by selecting from a set of predetermined answer options provided by the information processing device 20.

[0021] In the following, we will use as a primary example that the user is a customer with a mobile device such as a smartphone and provides various consultations regarding troubleshooting and contract confirmation as part of customer support related to this. However, the information processing system 100 can provide arbitrary content of response services to the user via text. In other words, while the customer support example used in the explanation assumes that the user is in a somewhat active or proactive psychological state and desires a response, the information processing system 100 can also provide response services to users that are (at least initially) in a passive or unresponsive psychological state, such as product recommendation services, solicitation services for donations or volunteer activities, or persuasion services to encourage a healthy lifestyle.

[0022] The information processing system 100 of this embodiment employs the RAG method, making it possible to automatically provide responses to users not only for customer support as described in this example, but also for any matter the user wishes to consult and resolve. Here, assuming that a database of a certain size already exists and is accessible containing case studies and manuals (procedures) of resolution procedures in the relevant field, the information processing system 100 can provide responses using the RAG method.

[0023] Figure 2 is a functional block diagram of an information processing device 20 according to one embodiment, which comprises a standard response unit 1, a summary generation unit 2, a similar case search unit 3, a response text generation unit 4, a standard response DB (database) 5, and a case DB 6.

[0024] The summary generation unit 2, the similar case search unit 3, and the response text generation unit 4 all have an internal LLM unit 7 that is pre-trained on a large amount of training data and capable of general-purpose LLM processing. Figure 3 shows the processing of the LLM unit 7, which can accept input text containing questions or instructions and output response text, and can also realize arbitrary text dialogues in the form of text input and output, not limited to the format of returning answers to questions, etc.

[0025] The input / output processing framework of this LLM unit 7 is the same as existing LLMs, and the input and output statements are also called input prompts and output prompts, respectively, as is commonly used in the technical field. Furthermore, the configuration of the LLM unit 7 (the configuration when the LLM unit 7 is extracted as a standalone component of the information processing device 20) can also utilize any existing learning model LLM, such as the one mentioned in Non-Patent Document 1 above.

[0026] In this embodiment, the input sentence to the LLM unit 7 is prepared internally by the summary generation unit 2, the similar case search unit 3, and the response sentence generation unit 4. This frees the user from the trouble of directly handling the LLM (for example, the trouble of trying out various input sentences to obtain the desired answer and checking each time whether the answer is appropriate).

[0027] As will be explained in more detail later, the similar case search unit 3 searches for similar cases from the case database 6, and optionally uses the LLM unit 7 to select (filter) appropriate cases from the searched similar cases. In other words, the LLM unit 7 in the similar case search unit 3 is for the purpose of selection, and in embodiments where selection is not performed, the LLM unit 7 can be omitted from the similar case search unit 3.

[0028] The following describes the operation of the information processing system 100 according to one embodiment by explaining the details of each functional block of the information processing device 20 in Figure 2. As mentioned above, the example used for explanation is when the user is a customer of a mobile phone or the like, and the information processing system 100 provides customer support to that customer using the RAG method.

[0029] Figure 4 shows an example configuration of the case DB6 according to one embodiment, and the case DB6 comprises a response case DB61 and a manual DB62. (Note that the case DB6 may be configured to include only one of these DB61 and DB62.) Figure 4 shows an example configuration of the case DB6 for realizing customer support as an explanatory example, and the response case DB61 stores various successful cases of past customer support responses as text for each case ID, and can be configured as a collection of text that has been automatically recorded by a computer and / or text that has been manually recorded by operators who have provided support. The manual DB62 stores operation manuals and troubleshooting procedures for mobile phones and network equipment that are covered by a line contract as text, and can be configured as manuals prepared by, for example, the company that develops and manufactures the equipment.

[0030] It is preferable that the case database 6 be pre-built as a comprehensive collection of information that directly or indirectly leads to solutions in the areas where the user seeks assistance. As with the records in this case database 6, it is usually difficult for a user to find the desired solution through manual methods such as keyword searches from comprehensive and generally vast amounts of information. However, in this embodiment, using the RAG method, the user only needs to repeatedly provide routine answers to the information processing device 20, and the information processing device 20 will process the results as appropriate case search results from the case database 6, or as response text that has been further processed to make it easier for the user to understand.

[0031] Figure 5 is a flowchart of the operation of an information processing device 20 according to one embodiment, and as shown in the figure, steps S1 to S4 are executed in this order. Details of each step are as follows.

[0032] ●Step S1…Standard response unit 1 and standard response DB5 In step S1, the standard response unit 1 receives response requests from users who have requests for assistance regarding troubleshooting or answering questions about contract details. The unit then performs standard responses to these requests with the user, which are pre-recorded in the standard response DB 5. The history of these standard responses is recorded and then output to the summary generation unit 2.

[0033] The standardized responses of the standardized response unit 1 can be implemented, for example, using a method known as a chatbot, by repeatedly issuing a series of rule-based questions to the user, as schematically shown at the top of Figure 6, and then sorting those questions into lower-level questions based on the answers received from the user. The standardized response DB 5 stores a predetermined method for sorting based on such a series of questions and answers, and the standardized response unit 1 can obtain a history of questions and answers to the user according to this method.

[0034] In Figure 6, the process moves between nodes from the parent node to the child node by repeatedly asking and answering questions in a tree structure. When the terminal node is reached and a user response is obtained, the processing of the standard response unit 1 is completed. Two examples of the standard response history in this case, EX1 and EX2, are given below. Example EX1… Question Q1 → User Answer A1 → Question Q2 → User Answer A3 → Question Q4 → User Answer A7 Example EX2… Question Q1 → User Answer A2 → Question Q3 → User Answer A6

[0035] In the upper example of Figure 6, questions Q1 to Q5 are sent as text messages from the standard response unit 1 to the user, and answers A1 to A10 are the user's responses to each question sent to the standard response unit 1. Each answer may be selected by the user from a predetermined menu, or it may be freely entered as text by the user.

[0036] The lower example in Figure 6 is a text example of a schematic standard response history obtained in this manner, showing a schematic example of individual questions and answers when a standard response history is constructed following the flow of Example EX1 described above. As shown in the text "Please try ■■" in question Q4 of this example, all or some of the tree-structured questions such as questions Q1 to Q5 may include suggestions for specific responses to the user in order to realize the response the user requests (for example, in order to obtain specific answers to specific questions). These suggested responses can be predetermined as potential solutions as the user's situation becomes clearer sequentially through the tree-structured questions such as questions Q1 to Q5.

[0037] Furthermore, if the user possesses extensive skills and knowledge, and / or if the difficulty of the user's request is low, it is possible that the user's request will be satisfied when the chatbot processing by the standard response unit 1 is completed, for example, if the user has received a satisfactory answer to their question. In this case, the flow shown in Figure 5 can be terminated at step S1.

[0038] On the other hand, the information processing device 20 of this embodiment is most effective when, even after the chatbot processing by the standard response unit 1 has finished, the user's response request has not been satisfied, and it would normally be necessary to hand over the request to an operator with specialized knowledge. Therefore, the explanation below will continue assuming that the request has not been satisfied. In such cases, as shown in the example at the bottom of Figure 6 as "Inquiry to Smart Chat," it is generally necessary to connect the user to an operator with specialized knowledge for issues that cannot be resolved by the chatbot alone, which incurs costs due to the need for human intervention. However, with this embodiment, it is expected that costs can be reduced by automating this human-intensive handover process using the RAG method of the information processing device 20.

[0039] Furthermore, criteria for automatically determining whether a handover is necessary, such as whether the user's response request has been met when the processing of the standard response unit 1 is completed, can also be pre-set as rules in the standard response DB 5. For example, in the example EX1 mentioned above, if the user ultimately provides answer A7 to question Q4, it can be set to indicate that the user's response request has not been met and that a handover process corresponding to step S2 and beyond is necessary.

[0040] Even in the case of a recommendation service, rather than customer support, the standard response unit 1 repeatedly asks the user standardized questions about their age, gender, occupation, personality, etc., and then classifies the user into a tree structure based on their answers. As a final question, it asks the user, for example, "We recommend three countries as travel destinations for you: A, B, and C. Which of the three would you like us to recommend?" If the user answers, "I would like to go to country A," the information processing device 20 of this embodiment takes over further and can provide detailed support, such as specifying which places to visit in country A and what kind of schedule to follow.

[0041] ●Step S2…Summary generation unit 2 (and LLM unit 7 as its internal processing) In step S2, the summary generation unit 2 summarizes the standard response history obtained from the standard response unit 1 in step S1 and outputs it to the similar case search unit 3. The summary generation unit 2 can obtain a summary as a response from the LLM unit 7 by inputting an input prompt to the LLM unit 7 that specifically lists the "standard response history" and requests it to summarize it in natural language text.

[0042] The method for creating input prompts to request summaries of standardized response histories can be pre-configured using a rule-based system, tailored to the specific service content provided by the information processing device 20, such as customer support. The wording for requesting a summary and the conditions to be included in the summary request may also be changed depending on the type of standardized response history content.

[0043] The type of content in the standard response history can be determined by, for example, which path was followed in the question-answer tree structure shown in the upper part of Figure 6. For example, in example EX1, it falls under the first type, and in example EX2, it falls under a different second type.

[0044] Figure 7 shows a schematic example of the processing of the summary generation unit 2. The upper part (sections 701-704) is an example of an input prompt that the summary generation unit 2 automatically creates and inputs to the LLM unit 7, and the lower part (sections 705, 706) is an example of an output prompt that the LLM unit 7 responds to the input prompt. Here, section 701, which instructs how to summarize, is the input prompt that needs to be designed in advance, and section 703, which is the object to be summarized, can use the content output by the standard response unit 1 as is, or it can use the content with a standard phrase indicating that it is the object to be summarized, such as "Please summarize the following," added to it.

[0045] In this embodiment, an input prompt can be prepared by designing a "section for instructing how to summarize" 701 in which a predetermined item to be used when creating a summary is specified according to the result of determining the content type of the standard response history. In the example in Figure 7, as shown in section 701, three items are given as the predetermined items: "item △△△", "item ○○○", and "item ■■■", and as shown in section 705, the LLM unit 7 provides a summary in the form of listing the contents of the three items as its response.

[0046] In one embodiment, in addition to specifying items for summary creation, such as the three items mentioned above, the "instruction on how to summarize" 701 may also include an "instruction to create the summary emphasizing the main points," as shown in section 702. As a result of including this "designated section to emphasize the main points" 702, section 706, which contains the "main points," appears in the output prompt. (That is, an embodiment is also possible in which section 706 does not appear in the output prompt by using an input prompt that omits the instruction to emphasize the main points in section 702.)

[0047] Furthermore, in response to instruction 702, which emphasizes the importance of key points, the LLM may be allowed to automatically determine which items in the standard response history constitute the key points without providing any prior knowledge. Alternatively, in a standard response history that takes a predetermined structure such as a tree structure as illustrated in Figure 6, it may be pre-configured to determine which one or more user responses constitute the key points, and this setting may be used as prior knowledge to construct the input prompt and give instructions to the LLM. Figure 7 shows an example of an input prompt using prior knowledge, where it is given as prior knowledge that the last part 704 of the standard response history 703 to be summarized constitutes the key points.

[0048] For example, in a customer support scenario like the one shown in Figure 6, where a user wants to resolve a problem, it is generally assumed that the last part of the standardized response history constitutes the key points of the user's response, and this can be used as prior knowledge to determine the key points. In the case of a travel recommendation service, for example, it is assumed that all or part of the travel destination, timing, and budget constitute the key points of the user's response, and in this case, it can be used as prior knowledge that these key points are not only in the last part of the standardized response history but also in the user's responses in between.

[0049] Regarding the distinction between items like the three items △△△, ○○○, and ■■■ in the example in Figure 7 and the main points, while "item" is a broad term in general, and "main points" may sometimes be considered a type of "item," in this embodiment, "item" and "main points" are distinguished and used as follows: "Items" play the role of selecting the most appropriate from among similar cases found in the similar case search unit 3, which will be described later, while "main points" concisely represent what the user who desires customer support or other assistance considers most important in the information processing device 20, that is, what the user specifically wants assistance with.

[0050] ●Step S3…Similar Case Search Unit 3 (Filtering by LLM Unit 7 can also be added) In step S3, the summary obtained from the summary generation unit 2 in step S2 is used as a search query, and the similar case search unit 3 searches the case DB 6 for cases similar to the summary. In one embodiment, the search results are output to the response text generation unit 4, or, as shown by the dotted arrow in Figure 2, the search results may be output as the response content to the user in their original form.

[0051] In the similar case search unit 3, for each case stored in the case DB 6, a summary is generated for each case using the same processing as the summary generation unit 2 used by the LLM unit 7. Then, those summaries determined to be similar to the search query can be used as search results. The similarity between summaries, which are documents, can be determined using any existing method for evaluating document similarity. For example, documents can be vectorized using BoW or TF-IDF based on the number of times each word appears in the document, and the similarity between document vectors can be evaluated as cosine similarity. If the similarity is determined to be high by a threshold, the summaries, which are documents, can be determined to be similar.

[0052] In the case database 6, to enable the above-mentioned search by the similar case search unit 3 at high speed, it is also possible to pre-store a summary of each case processed by the LLM unit 7 of the summary generation unit 2, or a document vector version of that summary, linked to the direct text of each case. Note that for the purpose of searching for similar cases by the similar case search unit 3, it is sufficient to store only the summarized versions in the case database 6. However, when outputting the search results from the similar case search unit 3 directly to the user, or when further processing them into response texts in the response text generation unit 4 described later, it is necessary to store the original text of the case (the case document) linked to the summary or document vector in the case database 6 and refer to that information.

[0053] As shown in Figure 4 above, when the case DB6 consists of the response case DB61 and the manual DB62, the similar case search unit 3 searches for similar cases within each DB. Specifically, it searches for individual response cases that are determined to be similar from the response case DB61, and searches for individual manuals that are determined to be similar from the manual DB62.

[0054] Furthermore, if the summary obtained from the summary generation unit 2 is generated under instructions to emphasize the key points, it is desirable to assign greater weight to the similarity judgment for the key points within the summary. For example, if the three items in the summary V obtained from the summary generation unit 2 are vectorized as document vectors v1, v2, and v3, and the content of the key points is vectorized as document vector v0, and a case W in the case DB 6 is similarly summarized so that the contents of the three items W1, W2, and W3 are vectorized as document vectors w1, w2, and w3, and the content of the key points is vectorized as document vector w0, then the similarity between the summary V and the case W, Sim(V,W), may be evaluated as shown in the following equation (1). Sim(V,W)=sim(v1,w1)+sim(v2,w2)+sim(v3,w3)+a*sim(v0,w0) …(1)

[0055] In equation (1) above, sim(x,y) is the similarity between document vectors x and y (e.g., cosine similarity), and a is a weight used to evaluate similarity by giving more weight to the main points than to the three items, with a predetermined value a>1. Furthermore, if the summary generated by the summary generation unit 2 does not use an instruction to give more weight to the main points, the summary will consist only of summaries for each item and will not contain a description of the main points. For example, if there are three items, the similarity can be evaluated by setting a=0 in the above equation.

[0056] Equation (1) evaluates the similarity between the summary V obtained from the summary generation unit 2 and the case W in the case DB 6 by evaluating the similarity for each item constituting the summary (items and key points if there are key points) and summing them up, to evaluate the similarity Sim(V,W) between the summary V obtained from the summary generation unit 2 and the case W in the case DB 6. In another embodiment, the entire summary V may be treated as a single document and represented as a single document vector v, and the entire case W may be treated as a single document and represented as a single document vector w, and the similarity between these documents may be evaluated as shown in the following equation (2). Sim(V,W) = sim(v,w) …(2)

[0057] Figure 8 is a schematic example of the processing in the similar case search unit 3, where the summary obtained by the summary generation unit 2 as the search query is shown as example AB, and it shows an example where one response case D61 in the response case DB 61 and one manual D62 in the manual DB 62 were found to be similar to summary AB. In other words, since it was determined that the summary AB61 of response case D61 and the summary AB62 of manual D62 were similar to the search query summary AB, the search results were obtained by finding that the original response case D61 and manual D62 before summarization were similar to summary AB.

[0058] Furthermore, in the response case DB61 and manual DB62, the number of results determined to be similar to the summary, which is the search query, is not limited to one, as shown in the example in Figure 8, but can be any number (it may even be zero).

[0059] As mentioned above, the similar case search unit 3 may further filter the search results using the LLM unit 7, so that only the selected results are included in the final search results. By inputting natural language text of a selection request based on usefulness as an input prompt to the LLM unit 7, the selection results can be obtained in the output prompt.

[0060] The input prompt can be pre-designed as a predetermined text, for example, as shown below. ●Example of an input prompt: "From the 'search results for similar items,' please select only those that are useful as a response to the 'summary' we are focusing on." Furthermore, the "summary" mentioned above refers to the summary generated by the summary generation unit 2.

[0061] Regarding selection requests based on usefulness, the method for determining usefulness may or may not be explicitly specified in the input prompt. (The above example of an input prompt is an example where it is not explicitly specified.) If the method for determining usefulness is explicitly specified, it may be done as follows, for example.

[0062] In other words, regarding how to set selection criteria after specifically determining usefulness, you can set predetermined selection criteria according to the content type automatically determined in the standard response history (standard response history before summarization) corresponding to the summary used as a search query, similar to how the content type of the standard response history was automatically determined by referring to the example in the upper part of Figure 6 above. Alternatively, if the summary consists of each item and its evaluation result, you can select those where the evaluation results for the same or corresponding items match or are similar. If the summary is generated with emphasis on key points, you can similarly select those where the key points match or are similar.

[0063] Figures 9 and 10 show schematic examples of input prompts (and output prompts) used in the similar case search unit 3 to select useful cases as described above. Figure 9 shows prompts for further filtering and selection from similar response cases in the response case DB 61, and Figure 10 shows prompts for further filtering and selection from similar manuals in the manual DB 62.

[0064] The prompt example in Figure 9 is an example of providing a specific criterion for determining usefulness by setting the condition that the evaluation results of items such as "item △△△" match (or are similar). Similarly, the prompt in Figure 10 sets the condition that "the inquiry can be resolved with a similar response" as a condition, and provides a specific criterion for determining usefulness based on the consistency of the item evaluation results. (In these examples, the inquiry consists of a description that allows the LLM unit 7 to specifically understand the summary generated by the summary generation unit 2.) These examples in Figures 9 and 10 are examples of allowing the LLM unit 7 to perform selection using a similar approach to the similarity evaluation using the following four similarity evaluation terms of formula (1) mentioned above. For example, if the similarity sim(v1,w1) is low, it is likely that there is a large content mismatch between the sentences V1 and W1 representing the corresponding original items, but such content mismatches can be automatically determined and selection can be achieved by input prompts to the LLM. sim(v1,w1), sim(v2,w2), sim(v3,w3), sim(v0,w0)

[0065] ●Step S4...Response text generation unit 4 (and LLM unit 7 as its internal processing) In step S4, the response text generation unit 4 generates a response text from the similar cases obtained by the similar case search unit 3 in step S3, and outputs it to the user as the response content, thus completing the flow shown in Figure 5.

[0066] As mentioned above, the search results from the similar case search unit 3 in step S3 may be output as the response content to the user. However, by generating a response text from the search results using the response text generation unit 4 in step S4 and then outputting it to the user as the response content, it is expected that the response content will be clearer to the user, resulting in a more valuable response. For example, if similar response cases found in the response case DB 61 or similar manuals found in the manual DB 62 are lengthy and include information other than what the user wants to know, generating a response text will be expected to extract only the information that the user wants to know.

[0067] Furthermore, both the similar case search results from the similar case search unit 3 in step S3 and the response text generated by the response text generation unit 4 in step S4 may be output as the response content to the user, or only one of them may be output as the response content.

[0068] The response text generation unit 4 can input an input prompt to the LLM unit 7 and generate a response text as an output prompt. Figure 11 shows a schematic example of the processing in the response text generation unit 4, and is a common example with the prompt examples described in each figure (Figures 7, 9, and 10) above.

[0069] As shown in the example in Figure 11, the response text generation unit 4 creates an input prompt under the following predetermined policy, and can obtain an output prompt, which is the response from the LLM unit 7, as a response text. The summary obtained by the summary generation unit 2 is used to create an input prompt, conveying the content the user desires to be addressed. The LLM unit 7 creates an input prompt to convey that its role as a virtual personality is to provide appropriate responses to the user. The similar cases obtained by the similar case search unit 3 are used to create an input prompt that indicates they are similar cases that can be used as reference when providing an appropriate response to the user.

[0070] As described above, according to this embodiment, by using the RAG method that utilizes the summaries generated by the summary generation unit 2, information that contributes to appropriate responses to the user can be obtained based on a large amount of successful response information pre-stored in the case DB 6 for matters that the user wishes to be addressed. Furthermore, from the user's perspective, after responding to the expected mechanical case-by-case responses with the standard response unit 1, which can be configured as a so-called chatbot, the user can quickly receive information on responses of a quality that is generally equivalent to that of a human response by an operator with specialized knowledge, etc. (without the user directly interacting with the LLM), and can gain satisfaction from the experience of having problems solved sequentially.

[0071] The following describes various supplementary and alternative examples relating to embodiments of the present invention.

[0072] (1) The embodiments of the present invention can provide further convenience to RAG, which is currently being actively researched and developed as a promising application of LLM, a type of generative AI (artificial intelligence) that performs automatic text generation, and can contribute to Goal 9 of the United Nations Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0073] (2) In the above embodiments, the response was realized by inputting and outputting text data, but a similar response may be realized for the user by inputting and outputting voice data. For example, instead of inputting text, the user may output the content that should be input as text data by speaking it to the terminal 10, record the voice data with the microphone provided in the terminal 10, convert the voice data into text data using any existing method such as deep learning, and then perform the same processing as above. Alternatively, for example, instead of the user seeing the result text obtained from the information processing device 20 on the display provided in the terminal 10, the result text may be converted from text data to voice data by voice reading processing using any existing method in the terminal 10, and the voice may be played back from the speaker provided in the terminal 10, so that the user can obtain the same information as the result text audibly. Input and output on the user side may be realized by any combination of voice recording / playback and text input / display.

[0074] (3) Figure 12 is a diagram showing an example of the hardware configuration of a typical computer device 70. The terminal 10 and the information processing device 20 in the information processing system 100 can each be realized as one or more computer devices 70 having such a configuration. When the terminal 10 or the information processing device 20 is realized with two or more computer devices 70, information necessary for processing may be sent and received via a network. The computer device 70 includes a CPU (Central Processing Unit) 71 that executes predetermined instructions, one or more dedicated processors 72 such as a GPU (Graphics Processing Unit) and / or NPU (Neural Network Processing Unit) that execute some or all of the execution instructions of the CPU 71 on behalf of or in cooperation with the CPU 71 and are specialized for specific calculations, RAM 73 as main memory that provides a work area to the CPU 71 (and dedicated processors 72), ROM 74 as auxiliary memory, a communication interface 75, a display 76 that outputs a display, an input interface 77 that accepts user input via a mouse, keyboard, touch panel, etc., a speaker 78 that outputs sound, a microphone 79 that receives sound input, and a bus BS for sending and receiving data between these.

[0075] Each functional unit of the information processing device 20 can be implemented by a CPU 71 and / or a dedicated processor 72 that reads and executes a predetermined program corresponding to the function of each unit from the ROM 74. Both the CPU 71 and the dedicated processor 72 are types of arithmetic units (processors). When display-related processing is performed, the display 76 also operates in conjunction; when communication-related processing for data transmission and reception is performed, the communication interface 75 also operates in conjunction; and when audio input / output-related processing is performed, the speaker 78 and microphone 79 also operate in conjunction.

[0076] When enabling text data and audio input / output to the user on terminal 10, the display 76, input interface 77, speaker 78, microphone 79, etc., can be used. [Explanation of Symbols]

[0077] 100... Information processing system, 10... Terminal, 20... Information processing device 1…Standard response unit, 2…Summary generation unit, 3…Similar case search unit, 4…Response text generation unit, 5…Standard response database, 6…Case database, 7…LLM unit 61...Response Case Database, 62...Manual Database

Claims

1. An information processing device that generates information related to the response to a user by taking over the history of questions and answers between a user who requests a response and the user who provides the response, A summary generation process that generates a summary of the history, The process involves performing a similarity search to find items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. The search results found to be similar to the above summary are used as the results of generating information regarding the interaction with the user. In the summary generation process, a text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation process, the text to be summarized is provided, including the key points of the matters the user wishes to be addressed, in accordance with the content of the history. In the summary generation process described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search process described above, similar items are searched by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing device characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

2. An information processing device that takes over the history of questions and answers between a user who requests assistance and the user who provides the assistance, and generates a response message which is information related to the assistance provided to the user, A summary generation process that generates a summary of the history, The process involves performing a similarity search to find items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. Further, a response text generation process is performed to generate the response text from the search results of similar items. In the response text generation process, the large-scale language model is given text as input text stating that the response should be provided to the user in the summary situation, with reference to the response implementation information found as similar, and the large-scale language model generates the response text as output text. In the summary generation process, a text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation process, the text to be summarized is provided, including the key points of the matters the user wishes to be addressed, in accordance with the content of the history. In the summary generation process described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search process described above, similar items are searched by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing device characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

3. The information processing device according to claim 1 or 2, characterized in that the multiple response realization information sets up in advance as a database includes information on customer support response cases.

4. The information processing device according to claim 1 or 2, characterized in that the multiple response realization information sets that have been pre-constructed as a database include information on the operation procedures of the equipment.

5. The information processing device according to claim 1 or 2, characterized in that the multiple response implementation information sets up in advance as a database includes information on troubleshooting procedures.

6. An information processing method that generates information related to the response to a user by taking over the history of the question-and-answer exchange between the questions provided to the user who desires a response and the answers obtained from the user, A summary generation procedure for generating a summary of the history, The computer performs a similarity search procedure to search for items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. The search results found to be similar to the above summary are used as the results of generating information regarding the interaction with the user. In the summary generation procedure described above, text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation procedure, the text to be summarized is provided in a manner that includes the key points of the matters the user wishes to be addressed, corresponding to the content of the history. In the summary generation procedure described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search procedure described above, similar items are searched for by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing method characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

7. An information processing method that generates a response text, which is information related to the response to a user, by taking the history of the question-and-answer exchange between the questions provided to a user who desires a response and the answers obtained from that user, A summary generation procedure for generating a summary of the history, The computer performs a similarity search procedure to search for items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. The computer further executes the response text generation procedure, which generates the response text from the search results of similar items, In the above response text generation procedure, the input text to the large-scale language model is a text stating that the response should be provided to the user in the summary situation, with reference to the response implementation information found as similar, and the output text from the large-scale language model is the response text. In the summary generation procedure described above, text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation procedure, the text to be summarized is provided in a manner that includes the key points of the matters the user wishes to be addressed, corresponding to the content of the history. In the summary generation procedure described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search procedure described above, similar items are searched for by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing method characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

8. An information processing program that takes over the history of questions and answers between a user who requests assistance and the user who provides the assistance, and generates information related to the assistance provided to that user, A summary generation procedure for generating a summary of the history, The computer is instructed to perform a similarity search procedure, which searches for items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. The search results found to be similar to the above summary are used as the results of generating information regarding the interaction with the user. In the summary generation procedure described above, text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation procedure, the text to be summarized is provided in a manner that includes the key points of the matters the user wishes to be addressed, corresponding to the content of the history. In the summary generation procedure described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search procedure described above, similar items are searched for by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing program characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

9. An information processing program that takes the history of questions and answers between a user who requests assistance and the user who provides the assistance, and generates a response message which is information related to the assistance given to the user, A summary generation procedure for generating a summary of the history, The computer is instructed to perform a similarity search procedure, which searches for items similar to the summary from among multiple sets of response implementation information that have been pre-built as a database. The computer is then made to execute the response text generation procedure, which generates the response text from the search results of similar items. In the above response text generation procedure, the input text to the large-scale language model is a text stating that the response should be provided to the user in the summary situation, with reference to the response implementation information found as similar, and the output text from the large-scale language model is the response text. In the summary generation procedure described above, text indicating that the history should be summarized is provided as input text to the large-scale language model, and the summary is generated as output text from the large-scale language model. In the summary generation procedure, the text to be summarized is provided in a manner that includes the key points of the matters the user wishes to be addressed, corresponding to the content of the history. In the summary generation procedure described above, the summary is divided into multiple items including the key points, and a summary is generated for each item. In the similarity search procedure described above, similar items are searched for by calculating the similarity between each of the multiple response realization information and the summary, The similarity is calculated by weighting the similarity sum of the texts for each of the multiple items in each of the multiple response realization information, and the texts for each of the items in the summary. An information processing program characterized in that, as the weights used in the weighted sum, the weight for the key point among the multiple items is greater than the weights for the items other than the key point.

Images