Response device and program

Abstract

A response device according to the present invention comprises: an input unit that is configured to receive input of first text which indicates a question related to a certain task; a search unit that is configured to acquire, from among information pertaining to the task, similarity information which is similar to the first text; a prediction unit that is configured to acquire related information which is stored in association with the similarity information, generate second text which requests the generation of a response to the first text with reference to the similarity information and the related information, and input the second text into a machine learning model which is trained to generate text according to input text; and an output unit that is configured to output text which has been generated by the machine learning model. Thus, it is possible to provide professional answers to questions in an easily understandable format.

Description

Response device and program

[0001] The present invention relates to a response device and a program.

[0002] In recent years, the task of maintaining and operating IT systems has become more difficult. Today's IT systems are composed of a complex combination of diverse components, and a failure in one component can affect multiple functions or other components, making it extremely difficult to identify and resolve the cause of the failure. As a result, the amount of knowledge required of operators who operate the systems to keep them running constantly is much greater than before, placing a significant burden on operators. In response to this increasing difficulty in operations, recommendation systems are being considered that recommend useful information such as the next step to take based on past similar events.

[0003] A recommendation system is a system that returns necessary information in response to an information search query from an operator, such as an error message or alarm message, by using documents such as response history (hereinafter referred to as "action history") of past events written by other operators and response manuals.By using a recommendation system, even an inexperienced operator, or even an operator who has no experience operating the target system, can obtain information from past events and related documents and proceed with the response.

[0004] An example of a recommendation system in system operation is a ticket routing method that determines the assignment destination of a trouble ticket (action history) (Non-Patent Document 1). This method predicts the operator who will shorten the resolution time based on the description of the action history.

[0005] Shao, Qihong, et al., "EasyTicket: a ticket routing recommendation engine for enterprise problem resolution", Proceedings of the VLDB Endowment 1.2 (2008): 1436-1439

[0006] However, conventional recommendation systems are unable to return answers that are easy for humans to understand and that will truly improve the efficiency of operators' work.

[0007] Most conventional recommendation systems focus on technology that returns the most relevant information for given information. For example, an expert system that outputs a history of related actions is an example of a recommendation system. In this case, however, the system must read the history of past actions, interpret it by substituting it for the current problem, and then derive an appropriate response. This type of work is difficult for inexperienced operators, and it is difficult to say that it truly improves work efficiency.

[0008] Generative AI, or large-scale language models (LLMs), have recently attracted considerable attention as a general-purpose work support system that goes beyond IT system operations. LLMs are a collective term for systems and techniques that train models to provide appropriate responses to users based on large amounts of data (usually text data, but often also images and audio). These systems then respond to queries with highly natural-sounding text (or images or audio). A major advantage of LLMs is their ability to generate natural, reliable answers, even when users' questions are in non-standard formats, such as natural language or images. Because they can respond to ambiguous questions when users themselves are unable to clearly articulate their problems, LLMs are also expected to be useful in support systems for inexperienced operators, i.e., recommendation systems.

[0009] On the other hand, LLMs are known to be prone to giving incorrect answers in situations that require specialized knowledge of specific systems.

[0010] LLMs are said to have hundreds of millions of parameters to ensure they can return accurate responses, and they are trained using a huge amount of data available around the world. As a result, LLMs' answers are based on general knowledge and do not have knowledge of specific systems, especially specialized knowledge of in-house systems that are only shared within a company. For this reason, simply using LLMs makes it difficult to create a recommendation system that is specialized for dealing with specific systems.

[0011] In summary, conventional recommendation systems place emphasis on acquiring specialized knowledge, making it difficult to improve the work efficiency of operators, while LLMs have the problem of being unable to return appropriate information because they do not have the specialized knowledge.

[0012] To improve the efficiency of operators, a recommendation system is needed that can return specialized knowledge that exists only within an organization, such as within a company, in a format that is easy for operators to understand.

[0013] The present invention has been made in view of the above points, and has as its object to provide specialized answers to questions in an easy-to-understand format.

[0014] In order to solve the above problem, the response device has an input unit configured to input a first text indicating a question about a certain business; a search unit configured to acquire similar information that is similar to the first text from information related to the business; a prediction unit configured to acquire related information stored in association with the similar information, generate second text that requests that a response to the first text be generated by referring to the similar information and the related information, and input the second text to a machine learning model trained to generate text corresponding to the input text; and an output unit configured to output the text generated by the machine learning model.

[0015] To provide specialized answers to questions in an easy-to-understand format.

[0016] 1 is a diagram illustrating an example of a hardware configuration of the response device 10 in the first embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of the response device 10 in the first embodiment. FIG. 3 is a flowchart for explaining an example of a processing procedure executed by the response device 10. FIG. 4 is a diagram illustrating an example of a question query. FIG. 5 is a diagram illustrating an example of similar information. FIG. 6 is a diagram illustrating an example of a first format of a context-attached sentence. FIG. 7 is a diagram illustrating an example of a second format of a context-attached sentence. FIG. 8 is a diagram illustrating an example of a context-attached sentence in the first embodiment. FIG. 9 is a diagram illustrating an example of a response sentence in the first embodiment. FIG. 10 is a diagram illustrating an example of an input / output interface in the present embodiment. FIG. 11 is a diagram illustrating an example of a functional configuration of the response device 10 in the second embodiment. FIG. 12 is a diagram illustrating an example of a process. FIG. 13 is a diagram illustrating an example of an action history to which a work name is added. FIG. 14 is a diagram illustrating an example of a context-attached sentence in the second embodiment. FIG. 15 is a diagram illustrating an example of a response sentence in the second embodiment.

[0017] The first embodiment will be described below with reference to the drawings. Fig. 1 is a diagram showing an example of the hardware configuration of a response device 10 in the first embodiment. The response device 10 in Fig. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, which are all interconnected via a bus B.

[0018] A program that realizes processing in the response device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0019] When an instruction to start a program is received, the memory device 103 reads and stores the program from the auxiliary storage device 102. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the response device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0020] 2 is a diagram showing an example of the functional configuration of the response device 10 in the first embodiment. In FIG. 2, the response device 10 has an input unit 11, a search unit 12, a prediction unit 13, a response unit 14, and an output unit 15. Each of these units is realized by a process in which one or more programs installed in the response device 10 are executed by the processor 104. Furthermore, each of these units may function on one computer (a physical or virtual computer) or may function in a distributed manner on multiple computers. In other words, the response device 10 may be composed of multiple computers.

[0021] The input unit 11 receives text (hereinafter referred to as a "query") indicating a question about a certain task from a user. The user is, for example, an operator who performs tasks related to the operation of an IT system. In this case, the question is a question about the operation of the IT system.

[0022] The search unit 12 acquires (extracts) information similar to the question received by the input unit 11 (hereinafter referred to as "similar information") from a knowledge DB 20, which is a collection of specialized knowledge related to the user's work. Here, specialized knowledge is a general term for information related to a system (hereinafter referred to as an "operation system") that the operator, who is the user in this embodiment, is operating and wants to resolve a problem. There may be one or more knowledge DBs 20. FIG. 2 shows an action history DB 21 and a manual DB 22 as examples of the knowledge DB 20. Each of these databases can be realized using, for example, an external system such as the auxiliary storage device 102 or a storage device connectable to the response device 10 via a network.

[0023] The action history DB 21 stores information indicating, in chronological order, the actions taken when past operational system problems occurred (hereinafter referred to as "action history").

[0024] The manual DB 22 stores a maintenance manual for each alert message of the operational system.

[0025] The prediction unit 13 acquires information (hereinafter referred to as "related information") associated with the similar information acquired by the search unit 12 and stored in the knowledge DB 20, generates text requesting that a response to the question query be generated by referring to the similar information and related information, and inputs the text to the response unit 14.

[0026] The response unit 14 is a machine learning model trained to generate text corresponding to input text. Specifically, the response unit 14 is an LLM such as chatGPT or Llama. The response unit 14 performs calculations based on learned parameters on the text input by the prediction unit 13, and generates appropriate text as a response (an answer to a question). Note that an LLM is a large language model. However, as long as the requirement that the input and output are text is satisfied, a simple language model other than an LLM, such as Transformer or its advanced form, BERT, may also be used as the response unit 14.

[0027] The output unit 15 outputs the text generated by the response unit 14 .

[0028] The following describes the processing procedure executed by the response device 10. FIG.

[0029] In step S101, the input unit 11 accepts a question query about the operational system from the user.

[0030] Fig. 4 is a diagram showing an example of a query. The query q1 shown in Fig. 4 is text indicating the content of an alarm output by the operation system (the content of an event that has occurred in the operation system) and an inquiry about what should be done in response to the alarm (event).

[0031] Next, the search unit 12 acquires some of the texts with the highest scores from the text groups stored in each knowledge DB 20 as similar information (S102). In a general search, the score of each text represents the degree of similarity between the query and the text. Therefore, the most similar text is acquired as similar information.

[0032] FIG. 5 is a diagram illustrating an example of similar information. FIG. 5 illustrates an example in which two action histories d1 and d2, which are texts having action history IDs of 100000001 and 100000103, are acquired as similar information. It is assumed in this example that the similarity between these two action histories is relatively high because both contain the word string "session request timeout" included in the query. In this embodiment, the action history is divided into phrases (units of a document separated by blank lines), and each phrase is considered to be a single piece of text. In the example of FIG. 5, of the two action histories having action history IDs of 100000001 and 100000103, phrases p11 and p21, which are the phrases with the highest similarity to the query (here, the first phrases of each), are acquired as similar information.

[0033] Generally, text similarity is often calculated based on vectorization of documents (see "Reference 1" below), and this embodiment also assumes the use of a similar method.

[0034] In this vectorization technique, a large amount of document data is given, and parameters are pre-trained to convert all documents into vectors with the same number of dimensions based on the frequency and order of the words and characters they contain. At this time, the vectors of each document are trained so that the more similar the words and characters in the documents, the closer the vectors are to each other. Using the trained parameters, any word string (or word set) can be converted into a vector, so all documents in the knowledge DB 20 are vectorized. Queries are also similarly treated as word strings and converted into vectors.

[0035] In order to obtain documents that are as similar as possible to the query, the search unit 12 selects documents having vectors with a relatively large inner product with the vector of the query. The larger the inner product, the closer the frequency and order of appearance of words and characters in the two documents.

[0036] Although we have described the vector-based method above, in practice, any method that can define the similarity between two texts is acceptable. Alternatively, we can simply create a frequency distribution of characters in a document and select text that has a frequency distribution relatively close to the frequency distribution of the query.

[0037] The number of similar information pieces obtained by the search unit 12 does not have to be one, as in the above example.

[0038] For example, it is possible to extract k texts with high similarity (k is a setting value provided by the system administrator in this embodiment), extract an indefinite number of texts that fall into the context described below, or obtain text from each of multiple knowledge DBs 20.

[0039] Next, the prediction unit 13 generates text to be input to the LLM (answer model) as a context-attached sentence based on the similarity information acquired by the search unit 12 (S103). A context refers to a sentence that is passed along with a question sentence in the LLM and contains information that leads to (is related to) an answer to the question.

[0040] The role of the prediction unit 13 is to extract "expert knowledge" from the similar information that will contribute to providing a response to the user. The information extracted by the prediction unit 13 as expert knowledge is hereinafter referred to as "related information." Because similar information is simply text similar to the query, it is not necessarily the information the user wants to obtain. For example, suppose a user wants to know how to deal with a certain error message. In this case, for the error message included in the query by the user, the search unit 12 extracts text containing the error message from the knowledge DB 20 as similar information. However, what the user wants to know is not the error message itself, but the corresponding solution. To meet the user's request, it is necessary to obtain text describing the solution from the similar information and respond to the request. Therefore, the role of the prediction unit 13 is to acquire (extract) the "expert knowledge" of the solution as related information from the "similar information" that describes the error message.

[0041] A simple implementation of the prediction unit 13 is to store text containing (associated with) both a question query and specialized knowledge in the knowledge DB 20. For example, an alarm response manual, which is one of commonly used operation manuals, describes the actions to be taken in response to an alert message.

[0042] In this case, the search unit 12 searches for the most similar pair of question query and alert message from among a set of pairs (alert message, action to be taken) included in the response manual stored in the manual DB 22, and passes the pair to the prediction unit 13 as similar information. The prediction unit 13 simply acquires the "action to be taken" of the pair obtained as input as related information, and uses the related information as context to create a sentence with context. Specifically, the prediction unit 13 generates a sentence with context by inserting the context ("action to be taken") and the question query into a predetermined format (template).

[0043] For example, the prediction unit 13 can generate a sentence with a context by inserting an alert message, a measure to be taken, and a question query into the {{alert message}}, {{measure to be taken}}, and {{question query}} parts, respectively, in a format such as that shown in Fig. 6. The format at this time is assumed to be given in advance.

[0044] In this case, the terms that can be included in the question query may be restricted, for example, by allowing only alert messages to be specified in the question query, or by making the question query an alert selection form rather than a free text entry.

[0045] Alternatively, in the case of a knowledge DB 20 such as the action history DB 21 having a chronological structure, similar information and specialized knowledge can be linked using two or more consecutive phrases. Generally, the action history records the actions taken by an operator in chronological order, i.e., the order in which the tasks were performed. Therefore, if a certain task A is described in a certain phrase in the action history, the phrase following phrase A is likely to represent the task performed after A. Therefore, when the most similar phrase B is obtained (as similar information) for a certain query, if the phrase C following phrase B is considered to be specialized knowledge (related information), then phrase C is likely to be the task that the user should perform next.

[0046] 5 is searched by the search unit 12, if the phrases p11 and p21 are acquired as similar information, the prediction unit 13 acquires the following phrases p12 and p22 in addition to these as specialized knowledge (related information = context) indicating the next task to be performed by the operator. In this case, the prediction unit 13 inserts the acquired specialized knowledge (related information = context) into {{context 1}}, {{subsequent sentence of context 1}}, {{context 2}}, {{subsequent sentence of context 2}}, and {{question query}}, respectively, in a format (template) as shown in FIG. 7, thereby generating a context-added sentence c1 as shown in FIG.

[0047] It should be noted that when using such a prediction unit 13, it is an implicit requirement that the action history DB 21 stores the work procedures in chronological order. As already mentioned, since the action history generally records the work contents in chronological order, this assumption is valid for many system operations.

[0048] Next, the prediction unit 13 inputs the generated context-attached sentence to the response unit 14 (S104).

[0049] The response unit 14, which is an LLM, generates a response sentence based on the context-attached sentence and outputs the response sentence to the output unit 15 (S105).

[0050] The output unit 15 outputs the response sentence (S106).

[0051] FIG. 9 is a diagram showing an example of a response sentence in the first embodiment. Response sentence a1 shown in FIG. 9 is an example of a response sentence output in response to the question query shown in FIG. 4. In this embodiment, it is assumed that an LLM is used as the response unit 14, and a context-attached sentence is provided to the LLM, and the response returned by the LLM is output as is. For example, input / output (interaction) between the user and the response device 10 is performed via a screen g1 as shown in FIG. 10. Note that in FIG. 10, the same parts as those in FIG. 4 or FIG. 9 are assigned the same reference numerals.

[0052] According to this embodiment, the user does not need to acquire or understand specialized knowledge. From the input / output example in FIG. 10 , it can be seen that the answer appears to the user to have been obtained using a conventional LLM. In reality, the system internally searches for documents similar to the question (query), extracts necessary related information from the documents as context, and generates the answer together with the context using the LLM. However, because the user does not see the specialized knowledge (related information), there is no need for interpretation, allowing the user to perform the task smoothly.

[0053] The prediction unit 13 is the most distinctive component that distinguishes this embodiment from other related techniques. The configuration of this embodiment has the advantage that the prediction unit 13 can be freely changed.

[0054] In a related technique, Search Expansion Generation (RAG), a query x ｔ Similar information z with a vector having a large inner product ｔ After extracting the similar information, the set of query and past responses (z ｔ , x ｔ , y １ , ..., y ｔ－１ ) and the response y ｔ In other words, the existing method has a configuration in which the search unit 12 and the response unit 14 are directly connected.

[0055] In contrast, in this embodiment, z ｔ The prediction unit 13 calculates the specialized knowledge (related information) w required for the response using the ｔ The response unit 14 additionally extracts similar information, specialized knowledge (related information), a set of questions and past responses (z ｔ , x ｔ , w ｔ , y １ , ..., y ｔ－１ ) to obtain the response y ｔ By using the prediction unit 13, the expert knowledge w ｔ The advantage of this embodiment is that it is possible to incorporate the above into the reference information of the response.

[0056] Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the second embodiment may be the same as those in the first embodiment.

[0057] In the first embodiment, the prediction unit 13 receives similar information of a question query and acquires subsequent sentences of the similar information as related information. As another example, related information that can improve the accuracy of answers can be acquired as context by using a knowledge DB 20 separate from the action history DB 21 and the manual DB 22.

[0058] In the second embodiment, an example will be described in which a business process is provided to the prediction unit 13 as additional information to realize more appropriate presentation of the next task.

[0059] Fig. 11 is a diagram showing an example of the functional configuration of the response device 10 in the second embodiment. In Fig. 11, the same parts as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted.

[0060] In FIG. 11, a process DB 23 is added to the knowledge DB 20 that the search unit 12 accesses.

[0061] The process DB 23 stores "processes," which are information indicating the procedures for tasks related to a user's business. A process is a flowchart-type graph that represents the tasks required to perform a certain business and the correct order in which they should be performed.

[0062] Fig. 12 is a diagram showing an example of a process. Fig. 12 shows a process that represents the correct response procedure when an alarm "Session request timeout" is output from the operational system. When no symbol is present in the diamond in the diagram, only the task at the end of one of the arrows (for example, either "Issue early warning" or "Watch and wait") is performed. When a diamond has a + symbol, it means that all the tasks at the end of the arrows (for example, "Contact verification," "Detour verification," and "Inquire with the other department") are performed in any order.

[0063] In the second embodiment, after acquiring the similarity information, the search unit 12 acquires a process from the process DB 23 using the similarity information and the query, and passes them to the prediction unit 13 .

[0064] Each process has a process name that indicates what business the process is for. The search unit 12 retrieves appropriate processes with corresponding process names (for example, processes whose process names include a character string corresponding to "event") from the process DB 23 using the character string "event" in the action history as similar information as a key. From a broader perspective, all processes stored in the process DB 23 correspond to various types of business that make up the user's business (business related to the operation of the IT system).

[0065] In the second embodiment, the prediction unit 13 acquires related information based on a process.

[0066] Methods for acquiring process-based related information (context) can be broadly classified into two types.

[0067] The first method is for the prediction unit 13 to acquire the process diagram (graph) itself as related information. Recent LLMs have become capable of reading diagrams, and by inputting the diagram and adding an explanation of the notation, it is expected that the response unit 14 will be able to understand the business content.

[0068] The second method utilizes a predictive process monitoring (PPM) technique in the prediction unit 13 to predict the next task based on similarity information and the process, and acquires information related to the predicted task from the information stored in the knowledge DB 20 as related information. PPM is a technique that inputs a certain process and a log of the history of a certain task corresponding to that process up to the middle, and predicts all the next tasks along that process. Examples of PPM techniques include the technique described in Reference 2 below. In addition to the process (a diagram showing it), PPM can predict the next task by providing a task log with similarity information for all queries entered in the dialogue (dialogue between the user and the response device 10) regarding a certain event up to that point. However, existing techniques assume that the task log is a sequence of task IDs indicating which tasks have been performed, and similarity information (e.g., each phrase in the action history) is likely to not include task IDs as is. Therefore, for each clause described in chronological order in the action history, the task name of the task corresponding to that clause (as a task ID) should be included in a predetermined format so that the search unit 12 can recognize it as a task name.

[0069] Fig. 13 is a diagram showing an example of an action history to which a task name has been added. Fig. 13 shows an example in which the task name has been added to each phrase in the format of "<time> [<task name>]" for the action history d1 shown in Fig. 5. Specifically, descriptions d1, d2, and d3 are descriptions that include the task name.

[0070] In this case, the search unit 12 acquires similar information (each phrase in the action history) by treating the sequence of question queries as a single question query. The search unit 12 also acquires the task names added in a predetermined format from the acquired similar information. In the case of the action history d1 in FIG. 13, a sequence of task names such as {alarm occurrence, communication check, detour communication check, ...} (hereinafter referred to as the "task name sequence") is acquired. When multiple pieces of similar information are acquired, a task name sequence is acquired for each piece of similar information. The search unit 12 inputs all of the acquired task name sequences to the prediction unit 13.

[0071] The prediction unit 13 predicts the next task name for each task name string using PPM, acquires a phrase containing the predicted task name as related information from similar information (action history) corresponding to the task name string, and includes the acquired phrase (related information) in a sentence with context.

[0072] 14 is a diagram showing an example of a context-attached sentence in the second embodiment. In the context-attached sentence c2 shown in FIG. 14, a description c21 is a portion including a phrase obtained by PPM. The example shows that the work name predicted by the prediction unit 13 is "inquiry to the other department," and a phrase including the work name is added as description c21.

[0073] The main advantages of using a process are that it allows for more accurate task recommendations by adding next task candidates, and prevents the wrong (and sometimes wrong) tasks from being performed. In the example process shown in FIG. 12, after an alarm occurs, not only "communication check" but also "inquiry to other department" may be performed. Therefore, the subsequent sentence in context 1 includes a description of the past action history corresponding to "inquiry to other department" in addition to the description corresponding to "communication check." Description c21 can be obtained by adding the task name to each phrase in the action history in advance, as described above.

[0074] The response unit 14 receives such a context-attached sentence and generates, for example, a response sentence a2 as shown in Fig. 15. While the response sentence a1 (Fig. 9) of the first embodiment was only able to recommend a communication check, the response sentence a2 can generate a response that takes into account both of the two possible tasks.

[0075] In addition, LLMs are known to suffer from a phenomenon called hallucination, in which they return inappropriate answers, but the LLM's responses can be limited so that it does not recommend work that does not follow the process.

[0076] For example, in the process of Figure 12, we want to avoid a situation where an "alarm is generated" is issued immediately after an "alarm is generated." Also, among the tasks that can make up a process, there are tasks such as "device reset" that have a large impact on the system and can sometimes lead to larger problems, so it can be dangerous for the LLM to make unlimited decisions about which tasks should be performed.

[0077] According to the above, immediately after an "alarm occurrence," tasks that may occur immediately after the "alarm occurrence" ("communication confirmation," "inquiry with the other department") are included in a context-attached statement using a process, thereby reducing the possibility of other tasks being recommended. Furthermore, because processes can be edited arbitrarily, by linking with the process DB 23, the system administrator can implicitly control the tasks that users should perform. For example, by editing a process, it is possible to prevent dangerous tasks from being recommended.

[0078] As described above, according to each of the above embodiments, it is possible to provide specialized answers to questions in an easy-to-understand format, which makes it possible to realize, for example, highly efficient support for IT system operation by inexperienced operators, which was not possible with conventional recommendation systems.

[0079] [Reference 1] Lewis, Patrick, et al. "Retrieval-augmented generation for knowledge-intensive nlp tasks." Advances in Neural Information Processing Systems 33 (2020): 9459-9474 [Reference 2] Watanabe Satoru, Ikeuchi Mitsuki, Takahashi Yosuke, "Predictive process monitoring method for decision support in troubleshooting," Institute of Electronics, Information and Communication Engineers General Conference, B-14-11, 2023 The above describes in detail the embodiments of the present invention, but the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0080] REFERENCE SIGNS LIST 10 Response device 11 Input unit 12 Search unit 13 Prediction unit 14 Response unit 15 Output unit 20 Knowledge DB 21 Action history DB 22 Manual DB 23 Process DB 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device B Bus

Claims

1. A response device comprising: an input unit configured to input a first text indicating a question about a certain business; a search unit configured to retrieve similar information similar to the first text from information related to the business; a prediction unit configured to retrieve related information stored in association with the similar information, generate second text requesting that a response to the first text be generated by referring to the similar information and the related information, and input the second text to a machine learning model trained to generate text corresponding to the input text; and an output unit configured to output the text generated by the machine learning model.

2. The response device according to claim 1, wherein the machine learning model is a large-scale language model.

3. The response device according to claim 1, characterized in that the search unit is further configured to acquire information indicating the procedure of a task related to the business, and the prediction unit predicts the next task after the task corresponding to the similar information based on the similar information and the information indicating the procedure, and acquires information related to the next task from among the information stored in association with the similar information as the related information.

4. A program that causes a computer to execute the following steps: an input step of inputting first text indicating a question about a certain business; a search step of retrieving similar information that is similar to the first text from information related to the business; a prediction step of retrieving related information stored in association with the similar information, generating second text that requests the generation of a response to the first text by referring to the similar information and the related information, and inputting the second text into a machine learning model that has been trained to generate text corresponding to the input text; and an output step of outputting the text generated by the machine learning model.

Images