Automated response method, automated response system, and computer program

Abstract

This provides an automated response method that enables the generation of highly accurate answers even when using generational AI. [Solution] The automated response method includes the steps of: receiving user input information from a user terminal via a communication network; extracting question information from the user input information; when an appropriate answer candidate for the question information is obtained by a non-generating AI conversational system, sending response information including the appropriate answer candidate as the answer to the user terminal; when an appropriate answer candidate for the question information is not obtained by the non-generating AI conversational system, providing an input prompt to at least one generating AI requesting the generation of a counter-question for the question information to be output by the at least one generating AI; and sending response information including the counter-question to the user terminal.

Description

[Technical Field]

[0001] This disclosure relates to automated response technologies in conversational systems such as chatbots and automated response robots, and more particularly to automated response technologies in conversational systems using Generative Artificial Intelligence (AI). [Background technology]

[0002] In recent years, conversational systems such as chatbots and automated response robots have been utilized in various fields. A conversational system is a computer system configured to interact with a user, and is also called a conversational agent. Generative AI based on a language model (LM) is one form of conversational agent technology. As an example of generative AI, ChatGPT (Chat Generative Pre-trained Transformer) provided by OpenAI is widely known. Generally, generative AI has the function of receiving input data called a prompt and generating content data such as language data, sound data, or image data in response to that prompt. The technique of controlling the content of the prompt to obtain the desired content data response is called prompt engineering.

[0003] Prior art related to prompt engineering is disclosed, for example, in Japanese Patent No. 7511068 (Patent Document 1). The information processing device disclosed in Patent Document 1 is configured to perform prompt engineering using a generation AI (first model) to generate an extended prompt from a user's question, input the extended prompt to an external generation AI (second model), and obtain a response corresponding to the extended prompt from this second model.

[0004] It is known that in generative AI, a phenomenon called hallucination occurs, where it generates plausible but factually incorrect misinformation as an answer. As one of the techniques for suppressing hallucination, the Retrieval-Augmented Generation (RAG) technique that uses search information obtained from external knowledge sources, which has not yet been learned in the large language model (LLM), is known. The RAG technique is disclosed, for example, in Non-Patent Document 1 below.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Although hallucination suppression technologies such as the aforementioned RAG technology exist, a technology that completely prevents hallucination has not yet been established. This poses a problem when attempting to introduce generating AI into automated response systems that require high reliability.

[0008] To avoid such problems, rule-based chatbots with multiple predefined rule patterns can be introduced without implementing generative AI. However, rule-based chatbots have the challenge of being unable to generate appropriate answers to ambiguous inquiries that fall outside the scope of the rule patterns.

[0009] In view of the foregoing, the purpose of this disclosure is to provide an automated response method, an automated response system, and a computer program that enable the generation of highly accurate responses even when using generational AI. [Means for solving the problem]

[0010] An automated response method according to a first aspect of the present disclosure is an automated response method performed in an automated response system configured to communicate with a user terminal via a communication network, comprising the steps of: receiving user input information from the user terminal via the communication network; extracting question information from the user input information; when an appropriate answer candidate for the question information is obtained by a non-generating AI dialogue system, transmitting response information including the appropriate answer candidate as an answer to the user terminal; when an appropriate answer candidate for the question information is not obtained by the non-generating AI dialogue system, providing an input prompt to at least one generating AI requesting the generation of a counter-question for the question information, causing the at least one generating AI to output the counter-question; and transmitting response information including the counter-question to the user terminal.

[0011] An automated response system according to a second aspect of this disclosure is configured to perform an automated response method according to a first aspect.

[0012] A computer program according to a third aspect of the present disclosure is a computer program comprising a series of instructions, which, when executed by one or more processors, causes the one or more processors to execute an automated response method according to the first aspect. [Effects of the Invention]

[0013] According to the first to third aspects of this disclosure, when a non-generating AI conversational system obtains an appropriate candidate answer to a question extracted from user input information, it can send response information containing the appropriate candidate answer as the answer to the user terminal. If no appropriate candidate answer to the question information is obtained, it can provide the generating AI with an input prompt requesting the generation of a counter-question for the question information, causing the generating AI to output a counter-question. Therefore, the generating AI is used to generate a counter-question for the question information extracted from user input information, but it is not used to directly generate an answer to the question information. A non-generating AI conversational system may be used to generate highly accurate answers to the question information. Thus, by using a non-generating AI conversational system and a generating AI in combination, it is possible to generate highly accurate answers and provide a reliable automated response system. [Brief explanation of the drawing]

[0014] [Figure 1] This is a schematic diagram of an automated response system according to one embodiment of the present disclosure. [Figure 2] This figure shows an example of an avatar image displayed on the user's terminal screen. [Figure 3] This is a schematic diagram of the input prompt configuration. [Figure 4] This is a diagram illustrating a system prompt. [Figure 5] This is a schematic diagram of a computer hardware configuration example in an automated response system according to one embodiment of the present disclosure. [Figure 6]This flowchart schematically shows an example of the procedure for an automated response processing according to one embodiment of the present disclosure. [Figure 7] This diagram shows examples of user questions and counter-questions generated by the AI ​​in a table format. [Modes for carrying out the invention]

[0015] Next, embodiments and modified examples thereof relating to this disclosure will be described in detail with reference to the drawings. Components denoted by the same reference numerals throughout the drawings are assumed to have the same configuration and function.

[0016] Figure 1 is a schematic diagram of an automated response system 1 according to one embodiment of the present disclosure. As shown in Figure 1, the automated response system 1 consists of multiple user terminals T1, ..., T N The system comprises a front system 10 having a configuration that enables mutual communication with each of the AIs via a communication network NW, and an interactive system 20 connected to the front system 10 in a manner that enables mutual communication. The interactive system 20 comprises the generating AIs 311, 312, ..., 31 that constitute the generating AI group 2. K Each of them is connected in a way that allows for mutual communication.

[0017] A communication network (NW) can be any network that includes one or more of the following: a local area network (LAN), a wide area network (WAN) such as the internet, a mobile communication network, and a fixed communication network (such as a fiber optic network or a CATV network).

[0018] In this specification, generative AI is an interactive system based on a language model (LM) consisting of an artificial neural network. (Generative AI 311, 312, ..., 31) K These are, respectively, Large-Scale Language Models (LLMs) 321, 322, ..., 32 K It is constructed based on the following, and generates AI311,312,...,31 KEach of them is configured to operate in cooperation with the interactive system 20. Each generative AI preferably has a large language model with more than tens of billions of model parameters as a basis, but is not limited thereto. Instead of the LLM, a small language model (SLM) with hundreds of millions to tens of billions of model parameters may be used.

[0019] In this embodiment, the number of generative AIs 311, 312, …, 31 K is an integer of 3 or more, but is not limited thereto. The number of generative AIs may be 1 or 2. Also, although the generative AI group 2 is provided separately from the automatic response system 1, it is not limited thereto. The configuration of the automatic response system 1 may be changed to include at least one generative AI.

[0020] User terminals T1, …, T N each have a configuration that can be operated by users U1, …, U N respectively. Each of the user terminals T1, …, T N is, for example, a PC (Personal Computer) having a communication function, or a mobile communication device such as a mobile phone terminal or a smartphone. Also, each of the user terminals T1, …, T N is equipped with a computer having one or more processors capable of executing a computer program including a series of instructions. Each of the user terminals T1, …, T N is installed with software (a set of programs constituting an application) for using the interactive automatic response service provided by the automatic response system 1. The number of user terminals T1 to T in this embodiment N is an integer of 2 or more, but is not limited thereto. The number of user terminals may be 1.

[0021] Referring to FIG. 1, the front system 10 includes a response control unit 11, a voice recognition unit 12, a synthesized voice generation unit 13, and an avatar image generation unit 14.

[0022] The response control unit 11 controls the user terminal T n When user input information is received from the user input information, voice data is extracted from the user input information and provided to the voice recognition unit 12 (n is an arbitrary integer in the range of 1 to N). The voice data is received from the user terminal T n This is digital data representing the acoustic signal received by the microphone. The response control unit 11 can also extract text data other than the spoken text data from the user input information. This text data is stored at the user terminal T. n In User U n It is digital data that is input by operating an input device (for example, a keyboard input device).

[0023] Voice data extracted from user input information is used by user U n If speech data is present, the speech recognition unit 12 can generate speech text data by performing speech recognition processing on the speech data using an acoustic model and a language model. Here, neural network models trained using deep learning can be used as the acoustic model and language model, respectively. In other words, the speech recognition unit 12 performs STT (Speech-To-Text) processing to convert the speech data into speech text data. The response control unit 11 can generate query information Q0 containing the generated speech text data and send it to the interactive system 20.

[0024] On the other hand, if the user input information includes text data other than spoken text data, the response control unit 11 can also generate query information Q0 containing such text data and send it to the interactive system 20. As will be described later, the interactive system 20 sends response information RS to the front system 10 for the query information Q0. The response information RS contains text data for the response.

[0025] When the response control unit 11 receives response information RS from the interactive system 20, it provides the text data contained in the response information RS to the synthesized speech generation unit 13. The synthesized speech generation unit 13 performs text-to-speech synthesis (TTS) on the text data to generate speech waveform data. For example, the synthesized speech generation unit 13 can analyze the text data to estimate speech features and generate speech waveform data from these speech features using a neural network model.

[0026] The avatar image generation unit 14 has the function of generating an image of a digital character, i.e., an avatar image, that visually represents the dialogue agent (chatbot). The avatar image generation unit 14 can generate an avatar image (moving image) based on the text data included in the response information RS and the speech feature data such as prosodic information generated by the synthesized speech generation unit 13.

[0027] The response control unit 11 sends the response information, including the generated audio waveform and avatar image data, to the user terminal T. n Send to user terminal T. n This allows the avatar image to be displayed on the screen and the audio waveform to be output from the speaker. Figure 2 shows user terminal T. n This figure shows an example of an avatar image Avt displayed on the display DW. In the example in Figure 2, the user terminal T n User images (Usr) captured by the built-in digital camera are also displayed. n It is possible to interact with the avatar while viewing the avatar image Avt and the audio output from the speaker. In this form, the interactive system 20 sends response information RS to the user terminal T via the front system 10. n It can be sent to

[0028] Referring to Figure 1, the interactive system 20 is configured as a non-generative AI interactive system, that is, an interactive system of a type other than generative AI. The interactive system 20 in this embodiment is a rule-based interactive system. The interactive system 20 comprises a dialogue control unit 21, a query analysis unit 24, a search unit 25, and an FAQ database 26. The dialogue control unit 21 includes a prompt control unit 22 and an AI response selection unit 23.

[0029] When the dialogue control unit 21 receives query information Q0 from the front system 10, it provides the query information Q0 to the query analysis unit 24. The query analysis unit 24 can analyze the text data (including spoken text data) contained in the query information Q0 and extract question information from the query information Q0.

[0030] The FAQ database 26 stores multiple predefined rule patterns, each of which consists of a dataset of pairs of hypothetical questions and corresponding candidate answers. For example, each rule pattern can consist of a hypothetical question, "Can I get a replacement if I lose my insurance policy?" and a corresponding candidate answer, "You can get a replacement insurance policy. You can request a replacement insurance policy through the customer-only website xxxx." Such datasets contain highly accurate answers whose accuracy is guaranteed by prior human verification. The search unit 25 attempts to find hypothetical questions that are highly semantically related to the question information by performing a search of the FAQ database 26. If a hypothetical question highly related to the question information is found as a result of the search, the search unit 25 can retrieve the candidate answer corresponding to that hypothetical question (i.e., the candidate answer that pairs with the hypothetical question) from the FAQ database 26 as an appropriate candidate answer for the question information. For example, if the user's question is "I've lost my insurance policy. What should I do?", the search unit 25 will search the rule patterns in the FAQ database 26 for a hypothetical question that is semantically and conceptually relevant to the question ("Can I get a replacement if I lose my insurance policy?"), and obtain a suitable answer candidate that pairs with the hypothetical question ("You can get a replacement insurance policy. You can apply for a replacement insurance policy through the customer-only website xxxx."). The search unit 25 provides this answer candidate to the dialogue control unit 21. The dialogue control unit 21 sends response information RS, which includes this answer candidate as an answer to the user, to the user terminal T via the front system 10. n It can be sent to

[0031] More specifically, the FAQ database 26 stores M rule patterns (where M is an integer greater than or equal to 2), and each rule pattern contains one or more assumed questions q(m,n m It can be constructed to include a pair of ) and a corresponding answer candidate a(m). Here, m is an integer in the range of 1 to M that indicates the rule pattern number, and n mis the number of the assumed question belonging to the m-th rule pattern. N is the number of assumed questions included in the m-th rule pattern. m If we were to express it as follows, the expected question q(m,n m If there is only one ), then n m =N m =1, but the expected question q(m,n m If there are multiple instances of ), then n m is 1 to N m It takes an integer value within the range. In this example, the total number of anticipated questions in the entire FAQ database 26 is N. m The sum of m (=N1+N2+…+N) M ) and the total number of answer candidates is M. If the m-th rule pattern contains multiple hypothetical questions q(m,1), q(m,2), ... these hypothetical questions q(m,1), q(m,2), ... can be set to sentences or phrases that are semantically similar to each other. For example, hypothetical question q(m,1) can be set to the sentence "Can I get a replacement if I lose my insurance policy?", hypothetical question q(m,2) to the sentence "Can the policy be replaced?", hypothetical question q(m,3) to the phrase "Lost policy", and hypothetical question q(m,4) to the sentence "I lost my policy", and answer candidate a(m) corresponding to these hypothetical questions q(m,1) to q(m,4) can be set to the sentence "You can get a replacement insurance policy. You can apply for a replacement insurance policy through the customer-only website xxxx."

[0032] If user question information is represented by the symbol q, the search unit 25 can compare the question information q with individual hypothetical questions in M ​​rule patterns in the FAQ database 26 and calculate the similarity between these hypothetical questions and the question information q. Based on the calculated similarity, the search unit 25 can then generate hypothetical questions q(m,n m ) and the appropriate answer candidate can be obtained from the corresponding answer candidates a(m).

[0033] Any known method can be used to calculate the similarity, and the method is not particularly limited. For example, the search unit 25 uses a known TF-IDF (Term Frequency and Inverse Document Frequency) method or a trained neural network model to calculate numerical vectors that characterize the question information q in linguistic units such as words, phrases, and sentences, and then compares these calculated numerical vectors with multiple hypothetical questions q(m,n) in the FAQ database 26. m The cosine similarity between the numerical vectors characterizing each of the elements can be calculated. The numerical vectors of distributed representations calculated using a trained neural network model are called embedding vectors.

[0034] When a hypothetical question is found whose similarity to the question information q meets a predetermined criterion, the search unit 25 can acquire the answer candidates that form a pair with the hypothetical question as appropriate answer candidates. For example, if there are similarities within a predetermined threshold range (for example, similarities exceeding a predetermined threshold) among the calculated similarities, the search unit 25 can acquire the answer candidates that form a pair with the hypothetical question used to calculate the highest similarity among those similarities as appropriate answer candidates.

[0035] On the other hand, if the search results do not find any hypothetical questions highly relevant to the question information q, the search unit 25 notifies the dialogue control unit 21 of the search results indicating that it was unable to obtain appropriate answer candidates for the question information q. At this time, the prompt control unit 22 in the dialogue control unit 21 generates an input prompt requesting the generation of a counter-question for the question information q, and generates query information Q1 including this input prompt, as shown in Figure 1, AI311,312,...,31 K Provided in parallel, generate AI311,...,31 K Therefore, the following are candidate counter-questions for the relevant question information q: FQ1, FQ2, ..., FQ K Output the following.

[0036] As schematically shown in Figure 3, the input prompt 50 is a combined prompt consisting of a system prompt 51 that specifies the range of the counter-questions to be generated and requests the generation of counter-questions for the question information q, and a user prompt 52 that includes the question information q (question text). Figure 4 is an example of the system prompt 51A.

[0037] The AI ​​response selection unit 23 in the dialogue control unit 21 generates AI 311, ..., 31 based on a predetermined algorithm. K The counter-question candidates outputted were FQ1, FQ2, ..., FQ K It has a function to select a counter-question from among the counter-question candidates FQ1, FQ2, ..., FQ K From among them, the candidate for the counter-question that was output with the shortest response time (a candidate for the counter-question with high real-time performance) may be selected, or the counter-question with the highest evaluation may be selected from among the candidates for the counter-question that were output within a predetermined response time. The dialogue control unit 21 generates AI 31 k The system generates response information RS, which includes the inverse question FQ1 obtained from (k is an integer in the range of 1 to K), and sends this response information RS to the user terminal T via the front system 10. n It can be sent to

[0038] The automated response system 1 described above is comprised of one computer or multiple computers interconnected via a communication path. Each computer in the automated response system 1 is equipped with one or more processors capable of executing a computer program containing a series of instructions. The processor in each computer includes one or more arithmetic units that execute processing based on a series of instructions from application software (a collection of computer programs for a specific purpose) read from non-volatile memory (a computer-readable recording medium). For example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Network Processing Unit) can be used as the arithmetic unit. All or part of the functions of the automated response system 1 may be implemented in an on-premise configuration where the provider of the interactive automated response service manages the hardware equipment itself, or in a cloud configuration where the provider does not need to manage the hardware equipment itself.

[0039] Figure 5 shows a schematic configuration of an information processing device 60, which is an example of the basic hardware configuration of a computer in the automated response system 1. The information processing device 60 consists of a processor 61, random access memory (RAM) 62, non-volatile memory 63, large-capacity storage 64 such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), input / output interface circuit 65, communication circuit 66, and signal path 67. The signal path 67 is a bus for interconnecting the processor 61, RAM 62, non-volatile memory 63, large-capacity storage 64, input / output interface circuit 65, and communication circuit 66. RAM 62 is a data storage area used when the processor 61 performs digital signal processing. The processor 61 includes an arithmetic unit such as a CPU, GPU, or NPU, or a combination thereof. The non-volatile memory 63 has a data storage area for storing data of application software executed by the processor 61. The communication interface circuit 66 has a configuration that allows communication with external communication devices via a communication network NW.

[0040] Next, we will explain the processing procedure by the automated response system 1. Figure 6 is a flowchart that schematically shows an example of the automated response processing procedure by the automated response system 1.

[0041] Referring to Figure 6, the front system 10 of the automated response system 1 is connected to the user terminal T nThe system waits until it receives user input information (NO in step S11). When the front system 10 receives user input information (YES in step S11), it processes the user input information and generates query information Q0 (step S12). That is, as described above, the response control unit 11 extracts voice data from the user input information and provides the voice data to the voice recognition unit 12. The voice recognition unit 12 can perform STT processing to convert the utterance data contained in the voice data into utterance text data. The response control unit 11 can generate query information Q0 that includes the utterance text data. On the other hand, if the user input information includes text data other than utterance text data, the response control unit 11 can also generate query information Q0 that includes that text data. The query information Q0 is transmitted to the interactive system 20.

[0042] Subsequently, the query analysis unit 24 in the interactive system 20 analyzes the query information Q0 and extracts question information q from the query information Q0 (step S13). Next, the search unit 25 searches the question information q for the assumed questions q(m,n) in the M rule patterns in the FAQ database 26. m ) Compare with the expected question q(m,n m The similarity between the set of assumed questions {q(m,n)} and the question information q is calculated (step S14). Here, as described above, m is an integer from 1 to M. m If a hypothetical question is found among them whose similarity meets a predetermined standard (YES in step S15), the dialogue control unit 21 acquires a candidate answer corresponding to the hypothetical question (a candidate answer paired with the hypothetical question) as an appropriate candidate answer to the question information q (step S21), and generates response information RS that includes the candidate answer as the answer (step S22).

[0043] The dialogue control unit 21 then sends the response information RS to the user terminal T via the front system 10. n It is sent to (step S23). Then the front system 10 sends to the user terminal T n The system waits until it receives the next user input information (step S11).

[0044] On the other hand, in step S15, the set of anticipated questions {q(m,n m If no hypothetical question is found among the )} that satisfies the predetermined similarity criteria (NO in step S15), the prompt control unit 22, in response to notification from the search unit 25, generates an input prompt requesting the generation of a counter-question for the question information q (step S31), and generates query information Q1 including the input prompt AI311, ..., 31 K Give to (step S32).

[0045] Subsequently, the dialogue control unit 21 of the interactive system 20 generates AI311, ..., 31 K From the list of potential counter-questions FQ1, ..., FQ K Waiting until a signal is received (NO in step S33). The dialogue control unit 21 generates AI311, ..., 31 K From the list of potential counter-questions FQ1, ..., FQ K Upon receiving (YES in step S33), the AI ​​response selection unit 23 selects the counter-question candidates FQ1, FQ2, ..., FQ based on a predetermined algorithm. K A counter-question is selected from the options (step S34). Then, the dialogue control unit 21 generates response information RS including the counter-question (step S35), and this response information RS is sent to the user terminal T via the front system 10. n It is sent to (step S23). Then the front system 10 sends to the user terminal T n The system waits until it receives the next user input information (step S11).

[0046] As described above, when the non-generating AI conversational system 20 obtains an appropriate answer candidate for a question extracted from user input information (YES in step S15 and step S21), it sends response information RS containing the appropriate answer candidate as an answer to the user terminal T via the front system 10. nThe system sends the information to the AI, and if no suitable answer candidate is obtained for the question information (NO in step S15), it provides the generating AI with an input prompt requesting the generation of a counter-question for the question information, causing the generating AI to output the counter-question (step S32). The generating AI may experience a phenomenon called hallucination, in which it generates plausible but false information as an answer, but the generating AIs 311~31 of the automatic response system 1 of this embodiment K This is used to generate counter-questions to the question information, and not to directly generate answers to the question information. A non-generating AI conversational system 20 is used to generate highly accurate answers to the question information, so that answers that contradict the answer candidates recorded in the FAQ database 26 are not used by user U. n This can be avoided. For example, businesses in regulated industries such as insurance and healthcare are required to strictly comply with laws and regulations, so having a generating AI generate answers carries risks. However, by adopting the automated response system 1 of this embodiment, such risks can be significantly reduced.

[0047] Also, User U n Even if the user input information is ambiguous, the automated response system 1 will generate AI311~31 K The counter-question generated by user U n Since you can reply to user U n This allows the user U to continue interacting with the automated response system 1. Therefore, the user U can access the content of the highly accurate answer candidates recorded in the FAQ database 26. n The probability of being able to give a result improves. Figure 7 is a table showing examples of ambiguous questions asked by users and the counter-questions actually generated using the generation AI in response to them.

[0048] Although various embodiments and modifications thereof relating to this disclosure have been described above, these embodiments and modifications thereof are merely illustrative and do not limit the scope of the present invention. It should be understood that the above embodiments can be modified, added to, and improved as appropriate without departing from the spirit and scope of the present invention. The scope of the present invention should be interpreted based on the claims and further understood to include their equivalents. [Explanation of Symbols]

[0049] NW: Communication Network, T1~T N :User terminal, 1:Automatic response system, 2:Generating AI group, 10:Front system, 11:Response control unit, 12:Speech recognition unit, 13:Synthesized speech generation unit, 14:Avatar image generation unit, 20:Non-generating AI conversational system, 21:Dialogue control unit, 22:Prompt control unit, 23:AI response selection unit, 24:Query analysis unit, 25:Search unit, 26:FAQ database, 311~31 K :Generation AI, 321~32 K :Large-scale language model (LLM), 60: Information processing device, 61: Processor, 62: Random access memory (RAM), 63: Non-volatile memory, 64: Mass storage, 65: Input / output interface circuit, 66: Communication interface circuit, 67: Communication channel 67.

Claims

1. An automated response method performed in an automated response system configured to communicate with a user terminal via a communication network, The steps include receiving user input information from the user terminal via the communication network, The steps include extracting question information from the user input information, When an appropriate candidate answer to the question information is obtained by a non-generating AI conversational system, the system sends response information containing the appropriate candidate answer as the answer to the user terminal. If the non-generating AI conversational system does not obtain an appropriate candidate answer to the question information, the system provides an input prompt to at least one generating AI requesting the generation of a counter-question to the question information, causing the at least one generating AI to output the counter-question. The steps include sending response information, including the aforementioned counter-question, to the user terminal. An automated response method.

2. An automated response method according to claim 1, wherein the at least one generating AI is an interactive system based on a language model comprising an artificial neural network.

3. An automated response method according to claim 1, The aforementioned interactive system includes a database in which multiple rule patterns, each containing a set of anticipated questions and candidate answers, are pre-stored. The automated response method further comprises the steps of comparing the question information with a plurality of hypothetical questions in the plurality of rule patterns, and obtaining the appropriate answer candidate from a plurality of answer candidates in the plurality of rule patterns based on the similarity between the plurality of hypothetical questions and the question information.

4. An automated response method according to claim 3, The steps include: calculating a numerical vector that characterizes the aforementioned question information; The steps include calculating the similarity between the calculated numerical vector and the numerical vectors characterizing the plurality of hypothetical questions, and An automated response method that further enhances this feature.

5. An automated response method according to claim 1, An automated response method further comprising the step of generating a combined prompt as the input prompt, which consists of a user prompt containing the question information and a system prompt that specifies the range of counter-questions to be generated and requests the generation of counter-questions for the question information, when the interactive system does not obtain a suitable candidate answer to the question information.

6. An automated response method according to claim 1, The aforementioned at least one generative AI is a plurality of generative AIs, The step of causing the aforementioned counter-question to be output by the at least one generating AI is: The steps include providing the input prompt to the multiple generating AIs in parallel, causing each of the multiple generating AIs to output multiple candidate counter-questions for the question information, The steps include selecting the counter-question from among the multiple candidate counter-questions based on a predetermined algorithm, and An automated response method, including

7. An automated response system configured to perform the automated response method described in any one of claims 1 to 6.

8. A computer program comprising a series of instructions, wherein, when the series of instructions is executed by one or more processors, the one or more processors cause the one or more processors to execute the automatic response method described in any one of claims 1 to 6.

Images