Automatic response device and automatic response method

Abstract

This automatic response device comprises: a first acquisition unit that acquires, from a large language model, a first answer to a user's question by inputting, to the large language model, a first instruction for instructing the same to generate the first answer in response to the question on the basis of the question and first data indicating a search result of a knowledge database relating to the user's question; and a second acquisition unit that acquires a first counter question sentence for clarifying the intention of the question from the large language model by inputting, to the large language model, a second instruction for instructing the same to generate the first counter question sentence on the basis of the first data and the question. The first acquisition unit inputs the first instruction to the large language model before the second acquisition unit acquires the first counter question sentence.

Description

Automatic Response Device and Automatic Response Method

[0001] The present invention relates to an automatic response device and an automatic response method.

[0002] In a system that presents information in response to a user's inquiry, a RAG (Retrieval-Augmented Generation) system that provides more reliable information by collaborating with an external knowledge database and a large language model is used. For example, Patent Document 1 discloses a browsing support system that outputs creation information for creating an electronic medical record to a RAG system, receives an answer from the RAG system, and supports the creation of an electronic medical record.

[0003] Japanese Patent No. 7601361

[0004] However, in the browsing support system of Patent Document 1, since the generation of an answer by a large language model is performed based on the search results of the knowledge database, the search process of the knowledge database and the generation process of the answer by the large language model are executed sequentially. Since both the search of the knowledge database and the generation of the answer by the large language model require a certain amount of time, there is a problem that it takes time from when the user asks a question until an answer is obtained.

[0005] The present invention has been made to solve the above problems, and an object thereof is to shorten the waiting time of the user.

[0006] An automated response device according to a preferred embodiment of the present invention includes: a first acquisition unit that acquires a first response from a large-scale language model by inputting a first instruction to the large-scale language model instructing it to generate a first response to a question based on first data indicating search results from a knowledge database relating to a question made by a user and the question; a second acquisition unit that acquires a first follow-up question from the large-scale language model by inputting a second instruction to the large-scale language model instructing it to generate a first follow-up question to clarify the intent of the question based on the first data and the question; and a display control unit that displays a response based on the first response on the user's display device when the user's second response to the first follow-up question indicates that the intent of the question corresponds to the first response, wherein the first acquisition unit inputs the first instruction to the large-scale language model before the second acquisition unit acquires the first follow-up question.

[0007] A preferred embodiment of the present invention provides an automated response method comprising: a first acquisition process for acquiring a first answer from a large-scale language model by inputting a first instruction to the large-scale language model instructing it to generate a first answer to a question based on first data indicating search results from a knowledge database relating to a question made by a user and the question; a second acquisition process for acquiring a first clarification question from the large-scale language model by inputting a second instruction to the large-scale language model instructing it to generate a first clarification question to clarify the intent of the question based on the first data and the question; and a display control process for displaying an answer based on the first answer on the user's display device if the user's second answer to the first clarification question indicates that the intent of the question corresponds to the first answer, wherein the first acquisition process inputs the first instruction to the large-scale language model before the first clarification question is acquired by the second acquisition process.

[0008] According to the automatic response device and automatic response method of the present invention, the user's waiting time can be reduced.

[0009] This figure shows the overall configuration of the automated response system according to the first embodiment. This is a block diagram showing an overview of the RAG system according to the first embodiment. This is a block diagram showing an example configuration of the user terminal device in Figure 1. This is a block diagram showing an example configuration of the database server in Figure 1. This is a block diagram showing an example configuration of the generation AI server in Figure 1. Figure 6 is a block diagram showing an example configuration of the automated response server in Figure 1. This is a flowchart showing an example of the overall operation of the processing unit in Figure 6. This is a flowchart showing an example of the overall operation of the processing unit in Figure 6. This is a block diagram showing an example of the overall operation of the processing unit in Figure 6. This is a block diagram showing an overview of the RAG system according to Modification 1.

[0010] 1. The configuration of the automatic response device according to the first embodiment of the present invention will be described below with reference to Figures 1 to 9.

[0011] 1.1. Configuration of the First Embodiment 1.1.1. Configuration of the Automatic Response System Figure 1 shows the overall configuration of the automatic response system 1 according to the first embodiment. The automatic response system 1 includes a user terminal device 10, an automatic response server 20, a database server 30, and a generation AI (Artificial Intelligence) server 40.

[0012] The automated response system 1 is a system that interacts with user U and presents answers to user U's inquiries, questions, etc.

[0013] A communication network (NET) is a telecommunications line, such as a mobile communication network, managed by a telecommunications carrier providing communication services. A communication network (NET) includes either or both wired and wireless communication networks. For example, a communication network (NET) may be connected via the Internet to other networks (not shown) managed by other telecommunications carriers.

[0014] In the automated response system 1, the user terminal device 10, the automated response server 20, the database server 30, and the generation AI server 40 are connected to each other via a communication network NET so that they can communicate with one another. However, the automated response server 20, the database server 30, and the generation AI server 40 may be directly connected to each other without using the communication network NET.

[0015] The user terminal device 10 is a device used by user U. The user terminal device 10 includes personal computers, tablet terminals, smartphones, smartwatches, etc. In this embodiment, only one user terminal device 10 is shown, but the automated response system 1 may be connected to multiple user terminal devices, each corresponding to a different user.

[0016] The automated response server 20 is a device that inputs data related to questions sent from the user terminal device 10 based on operations by the user U, and outputs the response result to the user terminal device 10 based on the input data. In this embodiment, the automated response server 20 is assumed to be a chatbot that serves as a customer support window for a mobile phone service.

[0017] The database server 30 is a device that manages a knowledge database for which the automated response server 20 searches for information necessary to answer user U's questions. In this embodiment, it is assumed that response manuals for customer support of mobile phone services are managed. In this embodiment, customer support for mobile phone services is used as an example, but the use of the automated response system 1 is not limited to customer support for mobile phone services. The automated response system 1 can be used for customer support of various companies and for response services for citizens by government agencies.

[0018] The generation AI server 40 is a device that has a large-scale language model. In this embodiment, it is assumed that it generates various question sentences and answer sentences based on instructions from the automatic response server 20.

[0019] The automated response server 20, the database server 30, and the generation AI server 40 constitute the RAG system.

[0020] 1.1.2. RAG System Overview Diagram 2 is a block diagram illustrating the overview of the RAG system 100 according to the first embodiment. The RAG system 100 is a system in which two processes, "Retrieval" and "Generation," are executed in cooperation with each other.

[0021] The knowledge database DB1 of the database server 30 includes multiple classification databases DB11 to DB1n, each corresponding to a single category. If the knowledge database DB1 includes customer service manuals for mobile phone services, the categories correspond, for example, to multiple mobile phone service products in a one-to-one manner. In order to retrieve the necessary information about a user U's question, the database server 30 performs a classification process that determines an evaluation value that serves as an indicator of which of the multiple classification databases DB11 to DB1n to search, in accordance with the purpose or intent of the user U's question.

[0022] The larger the amount of information in a database, the lower the accuracy and speed of the search process tend to be. However, as shown in Figure 2, by dividing the database by category, the amount of information in each database can be reduced, thereby suppressing the decrease in search accuracy and speed.

[0023] In the RAG system 100, the basic flow of the process by which the automatic response server 20 answers a question from user U is as follows:

[0024] First, in step S11, the automated response server 20 receives the question entered by user U from the user terminal device 10. Next, in step S12, the automated response server 20 vectorizes user U's question and sends it to the database server 30.

[0025] In step S13, the database server 30 determines an evaluation value for each of the classification databases DB11 to DB1n in the classification processing unit 312 and sends it to the automatic response server 20. The evaluation value is, for example, a value between 0 and 1. Each of the classification databases DB11 to DB1n is tagged with one or more keywords related to the contents of the classification database, and the evaluation value is determined based on the similarity between the question from user U and one or more keywords. For example, the higher the similarity, the higher the evaluation value is set.

[0026] In step S14, the automated response server 20 determines the classification result. The classification result indicates whether the evaluation value for a certain category is significantly higher than the evaluation values ​​for other categories. Whether the evaluation value for a certain category is significantly higher than the evaluation values ​​for other categories can be determined, for example, by whether the highest evaluation value among multiple evaluation values ​​is greater than or equal to the threshold TH.

[0027] If the highest evaluation value is greater than or equal to the threshold TH, and the category corresponding to the highest evaluation value is the first category, in step S15, the automated response server 20 searches the first classification database DB 11 corresponding to the first category for documents that have a high degree of similarity to the user U's question.

[0028] In step S16, the automated response server 20 obtains search result data for documents with high similarity. In step S17, the automated response server 20 combines the retrieved documents and the user U's question to generate an instruction statement for the large-scale language model LLM in the generating AI server 40, and sends the question, search result data, and instruction statement to the generating AI server 40.

[0029] In step S18, the automated response server 20 retrieves the response generated by the large-scale language model LLM from the large-scale language model LLM while referring to the retrieved document. In step S19, the automated response server 20 presents the response retrieved from the generation AI server 40 to user U.

[0030] With only the Large-Scale Language Model (LLM), responses are generated based only on information that the LLM has already learned, making it difficult to handle the latest and most specialized information. The RAG system 100 generates responses while referencing information from the specialized knowledge database DB1 in real time, thus enabling the generation of more reliable responses.

[0031] Although not shown in Figure 2, if the intent of user U's question is unclear, the automated response server 20 will present user U with a follow-up question. The first follow-up question is a question intended to clarify the intent of user U's question. If the intent of user U's question is unclear, the automated response server 20 inputs an instruction to the large-scale language model LLM to generate the first follow-up question, and thereby obtains the first follow-up question from the large-scale language model LLM.

[0032] Furthermore, if the category to which the user U's question belongs is unclear, the automated response server 20 presents the user U with a follow-up question. The second follow-up question is a question intended to clarify the category to which the user U's question belongs. If the category to which the user U's question belongs is unclear, the automated response server 20 inputs a third instruction into the large-scale language model LLM, instructing it to generate the second follow-up question, and thereby obtains the second follow-up question from the large-scale language model LLM. The third instruction is an example of a third instruction.

[0033] 1.1.3. User Terminal Device Configuration Diagram 3 is a block diagram showing an example configuration of the user terminal device 10 shown in Figure 1. As shown in Figure 3, the user terminal device 10 comprises a processing unit 11, a storage device 12, a communication device 13, a display device 14, and an input device 15. Each element of the user terminal device 10 is interconnected by one or more buses for communicating information.

[0034] The processing unit 11 is a processor that controls the entire user terminal device 10, and is configured, for example, using one or more chips. The processing unit 11 is configured using a central processing unit (CPU) that includes, for example, interfaces with peripheral devices, arithmetic units, registers, etc. Some or all of the functions of the processing unit 11 may be implemented by hardware such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array). The processing unit 11 executes various processes in parallel or sequentially.

[0035] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 includes, for example, non-volatile memory and volatile memory. Non-volatile memory includes, for example, ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Volatile memory includes, for example, RAM (Random Access Memory).

[0036] The storage device 12 stores multiple programs, including the control program PR1, which is executed by the processing device 11. The storage device 12 also functions as a work area for the processing device 11. The control program PR1 is a program that controls the entire processing device 11.

[0037] The communication device 13 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 13 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 13 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with Wi-Fi® and Bluetooth®.

[0038] The display device 14 is a device that displays images and text information. The display device 14 displays various images based on control by the processing device 11. For example, various display panels such as liquid crystal panels and organic EL (Electro-Luminescence) panels are preferably used as the display device 14.

[0039] The input device 15 accepts operations from the user U. For example, the input device 15 is configured to include a pointing device such as a keyboard, touchpad, touch panel, or mouse. If the input device 15 is configured to include a touch panel, it may also function as the display device 14.

[0040] The processing unit 11 functions as a receiving unit 111, a transmitting unit 112, an acquisition unit 113, and a display control unit 114, for example, by reading and executing the control program PR1 from the storage device 12.

[0041] The reception unit 111 receives user U's operations on the input device 15. User U's operations include, for example, tapping on the touch panel which is the input device 15.

[0042] The transmission unit 112 transmits the text input using the input device 15 to the automatic response server 20 via the communication device 13.

[0043] The acquisition unit 113 acquires text information transmitted from the automatic response server 20 via the communication device 13. The text information includes a question sentence and an answer sentence by the automatic response server 20.

[0044] The display control unit 114 causes the display device 14 to display various information.

[0045] 1.1.4. Configuration of Database Server Fig. 4 is a block diagram showing a configuration example of the database server 30 in Fig. 1. As shown in Fig. 4, the database server 30 includes a processing device 31, a storage device 32, and a communication device 33. Each element included in the database server 30 is interconnected by one or more buses for communicating information.

[0046] The processing device 31 is a processor that controls the entire database server 30, and is configured using, for example, one or more chips. The processing device 31 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that some or all of the functions included in the processing device 31 may be realized by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0047] The storage device 32 is a recording medium that can be read and written by the processing device 31. The storage device 32 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM, an EPROM, or an EEPROM. The volatile memory is, for example, a RAM.

[0048] The storage device 32 stores a plurality of programs including a control program PR3 for the processing device 31 to execute and a knowledge database DB1. Further, the storage device 32 functions as a work area for the processing device 31. The control program PR3 is a program that controls the entire processing device 31.

[0049] Knowledge database DB1 contains information such as customer service manuals, product instruction manuals, and contracts for various services. The information in knowledge database DB1 is pre-vectorized using an embedding model. An embedding model is a learning model that performs the process of distributing unstructured data such as text, images, and audio, that is, the process of converting unstructured data into vectors. Examples of embedding models include Word2Vec, GloVe, FastText, and BERT (Bidirectional Encoder Representations from Transformers).

[0050] Knowledge database DB1 includes multiple classification databases DB11 to DB1n, each corresponding to a single category. The first classification database DB11 corresponds to category #1. The second classification database DB12 corresponds to category #2. The nth classification database DB1n corresponds to category #n.

[0051] The communication device 33 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 33 is also called, for example, a network device, network controller, network card, communication module, etc. The communication device 33 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 33 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with Wi-Fi® and Bluetooth®.

[0052] The processing unit 31 functions as an interface unit 311, a classification processing unit 312, and a search processing unit 313, for example, by reading and executing the control program PR3 from the storage device 32.

[0053] The interface unit 311 receives a search instruction from the automatic response server 20. The interface unit 311 transmits the evaluation value determined by the classification processing unit 312 to the automatic response server 20. The interface unit 311 transmits the search results retrieved using the knowledge database DB1 to the automatic response server 20.

[0054] The classification processing unit 312 determines an evaluation value to determine whether the question posed by user U belongs to category #1, category #2, ..., or category #n. As described above, the evaluation value is, for example, a value between 0 and 1. The evaluation value is higher the greater the similarity between the question posed by user U and one or more keywords tagged in the classification databases DB11 to DB1n, respectively.

[0055] The search processing unit 313 processes the user U's query and retrieves the necessary information from the knowledge database DB1. The search processing unit 313 executes the search using a structured query language such as SQL (Structured Query Language).

[0056] 1.1.5. Configuration Diagram 5 of the Generating AI Server is a block diagram showing an example configuration of the Generating AI Server 40 shown in Figure 1. As shown in Figure 5, the Generating AI Server 40 comprises a processing unit 41, a storage device 42, and a communication device 43. Each element of the Generating AI Server 40 is interconnected by one or more buses for communicating information.

[0057] The processing unit 41 is a processor that controls the entire generation AI server 40, and is configured, for example, using one or more chips. The processing unit 41 is configured, for example, using a central processing unit (CPU) that includes interfaces with peripheral devices, an arithmetic unit, and registers. Some or all of the functions of the processing unit 41 may be implemented by hardware such as a DSP, ASIC, PLD, FPGA, etc. The processing unit 41 executes various processes in parallel or sequentially.

[0058] The storage device 42 is a recording medium that can be read from and written to by the processing device 41. The storage device 42 includes, for example, non-volatile memory and volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM.

[0059] The storage device 42 stores multiple programs, including the control program PR4 for execution by the processing device 41, and large-scale language models LLM. The storage device 42 also functions as a work area for the processing device 41. Examples of large-scale language models LLM include BERT (Bidirectional Encoder Representations from Transformers), GPT-4o, Gemini, and Llama2.

[0060] The communication device 43 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 43 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 43 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 43 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with Wi-Fi® and Bluetooth®.

[0061] The processing unit 41 functions as a first interface unit 411 and a second interface unit 412, for example, by reading and executing the control program PR4 from the storage device 42.

[0062] The first interface unit 411 receives instructions from the automatic response server 20 via the communication device 43 to generate a response sentence or a follow-up question sentence for user U. The first interface unit 411 then transmits the generated response sentence or follow-up question sentence to the automatic response server 20 via the communication device 43.

[0063] The second interface unit 412 inputs a command to generate a response sentence or a follow-up question sentence to the large-scale language model LLM. The second interface unit 412 retrieves the generated response sentence or follow-up question sentence from the large-scale language model LLM.

[0064] 1.1.6. Configuration of the Automated Response Server Figure 6 is a block diagram showing an example configuration of the automated response server 20 shown in Figure 1. As shown in Figure 6, the automated response server 20 comprises a processing unit 21, a storage device 22, and a communication device 23. Each element of the automated response server 20 is interconnected by one or more buses for communicating information.

[0065] The processing unit 21 is a processor that controls the entire automatic response server 20, and is configured, for example, using one or more chips. The processing unit 21 is configured, for example, using a central processing unit (CPU) that includes interfaces with peripheral devices, an arithmetic unit, and registers. Some or all of the functions of the processing unit 21 may be implemented by hardware such as a DSP, ASIC, PLD, FPGA, etc. The processing unit 21 executes various processes in parallel or sequentially.

[0066] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 includes, for example, non-volatile memory and volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM.

[0067] The storage device 22 stores multiple programs, including the control program PR2 for execution by the processing device 21, and the Embedding model EBM. The storage device 22 also functions as a work area for the processing device 21.

[0068] The communication device 23 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 23 is also called, for example, a network device, network controller, network card, communication module, etc. The communication device 23 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 23 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with Wi-Fi® and Bluetooth®.

[0069] The processing unit 21 functions as a first acquisition unit 211, a second acquisition unit 212, a third acquisition unit 213, a determination unit 216, a search unit 217, and a display control unit 218, for example, by reading and executing the control program PR2 from the storage device 22.

[0070] The first acquisition unit 211 inputs a first instruction sentence to the large-scale language model LLM of the generation AI server 40, thereby acquiring a first answer to a question from user U from the large-scale language model LLM. The first instruction sentence is an instruction to generate a first answer based on the first data and the question from user U. The first instruction sentence is an example of a first instruction. The first data is data showing the search results from the knowledge database DB1 regarding the question from user U.

[0071] Below is an example of the first instruction. Note that the question from user U falls under [Inquiry]. "The following text is a user inquiry submitted to online procedural support. Based on the search results below, please consider and output a response to the inquiry. Search results are reference texts that may provide clues to resolving the inquiry. [Search Results] How do I proceed if I am a new subscriber to docomo Hikari (registered trademark)?... (omitted) [Inquiry] I want to subscribe to the Hikari service."

[0072] The second acquisition unit 212 determines whether the intent of the question asked by user U is clear or unclear, based on the first data and the question asked by user U. More specifically, the second acquisition unit 212 determines whether the intent of the question asked by user U is clear or unclear based on the cosine similarity between the question asked by user U and the search results.

[0073] The second acquisition unit 212, if the intent of the user U's question is unclear, inputs a second instruction sentence into the large-scale language model LLM to acquire the first follow-up question sentence from the large-scale language model LLM. The first follow-up question sentence is a question sentence intended to clarify the intent of the user U's question. The second instruction sentence is an instruction to generate the first follow-up question sentence based on the first data and the question from user U. The second instruction sentence is an example of the second instruction. Note that if the intent of the user U's question is clear, the second acquisition unit 212 does not input the second instruction sentence into the large-scale language model LLM.

[0074] Below is an example of the second instruction: "The following text is a user inquiry submitted to our online procedural support. Please formulate and output a question to confirm whether the purpose of the inquiry matches the situation described in the search results below. Search results are reference texts that may provide clues to resolving the inquiry. [Search Results] How do I proceed if I am a new subscriber to docomo Hikari (registered trademark)?... (abbreviated) [Inquiry] I want to subscribe to the Hikari service."

[0075] The first acquisition unit 211 inputs the first instruction sentence into the large-scale language model LLM before the second acquisition unit 212 acquires the first follow-up question sentence.

[0076] Before the second acquisition unit 212 acquires the first follow-up question, the first follow-up question has not been presented to user U. Therefore, at this point, the second answer, which is the response to user U's first follow-up question, has not been obtained, and the intent of user U's question is not clear. However, after the second answer is obtained, if the display control unit 218 indicates that the second answer corresponds to the intent of user U's question and the first answer, it displays the answer based on the first answer on user U's display device 14.

[0077] Furthermore, if the display control unit 218 does not indicate that the second answer corresponds to the intent of the question posed by user U and the first answer, it will not display the answer based on the first answer on the display device 14. In this case, the first acquisition unit 211 inputs an instruction to the large-scale language model LLM to generate an answer to user U's question based on the search results of the knowledge database DB1 regarding user U's question and the second answer, and user U's question and the second answer, and acquires an answer to that instruction from the large-scale language model LLM.

[0078] Furthermore, a question from user U may have multiple intentions. These multiple intentions include a first intention, and the first answer includes a candidate answer corresponding to the first intention. If a question from user U has multiple intentions, the display control unit 218 displays the candidate answer on user U's display device 14 as an answer based on the first answer, if the second answer indicates the first intention.

[0079] The determination unit 216 determines the classification result based on the questions asked by the user U. The classification result has an evaluation value that indicates the degree of probability of belonging to each of the multiple categories. The evaluation value is, for example, a value in the range of 0 to 1.

[0080] The third acquisition unit 213, when the highest evaluation value among multiple evaluation values ​​corresponding one-to-one to multiple categories is less than the threshold TH, inputs a third instruction to the large-scale language model LLM, thereby acquiring a second follow-up question from the large-scale language model LLM. The third instruction is an instruction that, based on the classification result and the question asked by user U, generates a second follow-up question to clarify the category to which the question asked by user U belongs. The third instruction is an example of a third instruction.

[0081] Below is an example of the third instruction: "The following text is a user inquiry received through online procedural support. Please create and output a question to confirm whether the purpose of the inquiry falls under the following categories. [Category] docomo Hikari (registered trademark) [Inquiry] I want to subscribe to the optical service."

[0082] Furthermore, the first acquisition unit 211 acquires the third response, which is the user U's answer to the second follow-up question. If the highest evaluation value among the multiple evaluation values ​​is less than the threshold TH, the first acquisition unit 211 inputs the fourth instruction sentence to the large-scale language model LLM, thereby acquiring the fourth response from the large-scale language model LLM. That is, the fourth response is a response based on the fourth instruction sentence. The fourth instruction sentence is an instruction sentence that instructs the system to generate the fourth response to the question asked by user U, based on the second data and the question asked by user U. The fourth instruction sentence is an example of the fourth instruction.

[0083] The fourth acquisition unit 214 acquires the question text from user U from the user terminal device 10.

[0084] The distributed representation unit 215 vectorizes the acquired question text using an Embedding Model (EBM). The Embedding Model (EBM) is a learning model that performs the process of distributing unstructured data such as text, images, and audio, that is, the process of converting unstructured data into vectors. Examples of Embedding Models (EBM) include Word2Vec, GloVe, FastText, and BERT (Bidirectional Encoder Representations from Transformers).

[0085] The search unit 217 sends a question from user U to the database server 30 and causes the database server 30 to perform classification processing based on the question from user U. If the category corresponding to the highest evaluation value among multiple evaluation values ​​that correspond one-to-one to multiple categories is the first category, the search unit 217 retrieves the first data by performing a search for the question from user U using the first classification database DB 11, which corresponds to the first category among the multiple classification databases.

[0086] If the search unit 217 confirms, based on the third response, that the category to which the question by user U belongs is the second category, it retrieves the second data by performing a search for the question by user U using the classification database corresponding to the second category.

[0087] The display control unit 218 displays the answer based on the first response on the user U's display device 14 if the highest evaluation value among the multiple evaluation values ​​is equal to or greater than the threshold TH, and displays the answer based on the fourth response on the user U's display device 14 instead of the first response if the highest evaluation value among the multiple evaluation values ​​is less than the threshold TH.

[0088] Furthermore, a question from user U may have multiple intentions, including a first intention. Therefore, the first answer includes answer candidates corresponding to the first intention, answer candidates corresponding to the second intention, and so on. When the second answer, which is user U's response to the first follow-up question, indicates the first intention, the display control unit 218 displays the answer candidates corresponding to the first intention as the answer based on the first answer on user U's display device 14.

[0089] 1.2. Operation of the Automated Response Server According to the First Embodiment 1.2.1. Overall Operation of the Processing Unit 21 Figures 7 and 8 are flowcharts illustrating an example of the overall operation of the processing unit 21 shown in Figure 6. The overall operation of the processing unit 21 will be described below with reference to Figures 7 and 8. The routines in Figures 7 and 8 are started, for example, when the processing unit 21 is powered on, and are executed at regular intervals.

[0090] In step S102, the processing unit 21 functions as a fourth acquisition unit 214 to acquire a question from user U from user terminal device 10.

[0091] In step S104, the processing unit 21 functions as a distributed representation unit 215 to vectorize the question from user U.

[0092] In step S106, the processing unit 21 functions as a search unit 217, causing the database server 30 to perform classification processing. That is, in step S106, the processing unit 21 causes the category to which the question belongs to to determine an evaluation value.

[0093] In step S108, the processing unit 21, by functioning as a search unit 217, determines whether or not it is necessary to ask for clarification for classification processing. In other words, in step S108, the processing unit 21 determines whether or not the highest evaluation value among multiple evaluation values ​​that correspond one-to-one to multiple categories is less than the threshold TH.

[0094] If it is determined that no further questioning is necessary for classification processing, that is, if the determination result in step S108 is negative, the processing unit 21, in step S118, functions as a search unit 217 and obtains first data by searching the classification database corresponding to the first category for the question. Here, the first category is the category corresponding to the highest evaluation value. After that, the routine proceeds to step S120 in Figure 8.

[0095] On the other hand, if it is determined that a follow-up question is necessary for classification processing, that is, if the determination result in step S108 is positive, the processing unit 21, in step S110, functions as a search unit 217 and obtains first data by searching for the question using the classification database corresponding to the first category.

[0096] In step S112, the processing unit 21, by functioning as a third acquisition unit 213, inputs the third instruction sentence to the large-scale language model LLM, thereby acquiring the second follow-up question sentence from the large-scale language model LLM.

[0097] In step S114, the processing device 21 functions as a display control unit 218 and displays the acquired second follow-up question on the display device 14.

[0098] In step S116, the processing unit 21, functioning as a search unit 217, determines from the user U's answer whether or not the category to which the question belongs is the second category. The second category is a different category from the first category.

[0099] If user U's response confirms that the category to which the question belongs is the first category and not the second category, that is, if the determination result in step S116 is negative, this routine proceeds to step S122 in Figure 8.

[0100] On the other hand, if it is confirmed from user U's response that the category to which the question belongs is the second category, that is, if the determination result in step S116 is affirmative, the processing unit 21, in step S120, functions as a search unit 217 and obtains second data by searching for the question using the classification database corresponding to the second category. After that, this routine proceeds to step S122 in Figure 8.

[0101] Next, referring to Figure 8, in step S122, the processing unit 21 functions as a search unit 217 to determine whether or not it is necessary to ask for clarification for the search process. Whether or not it is necessary to ask for clarification for the search process can be determined, for example, by whether or not the similarity between the search result and the user U's question is less than a predetermined value.

[0102] If it is determined that a follow-up question is necessary for the search process, that is, if the determination result in step S122 is affirmative, the processing unit 21, in step S124, functions as a first acquisition unit 211 and inputs the first instruction sentence to the large-scale language model LLM, thereby obtaining the first or fourth response from the large-scale language model LLM. The first response is a response generated based on the first data, and the fourth response is a response generated based on the second data.

[0103] In step S126, the processing unit 21 functions as a second acquisition unit 212 and inputs the second instruction sentence to the large-scale language model LLM, thereby acquiring the second response from the large-scale language model LLM.

[0104] In step S128, the processing unit 21 functions as a display control unit 218 and displays the acquired first follow-up question on the display device 14.

[0105] On the other hand, if it is determined that a follow-up question is not necessary for the search process, that is, if the determination result in step S122 is negative, the processing unit 21, in step S130, functions as the first acquisition unit 211 and inputs the first instruction sentence to the large-scale language model LLM, thereby obtaining the first answer from the large-scale language model LLM.

[0106] In step S132, the processing unit 21, functioning as a display control unit 218, determines whether or not the user U's answer indicates that the intent of the question corresponds to the first answer.

[0107] If user U's response indicates that the intent of the question corresponds to the first response, that is, if the determination result in step S132 is affirmative, the processing unit 21, in step S134, functions as a display control unit 218 to display the first response on the display device 14 and terminates this routine.

[0108] When User U's answer indicates that the intent of the question and the first answer correspond, it means that User U's answer confirms that the first answer is in line with the intent of the question. For example, if the first answer is an explanation of the procedures for joining docomo Hikari, and the first follow-up question is, "Are you asking about the procedures for joining docomo Hikari?", and User U's answer to the first follow-up question is affirmative, then it can be said that the intent of the question and the first answer correspond.

[0109] On the other hand, if user U's answer does not indicate a correspondence between the intent of the question and the first answer, that is, if the determination result in step S132 is negative, the processing unit 21, in step S136, functions as a search unit 217 to obtain search results from the knowledge database DB1 regarding the question made by user U and user U's answer.

[0110] Next, the processing unit 21 executes the processes from step S122 onward, and repeats this routine until it executes the process in step S134.

[0111] 1.2.2. Partial Operation of Processing Unit 21 Figure 9 is a flowchart illustrating an example of the partial operation of the processing unit 21 shown in Figure 6. The partial operation of the processing unit 21 will be explained below with reference to Figure 9. The routine in Figure 9 is started, for example, when the processing unit 21 is powered up, and is executed at regular intervals.

[0112] In step S202, the processing unit 21 functions as a first acquisition unit 211 and inputs the first instruction sentence into the large-scale language model LLM.

[0113] In step S204, the processing unit 21 functions as a second acquisition unit 212 and inputs the second instruction sentence into the large-scale language model LLM.

[0114] In step S206, the processing unit 21 functions as a first acquisition unit 211 to acquire a first response from the large-scale language model LLM.

[0115] In step S208, the processing unit 21 functions as a second acquisition unit 212 to acquire the first follow-up question sentence from the large-scale language model LLM, and then terminates this routine.

[0116] As shown in Figure 9, the processing unit 21 inputs the first instruction sentence to the large-scale language model LLM before obtaining the first follow-up question sentence from the large-scale language model LLM. The order of the processing in step S202 and step S204 is not particularly important. The processing in step S202 and step S204 may be performed simultaneously. The order of the processing in step S206 and step S208 is also not particularly important. The processing in step S206 and step S208 may be performed simultaneously.

[0117] 1.3. Effects of the First Embodiment As described above, the automatic response server 20 according to the first embodiment comprises a first acquisition unit 211, a second acquisition unit 212, and a display control unit 218. The first acquisition unit 211 acquires a first response from the large-scale language model LLM by inputting a first instruction sentence into the large-scale language model LLM. The first instruction sentence is a sentence that instructs the system to generate a first response to a question by user U based on first data and the question by user U. The first data is data showing the search results of the knowledge database DB1 related to the question by user U.

[0118] The second acquisition unit 212 acquires the first follow-up question from the large-scale language model LLM by inputting the second instruction sentence into the LLM. The first follow-up question is a question intended to clarify the intent of the question asked by user U. The second instruction sentence is a sentence that instructs the generation of the first follow-up question based on the first data and the question asked by user U.

[0119] The display control unit 218 displays the answer based on the first answer on the user U's display device 14 if the user U's second answer to the first follow-up question indicates that the intent of the question by the user U corresponds to the first answer. The first acquisition unit 211 inputs the first instruction sentence into the large-scale language model LLM before the second acquisition unit 212 acquires the first follow-up question.

[0120] According to this embodiment, the instruction to generate the first answer for the large-scale language model LLM is disclosed without waiting for the acquisition of the first follow-up question about the search results in the knowledge database DB1, thus shortening the time from when user U asks the first question until the final answer is obtained.

[0121] Furthermore, the knowledge database DB1 includes multiple classification databases that correspond one-to-one to multiple categories, including the first category. The automated response server 20 according to the first embodiment further comprises a determination unit 216 and a search unit 217. The determination unit 216 determines a classification result based on a question from user U. The classification result has an evaluation value indicating the degree of probability of belonging to each of the multiple categories. If the category corresponding to the highest evaluation value among the multiple evaluation values ​​that correspond one-to-one to the multiple categories is the first category, the search unit 217 obtains the first data by performing a search for the question using the classification database corresponding to the first category from among the multiple classification databases.

[0122] According to this embodiment, the search process in the knowledge database DB1 is started without waiting for the classification results of the classification process, thus shortening the time from when user U asks the first question until they receive the final answer.

[0123] Furthermore, multiple categories include a second category. The automated response server 20 according to the first embodiment further comprises a third acquisition unit 213. The third acquisition unit 213 acquires a second follow-up question from the large-scale language model LLM by inputting a third instruction sentence to the large-scale language model LLM when the highest evaluation value among the multiple evaluation values ​​is less than the threshold TH. The third instruction sentence is a sentence that instructs the generation of a second follow-up question sentence to clarify the category to which the question by user U belongs, based on the classification result and the question by user U.

[0124] The first acquisition unit 211 acquires the third response from user U to the second follow-up question. Based on the third response, the search unit 217 confirms that the category to which user U's question belongs is the second category, and acquires the second data by performing a search for user U's question using the classification database corresponding to the second category. If the highest evaluation value is less than the threshold TH, the first acquisition unit 211 inputs the first instruction sentence into the large-scale language model LLM, thereby acquiring the fourth response from the large-scale language model LLM. Here, the first instruction sentence is a sentence that instructs the system to generate the fourth response to user U's question based on the second data and user U's question.

[0125] The display control unit 218 displays the answer based on the first response on the user U's display device 14 if the highest evaluation value is equal to or greater than the threshold TH, and displays the answer based on the fourth response on the user U's display device 14 instead of the first response if the highest evaluation value is less than the threshold TH.

[0126] According to this embodiment, if, as a result of obtaining User U's third response to the second follow-up question, it is confirmed that the category to which User U's question belongs is a second category different from the first category, a search is performed using the classification database corresponding to the second category. Therefore, a search process that correctly reflects the intent of User U's question can be performed.

[0127] Furthermore, a question from user U has multiple intentions, including a first intention. The first answer includes a candidate answer corresponding to the first intention. If the second answer indicates the first intention, the display control unit 218 displays the candidate answer corresponding to the first intention as the answer based on the first answer on user U's display device 14.

[0128] According to this embodiment, even if the intent of the question posed by user U is ambiguous, a candidate answer to the ambiguous question posed by user U is generated before the intent of user U is confirmed by "requesting clarification." If the second answer obtained through "requesting clarification" indicates the first intent, the generated candidate answer is displayed on the display device 14. Therefore, since a candidate answer is generated without waiting for the second answer, the time from when user U asks the first question until the final answer is obtained can be shortened.

[0129] Furthermore, the second acquisition unit 212 determines whether the intent of the question asked by user U is clear or unclear, based on the first data and the question asked by user U. If the result of the determination is unclear, the second acquisition unit 212 inputs a second instruction sentence into the large-scale language model LLM, thereby obtaining the first follow-up question sentence from the large-scale language model LLM.

[0130] In this embodiment, if the result of the judgment is unclear, a first follow-up question is obtained and presented to user U, thereby generating an answer that more accurately reflects the intent of user U's question.

[0131] Furthermore, if the first acquisition unit 211 does not confirm that the category to which the question from user U belongs is the second category, it inputs the first instruction sentence into the large-scale language model LLM and acquires the first answer from the large-scale language model LLM.

[0132] According to this embodiment, if it is confirmed that the category to which user U's question belongs is the first category and not the second category, the first data has already been obtained at the time user U's answer to the second follow-up question is acquired, and therefore the first instruction can be immediately input into the large-scale language model LLM. Consequently, the time from when user U asks the first question until the final answer is obtained can be shortened.

[0133] The automated response method according to the first embodiment includes a first acquisition process, a second acquisition process, and a display control process. The first acquisition process is a process of acquiring a first response from the large-scale language model LLM by inputting a first instruction sentence into the large-scale language model LLM. The first instruction sentence is a sentence that instructs the system to generate a first response to a question by user U based on first data and the question by user U. The first data is data showing the search results of the knowledge database DB1 related to the question by user U.

[0134] The second acquisition process is the process of acquiring the first follow-up question from the large-scale language model LLM by inputting the second instruction sentence into the LLM. The first follow-up question is a question sentence intended to clarify the intent of the question asked by user U. The second instruction sentence is a sentence that instructs the system to generate the first follow-up question sentence based on the first data and the question.

[0135] The display control process is the process of displaying the answer based on the first answer on the user U's display device 14 when the user U's second answer to the first follow-up question indicates that the intent of the question by the user U corresponds to the first answer. Furthermore, the first acquisition process inputs the first instruction sentence into the large-scale language model LLM before the first follow-up question is acquired by the second acquisition process.

[0136] According to this embodiment, the instruction to generate the first answer for the large-scale language model LLM is disclosed without waiting for the acquisition of the first follow-up question about the search results in the knowledge database DB1, thus shortening the time from when user U asks the first question until the final answer is obtained.

[0137] 2. Modifications This disclosure is not limited to the embodiments illustrated above. Specific examples of modifications are given below. Two or more embodiments may be arbitrarily selected from the following examples and combined. Furthermore, the embodiments of the above embodiments and the modifications below can be combined arbitrarily as long as they do not contradict each other.

[0138] 2.1. Modification 1 In the first embodiment, an example was described in which the knowledge database DB1 of the database server 30 has a plurality of classification databases DB11 to DB1n, and the classification process determines which classification database to use for searching. However, the present invention is not limited to a configuration in which the knowledge database DB1 has a plurality of classification databases.

[0139] Figure 10 is a block diagram showing an overview of the RAG system 100A according to Modification 1. In the RAG system 100A, the automatic response server 20A and the database server 30A differ from the automatic response server 20 and database server 30 according to the first embodiment, respectively. More specifically, the automatic response server 20A does not have a determination unit. The database server 30A does not have a classification processing unit and a classification database.

[0140] In the RAG system 100A, the basic flow of the process by which the automated response server 20A answers a question from user U is as follows:

[0141] First, in step S21, the automated response server 20A receives the question entered by user U from the user terminal device 10. Next, in step S22, the automated response server 20A vectorizes user U's question and searches the knowledge database DB1 for documents that are highly similar to user U's question.

[0142] In step S23, the automated response server 20A obtains search result data for documents with high similarity. In step S24, the automated response server 20A combines the retrieved documents and the user U's question to generate an instruction statement for the large-scale language model LLM in the generating AI server 40, and sends the question, search result data, and instruction statement to the generating AI server 40.

[0143] In step S25, the automated response server 20A retrieves the response generated by the large-scale language model LLM from the large-scale language model LLM while referring to the retrieved document. In step S26, the automated response server 20A presents the response retrieved from the generation AI server 40 to the user U.

[0144] The RAG system 100A shown in Figure 10 has a configuration in which the knowledge database DB1 does not have multiple classification databases. However, before the second acquisition unit 212 acquires the first follow-up question from the large-scale language model LLM, the first acquisition unit 211 can input the first instruction sentence into the large-scale language model LLM.

[0145] 2.2. Modification 2 In the first embodiment, the database server 30 was configured to include a classification processing unit, but the automatic response server 20 may be configured to include a classification processing unit, and the processing unit 21 may determine the evaluation value.

[0146] 2.3. Modification 3 In the first embodiment, the automatic response system 1 is configured such that the automatic response server 20, the database server 30, and the generation AI server 40 are connected by a communication network NET. However, the automatic response server 20, the database server 30, and the generation AI server 40 may be consolidated into a single server device. At least two of the automatic response server 20, the database server 30, and the generation AI server 40 may be consolidated into a single server device, or all three may be consolidated into a single server device.

[0147] 2.4. Modification 4 In the first embodiment, one large-scale language model is used as the large-scale language model LLM, but it is also possible to use multiple large-scale language models to generate various answers and various follow-up questions in parallel.

[0148] 3. Other (1) In the embodiments described above, the storage devices 12, 22, 32, and 42 are exemplified by ROM and RAM, but they can also be flexible disks, magneto-optical disks (e.g., compact disks, digital multipurpose disks, Blu-ray® disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy® disks, magnetic strips, databases, servers, and other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communication network NET via a telecommunications line.

[0149] (2) In the embodiments described above, the information, signals, etc. may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0150] (3) In the embodiments described above, the input and output information may be stored in a specific location (e.g., memory) or managed using a management table. The input and output information may be overwritten, updated, or appended to. The output information may be deleted. The input information may be transmitted to other devices.

[0151] (4) In the embodiments described above, the determination may be made by a value represented using one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0152] (5) The processing procedures, sequences, flowcharts, etc., exemplified in the embodiments described above may be rearranged in order, as long as there is no contradiction. For example, the methods described in this disclosure present various step elements using an exemplary order and are not limited to the specific order presented.

[0153] (6) Each function illustrated in Figures 1 to 10 is realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each function block is not particularly limited. That is, each function block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired, wireless, etc.). A function block may also be realized by combining software with the one or more devices.

[0154] (7) The programs illustrated in the embodiments described above should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages ​​or by other names.

[0155] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0156] (8) In each of the above-mentioned forms, the terms “system” and “network” shall be used interchangeably.

[0157] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information.

[0158] (10) In the embodiments described above, the user terminal device 10 may be a mobile station (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or several other appropriate terms. In this disclosure, terms such as “mobile station,” “user terminal,” “user equipment (UE),” and “terminal” may be used interchangeably.

[0159] (11) In the embodiments described above, the terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain and optical (both visible and invisible) domain.

[0160] (12) In the embodiments described above, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on".

[0161] (13) The terms “determining” and “determining” as used in this disclosure may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0162] (14) In the embodiments described above, where “include,” “including,” and variations thereof are used, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to be exclusive OR.

[0163] (15) In the present disclosure, if articles are added by translation, such as a, an, and the in English, the present disclosure may include the fact that the noun following these articles is plural.

[0164] (16) In this disclosure, the term “A and B are different” may mean “A and B are different from each other.” The term may also mean “A and B are each different from C.” Terms such as “separate” and “combine” may be interpreted in the same way as “different.”

[0165] (17) Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of the specified information (e.g., notification that "it is X") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0166] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Accordingly, the descriptions in the present disclosure are illustrative and not restrictive in any way.

[0167] 10...User terminal device, 20, 20A...Automatic response server, 30, 30A...Database server, 40...Generating AI server, 211...First acquisition unit, 212...Second acquisition unit, 213...Third acquisition unit, 216...Decision unit, 217...Search unit, 218...Display control unit, DB1...Knowledge database, DB11, DB12, DB1n...Classification database, LLM...Large-scale language model, TH...Threshold, U...User.

Claims

1. An automatic response device comprising: a first acquisition unit that acquires a first answer from a large-scale language model by inputting a first instruction to the large-scale language model instructing it to generate a first answer to a question based on first data showing search results from a knowledge database relating to a question made by a user and the question; a second acquisition unit that acquires a first follow-up question sentence from the large-scale language model by inputting a second instruction to the large-scale language model instructing it to generate a first follow-up question sentence to clarify the intent of the question based on the first data and the question; and a display control unit that displays an answer based on the first answer on the user's display device when the user's second answer to the first follow-up question sentence indicates that the intent of the question corresponds to the first answer, wherein the first acquisition unit inputs the first instruction to the large-scale language model before the second acquisition unit acquires the first follow-up question sentence.

2. The automatic response device according to claim 1, further comprising: a knowledge database including a plurality of classification databases that correspond one-to-one to a plurality of categories including a first category; a determination unit that determines a classification result having an evaluation value indicating the degree of probability of belonging to each of the plurality of categories based on the question; and a search unit that, when the category corresponding to the highest evaluation value among the plurality of evaluation values ​​that correspond one-to-one to the plurality of categories is the first category, obtains the first data by performing a search for the question using the classification database corresponding to the first category among the plurality of classification databases.

3. The plurality of categories includes a second category, and the system further comprises a third acquisition unit that acquires the second follow-up question from the large-scale language model by inputting a third instruction to the large-scale language model instructing it to generate a second follow-up question to clarify the category to which the question belongs, based on the classification result and the question, if the highest evaluation value among the plurality of evaluation values ​​is less than a threshold, the first acquisition unit acquires the user's third response to the second follow-up question, the search unit acquires second data by performing a search for the question using the classification database corresponding to the second category, if it is confirmed that the category to which the question belongs is the second category based on the third response, the first acquisition unit acquires the fourth response from the large-scale language model by inputting a first instruction to the large-scale language model instructing it to generate a fourth response to the question, based on the second data and the question, if the highest evaluation value is less than a threshold, the display control unit displays the response based on the first response on the user's display device if the highest evaluation value is greater than or equal to the threshold. The automatic response device according to claim 2, wherein if the highest evaluation value is less than a threshold, the response based on the fourth response is displayed on the user's display device instead of the first response.

4. The question has multiple intentions, including a first intention; the first answer includes a candidate answer corresponding to the first intention; and the display control unit displays the candidate answer as an answer based on the first answer on the user's display device when the second answer indicates the first intention. The automatic response device according to claim 1.

5. The automatic response device according to claim 1, wherein the second acquisition unit determines whether the intent of the question is clear or unclear based on the first data and the question, and if the result of the determination is unclear, it inputs the second instruction to the large-scale language model to acquire the first follow-up question from the large-scale language model.

6. The automated response device according to claim 3, wherein if the first acquisition unit does not confirm that the category to which the question belongs is the second category, it inputs the first instruction to the large-scale language model to acquire the first answer from the large-scale language model.

7. An automated response method comprising: a first acquisition process for obtaining a first answer from a large-scale language model by inputting a first instruction to the large-scale language model instructing it to generate a first answer to a question based on first data showing search results from a knowledge database relating to a question made by a user and the question; a second acquisition process for obtaining a first follow-up question sentence from the large-scale language model by inputting a second instruction to the large-scale language model instructing it to generate a first follow-up question sentence to clarify the intent of the question based on the first data and the question; and a display control process for displaying an answer based on the first answer on the user's display device if the user's second answer to the first follow-up question sentence indicates that the intent of the question corresponds to the first answer, wherein the first acquisition process inputs the first instruction to the large-scale language model before the first follow-up question sentence is obtained by the second acquisition process.

Images