Interactive automated response device, interactive automated response method, and program

Abstract

The present invention provides an interactive automated response system that provides visual information, such as diagrams or tables, related to the answer. [Solution] An interactive automated response device is provided, comprising: an answer generation unit that generates answers to user questions; a search unit that searches a database for visual information related to the answers; and a user interface that simultaneously presents the answers and the retrieved visual information. In the interactive automated response device, the visual information is a figure, a diagram, or an image. The interactive automated response device includes an intent interpretation unit that interprets the intent of the user's question and extracts keywords.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an interactive automatic response device, an interactive automatic response method, and a program.

BACKGROUND ART

[0002] In recent years, with the rapid development of artificial intelligence (AI: Artificial Intelligence) technology, the utilization of interactive automatic response devices (chatbots) has expanded in various fields. A chatbot is a computer program that uses AI to interact with humans through text or voice, and has the ability to generate appropriate responses to inputs from users and conduct interactions using natural language processing and machine learning technologies.

[0003] Related chatbot systems mainly employ the following methods.

[0004] ·Rule-based method: A method that generates responses based on predefined rules and pattern matching. While this method is relatively easy to implement, it has the problem of lacking flexibility and being unable to handle unexpected inputs.

[0005] ·Search-based method: A method that searches for appropriate answers from a large-scale Q&A database. Since this method only uses existing answers, it has a problem that it is difficult to respond to new situations.

[0006] ·Machine learning model method: A method that mainly uses deep learning to generate responses. While this method is capable of generating responses that take context into account, it has high computational costs and a risk of generating inappropriate responses. In addition, this method requires much effort for re-learning to update information.

[0007] ·Knowledge base integration method: A method that links with an external knowledge base or information source. While this method can provide up-to-date and accurate information, updating and managing knowledge presents a challenge.

[0008] Patent Document 1 discloses a program for easily generating suitable prompts for artificial intelligence. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2025-1048 [Overview of the project] [Problems that the invention aims to solve]

[0010] A conversational automated response system (chatbot) is a computer program that utilizes artificial intelligence (AI) to enable natural language dialogue with humans through text or voice. It has the ability to generate appropriate responses to user input using natural language processing and machine learning techniques and to continue the conversation. However, the chatbots in question are mainly limited to providing text-based answers and do not provide visual information such as diagrams or tables related to the answers. Therefore, the purpose of this disclosure is to provide a conversational automated response device that provides visual information such as diagrams or tables related to the answers. [Means for solving the problem]

[0011] The interactive automated response system of this disclosure is An answer generation unit that generates answers to user questions, A search unit that retrieves visual information related to the aforementioned answer from a database, A user interface that simultaneously presents the aforementioned answer and the retrieved visual information, It is an interactive automated response system equipped with [the following features].

[0012] The interactive automated response method described herein is: Generate answers to user questions, Search the database for visual information related to the above answer. The aforementioned answer and the retrieved visual information are presented simultaneously. This is an interactive automatic response method.

[0013] The program of the present disclosure is generates a response to a question from a user, searches a database for visual information related to the response, is a program that causes an information processing apparatus to execute simultaneously presenting the response and the retrieved visual information. [Effects of the Invention]

[0014] The present disclosure provides an interactive automatic response apparatus or the like that provides visual information such as diagrams or tables related to responses. [Brief Description of the Drawings]

[0015] [Figure 1] This is a block diagram showing the configuration of a related interactive automatic response apparatus. [Figure 2] This is a block diagram showing the configuration of the interactive automatic response apparatus according to the present disclosure. [Figure 3] This is a block diagram showing the configuration of a related interactive automatic response system. [Figure 4] This is a block diagram showing the configuration of the interactive automatic response system according to the present disclosure. [Figure 5] This is a diagram showing an example of a question input to a related interactive automatic response apparatus. [Figure 6] This is a diagram showing an example of a response from a related interactive automatic response apparatus. [Figure 7] This is a diagram showing Display Example 1 of a question for the chatbot according to the present disclosure. [Figure 8] This is a diagram showing Display Example 1 of a response from the chatbot according to the present disclosure. [Figure 9] This is a diagram showing Display Example 2 of a question for the chatbot according to the present disclosure. [Figure 10] This is a diagram showing Display Example 2 of a response from the chatbot according to the present disclosure. [Figure 11] This is a table showing Example 1 of diagram description information according to the present disclosure. [Figure 12] This is a diagram showing display example 3 of the chatbot according to the present disclosure. [Figure 13] This is a diagram showing display example 4 of the chatbot according to the present disclosure. [Figure 14] This is a table showing example 2 of diagram explanation information according to the present disclosure. [Figure 15] This is a block diagram showing the configuration of the information processing apparatus according to the present disclosure. [Figure 16] This is a block diagram showing the configuration of the interactive automatic response apparatus according to the present disclosure. [Mode for Carrying Out the Invention]

[0016] (Description of Related Chatbot) FIG. 3 is a block diagram showing the configuration of a related interactive automatic response system. FIG. 5 is a diagram showing an example of a question posed to a related interactive automatic response apparatus. FIG. 6 is a diagram showing an example of a response from a related interactive automatic response apparatus. The related interactive automatic response system will be described with reference to FIG. 3, FIG. 5 and FIG. 6.

[0017] As shown in FIG. 3, the interactive automatic response system 300 includes a chatbot 301, an API 316, and a relevance analysis engine 311. The chatbot 301 includes a GUI (Graphical User Interface) 302, an intention interpretation unit 303, a user response generation unit 304, a procedure control unit 305, sequence data 306, a question generation unit 307, and a response determination unit 308.

[0018] The chatbot 301 is an interactive automatic response apparatus, which is a computer program that interacts with humans via text or voice using artificial intelligence (AI). It has a function of generating an appropriate response to an input from a user using natural language processing and machine learning technologies, and continuing the interaction.

[0019] GUI302 is a graphical user interface. GUI302 has an information input unit 317 for receiving questions from the user and an information output unit 318 for displaying answers to the user.

[0020] The intent interpretation unit 303 interprets the intent of the question from the question entered into the GUI 302, detects the keywords necessary to answer the question, and passes them to the procedure control unit 305.

[0021] The procedure control unit 305 selects the optimal procedure for answering a question from the sequence data 306, based on keywords obtained from the intent interpretation unit 303, using the scenario procedure and LLM query procedure 319. Furthermore, the procedure control unit 305 is connected to the question generation unit 307, the answer determination unit 308, and the user answer generation unit 304.

[0022] The function of the procedure control unit 305 is: 1. The question generation process is started in the question generation unit 307. 2. Obtain information from the response determination unit 308. 3. The user response generation unit 304 receives the information for the response and initiates the process of notifying the user of the information. These processes are executed based on the processing procedure obtained from the sequence data 306.

[0023] The user response generation unit 304 receives response information from the procedure control unit 305 and generates responses to user questions according to control instructions. The user response generation unit 304 then passes the information to the GUI 302 and starts processing in the information output unit 318.

[0024] The sequence data 306 stores the scenario procedure / LLM query procedure 319 for answering the question. The sequence data 306 provides the optimal processing procedure to the procedure control unit 305.

[0025] The question generation unit 307 receives a question based on instructions from the procedure control unit 305 and appropriately processes the prompt to obtain information highly relevant to the desired answer. The question generation unit 307 also has the function of passing this prompt to the relevance analysis engine 311, which is composed of an LLM (Large Language Model) with RAG (Retrieval-Augmented Generation) functionality, via API 316.

[0026] The response determination unit 308, activated by the procedure control unit 305, receives responses from the relevance analysis engine 311, which is composed of an LLM with RAG functionality, via API 316. The response determination unit 308 has the function of processing these responses into information appropriate to the application and passing them to the procedure control unit 305.

[0027] The relevance analysis engine 311 comprises a RAG 312, a knowledge base 313, and an LLM 315. The relevance analysis engine 311 consists of an LLM with RAG functionality. The relevance analysis engine 311 uses an algorithm that analyzes semantic relevance from information registered in the knowledge base 313 in response to the content of a prompt received from the chatbot 301 via API 316, and then provides the most appropriate text response.

[0028] RAG312 analyzes question 332 received from chatbot 301 via API 316 and retrieves the most relevant information from knowledge base 313. RAG312 also has the function of creating a new context to supplement information that LLM315 has not previously learned and asking questions to LLM315, and passing the answer 333 from LLM315 to chatbot 301 via API 316.

[0029] Knowledge Base 313 is a large, structured external information repository used by RAG312. Knowledge Base 313 is a comprehensive data store containing text data, metadata, and associated indexing information.

[0030] LLM315 refers to a large-scale language model. LLM315 is a deep learning model trained on a massive amount of text data. It possesses advanced capabilities in natural language comprehension and generation, and is a versatile language processing method applicable to a variety of tasks.

[0031] API316 is a program interface that relays the question 332 from the question generation unit 307 to the relevance analysis engine 311, and the answer 333 from the relevance analysis engine 311 to the answer determination unit 308.

[0032] The associated interactive automated response system receives questions from the user and, based on the received questions, queries an external relevance analysis engine to obtain answers. The answers are displayed to the user in text format.

[0033] As shown in Figure 5, the user asks the question 330, "What is the second animal of the Chinese zodiac?". The question text entered by the user is received by the GUI 302, and the intent interpretation unit 303 extracts keywords from the input information. The procedure control unit 305 selects appropriate sequence data from the sequence data 306 based on the keywords of the question, and passes the user question to the question generation unit 307 based on the selected sequence data.

[0034] The question generation unit 307 uses the stored question format and user questions to create a user question prompt for querying the relevance analysis engine 311. The question generation unit 307 uses API 316 to send the created user question prompt to the relevance analysis engine 311.

[0035] The response determination unit 308 waits for a response from the relevance analysis engine 311. RAG312 searches the knowledge base 313 for information related to the user question and retrieves the relevant information. Based on the retrieved relevant information and the user question prompt, RAG312 sends the question to LLM315. RAG312 uses API316 to send the response generated by LLM315 to the chatbot.

[0036] The response determination unit 308 determines necessary and unnecessary response information from the received responses and discards the unnecessary response information. The procedure control unit 305 passes the necessary response information regarding the user question obtained from the response determination unit 308 to the user response generation unit 304.

[0037] The user response generation unit 304 creates an answer to the user based on the answer information regarding the user's question obtained from the procedure control unit 305, and displays the text-based answer using the GUI 302. As shown in Figure 6, the answer 331 is obtained which states, "The second animal of the Chinese zodiac is the ox."

[0038] A conversational automated response system is a computer program that utilizes artificial intelligence (AI) to enable natural language dialogue with humans through text or voice. It has the ability to generate appropriate responses to user input using natural language processing and machine learning techniques, and to continue the conversation. However, related chatbots are primarily limited to providing text-based answers and do not provide visual information such as diagrams or tables related to the answers.

[0039] There are several ways to generate responses from a chatbot, including the following: 1. Rule-based approach: This is the most basic method, which detects keywords or specific phrases from the user's question and returns an answer based on predefined rules and pattern matching. However, this method lacks flexibility in responses and has the challenge of not being able to handle unexpected inputs.

[0040] 2. Utilization of Large-Scale Language Models (LLMs): Using LLMs trained on large amounts of dialogue data enables flexible responses that take context into account. However, LLMs such as GPT-4® or Claude® 3.5 have difficulty responding to specialized information that they have not been pre-trained on.

[0041] 3. Introduction of Search and Generate (RAG) Technology: To overcome the limitations of LLM, RAG technology was adopted, and a method was developed to generate answers by utilizing information from an external knowledge base in which specialized information has been pre-registered.

[0042] However, despite these advancements, the following challenges remain for related chatbot systems. 1. Lack of visual information: Chatbots, LLMs, and RAGs all primarily use text as their input and output, and do not provide the functionality to display diagrams related to their responses.

[0043] 2. Discrepancy between understanding and displaying figures and tables: RAG has the function of recognizing and understanding figures and tables. However, its purpose is to convert the figures into text and into a context that LLM can handle, not to directly display the figures.

[0044] 3. Limitations of conveying complex information: Text-only responses make it difficult to effectively convey complex concepts or information.

[0045] 4. Limitations on user comprehension: Due to the lack of visual aids, there are limitations to the user's ability to understand the content.

[0046] 5. Reduced efficiency of dialogue: The lack of supplementary explanations using diagrams and charts makes it time-consuming to explain complex information or procedures, reducing the efficiency of the dialogue.

[0047] These challenges limited the effectiveness of the relevant interactive automated response systems in terms of communication and information transfer with users, thus limiting improvements in user satisfaction and learning outcomes.

[0048] (Description of the interactive automated response system according to the embodiment) Figure 4 is a block diagram showing the configuration of the interactive automated response system according to this disclosure. Figure 7 is a diagram showing example 1 of the display of a question from the chatbot according to this disclosure. Figure 8 is a diagram showing example 1 of the display of an answer from the chatbot according to this disclosure. Figure 9 is a diagram showing example 2 of the display of a question from the chatbot according to this disclosure. Figure 10 is a diagram showing example 2 of the display of an answer from the chatbot according to this disclosure. Figure 11 is a table showing an example of diagram explanatory information according to this disclosure. Figure 12 is a diagram showing example 3 of the display of the chatbot according to this disclosure. Figure 13 is a diagram showing example 4 of the display of the chatbot according to this disclosure. Figure 14 is a table showing example 2 of the diagram explanatory information according to this disclosure. The interactive automated response system according to the embodiment will be described with reference to Figures 4 and 7 through 14.

[0049] As shown in Figure 4, the interactive automated response system 400 according to this embodiment comprises a chatbot 401, an API 416, and a relevance analysis engine 411. Although the interactive automated response system 400 is configured to include the relevance analysis engine 411 externally, both may be considered as a single interactive automated response device. The chatbot 401 comprises a GUI 402, an intent interpretation unit 403, a user response generation unit 404, a procedure control unit 405, sequence data 406, a question generation unit 407, an answer determination unit 408, a figure search unit 409, and a figure database 410.

[0050] Chatbot 401 is an interactive automated response system. It is a computer program that utilizes artificial intelligence (AI) to interact with humans through text or voice. It has the ability to generate appropriate responses to user input using natural language processing and machine learning techniques, and to continue the conversation.

[0051] Chatbot 401 has the ability to display images in addition to providing text responses. It is called a multimodal chatbot because it processes different types of information sources.

[0052] GUI402 is a graphical user interface. GUI402 includes an information input unit 417 for receiving questions from the user and an information output unit 418 for displaying answers to the user. The information output unit 418 includes a diagram display unit 419 for displaying visual information related to the answer. This allows for the simultaneous presentation of textual answers and visual information.

[0053] The intent interpretation unit 403 interprets the intent of the question from the question entered into the GUI 402, detects the keywords necessary to answer the question, and passes them to the procedure control unit 405. The intent interpretation unit 403 can also determine from the question entered into the GUI 402 whether or not a diagram needs to be displayed in the answer.

[0054] The user response generation unit 404 receives information for the response or information about the diagram from the procedure control unit 405. The user response generation unit 404 generates a response to the user's question according to the control instructions, passes the information to the GUI 402, and starts the processing of the information output unit 418.

[0055] The procedure control unit 405 selects the optimal procedure from the sequence data 406 to answer the question based on keywords obtained from the intent interpretation unit 403, choosing from scenario procedures, LLM query procedures, and diagram display procedures 420. Furthermore, the procedure control unit 405 is connected to the question generation unit 407, the answer determination unit 408, the user answer generation unit 404, and the diagram search unit 409.

[0056] The function of the procedure control unit 405 is: 1. The question generation process is started in the question generation unit 407. 2. Obtain information from the answer determination unit 408. 3. Information for the answer or information for the answer diagram is passed to the user answer generation unit 404, and the process of notifying the user of the information is initiated. 4. The figure search unit 409 receives information about the figure obtained from the answer determination unit 408 to obtain figure information related to the answer. These processes are performed based on the processing steps obtained from sequence data 406.

[0057] The sequence data 406 stores scenario procedures, LLM query procedures, and diagram display procedures 420 for answering questions, providing the procedure control unit 405 with the optimal processing procedure.

[0058] The question generation unit 407 receives a question based on instructions from the procedure control unit 405 and appropriately processes the prompt to obtain the desired answer text or information on a figure highly relevant to the answer. The question generation unit 407 has the function of passing this prompt to the relevance analysis engine 411, which is configured with an LLM with RAG functionality, via API 416.

[0059] The response determination unit 408, activated by the procedure control unit 405, receives the response 433 from the relevance analysis engine 411, which is composed of an LLM with RAG functionality, via the API 416. The response determination unit 408 has the function of processing the response 433 into information appropriate to its purpose and passing it to the procedure control unit 405.

[0060] The figure search unit 409, upon instruction from the procedure control unit 405, uses the information obtained from the answer determination unit 408 to transmit to the procedure control unit 405 multiple visual pieces of information with a high degree of relevance or the visual piece of information most relevant to the answer from the figure database 410.

[0061] The diagram database 410 is a dedicated database that stores visual information such as various shapes, charts, images, block diagrams, sequence diagrams, or flowcharts. The diagram database 410 manages each piece of visual information using a unique identifier. The diagram database 410 provides visual information in response to requests from the diagram search unit 409.

[0062] The relevance analysis engine 411 comprises a RAG 412, a knowledge base 413, and an LLM 415. The relevance analysis engine 411 consists of an LLM with RAG functionality. The relevance analysis engine 411 receives prompts from the chatbot 401 via API 416. The relevance analysis engine 411 uses an algorithm that analyzes semantic relevance from the information registered in the knowledge base 413 and the diagram caption information 414 to provide the chatbot 401 with the most appropriate text response to the prompt and the most appropriate visual information to the response in response 433.

[0063] RAG412 is a Retrieval-Augmented Generation (RAG) unit. RAG412 has the function of analyzing question 432 received from chatbot 401 via API 416 and retrieving the most relevant information from knowledge base 413. RAG412 also has the function of creating a new context to supplement information that LLM415 has not previously learned and asking questions to LLM415. RAG412 also has the function of passing the answer 433 from LLM415 to chatbot 401 via API 416.

[0064] Knowledge Base 413 is a large, structured external information repository used by the RAG412 functionality. It is also a comprehensive data store containing text data, metadata, and associated indexing information.

[0065] Figure caption information 414 is a database that stores descriptive data for each piece of visual information. Figure caption information 414 includes information such as the content, subject, and keywords of the visual information, and is processed into information suitable for searching. Figure caption information 414 is stored in a knowledge base.

[0066] LLM415 is a Large Language Model. It is a deep learning model trained on a massive amount of text data. LLM415 possesses advanced capabilities in natural language comprehension and generation, and is a general-purpose language processing method applicable to a variety of tasks.

[0067] API 416 is an Application Programming Interface. API 416 is a program interface that relays the question 432 from the question generation unit 407 to the relevance analysis engine 411, and the answer 433 from the relevance analysis engine 411 to the answer determination unit 408.

[0068] A multimodal chatbot receives questions from users, queries an external relevance analysis engine based on the received questions, and retrieves answers. The answers are displayed to the user in the form of text and relevance diagrams. Thus, a multimodal chatbot provides users with detailed information and visual support.

[0069] This explains how the multimodal chatbot works. Before proceeding, please perform the following preparations. • Register specialized information that LLM415 has not yet learned in Knowledge Base 413. This registration of specialized information is done using RAG412. The diagram caption information shown in Figure 11 is pre-registered in the knowledge base 413. The diagram caption information shown in the table is converted into a format that is easy for RAG 412 to search and is registered in the knowledge base 413 as diagram caption information 414. The files indicated in the table in Figure 11, specifically the figure filenames, should be stored in the chatbot's figure database 410 beforehand.

[0070] The operation flow of the interactive automated response method described herein is as follows: The question text entered by the user is received by GUI402, and the intent interpretation unit 403 detects keywords for the question from the input information.

[0071] The user asks question 430, "What is the second of the twelve zodiac signs?" as shown in Figure 7. Or the user asks question 430, "What is the second of the twelve zodiac signs? Please provide a diagram." as shown in Figure 9. Question 430 is entered into GUI 402. The intent interpretation unit 403 detects from the state of the checkbox in Figure 7 that a diagram is required in the answer. Similarly, the intent interpretation unit 403 detects from the content of the text in Figure 9 that a diagram is required in the answer.

[0072] Another method for detecting whether a diagram is needed in the answer is to query the LLM415 to determine whether the intent of the question necessitates the display of a diagram. The procedure control unit 405 selects appropriate sequence data from the sequence data 406 based on the keyword of the question. Based on the selected sequence data, the procedure control unit 405 passes the user's question to the question generation unit 407.

[0073] The question generation unit 407 uses the stored question format and the user's question to create a user question prompt for the relevance analysis engine 411. The question generation unit 407 uses API 416 to send the created user question prompt to the relevance analysis engine 411.

[0074] The response determination unit 408 waits for a response from the relevance analysis engine 411. The RAG 412 searches the knowledge base 413 for information related to the user's question and retrieves the relevant information. Based on the retrieved relevant information and the user question prompt, the RAG 412 sends the question to the LLM 415. The RAG 412 uses the API 416 to send the response generated by the LLM 415 to the chatbot 401.

[0075] The response determination unit 408 determines from the received responses which response information is necessary and which is unnecessary, and discards the unnecessary response information.

[0076] The procedure control unit 405 holds the necessary answer information regarding the user's question obtained from the answer determination unit 408. The procedure control unit 405 passes the answer information regarding the user's question to the question generation unit 407 in order to perform a figure search.

[0077] The question generation unit 407 uses the stored figure search question format and the answer information obtained from the procedure control unit 405 to create a figure search prompt for the relevance analysis engine 411. The question generation unit 407 uses the API 416 to send the created figure search prompt to the relevance analysis engine 411.

[0078] The response determination unit 408 waits for a response from the relevance analysis engine 411. The RAG 412 searches for information related to the figure search question from the figure description information 414 in the knowledge base 413 and obtains the relevant information. Based on the relevant information obtained from the knowledge base 413 and the figure search prompt, the RAG 412 sends the question to the LLM 415. The RAG 412 sends the answer generated by the LLM 415 to the chatbot using the API 416.

[0079] The response determination unit 408 determines from the received responses which response information is necessary and which is unnecessary, and discards the unnecessary response information.

[0080] The procedure control unit 405 passes the necessary answer information regarding the figure search obtained from the answer determination unit 408 to the figure search unit 409. Based on the answer information regarding the figure search obtained from the procedure control unit 405, the figure search unit 409 searches the figure database 410 and obtains the file names of the relevant figures. The procedure control unit 405 passes the answer information regarding the user's question that it had held and the file names of the figures obtained from the figure search unit 409 to the user answer generation unit 404.

[0081] The user response generation unit 404 creates a response to the user based on the answer information regarding the user's question obtained from the procedure control unit 405. The chatbot 401 displays a text-based response using the GUI 402. The chatbot 401 also displays a diagram related to the user's question using the diagram display unit 419 based on the file name of the diagram obtained from the procedure control unit 405.

[0082] As shown in Figure 8, a picture of an ox is displayed along with the text response, "The second of the twelve zodiac signs is the ox (Ushi)." Also, as shown in Figure 10, a picture of an ox is displayed along with the text response, "The second of the twelve zodiac signs is the ox (Ushi)." This is an example of a response based on the information in the table in Figure 11.

[0083] In practice, the information registered in knowledge base 413 consists of specialized data that LLM415 has not learned. An example of such information registered in knowledge base 413 is shown in the table in Figure 14. Based on the information shown in the table in Figure 14, the chatbot responses shown in Figure 12 or Figure 13 are generated.

[0084] The above configuration improves comprehension through the integration of visual and linguistic information. Simultaneous display of text responses and related diagrams enhances user understanding. Visually supplementing complex concepts or procedures increases the efficiency of information transmission.

[0085] The above configuration optimizes information delivery through highly adaptable chart and graph selection. The relevance analysis engine selects the most suitable chart and graph for each question and answer. Appropriate visual information is provided according to the context of the user's question and dialogue.

[0086] The above configuration ensures that the latest information is provided through knowledge base updates. The knowledge base update mechanism using RAG allows for updating information related to answers, diagram descriptions, and the diagram database. This enables the provision of up-to-date visual information.

[0087] The above configuration ensures consistency between text responses and visual information. The relevance analysis engine analyzes the semantic relationship between responses from the LLM and the explanatory texts in the figure description database, selecting the most appropriate figure. This ensures consistency between text responses and visual information.

[0088] The above configuration enables efficient management and utilization of visual information. The integration of the diagram database and the diagram caption database allows for the efficient management and utilization of a wealth of visual information.

[0089] The above configuration enables the generation of answers that include specialized information and the selection of relevant diagram explanatory information. The Search Enhancement Generation (RAG) function creates new contexts that supplement information that the LLM has not learned. This makes it possible to generate answers that include specialized information and to select the most appropriate information from the diagram explanatory information.

[0090] The above configuration allows for the presentation of easily understandable information. The diagram display control appropriately displays the answers and related diagrams on the GUI. This ensures that information is presented in a way that is easy for the user to understand.

[0091] The above configuration ensures system scalability and flexibility. It is designed to flexibly integrate the latest natural language processing technologies, such as semantic relationship analysis using an LLM with RAG functionality. This allows it to adapt to future information changes.

[0092] (Description of other related interactive automated response systems) Figure 1 is a block diagram showing the configuration of a related interactive automated response system. The related interactive automated response system will be described with reference to Figure 1.

[0093] As shown in Figure 1, the related interactive automated response device (chatbot) 101 comprises a GUI 102, an intent interpretation unit 103, a user response generation unit 104, a procedure control unit 105, and sequence data 106.

[0094] Chatbot 101 is a computer program that uses artificial intelligence (AI) to interact with humans through text or voice. It has the ability to generate appropriate responses to user input using natural language processing and machine learning techniques, and to continue the conversation.

[0095] GUI102 is a graphical user interface. GUI102 includes an information input unit 107 for receiving questions from the user and an information output unit 108 for displaying answers to the user.

[0096] The intent interpretation unit 103 interprets the intent of the question from the question entered into the GUI 102, detects the keywords necessary to answer the question, and passes them to the procedure control unit 105.

[0097] The procedure control unit 105 selects the optimal scenario from the scenario procedure 109 to answer the question from the sequence data 106 based on keywords obtained from the intent interpretation unit 103. The procedure control unit 105 is connected to the user response generation unit 104. The procedure control unit 105 has the function of passing information for the answer to the user response generation unit 104 and initiating the information notification process to the user. These processes are executed based on the processing procedure obtained from the sequence data 106.

[0098] The user response generation unit 104 receives response information from the procedure control unit 105 and generates responses to user questions according to control instructions. The user response generation unit 104 passes the information to the GUI 102 and starts processing in the information output unit 108.

[0099] The sequence data 106 stores the scenario procedure for answering question 110. The sequence data 106 provides the optimal processing procedure to the procedure control unit 105.

[0100] The related chatbot works as follows: The question text entered by the user is received by the GUI 102, and the intent interpretation unit 103 detects keywords of the question from the input information. The procedure control unit 105 selects an appropriate scenario procedure 109 from the sequence data 106 based on the keywords of the question.

[0101] The procedure control unit 105 passes answer information regarding the user's question to the user response generation unit 104 based on the scenario procedure 109. The user response generation unit 104 generates an answer to the user based on the answer information regarding the user's question obtained from the procedure control unit 105. The chatbot 101 displays the answer 111 using the GUI 102.

[0102] (Description of an interactive automatic response device according to other embodiments) Figure 2 is a block diagram showing the configuration of the interactive automated response device according to this disclosure. Other embodiments of the interactive automated response device will be described with reference to Figure 2.

[0103] Other embodiments of the interactive automated response device differ from the interactive automated response system of the embodiment in that the LLM with RAG function in the relevance analysis engine is replaced with a deep learning model.

[0104] As shown in Figure 2, the interactive automated response device (chatbot) 201 comprises a GUI 202, an intent interpretation unit 203, a user response generation unit 204, a procedure control unit 205, sequence data 206, a question generation unit 207, an answer determination unit 208, a figure search unit 209, a figure database 210, and a relevance analysis engine 211.

[0105] Chatbot 201 is an interactive automated response system. It is a computer program that utilizes artificial intelligence (AI) to interact with humans through text or voice. Chatbot 201 has the ability to generate appropriate responses to user input using natural language processing and machine learning techniques, and to continue the conversation.

[0106] GUI202 is a graphical user interface. GUI202 includes an information input unit 215 for receiving questions from the user and an information output unit 216 for displaying answers to the user. The information output unit 216 includes a diagram display unit 217 for displaying visual information related to the answers.

[0107] The intent interpretation unit 203 interprets the intent of the question 220 entered into the GUI 202, detects the keywords necessary to answer 221, and passes them to the procedure control unit 205.

[0108] The user response generation unit 204 receives information for the response or information about the diagram from the procedure control unit 205. The user response generation unit 204 generates a response to the user's question according to the control instructions, passes the information to the GUI 202, and starts processing in the information output unit 216.

[0109] The procedure control unit 205 selects the optimal procedure from the sequence data 206 to answer the question, based on keywords obtained from the intent interpretation unit 203, from the scenario procedure and diagram display procedure 218. Furthermore, the procedure control unit 205 is connected to the question generation unit 207, the answer determination unit 208, the user answer generation unit 204, and the diagram search unit 209.

[0110] The function of the procedure control unit 205 is: 1. The question generation process is started in the question generation unit 207. 2. Obtain information from the response determination unit 208. 3. Information for the answer or information for the answer diagram is passed to the user answer generation unit 204, and the process of notifying the user of the information is initiated. 4. The figure search unit 209 receives information about the figure obtained from the answer determination unit 208 to obtain figure information related to the answer. These processes are performed based on the processing steps obtained from the sequence data 206.

[0111] The sequence data 206 stores scenario procedures and diagrammatic display procedures 218 for answering questions, providing the procedure control unit 205 with the optimal processing procedure.

[0112] The question generation unit 207 receives a question based on instructions from the procedure control unit 205 and appropriately processes the prompt to obtain the desired answer text or information on a figure highly relevant to the answer. It also has the function of passing this prompt to the relevance analysis engine 211.

[0113] The response determination unit 208 receives a response from the relevance analysis engine 211 upon activation by the procedure control unit 205. The response determination unit 208 has the function of processing the response into information appropriate to its purpose and passing it to the procedure control unit 205.

[0114] The figure search unit 209, upon instruction from the procedure control unit 205, uses the information obtained from the answer determination unit 208 to transmit to the procedure control unit 205 multiple visual pieces of information with a high degree of relevance or the visual piece of information most relevant to the answer from the figure database 210.

[0115] The figure database 210 is a dedicated database that stores various visual information such as figures, diagrams, and images. The figure database 210 manages each piece of visual information using a unique identifier. The figure database 210 provides visual information in response to requests from the figure search unit 209.

[0116] The relevance analysis engine 211 is composed of a deep learning model 213. The relevance analysis engine 211 passes figure description information 214 to the answer determination unit 208 according to the content of the prompt received from the question generation unit 207. Furthermore, based on pre-learned content, the relevance analysis engine 211 uses an algorithm to analyze semantic relevance and passes the most appropriate text answer and the visual information most appropriate for that answer to the answer determination unit 208.

[0117] The deep learning model 213 has a structured repository, which is formed by learning about the knowledge of the domain to which the chatbot 201 is applied. The deep learning model 213 responds to the content of the prompt received from the question generation unit 207 from text data, metadata, related index information, and figure description information 214. Based on the content of the prompt, the deep learning model 213 uses an algorithm that analyzes semantic relevance to pass the most appropriate text response and visual information appropriate to that response to the response determination unit 208.

[0118] Figure caption information 214 is a database that stores descriptive data corresponding to each piece of visual information. Figure caption information 214 includes information such as the content, subject, or keywords of the visual information, which is processed into information suitable for retrieval and trained on the deep learning model 213.

[0119] The above configuration provides the same effects as the interactive automated response system according to the embodiment.

[0120] (Description of an interactive automatic response device according to an embodiment) Figure 16 is a block diagram showing the configuration of the interactive automated response device of this disclosure. An embodiment of the interactive automated response device will be described with reference to Figure 16.

[0121] As shown in Figure 16, the interactive automatic response device 1600 according to this embodiment includes an answer generation unit 1601, a search unit 1602, and a user interface 1603.

[0122] The response generation unit 1601 generates answers to questions from the user. The response generation unit 1601 corresponds to, for example, the relevance analysis engine 411 of the interactive automated response system 400 according to the embodiment. The response generation unit 1601 corresponds to the relevance analysis engine 211 of the chatbot 201 according to another embodiment.

[0123] The search unit 1602 searches the database for visual information related to the answer. The search unit 1602 corresponds to, for example, the figure search unit 409 of the interactive automated response system 400 according to one embodiment. The search unit 1602 corresponds to, for example, the figure search unit 209 of the chatbot 201 according to another embodiment.

[0124] The user interface 1603 simultaneously presents the answer and the retrieved visual information. The user interface 1603 corresponds to, for example, the GUI 402 of the interactive automated response system 400 according to one embodiment. The user interface 1603 also corresponds to, for example, the GUI 202 of the chatbot 201 according to another embodiment.

[0125] The above configuration provides an interactive automated response system that provides visual information, such as diagrams or tables, related to the answer.

[0126] Computer programs are executed and processed by information processing devices. Figure 15 is a block diagram showing the configuration of an information processing device according to this disclosure. The information processing device 1500 includes a processor 1501 that executes and processes programs, and a memory 1502 that stores programs. The information processing device 1500 may consist of one device or multiple devices. The information processing device 1500 may also be a cloud server that processes some or all of its functions in a distributed manner.

[0127] Some or all of the processing in the information processing device 1500 described above can be implemented as a computer program. Such a program can be stored using various types of non-temporary computer-readable media and supplied to a computer. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer using various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0128] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0129] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate.

[0130] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An answer generation unit that generates answers to user questions, A search unit that retrieves visual information related to the aforementioned answer from a database, A user interface that simultaneously presents the aforementioned answer and the retrieved visual information, An interactive automated response system equipped with [a specific feature]. (Note 2) The aforementioned visual information is a figure, chart, image, block diagram, sequence diagram, or flowchart, as described in Appendix 1 of the interactive automated response device. (Note 3) The interactive automated response device described in Appendix 1, comprising an intent interpretation unit that interprets the intent of the user's question and extracts keywords. (Note 4) An interactive automated response device as described in Appendix 3, comprising a question generation unit that generates questions for a Large Language Model (LLM). (Note 5) The interactive automatic response device described in Appendix 4, comprising a response determination unit that processes the response from the LLM to create information to be searched by the search unit. (Note 6) The interactive automatic response device described in Appendix 5, comprising a procedure control unit that controls the coordination between the question generation unit, the answer determination unit, the answer generation unit, and the search unit based on the detected keyword. (Note 7) An interactive automatic response device as described in Appendix 6, comprising sequence data for storing scenario procedures and diagram display procedures, which provides processing procedures to the procedure control unit. (Note 8) A database of diagrammatic information that stores explanatory data corresponding to the aforementioned visual information, An interactive automatic response device equipped with [the following features]. (Note 9) An interactive automated response device as described in Appendix 8, which provides a text response and visual information suitable for the text response by using an algorithm that analyzes semantic relationships from a prompt generated from the aforementioned question and explanatory data corresponding to the aforementioned visual information. (Note 10) An interactive automated response device as described in Appendix 9, in which an LLM with RAG (Retrieval-Augmented Generation) functionality is used to employ the algorithm for analyzing the semantic relationships. (Note 11) An interactive automated response device as described in Appendix 9, wherein a deep learning model is used with an algorithm that analyzes the semantic relationships mentioned above. (Note 12) Generate answers to user questions, Search the database for visual information related to the above answer. The aforementioned answer and the retrieved visual information are presented simultaneously. Interactive automated response method. (Note 13) Generate answers to user questions, Search the database for visual information related to the above answer. A program that causes an information processing device to simultaneously present the aforementioned answer and the retrieved visual information.

[0131] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 11 that are dependent on Appendice 1 {e.g., device} may also be dependent on Appendices 12 {e.g., method} and 13 {e.g., program} in the same way as in Appendices 2 to 11. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software. [Explanation of symbols]

[0132] 101 Chatbot, 102 GUI, 103 Intent Interpretation Unit, 104 User Response Generation Unit, 105 Procedure Control Unit, 106 Sequence Data, 107 Information Input Unit, 108 Information Output Unit, 109 Scenario Procedure, 110 Question, 111 Answer, 201 Chatbot, 202 GUI, 203 Intent Interpretation Unit, 204 User Response Generation Unit, 205 Procedure Control Unit, 206 Sequence Data, 207 Question Generation Unit, 208 Answer Determination Unit, 209 Figure Search Unit, 210 Figure Database, 211 Relevance Analysis Engine, 213 Deep Learning Model, 214 Figure Description Information, 215 Information Input Unit, 216 Information Output Unit, 217 Figure Display Unit, 218 Scenario Procedure / Figure Display Procedure, 220 Question, 221 Answer, 300 Interactive Automated Response System, 301 Chatbot, 302 GUI, 303 304 Intent Interpretation Unit, 305 User Response Generation Unit, 306 Procedure Control Unit, 307 Sequence Data, 307 Question Generation Unit, 308 Response Determination Unit, 311 Relevance Analysis Engine, 312 RAG, 313 Knowledge Base, 315 LLM, 316 API, 317 Information Input Unit, 318 Information Output Unit, 319 Scenario Procedure / LLM Inquiry Procedure, 330 Question, 331 Response, 332 Question, 333 Response, 400 Interactive Automated Response System, 401 Chatbot, 402 GUI, 403 Intent Interpretation Unit, 404 User Response Generation Unit, 405 Procedure Control Unit, 406 Sequence Data, 407 Question Generation Unit, 408 Response Determination Unit, 409 Figure Search Unit, 410 Figure Database, 411 Relevance Analysis Engine, 412 RAG, 413 Knowledge Base, 414 Figure Description Information, 415 LLM, 416 API, 417 Information Input Unit, 418 Information Output Unit, 419 Diagram Display Unit, 420 Scenario Procedure / LLM Inquiry Procedure / Diagram Display Procedure, 430 Question, 431 Answer, 432 Question, 433 Answer, 1500 Information Processing Device, 1501 Processor, 1502 Memory, 1600 Interactive Automated Response Device, 1601 Answer Generation Unit, 1602 Search Unit, 1603 User Interface

Claims

1. An answer generation unit that generates answers to user questions, A search unit that retrieves visual information related to the aforementioned answer from a database, A user interface that simultaneously presents the aforementioned answer and the retrieved visual information, An interactive automated response system equipped with [a specific feature].

2. The interactive automatic response device according to claim 1, wherein the visual information is a figure, chart, image, block diagram, sequence diagram, or flowchart.

3. The interactive automated response device according to claim 1, further comprising an intent interpretation unit that interprets the intent of the user's question and extracts keywords.

4. The interactive automatic response device according to claim 3, comprising a question generation unit that generates questions for a Large Language Model (LLM).

5. The interactive automatic response device according to claim 4, further comprising a response determination unit that processes the response from the LLM to create information to be searched by the search unit.

6. The interactive automatic response device according to claim 5, further comprising a procedure control unit that controls the coordination between the question generation unit, the answer determination unit, the answer generation unit, and the search unit based on the detected keyword.

7. The interactive automatic response device according to claim 6, further comprising sequence data for storing scenario procedures and diagram display procedures, which provides a processing procedure to the procedure control unit.

8. A database of diagrammatic information that stores explanatory data corresponding to the aforementioned visual information, An interactive automatic response device according to claim 1, comprising:

9. Generate answers to user questions, Search the database for visual information related to the above answer. The aforementioned answer and the retrieved visual information are presented simultaneously. Interactive automated response method.

10. Generate answers to user questions, Search the database for visual information related to the above answer. A program that causes an information processing device to simultaneously present the aforementioned answer and the retrieved visual information.

Images