Information processing system and information processing device

Abstract

A virtual agent conversation system (10) conducts a conversation with a user by using a main agent (22A) and a plurality of different specialized agents (22B). The virtual agent conversation system (10) comprises: a conversation unit (140A) that instructs an artificial intelligence system (12) to generate conversation content of a virtual agent (22) for the user, acquires the conversation content generated by the artificial intelligence system (12), and presents the conversation content to the user; and a dispatch LLM (144A) that, when an artificial intelligence system (12A) corresponding to the main agent (22A) has determined that generation of conversation content based on an utterance of the user is to be handed over to a specialized agent (22B), uses artificial intelligence to select the specialized agent (22B) to take over the conversation with the user.

Description

Information Processing System and Information Processing Apparatus

[0001] The present disclosure relates to an information processing system and an information processing apparatus.

[0002] In recent years, virtual agent systems have been developed that use a trained conversation model obtained by machine learning to conduct conversations with users in natural language.

[0003] In such a virtual agent system, a terminal such as a smartphone used by a user displays an image of a character corresponding to a virtual agent, and receives inputs such as questions from the user. The virtual agent is, for example, a two-dimensional or three-dimensional animation character or the like set as the user's conversation partner. In some cases, the user may use a plurality of virtual agents with different specialties or the like separately to conduct conversations.

[0004] Patent Document 1 describes calculating the similarity between a user utterance text from a user for the currently selected agent and each of the user assumption texts included in all agents, and switching to another agent that includes the user assumption text with a similarity equal to or higher than a predetermined threshold and the highest similarity.

[0005] Japanese Unexamined Patent Publication No. 2020-034626

[0006] In Patent Document 1, another agent to be switched to is selected based on the similarity between the user utterance text and the user assumption text. However, there is a possibility that the selection of the other agent to be switched to cannot be performed appropriately by the similarity calculation.

[0007] Accordingly, an object of the present disclosure is to provide an information processing system and an information processing apparatus that enable more appropriate handover of a conversation to a virtual agent.

[0008] The information processing system disclosed herein is an information processing system that engages in conversation with a user through a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user. The system comprises a conversation unit that instructs a task processing unit, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing unit and presents it to the user. The system comprises a selection unit that, when the task processing unit corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to a specialized virtual agent, uses artificial intelligence to select a specialized virtual agent to take over the conversation with the user.

[0009] The information processing device disclosed herein is an information processing device that engages in conversation with a user through a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user. The device comprises a conversation unit that instructs a task processing device, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing device and presents it to the user. The task processing device corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to the specialized virtual agents, and a selection unit that uses artificial intelligence to select the specialized virtual agent to take over the conversation with the user.

[0010] This disclosure will enable the transfer of conversations to more appropriate virtual agents.

[0011] Figure 1 is a diagram showing the configuration of the virtual agent conversation system of this embodiment. Figure 2 is a diagram showing the hardware configuration of the user terminal of this embodiment. Figure 3 is a diagram showing the hardware configuration of the server of this embodiment. Figure 4 is a functional block diagram related to conversation handover of this embodiment. Figure 5 is a flowchart showing the flow of conversation processing when a conversation is handed over from the main agent to a specialist agent in this embodiment.

[0012] Embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are merely examples of how this disclosure may be implemented, and this disclosure is not limited to the specific configurations described below. In implementing this disclosure, specific configurations may be adopted as appropriate depending on the embodiment.

[0013] Figure 1 shows an example of the configuration of the virtual agent conversation system 10 of this embodiment. As shown in Figure 1, the virtual agent conversation system 10 of this embodiment is an information processing system that uses an artificial intelligence system (generative artificial intelligence model) 12 to generate conversation content between a virtual agent (AI agent) and a user.

[0014] The virtual agent conversation system 10 of this embodiment comprises an artificial intelligence system 12, a user terminal 14, and a distribution server 16. The user terminal 14 is owned by a user who uses the virtual agent conversation system 10. Multiple distribution servers 16 and artificial intelligence systems 12 may be provided, as will be described in detail later.

[0015] The virtual agent conversation system 10 may include other servers, terminals, and other equipment. The user terminal 14 and the distribution server 16 communicate with each other via a network NW. The distribution server 16 and the artificial intelligence system 12 also communicate with each other via a network NW. The network NW consists of the Internet, LAN (Local Area Network), various mobile communication systems such as wireless base stations, etc. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the Internet via predetermined access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, Bluetooth®, etc. When connecting via a wired connection, the network also includes connections made directly by USB (Universal Serial Bus) cables, etc.

[0016] The user terminal 14 is an information processing device equipped with a touch panel display 20 and used by the user. The user terminal 14 accepts user operations. The user terminal 14 launches the application of this embodiment (hereinafter sometimes simply referred to as "app") based on user operations. The app of this embodiment is a virtual agent app that provides a conversation service with the virtual agent 22 displayed on the touch panel display 20 of the user terminal 14. The user terminal 14 requests content to be displayed or played by the app from the distribution server 16 based on user operations. The user terminal 14 may display or play content using any app, as long as it is capable of displaying or playing the content distributed by the distribution server 16. For example, the user terminal 14 may display or play content using a dedicated application such as the virtual agent app of this embodiment, or it may display or play content including the virtual agent 22 using a browser app (hereinafter sometimes simply referred to as "browser").

[0017] The user terminal 14 can be implemented as, for example, a smartphone, tablet, notebook PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), etc. In this embodiment, the user terminal 14 is a smartphone with touch panel functionality.

[0018] In this embodiment, as an example, the user terminal 14 has the function of a voice response system (speech recognition), detects voice information (hereinafter also referred to as "utterances") spoken by the user, and accepts the detected utterances as input information. For example, the user terminal 14 is a terminal device that processes the user's utterances. When the user terminal 14 detects an utterance by the user, it sends the input information based on the user's utterance to the distribution server 16, receives content corresponding to the user's utterance, and presents it to the user.

[0019] The distribution server 16 receives a distribution request from the user's user terminal 14 and distributes content related to the virtual agent 22 that interacts with the user in the virtual agent application launched on the user terminal 14. The distribution server 16 uses the artificial intelligence system 12 to generate the conversation content of the virtual agent 22 to the user and distributes the generated conversation content to the user terminal 14. The distribution server 16 is composed of at least a computer having communication and program execution functions. The distribution server 16 may be implemented with one information processing device or with multiple information processing devices.

[0020] Furthermore, the distribution server 16 may have a speech recognition function. Also, the distribution server 16 may be able to obtain information from a speech recognition server that provides speech recognition services. In this case, the virtual agent conversation system 10 may include a speech recognition server. In this embodiment, it is assumed that the user terminal 14, distribution server 16, and speech recognition server recognize user utterances and identify the user who made the utterances using various conventional technologies as appropriate, so a detailed explanation is omitted.

[0021] The artificial intelligence system 12 is an information processing device that internally incorporates an LLM (Large Language Model) and is capable of outputting output sentences (responses to prompts) in response to input sentences called prompts. For example, the artificial intelligence system 12 may include ChatGPT, OpenAI GPT, PerplexityAsk, BingAI, BERT, etc. The artificial intelligence system 12 may be operated by the business operator providing the virtual agent application of this embodiment, or it may be used via an API (Application Programming Interface) provided by an external artificial intelligence system 12 operated by another business operator.

[0022] The artificial intelligence system 12 is equipped with a dialogue (chat) function. The distribution server 16 can provide the artificial intelligence system 12 with arbitrary inquiries or commands in text form, and obtain answers to those inquiries or commands. Thus, the artificial intelligence system 12 of this embodiment is a task processing device that uses artificial intelligence to process tasks in response to task processing instructions such as arbitrary inquiries or commands.

[0023] In this embodiment, the distribution server 16 sends a prompt to the artificial intelligence system 12 to cause the artificial intelligence system 12 to generate the content of the conversation between the virtual agent 22 and the user, or the content of the conversation for the virtual agent to speak to the user, thereby obtaining the conversation content as a response.

[0024] Furthermore, in this embodiment, the artificial intelligence system 12 is not limited to text-based dialogue. For example, the artificial intelligence system 12 may be capable of voice-based dialogue. In this case, the artificial intelligence system 12 may receive voice data of the user's utterance (for example, the voice data of the utterance itself, or the voice data of the utterance with noise canceled out) along with the user's identification information, and generate the conversation content of the virtual agent. Alternatively, the artificial intelligence system 12 may be an image generation AI system such as Midjourney or Stable Diffusion that can generate image data.

[0025] Figure 2 shows an example of the hardware configuration of the user terminal 14 in this embodiment. The user terminal 14 is a computer (information processing device) such as a smartphone, tablet, notebook PC (Personal Computer), desktop PC, mobile phone, or PDA (Personal Digital Assistant).

[0026] The user terminal 14 is physically configured as a computer including a processor 100, memory 102, storage 104, communication device 106, input device 108, output device 110, and a bus connecting these. Each of these devices operates on power supplied from a battery (not shown). In the following description, the term "device" can be read as a circuit, device, unit, etc. The hardware configuration of the user terminal 14 may include one or more of the devices shown in Figure 2, or some of the devices may be omitted. Alternatively, multiple devices with different enclosures may be connected via communication to constitute the user terminal 14.

[0027] Each function in the user terminal 14 is realized by loading predetermined software (programs) onto hardware such as the processor 100 and memory 102, which causes the processor 100 to perform calculations, control communication by the communication device 106, and control at least one of the reading and writing of data in the memory 102 and storage 104.

[0028] The processor 100 controls the entire computer, for example, by running the operating system. The processor 100 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. Also, for example, a baseband signal processing unit or a call processing unit may be implemented by the processor 100.

[0029] The processor 100 reads programs (program code), software modules, data, etc., from at least one of the storage 104 and the communication device 106 into the memory 102 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described later. Functional blocks of the user terminal 14 may be stored in the memory 102 and implemented by control programs that run on the processor 100. Various processes may be executed by one processor 100, or they may be executed simultaneously or sequentially by two or more processors 100. The processor 100 may be implemented by one or more chips. The program may also be transmitted to the user terminal 14 via a telecommunications line.

[0030] Memory 102 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may also be called a register, cache, main memory, etc. Memory 102 can store executable programs (program code), software modules, etc., for carrying out the method of this embodiment.

[0031] The storage 104 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., Compact Disc, Digital Multipurpose Disc, Blu-ray® Disc), a smart card, flash memory (e.g., a card, stick, key drive), a floppy® disk, a magnetic strip, etc. The storage 104 may also be called an auxiliary storage device.

[0032] The communication device 106 enables communication with the distribution server 16. The communication device 106 may be configured to include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the transmitting and receiving antennas, amplifier section, transmitting and receiving section, transmission path interface, etc., may be implemented by the communication device 106. The transmitting and receiving section may be implemented in a physically or logically separated manner, with a transmitting section and a receiving section.

[0033] The input device 108 is an input device that accepts input from the outside (for example, keys, microphones, cameras, switches, buttons, position information sensors, motion sensors, and various other sensors). The output device 110 is an output device that outputs to the outside (for example, displays, speakers, LED lamps, etc.). In this embodiment, the user terminal 14 includes a touch panel display 20 in which the input device 108 and the output device 110 are integrated.

[0034] Each device, such as the processor 100 and the memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used for each device.

[0035] The user terminal 14 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by this hardware. For example, the processor 100 may be implemented using at least one of these hardware components.

[0036] The distribution server 16 is an information processing device, and as will be described in detail later, it generates responses to questions and other information entered by the user via the user terminal 14 using generative AI (Artificial Intelligence) built with Large Language Models (LLM).

[0037] Figure 3 shows an example of the hardware configuration of the distribution server 16 in this embodiment. The distribution server 16 is an information processing device implemented by, for example, a computer, server device, cloud system, etc. Physically, the distribution server 16 is configured as a computer device including a processor 120, memory 122, storage 124, communication device 126, input device 128, output device 130, and a bus connecting these. The processor 120, memory 122, storage 124, input device 128, and output device 130 are the same hardware as the processor 100, memory 102, storage 104, input device 108, and output device 110 of the user terminal 14, so their description is omitted.

[0038] The communication device 126 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 126 enables communication with the user terminal 14.

[0039] The hardware configuration of the artificial intelligence system 12 in this embodiment is the same as that of the distribution server 16, so its description will be omitted.

[0040] In the virtual agent conversation system 10, as described above, the user converses with the virtual agent 22 using the virtual agent application installed on the user terminal 14.

[0041] The virtual agent application of the present embodiment enables conversations between a user and a plurality of virtual agents 22 of different types. For this reason, the user uses the virtual agent application to select one virtual agent 22 from the plurality of virtual agents 22 of different types to have a conversation. The virtual agent application displays different 2D or 3D characters on the touch panel display 20 of the user terminal 14 according to the virtual agent 22 selected by the user.

[0042] As an example, the virtual agents 22 include a main agent and sub-agents. The main agent is a virtual agent 22 that has low expertise but conducts general-purpose conversations with the user. For example, the user has daily conversations with the main agent. The sub-agent is a virtual agent 22 that conducts more specialized conversations than the main agent, and is referred to as a specialized agent in the following description. The specialized agents are categorized, for example, by categories such as law, home appliances, cooking, and childcare. The user asks specialized questions and the like to the specialized agent.

[0043] In the virtual agent conversation system 10 of the present embodiment, as an example, a different artificial intelligence system 12 (LLM) is constructed for each of the main agent and the specialized agents. In particular, the LLM of the artificial intelligence system 12 corresponding to a specialized agent has undergone learning specialized for the specialized content of the relevant specialized agent. In addition, the main agent and each specialized agent of the virtual agent conversation system 10 of the present embodiment execute responses to the user via the user terminal 14 by having different distribution servers 16 perform the respective processes.

[0044] Further, the business operators that operate the main agent and the specialized agents may be different. In such a form, the distribution server 16 that operates the main agent and the distribution server 16 that operates the specialized agent are operated by different business operators.

[0045] In this embodiment, the virtual agent conversation system 10 allows a specialist agent to take over a conversation between a user and a main agent while the user and the main agent are in the middle of the conversation. This conversation can be taken over not only by a user requesting it from the main agent, but also by the main agent proactively deciding to hand over the conversation to a specialist agent. For example, if a user asks a question to the main agent, and the question is technical, and it is appropriate for the user to have a conversation with a specialist agent, the conversation may be taken over. Even if the user's question is not technical, the conversation may be taken over if, for other reasons, it is deemed more appropriate for another virtual agent 22 to take over the conversation.

[0046] The determination of whether a conversation needs to be taken over is made by the artificial intelligence system 12 corresponding to the virtual agent 22 that is conversing with the user, for example, based on whether the user's utterance satisfies pre-set conditions. Then, based on the content of the conversation between the user and virtual agent 22, an appropriate other virtual agent 22 to take over the conversation is selected.

[0047] Figure 4 is a functional block diagram relating to the handover of conversation from the main agent 22A to the specialized agent 22B. In the example in Figure 4, the operator of the main agent 22A and the operator of the specialized agent 22B are different operators. In the following explanation, the operator of the main agent 22A will be referred to as Operator A, and the operator of the specialized agent 22B will be referred to as Operator B. In Figure 4, the artificial intelligence system 12, distribution server 16 and its functions corresponding to the main agent 22A will be denoted with "A" at the end of their symbols, and the artificial intelligence system 12, distribution server 16 and its functions corresponding to the specialized agent 22B will be denoted with "B" at the end of their symbols to distinguish them and explain them accordingly.

[0048] The artificial intelligence system 12A may be operated by the operator A, or may be operated by an operator different from the operator A or the operator B. The LLM of the artificial intelligence system 12A is constructed as a highly versatile trained model that is not specialized. Further, the artificial intelligence system 12A of the present embodiment determines whether or not to hand over the generation of conversation content based on an utterance from a user to the specialized agent 22B.

[0049] The distribution server 16A is a distribution server 16 operated by the operator A, and includes a conversation unit 140A, a storage control unit 142A, a scheduling LLM 144A, and a storage 124A. The conversation unit 140A, the storage control unit 142A, and the scheduling LLM 144A are functions provided by the processor 120 of the distribution server 16A.

[0050] The conversation unit 140A instructs the artificial intelligence system 12A to generate conversation content of the main agent 22A for the user, acquires the conversation content generated by the artificial intelligence system 12A, and presents it to the user. The above instruction from the conversation unit 140A is a prompt. The prompt is generated based on a question or the like from the user input via the user terminal 14. The presentation of the conversation content to the user is, for example, an utterance of the main agent 22A displayed on the touch panel display 20 of the user terminal 14.

[0051] The storage control unit 142A stores information related to the content of conversations with the user for each virtual agent 22 in a database (hereinafter referred to as "conversation DB"). For this purpose, the artificial intelligence system 12A transmits the content of the conversation between the main agent 22A and the user to the storage control unit 142A, and causes the conversation content to be stored in the conversation DB. The conversation DB is held in the storage 124A. The conversation unit 140A instructs the artificial intelligence system 12A to generate conversation content such that the conversation content is generated with reference to the conversation DB.

[0052] The LLM 144A, when the artificial intelligence system 12A determines that it should hand over the conversation with the user to a specialist agent 22B, functions as a selection unit that uses artificial intelligence (LLM) to select a specialist agent 22B to take over the conversation with the user. The LLM 144A selects one specialist agent 22B from among several pre-registered specialist agents 22B according to the content of the conversation between the user and the main agent 22A.

[0053] The LLM of the artificial intelligence system 12B, which corresponds to the specialist agent 22B, is constructed as a specialized trained model corresponding to, for example, law, home appliances, childcare, or cooking. The artificial intelligence system 12B may be operated by business operator B, or by a business operator different from business operator A or business operator B. In this embodiment, the artificial intelligence system 12B transmits the content of the conversation between the specialist agent 22B and the user to the memory control unit 142A, and stores the conversation content in the conversation DB.

[0054] The distribution server 16B is a distribution server 16 operated by business operator B, and is equipped with a conversation unit 140B. The distribution server 16B is provided for each specialized field corresponding to the specialized agent 22B, and may be operated by a business operator specializing in each specialized field.

[0055] The conversation unit 140B has the same functions as the conversation unit 140A, and instructs the artificial intelligence system 12B to generate conversation content for the expert agent 22B to the user, and retrieves the conversation content generated by the artificial intelligence system 12B and presents it to the user.

[0056] In this embodiment, the conversation unit 140B can instruct the artificial intelligence system 12B to generate conversation content by referring to information about conversations between the user and the main agent 22A, which is another virtual agent 22, stored in the conversation DB. In this embodiment, the conversation unit 140B can instruct the artificial intelligence system 12B to generate conversation content by referring to the conversation DB held by the storage 124A.

[0057] Next, we will explain the handover of conversation from the main agent 22A to the specialist agent 22B by the artificial intelligence system 12A and the management LLM 144A.

[0058] As described above, according to the artificial intelligence system 12A of this embodiment, the artificial intelligence system 12A actively determines whether to hand over the generation of conversation content with the user from the main agent 22A to the specialist agent 22B, and the management LLM 144A selects the specialist agent 22B. As a result, the user does not need to select the specialist agent 22B themselves. Since the management LLM 144A uses artificial intelligence to select the specialist agent 22B, it becomes possible to hand over the conversation to a more appropriate specialist agent 22B.

[0059] More specifically, while the user and the main agent 22A are conversing, the artificial intelligence system 12A determines from the conversation history of the user's statements whether the user intends to have a specialized conversation or seek advice, and uses this as a trigger to call the specialist agent 22B.

[0060] In this embodiment, the artificial intelligence system 12A determines, as an example, to hand over the conversation to the specialist agent 22B in any of the following three cases.

[0061] Pattern 1: When the user's utterance is technical. Pattern 2: When the system is unable to respond to the user's utterance. Pattern 3: When the system is unable to respond to the user's utterance due to the attributes of the main agent 22A.

[0062] Pattern 1 is the case when, for example, the user makes an utterance that asks the main agent 22A a question about a specialized topic. For example, the user might ask, "I want to pay my taxes, but how do I do it?" In cases like this, where the user asks a highly specialized question, it is assumed that the main agent 22A will not be able to respond to the user's question, so it is better to hand over the conversation to the specialist agent 22B.

[0063] Pattern 2 is, for example, when a user asks the same question multiple times. Pattern 2 occurs when, for instance, the user asks, "What are the winter fashion trends this year?", and after the main agent 22A answers this question, the user asks, "Isn't that last year's trend?", and after the main agent 22A answers this question, the user says, "I saw on TV that XX is going to be popular this year." In such cases, it is assumed that the main agent 22A is unable to respond to the user's question, so it is better to hand it over to the specialist agent 22B.

[0064] Pattern 3 is, as an example, when the user's utterance relates to content pre-set according to the attributes of the main agent 22A (hereinafter referred to as "unanswerable content"). The attributes of the main agent 22A include, for example, the properties and characteristics of the business operator that operates the main agent 22A, or, if the main agent 22A is modeled after a specific character, the properties and characteristics of the character.

[0065] Pattern 3, which refers to content that cannot be answered, includes, for example, cases where it is difficult for Main Agent 22A to give a clear answer to a user's question due to the characteristics of the business of Business A, which operates Main Agent 22A, or questions that Business A does not want Business A to answer. More specifically, for example, if a user says something like, "Please tell me your recommendations for XX home appliances," even though Business A, which operates Main Agent 22A, also sells home appliances, Main Agent 22A may answer with products from other home appliance manufacturers.

[0066] Furthermore, questions concerning war, ideology, politics, etc., which the operator of Main Agent 22A, Company A wishes to avoid answering, will also be deemed unanswerable. Additionally, if Main Agent 22A is modeled after a specific character, questions from users that the character should not be answering will also be deemed unanswerable.

[0067] In order to determine whether a conversation with the user falls under the above patterns, the LLM of the artificial intelligence system 12A is trained, or prompts are input to the artificial intelligence system 12A. In addition, to handle pattern 3, for example, RAG (Retrieval Augmented Generation) may be used. For this reason, a list indicating unanswerable content is stored in storage 124A, and the artificial intelligence system 12A refers to this list each time it converses with the user.

[0068] The artificial intelligence system 12A then outputs a trigger to call the control LLM 144A if the user's utterance is one of the patterns 1 to 3 described above. The artificial intelligence system 12A may also decide to hand over the conversation to the specialist agent 22B in cases of other patterns, not limited to patterns 1 to 3.

[0069] When the dispatching LLM 144A receives this trigger, it selects a specialist agent 22B (artificial intelligence system 12B) to take over the conversation. As described above, the dispatching LLM 144A uses artificial intelligence (LLM) to select the specialist agent 22B to take over the conversation with the user. In other words, the dispatching LLM 144A in this embodiment is, as an example, a trained model that has been machine-learned to handle the user's utterances and the specialist agent 22B that responds to them.

[0070] Furthermore, the multiple specialized agents 22B that are selected by the dispatching LLM 144A may be classified, for example, by specialty field. One classification may be further divided into multiple classifications. The dispatching LLM 144A selects a specialized agent 22B from the multiple classified specialized agents 22B that corresponds to the user's question asked to the main agent 22A. The dispatching LLM 144A selects one specialized agent 22B from among the multiple classified specialized agents 22B that is appropriate for the user's utterance.

[0071] After the dispatcher LLM 144A selects specialist agent 22B, main agent 22A asks the user whether they want to hand over the conversation to specialist agent 22B. If the user requests to hand over to specialist agent 22B, the conversation with the user is handed over from main agent 22A to specialist agent 22B. This allows the user to decide whether or not to hand over to specialist agent 22B.

[0072] The main agent 22A's inquiry regarding taking over the conversation from the user would be as follows:

[0073] First, the main agent 22A asks the user whether they want to be transferred to the specialist agent 22B, for example, by saying, "There's someone who knows more about this, would you like me to call them?" If the user replies, "Yes, I'd like to talk to them now," the main agent 22A responds, "Okay! I'll pass on the details of your consultation and any necessary information," explaining to the user that the details of their consultation and user information will be passed on in advance. After that, the conversation with the user is handed over to the specialist agent 22B, and the user and the specialist agent 22B then converse.

[0074] Furthermore, when the LLM 144A selects a specialist agent 22B, it sets a flag indicating the selected specialist agent 22B. In the virtual agent conversation system 10 of this embodiment, as an example, with the LLM 144A holding the flag, the main agent 22A asks the user whether or not to hand over the conversation to the specialist agent 22B. If the user requests to hand over to the specialist agent 22B, the conversation with the user is handed over to the specialist agent 22B that has the flag set. This allows for a smooth handover to the specialist agent 22B when a handover request is made by the user.

[0075] Furthermore, the transfer of a user conversation to a flagged specialist agent 22B is not limited to cases where the user requests the transfer to specialist agent 22B. The transfer of a user conversation to a flagged specialist agent 22B may occur under other conditions. In other words, the transfer of the user conversation to a flagged specialist agent 22B occurs when it is determined that the generation of the conversation content should be transferred to specialist agent 22B.

[0076] Figure 5 is a flowchart showing the flow of conversation processing when the conversation is handed over from the main agent 22A to the specialist agent 22B in this embodiment.

[0077] First, in step 100, a conversation takes place between the user and the main agent 22A. During this conversation, the artificial intelligence system 12A determines whether or not to hand over the generation of the conversation content from the main agent 22A to the specialist agent 22B.

[0078] In step 102, if the artificial intelligence system 12A determines that it is necessary to hand over the conversation to the specialist agent 22B, it proceeds to step 106; otherwise, it proceeds to step 104.

[0079] In step 104, the conversation unit 140A determines whether the conversation between the user and the main agent 22A has ended. If the determination is positive, the conversation process ends; otherwise, the process returns to step 100.

[0080] In step 106, the operator LLM 144A selects a specialist agent 22B to take over the conversation. The operator LLM 144A sets a flag indicating the selected specialist agent 22B.

[0081] In the next step 108, the main agent 22A asks the user whether it is necessary to hand over to the specialist agent 22B during the conversation between the main agent 22A and the user.

[0082] In the next step 110, the artificial intelligence system 12A determines whether the user has requested that the conversation be handed over to the specialist agent 22B. If the determination is positive, the system proceeds to step 112; otherwise, it returns to step 100. The user may not request a handover to the specialist agent 22B if, for example, the user is satisfied with the conversation with the main agent 22A up to that point.

[0083] In step 112, the conversation with the user is handed over to the flagged specialist agent 22B, and the conversation between the user and the main agent 22A is terminated.

[0084] Although this disclosure has been described above using the embodiments described above, the technical scope of this disclosure is not limited to the scope described in the embodiments above. Various modifications or improvements can be made to the embodiments without departing from the gist of the disclosure, and such modified or improved forms are also included in the technical scope of this disclosure.

[0085] This disclosure may be applied to a system consisting of multiple devices or to a single device. Furthermore, this disclosure is also applicable when an information processing program that implements the functions of this embodiment is supplied to a system or device and executed by an internal processor. The technical scope of this disclosure includes programs installed on a computer to implement the functions of this disclosure on a computer, or the medium on which such programs are stored, a server that downloads such programs, and a processor that executes such programs. In particular, at least a non-transitory computer-readable medium containing a program that causes a computer to execute the processing steps included in this embodiment described above is included in the technical scope of this disclosure.

[0086] For example, in the above embodiment, the conversation unit 140, the management LLM 144A, and the memory control unit 142A were described as being provided on a server 16, but this disclosure is not limited thereto. For example, at least one of the conversation unit 140, the management LLM 144A, and the memory control unit 142A may be provided on a user terminal 14. Alternatively, the conversation unit 140, the management LLM 144A, and the memory control unit 142A may each be provided on different servers 16. Furthermore, the server 16 or the terminal device 14 or the server 16 may have the functions of an artificial intelligence system 12 (LLM).

[0087] Furthermore, although the above embodiment describes a configuration in which the distribution LLM 144A is provided on the distribution server 16A, this disclosure is not limited thereto. The distribution LLM 144A may also be provided as a function of the artificial intelligence system 12A, which is an LLM. For example, the distribution LLM 144 can be constructed by making the prompt to the artificial intelligence system 12A two-stage. Specifically, in the first stage, the user obtains an answer regarding the content and area of ​​support needed from the user's utterance. In the second stage, the LLM is instructed to select an appropriate agent from the obtained answer and a prompt that describes the group of specialized agents the user possesses. Regarding the group of specialized agents, the prompt may include weightings such as which agents are more likely to be selected or less likely to be selected, based on the intention of the business operator providing the group of specialized agents to the user.

[0088] Furthermore, although the above embodiment describes a configuration in which the main agent 22A and the specialized agent 22B are operated by different operators, this disclosure is not limited thereto. The main agent 22A and the specialized agent 22B may be operated by the same operator.

[0089] Furthermore, although the above embodiment describes a configuration in which the user terminal 14 has a speech recognition function, detects speech information uttered by the user, and accepts the detected speech as input information, this disclosure is not limited to this. For example, the user terminal 14 may accept text information, image information, and speech information entered by the user as input information.

[0090] The following additional notes are provided regarding the embodiments described above.

[0091] [Note 1] The information processing system described in Note 1 is an information processing system that engages in conversation with a user through a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user. The system comprises a conversation unit that instructs a task processing unit, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing unit and presents it to the user; and a selection unit that, when the task processing unit corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to the specialized virtual agents, uses artificial intelligence to select the specialized virtual agent to take over the conversation with the user.

[0092] In this configuration, the task processing unit proactively determines whether to hand over the generation of user conversation content from the main virtual agent to a specialized agent, and the selection unit selects the specialized virtual agent. This eliminates the need for the user to select the specialized virtual agent themselves. Since the selection unit uses artificial intelligence to select the specialized virtual agent, it is possible to select a more appropriate specialized virtual agent.

[0093] [Note 2] In the information processing system of Note 1, the task processing device corresponding to the main virtual agent may decide to hand over the conversation to the specialized virtual agent in any of the following cases: first, when the user's utterance is specialized; second, when the main virtual agent is unable to respond to the user's utterance; or third, when the main virtual agent is unable to respond to the user's utterance due to its attributes.

[0094] This configuration allows for the user to speak and then, at the appropriate time, take over to a specialized virtual agent.

[0095] [Note 3] In the information processing system described in Note 2, in the case of the second example above, it is acceptable even if the user asks the same question multiple times.

[0096] This configuration makes it easy to determine if the system is unable to handle task processing instructions based on user utterances.

[0097] [Note 4] In the information processing system described in Note 2 or Note 3, in the third case, the user's utterance may also relate to content pre-configured according to the attributes of the main virtual agent.

[0098] This configuration makes it easy to determine if a user's utterance cannot be handled due to the attributes of the main virtual agent.

[0099] [Note 5] In any of the information processing systems described in Notes 1 to 4, the main virtual agent may, after the selection unit has selected the specialized virtual agent, inquire with the user whether or not to transfer the conversation to the specialized virtual agent, and if the user requests to transfer the conversation to the specialized virtual agent, the conversation with the user may be transferred from the main virtual agent to the specialized virtual agent.

[0100] This configuration allows the user to decide whether or not to hand over the task to a specialized virtual agent.

[0101] [Note 6] In any of the information processing systems described in Notes 1 to 5, if the selection unit selects the specialized virtual agent, it may set a flag indicating the selected specialized virtual agent, and if it determines that the generation of the conversation content should be handed over to the specialized virtual agent, the conversation with the user may be handed over to the specialized virtual agent to which the flag has been set.

[0102] This configuration allows for a smooth handover of specialized virtual agents when a user requests a transfer of responsibilities.

[0103] [Note 7] The information processing device in Note 7 is an information processing device that engages in conversation with a user using a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user. The device comprises a conversation unit that instructs a task processing device, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing device and presents it to the user. The task processing device corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to the specialized virtual agents, and a selection unit that uses artificial intelligence to select the specialized virtual agent to take over the conversation with the user.

[0104] 10 Virtual agent conversation system 12 Artificial intelligence system (task processing unit) 14 User terminal (information processing unit) 16 Server (information processing unit) 22A Main agent (principal virtual agent) 22B Specialist agent (specialized virtual agent) 140 Conversation unit 144A LLM (selection unit)

Claims

1. An information processing system that engages in conversation with a user through a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user, comprising: a conversation unit that instructs a task processing unit, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing unit and presents it to the user; and a selection unit that, when the task processing unit corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to one of the specialized virtual agents, uses artificial intelligence to select the specialized virtual agent to take over the conversation with the user.

2. The information processing system according to claim 1, wherein the task processing device corresponding to the main virtual agent determines to hand over the conversation to the specialized virtual agent in any of the following cases: first, when the user's utterance is specialized; second, when the main virtual agent is unable to respond to the user's utterance; or third, when the main virtual agent is unable to respond to the user's utterance due to the attributes of the main virtual agent.

3. The information processing system according to claim 2, wherein the second case is when the user asks the same question multiple times.

4. The information processing system according to claim 2 or 3, wherein in the third case, the utterance from the user relates to content pre-configured according to the attributes of the main virtual agent.

5. The information processing system according to claim 1 or 2, wherein the main virtual agent, after the selection unit has selected the specialized virtual agent, inquires with the user whether or not to hand over the conversation to the specialized virtual agent, and if the user requests to hand over to the specialized virtual agent, the conversation with the user is handed over from the main virtual agent to the specialized virtual agent.

6. The information processing system according to claim 1 or 2, wherein the selection unit, upon selecting the specialized virtual agent, sets a flag indicating the selected specialized virtual agent, and if it determines that the generation of the conversation content should be handed over to the specialized virtual agent, the conversation with the user is handed over to the specialized virtual agent for which the flag has been set.

7. An information processing device that engages in conversation with a user through a main virtual agent, which is a virtual agent that engages in general-purpose conversations with the user, and a plurality of different specialized virtual agents, which are virtual agents that engage in specialized conversations with the user, comprising: a conversation unit that instructs a task processing device, which processes tasks using artificial intelligence in response to task processing instructions, to generate conversation content for the virtual agents to the user, and acquires the conversation content generated by the task processing device and presents it to the user; and a selection unit that, when the task processing device corresponding to the main virtual agent determines that the generation of conversation content based on the user's utterances should be handed over to one of the specialized virtual agents, uses artificial intelligence to select one of the specialized virtual agents to take over the conversation with the user.

Images