Information processing method, computer and program

Abstract

To appropriately switch an agent capable of performing interaction associated with a user.SOLUTION: An information processing method according to an aspect of the present disclosure is a method executed by a first computer in a dialogue system capable of interacting with a user, the method including, while connecting to a second computer on a network via a communication device mounted in the vehicle or a communication terminal capable of communicating with the first computer, using a microphone and a speaker mounted in the vehicle to interface a first dialogue process with the user by a first AI agent implemented in the second computer, and recording a first conversation log related to a conversation content between the first AI agent and the user in a memory in the first computer, when the connection to the second computer is disconnected, the second interactive processing with the user is executed based on the first conversation log by the second AI agent implemented in the communication terminal or the first computer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, a computer, and a program. [Background technology]

[0002] Conventionally, a technology has been known for calling a voice dialogue agent having the functions desired by a user when the services of multiple voice dialogue agents are available in a computer such as an on-board device installed in a vehicle (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-117302 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in a dialogue agent system that can interact with users, from the perspective of computational complexity, a voice dialogue agent (AI agent inference processing) that can engage in dialogue linked to the user may be implemented on the cloud side, and an edge device such as an in-vehicle computer may act as the user interface for that AI agent. In such dialogue agent systems, situations may arise where communication with the called AI agent is not possible due to, for example, an internet disconnection.

[0005] One of the problems that the present disclosure aims to solve is how to appropriately switch between agents that are linked to a user and that can have a conversation. [Means for solving the problem]

[0006] An information processing method according to one embodiment of the present disclosure is an information processing method executed by a first computer mounted on a vehicle in an interactive agent system capable of interacting with a user, wherein the method connects to a second computer on a network via a communicator mounted on the vehicle or a communication terminal capable of communicating with the first computer, and uses a microphone and speaker mounted on the vehicle to interface a first interaction process with the user by a first AI agent implemented on the second computer, records a first conversation log regarding the content of the conversation between the first AI agent and the user in memory within the first computer, and when the connection to the second computer is disconnected, causes the communication terminal or a second AI agent implemented in the first computer to execute a second interaction process with the user based on the first conversation log. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to appropriately switch between agents that are linked to a user and with whom a conversation can be conducted. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of a dialogue agent system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the interactive agent system according to the embodiment. [Figure 3] FIG. 3 is a sequence diagram showing an example of API exchanges between a client (information terminal or vehicle) and a server (cloud) via a wide area communication network in the interactive agent system according to the embodiment. [Figure 4] FIG. 4 is a sequence diagram showing an example of a processing flow in which a client downloads and uses agent attribute information in the interactive agent system according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a functional configuration of the dialogue agent system according to the embodiment. [Figure 6] FIG. 6 is a sequence diagram illustrating an example of a processing flow for detecting a network disconnection and switching the server that executes the inference processing using the AI ​​model to the edge side in the interactive agent system according to the embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of a processing flow when the client side (UI side) changes the agent in the interactive agent system according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a data structure of a task status notification in the interactive agent system according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a processing flow when the server side (AI execution side) changes the agent in the interactive agent system according to the embodiment. [Figure 10] FIG. 10 is a sequence diagram showing an example of the flow of processing that is handed over from the vehicle agent to the cloud agent when the network is restored in the interactive agent system according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a notification in the dialogue agent system according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a process flow for managing a usage log of a partner agent in the interactive agent system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, with reference to the drawings, embodiments of an information processing method, a computer, a program, a communication terminal, and a dialogue agent system according to the present disclosure will be described.

[0010] In recent years, technological development of AI agents using large-scale language models (LLMs) has progressed. AI agents possess short-term and long-term memory (user usage logs or portions of their contents) and can autonomously communicate with external applications and web services over a network, launching and operating other applications and web services. This allows AI agents to set or update goals through text or voice communication with users (instructions to the AI ​​are also called prompts), autonomously generate tasks necessary to achieve those goals, and execute the information processing of the generated tasks sequentially, either autonomously or through communication with the user, to achieve their final goal. AI is a computer system, or software program, that can process information input or output not only in a single modal (data format) such as text, but also in a combination of multiple modalities such as voice and images. This is called multimodal AI.

[0011] AI agents can be given specializations and characteristics depending on the databases they refer to when processing information and the algorithms they use to generate tasks. This allows them to be implemented as highly specialized agents specializing in specific functions, for example. On the other hand, AI agents can also be implemented as personalized AI agents that get close to individual users by learning the preferences, biometric information, and past behavioral history of the users they communicate with, and by accessing databases that store such personal user data (hereinafter referred to as user attribute information). The former type of AI agent is sometimes called a specialized agent, and the latter type of AI agent is sometimes called a partner-type agent.

[0012] Partner-type agents are thought to be particularly effective partners in mobility spaces. This is because when a user travels by vehicle to a place outside their usual range of activity and has a new (or unusual) experience, a partner-type agent can act as an appropriate navigator that is attuned to the user's individuality. For example, when navigating a vehicle's route, a partner-type agent can make selections or suggestions that reflect the user's preferences. Examples of user preferences include whether the user prefers the shortest route, roads that are easy to drive on with separated sidewalks, or whether they like to stop by tourist spots.

[0013] The inventors have considered a series of user experiences regarding the use of vehicles and AI agents. The first scenario envisioned was a scenario in which an AI agent, with AI inference processing running on a server, is accessed from the vehicle's onboard system via a computer network. In such a scenario, there is a need to seamlessly continue interaction with the AI ​​agent even if the vehicle enters an area with poor communication conditions and the network becomes unavailable. However, when AI inference processing is performed on a server, there are challenges in terms of customer value and implementation, such as how to appropriately continue communication with the user and ongoing tasks when the network becomes disconnected or unstable.

[0014] First, we will explain the definition of an AI agent (Artificial Intelligence Agent). An AI agent is software or a mechanism for achieving a predefined goal. An AI agent is designed to autonomously generate, select, and execute actions to achieve the goal based on communication with the user, the situation in the user's surrounding space S, and external information acquired via a network regarding the interaction with the user. Communication with the user includes all means for conveying the user's emotions, will, and thoughts. For example, this includes one or more of the following: a voice user interface (VUI) that expresses intentions through audio information such as conversation, pauses before speaking, and tone of voice; a graphical user interface (GUI) that expresses intentions through visual information such as text and symbols; a physical user interface (PUI) that expresses intentions through physical operations such as buttons and switches; and a natural user interface (NUI) that expresses intentions through bodily actions such as facial expressions, gaze, posture, and gestures. In this embodiment, an AI agent is also simply referred to as an agent. To facilitate communication with the user, an agent may be represented as a unique character, in which case it has the character's attribute information. The agent's attribute information includes, for example, information about one or more of the character's appearance, body, clothing, accessories, gestures, facial expressions, voice, personality, preferences, habits, knowledge, and experience (records of past interactions with users).

[0015] FIG. 1 is a diagram illustrating an example of the overall configuration of a dialogue agent system according to an embodiment. In this disclosure, an agent system that communicates with a user as described above is referred to as a "dialogue agent," a "dialogue agent system," or simply an "agent." The dialogue agent system according to the embodiment is an example of a dialogue agent system that can dialogue with a user. In this disclosure, communication between a user and an agent system is not limited to dialogue, but may also include the other types of communication described above. In this disclosure, for ease of reading, terms such as "dialogue capable" may be used, but these terms may be appropriately replaced with "communication capable."

[0016] As shown in Fig. 1, the interactive agent system according to the embodiment includes at least one information terminal 1, at least one vehicle 2, and at least one cloud 3. The information terminal 1, the vehicle 2, and the cloud 3 are communicably connected to each other via a network N. The network N is a wide area communication network such as the Internet. The information terminal 1 and the vehicle 2 are communicably connected to each other via the network N or short-range wireless communication C in a P2P (Peer to Peer) manner.

[0017] 1 illustrates an example of a dialogue agent system including one information terminal 1, one vehicle 2, and two clouds 3, but the number of each can be changed as appropriate. For example, the dialogue agent system may include zero or two or more information terminals 1. Similarly, the dialogue agent system may include zero or two or more vehicles 2. Similarly, the dialogue agent system may include zero, one, or three or more clouds 3.

[0018] The user's information terminal 1 stores information about an electronic key for unlocking and starting a vehicle 2.

[0019] In the dialogue agent system according to the embodiment, each agent is implemented in each of the information terminal 1, the vehicle 2, and the clouds 3a and 3b by recording a pre-trained AI model such as a large-scale language model (LLM) in each of these devices. Furthermore, the AI ​​model is used in combination with a framework (software) for operating it, thereby executing the interaction (dialogue) of each agent with the user P.

[0020] For example, the framework inputs information based on user P's utterance (or information in response to a request) into the AI ​​model. For example, the AI ​​model performs inference to output information in response to the input information. For example, the framework acquires the information output from the AI ​​model, generates a response to the user based on the acquired information, and presents the generated response to user P.

[0021] Here, recording an AI model may mean recording the structure of the AI ​​model (e.g., the structure of a neural network) and learned parameters in the memory of the device, and training an AI model may mean optimizing or determining (updating) the parameters that define the AI ​​model according to the purpose of the AI ​​model.

[0022] The smartphone agent 10 is an AI agent implemented in the information terminal 1. Specifically, the smartphone agent 10 is software that operates by running an inference process using an AI model stored in the information terminal 1, such as a smartphone. The smartphone agent 10 is an agent that acts as a daily companion for the user, and is associated with an electronic key for using the vehicle 2.

[0023] The vehicle agent 20 is an AI agent implemented in at least one computer (on-board system) installed in the vehicle 2. Specifically, the vehicle agent 20 is software that operates by running an inference process using an AI model recorded in the on-board system of the vehicle 2.

[0024] The cloud agent 30a is an AI agent implemented in at least one computer (server) that realizes the cloud 3a. The other cloud agent 30b is an AI agent implemented in at least one computer that realizes the other cloud 3b. Specifically, each cloud agent 30 operates in a server-client configuration. The cloud agent 30 performs inference processing in response to a request received on the cloud 3 (server), generates a response, and returns it to the client. The cloud agent 30 and smartphone agent 10 are agents that can be used from any device (client) used by user P. They are agents (partner-type agents) that can interact with user P by working together to complete daily tasks and gaining a deep understanding of user P's behavior, thoughts, and experiences. Here, the cloud agent 30 according to the embodiment is an example of a first AI agent implemented on a second computer.

[0025] The interactive agent system according to the embodiment may switch the services provided by the vehicle agent 20 depending on the user's location. For example, the vehicle agent 20 is an agent available in an in-vehicle system, developed primarily for interacting with a user P while in the vehicle, such as providing driving assistance, route guidance, or answering questions about the vehicle 2. The vehicle agent 20 has a feature that it changes the response / scope of response to requests between a user P in the vehicle and a user P communicating remotely from a remote location. For example, by changing the services and capabilities it can provide depending on whether the user P is inside or outside the vehicle, it is possible to reduce the risk of attacks against the in-vehicle system via a network. For example, if the in-vehicle system of the vehicle 2 on which the vehicle agent 20 is running directly interacts with the user P through video, audio, or physical operations (e.g., touching the screen) or if the user P's information terminal 1 is within a predetermined distance from the vehicle 2, it may be determined that the user is inside the vehicle. For example, if the in-vehicle system interacts with the user indirectly via a network with another computer system, such as a smartphone, or if the device serving as the user P's user interface is not the in-vehicle system of the vehicle 2, it may be determined that the user P is outside the vehicle.

[0026] To give one specific example, user P can use information terminal 1 to query vehicle agent 20 via the network from a location far away from vehicle 2 to find out the interior temperature of vehicle 2. However, in this case, user P cannot activate the air conditioner via vehicle agent 20 to adjust the interior temperature of vehicle 2. User P can activate the air conditioner of vehicle 2 via vehicle agent 20 only when user P is inside the vehicle or when the in-vehicle system of vehicle 2 is using it as a user interface device that communicates with user P.

[0027] Furthermore, the smartphone agent 10 and vehicle agent 20 can operate using only the computer resources and data of the device even in situations where there is no Internet connection, such as when user P drives vehicle 2 through an area with poor reception. On the other hand, when any agent has an Internet connection, it searches for and obtains the necessary information from the Internet and generates a response.

[0028] Furthermore, while driving vehicle 2, user P communicates with cloud agent 30 running on cloud 3 (server) using the vehicle 2's in-vehicle system as a client via video, audio, text information, etc. In this case, other agents (vehicle agent 20 or smartphone agent 10) or the client terminal used by the user (or an app running there) may explicitly participate in the interaction between user P and cloud agent 30, or may monitor the interaction without participating, or may not monitor it at all.

[0029] In some cases, the AI ​​agent is operated by an information terminal 1 implemented as a mobile device such as a smartphone. However, mobile devices such as smartphones may have poor reception performance compared to, for example, the vehicle 2, and may be prone to losing connection to the network N (Internet connection). Therefore, when the Internet connection is about to be lost, an in-vehicle system in the vehicle 2 with better radio wave reception performance may connect to the Internet, and the information terminal 1 may be connected by tethering, thereby making it less likely that the information terminal 1 will lose its connection to the Internet. Tethering by the vehicle 2 may be performed automatically when the vehicle 2 enters an area with low radio wave strength, by referring to a radio wave strength map of the carrier with which the user P has a contract. Alternatively, tethering may be performed constantly for communication devices such as the information terminal 1 brought into the vehicle 2. Alternatively, when the radio wave reception strength becomes low, the information terminal 1 may request tethering from the vehicle 2, and tethering may be initiated.

[0030] The electronic key stored in the information terminal 1 is used to authenticate unlocking and starting the vehicle 2. The model number of the information terminal 1 and information on the contracted carrier may be stored in association with the electronic key information. When the vehicle 2 is started using the electronic key of the information terminal 1, the model number of the smartphone and the contracted carrier information may be notified to the vehicle 2. Furthermore, upon acquiring the carrier information, the vehicle 2 may update the radio wave intensity map and store it in the memory of the vehicle 2.

[0031] FIG. 2 is a diagram illustrating an example of a hardware configuration of the interactive agent system according to the embodiment.

[0032] The information terminal 1 includes at least a sensor 101, a UI unit 103, a calculation unit 104, a memory 105, and a communication unit .

[0033] The sensor 101 acquires at least one of video information, audio information, and physical quantities of the surrounding environment.

[0034] The UI unit 103 receives button presses, touch operations, etc. from the user. The UI unit 103 has a display that displays a GUI, and a speaker and microphone that input and output the VUI.

[0035] The calculation unit 104 performs information processing such as various calculations and information drawing performed in the information terminal 1. The calculation unit 104 is an example of a processor that causes the smartphone agent 10 (for example, a partner-type agent) to execute the generation processing.

[0036] Memory 105 stores data and files used by calculation unit 104. Memory 105 is an example of a memory that records at least one of access information including the address of an agent to be called (summoned) to information terminal 1, agent attribute information related to each agent, and an AI model that executes inference processing on information terminal 1. Memory 105 may also record a usage log of each agent by a user. Memory 105 may also record a usage log related to at least one agent available to a user. Memory 105 may also manage (record) information related to at least one electronic key available to a user.

[0037] The communication unit 106 communicates via the network N with other computers on the communication network.

[0038] When an application that executes an AI agent is installed in information terminal 1, the program and necessary data are recorded in memory 105 of information terminal 1, and the program is executed by calculation unit 104. This application may be an AI agent that executes inference processing on information terminal 1, or may be an AI agent that executes inference processing on a computer other than information terminal 1 via a network.

[0039] In this embodiment, a smartphone is used as an example of the information terminal 1, but the information terminal 1 is not limited to this. The information terminal 1 may be in the form of a wristwatch-type smart watch, smart glasses-type eyeglasses, smart earphones worn on the ears, a smart ring-type finger ring, a smart speaker operated by voice, or a robot with movable parts. Here, the information terminal 1 according to the embodiment is an example of a communication terminal capable of communicating with a first computer.

[0040] The vehicle 2 has at least a movable unit 201 , a light 202 , a sensor 203 , a UI unit 204 , a key control unit 205 , a calculation unit 206 , a memory 207 and a communication unit 208 .

[0041] The movable part 201 moves the vehicle 2 and moves devices (seats, etc.) in the vehicle interior space.

[0042] The lighting 202 illuminates the surroundings of the vehicle 2 and the interior of the vehicle.

[0043] The sensor 203 detects the positions and states of people and cars around the vehicle 2, as well as people and objects inside the vehicle.

[0044] The UI unit 204 provides various types of video and audio information to the occupants (passengers) of the vehicle 2, and also receives inputs such as touch operations and voice operations from the occupants.

[0045] The key control unit 205 authenticates the key to be used for unlocking and controls the locking / unlocking of the doors of the vehicle 2.

[0046] The calculation unit 206 executes various processes related to the vehicle's core system and vehicle functions. The calculation unit 206 is an example of a processor that stores an AI model in the memory 207 and executes software of the vehicle agent 20 that can be executed by the in-vehicle system.

[0047] The memory 207 stores data and files used by the calculation unit 206. The memory 207 is an example of a memory that records at least one of access information including the address of an agent to be called (summoned) on the in-vehicle system, agent attribute information on each agent, and an AI model that executes inference processing on the in-vehicle system. The memory 207 may also record a usage log of each agent by a user. The memory 207 may also record a usage log on at least one agent available to a user. The memory 207 may also record various data including a program for the vehicle core system and a key management database.

[0048] The communication unit 208 performs wireless communication with external devices via a network N or short-range wireless communication C. Here, the communication unit 208 is an example of a communication device mounted on a vehicle.

[0049] In this embodiment, the UI unit 204, the key control unit 205, the calculation unit 206, the memory 207, and the communication unit 208 are realized by an in-vehicle system mounted on the vehicle 2. Here, the memory 207 is an example of a memory that stores a program for causing the calculation unit 206 to execute predetermined information processing. This predetermined information processing includes processing executed by the acquisition unit 21, the setting unit 22, the execution unit 23, and the handover unit 24 (see FIG. 5 ), which will be described later.

[0050] The information terminal 1, the vehicle 2, and the cloud 3 may communicate with each other via other communication means than the network N. For example, the unlocking process between the vehicle 2 and the information terminal 1 may be performed using short-range wireless communication C.

[0051] The cloud 3 includes at least a communication unit 301 , a memory 302 , and a calculation unit 303 .

[0052] The communication unit 301 communicates with other computers on a network N (wide area network).

[0053] The memory 302 stores data and files used by the calculation unit 303. The memory 302 stores information about the vehicle 2 and the user, a management program for the information, and the like.

[0054] The calculation unit 303 performs various types of data processing and is an example of a processor that causes the cloud agent 30 (for example, a partner-type agent) to execute the generation process.

[0055] FIG. 3 is a sequence diagram showing an example of API exchanges between a client (information terminal 1 or vehicle 2) and a server (cloud 3) via a network N in the interactive agent system according to the embodiment.

[0056] First, the premise of the dialogue agent system according to this embodiment will be explained. Due to the limited computational resources required to run large-scale language models in current AI, in many cases, an information terminal 1 (client) such as a smartphone is used as a UI terminal. A user inputs questions or requests via text or voice into an app or web browser on the information terminal 1. The app or web browser on the information terminal 1 then transmits the input data to the cloud 3 (such as an address indicated by a URL (API endpoint) specified for using the AI ​​model) via an API or HTTP / HTTPS protocol. Hereinafter, the computer system that actually executes the inference process of the AI ​​model may also be simply referred to as a server.

[0057] Here, at least one computer that implements a client according to an embodiment is an example of a first computer, and at least one computer that implements a server according to an embodiment is an example of a second computer.

[0058] The API (Application Programming Interface) and HTTP / HTTPS protocol are rules for communication between the app or web browser on information terminal 1 and the web app on cloud 3, and are responsible for exchanging data sent and received between them in a specified format (e.g., HTTPS request / response). After receiving a request, cloud 3 returns the response to information terminal 1 in a specified format (e.g., HTTPS response). The API and HTTP / HTTPS protocol are responsible for authentication to ensure security, and for sending and receiving requests and responses. Cloud 3 (server) processes the data and generates a response. Cloud 3 processes requests using vast computing resources and large amounts of data. Inference processing using an AI model is performed on cloud 3, where data processing such as image processing, audio processing, data processing, and natural language processing is performed in response to the request, and the response is returned to the client via a specified data communication method. This allows interaction between the agent and the user.

[0059] In the configuration of the interactive agent system according to this embodiment, the client, which is the information terminal 1 or the in-vehicle system, is the UI unit, and the server, which is the cloud 3, is the calculation unit that processes data and generates a response, and the API / protocol can be thought of as the rules for data communication between these two different devices.

[0060] Here, the client as the UI unit may be the information terminal 1, the vehicle 2, or software running on them, and may take any form as long as it can receive requests from the user and notify a response. Also, the server as the computing unit may be the cloud 3, the information terminal 1, the vehicle 2, or software running on them, and may take any form as long as it can execute processing using an AI model in response to a request, generate a response, and send the response back. In other words, when the client and server are different devices, communication is performed via a network using a predetermined API / protocol to perform processing.

[0061] Furthermore, even when the client and server are implemented within a single terminal (device), they can communicate and process using the same predetermined API / protocol via a bus within the terminal, and the configuration of the interactive agent system according to the embodiments has a high degree of freedom in combination. Each embodiment of the present disclosure may be implemented in any form.

[0062] For example, the processing described in FIG. 3 as being executed by the information terminal 1 may be executed by an application acting as a client running on the in-vehicle system of the vehicle 2. Alternatively, the processing described as being executed by a client and a server may be executed by the UI unit 103 and the calculation unit 104, respectively, within a single information terminal 1. As long as the same predetermined API and protocol are used, the function of the UI unit (or client) in the interactive agent system may be implemented or executed in the information terminal 1 or the vehicle 2, and the function of the calculation unit (or server) that performs inference processing of the AI ​​model may be implemented or executed in the cloud 3, the information terminal 1, or the vehicle 2. Of course, these may be realized by software running on the information terminal 1, the vehicle 2, or the cloud 3. In other words, all of the disclosures in the present embodiment can be realized by any of these configurations, and may be realized by any system configuration.

[0063] For example, a client side such as information terminal 1 requests user authentication from a server side such as cloud 3 (S301). If the server side succeeds in authenticating the user (S302), it notifies the client side that the user authentication was successful (S303). Upon being notified that the user authentication was successful, the client side acquires a user request from communication (such as a conversation) with the user (S304) and sends the acquired request to the server side (S305).

[0064] Figure 3 also shows an example of an HTTPS request (Request 1) sent from the client browser or application to the server in S305. The first line of this HTTPS request indicates that the request is sent using the HTTP version 1.1 POST method to the URL "https: / / CloudAgent / {session_ID} / messages." Here, "{session_ID}" contains the session ID for this communication session. The second line contains the Authorization header, specifying the access token used for authentication with the cloud 3 (server) on which the cloud agent 30 runs in the "{access_token}" portion. The access token is a secret authentication code issued to a user or application and is used by the server to approve the request. The third line contains the Content-Type header, specifying that the data format to be sent is JSON. The following is the request body, which describes the content of the message to be sent in JSON format. Here, the message contains text data: "What's the weather in Osaka?"

[0065] The server side executes data processing in response to the request received from the client side (S306), and notifies the client side of the result as a response (S307).

[0066] Figure 3 also shows an example of an HTTPS response, "Response 1," sent from the server to the client during S307. In this HTTPS response example, the first line indicates that the request was processed successfully by returning the status code "200 OK." The second line specifies in the Content-Type header that the data format to be sent is JSON. The following is the response body, which describes the message returned by the server in JSON format. In this case, the message is text data that reads, "Sunny, then rainy."

[0067] Thereafter, the client side outputs (notifies) the notified response to the user (S308). Thereafter, the client side and the server side repeat the exchange of requests and responses in the same manner (S309 to S313).

[0068] FIG. 4 is a sequence diagram showing an example of a processing flow in which a client downloads and uses agent attribute information in the interactive agent system according to the embodiment.

[0069] First, the client side requests the server side to authenticate the user (S401). If the server side succeeds in authenticating the user (S402), it acquires the agent attribute information of the agent that the user previously designated (S403) and transmits the acquired agent attribute information to the client side (S404).

[0070] Upon receiving the agent attribute information, the information terminal 1 sets the agent attribute information (S405) and activates the agent based on the set agent attribute information (S406). This allows the user to recognize the specified agent and understand that communication, such as talking to the user, can begin based on the agent's status displayed on the UI unit 103. The activated agent then acquires a request based on the user's speech (S407) and transmits the acquired request to the cloud 3 based on a predetermined protocol / API (S408). The cloud 3 executes data processing in response to the request received from the information terminal 1 (S409) and notifies the information terminal 1 of the result as a response based on the predetermined protocol / API (S410). The cloud 3 updates the user attribute information based on the interaction with the user (S411) and updates the agent usage log (a database recording the date and time of interactions between the user and the agent, the content of the conversation, etc.) (S412). Furthermore, the information terminal 1 responds to the user via the agent displayed on the UI unit 13 in accordance with the received response (S413). Thereafter, the information terminal 1 and the cloud 3 repeatedly exchange requests and responses.

[0071] In this way, the client (first computer) connects to the server (second computer) and interfaces with the first interaction process with the user by an agent (first AI agent) implemented on the server.

[0072] User attribute information is a database that includes one or more of the following: the user's name (nickname), age, gender, career history, interests, preferences, past conversation history, thoughts, experiences, schedule information, frequently used or subscribed external information / services, incomplete or unresolved tasks, unique information about the device used by the user (identification information, access information), biometric information, medical history, and behavioral history (movement history).

[0073] By storing agent attribute information on the cloud 3 (server side), the agent attribute information can be displayed on any information terminal 1 (client) when accessed from that information terminal 1. The agent attribute information is a data set used to represent an agent, including, for example, one or more of the following: 3D model data (data defining the agent's appearance and 3D physique, including texture, clothing, etc.), animation data (data on facial expressions, mouth, and gestures for reproducing natural movements), voice data (data for reproducing the characteristics of the agent's vocalizations), emotion data (data for reproducing specific behavioral patterns based on emotions), and control script (control code for ensuring consistency of the agent's actions and overall behavior when responding).

[0074] In cloud 3, a usage log (user attribute information) recording interactions between the agent and the user is stored and managed while being updated continuously in memory 302. This makes it possible to search for past events with the user and provide answers or suggestions. For example, if a user asks the agent about the final status of a specific matter, the agent can refer to the conversation history in the user attribute information to find the final status of that matter and generate an answer. This type of technology is known as RAG (Retrieval-Augmented Generation), a natural language processing technology that combines information retrieval and generative modeling.

[0075] The cloud 3 may update user attribute information (interests, preferences, thoughts, etc.) obtained through interactions between the agent and the user in memory 302. In addition, by recording, managing, and updating information about the user's knowledge system and experience as user attribute information, it becomes possible to provide replies and suggestions based on the user's knowledge system (which allows interactions based on the user's knowledge in areas in which the user is knowledgeable) and experience (recorded data such as what the user has experienced in the past and events that have occurred).

[0076] The contents of the request and the response may be text data such as text chat, video / still image data, audio data, or other data that has undergone predetermined processing, or may be expressed as one of the parameters included in an API, etc. Furthermore, if various sensors are provided on the client side, such as the information terminal 1 on which an agent that interacts with the user operates, the contents of the request and the response may include modal data other than the above-mentioned text, image, and audio.

[0077] FIG. 5 is a diagram illustrating an example of a functional configuration of the dialogue agent system according to the present embodiment.

[0078] In the vehicle 2 (on-board system) according to this embodiment, the calculation unit 206 executes a program recorded in the memory 207, thereby realizing each function of the vehicle 2 (on-board system) including the acquisition unit 21, the setting unit 22, the execution unit 23, and the handover unit 24. Note that each function of the vehicle 2 (on-board system) including the acquisition unit 21, the setting unit 22, the execution unit 23, and the handover unit 24 may be realized in cooperation (linkage) with the user's information terminal 1 and the cloud 3.

[0079] The acquisition unit 21 acquires authentication information from the client. Here, the authentication information may include information on an electronic key that enables the user to use the vehicle 2.

[0080] Furthermore, if the acquisition unit 21 determines that the authentication information is valid, it acquires access information (for example, a connection address such as an API endpoint) for accessing the agent used by the user from a server (device) on which the agent is implemented or a computer on the network that is accessible by the in-vehicle system of the vehicle 2. The acquisition of access information may be performed if it is determined, based on the authentication information acquired from the client, that information about an agent available to the user is not registered in the memory 207 of the in-vehicle system.

[0081] Furthermore, if the acquisition unit 21 determines that the authentication information is valid, it may establish a connection to a server running a multimodal agent. In this case, the acquisition unit 21 acquires sensing data acquired via one or more sensors 101 provided in the information terminal 1 and one or more sensors 203 provided in the vehicle 2 while the user is using the vehicle 2. Here, the sensing data may include at least one of the user's facial expressions, gestures, emotions, biometric information, the interior environment of the vehicle 2, the surrounding conditions around the information terminal 1, the surrounding conditions around the vehicle 2, the movement state of the information terminal 1, and the driving state of the vehicle 2. Here, output data transmitted to the server based on the sensing data may include at least one of the type of data from which the sensing data was extracted, an identification code for identifying at least one of the information terminal 1, the vehicle 2, the user's state, and an event, the user's state, and a timestamp indicating the time when the event occurred. Here, the multimodal agent is not limited to the cloud agent 30 and may be, for example, the smartphone agent 10 or the vehicle agent 20.

[0082] Then, the acquiring unit 21 converts the acquired sensing data into output data of a format and type that the agent can handle, for each unit of data acquired during a predetermined period. Furthermore, when the acquiring unit 21 determines that the authentication information is valid, the acquiring unit 21 may directly acquire data format information (i.e., output data) indicating a data format and data type that the agent can handle, from the client. In this case, the acquiring unit 21 may transmit the data acquired from the client as output data as is to the server.

[0083] Here, when the agent is implemented in an information terminal 1 (server) that can communicate via a network N or short-range wireless communication C, the access information may include an address for accessing the agent in the information terminal 1 via the network N or short-range wireless communication C. Also, when the agent is implemented in a cloud 3 (server) that can communicate with the in-vehicle system of the vehicle 2 via the network N, the access information may include an address for accessing the agent in the cloud 3 via the network N.

[0084] Here, the address included in the access information may be a global address for accessing an agent implemented on a server that can communicate with the in-vehicle system via the network N, or a local address for accessing an agent implemented on a server that can communicate with the in-vehicle system without going through the network N. Connection to the information terminal 1 (server) may be performed via a common API, regardless of whether the address included in the access information is a global address or a local address. This allows the in-vehicle system to connect to both the agent on the server side and the agent on the edge side (information terminal 1 such as a smartphone) using the common API.

[0085] Furthermore, based on the access information, the acquisition unit 21 acquires agent attribute information for representing the agent's character from an agent database implemented by a server on which the AI ​​is implemented. That is, the acquisition unit 21 acquires agent attribute information including information on the agent's UI, etc. from the server. Here, the agent database may be located in another computer with which the in-vehicle system of the vehicle 2 can communicate via a network N, etc.

[0086] Furthermore, if the acquisition unit 21 determines that the authentication information is valid, it acquires access information for accessing the agent from the memory 207 or the like. Furthermore, the acquisition unit 21 outputs (transmits) agent attribute information for representing the agent's character to the setting unit 22, causing at least one of a GUI and a VUI to be set based on the agent attribute information.

[0087] The setting unit 22 sets at least one of a GUI and a VUI representing the character of the agent in the execution unit 23 based on the agent attribute information.

[0088] While connected to the server, the execution unit 23 interfaces with the agent implemented in the connected server to perform interactive processing with the user, using a microphone and speaker mounted on the vehicle 2. Here, interfacing with interactive processing with the agent means that the agent represented in the UI unit 204 communicates with the user, acquires the user's responses and requests, and conveys responses to the responses and requests to the user.

[0089] Specifically, the execution unit 23 connects to the server on which the agent is implemented based on the access information, and executes dialogue (communication) with the user using at least one of the GUI and the VUI in accordance with the communication generated by the agent. Here, the GUI may be displayed on a display included in the UI unit 204. The VUI may be input and output via a speaker and a microphone included in the UI unit 204. When the execution unit 23 receives a request from the user via the microphone of the UI unit 204 included in the in-vehicle system during a period when the in-vehicle system is unable to use the agent implemented in the server realized by an external device, the execution unit 23 may execute a response process in response to the request using the vehicle agent 20 implemented in the in-vehicle system.

[0090] The handover unit 24 records a usage log (first conversation log) relating to the content of the conversation between the agent and the user in the memory 207. Furthermore, when the network connection with the AI ​​agent implemented by an external server is cut off, the handover unit 24 causes an agent implemented in another available server to continue the dialogue processing with the user based on the usage log (first conversation log).

[0091] In the information terminal 1 according to this embodiment, the calculation unit 104 executes a program recorded in the memory 105, thereby realizing each function of the information terminal 1, including the communication unit 11 and the processing unit 12. Note that each function of the information terminal 1, including the communication unit 11 and the processing unit 12, may be realized in cooperation (linkage) with the in-vehicle system of the vehicle 2 and the cloud 3.

[0092] Furthermore, the cloud 3 according to this embodiment realizes each function of the cloud 3, including the communication unit 31 and the processing unit 32, by the calculation unit 303 executing a program recorded in the memory 302. Note that each function of the cloud 3, including the communication unit 31 and the processing unit 32, may be realized in cooperation (linkage) with the information terminal 1 or the in-vehicle system of the vehicle 2.

[0093] The communication unit 11 transmits authentication information for the user to use the vehicle 2 to the in-vehicle system of the vehicle 2. Furthermore, if the communication units 11, 31 determine that the authentication information is valid, they transmit access information for accessing the agent to the in-vehicle system of the vehicle 2. Furthermore, the communication units 11, 31 transmit agent attribute information for representing the agent's character to the in-vehicle system of the vehicle 2, and cause the in-vehicle system to set at least one of a GUI and a VUI based on the agent attribute information.

[0094] When the processing units 12 and 32 receive a request from a user from the client side, they generate a response (response content) according to the request using an implemented agent and return it to the client via the communication units 11 and 31.

[0095] 6 is a sequence diagram showing an example of a process for detecting a network disconnection and switching the server that executes the inference process using the AI ​​model to the edge server in the interactive agent system according to the embodiment. This diagram shows an example of the flow of the process for handing over from the cloud agent 30 to the vehicle agent 20.

[0096] This example illustrates a case in which a user is writing a reply to an email or social media message while communicating with a cloud agent 30 via the vehicle 2's in-vehicle system (client), and the vehicle 2 loses connection to the Internet. The remaining task is then handed over to a vehicle agent 20, one of the edge-side (client-side) agents. Here, the edge-side agent is, for example, an agent implemented on a device located in the user's surrounding space S (see Figure 1). This agent may also be an agent implemented on a device that can communicate without the Internet. In the example of Figure 1, the edge-side agent may be the vehicle agent 20, as shown in the figure, or it may be the smartphone agent 10.

[0097] The client's UI unit 204 executes a GUI or VUI (a physical representation of the cloud agent 30, which functions as a UI for the user) representing the cloud agent 30. The GUI or VUI acquires the user's request 1 (S501) and transmits the acquired request to the server according to a predetermined API / protocol (S502). The server executes data processing, including inference processing (a response generation function that serves as the brain of the cloud agent 30) using the AI ​​model of the cloud agent 30 in response to the request received from the client (S503), updates the usage log of the cloud agent 30 (S504), and notifies the client of the result as a response according to a predetermined API / protocol (S505). The client's physical representation of the cloud agent 30 responds according to the response (S506), and updates the usage log of the cloud agent 30 (S507). The client and server then repeatedly exchange requests and responses, the agent's physical representation on the client's UI unit (user communication via the GUI and / or VUI), and update the usage log (S508-S514).

[0098] For example, in the example of Fig. 6, the cloud agent 30 responds to the user's "Request 1" of "Write a reply. In a positive tone" with "Response 1" of "How about...?". Also, in the example of Fig. 6, the cloud agent 30 responds to the user's "Request 2" of "Delete..." after receiving a response according to "Response 1" with "I did it like this. What do you think?".

[0099] Here, the explanation will be continued by taking as an example a case where the client detects that the internet connection is unstable or has been disconnected and notifies the user of the internet disconnection (S515).

[0100] The client organizes the progress of the tasks that the user and the cloud agent 30 have been working on (S516), extracts candidate agents that can take over, and selects one or more agents to take over from the extracted candidate agents (S517).

[0101] Here, we will continue the explanation using the example where vehicle agent 20, which can perform inference processing using an AI model in the vehicle 2's onboard system (and can operate on the edge side without using the Internet), is selected as the agent to take over.

[0102] The client requests user authentication from the in-vehicle system (server) in which the selected vehicle agent 20 is implemented (S518). If the server succeeds in authenticating the user (S519), it acquires agent attribute information of the vehicle agent 20 (S520) and transmits the acquired agent attribute information to the client (S521).

[0103] When the client receives the agent attribute information of the vehicle agent 20 to be handed over, it sets the agent attribute information (S522), starts the vehicle agent 20 based on the set agent attribute information (S523), and notifies the user of the handover from the cloud agent 30 to the vehicle agent 20 (S524). The client also sends a request (task status notification) regarding the task handover to the server (S525). This request (task status notification) includes at least one of the progress of the current task, the history of interactions and / or a summary thereof, and information about the previous agent, and is input to the agent to be handed over.

[0104] The server executes data processing in response to the task status notification (task handover request) received from the client (S526), ​​updates the usage log of the vehicle agent 20 (S527), and notifies the client of the result as a response (S528). The client also runs a GUI or VUI (a physical representation of the vehicle agent 20, functioning as a UI for the user) that represents the vehicle agent 20, and uses it to represent the received response and notify the user (S529), and updates the usage log of the vehicle agent 20 (S530). Thereafter, the client and server repeat the exchange of requests and responses regarding the vehicle agent 20 that is the handover destination, and update the physical representation of the agent in the client's UI unit and the usage log (not shown).

[0105] For example, in the example in Figure 6, the user has not responded to "Response 2" from the cloud agent 30 before the handover, which states, "We did it this way. What do you think?" In this situation, the server's vehicle agent 20 takes over the in-progress task described in the task status notification (Request 4) received from the client, and responds with "Response 4" which states, "How about it?"

[0106] In this way, the client (first computer) connects to the server (second computer) and interfaces with the agent (first AI agent) installed on the server to perform a first dialogue with the user. The client also records a usage log (first conversation log) related to the content of the conversation between the agent installed on the server and the user in its memory. If the connection to the server becomes unstable or is disconnected, the client causes an agent (second AI agent) installed on another server to perform a second dialogue with the user based on the usage log of the previous agent.

[0107] Furthermore, when the client executes a second dialogue process with a user based on the usage log of the previous agent, after detecting a disconnection of the connection to the server executing the first dialogue process, the client identifies, based on the usage log, one or more tasks that were requested of the previous agent before the disconnection.The client then selects one agent capable of executing one or more tasks from one or more agents (second AI agents) implemented in another server.The client also inputs task status information (request) including the content and progress of the one or more tasks to the selected agent.As a result, the client executes a second dialogue process for one or more tasks based on the task status information to the selected successor agent.

[0108] As an example, this agent to be handed over is an information terminal 1 (another server or server software executed therein) that can communicate with the client (an in-vehicle system or client software executed therein) via P2P, or an agent implemented in the in-vehicle system (another server or server software executed therein). If the agent to be handed over is an information terminal 1 (another server) that can communicate with the client via P2P, the client sends a task status notification (request) to the information terminal 1.

[0109] The task status notification (task handover request information from the client to the server) described here may be historical information on interactions, extracted from a usage log that records the history of interactions between the user and the agent, at least the portion related to the most recent topic. In this case, processing such as analyzing and summarizing the interactions becomes unnecessary, which simplifies the process of generating task status notification requests on the client.

[0110] Additionally, while the agent to be handed over is executing the second dialogue process, the client also records a usage log (second conversation log) related to the content of the dialogue process in its memory. When the connection to another server (e.g., the previous server that was executing the dialogue process before the disconnection) is restored, the client transmits task status information generated based on the usage log of the agent to be handed over to the other server. As a result, while connected to the server that realizes the agent to be handed over after the recovery, the client interfaces with the user the third dialogue process by the agent to be handed over after the recovery (e.g., the first AI agent) using the microphone and speaker mounted on vehicle 2.

[0111] Furthermore, after detecting the restoration of the connection to another server (e.g., the previous server that was executing the dialogue processing before the disconnection), the client identifies, based on the usage log (second conversation log) of the agent to be taken over, one or more tasks that were requested of the previous agent (first AI agent) before the connection was disconnected or that were newly requested of the agent to be taken over during the period when the connection was disconnected.The client then generates task status information including the content and progress of the one or more tasks, sends it to a server (e.g., the first computer) that will realize the agent to be taken over after the recovery, and causes the agent to be taken over after the recovery to execute a third dialogue processing related to the one or more tasks based on the task status information (request).

[0112] In the interactive agent system according to this embodiment, an application (app) is installed in the client and executes to control the appearance and voice of the cloud agent 30 displayed by the client's UI units 103 and 204, i.e., to control at least one of the GUI and VUI, and monitors and records interactions between the user and the cloud agent 30. With this configuration, even if a situation arises in which communication between the client and the server is suddenly disabled, such as a network disconnection, the remaining tasks can be smoothly handed over to another edge-side agent, such as the vehicle agent 20, based on the history of the interaction up to that point. For example, in a configuration in which the client operates on a browser-based system, such a method cannot be adopted because the client is designed not to store the history of the interaction. On the other hand, in a configuration in which an application that controls the appearance and voice of the cloud agent 30 during interaction is executed on the client, the history of the interaction can be stored in memory or a file, and in an emergency, an unfinished task can be handed over to another agent. In other words, the agent control application running on the client not only serves to interact with the user, but also has the unprecedented advantage of being able to remember that interaction, for example, for a short period of time, and thus be able to ask another available agent to continue processing an ongoing task if there is a disconnection from the network or a problem occurs on the connected cloud side, making communication or processing impossible.

[0113] Furthermore, in the interactive agent system according to this embodiment, when communication between the client and the server is not possible, such as when the network is disconnected, candidate agents that can take over are searched for, and a successor agent is selected from the candidates. In this case, the agent selected as the successor agent is not limited to the vehicle agent 20, and the smartphone agent 10 may also be selected. The selection of the successor agent and its candidates will be described later.

[0114] Furthermore, in the interactive agent system according to this embodiment, at the time of handover, the progress status of the task up to that point is organized and notified to the edge-side agent (vehicle agent 20 in the example of FIG. 6). This task status notification is a handover document that includes one or more pieces of information from among the task status, interaction history and / or its summary, and previous agent information. This configuration allows for a smooth agent handover, improving the UX. The UX associated with agent handover will be described later.

[0115] FIG. 7 is a flowchart showing an example of a processing flow when a client (UI function provider) changes agents in the interactive agent system according to the embodiment.

[0116] The client, or software executed by the client (same below), determines whether the Internet connection is unstable or disconnected (S601). If the Internet connection is stable or not disconnected (S601: No), the flow in Figure 7 ends. On the other hand, if the Internet connection is unstable or disconnected (S601: Yes), the client organizes the current task status, such as the task progress status (information included in the task status notification in Figure 6) and access information to the agent (S602), and obtains access information to a server with which the client can communicate without going through the Internet, or to an agent executed by software executed by the server (same below) that can execute inference processing of an AI model without going through the Internet (S603).

[0117] The client selects one agent from one or more agents that can communicate and run without the Internet (S604), and accesses the selected agent (user authentication) (S605). Specifically, the client requests user authentication from the server that runs the selected agent. If the server successfully authenticates the user, it acquires agent attribute information of the agent that will take over and sends the acquired agent attribute information to the client. Furthermore, upon receiving the agent attribute information of the agent that will take over, the client sets the agent attribute information, starts the agent that will take over based on the set agent attribute information, and notifies the user that the agent has been switched over to the agent that will take over.

[0118] The client also requests the server to continue the task by sending a task status notification (request) to notify the server of the current task progress (S606). After that, the client and server repeatedly exchange requests and responses with the agent to be handed over, and update the agent's physical representation in the client's UI and usage log (not shown).

[0119] In this interactive agent system capable of interacting with a user, the cloud 3 (second computer) is capable of communicating with an in-vehicle system (first computer) mounted on the vehicle 2 and is equipped with a cloud agent 30 (first AI agent) used by the user. While connected to the in-vehicle system, the cloud agent 30 executes a first dialogue with the user via the in-vehicle system, using a microphone and speaker mounted on the vehicle 2 for conversation with the user, a camera for capturing the user's facial expressions and other non-verbal responses, and a touch panel for receiving the user's intentions. The in-vehicle system also records a usage log (first conversation log) related to the content of the conversation between the cloud agent 30 and the user in the memory 207 of the in-vehicle system of the vehicle 2. Based on the usage log, the in-vehicle system identifies one or more tasks requested of the cloud agent 30, transmits task status information (request) including the content and progress of the one or more tasks to the vehicle agent 20, which is an agent on the edge side, and causes a vehicle agent 20 (second AI agent) available to the in-vehicle system to execute a second dialogue related to the one or more tasks based on the task status information.

[0120] In the above explanation, the task is handed over to the vehicle agent 20, but of course the present disclosure is not limited to this, and a request to hand over may be issued to another AI agent (e.g., smartphone agent 10) that is available without going through the Internet.

[0121] In the above explanation, one or more tasks are identified based on the usage log, but the present disclosure is not limited to this, and a portion of the most recent data exchange in the usage log may be sent to the AI ​​agent to which the task is being handed over as task status information.

[0122] In the interactive agent system according to this embodiment, if communication with the agent currently in dialogue becomes impossible due to an unstable or lost internet connection, the client can switch agents. For example, the client searches for an agent that can communicate with the client via P2P (Peer to Peer) communication without using a network N, such as short-range wireless communication C (e.g., Wi-Fi®) or wired communication (e.g., USB cable), and that can execute inference of the AI ​​model without requiring an internet connection. The client then selects one agent from the candidate agents it has found to take over the task, and notifies the server that executes the selected agent of the task status, thereby indicating the status of the ongoing task and requesting the server to take over (continue processing).

[0123] If the agent to be handed over can be executed within the client's device, that is, if the agent to be handed over can be executed on-device, one agent to be handed over may be selected, including the client's own terminal.

[0124] FIG. 8 is a diagram illustrating an example of a data structure of a task status notification in the interactive agent system according to the embodiment.

[0125] Here, with reference to FIG. 8, an example of a task status notification sent from a client to a server when taking over a task or making a new request will be described. FIG. 8 illustrates an example of a task status notification expressed as an HTTPS request. Note that while the description here assumes that a client makes a request to a server, the present disclosure is not limited to this. An AI agent on a server running an AI model may voluntarily notify other AI agents on the Internet of the task status or make a request via an API to request a new task. The task status notification described in this disclosure can be universally used when part or all of task processing is transferred from one AI agent to another.

[0126] In the example of FIG. 8, the first line of the task status notification that notifies the task status is a request line, which indicates that the vehicle agent 20 to take over is being asked to take over the interaction identified by {session_id} that took place between the cloud agent 30 and the user. The task status notification is information that indicates the progress of the task that the user has been working on with the previous agent. This task status notification includes, for example, at least one of the following information: task overview information, information on the history and / or summary of the most recent interaction between the user and the previous agent, and access information related to the user, the previous agent, and the interaction there.

[0127] The task summary (task_status) information is an overview of the task requested by the user or set by the previous agent. The task summary information includes the task name or identification number (name), content (description), current progress (progress) showing what has been completed and what remains to be done, and information (constraints) showing the restrictions and conditions for task execution.

[0128] The history of the most recent interaction between the user and the previous agent, or its summary information, includes text data expressing the interaction (at least one of the history of when, who, and what was communicated), or its summary information (conversation_summary). This summary information may be, for example, text data that briefly includes decisions and pending matters, the user's intentions, restrictions, and limitations regarding the interaction between the user and the previous agent.

[0129] The access information (access_information) about the user, the previous agent, and their interactions includes user authentication information (user_id), the access token or API key (access_token) granted to the user or app, information identifying the interaction between the user and the previous agent (session_id), the agent's access information (agent_URL), a link to the text log showing the interaction with the agent (chat_URL), and information indicating the language setting used by the user (language_code).

[0130] Note that this information is merely an example, and the same form may be expressed in a different data type or format, and may be implemented as an API rather than being limited to the form of an HTTP request.

[0131] As an example of the text data describing the conversation mentioned above (at least one of the histories of when, who, and what was communicated), the right side of the figure shows a case where the conversation history is shared without being summarized. Here, the conversation history (conversation_history) may be written in chronological order, paired with information identifying whether the speech was made by the user or the AI ​​agent (speaker) and information indicating the content of the speech (utterance).

[0132] It should be noted that the interaction history or summary included in the task status notification described here is not limited to text data. For example, it may be audio data that records the audio of the interaction between the user and the agent, or video data that records video of the interaction. The task status notification only needs to record the most recent interaction between the user and the agent, and various data formats are possible.

[0133] FIG. 9 is a flowchart showing an example of a processing flow when a server (a computer that executes an inference process of an AI model, or software executed thereon) changes an agent in the interactive agent system according to the embodiment.

[0134] The server determines whether the usage frequency or cost exceeds a predetermined set value (S701). If the usage frequency or cost exceeds the predetermined set value (S701: Yes), the server acquires access information to agents with a sufficient usage frequency or cost (S702). After that, the flow in FIG. 9 proceeds to the processing of S707.

[0135] If the usage frequency or cost does not exceed a predetermined set value (S701: No), the server determines whether there is another agent more suitable for the task content, i.e., whether there is another agent more qualified than the agent currently executing the dialogue processing (S703). If there is another agent more suitable for the task content (S703: Yes), the server obtains access information for an agent with high performance and evaluation for the task content (S704). Then, the flow in Figure 9 proceeds to the processing of S707.

[0136] If there is no other agent more suitable for the task content (S703: No), the server determines whether the remaining battery power of the device on the server that is executing inference using the AI ​​model is less than a predetermined threshold (S705). If the remaining battery power of the executing device is equal to or greater than the predetermined threshold (S705: No), the flow in FIG. 9 ends.

[0137] If the remaining battery charge of the device in operation is less than a predetermined threshold (S705: Yes), the server acquires access information for agents in the device in operation whose remaining battery charge is equal to or greater than a predetermined value and / or for agents that are powered on (S706).

[0138] The server then organizes the current task status, such as the task progress status and access information (S707), and selects one target agent based on the acquired access information (S708). The server then requests the server executing the target agent to continue the task by sending a task status notification (request) to notify the selected target agent of the current task status, or by requesting task execution via an API (S709). This notification may be sent via the client or directly between servers. The client and server then repeatedly exchange requests and responses regarding the target agent, display the agent's physical representation in the client's UI, and update the usage log (not shown).

[0139] In this interactive agent system capable of interacting with a user, the cloud 3 (second computer) is capable of communicating with an in-vehicle system (first computer) installed in the vehicle 2 and is equipped with a cloud agent 30 (first AI agent) used by the user. While connected to the in-vehicle system, the cloud 3 (second computer) executes a first dialogue process with the user using a microphone and speaker used for conversation with the user, a camera for capturing the user's facial expressions and other non-verbal responses, and a touch panel for receiving the user's intentions. The cloud 3 also records a usage log (first conversation log) related to the content of the conversation between the cloud agent 30 and the user in the memory 302 of the cloud 3. Based on the usage log, the cloud 3 identifies one or more tasks requested of the cloud agent 30, transmits task status information (request) including the content and progress of the one or more tasks to the in-vehicle system, and causes a vehicle agent 20 (second AI agent) available to the in-vehicle system to execute a second dialogue process related to the one or more tasks based on the task status information.

[0140] In the interactive agent system according to this embodiment, handover can also be performed for reasons other than a network disconnection. Specifically, in the interactive agent system, if the cloud agent 30 determines that it cannot or should not process one or more tasks, it will hand over the tasks to another agent. For example, some or all of the tasks may be handed over to another agent due to restrictions imposed by the AI ​​model's usage contract or fees, reduction of the computational load on the cloud 3 (server side), assigning the task to an agent suited to the task, or low battery life on the edge terminal (client) running the AI ​​model. In this case, the server running the agent may notify the computer system that could potentially be the other server running the other agent of the task status and request a handover.

[0141] For example, if the frequency of usage exceeds a set value, such as exceeding the limit on the number of API calls within a specified period of time or the upper limit on the amount of input / output tokens, or if a charge is incurred based on the frequency of usage and the charge exceeds the set value, the server can hand over the interactive processing to one of the agents that can be used for free or that still has room to reach the set value.

[0142] For example, if there is another agent whose performance capability for the task content is higher or higher by a predetermined amount, the server may hand over subsequent processing to the agent with the higher performance capability. Note that the server may request the processing of a specific task from another server via an API or the like, so that the server returns the results of processing the task, thereby preventing the handover of agents.

[0143] For example, if the battery level of the device that implements the server is lower than a predetermined value, such as when the in-vehicle system is tethered via Wi-Fi (registered trademark) to have the information terminal 1 perform AI processing and the information terminal 1 is not sufficiently charged, the server can hand over to an agent running on a device with remaining battery power or a device that is powered.

[0144] FIG. 10 is a sequence diagram showing an example of the flow of processing taken over from the vehicle agent 20 to the cloud agent 30 when the network is restored in the interactive agent system according to the embodiment.

[0145] After notifying the user that the vehicle agent 20 has been handed over due to a network disconnection or the like (S801), the client sends a request regarding the task status (task status notification) to the server that executes the vehicle agent 20 that will be the handover destination (S802). The server executes data processing in accordance with the task status notification (request) received from the client or the API requesting task handover (or execution) (S803), updates the usage log of the vehicle agent 20 (S804), and notifies the client of the result as a response (S805). The client also responds to the user in accordance with the response via the vehicle agent 20 as a user interface represented by the UI unit 204 (S806), and updates the usage log of the vehicle agent 20 (S807). Thereafter, the client and server repeat the exchange of requests and responses with respect to the handover destination vehicle agent 20, the physical representation of the vehicle agent 20 on the UI unit of the client, and the updating of the usage log (S808 to S814).

[0146] For example, in the example of Figure 10, the situation is one in which the user has not responded to "Response 2" from the agent (e.g., cloud agent 30) before the handover, which is "I did it this way. What do you think?" (see Figure 6). In this situation, the vehicle agent 20 takes over the task from the cloud agent 30 based on the task status notification (Request 4 in Figure 10) received from the client, and responds as "Response 4" with "How about this?". Also, in the example of Figure 10, for example, in response to "Request 5" from the client, which is "That's OK. Send it," the vehicle agent 20 responds as "Response 5" with "Got it. I'll send it after the network is restored."

[0147] The explanation will be continued by taking as an example a case where the client subsequently detects recovery of the Internet connection (net recovery) and notifies the user of the recovery of the internet connection (S815).

[0148] The client organizes the task status of the vehicle agent 20 before recovery (S816), extracts candidate agents to take over after recovery, and selects one or more agents to take over after recovery from the extracted candidate takeover agents (S817).

[0149] Here, the explanation will be continued by taking as an example a case where the cloud agent 30 is selected as the agent to take over after recovery.

[0150] The client requests user authentication from the cloud 3 (server) in which the selected cloud agent 30 is implemented (S818). If the server succeeds in authenticating the user (S819), it acquires agent attribute information of the cloud agent 30 (S820) and transmits the acquired agent attribute information to the client side (S821).

[0151] When the client receives the agent attribute information of the cloud agent 30 that will take over after recovery, it sets the agent attribute information (S822), executes the physical expression of the cloud agent 30 using the UI unit 204 based on the set agent attribute information (S823), notifies the user of the changeover to the cloud agent 30, and also confirms with the user the execution of any uncompleted tasks (S824). In addition, the client sends a request regarding the task status (task status notification) to the server (S825).

[0152] The server executes data processing in response to the task status notification (request) received from the client or the API requesting task handover (or execution) (S826), updates the usage log of the cloud agent 30 (S827), and notifies the client of the result as a response (S828). The client also executes the physical expression of the cloud agent 30 in accordance with the received response (S829) and updates the usage log of the cloud agent 30 (S830). Thereafter, the client and server use the cloud agent 30 that will take over after recovery to repeatedly exchange requests and responses, update the physical expression of the agent on the client's UI, and update the usage log (not shown).

[0153] 10, the task status of the vehicle agent 20 before recovery is that the final draft has been completed and the only remaining task is to reply to it. In this situation, the cloud agent 30, which is the takeover destination after recovery, takes over the remaining tasks of the vehicle agent 20 based on the task status notification (Request 7) ​​received from the client side requesting a reply with the final draft, or an API describing an equivalent request, and sends a reply with the final draft that has been confirmed, responding as "Response 7" saying "The reply has been sent."

[0154] In this way, in the interactive agent system of this embodiment, when the network is restored, the client's agent control application accesses the cloud agent 30 and notifies it of the task status, and the cloud agent 30 that takes over after the recovery can carry out the unfinished tasks of the vehicle agent 20 before the takeover.

[0155] Specifically, in the interactive agent system according to this embodiment, an application that controls the agent's appearance and voice responses runs on the client device, and monitors and records the interaction between the user and the agent (vehicle agent 20 in the example of FIG. 10) before recovery. Therefore, when the network is restored, the history up to that point can be compiled, and the remaining tasks can be smoothly handed over to the agent (cloud agent 30 in the example of FIG. 10) that will take over after recovery. For example, in a configuration in which the client operates on a browser-based system, interaction history is generally not recorded, and this is not possible. On the other hand, in a configuration in which an application that controls the agent's appearance and voice conversation during a conversation is executed on the client, the interaction history can be recorded in memory or as a file, making it possible to accurately request another agent to execute an uncompleted task (or a new task) based on the past interaction history.

[0156] The usage log (second conversation log) of the agent that takes over when the network is disconnected is divided into one or more segments corresponding to one or more tasks and managed in the client's memory. The task status information may also include a summary of the segments of the usage log that correspond to tasks that should be taken over by the agent that takes over after recovery (for example, the first AI agent).

[0157] Furthermore, in the interactive agent system according to this embodiment, when the network is restored, the task status is organized and notified to the agent who will take over. This configuration allows for a smooth agent handover, improving the UX. The UX associated with agent handover will be described later.

[0158] The takeover of the agent when the network is restored may be initiated by the agent that took over when the network was disconnected (vehicle agent 20 in the example of FIG. 10). For example, when the vehicle agent 20 detects that the network has been restored, it may send a task status notification to the original agent that took over (cloud agent 30 in the example of FIG. 10) to request a takeover. This may allow the cloud agent 30 to resume interaction with the user, summarize the interaction with the vehicle agent 20, or review the history of the interaction in chronological order, and ask the user to confirm the execution of any uncompleted tasks.

[0159] FIG. 11 is a diagram illustrating an example of a notification in the dialogue agent system according to the embodiment.

[0160] Here, an example of a UX for agent handover will be described with reference to Fig. 11. Fig. 11 illustrates an example of an agent display layout.

[0161] For example, when an agent is displayed on the UI unit 204 of the vehicle 2 (in-vehicle system), the agent (first object) currently executing the interactive process is placed horizontally farther from the user P in the display area of ​​the UI unit 204. Here, placing an agent in the display area of ​​the UI unit 204 refers to displaying an object representing the character of the agent in the display area. Also, for example, information 501, 509 to be displayed to the user P (in this case, text sent to the user P, such as an email, and / or text being composed as a reply) is placed horizontally closer to the user P. Here, placing information 501, 509 in the display area of ​​the UI unit 204 refers to displaying an object (second object) containing text information intended for the user P in the display area. In other words, the normal layout is to display the information to be presented to the user P in a position closer to the user P and the agent in a position farther from the user P.

[0162] Note that information 515 (information on uncompleted tasks in this example) displayed to user P is also an object containing text information for user P, similar to 501 and 509.

[0163] 11, user P, who is speaking voices 401, 407, and 413, is sitting on the left side of the display area of ​​UI unit 204, and therefore the vehicle agent 20 and cloud agent 30 with whom he is currently interacting are displayed on the right side as seen from user P. In addition, the acoustics around user P and the audio output of the left and right speakers may be adjusted so that voices 403, 405, 409, and 411 from the agents are heard from the direction of the agent's display position. In the example of this figure, the audio output of the speaker on user P's right side may be controlled to be louder than the audio output of the speaker on the left side so that the agent's voice is heard louder from the right side of user P.

[0164] When an unstable connection to network N or a network disconnection is detected, a notification 503 such as "Network connection is unstable!" is displayed in the display area of ​​the UI unit 204, superimposed on other displays or displayed in an easily recognizable position, size, and color scheme. This notification 503 is an example of a notification to the user that the connection to network N is unstable or that the network has been disconnected.

[0165] When the internet connection is unstable and a handover to the vehicle agent 20 is notified, the previous cloud agent 30 is placed horizontally closer to the user P in the display area of ​​the UI unit 204, and the vehicle agent 20 to be handed over is placed further away. Alternatively, in accordance with the convention for representing time-series changes, the previous agent may be placed on the left side of the user P's view and the new agent on the right side. In this case, icons 505 and 507 indicating whether or not the agent is connected to the network N may be placed in the display area of ​​the UI unit 204 together with each agent. In the example of FIG. 11 , icon 505 indicating an agent for which an internet connection is available or was available is placed together with the cloud agent 30 before the internet disconnection. Furthermore, icon 507 indicating an agent for which an internet connection is unavailable or was unavailable is placed together with the vehicle agent 20 to be handed over. This icon 507 may be placed in the display area of ​​the UI unit 204 until the internet connection is restored.

[0166] For example, when an uncompleted task is created during interaction with the agent to which the task is to be taken over, information 511 indicating the number or details of the task is displayed in the display area or is notified by voice. Here, information 511 is an example of text information for user P, and is an example of information about at least one uncompleted task that has been requested by the agent but has not yet been completed. Furthermore, the display of information 511 is maintained and updated even after the agent to which the task is to be taken over begins to process the dialogue. This allows user P to easily understand how many uncompleted tasks there are while engaging in various interactions with the agent offline. Furthermore, when the last uncompleted task is performed, user P will have a good memory of the uncompleted tasks, and can easily confirm that there are no omissions in the list of uncompleted tasks.

[0167] For example, when recovery of the internet connection is detected, a notification 513 such as "Internet connection has been restored!" is displayed in the display area of ​​the UI unit 204, superimposed on other displays or displayed in an easily recognizable position, size, and color scheme. This notification 513 is an example of a notification to the user that the internet connection has been restored.

[0168] For example, after the internet is restored, the cloud agent 30 places information 515 in the display area of ​​the UI unit 204 indicating the remaining incomplete tasks, and finally confirms with the user P whether to complete them. Here, the information 515 is an example of text information for the user P, and is an example of information about at least one incomplete task that was requested by the previous agent but has not yet been completed.

[0169] If user P gives a reaction of approval, the agent executes and completes these uncompleted tasks. This reaction of approval by user P may be, for example, touching the OK button on UI unit 204, a voice utterance 413 such as "Perform the uncompleted tasks" or "Do it," or a gesture of affirmation such as nodding in response to the agent's question. The agent (cloud agent 30 in the example of FIG. 11) that takes over after recovery monitors whether all of these uncompleted tasks have been executed and may notify the user if they have all been successfully completed. This allows user P to clearly recognize that all uncompleted tasks have been completed. Furthermore, if some or all of the tasks could not be completed, the user may be notified of this, and the uncompleted tasks may be displayed in the display area or a voice notification may be continued.

[0170] FIG. 12 is a flowchart illustrating an example of a process flow for managing a usage log of a partner agent in the interactive agent system according to the embodiment.

[0171] First, the server manages a usage log (history) of interactions between the user and the agent, broken down by at least one of date and content (S901).The server also determines and stores the user's knowledge system, interests, and experiences from the interactions with the user, and records and manages a concise representation of these as user profile data (or user attribute information) in the server's memory (S902).

[0172] The server also determines whether the data size of the usage log is greater than a predetermined value (S903). If the data size of the usage log is equal to or less than the predetermined value (S903: No), the flow in FIG. 12 ends. On the other hand, if the data size of the usage log is greater than the predetermined value (S903: Yes), the server individually evaluates the future utility value of the managed usage log for at least one of the date and content (S904), and compresses, summarizes, or deletes the usage log according to the evaluation of its future utility value so that the data size is equal to or less than the predetermined value. Note that the predetermined value for the data size after compression, summarization, or deletion may be the same as or smaller than the predetermined value for data size used in the processing of S903.

[0173] The flow of FIG. 12 may be executed not only by the server but also by the client.

[0174] Thus, in the interactive agent system according to this embodiment, the usage logs of the partner-type agent are managed separately by date and content. As an example, a computer equipped with an AI agent capable of interacting with a user executes an interactive process with the AI ​​agent using the user. The computer's memory records the history of the AI ​​agent's use as multiple usage logs, divided by date, content, or date and content. When the data size of the multiple usage logs exceeds a predetermined value, the computer evaluates the future utility value of each of the multiple usage logs and compresses, summarizes, or deletes each of the multiple usage logs according to its future utility value.

[0175] For example, the usage fee for a partner-type agent could be calculated based on the storage fee for the usage log (the fee for becoming smarter as the learning data increases with use) and the usage fee for the AI ​​model (the fee for using the smart model). The more frequently a user uses a partner-type agent that provides daily support to the user, the larger the usage log volume will become, resulting in higher management costs. Furthermore, as the usage log becomes larger, searching it using RAG becomes more difficult due to the processing time and memory constraints. At the same time, there is also concern that the amount of usage log data, such as information that is only useful at the time (such as current road congestion), will increase, making it difficult to use as learning data for improving interactions with users in the future.

[0176] In this context, the interactive agent system according to this embodiment manages usage logs by the user who performed the interaction, the date, and the content of the interaction. If the usage log is determined to be highly useful in generating responses during interactions with the user, the detailed data size is retained in a large state. If the usage log is determined to be of low value, the data is summarized to reduce its size, or if it is video or audio data, it is compressed at a higher compression rate to reduce its size, or the usage log data itself is deleted to reduce its size. It is also conceivable that the user may instruct all usage logs relating to a certain matter to be summarized / deleted. With this configuration, it is easy to search for the relevant usage log in accordance with the user's instructions (settings), and the target usage log may be summarized / deleted.

[0177] Furthermore, in the interactive agent system according to this embodiment, the future utility value of the usage log may be determined, for example, by a computing unit of a device that realizes a server, and may be recorded and managed in a memory installed in the device.

[0178] For example, the future utility value of a usage log may be determined as an evaluation value that is higher the more likely the user is to view the usage log data in the future, and lower the more likely the user is to view the usage log data in the future. For example, the future utility value may be evaluated higher for video data of the user's family and friends, and lower for data on the route to a destination that the user searched for on a certain day or traffic conditions at that time.

[0179] Furthermore, for example, the future utility value of a usage log may be determined as an evaluation value such that the higher the probability that the usage log data will be referenced to generate future interactions with the user, the higher the evaluation value, and the lower the probability that the usage log data will be referenced to generate future interactions with the user. For example, the future utility value may be evaluated as high if the information is information about the user's career, work, interests, and family structure, and low if the information is determined to be easily searchable on the Internet.

[0180] Furthermore, in the interactive agent system according to this embodiment, the agent may be configured to interact with the user in accordance with the user's knowledge and level of interest, based on user attribute information that has been determined and stored from interactions with the user, such as the user's knowledge system, interests, and experiences.

[0181] Furthermore, in the interactive agent system according to this embodiment, an AI model trained to make the same judgments and reactions as the user based on user attribute information and usage logs may be created, and the AI ​​model may be used to perform simple prejudgment information processing and judgments. For example, if an introduction to a specific product or service is sent to a user (email delivery, web / SNS advertisement display, etc.), and the AI ​​model that has learned the user's judgments and reactions based on the user attribute information and usage logs determines that the user has a low level of interest in that product or service, the advertisement or information delivery may be treated as low priority, or it may be rejected, not displayed, or the received information may be discarded. On the other hand, a partner-type agent may be configured to curate social media posts and news that the user is interested in, and introduce them to the user periodically or in an event-driven manner, or to compile and report them in a report.

[0182] In the above embodiment, an example has been given of an agent capable of dialogue linked to a user being realized in a dialogue agent system including a vehicle 2, but the present disclosure is not limited to this. The dialogue agent system according to the present disclosure may be realized by including a computer system installed in a space such as a home, office, or store, in addition to or instead of a vehicle.

[0183] For example, if there is a computer system (e.g., an IoT system with a smart speaker as a hub) that controls home appliances and equipment, and a user can communicate with the computer system using natural language, this computer system is equivalent to an in-vehicle system in a vehicle 2. Therefore, what can be achieved with the configuration of information terminal 1, vehicle 2 (in-vehicle system), and cloud 3 described in this disclosure can also be achieved with a configuration of information terminal 1, home (smart speaker), and cloud 3.

[0184] In each of the above-mentioned embodiments, GUI and VUI have been used as a means of communication between the AI ​​agent and the user, but a means of expressing intention via physical operation (PUI) may also be used, or a means of expressing intention via the user's bodily movements (NUI) may also be used, or two or more of these means of expressing intention may be combined to be realized.

[0185] In each of the above-mentioned embodiments, the determination of "whether it is A or not" may be realized by determining only that it is A, or by determining only that it is not A, or by determining both of these.

[0186] In each of the above-described embodiments, "any of A" means "at least one of A."

[0187] The programs executed by each device of the interactive agent system according to each of the above-described embodiments may be provided as files recorded in an installable or executable format on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0188] The programs executed by each device in the dialogue agent system according to each of the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.The programs executed by each device in the dialogue agent system according to the above-described embodiments may be provided or distributed via a network such as the Internet.

[0189] Furthermore, the programs executed by the devices of the interactive agent system according to the above-described embodiments may be provided by being pre-installed in a ROM or the like.

[0190] According to at least one of the embodiments described above, it is possible to appropriately switch between agents that are linked to a user and that can be interacted with. Therefore, even in a situation where an agent needs to be switched, it is possible to appropriately take over a task before and after the switch.

[0191] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0192] 1. Information terminal 10 Smartphone Agent 11 Communications Department 12 Processing section 101 Sensors 103 UI section 104 Arithmetic section 105 memory 106 Communications Department 2 vehicles 20 Vehicle Agents 21 Acquisition Department 22 Setting section 23 Executive Department 24 Handover Section 201 Moving parts 202 Lighting 203 Sensor 204 UI ​​section 205 Key control unit 206 Arithmetic section 207 Memory 208 Communications Department 3,3a,3b Cloud 30, 30a, 30b Cloud Agent 31 Communications Department 32 Processing section 301 Communications Department 302 memory 303 Arithmetic section C Near field communication N Network P user S The user's surrounding space

Claims

1. 1. An information processing method executed by a first computer mounted on a vehicle in an interactive agent system capable of interacting with a user, comprising: a first AI agent implemented in the second computer is connected to the vehicle via a communication device mounted on the vehicle or a communication terminal capable of communicating with the first computer, and a first dialogue process between the vehicle and the user is performed using a microphone and a speaker mounted on the vehicle while connecting to a second computer on a network; recording a first conversation log relating to the content of the conversation between the first AI agent and the user in a memory within the first computer; When the connection to the second computer is disconnected, a second AI agent implemented in the communication terminal or the first computer is caused to execute a second dialogue process with the user based on the first conversation log. Information processing methods.

2. In the execution of the second dialogue processing based on the first conversation log, after detecting a disconnection of the second computer, identifying, based on the first conversation log, one or more tasks that were requested of the first AI agent prior to the disconnection; Selecting one second AI agent capable of executing the one or more tasks from the communication terminal capable of P2P communication with the first computer or one or more second AI agents implemented in the first computer as the second AI agent; inputting task status information including the content and progress status of the one or more tasks into the second AI agent; causing the second AI agent to execute the second interaction process related to the one or more tasks based on the task status information; The information processing method according to claim 1 .

3. The second AI agent is implemented in the communication terminal capable of P2P communication with the first computer, transmitting the task status information to the communication terminal when the connection to the second computer is disconnected; The information processing method according to claim 2 .

4. Prior to the first dialogue processing by the first AI agent, Acquire agent attribute information for representing the character of the first AI agent from the second computer; setting, in the first computer, at least one of a GUI and a VUI representing the character of the first AI agent based on the agent attribute information; interfacing the first interaction process by the first AI agent using at least one of the GUI and the VUI; The information processing method according to claim 1 .

5. The control of at least one of the GUI and the VUI and the recording of the first conversation log are performed by an application installed in the first computer. The information processing method according to claim 4.

6. the GUI is displayed on a display mounted on the vehicle; the GUI includes a first object representing a character of the first AI agent and a second object including text information directed to the user; In the display area of ​​the display, the first object is displayed at a position farther from the user than the second object. The information processing method according to claim 4.

7. the textual information includes information about at least one uncompleted task that has been requested of the first AI agent but has not yet been completed; The information processing method according to claim 6.

8. maintaining the display of the uncompleted task even after the second dialogue process by the second AI agent is initiated; The information processing method according to claim 7.

9. While the second AI agent is executing the second dialogue process, a second conversation log relating to the content of the second dialogue process is recorded in the memory of the first computer; When the connection to the second computer is restored, task status information generated based on the second conversation log is transmitted to the second computer; and interfacing a third interaction between the first AI agent and the user using the microphone and the speaker mounted on the vehicle while connected to the second computer. The information processing method according to claim 1 .

10. In the execution of the third dialogue process based on the second conversation log, After detecting the restoration of the connection to the second computer, identify, based on the second conversation log, one or more tasks that were requested of the first AI agent before the connection was lost or that were newly requested of the second AI agent during the period when the connection was lost; generating the task status information including the content and progress of the one or more tasks; sending the task status information to the first computer, and causing the first AI agent to execute the third interaction process related to the one or more tasks based on the task status information; The information processing method according to claim 9.

11. the second conversation log is managed in the memory by being divided into one or more segments each corresponding to one or more tasks, the task status information includes a summary of a segment of the second conversation log corresponding to the task to be handed over to the first AI agent; The information processing method according to claim 10.

12. Before the third dialogue processing by the first AI agent, Acquire agent attribute information for representing the character of the first AI agent from the second computer; setting, in the first computer, at least one of a GUI and a VUI representing the character of the first AI agent based on the agent attribute information; interfacing the third interaction process by the first AI agent using at least one of the GUI and the VUI; The information processing method according to claim 9.

13. The recording of the second conversation log and the control of at least one of the GUI and the VUI are performed by an application installed in the first computer. The information processing method according to claim 12.

14. A computer installed in a vehicle, a processor; a memory storing a program for causing the processor to execute the information processing method according to any one of claims 1 to 13 as the first computer; computer.

15. A program for causing the first computer to execute the information processing method according to any one of claims 1 to 13.

16. An information processing method for a dialogue agent system capable of dialogue with a user, the method being executed by a second computer that can communicate with a first computer mounted on a vehicle and that has a first AI agent used by the user implemented therein, the method comprising: a first dialogue process between the first AI agent and the user, using a microphone and a speaker mounted on the vehicle while connecting to the first computer via a communication device mounted on the vehicle or a communication terminal of the user; recording a first conversation log relating to the content of the conversation between the first AI agent and the user in a memory within the second computer; Identifying one or more tasks requested of the first AI agent based on the first conversation log; sending task status information including the content and progress of the one or more tasks to the first computer, and having a second AI agent available to the first computer execute a second dialogue process related to the one or more tasks based on the task status information; Information processing methods.

17. The sending of the task status information is performed when the first AI agent determines that it cannot or should not process the one or more tasks.

17. The information processing method according to claim 16.

18. The task status information is transmitted when the usage frequency or usage fee of the first AI agent exceeds a predetermined set value.

17. The information processing method according to claim 16.

19. The sending of the task status information is executed when it is determined that there is another AI agent more suitable than the first AI agent based on the content of the one or more tasks.

17. The information processing method according to claim 16.

20. the first dialogue processing by the first AI agent is executed by connecting to the second computer on which the first AI agent is implemented via the communication terminal; the task status information is transmitted when the remaining battery power of the communication terminal falls below a predetermined value; The second AI agent is an AI agent implemented within the first computer.

17. The information processing method according to claim 16.

21. a processor; a memory storing a program for causing the processor to execute the information processing method according to any one of claims 16 to 20 as the second computer; computer.

22. A program for causing the second computer to execute the information processing method according to any one of claims 16 to 20.

23. 1. An information processing method implemented on a computer that implements an AI agent capable of interacting with a user, comprising: executing a dialogue between the AI ​​agent and the user using a microphone and a speaker installed on the computer or another computer that can communicate with the computer; The history of use of the AI ​​agent is stored in the memory of the computer as a plurality of usage logs divided by date, by content, or by date and content, When the data size of the plurality of usage logs exceeds a predetermined value, a future utility value of each of the plurality of usage logs is evaluated; compressing, summarizing, or deleting each of the plurality of usage logs according to the future usage value; Information processing methods.

24. a processor; a memory storing a program for causing the processor to execute the information processing method according to claim 23; computer.

25. A program for causing a processor to execute the information processing method according to claim 23.

Images