Information processing system, information processing device, and information processing method

Abstract

To improve user satisfaction with virtual characters that can converse with the user. [Solution] The information processing system includes a user analysis unit that analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on a conversation between the user and a virtual character. This technology can be applied, for example, to devices and systems that can converse with a user using a virtual character.

Description

Technical Field

[0001] The present technology relates to an information processing system, an information processing apparatus, and an information processing method, and more particularly to an information processing system, an information processing apparatus, and an information processing method for realizing a virtual character capable of conversing with a user.

Background Art

[0002] In recent years, the use of AI (Artificial Intelligence) in vehicles has been promoted. For example, the use of a voice agent that operates devices according to a user's voice instruction has been promoted (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, technologies related to virtual characters capable of conversing with users, such as voice agents, are still in the development stage and there is room for improvement.

[0005] The present technology has been made in view of such a situation, and aims to improve the user satisfaction with respect to a virtual character capable of conversing with a user.

Means for Solving the Problems

[0006] An information processing system according to one aspect of the present technology includes a user analysis unit that analyzes characteristics of the user including at least one of the user's habits, opinions, desires, and ways of thinking based on a conversation between the user and a virtual character.

[0007] One aspect of this technology is an information processing device which includes a user analysis unit that analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on a conversation between the user and a virtual character.

[0008] One aspect of this technology is an information processing method in which an information processing device analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on a conversation between the user and a virtual character.

[0009] In one aspect of this technology, the user's characteristics, including at least one of the user's habits, opinions, desires, and ways of thinking, are analyzed based on conversations between the user and a virtual character. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing one embodiment of an information processing system to which this technology is applied. [Figure 2] This is a block diagram showing an example of a main server configuration. [Figure 3] This is a flowchart to explain the process of acquiring user information. [Figure 4] This is a diagram to explain the news notification process. [Figure 5] This is a flowchart explaining the process of notifying weather information. [Figure 6] This is a flowchart explaining the restaurant recommendation process. [Figure 7] This is a flowchart illustrating the details of the destination setting process. [Figure 8] This is a diagram showing an example of a computer configuration. [Modes for carrying out the invention]

[0011] The following describes the configurations for implementing this technology. The explanation will proceed in the following order. 0. Background of this technology 1. Embodiment 2. Variation Example 3. Others

[0012] <<0. Background of the Present Technology>> First, the background of the present technology will be described.

[0013] [[ID=,12]] As described above, in recent years, the utilization of voice agents in vehicles has been progressing.

[0014] Conventional voice agents mainly focus on a convenient function of operating devices on behalf of the user according to the user's voice instructions.

[0015] In contrast, in the present technology, as the user utilizes conversations and functions with the character, the character is personalized. That is, as the character's understanding of the user's characteristics progresses, it is customized to the user specifications, and it becomes possible to make proposals, provide information, and execute actions that are more suitable for the user.

[0016] In addition, the character executes a conversation catch ball, rather than a simple Q&A format, and a dialogue design that allows the user to empathize is constructed.

[0017] As a result, the user's satisfaction with the character is improved.

[0018] <<1. Embodiment>> Next, referring to FIGS. 1 to 11, the embodiments of the present technology will be described.

[0019] <Configuration Example of Information Processing System 101> FIG. 1 shows an embodiment of an information processing system 101 to which the present technology is applied.

[0020] The information processing system 101 is a system that provides a service (hereinafter referred to as a character service) that executes conversations and various functions using a virtual character to a user. The character is an AI agent configured by a virtual human model.

[0021] The functions provided by the information processing system 101 include, for example, notification functions for weather information and news, restaurant recommendation functions, content playback functions for music and other content, and remote control functions for various devices.

[0022] The information processing system 101 comprises a main server 111, an external service server 112, an AI server 113, and edge terminals 114-1 to 114-n. The main server 111, the external service server 112, the AI ​​server 113, and edge terminals 114-1 to 114-n are interconnected via a network 121 such as the Internet.

[0023] Hereafter, when there is no need to distinguish between edge terminals 114-1 to 114-n individually, they will simply be referred to as edge terminal 114.

[0024] The main server 111 is an information processing device that provides character services to each edge terminal 114. The main server 111 receives user utterance data, character utterance data, user status data, and surrounding status data from the edge terminal 114 via the network 121.

[0025] User speech data is data that shows the content of the user's speech. Character speech data is data that shows the content of the character's speech that was actually output at the edge terminal 114. User situation data is data about the user's situation. Surrounding situation data is data about the situation around the user.

[0026] The main server 111 analyzes and understands the user's characteristics, the context of the conversation between the user and the character (hereinafter referred to as the conversation context), etc., based on user utterance data, character utterance data, and user situation data.

[0027] The main server 111, as needed, uses various services provided by the external service server 112 via the network 121 to obtain various information (hereinafter referred to as external information) from the external service server 112. The main server 111, as needed, sends prompts to the AI ​​server 113 via the network 121 and receives data generated by the AI ​​based on the prompts (hereinafter referred to as AI-generated data) from the AI ​​server 113.

[0028] The main server 111 generates behavior control information that controls the character's actions, such as speech, based on user characteristics, conversation context, external information, and AI-generated data. The main server 111 transmits the behavior control data to the edge terminal 114 via the network 121 and controls the character's actions, such as speech, on the edge terminal 114. In addition, the main server 111 provides various functions to the user through the character by controlling the character's actions on the edge terminal 114.

[0029] External service server 112 is a server that provides various services such as search services, weather forecast services, news services, and restaurant information services.

[0030] The AI ​​server 113 is a server that provides generative AI such as LLM (Large Language Model). The AI ​​server 113 may provide multiple types of generative AI.

[0031] The edge terminal 114 is a device that allows users to access character services. The edge terminal 114 consists of, for example, a vehicle, an in-vehicle device, a smartphone, a PC (personal computer), etc. The edge terminal 114 performs processing related to character services by executing a predetermined application program based on behavior control information received from the main server 111, for example.

[0032] In the following, when each device of the information processing system 101 communicates via the network 121, the phrase "via the network" will be omitted. For example, when the main server 111 and the edge terminal 114 communicate via the network 121, it will simply be stated that the main server 111 and the edge terminal 114 communicate.

[0033] Furthermore, the following explanation will focus on examples where the edge terminal 114 is used in a vehicle. In this case, the edge terminal 114 may be the vehicle itself, an in-vehicle device, or a device brought into the vehicle and used therein.

[0034] <Example configuration for main server 111> Figure 2 shows an example configuration of the main server 111.

[0035] The main server 111 includes an information processing unit 151, a character information storage unit 152, a user information storage unit 153, a conversation log storage unit 154, an action history storage unit 155, a setting information storage unit 156, and an information DB (database) 157.

[0036] The information processing unit 151 includes, for example, a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and executes processing related to character services. The information processing unit 151 also includes an action processing unit 161, a user analysis unit 162, and a communication unit 163.

[0037] The action processing unit 161 executes processing related to the character's actions, such as conversations. The action processing unit 161 includes an information gathering unit 171, a context analysis unit 172, and an action control unit 173.

[0038] The information gathering unit 171 collects information to analyze and understand the user's characteristics and conversational context.

[0039] For example, the information gathering unit 171 receives user speech data, character speech data, user status data, and surrounding status data from each edge terminal 114 via the communication unit 163.

[0040] For example, the information gathering unit 171 may receive audio data representing each user's speech from each edge terminal 114, perform speech recognition processing on the audio data, and generate user speech data representing the recognized speech content.

[0041] User status data may include, for example, data relating to the user's gaze direction, posture, behavior, and physical condition. User behavior includes, for example, user operations on edge terminals 114 or character services. The method of user operation is not particularly limited and includes, for example, manual operation of devices such as edge terminals 114, operation by gesture, operation by voice commands, etc. User status data may also include, for example, image data (video data or still image data) of the user.

[0042] Surrounding environment data may include, for example, data about the user's surroundings and data about objects around the user. Data about the user's surroundings may include, for example, data about the user's current location, surrounding weather, brightness, temperature, etc. Data about objects around the user may include, for example, information about people, vehicles, obstacles, structures, traffic lights, traffic signs, etc. Surrounding environment data may also include, for example, image data (video data or still image data) taken of the user's surroundings.

[0043] For example, if the edge terminal 114 is used in a vehicle, the surrounding situation data may include data related to the vehicle's status. For example, the data related to the vehicle's status may include data on the vehicle type, speed, acceleration, direction of travel, and passengers. For example, the data related to the vehicle's status may include image data (video data or still image data) taken inside the vehicle and image data (video data or still image data) taken outside the vehicle.

[0044] For example, the information gathering unit 171 obtains setting information related to user settings from the setting information storage unit 156. The setting information may include, for example, information about the user's default settings such as character, language, and time zone.

[0045] For example, the information gathering unit 171 acquires user information regarding the user's characteristics from the user information storage unit 153.

[0046] User information may include, for example, information about the user's objective characteristics and information about the user's inner self. Information about the user's objective characteristics may include, for example, the user's external characteristics, profile, episodes, etc. User episodes may include, for example, information about events the user has experienced and information about the user's plans. Information about the user's inner self may include, for example, information about the user's habits, opinions, desires, way of thinking, preferences, etc.

[0047] A habit is, for example, a pattern of behavior or thought that a user repeatedly performs.

[0048] An opinion, for example, is a user's thought or feeling, and includes principles, assertions, beliefs, subjectivity, views, values, emotions, etc.

[0049] Desires, for example, are the physiological or psychological needs or wishes of the user.

[0050] A way of thinking, for example, refers to the direction a user takes, their thought patterns, and includes their approach, perspective, and way of dealing with things.

[0051] Preferences, for example, represent a user's tastes, interests, and concerns.

[0052] For example, the information gathering unit 171 obtains character information about the character that interacts with the user from the character information storage unit 152.

[0053] Character information includes, for example, personal data that represents the individuality of each character, and image data of each character.

[0054] For example, the information gathering unit 171 acquires conversation logs related to ongoing conversations between the user and the character from the conversation log storage unit 154.

[0055] A conversation log is a log of ongoing conversations between a user and a character. For example, a conversation log includes the conversation history, conversation context information, and various pieces of information used or present in the conversation.

[0056] Conversation context information is information that includes the results of an analysis of the conversation context. Conversation context includes, for example, the context or content of a conversation between a user and a character, the user's intentions, the user's situation, the circumstances surrounding the user, etc.

[0057] The information gathering unit 171 supplies necessary information from among user utterance data, character utterance data, user situation data, surrounding situation data, user information, character information, and conversation logs to the context analysis unit 172 and the user analysis unit 162.

[0058] The context analysis unit 172 analyzes and understands the conversation context based on the information obtained from the information gathering unit 171. The context analysis unit 172 supplies the conversation context information, including the analysis results of the conversation context, to the planning unit 181.

[0059] The context analysis unit 172 generates or updates conversation logs and stores them in the conversation log storage unit 154. For example, the context analysis unit 172 updates the conversation log by adding some or all of the conversation context information, user utterance data, and character utterance data to the conversation log relating to the user's ongoing conversation.

[0060] The context analysis unit generates or updates user behavior history data and stores it in the behavior history storage unit 155. For example, the context analysis unit updates the behavior history data by adding some or all of the conversation context information, user utterance data, and character utterance data to the user behavior history data.

[0061] The behavioral history data includes, for example, the conversation history between the user and the character from the start of using the character service, and all or part of the conversation context information in the conversation between the user and the character.

[0062] The behavior control unit 173 controls the actions of the character on the user's edge terminal 114, including speech, based on conversation context information, user information, and character information. The behavior control unit 173 includes a planning unit 181 and a generation unit 182.

[0063] The planning unit 181 obtains character information from the character information storage unit 152. The planning unit 181 obtains user information from the user information storage unit 153. The planning unit 181 obtains information necessary for planning character actions from the information DB 157 as needed. The planning unit 181 obtains external information necessary for planning character actions from the external service server 112 via the communication unit 163 as needed.

[0064] The planning unit 181 plans the character's actions based on conversation context information, user information, character information, and information obtained from the information database 157 and the external service server 112 as needed. For example, the planning unit 181 supplies action plan information indicating the character's action plan to the generation unit 182.

[0065] The planning unit 181 adds some or all of the information obtained from the information database 157 and the external service server 112 to the conversation log of the user's ongoing conversations stored in the conversation log storage unit 154, as needed.

[0066] The generation unit 182 acquires character information from the character information storage unit 152. The generation unit 182 acquires user information from the user information storage unit 153. The generation unit 182 generates action control information based on the action plan information, user information, and character information, using the AI ​​server 113 via the communication unit 163 as needed.

[0067] Action control information is information used to control the execution of character actions planned by the planning unit 181. Action control information includes, for example, information about the character to be controlled, the content of the character's speech, and the movement of the character's image.

[0068] The generation unit 182 transmits behavior control information to the edge terminal 114 via the communication unit 163, thereby controlling the character's behavior at the edge terminal 114.

[0069] The user analysis unit 162 analyzes user characteristics based on information supplied from the information collection unit 171 and the user behavior history data stored in the behavior history storage unit 155, utilizing the AI ​​server 113 via the communication unit 163 as needed. Based on the analysis results of the user characteristics, the user analysis unit 162 generates or updates user information for each user and stores it in the user information storage unit 153.

[0070] The communication unit 163 communicates with the external service server 112, the AI ​​server 113, and each edge terminal 114.

[0071] The user information storage unit 153 stores user information for each user.

[0072] The character information storage unit 152 stores character information for each character.

[0073] The conversation log storage unit 154 stores conversation logs related to each user's ongoing conversations.

[0074] The behavior history storage unit 155 stores the behavior history data of each user.

[0075] The configuration information storage unit 156 stores the configuration information for each user.

[0076] Information DB157 stores various types of information used in each user's conversation.

[0077] In the following, when any part of the main server 111 communicates with an external device via the communication unit 163, the phrase "via the communication unit 163" will be omitted. For example, when the generation unit 182 communicates with the AI ​​server 113 via the communication unit 163, it will simply be stated that the generation unit 182 communicates with the AI ​​server 113.

[0078] <How to manage user information> Next, we will explain how to manage user information, including how to acquire and update it.

[0079] <How to update user information> First, let's explain how to update user information.

[0080] For example, the user analysis unit 162 updates some of the user information in real time based on one or more of the user utterance data, character utterance data, user situation data, surrounding situation data, and conversation logs supplied by the information collection unit 171. Specifically, for example, the user analysis unit 162 updates information related to the user's profile, episodes, preferences, opinions, and desires in real time based on the conversation context (conversation context or content, user situation, user's surroundings, etc.).

[0081] For example, a user's profile, obtained from direct expressions in their utterances, is stored in a key-value format without pre-determining the items. Furthermore, the profile keys and values ​​are expressed in natural language. For instance, the key "unacceptable food" might be associated with the value "canned fruit," and the key "name of pet iguana" might be associated with the value "Grande."

[0082] This increases the flexibility of the profiles that can be stored compared to when user profiles are stored using pre-determined fixed slots.

[0083] For example, the user analysis unit 162 updates a portion of each user's user information in batch processing at predetermined times (for example, at a predetermined time late at night each day) based on the user's behavioral history data stored in the behavioral history storage unit 155. For example, the user analysis unit 162 updates the user's habits, way of thinking, and a summary of their daily conversations at predetermined times.

[0084] For example, user habits can be obtained by aggregating user episodes (events and plans) and inductively analyzing trends. User habits may also include usage trends of character service functions (e.g., frequency of use of each function, time of day of use, method of use, etc.).

[0085] <Methods for analyzing user comments> Next, I will explain how to analyze user comments.

[0086] For example, the user analysis unit 162 uses the AI ​​server 113 (e.g., LLM) to analyze the user's statements step by step using prompts that include the following steps 1 to 5. This allows the user analysis unit 162 to extract not only superficial information but also the user's fundamental desires and opinions.

[0087] In the next step, write down the true meaning of what the other person said. Step 1: The context of the statement. What was the situation? Step 2: How is the other person feeling? Step 3: Why did the other person make that statement? Step 4: What are the other person's fundamental desires? Step 5: Identify the principles, beliefs, and opinions that the other person consistently holds.

[0088] Note that "the other party" in this prompt refers to the user.

[0089] Step 1 analyzes the context of the user's statements. Step 2 analyzes the user's feelings. Step 3 analyzes the reasons for the user's statements. Step 4 analyzes the user's (fundamental) desires. Step 5 extracts the user's (constant) opinions. The extracted user opinions include the user's principles and beliefs.

[0090] Furthermore, for example, the user analysis unit 162 can continuously deepen its understanding of the user by adding user information to the prompt, as shown below.

[0091] #instructions Using the supplementary information as a guide, please write down the true meaning of the other person's statement in the next step. # Supplementary information *User information **User name** Katsutoshi **User feedback** ***They value family events and creating memories. At the same time, they are prudent in acting cautiously while keeping financial realities in mind. They have a value system that dictates avoiding unnecessary expenses. **User experience and plans** ***I was the captain of the kendo club when I was in junior high school. ***I often made mistakes because of alcohol when I was younger. ***There were no physics problems that I couldn't solve. ***I ate a Big Mac yesterday. **User Habits** ***I make my own coffee in the morning. **User's favorite things** ***Belgian beer **Things users dislike** ***Canned fruit **Other user information** ***Family composition: Wife and two children ***Dog breed: Pomeranian ***Hobbies: Mathematical logic, game development Step 1: The context of the statement. What was the situation? Step 2: How is the other person feeling? Step 3: Why did the other person make that statement? Step 4: What are the other person's fundamental desires? Step 5: Identify the principles, beliefs, and opinions that the other person consistently holds.

[0092] Here, we will explain a specific example of how to analyze user comments.

[0093] For example, if a user first says, "I set up a drinking party for 8 PM, but I can't wait at all," the following analysis steps 1 to 5 will be performed.

[0094] Step 1: You probably have plans to attend a drinking party scheduled for 8 PM, and at this point, there is still plenty of time before that.

[0095] Step 2: While you are very much looking forward to the drinking party, you also feel impatient and excited.

[0096] Step 3: I wanted to express and share my feelings of being impatient while waiting because I was so excited, in order to alleviate those feelings or gain empathy.

[0097] Step 4: The desire to have fun as soon as possible. Also, the desire to share the exhilaration you are feeling with someone and find resonance with them.

[0098] Step 5: You have a personality that gets excited about fun plans. You tend to get impatient when you feel like you have to wait a long time. You tend to express your emotions and feelings frankly.

[0099] Next, if the user says, "I actually want to go home and see my kids and dog," the following analysis will be performed, as described in steps 1 through 5 below.

[0100] Step 1: You have plans to attend a drinking party, but deep down you have a strong attachment to spending time with your family and pets, and would rather go home. However, you may be attending the party due to expectations or schedules of those around you.

[0101] Step 2: A love for family and dogs, and a desire for relaxing time at home. On the other hand, there may be some guilt or remorse about being at a drinking party.

[0102] Step 3: While outwardly appearing to enjoy the party, reveal that inwardly you would prefer to prioritize time with family and pets. By expressing your true feelings a little, you can encourage empathy and self-awareness.

[0103] Step 4: I want to feel secure and happy spending time with my beloved family and pets. I want someone to understand my true feelings.

[0104] Step 5: Values ​​that cherish time with family and loved ones. Sometimes struggles between social obligations and expectations, and one's true feelings and desires. Wants to express one's true feelings honestly, but tends to worry about how others will perceive it.

[0105] Next, if the user says, "Well, in reality, they're a pain to deal with," the following analysis will be performed, as shown in steps 1 through 5 below.

[0106] Step 1: While feeling love for family and dogs, the reality is that caring for them involves time and effort. This gap is mentioned with a touch of humor.

[0107] Step 2: Complex emotions—caring deeply for family and dogs, yet sometimes finding their care burdensome. This contradiction is acknowledged with self-deprecating humor, and shared as an amusing topic of conversation.

[0108] Step 3: Because of their love, they try to elicit empathy and laughter from the other person by lightly making fun of their own situation, which involves accepting the reality that it is troublesome.

[0109] Step 4: You want someone to understand the realistic burdens of love, even while you feel loved. You want to acknowledge your conflicting emotions and let them go with a light heart.

[0110] Step 5: While affectionate, they tend to be honest about the practical burdens and hassles involved. They are good at sharing emotions through laughter and lighthearted sarcasm. They don't take the contradictions in relationships and everyday life too seriously, and tend to accept them with humor.

[0111] Note that the order of the steps in analyzing this user's statements may be changed. Also, steps may be added, modified, or deleted as needed.

[0112] In this way, by analyzing the user's true intentions, it is possible to extract the user's opinions (e.g., principles, beliefs) from the user's statements and store them as user information. Furthermore, for example, the generation unit 182 can utilize the stored user opinions in conversation and, using the AI ​​server 113, generate utterances that show empathy and suggestions while understanding the user.

[0113] For example, based on a user's statement, "Drinking coffee calms me down," the user's opinion, "Drinking coffee calms me down," is extracted and stored. In response to this, for example, if the user says, "I'm getting nervous. I need to calm down," the utterance, "How about drinking some coffee to calm down?" is generated and spoken by the character.

[0114] For example, based on a user's statement, "Taking a bath makes fatigue disappear," the user's opinion that "Taking a bath relieves fatigue" is extracted and stored. In response to this, for example, in response to a user's statement, "I was tired from today's meeting," the utterance, "You did a great job. Let's take a bath and wash away your fatigue," is generated and spoken by the character.

[0115] For example, based on a user's statement, "Meetings should begin with a clear statement of purpose," the user's opinion, "Meetings should clearly state their purpose," is extracted and stored. In response to a user's statement, for example, "I wonder what the purpose of today's meeting was," the utterance, "We should have stated the purpose at the beginning," is generated and spoken by the character.

[0116] For example, based on a user's statement, "I need to go to bed early tonight," the user's opinion that "you should go to bed early at night" is extracted and stored. In response to this, for example, if the user says, "Maybe I'll work a little harder," the utterance, "It would be better to go to bed early tonight and start tomorrow morning," is generated and spoken by the character.

[0117] For example, based on a user's statement, "Karaoke is the best way to relieve stress," the user's opinion that "Karaoke is ideal for stress relief" is extracted and stored. In response to this, for example, if the user says, "I think I've been feeling stressed lately," the utterance, "In that case, invite a friend to karaoke and relieve some stress," is generated and spoken by the character.

[0118] <Method for modifying and aggregating user information> Next, we will explain how to modify and consolidate user information.

[0119] For example, the user analysis unit 162 uses the AI ​​server 113 (e.g., LLM) to resolve inconsistencies in user information. For instance, if information such as "likes ramen" and "dislikes ramen" exists simultaneously, the user analysis unit 162 uses the AI ​​server 113 to detect these inconsistencies and adopt the newer information.

[0120] For example, the user analysis unit 162 inputs a list of the user's likes and dislikes to the AI ​​server 113 along with the following prompts, and detects inconsistent information.

[0121] Find all combinations of things you like and things you dislike that contradict each other. Please consider the following cases to be contradictory. When you like and dislike the same thing Example) Likes: Ramen, Dislikes: Ramen When you like one thing and dislike the things that encompass it. Example) Likes: Ramen, Dislikes: Noodles When you dislike one thing but like the things that encompass it. Example) Likes: Noodles, Dislikes: Ramen

[0122] For example, the user analysis unit 162 uses the AI ​​server 113 (e.g., LLM) to aggregate distributed user information. For example, if multiple pieces of information about the same event exist, the user analysis unit 162 combines them into one to generate new information.

[0123] For example, the user analysis unit 162 inputs a list of user events to the AI ​​server 113 along with the following prompt, and aggregates information about the same events.

[0124] #instructions Please follow the steps below to produce the output. Step 1: Cluster events that relate to the same event. Step 2: Summarize the events within the same cluster and group them together as a single, continuous series of events. Step 3: Output in the following format. If there are multiple entries, please use line breaks. when: ○○, content: ○○

[0125] Note that the output format for step 3 of this prompt omits the specific format of the timestamp.

[0126] For example, a list of events like the one below is entered into the AI ​​server 113 along with this prompt. Note that the specific timestamps for each event are omitted.

[0127] When: Weekend, Content: I'm going on a trip to Shizuoka. When: Today, Content: I ate ramen. When: Today, Content: The ramen was delicious. When: Today, Content: I went for a walk in the park. When: Today, Content: There was a company meeting. When: Today, Content: The bamboo shoots tasted strange. When: Today, Content: I went to the park. When: Today, Content: The soup was quite spicy. When: 3 days ago, Content: I played soccer with my friends. When: 3 days ago, Content: I scored a hat-trick. When: Yesterday, Content: I went shopping. When: Yesterday, Content: I bought kamaboko (fish cake) and a bicycle. When: Yesterday, Content: I enjoyed shopping.

[0128] In response, the following list of identical events is output. Note that the specific timestamps for each event are omitted.

[0129] When: Weekend, Content: I'm going on a trip to Shizuoka. When: Today, Content: I ate ramen and it was delicious, but the bamboo shoots had a strange taste and the soup was quite spicy. When: Today, Content: I went to the park and took a walk. When: Today, Content: There was a company meeting. When: 3 days ago, Content: I played soccer with my friends and scored a hat-trick. When: Yesterday, Content: I went shopping and bought kamaboko (fish cake) and a bicycle. I enjoyed shopping.

[0130] Furthermore, for example, the user analysis unit 162 can aggregate similar information and information with inclusion relationships using the AI ​​server 113 (e.g., LLM). For example, if there is information with subtle differences in expression or inclusion relationships, the user analysis unit 162 can combine that information into one and generate new information.

[0131] <User Information Acquisition Strategy> Next, we will explain the strategy for acquiring user information.

[0132] To provide users with appropriate news, weather, content, etc., it is necessary to collect useful user information in advance. However, simply collecting user information through everyday conversations with users is often insufficient to gather truly useful information.

[0133] In response, the information processing system 101 collects useful user information by strategically asking the user questions.

[0134] Now, referring to the flowchart in Figure 3, we will explain the user information acquisition process performed by the information processing system 101.

[0135] This process is executed as needed, for example, during a conversation with a user.

[0136] In step S1, the information processing system 101 analyzes the context of the conversation with the user.

[0137] For example, the information collection unit 171 receives user utterance data, character utterance data, user status data, and surrounding status data from the user's edge terminal 114 during a conversation. For example, the information collection unit 171 obtains user setting information from the setting information storage unit 156. For example, the information collection unit 171 obtains user information from the user information storage unit 153. The information collection unit 171 obtains character information about the character the user is conversing with from the character information storage unit 152. For example, the information collection unit 171 obtains conversation logs about the ongoing conversation between the user and the character from the conversation log storage unit 154.

[0138] Furthermore, the information gathering unit 171 is not necessarily required to acquire all of the above-mentioned information; it may acquire only the necessary information.

[0139] The information gathering unit 171 supplies the information necessary for the conversation context from the acquired information to the context analysis unit 172.

[0140] The context analysis unit 172 analyzes and understands the conversation context based on the information obtained from the information gathering unit 171. The context analysis unit 172 supplies the conversation context information, including the analysis results of the conversation context, to the planning unit 181.

[0141] In step S2, the planning unit 181 determines whether or not to ask a question based on the conversation context information. For example, if the planning unit 181 determines that it is appropriate to ask a question spontaneously, or if it is appropriate to change the topic, it proceeds to step S3. For example, if the planning unit 181 determines that it is appropriate to ask a question if there is information that needs to be obtained regarding the topic in the current conversation, it proceeds to step S3.

[0142] In step S3, the planning unit 181 acquires the question data.

[0143] For example, the information database 157 stores question data. This question data includes, for instance, effective question items tailored to specific uses, and defines the necessary components for generating question statements in the LLM.

[0144] Specifically, the question data includes, for example, "items," "method of asking or obtaining questions," and "purpose."

[0145] "Items" refer to the categories of information to be obtained through the questions. For example, "items" may include "favorite artist," "favorite athlete," "favorite entertainment genre," "favorite comedian," "frequently used media," "occupation," "hometown," and "favorite sports team."

[0146] "How to ask a question or how to obtain information" indicates how to ask a question or how to obtain the information registered in the "item".

[0147] Specifically, for example, under the item "Favorite Artist," information such as "Who is your favorite artist?" is registered.

[0148] For example, under the item "Favorite Entertainment Genre," information such as "Do you like comedy?", "Do you like games?", "Do you like movies?", "Do you like manga?", and "Do you like anime?" is registered. Also, under the item "Favorite Entertainment Genre," information such as "Asking 'What is your favorite entertainment genre?' doesn't yield the expected results, so we'll ask in a closed format" is registered.

[0149] For example, under the item "Who is your favorite comedian?", the following information is registered: "(For people who like comedy only) Who is your favorite comedian?" and "(For people who like comedy only, and in the context of comedian A) Do you like comedian A?".

[0150] For example, under the category "Frequently used media," information such as "Do you often watch YouTube (registered trademark)?", "Do you often watch TikTok?", "Do you often watch TV?", and "Do you listen to the radio?" is registered.

[0151] For example, under the "Occupation" field, information such as "Obtained from profile-like data?" is registered.

[0152] For example, under the "Local Area" category, the information registered is "Obtained from profile-like information?".

[0153] For example, under the category "Sports team you support," information such as "Which team (or player) do you support?" is registered.

[0154] "Purpose" indicates the intended use of the acquired information. For example, "Purpose" includes the type of function that will use the acquired information (e.g., "news," "weather," "music," etc.).

[0155] The planning unit 181 selects items to ask questions based on the current conversation context and the user information of the user in the conversation stored in the user information storage unit 153, and retrieves question data related to the selected items from the information DB 157. The planning unit 181 supplies the retrieved question data to the generation unit 182.

[0156] In step S4, the information processing system 101 asks the user a question. For example, the generation unit 182 sends a prompt containing question data to the AI ​​server 113, requests the generation of a question sentence, and retrieves the generated question sentence from the AI ​​server 113.

[0157] The generation unit 182 generates behavioral control information, including the generated question text, and transmits it to the user's edge terminal 114.

[0158] The edge terminal 114 asks the user a question by having a character speak based on behavioral control information.

[0159] The questions posed to the user include open-ended questions that can be answered in any format, and closed-ended questions that require answers from a limited set of options.

[0160] Open-ended questions might include things like favorite artists, favorite sports, occupation, or place of origin. Open-ended questions are also used, for example, when a character spontaneously asks a question or changes the topic.

[0161] Closed questions take into account the context of the conversation with the user and ask about their likes and dislikes, interests, etc., regarding specific items that are being discussed in the conversation. For example, when the topic is restaurant A, the question "Do you like restaurant A?" might be asked. For example, when election news is announced, the question "Are you interested in elections?" might be asked.

[0162] For example, information that is difficult to obtain with an open-ended question like "What book do you want to read?" can be obtained by repeatedly asking an easy-to-answer closed-ended question like "Do you want to read XX?", allowing for the collection of multiple pieces of information about books that people want to read in the medium to long term.

[0163] In this way, the questions asked by the character to the user are controlled based on the conversational context, and open-ended and closed-ended questions are used appropriately.

[0164] In step S5, the information processing system 101 obtains a response from the user.

[0165] For example, the user answers questions from the character.

[0166] In response, the edge terminal 114 performs speech recognition processing on the utterance representing the user's answer and generates user utterance data indicating the recognized utterance content. The edge terminal 114 sends character utterance data indicating the character's question content and user utterance data indicating the user's answer content to the main server 111.

[0167] The information gathering unit 171 of the main server 111 receives character speech data and user speech data from the edge terminal 114. The information gathering unit 171 supplies the character speech data and user speech data to the user analysis unit 162.

[0168] In step S6, the user analysis unit 162 updates the user information. For example, the user analysis unit 162 extracts user characteristics based on character utterance data and user utterance data, that is, based on the user's answers to the character's questions. Based on the extracted user characteristics, the user analysis unit 162 updates the user information of the user in the conversation stored in the user information storage unit 153.

[0169] After that, the user information retrieval process ends.

[0170] On the other hand, if it is determined in step S2 that no question is to be asked, the processing in steps S2 to S6 is skipped, and the user information acquisition process ends.

[0171] <News notification function> Next, we will describe the news notification function provided by the information processing system 101.

[0172] For example, based on the conversation context (e.g., the user's utterances) and user information, the information processing system 101 can notify the user of news that is valuable or of interest to the user during a conversation, even if the user does not give specific instructions directly.

[0173] For example, during a conversation with a user, the planning unit 181 generates a news query based on the conversation context and user information, as shown in Figure 4.

[0174] Specifically, for example, the planning unit 181 extracts intent parameters that indicate the user's intent based on the user's utterances included in the conversation context and the usage trends of character service functions included in the user information. The planning unit 181 extracts keywords from the intent parameters and generates an intent query.

[0175] Planning Unit 181 extracts keywords based on the user's profile, episodes, opinions, and habits included in the user information.

[0176] The planning unit 181 generates news queries based on intent queries, user preferences included in user information, and query candidates containing keywords. For example, the planning unit 181 sends the news queries to the external service server 112 and retrieves news information from the external service server 112 that shows the news searched based on the news queries. The planning unit 181 supplies the retrieved news information to the generation unit 182.

[0177] The generation unit 182, for example, sends a prompt containing news information to the AI ​​server 113, requests the generation of an utterance to notify the news, and retrieves the generated utterance from the AI ​​server 113. The generation unit 182 generates behavior control information including the generated utterance and sends it to the user's edge terminal 114.

[0178] The edge terminal 114 notifies the user of news by having a character speak based on behavioral control information.

[0179] This allows, for example, news articles that are likely to be of interest to the user to be prioritized and notified to the user from among the searched news articles.

[0180] <Weather information notification function> Next, we will describe the weather information notification function provided by the information processing system 101.

[0181] Now, referring to the flowchart in Figure 5, we will explain the weather information notification process performed by the information processing system 101.

[0182] This process is executed as needed, for example, during a conversation with a user.

[0183] In step S51, the context of the conversation with the user is analyzed, similar to the process in step S1 in Figure 3. The context analysis unit 172 supplies conversation context information, including the results of the conversation context analysis, to the planning unit 181.

[0184] In step S52, the planning unit 181 determines whether or not to notify the user of weather information based on the conversation context and user information. For example, the planning unit 181 determines whether or not to notify the user of weather information not only based on the user's intention to know the weather, but also based on the user's schedule, destination, etc., in the context of the conversation with the user. Alternatively, the planning unit 181 may use the AI ​​server 113 (e.g., LLM) to determine whether or not to notify the user of weather information. If it is determined that weather information should be notified, the process proceeds to step S53.

[0185] In step S53, the planning unit 181 acquires weather information. For example, based on user information, the planning unit 181 extracts combinations of location and date / time for acquiring weather information.

[0186] Specifically, for example, combinations of location and date / time are extracted based on the user's schedule. For example, if the user has a trip to Atami planned for the weekend, the combination (Atami, weekend) is extracted. For example, if the user has a drinking party planned in Shinagawa the day after tomorrow, the combination (Shinagawa, day after tomorrow) is extracted.

[0187] For example, combinations of location and date / time are extracted based on the user's recent actions and location information. For instance, combinations such as (near home, today), (near home, tomorrow), (near work, today), (near work, tomorrow) are extracted.

[0188] For example, combinations of location and date / time are extracted based on the user's associated region. For instance, combinations such as (near parents' home, today), (near parents' home, tomorrow), (family's address, today), (family's address, tomorrow), (hometown, today), (hometown, tomorrow) are extracted.

[0189] The planning unit 181 obtains weather information for each location and time from the external service server 112.

[0190] In step S54, the planning unit 181 selects the weather information to be notified. Specifically, the planning unit 181 calculates a score for each piece of weather information based on the location, date and time, weather conditions, etc. of the acquired weather information.

[0191] For example, the score for each weather forecast is calculated based on five scores: impact, location relevance, weather / date / time score, notification appropriateness, and importance.

[0192] The impact level indicates the degree to which the predicted weather will affect the user. For example, the impact level will be higher if severe weather such as typhoons, heavy rain, heavy snow, or strong winds is predicted. Furthermore, the more severe the predicted severe weather, the higher the impact level will be.

[0193] Location relevance indicates the user's relevance to the location where the weather is predicted. For example, the higher the relevance between the user's current location, home, workplace, family addresses, and upcoming plans and the location where the weather is predicted, the higher the location relevance.

[0194] The weather date and time score indicates the score relative to the date and time the weather was predicted. For example, the closer the predicted date and time is to the present, the higher the weather date and time score will be.

[0195] Notification appropriateness indicates, for example, how appropriate it would be to notify someone of the relevant weather information at that particular time. For example, for today's weather information, notification appropriateness is higher before commuting than before going to bed. For example, for tomorrow's weather information, notification appropriateness is higher before going to bed than before commuting.

[0196] Importance indicates the user's level of importance to the weather information in question. For example, importance is calculated based on the user's behavior patterns, schedule, interests, health status, etc. For instance, if a user's activities at the time the weather is predicted are likely to be affected by the weather, the importance will be higher. For example, if a user plans to be outdoors at the time the weather is predicted, the importance will be higher.

[0197] For example, the planning unit 181 selects the weather information with the highest score to be notified to the user. The planning unit 181 then supplies the selected weather information to the generation unit 182.

[0198] In step S55, the information processing system 101 notifies the weather information. For example, the generation unit 182 sends a prompt containing weather information to the AI ​​server 113, requests the generation of a speech statement to notify the weather information, and retrieves the generated speech statement from the AI ​​server 113.

[0199] The generation unit 182 generates behavior control information including the generated utterance and transmits it to the user's edge terminal 114.

[0200] The edge terminal 114 notifies the user of weather information by having a character speak based on behavioral control information.

[0201] This allows, for example, contextual weather information to be provided during a conversation with a user.

[0202] Specifically, users will be proactively notified of weather information they need. For example, they might receive notifications such as, "It looks like there's heavy snow in Toyama, where my family lives," "It's going to be sunny tomorrow, so it should be good for a run," "It looks like there will be a lot of pollen tomorrow, so be careful," or "It might rain during next week's camping trip."

[0203] For example, weather information could be sent proactively and regularly, tailored to the user's weather usage patterns. For instance, notifications might include messages like, "Today's weather will be partly cloudy," or "Tomorrow's weather will be rainy. Don't forget your umbrella."

[0204] On the other hand, if it is determined in step S1 that weather information should not be notified, the processes in steps S52 to S55 are skipped, and the weather information notification process ends.

[0205] <Restaurant recommendation function> Next, we will explain the restaurant recommendation function provided by the information processing system 101.

[0206] For example, collaborative filtering has traditionally been used in the recommendation process for restaurants.

[0207] Collaborative filtering requires data on the usage patterns and preferences of many users, which can result in a cold start when the system is launched. Furthermore, because it can only consider the inherent relationships within this data, providing recommendations tailored to the specific circumstances of each user requires an unrealistic amount of data.

[0208] For example, in order to recommend kishimen to a user who likes hoto, a sufficient amount of data is needed to show that other users with similar preferences also like kishimen.

[0209] While content-based methods can sometimes represent the similarities between items, it is difficult to represent all similarities with data. For example, the fact that it is noodles and that the raw material is wheat is well represented, but it is difficult to cover the finer details such as the fact that it is flat noodles with data.

[0210] Furthermore, for example, traditional methods for recommending restaurants have involved keyword matching based on preference information, or similar language processing.

[0211] However, in this case, for example, a user who likes curry may receive recommendations that do not suit their situation or preferences because other information is unavailable. For instance, when searching for restaurants while traveling, a chain restaurant may be recommended even if the user wants a local curry restaurant. Also, restaurants that serve dishes containing ingredients that the user cannot eat for religious reasons or allergies may be recommended.

[0212] Furthermore, it is difficult to recommend restaurants by considering features that do not appear in superficial data, such as whether the restaurant is clean or whether the customer's preferred background music is playing.

[0213] In contrast, the information processing system 101 does not require other users' preference data, but instead uses user information and restaurant review information (e.g., word-of-mouth) to recommend restaurants that take complex relationships into account.

[0214] Specifically, the information processing system 101 implements a restaurant recommendation function through three functions: pre-questioning, suggestion, and destination setting.

[0215] For example, preliminary questions are asked according to the user information acquisition process described above, as shown in Figure 3. The user's answers to the preliminary questions are obtained, and the user information is updated based on the obtained answers.

[0216] The pre-submitted questions include, for example, three types of questions: spontaneous questions, contextual questions, and questions asked when requesting restaurant recommendations.

[0217] Spontaneous questions are questions that a character asks spontaneously during a conversation with the user. For example, these might include questions like, "What did you have for lunch?", "What did you eat yesterday?", "Was the ramen you had yesterday delicious?", "What's your favorite food?", and "What's your least favorite food?".

[0218] Contextual questions are questions that a character asks in context during a conversation with the user. For example, when talking about a certain food, a question like "Do you like [food item]?" might be asked, or when talking about a certain restaurant, a question like "Do you like [restaurant name]?" might be asked.

[0219] The questions asked when requesting restaurant recommendations are asked only once when a user requests a restaurant recommendation. For example, possible questions include: "Is there anything you can't eat?", "Is there any food you're currently into?", and "Do you like limited-time offers?".

[0220] For example, the generation unit 182 generates user information useful for recommending restaurants based on the question data about restaurants obtained in step S3 of Figure 3 described above, and the user information stored in the user information storage unit 153.

[0221] Next, the generation unit 182 generates a prompt that instructs the user to select a pre-question from among several candidate questions, based on the context of the conversation with the user and the generated user information.

[0222] Specifically, the prompt includes "role," "user information," and "suggested questions."

[0223] "Role" includes instructions such as, "Consider the user information and context, select one appropriate question from the list of candidates, and ask the user the question."

[0224] "User information" is user information generated based on the question data and user information described above.

[0225] "Potential questions" include, for example, the following questions:

[0226] What did you eat for breakfast, lunch, or dinner this morning? Do you like trendy foods? • Favorite artist

[0227] The generation unit 182 sends the generated prompt to the AI ​​server 113, requests the generation of a question, and retrieves the generated question from the AI ​​server 113.

[0228] Then, using the acquired question text, the user is asked preliminary questions using a character, and the user information is updated based on the user's answers to the preliminary questions. This allows for the accumulation of user information that is useful for recommending restaurants to the user.

[0229] Next, with reference to the flowchart in Figure 6, the restaurant suggestion process performed by the information processing system 101 will be explained.

[0230] This process is executed as needed, for example, during a conversation with a user.

[0231] In step S101, the context of the conversation with the user is analyzed, similar to the process in step S1 in Figure 3. This allows for an understanding of, for example, the user's intentions. The context analysis unit 172 supplies conversation context information, including the context analysis results, to the planning unit 181.

[0232] In step S102, the planning unit 181 determines whether or not to suggest a restaurant based on the conversation context information. If it is determined that a restaurant should be suggested, the process proceeds to step S102.

[0233] For example, if a user says something like, "I want to eat lunch somewhere," "Can you recommend a good restaurant for lunch?", or "Are there any good ramen shops nearby?", the system will determine that it is time to suggest a restaurant.

[0234] In step S103, the planning unit 181 extracts search criteria for restaurants. For example, the planning unit 181 extracts the parameters necessary for searching for restaurant information based on the user's intent, etc., which it has understood.

[0235] In step S104, the planning unit 181 searches for restaurant information. Specifically, the planning unit 181 sets search conditions based on the extracted parameters. Based on the set search conditions, the planning unit 181 searches for restaurants that meet the search conditions from the information stored in the information DB 157 and obtains restaurant information related to the searched restaurants. The planning unit 181 also requests the external service server 112 to search for restaurants using the set search conditions and obtains restaurant information related to the searched restaurants from the external service server 112.

[0236] Restaurant information includes, for example, reviews (customer feedback), business hours, location, map, menu, and waiting time. The restaurant information in Information DB157 and the external service server 112 is updated with the latest information as needed.

[0237] The planning unit 181 stores the acquired restaurant information in the conversation log storage unit 154.

[0238] In step S105, the planning unit 181 filters restaurant information. For example, the planning unit 181 filters restaurant information based on reviews, waiting times, and user information. Here, restaurants that serve dishes that the user dislikes or cannot eat are excluded.

[0239] In step S106, the information processing unit 151 suggests restaurants. For example, the planning unit 181 obtains user information related to restaurant suggestions from the user information storage unit 153. Next, the generation unit 182 determines a recommendation policy for restaurants based on information such as destination, whether it is a holiday or not, and whether it is close to home. For example, a recommendation policy may be determined to prioritize independent restaurants over chain restaurants, or to prioritize restaurants catering to tourists. The planning unit 181 supplies the selected restaurant information, the acquired user information, the recommendation policy information, and the user's status information to the generation unit 182.

[0240] Based on the information obtained from the planning unit 181, the generation unit 182 generates a prompt to instruct the AI ​​server 113 to create a recommendation letter for a restaurant.

[0241] Prompts include, for example, "Instructions," "User Information," "Status," "Policy," and "List of Restaurants."

[0242] An "instruction" is a command to the AI ​​server 113 (for example, LLM). For example, an instruction might be: "You are an AI that recommends the most appropriate restaurant based on the user's information and situation. Using the user information and situation as a reference, select the best restaurant from the list and recommend it while explaining the reason. If you are unable to narrow it down, ask the user additional questions."

[0243] This instructs the AI ​​server 113 to select one restaurant and recommend it along with the reason. If it is unable to narrow down the list of recommended restaurants, it is allowed to ask additional questions.

[0244] "User information" includes, for example, the following information:

[0245] Favorite things: Iekei ramen, tuna, mayonnaise, yakiniku (grilled meat), Chinese food Things I dislike: Dirty restaurants, grilled fish, green beans, bell peppers Age: 35 Gender: Male Place of origin: Fukuoka Home: Saitama City, Saitama Prefecture Episode: I ate ramen with a friend yesterday. I ate yakiniku the day before yesterday. Opinions: Restaurants that advertise huge portions aren't eco-friendly. I could eat yakiniku every day. I like trying new restaurants. I prefer local eateries to major chains.

[0246] "Situation" includes, for example, the following information:

[0247] Current time: 2024 / 11 / 27 10:00 Planned visit date and time: 2024 / 11 / 27 noon Current location: 35.83,139.92 Number of passengers: 1 Purpose of travel: Travel Means of transportation: car

[0248] "Policies" include, for example, the following information:

[0249] Prioritize shops catering to tourists over chain stores. I will prioritize stores with parking lots.

[0250] The "List of Restaurants" contains a list of information about selected restaurants.

[0251] The generation unit 182 sends the generated prompt to the AI ​​server 113, requests the generation of a recommendation letter, and retrieves the generated recommendation letter from the AI ​​server 113.

[0252] The generation unit 182 generates behavioral control information, including the generated recommendation text, and transmits it to the user's edge terminal 114.

[0253] The edge terminal 114 suggests restaurants to the user by having a character speak content that includes recommendations, based on behavioral control information.

[0254] In step S107, similar to the process of step S1 in FIG. 3, the context in the conversation with the user is analyzed. As a result, for example, based on statements indicating the user's reaction to the proposed restaurant, etc., the user's opinion on the proposed restaurant is analyzed. The context analysis unit 172 supplies the conversation context information including the analysis result of the context to the planning unit 181.

[0255] In step S108, the planning unit 181 determines whether a proposal for another restaurant is requested based on the conversation context information. If it is determined that a proposal for another restaurant is requested, the process returns to step S106.

[0256] For example, when the user makes statements such as "What about others?", "Tell me more.", "I'm not in the mood for soba now.", etc., it is determined that a proposal for another restaurant is requested. [[ID=X]]

[0257] [[ID=X]] After that, in step S108, the processes of steps S106 to S108 are repeatedly executed until it is determined that a proposal for another restaurant is not requested.

[0258] On the other hand, in step S108, if it is determined that a proposal for another restaurant is not requested, the process proceeds to step S109.

[0259] In step S109, the planning unit 181 determines whether to set a destination based on the conversation context information. If it is determined to set a destination, the process proceeds to step S110.

[0260] For example, when the user makes statements such as "Let's go there.", "Set the destination.", etc., it is determined to set a destination.

[0261] In step S110, the information processing system 101 executes the destination setting process, and the restaurant proposal process ends.

[0262] Here, we will explain the details of the destination setting process by referring to the flowchart in Figure 7.

[0263] In step S151, the planning unit 181 acquires information about the proposed restaurant as needed. For example, if the planning unit 181 needs information to set the proposed restaurant as the destination, or if it needs additional information to present to the user regarding the proposed restaurant, it acquires the necessary information from the information DB 157 or the external service server 112.

[0264] In step S152, the planning unit 181 sets the destination. For example, the planning unit 181 generates destination setting information that includes information about the restaurant set as the destination (e.g., restaurant name, address, map, etc.). The planning unit 181 supplies the destination setting information to the generation unit 182.

[0265] Furthermore, the planning unit 181 transmits destination setting information to an application program or device that provides navigation and scheduling functions, as needed.

[0266] An application program or device that receives destination setting information sets the restaurant suggested to the user as the destination based on the destination setting information.

[0267] In step S153, the information processing system 101 notifies the user that the destination setting is complete. For example, it sends a prompt containing destination setting information to the AI ​​server 113, requests the generation of a speech statement notifying the user that the destination setting is complete, and retrieves the generated speech statement from the AI ​​server 113.

[0268] The generation unit 182 generates behavior control information including the generated utterance and transmits it to the user's edge terminal 114.

[0269] Based on the behavior control information, the edge terminal 114 notifies the user that the suggested restaurant has been set as the destination by having the character speak.

[0270] After that, the destination setting process is completed.

[0271] Returning to Figure 6, on the other hand, in step S109, if, for example, the user does not instruct to set a destination or does not indicate an intention to go to a suggested restaurant, it is determined that no destination is set, the process in step S110 is skipped, and the restaurant suggestion process ends.

[0272] Furthermore, if it is determined in step S102 that no restaurant should be suggested, the processes in steps S103 to S110 are skipped, and the restaurant suggestion process ends.

[0273] As described above, restaurants are recommended interactively, one by one, along with reasons for recommendation based on the user's situation and preferences. This makes the recommended restaurants more appealing to the user compared to when restaurant search results are displayed as a mechanical list.

[0274] Furthermore, flexible recommendations are achieved that are tailored not only to the user's preferences and tastes, but also to the situation and user characteristics. For example, instead of superficial matching based on the user's favorite categories or foods, it can recommend more appropriate restaurants by considering the user's niche preferences and what they have recently eaten. For instance, restaurants that the user is likely to frequent may be recommended, allowing the user to experience the value of personalization.

[0275] Furthermore, by resolving similarities and relationships that previously could only be inferred from data using language models (e.g., LLM) without using conventional methods such as collaborative filtering, it becomes possible to recommend appropriate restaurants to users even with limited data.

[0276] Specifically, for example, if you enjoyed the hoto noodles you had before, you can then recommend kishimen noodles next time.

[0277] For example, for a user's request such as "Please tell me a recommended restaurant for lunch." at the travel destination, it is prevented that chain restaurants are recommended.

[0278] And, for example, it becomes possible to recommend a local restaurant in Kyoto together with a message such as "There is a restaurant called ○○ Shijo Karasuma Store. Since you came to Kyoto especially, how about a buffet of Kyoto vegetables? You can eat as much fresh vegetables picked in the morning as you like!" This can give the user an impression that high-value-added information is being provided.

[0279] For example, for a user's request such as "I want to have dinner near here, is there any recommendation?" if the user likes ramen, even if the user had ramen for lunch, it is prevented that a ramen restaurant is recommended.

[0280] And, for example, it becomes possible to recommend a Japanese restaurant together with a message such as "Then how about Japanese cuisine ○○? Since you had ramen for lunch, you can have a healthy meal. There are also voices saying that the interior is stylish." This can give the user an impression such as "They are really considering my health!" or "They are matching my hobbies!"

[0281] For example, an appropriate restaurant is proposed based on the user's preferences and information on the restaurant review site.

[0282] For example, assume that the user's preferences are "emphasize the cleanliness of the store" and "want to go to a store that is easy to enter with children". Also, for example, stores A and B are searched on the review site, and the evaluation of store A on the review site is "cleanliness was thorough" and "there was a service for children", and the evaluation of store B is "there was a kids chair" and "the toilet was dirty".

[0283] In this case, for example, store A would be suggested with a message like, "How about store A? It has the cleanliness and child-friendly services that you value."

[0284] For example, a suitable restaurant is suggested based on the user's behavioral history, as well as their current status and plans.

[0285] For example, suppose the user's activity history is "I haven't been able to go to the gym lately because I've been going to a lot of drinking parties," and "I had a rich ramen for dinner last night," and the user's preference is "I value ratings on review sites." Also, suppose the user's current status and schedule are "I have an appointment at 1pm, so I want to grab a quick lunch nearby before then."

[0286] In this case, for example, a soba restaurant might be suggested along with a message like, "You've been to a lot of drinking parties lately, how about some soba? There's a highly-rated restaurant on review sites about 5 minutes from your destination. They also serve food quickly."

[0287] The following is an example of a conversation between a user and a character when using the restaurant recommendation feature.

[0288] User: "Can you recommend a good lunch spot?"

[0289] Character: "I recommend 'Restaurant A'! It has parking and a charming, secluded atmosphere. You can enjoy carefully crafted dishes such as homemade tofu and yuba (tofu skin), making it perfect for adding a special touch to your trip."

[0290] User: "Anything else?"

[0291] Character: "We also recommend 'Restaurant B'! It's very popular with women, and their creamy omelet rice on the lunch menu is highly acclaimed. It's a casual bistro cafe, perfect for a change of pace while traveling."

[0292] User: "I don't like eggs."

[0293] Character: "In that case, let's go back to 'Restaurant A'! You can enjoy fragrant soba noodles and exquisite tofu dishes while avoiding eggs. The calm, traditional Japanese atmosphere is also appealing for lunch."

[0294] User: "I'd like to know about other options as well."

[0295] Character: "How about 'Restaurant C'? They're known for their juicy lamb with no unpleasant odor, perfect for yakiniku lovers like you. It's healthy and satisfying!"

[0296] User: "Are there any others?"

[0297] Character: "How about 'Restaurant D'? It has a parking lot and a great selection of seafood dishes. It's perfect for a relaxed lunch while sightseeing, as you can enjoy a meal in a calm atmosphere!"

[0298] User: "I don't like shrimp."

[0299] Character: "Then how about 'Restaurant E'? It's a stylish bar where you can enjoy pasta and Spanish cuisine. It's perfect for a light lunch and you can also experience something out of the ordinary!"

[0300] User: "Which one do you think is best?"

[0301] Character: "I think 'Restaurant A' is the best choice! You can enjoy a special feeling during your trip, and the gentle flavors of tofu and soba noodles are perfect for lunch. It also has ample parking and is easily accessible."

[0302] In this way, user satisfaction with the characters can be improved.

[0303] <<2. Variant>> The following describes some modifications of the embodiments of the present technology described above.

[0304] For example, there may be multiple instances of the main server 111, the external service server 112, and the AI ​​server 113.

[0305] For example, two or more of the main server 111, external service server 112, and AI server 113 may be integrated. For example, the main server 111 may be equipped with a generative AI such as LLM.

[0306] For example, some or all of the processing of each device in the information processing system 101 may be performed by other devices. For example, some of the processing of the main server 111 described above may be performed by the edge terminal 114.

[0307] This technology is applicable to all devices and systems that can converse with users using virtual characters.

[0308] <<3.B>> <Description of a computer using this technology> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up that software are installed on a computer. Here, a computer includes computers built into dedicated hardware, as well as general-purpose personal computers, for example, that can perform various functions by installing various programs.

[0309] Figure 8 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program.

[0310] In a computer, the processing circuit 1001, ROM (Read Only Memory) 1002, and RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0311] An input / output interface 1005 is further connected to the bus 1004. An input / output interface 1005 is connected to an input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010.

[0312] The input unit 1006 may include physical or virtual operating means that the user operates to input information, such as a keyboard, mouse, or touch panel, as well as means that the user inputs information through voice, eye gaze, etc. Furthermore, the input unit 1006 may include sensors for inputting various physical quantities into the computer. For example, the input unit 1006 may include sensors that acquire physical quantities such as light (including infrared light other than visible light) or sound, such as a camera or microphone. Also, for example, the input unit 1006 may include sensors that acquire other physical quantities such as temperature, moisture content, acceleration, and distance. The output unit 1007 may include means that present information to the user by stimulating the user's perception, such as a display, speaker, or haptic device. The storage unit 1008 is composed of a hard disk, non-volatile or volatile memory, etc., and stores various types of information (including programs). The communication unit 1009 is a network interface, etc., and performs wired or wireless communication with the outside. The drive 1010 drives removable media 1011 such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory.

[0313] The processing circuit 1001 includes a processor that executes programs such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). The processing circuit 1001 (its processor) performs the series of processes described above by loading the program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it. The processing circuit 1001 can output the processing results of the series of processes from the output unit 1007, for example, via the bus 1004 and the input / output interface 1005, as needed. The processing circuit 1001 can also store the processing results in the memory unit 1008 or transmit them from the communication unit 1009.

[0314] The program executed by the computer (processing circuit 1001) can be provided by recording it on a removable medium 1011, such as a package medium. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0315] In a computer, a program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable media 1011 into the drive 1010. Alternatively, a program can be received by the communication unit 1009 from another device, such as a server, via a wired or wireless transmission medium, and installed in the storage unit 1008. Furthermore, programs can be pre-installed in the ROM 1002 or the storage unit 1008.

[0316] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0317] The processes that a computer performs according to a program do not necessarily have to follow the order described in the flowchart. In other words, the processes that a computer performs according to a program include processes that are executed in parallel or individually (e.g., parallel processing and object-based processing).

[0318] A program may be processed by a single computer (processor), or it may be processed in a distributed manner by multiple computers. Furthermore, a program may be transferred to a remote computer and executed there.

[0319] When the above-described series of processes are performed by a computer executing a program, for example, the processing circuit 1001 (processor) functions as the information processing unit 151 in Figure 2 by executing the program.

[0320] In this specification, a system means one component or a collection of multiple components (devices, modules (parts), etc.). Therefore, one or more components of a computer, for example, only the processor, or a combination of a processor and memory (for example, only the processing circuit 1001, or a combination of processing circuit 1001 to bus 1004, etc.), constitute a system. Regarding a collection of multiple components, it is not necessary whether all components reside in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, or a single device containing multiple modules within a single enclosure, are all systems. Furthermore, for example, an entire computer, or a combination of a computer and other devices such as a server (not shown), also constitute a system.

[0321] The components (blocks) of the apparatus illustrated in this specification are functional conceptual blocks, and the actual apparatus does not need to have the illustrated configuration. That is, the apparatus can have any configuration in which the functions of the illustrated components are divided and / or integrated into any unit, for example, a configuration having one block in which the functions of all components are integrated.

[0322] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0323] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0324] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0325] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0326] <Examples of configuration combinations> This technology can also be configured as follows:

[0327] (1) A user analysis unit analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on conversations between the user and a virtual character. Information processing systems that are equipped with these systems. (2) A context analysis unit that analyzes the context of the conversation between the user and the character, Based on the context and the user's characteristics, an action control unit controls the character's actions, including speech. The information processing system described in (1) further comprises the above. (3) The behavior control unit selects news to notify the user using the character, based on the context and the user's characteristics. The information processing system described in (2) above. (4) The behavior control unit controls the notification of weather information by the character based on the context and the user's characteristics. The information processing system described in (2) or (3) above. (5) The behavior control unit extracts candidate combinations of location and date / time for weather information to be notified to the user based on the context and the user's characteristics, and selects the weather information to be notified to the user from among the weather information for each location and date / time combination, based on the location, date / time, and weather of each piece of weather information, the context, and at least one of the user's characteristics. The information processing system described in (4) above. (6) The behavior control unit recommends restaurants using the character based on the context and the user's characteristics. An information processing system as described in any of (2) to (5) above. (7) The behavior control unit causes the character to speak a recommendation statement that includes the restaurant to recommend to the user and the reason for the recommendation. The information processing system described in (6) above. (8) The behavior control unit generates the recommendation text using a language model, as well as prompts containing the user's characteristics, the context, a policy for recommending restaurants, and information about the restaurants. The information processing system described in (7) above. (9) The behavior control unit controls the questions the character asks the user based on the context. The user analysis unit updates user information regarding the user's characteristics based on the user's answers to the questions. An information processing system as described in any of (2) to (8) above. (10) The behavior control unit uses open-ended and closed-ended questions based on the context. The information processing system described in (9) above. (11) The context includes at least one of the following: the context or content of the conversation between the user and the character, the user's situation, the circumstances surrounding the user, and the user's intentions. An information processing system as described in any of (2) to (10) above. (12) The user analysis unit updates some of the user's characteristics in real time based on the context, and updates some of the user's characteristics in batch processing based on the history of conversations between the user and the character. An information processing system as described in any of (2) to (11) above. (13) The user analysis unit uses a language model to extract the user's opinion from the user's statements. An information processing system as described in any of (1) to (12) above. (14) The user analysis unit analyzes the true meaning of the user's statements by analyzing the user's statements step by step using the language model. The information processing system described in (13) above. (15) The user analysis unit analyzes the user's utterances in the following order: the background of the user's speech, the user's feelings, the reason the user spoke, the user's desires, and the user's opinions. The information processing system described in (14) above. (16) The user analysis unit further analyzes user information regarding the user's characteristics using a language model, and performs at least one of the following: resolving inconsistencies in the user information and aggregating information on the same event. An information processing system as described in any of (1) to (15) above. (17) The user characteristics further include at least one of the user's profile, episodes, and preferences. An information processing system as described in any of (1) to (16) above. (18) A user analysis unit analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on conversations between the user and a virtual character. Information processing device. (19) Information processing device, Based on conversations between the user and a virtual character, the system analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and ways of thinking. Information processing methods.

[0328] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur. [Explanation of symbols]

[0329] 101 Information Processing System, 111 Main Server, 112 External Service Server, 113 AI Server, 114-1~114-n Edge Terminals, 151 Information Processing Unit, 161 Behavior Processing Unit, 162 User Analysis Unit, 171 Information Collection Unit, 172 Context Analysis Unit, 173 Behavior Control Unit, 181 Planning Unit, 182 Generation Unit

Claims

1. A user analysis unit analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on conversations between the user and a virtual character. Information processing systems that are equipped with these systems.

2. A context analysis unit that analyzes the context of the conversation between the user and the character, Based on the context and the user's characteristics, an action control unit controls the character's actions, including speech. The information processing system according to claim 1, further comprising:

3. The behavior control unit selects news to notify the user using the character, based on the context and the user's characteristics. The information processing system according to claim 2.

4. The behavior control unit controls the notification of weather information by the character based on the context and the user's characteristics. The information processing system according to claim 2.

5. The behavior control unit extracts candidate combinations of location and date / time for weather information to be notified to the user based on the context and the user's characteristics, and selects the weather information to be notified to the user from among the weather information for each location and date / time combination, based on the location, date / time, and weather of each piece of weather information, the context, and at least one of the user's characteristics. The information processing system according to claim 4.

6. The behavior control unit recommends restaurants using the character based on the context and the user's characteristics. The information processing system according to claim 2.

7. The behavior control unit causes the character to speak a recommendation statement that includes the restaurant to recommend to the user and the reason for the recommendation. The information processing system according to claim 6.

8. The behavior control unit generates the recommendation text using a language model, as well as prompts containing the user's characteristics, the context, a policy for recommending restaurants, and information about the restaurants. The information processing system according to claim 7.

9. The behavior control unit controls the questions the character asks the user based on the context. The user analysis unit updates user information regarding the user's characteristics based on the user's answers to the questions. The information processing system according to claim 2.

10. The behavior control unit uses open-ended and closed-ended questions based on the context. The information processing system according to claim 9.

11. The context includes at least one of the following: the context or content of the conversation between the user and the character, the user's situation, the circumstances surrounding the user, and the user's intentions. The information processing system according to claim 2.

12. The user analysis unit updates some of the user's characteristics in real time based on the context, and updates some of the user's characteristics in batch processing based on the history of conversations between the user and the character. The information processing system according to claim 2.

13. The user analysis unit uses a language model to extract the user's opinion from the user's statements. The information processing system according to claim 1.

14. The user analysis unit analyzes the true meaning of the user's statements by analyzing the user's statements step by step using the language model. The information processing system according to claim 13.

15. The user analysis unit analyzes the user's utterances in the following order: the background of the user's speech, the user's feelings, the reason the user spoke, the user's desires, and the user's opinions. The information processing system according to claim 14.

16. The user analysis unit further analyzes user information regarding the user's characteristics using a language model, and performs at least one of the following: resolving inconsistencies in the user information and aggregating information on the same event. The information processing system according to claim 1.

17. The user characteristics further include at least one of the user's profile, episodes, and preferences. The information processing system according to claim 1.

18. A user analysis unit analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and way of thinking, based on conversations between the user and a virtual character. Information processing device.

19. Information processing device, Based on conversations between the user and a virtual character, the system analyzes the user's characteristics, including at least one of the user's habits, opinions, desires, and ways of thinking. Information processing methods.

Images