Vehicle, information processing system, and computer program product with personalized ai assistant

Abstract

An information processing system includes a user analysis unit configured to analyze a characteristic of a user, the characteristic including at least one of a habit, an opinion, a desire, or a way of thinking of the user, based on a conversation between the user and a character that is virtual. The present technology is applicable to, for example, apparatuses and systems capable of having conversations with users using virtual characters.

Description

VEHICLE, INFORMATION PROCESSING SYSTEM, AND COMPUTER PROGRAM PRODUCT WITH PERSONALIZED AI ASSISTANT

[0001] The present technology relates to an information processing system, an information processing apparatus, and an information processing method, and particularly relates to an information processing system, an information processing apparatus, and an information processing method that realize virtual characters capable of having conversations with users.

[0002] In recent years, artificial intelligence (AI) has been widely applied to vehicles. For example, voice agents that operate devices and the like in response to voice instructions from users have been broadly adopted (for example, see PTL 1).

[0003] WO 2023 / 090057

[0004] However, the technologies for virtual characters capable of having conversations with users, including voice agents, are still under development, leaving room for improvement.

[0005] The present technology has been made in view of such circumstances, and aims to improve the user satisfaction with virtual characters capable of having conversations with users.

[0006] An information processing system according to an aspect of the present technology includes a user analysis unit configured to analyze a characteristic of a user, the characteristic including at least one of a habit, an opinion, a desire, or a way of thinking of the user, based on a conversation between the user and a character that is virtual.

[0007] An information processing apparatus according to an aspect of the present technology includes a user analysis unit configured to analyze a characteristic of a user, the characteristic including at least one of a habit, an opinion, a desire, or a way of thinking of the user, based on a conversation between the user and a character that is virtual.

[0008] An information processing method according to an aspect of the present technology includes, by an information processing apparatus, analyzing a characteristic of a user, the characteristic including at least one of a habit, an opinion, a desire, and a way of thinking of the user, based on a conversation between the user and a character that is virtual.

[0009] In one aspect of the present technology, a characteristic of a user is analyzed, the characteristic including at least one of a habit, an opinion, a desire, and a way of thinking of the user, based on a conversation between the user and a character that is virtual.

[0010] Fig. 1 is a block diagram illustrating an embodiment of an information processing system to which the present technology is applied.Fig. 2 is a block diagram illustrating a configuration example of a main server.Fig. 3 is a flowchart for explaining user information acquisition processing.Fig. 4 is a diagram for explaining news notification processing.Fig. 5 is a flowchart for explaining weather information notification processing.Fig. 6 is a flowchart for explaining restaurant recommendation processing.Fig. 7 is a flowchart for explaining details of destination setting processing.Fig. 8 is a diagram illustrating a configuration example of a computer.

[0011] Hereinafter, embodiments for implementing the present technology will be described. Description will be made in the following order. 0. Background of the Present Technology 1. Embodiment 2. Modifications 3. Others

[0012] 0. Background of the Present Technology First, the background of the present technology will be described.

[0013] Voice agents have been widely adopted in vehicles in recent years as described above.

[0014] Existing voice agents mainly focus on convenient functions of operating devices on behalf of users, following voice instructions from the user.

[0015] In contrast, in the present technology, a character is personalized as the user has conversations with and uses functions of the character. In other words, as the character improves the understanding of the user's characteristics, the character is customized for the user's specifications, making it possible to make proposals, provide information, and execute actions that are more suitable for the user.

[0016] In addition, the character executes an interactive conversation rather than a simple Q&A format, thus constructing an interaction design that users can empathize with.

[0017] This improves the user's satisfaction with the character.

[0018] 1. Embodiment Next, embodiments of the present technology will be described with reference to Figs. 1 to 11.

[0019] Configuration Example of Information Processing System 101 Fig. 1 illustrates an embodiment of an information processing system 101 to which the present technology is applied.

[0020] The information processing system 101 is a system that provides a user with a service (hereinafter referred to as a “character service”) that performs conversations and various functions by using a virtual character. The character is an AI agent configured by a virtual human model.

[0021] The functions provided by the information processing system 101 include, for example, a notification function for weather information, news, and the like, a restaurant recommendation function, a content reproduction function for music and the like, a remote operation function for various devices, and the like.

[0022] The information processing system 101 includes a main server 111, an external service server 112, an AI server 113, and edge terminals 114-1 to 114-n. The main server 111, the external service server 112, the AI server 113, and the edge terminals 114-1 to 114-n are connected to each other via a network 121 such as the Internet.

[0023] Hereinafter, when it is not necessary to distinguish the edge terminal 114-1 to the edge terminal 114-n individually, they are simply referred to as an edge terminal 114.

[0024] The main server 111 is an information processing apparatus that provides a character service to each edge terminal 114. The main server 111 receives user utterance data, character utterance data, user situation data, and surrounding situation data from the edge terminals 114 via the network 121.

[0025] The user utterance data is data indicating utterance content of the user. The character utterance data is data indicating the utterance content of the character actually output in the edge terminals 114. The user situation data is data related to situations of the user. The surrounding situation data is data regarding situations around the user.

[0026] The main server 111 analyzes and understands the characteristics of the user, the context in the conversations between the user and the character (hereinafter referred to as “conversation context”), and the like based on the user utterance data, the character utterance data, and the user situation data.

[0027] The main server 111 acquires various types of information (hereinafter referred to as “external information”) from the external service server 112 via the network 121 by using various types of services provided by the external service server 112 as necessary. The main server 111 transmits a prompt to the AI server 113 and receives data generated by the generative AI based on the prompt (hereinafter referred to as AI-generated data) from the AI server 113 via the network 121 as necessary.

[0028] The main server 111 generates action control information for controlling an action such as utterance of the character based on characteristics of the user, the conversation context, external information, AI-generated data, and the like. The main server 111 transmits the action control data to the edge terminals 114 via the network 121, and controls an action such as utterance of the character on the edge terminals 114. The main server 111 also provides various functions to the user through the character by controlling the action of the character on the edge terminals 114.

[0029] The external service server 112 is a server that provides various services such as a search service, a weather forecast providing service, a news providing service, and a restaurant information providing service.

[0030] The AI server 113 is a server that provides a generative AI such as a large language model (LLM). The AI server 113 may provide multiple types of generative AIs.

[0031] The edge terminal 114 is an apparatus for the user to use a character service. The edge terminal 114 is configured by, for example, a vehicle, an in-vehicle apparatus, a smartphone, a personal computer (PC), or the like. The edge terminal 114 executes a predetermined application program based on the action control information received from the main server 111, for example, thereby executing processing related to the character service.

[0032] Note that, in the following, when the respective apparatuses of the information processing system 101 perform communication via the network 121, the description “via the network” will be omitted. For example, when the main server 111 and the edge terminal 114 communicate with each other via the network 121, it is simply described that the main server 111 and the edge terminal 114 communicate with each other.

[0033] Further, hereinafter, an example in which the edge terminal 114 is used for a vehicle will be mainly described. In this case, the edge terminal 114 may be a vehicle itself, may be an in-vehicle apparatus, or may be an apparatus that is brought into and used in the vehicle.

[0034] Configuration Example of Main Server 111 Fig. 2 shows a configuration example of the main server 111.

[0035] The main server 111 includes an information processing unit 151, a character information storage unit 152, a user information storage unit 153, a conversation log storage unit 154, an action history storage unit 155, a setting information storage unit 156, and an information database (DB) 157.

[0036] The information processing unit 151 includes a processor, for example, a central processing unit (CPU), a graphics processing unit (GPU), or the like and executes processing related to character services. The information processing unit 151 includes an action processing unit 161, a user analysis unit 162, and a communication unit 163.

[0037] The action processing unit 161 executes processing related to actions such as conversations of characters. The action processing unit 161 includes an information collection unit 171, a context analysis unit 172, and an action control unit 173.

[0038] The information collection unit 171 collects information for analyzing and understanding the user's characteristics and conversation context.

[0039] For example, the information collection unit 171 receives the user utterance data, the character utterance data, the user situation data, and the surrounding situation data from each edge terminal 114 via the communication unit 163.

[0040] Note that, for example, the information collection unit 171 may receive voice data indicating the utterance voice of each user from each edge terminal 114, execute voice recognition processing on the voice data, and generate user utterance data indicating the recognized utterance content.

[0041] The user situation data may include, for example, data regarding the user's line-of-sight direction, posture, actions, physical conditions, and the like. The user's actions include, for example, a user operation on the edge terminal 114, character services, and the like. The method for the user operation is not particularly limited, but includes, for example, a manual operation, an operation by a gesture, an operation by a voice command, and the like for a device such as the edge terminal 114. The user situation data may include, for example, image data (moving image data or still image data) obtained by photographing the user.

[0042] The surrounding situation data may include, for example, data regarding the environment around the user and data regarding objects around the user. The data regarding the environment around the user may include, for example, data regarding the current position of the user, the weather nearby, the brightness, the temperature, and the like. The data regarding objects around the user may include, for example, information regarding persons, vehicles, obstacles, structures, traffic lights, traffic signs, and the like around the user. The surrounding situation data may include, for example, image data (moving image data or still image data) obtained by photographing the surroundings of the user.

[0043] For example, when the edge terminal 114 is used in a vehicle, the surrounding situation data may include data regarding situations of the vehicle. For example, the data regarding situations of the vehicle may include data relating to the vehicle type, speed, acceleration, traveling direction, fellow passengers, and the like. For example, the data regarding situations of the vehicle may include image data (moving image data or still image data) obtained by photographing the inside the passenger compartment of the vehicle and image data (moving image data or still image data) obtained by photographing outside of the vehicle.

[0044] For example, the information collection unit 171 acquires setting information related to the user settings from the setting information storage unit 156. The setting information may include, for example, information related to settings such as the character, language, and time zone used by the user as a default.

[0045] For example, the information collection unit 171 acquires user information related to the characteristics of the user from the user information storage unit 153.

[0046] The user information may include, for example, information regarding objective characteristics of the user and information regarding inner feelings of the user. Information regarding objective characteristics of the user may include, for example, characteristics of the appearance, profiles, episodes, and the like of the user. The user's episodes may include, for example, information about events the user has experienced and the user's schedule. The information regarding inner feelings of the user may include, for example, information regarding habits, opinions, desires, the way of thinking, preferences, and the like of the user.

[0047] The habits are, for example, a pattern of behavior or thinking that the user repeatedly performs.

[0048] The opinions are, for example, thoughts or feelings of the user, including principles, assertions, beliefs, subjectivity, viewpoints, values, emotions, and the like.

[0049] The desires are, for example, physiological or psychological needs or desires of the user.

[0050] The way of thinking is, for example, the direction of thinking and tendency of thinking of the user, including approaches, viewpoints, and the way of working on things.

[0051] The preferences represent, for example, a user's tastes, interests, or concerns.

[0052] For example, the information collection unit 171 acquires character information related to the characters having conversations with the user from the character information storage unit 152.

[0053] The character information includes, for example, personal data representing the personality of each character, image data of each character, and the like.

[0054] For example, the information collection unit 171 acquires conversation logs regarding ongoing conversations between the user and the characters from the conversation log storage unit 154.

[0055] The conversation logs are logs related to ongoing conversations between the user and the characters. The conversation logs include, for example, a conversation history related to the ongoing conversations, conversation context information, and various types of information being used or having been used for the conversations.

[0056] The conversation context information is information including analysis results of conversation context. The conversation context includes, for example, context or content in the conversations between the user and the characters, the user’s intentions, situations of the user, situations around the user, and the like.

[0057] The information collection unit 171 supplies necessary information among the user utterance data, the character utterance data, the user situation data, the surrounding situation data, the user information, the character information, and the conversation logs to the context analysis unit 172 and the user analysis unit 162.

[0058] The context analysis unit 172 analyzes and understands the conversation context based on the information acquired from the information collection unit 171. The context analysis unit 172 supplies conversation context information including the analysis results of the conversation context to a planning unit 181.

[0059] The context analysis unit 172 generates or updates conversation logs and stores the conversation logs in the conversation log storage unit 154. For example, the context analysis unit 172 updates a conversation log regarding an ongoing conversation of the user by adding a part or all of the conversation context information, the user utterance data, and the character utterance data to the conversation log.

[0060] The context analysis unit generates or updates the action history data of the user, and stores the action history data in the action history storage unit 155. For example, the context analysis unit updates the action history data by adding a part or all of the conversation context information, the user utterance data, and the character utterance data to the action history data of the user.

[0061] The action history data includes, for example, a conversation history of conversations between the user and the character from when the use of the character service started, and all or part of conversation context information in the conversations between the user and the character.

[0062] The action control unit 173 controls an action including an utterance of a character in the edge terminal 114 of the user based on the conversation context information, the user information, and the character information. The action control unit 173 includes the planning unit 181 and a generation unit 182.

[0063] The planning unit 181 acquires the character information from the character information storage unit 152. The planning unit 181 acquires the user information from the user information storage unit 153. The planning unit 181 acquires information necessary for planning the action of the character from an information DB 157 as necessary. The planning unit 181 acquires external information necessary for planning the action of the character from the external service server 112 via the communication unit 163 as necessary.

[0064] The planning unit 181 plans actions of the character based on the conversation context information, the user information, the character information, and information acquired from the information DB 157 and the external service server 112 as necessary. For example, the planning unit 181 supplies action plan information indicating an action plan of the character to the generation unit 182.

[0065] The planning unit 181 adds a part or all of the information acquired from the information DB 157 and the external service server 112 to a conversation log of an ongoing conversation of the user stored in the conversation log storage unit 154 as necessary.

[0066] The generation unit 182 acquires the character information from the character information storage unit 152. The generation unit 182 acquires the user information from the user information storage unit 153. The generation unit 182 generates action control information based on the action plan information, the user information, and the character information while using the AI server 113 via the communication unit 163 as necessary.

[0067] The action control information is information for controlling the execution of an action of the character planned by the planning unit 181. The action control information includes, for example, information related to a character to be controlled, the content of an utterance of the character, movement of the character in images displayed on the display, and the like.

[0068] The generation unit 182 transmits the action control information to the edge terminal 114 via the communication unit 163, and controls actions of the character in the edge terminal 114.

[0069] The user analysis unit 162 analyzes the characteristics of the user based on the information supplied from the information collection unit 171 and the action history data of each user stored in the action history storage unit 155 while using the AI server 113 via the communication unit 163 as necessary. The user analysis unit 162 generates or updates user information of each user based on the analysis result of the characteristics of the user, and stores the user information in the user information storage unit 153.

[0070] The communication unit 163 communicates with the external service server 112, the AI server 113, and each edge terminal 114.

[0071] The user information storage unit 153 stores user information of each user.

[0072] The character information storage unit 152 stores character information of each character.

[0073] The conversation log storage unit 154 stores conversation logs related to ongoing conversations of each user.

[0074] The action history storage unit 155 stores action history data of each user.

[0075] The setting information storage unit 156 stores setting information of each user.

[0076] The information DB 157 stores various types of information used in conversations of each user.

[0077] Note that, hereinafter, when each unit of the main server 111 communicates with an external device via the communication unit 163, the description of “via the communication unit 163” will be omitted. For example, when the generation unit 182 communicates with the AI server 113 via the communication unit 163, it is simply described that the generation unit 182 communicates with the AI server 113.

[0078] Management Method for User Information Next, a management method for acquiring and updating user information will be described.

[0079] Update Method for User Information First, an update method for user information will be described.

[0080] For example, the user analysis unit 162 updates a part of the user information in real time based on one or more of the user utterance data, the character utterance data, the user situation data, the surrounding situation data, and the conversation log supplied from the information collection unit 171. Specifically, for example, the user analysis unit 162 updates information related to a profile, an episode, a preference, an opinion, and a desire of the user in the user information in real time based on a conversation context (a context or content of a conversation, a situation of the user, a situation around the user, or the like).

[0081] For example, the profile of the user acquired from direct expression in an utterance (utterance content) of the user is stored in a key-value format without determining an item in advance. The key and the value of the profile are expressed in natural language. For example, a value “canned fruit” is registered for a key “food that should not be allowed”, and a value “grande” is registered for a key “name of a pet iguana”.

[0082] This increases the flexibility of the profiles that can be stored, compared to a case in which the profiles of the users are stored in advance using fixed slots.

[0083] For example, the user analysis unit 162 updates a part of the user information of each user through batch processing at a predetermined timing (for example, a predetermined time at midnight every day) based on the action history data of each user stored in the action history storage unit 155. For example, the user analysis unit 162 updates the user's habit, the way of thinking, the summary of a conversation in a day, and the like at a predetermined timing.

[0084] For example, the user's habit is obtained by aggregating the user's episodes (events and schedules) and analyzing the trend inductively. Note that the user's habit may include a use tendency of the function of the character service (for example, a use frequency of each function, a time slot in which each function is used, a use method, and the like).

[0085] Analysis Method for User’s Utterance Next, a method for analyzing a user’s utterance will be described.

[0086] For example, the user analysis unit 162 analyzes the user’s utterance in a step-by-step manner, using the AI server 113 (for example, LLM) by using a prompt including the following steps 1 to 5. Accordingly, the user analysis unit 162 extracts not only superficial information but also the user's fundamental desires, opinions, and the like.

[0087] Write the meaning of the utterance of the partner in the next step. Step 1: The background of the utterance. What is the situation? Step 2: What is the feeling of the partner? Step 3: Why did the partner make that utterance? Step 4: What is the partner's underlying desire? Step 5: Extract the principle, assertion, and opinion that the partner has constantly in mind.

[0088] The partner in these prompts is the user.

[0089] In step 1, the background of the user's utterance is analyzed. In step 2, the user's feeling is analyzed. In step 3, the reason that the user has made the utterance is analyzed. In step 4, the (underlying) desires of the user are analyzed. In step 5, the (constant) opinion of the user is extracted. Note that the extracted user's opinion includes the user's principle and assertion.

[0090] Furthermore, for example, the user analysis unit 162 can continuously deepen the understanding of the user by adding the user information to the prompts as follows.

[0091] #Indication Write the meaning of the utterance of the partner in the next step with reference to supplementary information. #Supplementary information * User information ** User's name Katsutoshi ** User's opinion *** The user values family events and creating memories. On the other hand, the user has a principle of carefully acting in anticipation of economic reality. The user holds the value of avoiding unnecessary expenses. ** User's experience and schedule *** The user was the captain of the Kendo club when he was a junior high school student. *** The user got into trouble with alcohol when he was young. *** There were no problems that the user couldn’t solve in physics. *** The user ate Big Mac yesterday. ** User’s habits *** The user brews his own coffee in the morning. ** User's likes *** Belgian beer ** User's dislikes *** Canned fruit ** Other Information of User *** Family composition: one wife and two children *** Type of pet dog: Pomeranian *** Hobbies: mathematical foundations, creating games. Step 1: The background of the utterance. What is the situation? Step 2: What is the feeling of the partner? Step 3: Why did the partner make that utterance? Step 4: What is the partner's underlying desire? Step 5: Extract the principle, assertion, and opinion that the partner has constantly in mind.

[0092] Here, a specific example of a method for analyzing the user’s utterance will be described.

[0093] For example, first, when a user says, “The drinking party is set for 20:00, but I just cannot wait at all”, the utterance is analyzed along the following steps 1 to 5.

[0094] Step 1: Probably, the user is scheduled to join the drinking party from 20:00, and it seems, at this moment, that there is still a considerable amount of time before that time.

[0095] Step 2: The user has a strong feeling of looking forward to the drinking party, but also has a feeling of impatience and excitement.

[0096] Step 3: The user wanted to express the feeling of impatience caused by the time that the user has to wait, and by sharing this with others, hoped to ease his own feelings and gain their empathy.

[0097] Step 4: Desire to spend the amusing time as soon as possible. In addition, the user wants to share the feeling of excitement with someone and receive their resonance.

[0098] Step 5: A character of having strong expectations for any scheduled pleasant events. The user is likely to be easily impatient when waiting for a long time. The user tends to express emotions and feelings straightforwardly.

[0099] Next, when the user says, “I really want to go home and see my kid and dog”, the utterance is analyzed along the following steps 1 to 5.

[0100] Step 1: Although the user has a schedule to join a drinking party, the user is strongly attached to the time to spent with the user’s family and pet in the user’s mind, and the user feels that the user rather wants to go home. However, there is a possibility that the user joins the drinking party due to expectations or schedules from people around the user.

[0101] Step 2: Feelings of seeking love for the user’s family and dog and healing time at home. Meanwhile, it is possible that the user may be feeling slightly guilty or hesitant about being at the drinking party.

[0102] Step 3: Even though the user appears to be enjoying the drinking party, the user wants to encourage empathy and awareness by revealing a little of the user’s true feelings that the user wants to prioritize the time with the user’s family and pet.

[0103] Step 4: The user wants to feel secure and happy by spending time with the user’s loved family and pets. The user wants anyone to understand the user’s true feelings.

[0104] Step 5: The user values time with the user’s family members and those close to the user. The user is sometimes indecisive between socially required plans and obligations and the user's true feelings and desires. The user tends to worry about how others would perceive the user’s decision even though the user wants to express the user’s feelings straightforwardly.

[0105] Next, when the user says “Well, in reality, being with them is a bit of a hassle”, the utterance is analyzed along the following steps 1 to 5.

[0106] Step 1: Even though the user feels affection for the user’s family and dog, caring for them actually requires effort and can be burdensome. The user makes comments about the gap with a light joke.

[0107] Step 2: Complex feelings that the user sometimes feels that the user's family and dog are annoyed even though the user cares for them very much. While self-deprecating about that contradiction, the user shares it as an interesting topic of conversation.

[0108] Step 3: Because the user has affection, the user lightly makes fun of the user’s own situation, accepting the reality that it takes effort, in order to draw out the other person's empathy and laughter.

[0109] Step 4: Even though the user feels affection, the user wants someone to understand the practical burden. A desire to acknowledge the user's conflicting feelings while lightly shedding them off.

[0110] Step 5: The user tends to take an honest attitude toward the practical burdens and labors despite having deep love. The user is good at sharing the user’s feelings through laughter and light sarcasm. Characteristics of perceiving human relations and daily contradictions with humor, without taking them too seriously.

[0111] Note that the order of the steps for analyzing the user's utterance in a step-by-step manner may be changed. Steps may also be added, modified, or deleted as appropriate.

[0112] In this manner, by analyzing the user’s true intention, for example, it is possible to extract the user's opinion (for example, the principles and assertions) from the user's utterance and store the user's opinion as user information. In addition, for example, the generation unit 182 can utilize the stored user’s opinions in conversation, and generate utterance sentences indicating empathy or proposals after understanding the user using the AI server 113.

[0113] For example, based on a user’s utterance “Drinking coffee calms me down, you know”, the user's opinion “Drinking coffee calms me down” is extracted and stored. Based on this information, for example, in response to the user's utterance “I'm getting nervous. I need to calm down”, an utterance sentence “How about drinking coffee to calm down?” is generated and uttered by the character.

[0114] For example, based on a user's utterance “bathing blows off my fatigue”, the user's opinion “taking a bath relieves fatigue” is extracted and stored. Based on this information, for example, in response to the user's utterance “Today’s conference was very tiresome”, an utterance sentence “You worked hard. How about taking a bath to relieve your fatigue?” is generated and uttered by the character.

[0115] For example, based on the user's utterance “The meeting should begin by clearly stating its purpose at the start”, the user's opinion “The purpose of the meeting should be clarified” is extracted and stored. Based on this information, for example, in response to the user's utterance “What was today's meeting for?”, an utterance sentence “We should have clearly stated the purpose at the beginning” is generated and uttered by the character.

[0116] For example, based on the user's utterance “People should go to bed early at night”, the user's opinion “It is good to go to bed early at night” is extracted and stored. Based on this information, for example, in response to the user's utterance “I'll try to work a little harder”, an utterance sentence “You’d better go to bed early today and do the work again from tomorrow morning” is generated and uttered by the character.

[0117] For example, based on the user's utterance “Karaoke is the best for stress relief”, the user's opinion “Karaoke is the best option for stress relief” is extracted and stored. Based on this information, for example, in response to the user's utterance “I’ve been feeling stressed lately”, an utterance sentence “In that case, let's invite a friend to karaoke and relieve some stress!” is generated and uttered by the character.

[0118] Method for Modifying and Aggregating User Information Next, a method for modifying and aggregating user information will be described.

[0119] For example, the user analysis unit 162 resolves the contradiction of the user information by using the AI server 113 (for example, LLM). For example, when there are two pieces of information, “I love Ramen” and “I hate Ramen” at the same time, the user analysis unit 162 detects a contradiction between the two using the AI server 113 and adopts the newer information.

[0120] For example, the user analysis unit 162 inputs a list of things that the user likes and dislikes to the AI server 113 together with the following prompt, and detects contradictory information.

[0121] Find all combinations of likes and dislikes where the content is contradictory. Determine that a case that matches any one of the following cases is contradictory. In a case where one likes and dislikes the same object: Example) Like: Ramen, Dislike: Ramen In a case where one likes an object and dislikes something included in that category: Example) Like: Ramen, Dislike: noodles In a case where one dislikes an object and likes something included in that category: Example) Like: noodles, Dislike: Ramen

[0122] For example, the user analysis unit 162 aggregates distributed user information by using the AI server 113 (for example, LLM). For example, when there are a plurality of pieces of information related to the same event, the user analysis unit 162 integrates the plurality of pieces of information into one piece to generate new information.

[0123] For example, the user analysis unit 162 inputs a list of user events to the AI server 113 together with the following prompts to aggregate information about the same event.

[0124] #Indication Output according to the following steps. Step 1: Cluster events by those relating to the same event. Step 2: Summarize the events in the same cluster, and organize them as a series of single events. Step 3: Output them in the following format. Enter a new line if there are a plurality of lines. When: XX, Content: XX

[0125] In the output format of step 3 of this prompt, the description of the specific format for time stamps is omitted.

[0126] For example, the following list of events is input into the AI server 113 together with the following prompts. Note that the specific time stamps of each event are not shown.

[0127] When: weekend, Content: I am going to travel to Shizuoka. When: today, Content: I ate Ramen. When: today, Content: Ramen was delicious. When: today, Content: I took a walk in the park. When: today, Content: There was a conference at the company. When: today, Content: The fermented bamboo tasted strange. When: today, Content: I visited the park. When: today, Content: The soup was quite spicy. When: three days ago, Content: I played soccer with friends. When: three days ago, Content: He scored a hat trick. When: yesterday, Content: I went shopping. When: yesterday, Content: I bought Kamaboko and a bicycle. When: yesterday, Content: I enjoyed shopping.

[0128] In this regard, a list in which the same events are aggregated is output as follows. Note that the specific time stamps for the events are not described.

[0129] When: weekend, content: I am going to travel to Shizuoka. When: today, content: I ate Ramen and it was delicious, but the fermented bamboo tasted strange and the soup was quite spicy. When: today, content: I took a walk in the park. When: today, content: There was a conference at the company. When: three days ago, content: I played soccer with friends and had a hat trick. When: yesterday, content: I went shopping and bought Kamaboko and a bicycle. I enjoyed shopping.

[0130] Further, for example, the user analysis unit 162 can aggregate similar information and information having an inclusion relationship by using the AI server 113 (for example, LLM). For example, when there are pieces of information having a difference in detailed expression or an inclusion relationship, the user analysis unit 162 can generate new information by integrating the pieces of information into one.

[0131] Strategy for Acquiring User Information Next, strategies for acquiring user information will be described.

[0132] In order to provide appropriate news, weather, content, and the like to a user, it is necessary to collect effective user information in advance. However, it is difficult to collect effective user information only by collecting user information in usual conversations with the user.

[0133] In contrast, the information processing system 101 collects effective user information by asking strategic questions to the user.

[0134] Here, a user information acquisition processing performed by the information processing system 101 will be described with reference to the flowchart of Fig. 3.

[0135] This processing is performed as appropriate, for example, during a conversation with the user.

[0136] In step S1, the information processing system 101 analyzes the context in a conversation with the user.

[0137] For example, the information collection unit 171 receives user utterance data, character utterance data, user situation data, and surrounding situation data from the edge terminal 114 of the user during the conversation. For example, the information collection unit 171 acquires setting information related to the user from the setting information storage unit 156. For example, the information collection unit 171 acquires user information related to the user from the user information storage unit 153. The information collection unit 171 acquires character information related to the character having the conversation with the user from the character information storage unit 152. For example, the information collection unit 171 acquires the conversation log regarding the ongoing conversation between the user and the character from the conversation log storage unit 154.

[0138] Note that the information collection unit 171 does not necessarily need to acquire all the above-described information, and may acquire only necessary information.

[0139] The information collection unit 171 supplies information necessary for the conversation context among the acquired information to the context analysis unit 172.

[0140] The context analysis unit 172 analyzes and understands the conversation context based on the information acquired from the information collection unit 171. The context analysis unit 172 supplies conversation context information including the analysis results of the conversation context to a planning unit 181.

[0141] In step S2, the planning unit 181 determines whether to ask a question based on the conversation context information. For example, when it is a timing at which the planning unit 181 may ask a question proactively or when it is a timing at which the planning unit 181 switches the topic, the planning unit 181 determines to ask a question, and the processing proceeds to step S3. For example, when there is information that needs to be acquired regarding the topic for the current conversation, the planning unit 181 determines to ask a question, and the processing proceeds to step S3.

[0142] In step S3, the planning unit 181 acquires question data.

[0143] For example, question data is stored in the information DB 157. The question data includes, for example, effective question items whose application has been intended, and components necessary for causing the LLM to generate question sentences are defined.

[0144] Specifically, the question data includes, for example, “item”, “the way of asking a question and acquisition”, and “application”.

[0145] The “item” indicates an item of information acquired by a question. For example, the “item” includes “favorite artist”, “favorite sports player”, “favorite entertainment genre”, “favorite comedian”, “frequently used media”, “occupation”, “stronghold”, “favorite sports team”, and the like.

[0146] The “way of asking a question and acquisition” indicates a way of asking a question to acquire information registered in the “item” or an information acquisition method.

[0147] Specifically, for example, for the item “favorite artist”, the following information is registered: “Who is your favorite artist?”.

[0148] For example, for the item “favorite entertainment genre”, the following information is registered: “Do you like comedy?”, “Do you like games?”, “Do you like movies?”, “Do you like comics?”, or “Do you like animations?”. In addition, for the item “favorite entertainment genre”, the following information is registered: “Because asking “What is your favorite entertainment genre?” returned no expected results, I need to use closed questions”.

[0149] For example, for the item “Who is your favorite comedian?”, the following information is registered: “Who is your favorite comedian (limited to persons who like comedy)?” and “Do you like comedian A (limited to persons who like comedy; in the context of comedian A)?”.

[0150] For example, for the item “frequently used media”, the following information is registered: “Do you often watch YouTube (registered trademark)?”, “Do you often watch TikTok?”, “Do you often watch TV?”, or “Do you listen to the radio?”.

[0151] For example, for the item “occupation”, the following information is registered: “Acquired from the profile?”.

[0152] For example, for the item “stronghold”, the following information is registered: “Acquired from the profile?”.

[0153] For example, for the item “favorite sports team”, the following information is registered: “Which team (player) do you root for?”.

[0154] The “application” indicates the application of the acquired information. For example, the “application” includes the type of function (for example, “news”, “weather”, “music”, or the like) for which the acquired information is used.

[0155] The planning unit 181 selects an item to be asked based on the current conversation context and the user information of the user in the conversation stored in the user information storage unit 153, and acquires question data about the selected item from the information DB 157. The planning unit 181 supplies the acquired question data to the generation unit 182.

[0156] In step S4, the information processing system 101 asks the user a question. For example, the generation unit 182 transmits a prompt including the question data to the AI server 113, requests the generation of a question sentence, and acquires the generated question sentence from the AI server 113.

[0157] The generation unit 182 generates action control information including the generated question sentence and transmits the action control information to the edge terminal 114 of the user.

[0158] The edge terminal 114 asks the user a question by causing the character to utter based on the action control information.

[0159] The question to the user includes an open question that can be answered in a free form and a closed question to answer from among limited options.

[0160] The open question asks about, for example, favorite artist, favorite sport, occupation, place of origin, and the like. The open question is used, for example, when the character proactively asks a question or changes the topic.

[0161] The closed question asks, taking the context of the conversation with the user into consideration, whether the user likes or dislikes a specific item that is a topic for the conversation, whether the user is interested in the item, and the like. For example, when the topic is restaurant A (or another facility), the question asked is “Do you like restaurant A?”. For example, when election news is reported, the question asked is “Are you interested in the election?”.

[0162] For example, with respect to information that is hard to acquire in open questions like “What is the book you want to read?”, a plurality of pieces of information about that book that the user wants to read over the mid- or long-term can be acquired by repeating a closed question that is easy to answer like “Do you want to read OO?”.

[0163] In this way, questions to the user made by the character are controlled based on the conversation context and the like, and open questions and closed questions are selectively used.

[0164] In step S5, the information processing system 101 acquires the answers from the user.

[0165] For example, the user answers in response to questions from the character.

[0166] In response to this, the edge terminal 114 executes a voice recognition processing on the utterance voice representing the user's answer, and generates user utterance data indicating the recognized utterance content. The edge terminal 114 transmits character utterance data indicating the question content of the character and user utterance data indicating the answer content of the user to the main server 111.

[0167] The information collection unit 171 of the main server 111 receives the character utterance data and the user utterance data from the edge terminal 114. The information collection unit 171 supplies the character utterance data and the user utterance data to the user analysis unit 162.

[0168] In step S6, the user analysis unit 162 updates the user information. For example, the user analysis unit 162 extracts the characteristics of the user based on the character utterance data and the user utterance data, that is, based on the answer content of the user to the question of the character. The user analysis unit 162 updates the user information of the user in conversations stored in the user information storage unit 153 based on the extracted characteristics of the user.

[0169] Then, the user information acquisition processing ends.

[0170] On the other hand, in step S2, when it is determined that no question should be asked, the processing of step S2 to step S6 is skipped, and the user information acquisition processing ends.

[0171] News Reporting Function Next, a news reporting function provided by the information processing system 101 will be described.

[0172] For example, the information processing system 101 can report news that may be valuable or of interest to the user based on the conversation context (for example, the content of the user's utterance) and the user information even if the user does not directly give a specific instruction during conversations with the user.

[0173] For example, during a conversation with the user, the planning unit 181 generates a news query based on the conversation context and the user information, as illustrated in Fig. 4.

[0174] Specifically, for example, the planning unit 181 extracts intent parameters indicating the user’s intention based on the content of the user’s utterance included in the conversation context and the use tendency of the function of the character service included in the user information. The planning unit 181 extracts keywords from the intent parameters and generates an intent query.

[0175] The planning unit 181 extracts keywords based on profiles, episodes, opinions, and habits included in the user information.

[0176] The planning unit 181 generates a news query based on the intent query, the user's favorites included in the user information, and query candidates including the keywords. For example, the planning unit 181 transmits a news query to the external service server 112 and acquires news information indicating news retrieved based on the news query from the external service server 112. The planning unit 181 supplies the acquired news information to the generation unit 182.

[0177] For example, the generation unit 182 transmits a prompt including news information to the AI server 113, requests the generation of an utterance sentence for reporting the news, and acquires the generated utterance sentence from the AI server 113. The generation unit 182 generates action control information including the generated utterance sentence and transmits the action control information to the edge terminal 114 of the user.

[0178] The edge terminal 114 reports the news to the user by causing the character to utter based on the action control information.

[0179] Accordingly, for example, news that is likely to be of interest or concern to the user is preferentially reported to the user from among the retrieved news articles.

[0180] Weather Information Reporting Function Next, a weather information reporting function provided by the information processing system 101 will be described.

[0181] Here, a weather information notification processing performed by the information processing system 101 will be described with reference to the flowchart of Fig. 5.

[0182] This processing is performed as appropriate, for example, during a conversation with the user.

[0183] In step S51, the context of the conversation with the user is analyzed, as in the processing of step S1 in Fig. 3. The context analysis unit 172 supplies conversation context information including the analysis results of the conversation context to a planning unit 181.

[0184] In step S52, the planning unit 181 determines whether to report weather information based on the conversation context and user information. For example, the planning unit 181 determines whether to report the weather information based not only on the user’s intention to know the weather information but also on the schedule, the destination, and the like of the user in the context of the conversation with the user. Note that, for example, the planning unit 181 may determine whether to report the weather information using the AI server 113 (for example, LLM). Then, when it is determined that the weather information is to be reported, the processing proceeds to step S53.

[0185] In step S53, the planning unit 181 acquires the weather information. For example, the planning unit 181 extracts a combination of the place and date and time at which the weather information was acquired, based on the user information.

[0186] Specifically, for example, a combination of a place and a date and time is extracted based on a schedule of the user. For example, when there is a schedule of a travel to Atami on the weekend, a combination of [Atami, weekend] is extracted. For example, when there is a schedule of a drinking party in Shinagawa on the day after tomorrow, a combination of [Shinagawa, the day after tomorrow] is extracted.

[0187] For example, a combination of the place and the date and time is extracted based on information about the latest action or location of the user. For example, combinations such as [around the home, today], [around the home, tomorrow], [around the workplace, today], and [around the workplace, tomorrow] are extracted.

[0188] For example, a combination of the place and date and time is extracted based on a region related to the user. For example, combinations such as [around the house, today], [around the house, tomorrow], [family address, today], [family address, tomorrow], [place of origin, today]), and [place of origin, tomorrow] are extracted.

[0189] The planning unit 181 acquires the weather information of each location and date and time from the external service server 112.

[0190] In step S54, the planning unit 181 selects the weather information to be reported. Specifically, the planning unit 181 calculates the score for each piece of weather information based on the location, date and time, weather, and the like of the acquired weather information.

[0191] For example, the score of each weather information is calculated based on five scores of the influence level, the place relevance level, the weather date and time score, the report appropriateness level, and the importance level.

[0192] The influence level indicates the level of influence of the predicted weather on the user. For example, when bad weather such as a typhoon, heavy rain, heavy snow, or strong wind is forecast, the influence level increases. In addition, the higher the predicted severity of bad weather, the greater the influence level.

[0193] The place relevance level indicates the relevance level of the user with respect to the place for which weather is forecast. For example, the higher the relevance between the place for which weather has been forecast and the place related to the user's current location, home, office, family address, or future schedule, the higher the place relevance level.

[0194] The weather date and time score indicates a score for the date and time for which weather has been forecast. For example, the weather date and time score increases as the date and time for which weather has been forecast gets closer to the present.

[0195] The report appropriateness level indicates, for example, an appropriateness level of target weather information reported at the current time point. For example, the report appropriateness level of today's weather information is higher before commuting than before going to bed. For example, the report appropriateness level of tomorrow's weather information is higher before going to bed than before commuting.

[0196] The importance level indicates the level of importance of the user with respect to the target weather information. For example, the importance level is calculated based on action patterns, schedules, interests, health conditions, and the like of the user. For example, in a case that a user's action at the date and time for which weather has been forecast is easily affected by the weather, the importance level is high. For example, when the user is scheduled for outdoor actions on a date and time for which weather has been forecast, the importance level is high.

[0197] For example, the planning unit 181 selects weather information having the highest score as weather information to be reported to the user. The planning unit 181 supplies the selected weather information to the generation unit 182.

[0198] In step S55, the information processing system 101 reports the weather information. For example, the generation unit 182 transmits a prompt including weather information to the AI server 113, requests the generation of an utterance sentence for reporting the weather information, and acquires the generated utterance sentence from the AI server 113.

[0199] The generation unit 182 generates action control information including the generated utterance sentence and transmits the action control information to the edge terminal 114 of the user.

[0200] The edge terminal 114 reports the weather information to the user by causing the character to utter based on the action control information.

[0201] Accordingly, for example, during a conversation with the user, weather information consistent with the context is reported.

[0202] Specifically, the weather information necessary for the user is proactively reported. For example, weather information as follows is reported: “Toyama, your hometown, seems to be getting heavy snow today”, “Tomorrow seems to be sunny, so you can run”, “Tomorrow seems to have a lot of pollen, so be careful”, “It might be rainy at the camp the next week”, and the like.

[0203] For example, the weather information is proactively and periodically reported in accordance with the user's tendency to use the weather information. For example, weather information as follows is reported: “Today's weather is sunny and sometimes cloudy”, “Tomorrow's weather is rainy. Remember your umbrella”, and the like.

[0204] On the other hand, when it is determined in step S1 that the weather information is not to be reported, the processing of step S52 to step S55 is skipped, and the weather information notification processing ends.

[0205] Restaurant Recommendation Function Next, a restaurant recommendation function provided by the information processing system 101 will be described.

[0206] For example, collaborative filtering has been used for existing restaurant recommendation processing.

[0207] When the collaborative filtering is used, use tendency data and preference data of many users are required, and a cold start may occur at the start of a system operation. In addition, since only the relevance that the data has cannot be taken into consideration, an unrealistic amount of data is required to make a recommendation in accordance with the detailed circumstances of the user.

[0208] For example, in order to recommend Kishimen Noodles to users who like Hoto Noodles, a sufficient amount of data indicating that other users having similar tendencies prefer Kishimen Noodles is required.

[0209] While content-based approaches may be used to express similarities between items, it is difficult to represent all similarities in data. For example, although the nature of noodles and raw materials such as wheat can be well expressed, it is difficult to cover even detailed features of flat noodles with data.

[0210] In addition, for example, existing processing of recommending a restaurant has been performed using keyword matching based on preference information or using similar language processing.

[0211] However, in this case, for example, a recommendation that does not match the situation or principle of a user may be made because other information cannot be used for users who like curry. For example, when a user searches for a restaurant in a travel destination, there is concern that a chain shop may be recommended even though the user desires a local curry shop. In addition, there is a concern that a restaurant that serves food containing an ingredient that the person cannot eat due to religious restrictions, allergies, or other reasons might be recommended.

[0212] In addition, it is difficult to recommend restaurants in consideration of features that do not appear as superficial data, such as whether the restaurant is clean, whether favorite background music (BGM) is being played, and the like.

[0213] In contrast, the information processing system 101 realizes recommendation of restaurants in consideration of complicated relevance using user information, restaurant review information (for example, customer feedback), and the like without requiring preference data of other users.

[0214] Specifically, the information processing system 101 realizes the restaurant recommendation function with three functions of preliminary questions, proposals, and destination settings.

[0215] The preliminary questions are made, for example, according to the user information acquisition processing described above with reference to Fig. 3, and the user's answer to the preliminary question is acquired, and the user information is updated based on the acquired answer.

[0216] The preliminary questions include, for example, three types of questions, namely, proactive questions, context-based questions, and questions triggered by restaurant recommendation requests.

[0217] The proactive questions are questions that the character proactively asks during conversations with the user. For example, assumable questions are: “What did you have for lunch?”, “What did you eat yesterday?”, “Was the Ramen you had yesterday delicious?”, “What is your favorite food?”, “What is your least favorite food?”, and the like.

[0218] The context-based questions are questions that the character asks in accordance with the context during conversations with the user. For example, examples of assumable questions include “Do you like XX?” with respect to a certain food when they are talking about the food, and “Do you love to go to XX?” with respect to a certain restaurant when they are talking about the restaurant.

[0219] A question made at the time of requesting recommendation of a restaurant is one asked only once when the user requests recommendation of a restaurant. For example, examples of assumable questions include “Are there any foods you can't eat?”, “Is there any food you've been really into lately?”, “Do you like seasonal foods?”, and the like.

[0220] For example, the generation unit 182 generates user information useful for recommendation of restaurants based on the question data related to restaurants acquired in the processing of step S3 in Fig. 3 described above and the user information stored in the user information storage unit 153.

[0221] Next, the generation unit 182 generates prompts for instructing to select a preliminary question from among a plurality of question candidates based on the context of the conversation with the user and the generated user information.

[0222] Specifically, the prompts include “role”, “user information”, and “question candidate.

[0223] The “role” includes, for example, an instruction such as “Select one appropriate question from the question candidates in consideration of the user information and context and ask a question to the user”.

[0224] The “user information” is user information generated based on the question data and user information described above.

[0225] The “question candidate” includes, for example, the following questions.

[0226] - What did you have for today's breakfast, lunch, yesterday's dinner, etc.? - Do you like the food you are talking about? - Favorite artists

[0227] The generation unit 182 transmits the generated prompts to the AI server 113, requests generation of a question sentence, and acquires the generated question sentence from the AI server 113.

[0228] Then, the character is used to ask the user about a preliminary question using the acquired question sentence, and the user information is updated based on the user's answer to the preliminary question. Accordingly, user information useful for recommending a restaurant to the user is stored.

[0229] Next, a restaurant proposing processing performed by the information processing system 101 will be described with reference to the flowchart of Fig. 6.

[0230] This processing is performed as appropriate, for example, during a conversation with the user.

[0231] In step S101, the context of the conversation with the user is analyzed, as in the processing of step S1 in Fig. 3. Accordingly, for example, the user’s intention or the like is understood. The context analysis unit 172 supplies conversation context information including the analysis results of the context to the planning unit 181.

[0232] In step S102, the planning unit 181 determines whether to propose a restaurant based on the conversation context information. If it is determined to recommend a restaurant, the processing proceeds to step S102.

[0233] For example, recommending a restaurant is determined when the user’s utterance indicates “I want to have lunch somewhere”, “Recommend restaurants for lunch to me”, “Is there any good Ramen restaurant nearby?”, or the like.

[0234] In step S103, the planning unit 181 extracts the search conditions for restaurants. For example, the planning unit 181 extracts parameters necessary for searching for information about restaurants based on the understood user’s intention, or the like.

[0235] In step S104, the planning unit 181 searches for information about restaurants. Specifically, the planning unit 181 sets search conditions based on the extracted parameters. The planning unit 181 searches for restaurants satisfying the set search conditions from the information stored in the information DB 157 based on the search conditions, and acquires restaurant information about the retrieved restaurants. In addition, the planning unit 181 requests the external service server 112 to search for restaurants using the set search conditions, and acquires restaurant information about the retrieved restaurants from the external service server 112.

[0236] The restaurant information includes, for example, reviews (customer feedback), business hours, locations, maps, menus, waiting times, and the like. Note that the restaurant information in the information DB 157 and the external service server 112 is updated to the latest information as appropriate.

[0237] The planning unit 181 causes the acquired restaurant information to be stored in the conversation log storage unit 154.

[0238] In step S105, the planning unit 181 selects restaurant information. For example, the planning unit 181 selects restaurant information based on reviews, waiting times, and the like about the restaurants, and user information. Here, restaurants that serve food that the user dislikes or food containing ingredients that the user cannot eat are excluded.

[0239] In step S106, the information processing unit 151 proposes a restaurant. For example, the planning unit 181 acquires the user information related to the proposal of the restaurant from the user information storage unit 153. Next, the generation unit 182 determines recommendation policies for a restaurant based on information such as the destination, whether it is a holiday, and whether it is close to home. For example, recommendation policies such as giving priority to independent restaurants over chain restaurants, giving priority to restaurants for tourists, and the like are determined. The planning unit 181 supplies the selected restaurant information, the acquired user information, the information indicating the recommendation policies, and the information indicating the user's situation to the generation unit 182.

[0240] The generation unit 182 generates prompts for instructing the AI server 113 to create a restaurant recommendation sentence based on the information acquired from the planning unit 181.

[0241] The prompts include, for example, “instruction”, “user information”, “situation”, “policy”, and “restaurant list”.

[0242] The “instruction” is an instruction to the AI server 113 (for example, LLM). For example, an instruction indicating “You are an AI recommending the most appropriate restaurant based on the user's situations. Select one optimum restaurant from the restaurant list with reference to the user information and the situations, and recommend one with the reasons. If you cannot narrow down the search range well, ask the user additional questions” is assumable.

[0243] Accordingly, the AI server 113 is instructed to select one restaurant and recommend the restaurant along with the reason. In addition, when the AI server cannot narrow down recommendable restaurants, additional questions are allowed.

[0244] The “user information” includes, for example, the following information.

[0245] Likes: Ie-kei Ramen, tuna, mayonnaise, grilled meat, and Chinese food Dislikes: Dirty restaurants, grilled fish, green beans, green bell pepper Age: 35 Gender: Male Place of origin: Fukuoka Residence: Saitama City, Saitama Prefecture Episodes: He ate Ramen with his friends yesterday. Ate grilled meat the day before yesterday. Opinion: Restaurants that specialize in extra-large servings are not eco-friendly. Grilled meat is ok every day. Love to try new restaurants. Prefers less-known, local spots to mainstream restaurants.

[0246] The “situation” includes, for example, the following information.

[0247] Current time: 10:00, Nov. 27th, 2024 Scheduled visit date and time: Afternoon, Nov. 27th, 2024 Current location: 35.83,139.92 Number of passengers: 1 Purpose of travel: Travel Transportation means: Car

[0248] The “policy” includes, for example, the following information.

[0249] Prioritize shops for tourists over chain shops. Prioritize shops with a parking lot.

[0250] The “restaurant list” includes a list of information about selected restaurants.

[0251] The generation unit 182 transmits the generated prompts to the AI server 113, requests generation of a recommendation sentence, and acquires the generated recommendation sentence from the AI server 113.

[0252] The generation unit 182 generates action control information including the generated recommendation sentence and transmits the action control information to the edge terminal 114 of the user.

[0253] The edge terminal 114 proposes a restaurant to the user by causing the character to utter a content including the recommendation sentence based on the action control information.

[0254] In step S107, the context of the conversation with the user is analyzed, as in the processing of step S1 in Fig. 3. Accordingly, for example, the user's opinion on the proposed restaurant is analyzed based on the utterance indicating the user's reaction to the proposed restaurant. The context analysis unit 172 supplies conversation context information including the analysis results of the context to the planning unit 181.

[0255] In step S108, the planning unit 181 determines whether proposal of another restaurant has been requested based on the conversation context information. If it is determined that the proposal of another restaurant has been requested, the processing returns to step S106.

[0256] For example, it is determined that the proposal of another restaurant has been requested if the user makes utterances indicating “What else?”, “More options, please”, “I’m not in the mood for noodles right now”, and the like.

[0257] Thereafter, the processing of step S106 to step S108 is repeated until it is determined in step S108 that no request for proposal of another restaurant has been received.

[0258] On the other hand, if it is determined in step S108 that no request for proposal of another restaurant has been received, the processing proceeds to step S109.

[0259] In step S109, the planning unit 181 determines whether to set the destination based on the conversation context information. If it is determined that the destination has been set, the processing proceeds to step S109.

[0260] For example, when the user makes an utterance such as “Let’s go there” or “Set the destination”, it is determined to set the destination.

[0261] In step S110, the information processing system 101 performs the destination setting processing, and then the restaurant proposing processing ends.

[0262] Next, the destination setting processing will be described in detail with reference to the flowchart of Fig. 7.

[0263] In step S151, the planning unit 181 acquires information about the proposed restaurant as necessary. For example, when there is information necessary for setting the proposed restaurant as a destination or information necessary to additionally present to the user the proposed restaurant, the planning unit 181 acquires the necessary information from the information DB 157 or the external service server 112.

[0264] In step S152, the planning unit 181 sets the destination. For example, the planning unit 181 generates destination setting information including information (for example, shop name, address, map, and the like) regarding the restaurant set as the destination. The planning unit 181 supplies the destination setting information to the generation unit 182.

[0265] In addition, the planning unit 181 transmits the destination setting information to an application program or an apparatus that provides a navigation function or a scheduling function, as necessary.

[0266] The application program or the apparatus that has received the destination setting information sets the restaurant proposed to the user as the destination based on the destination setting information.

[0267] In step S153, the information processing system 101 notifies the user of completion of the setting of the destination. For example, for example, a prompt including the destination setting information is transmitted to the AI server 113, the generation of an utterance sentence for notifying of completion of the setting of the destination is requested, and the generated utterance sentence is acquired from the AI server 113.

[0268] The generation unit 182 generates action control information including the generated utterance sentence and transmits the action control information to the edge terminal 114 of the user.

[0269] The edge terminal 114 notifies the user that the proposed restaurant has been set as the destination by causing the character to make an utterance based on the action control information.

[0270] Thereafter, the destination setting processing ends.

[0271] Returning to Fig. 6, on the other hand, in step S109, for example, when the user does not instruct the setting of the destination or does not indicate the intention to go to the proposed restaurant, it is determined that the destination is not set, the process of step S110 is skipped, and the restaurant proposing processing ends.

[0272] In addition, if it is determined in step S102 not to propose a restaurant, the processing of step S103 to step S110 is skipped, and the restaurant proposing processing ends.

[0273] As described above, restaurants are recommended one by one in an interactive manner, along with the recommendation reasons according to the user's situations, tastes, and the like. This makes the user feel more like to visit the recommended restaurant than if the restaurant search results are presented mechanically as a list.

[0274] In addition, flexible recommendation is realized in accordance with not only the user's tastes and preference tendencies but also the situations and user characteristics. For example, it is possible to recommend more appropriate restaurants in consideration of the user's niche taste, food that the user recently ate, and the like, instead of superficial matching according to the user's favorite category or food. For example, a restaurant that is likely to be visited frequently by the user is recommended, and the user can realize the value of personification.

[0275] Furthermore, without using an existing method such as collaborative filtering, by causing a language model (for example, LLM) to solve similarities and relationships that could only be inferred from data, it is possible to recommend an appropriate restaurant to the user even with a small amount of data.

[0276] Specifically, for example, because Hoto Noodles that the user ate previously were good, the system can recommend Kishimen noodles next time.

[0277] For example, in response to a user’s request “Tell me a restaurant recommendable for lunch” in a travel destination, recommending chain restaurants can be avoided.

[0278] Then, for example, it becomes possible to recommend a local Kyoto restaurant, along with a message like “There's a restaurant called XX Shijo Karasuma. Since you're in Kyoto, how about a Kyoto vegetable buffet? It's all-you-can-eat, with freshly harvested vegetables!” This gives the user the impression that the user is provided with high-value-added information.

[0279] For example, in response to the user's request “I want to have dinner around here. Could you recommend a restaurant?”, recommending a Ramen shop is prevented even when, if the user loves Ramen, the user is taking Ramen for lunch.

[0280] Then, a Japanese restaurant can be recommended with a message, for example, “Then, how about Japanese restaurant OO. Since you ate Ramen for lunch, you can have a healthy meal there. Some people say the interior is stylish”. This gives the user the impression that “The system is really considering my health” or “The system understands my taste well”.

[0281] For example, an appropriate restaurant is proposed based on the user's tastes and information about the restaurant on a review site.

[0282] For example, it is assumed that the user's tastes include “Cleanliness is most important for restaurants” and “Want to go to a restaurant that is easy to enjoy with kids”. For example, it is assumed that restaurant A and restaurant B were retrieved from a review site, the evaluation of restaurant A at the review site is “Impeccable cleanliness”, and “Child-friendly service”, and the evaluation of restaurant B is “Provide kid’s chairs” and “Dirty restroom”.

[0283] In this case, for example, restaurant A is recommended with the message “How about restaurant A? The restaurant is excellent in cleanliness and child-friendly services that XX values”.

[0284] For example, an appropriate restaurant is proposed based on the action history of the user, and the current states and schedules.

[0285] For example, it is assumed that the user's action history includes “Recently, I haven't been able to go to the gym due to a series of drinking parties” and “Yesterday’s dinner was heavy Ramen”, and the user's taste is “Emphasis on review site scores”. Furthermore, it is assumed that the user’s state and schedule are “I have an appointment starting from 1 p.m., so I want to have a quick lunch nearby before then”.

[0286] In this case, for example, a soba restaurant is proposed with the message “Since you've been having a lot of drinking parties lately, how about some soba? There's a restaurant highly rated at review sites just five minutes from your destination, and they serve the food quickly”.

[0287] Hereinafter, an example of a conversation between the user and the character when the restaurant recommendation function is used will be described.

[0288] User: “Give me some lunch recommendations.”

[0289] Character: “I recommend “restaurant A”! It offers parking and a sheltered and calming atmosphere. You can enjoy specialties such as self-made tofu and bean curd, making it perfect for a special travel feeling.”

[0290] User: “What else?”

[0291] Character: ““Restaurant B” is also recommendable! It is popular with women, and known for its lunchtime creamy omelet. It is a casual bistro cafe perfect for having travel moods.”

[0292] User” “I don’t like eggs.”

[0293] Character: “Then, let’s go back “restaurant A”! You can enjoy fragrant soba and superb tofu dishes, even if you're avoiding eggs. The calm Japanese atmosphere is also attractive for lunch.”

[0294] User: “Give me other candidate recommendations.”

[0295] Character: “How about “restaurant C”? It's known for its non-gamy juicy lamb meat, and perfect for grilled meat lovers like you. It's healthy and very satisfying!”

[0296] User: “Anything else?”

[0297] Character: “How about “restaurant D”? Ample parking is available, and the seafood dishes are abundant. It is best for lunch for travelers who want to enjoy eating slowly in a calm atmosphere!”

[0298] User: “I don’t like shrimp.”

[0299] Character: “Then, how about “restaurant E”? It's a fashionable bar that offers pasta and Spanish dishes. The place is perfect for light lunch and you can experience something special!”

[0300] User: “Which one do you think is the best?”

[0301] Character: “I think “restaurant A” is perfect! You can enjoy a special feeling of traveling, and the gentle flavors of tofu and soba are perfect for lunch. It has a convenient parking lot, ensuring easy accessibility.”

[0302] As described above, the user's satisfaction with the character can be improved.

[0303] 2. Modification Hereinafter, a modification of the embodiment of the present technology described above will be described.

[0304] For example, a plurality of main servers 111, a plurality of external service servers 112, and a plurality of AI servers 113 may be provided.

[0305] For example, two or more of the main server 111, the external service server 112, and the AI server 113 may be integrated. For example, the main server 111 may include a generative AI such as an LLM.

[0306] For example, a part or all of the processing of each apparatus of the information processing system 101 may be executed by another apparatus. For example, a part of the processing of the main server 111 described above may be executed by an edge terminal 114.

[0307] The present technology is applicable to all apparatuses and systems capable of having conversations with users using virtual characters.

[0308] 3. Others Description of Computer to Which Present Technology is Applied The series of processing operations described above can be executed by hardware or can be executed by software. In a case in which the series of processing operations is executed by software, a program constituting the software is installed in a computer. Herein, a computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer, for example, that can execute various functions by the installation of various programs, and the like.

[0309] Fig. 8 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing operations by using a program.

[0310] In the computer, a processing circuit 1001, a read only memory (ROM) 1002, and a random access memory (RAM) 1003 are connected to one another via a bus 1004.

[0311] Furthermore, an input / output interface 1005 is connected to the bus 1004. An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

[0312] The input unit 1006 may include a physical or virtual operation means for inputting information by a user's operation, such as a keyboard, a mouse, or a touch panel, and a means for inputting information by a user by using voices, lines of sight, or the like. Further, the input unit 1006 may include sensors for inputting various physical quantities to the computer. For example, the input unit 1006 can include sensors that acquire physical quantities of light (including infrared rays other than visible light), sound, or the like, such as a camera or a microphone. In addition, for example, the input unit 1006 can include sensors that acquire other physical quantities such as temperature, moisture content, acceleration, and distance. The output unit 1007 can include a means for presenting information to the user by stimulating the user's perception, such as a display that is visible from a driver’s seat, a speaker, or a haptic device. The storage unit 1008 is configured by a hard disk, a nonvolatile or volatile memory, or the like, and stores various types of information (including programs). The communication unit 1009 is a network interface or the like, and performs wired or wireless communication with the outside. The drive 1010 drives a removable medium 1011 such as a magnetic disk, an optical disc, a magneto-optical disc, or a semiconductor memory.

[0313] The processing circuit 1001 includes a processor that executes a program, such as a central processing unit (CPU) or a digital signal processor (DSP). The (processor of the) processing circuit 1001 loads the program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes the program, thereby performing the above-mentioned series of processing operations. The processing circuit 1001 can cause the output unit 1007 to output the processing result of the series of processing operations, for example, via the bus 1004 and the input / output interface 1005, as necessary. The processing circuit 1001 can also cause the storage unit 1008 to store the processing result, cause the communication unit 1009 to transmit the processing result, and the like.

[0314] The program executed by the computer (processing circuit 1001) can be provided by being recorded in the removable medium 1011 as a package medium or the like, for example. Further, the program can be provided via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.

[0315] In the computer, the program can be installed in the storage unit 1008 via the input / output interface 1005 by mounting the removable medium 1011 on the drive 1010. The program can also be received through the communication unit 1009 from another apparatus such as a server via a wired or wireless transmission medium and installed in the storage unit 1008. Other than the above manners, the program can be pre-installed in the ROM 1002 or the storage unit 1008.

[0316] The program executed by the computer may be a program in which processing is performed in a time-series manner in the order described in the present specification, or may be a program in which processing is performed in parallel or at a necessary timing such as a time when a call is made.

[0317] The processing performed by the computer according to the program does not necessarily need to be performed in a time-series manner in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or in an individual manner (for example, parallel processing or processing by an object).

[0318] The program may be processed by one computer (processor) or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

[0319] When the series of processing operations described above is performed by the computer executing the program, for example, for example, the processing circuit 1001 (the processor thereof) functions as the information processing unit 151 of Fig. 2 by executing the program.

[0320] In the present specification, the system means one component or a set of a plurality of components (apparatuses, modules (parts), and the like). Accordingly, one or a plurality of components of a computer, for example, only a processor or a combination of a processor and a memory (e.g., only the processing circuit 1001 or a combination of the processing circuit 1001 through the bus 1004), is a system. With respect to the set of a plurality of components, it does not matter whether all the components are in the same housing. Therefore, the system refers to both a plurality of devices housed in separate housings and connected via a network and one apparatus in which a plurality of modules are housed in one housing. Furthermore, for example, the entire computer or a combination of the computer and another apparatus such as a server, which is not illustrated, is also a system.

[0321] The components (blocks) of the apparatus described in the present specification and illustrated in the drawings are functional conceptual blocks, and an actual apparatus does not need to have the illustrated configuration. In other words, the apparatus may have any configuration in which the functions of the illustrated components are divided and / or integrated in any unit, for example, a configuration having one block in which the functions of all the components are integrated.

[0322] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

[0323] For example, the present technology can have a configuration of cloud computing in which one function is shared and processed in cooperation by a plurality of apparatuses via a network.

[0324] Further, each step described in the above-described flowchart can be performed by one apparatus or can be shared and executed by a plurality of apparatuses.

[0325] Further, when a single step includes multiple pieces of processing, the multiple pieces of processing included in that single step can be executed by a single device, or can be shared and executed by multiple devices.

[0326] Example of Combined Configurations The present technology may also have the following configurations. (1). A vehicle, comprising: a passenger compartment; a user interface that is visible from a driver’s seat; and an information processing apparatus having circuitry configured to acquire via the user interface data regarding an exchange of information data between a vehicle driver and a virtual character hosted on a computer-based system, develop a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver; and control an action of the virtual character through the user interface based on the perceived intention parameter of the vehicle driver. (2) The vehicle of (1), wherein the exchange of information data comprises a voice conversation between the vehicle driver and the virtual character, and the circuitry is configured to analyze a context of the voice conversation to develop the perceived intention parameter. (3) The vehicle of (1), wherein the user interface comprises a display configured to visually present the virtual character to the vehicle driver, and the circuitry is configured to control a movement of the virtual character on the display based on the perceived intention parameter. (4) The vehicle of (1), wherein the circuitry is further configured to: determine whether to propose a facility to the vehicle driver based on the perceived intention parameter, and control the user interface to output a proposal for the facility when it is determined to propose the facility. (5) The vehicle of (4), wherein the circuitry is further configured to: receive an acceptance of the proposal from the vehicle driver via the user interface, and set the facility as a destination for the vehicle in response to the acceptance. (6) The vehicle of (1), wherein the circuitry is further configured to: acquire weather information regarding a current location or a destination of the vehicle, and control the virtual character to report the weather information to the vehicle driver based on the perceived intention parameter. (7) The vehicle of (1), wherein the circuitry is further configured to: generate a question to be asked by the virtual character to the vehicle driver to acquire a missing piece of information regarding the perceived intention parameter, and update the perceived intention parameter based on a response of the vehicle driver to the question. (8) The vehicle of (1), wherein the circuitry is further configured to store the perceived intention parameter in a memory as a user profile associated with the vehicle driver, and the circuitry utilizes the user profile to personalize a subsequent conversation between the virtual character and the vehicle driver. (9) An information processing apparatus comprising: circuitry configured to acquire data regarding an exchange of information data between a vehicle driver and a virtual character hosted on a computer-based system, develop a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver, and control an action of the virtual character based on the perceived intention parameter of the vehicle driver. (10) The information processing apparatus of (9), wherein the circuitry is further configured to extract a keyword from the perceived intention parameter, and generate a query based on the keyword to acquire news information. (11) The information processing apparatus of (9), wherein the circuitry is further configured to: detect a contradiction between the perceived intention parameter and pre-existing information in a profile of the vehicle driver, and update the profile by adopting the perceived intention parameter as newer information when the contradiction is detected. (12) The information processing apparatus of (9), wherein the circuitry is further configured to select a facility to be proposed to the vehicle driver based on the perceived intention parameter and a current location of the vehicle driver, and generate output data for causing the virtual character to propose the facility. (13) The information processing apparatus of (9), wherein the circuitry is further configured to: analyze a context of the exchange of information data to determine if the vehicle driver has a specific interest, and initiate a new topic of conversation via the virtual character related to the specific interest. (14) The information processing apparatus of (9), wherein the circuitry is further configured to acquire weather information from an external server, and determine a timing for the virtual character to report the weather information based on the perceived intention parameter. (15) The information processing apparatus of (9), wherein the circuitry is further configured to generate a question for the vehicle driver when the perceived intention parameter is insufficient to determine a preference of the vehicle driver. (16) The information processing apparatus of (9), wherein the circuitry comprises a communication interface configured to communicate with a plurality of edge terminals, and the circuitry is configured to manage the perceived intention parameter as a user profile accessible by the plurality of edge terminals. (17) The information processing apparatus of (9), wherein the circuitry is further configured to analyze the exchange of information data according to a sequence of steps including analyzing a background, a feeling, a reason, and a desire of the vehicle driver. (18) The information processing apparatus of (9), wherein the circuitry is further configured to receive the data regarding the exchange of information data from a vehicle via a network, and transmit control data for the action of the virtual character to the vehicle via the network. (19) A non-transitory computer-readable medium storing instructions which, when executed by a computer, cause the computer to perform a method comprising: acquiring data regarding an exchange of information data between a vehicle driver and a virtual character; developing a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver; and controlling an action of the virtual character based on the perceived intention parameter of the vehicle driver. (20) The non-transitory computer-readable medium of 19, wherein the instructions cause the computer to perform: analyzing a voice input from the vehicle driver to identify a context; and generating a response of the virtual character that aligns with the context and the perceived intention parameter.

Claims

1. A vehicle, comprising: a passenger compartment; a user interface that is visible from a driver’s seat; and an information processing apparatus having circuitry configured to acquire via the user interface data regarding an exchange of information data between a vehicle driver and a virtual character hosted on a computer-based system, develop a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver; and control an action of the virtual character through the user interface based on the perceived intention parameter of the vehicle driver.

2. The vehicle of claim 1, wherein the exchange of information data comprises a voice conversation between the vehicle driver and the virtual character, and the circuitry is configured to analyze a context of the voice conversation to develop the perceived intention parameter.

3. The vehicle of claim 1, wherein the user interface comprises a display configured to visually present the virtual character to the vehicle driver, and the circuitry is configured to control a movement of the virtual character on the display based on the perceived intention parameter.

4. The vehicle of claim 1, wherein the circuitry is further configured to: determine whether to propose a facility to the vehicle driver based on the perceived intention parameter, and control the user interface to output a proposal for the facility when it is determined to propose the facility.

5. The vehicle of claim 4, wherein the circuitry is further configured to: receive an acceptance of the proposal from the vehicle driver via the user interface, and set the facility as a destination for the vehicle in response to the acceptance.

6. The vehicle of claim 1, wherein the circuitry is further configured to: acquire weather information regarding a current location or a destination of the vehicle, and control the virtual character to report the weather information to the vehicle driver based on the perceived intention parameter.

7. The vehicle of claim 1, wherein the circuitry is further configured to: generate a question to be asked by the virtual character to the vehicle driver to acquire a missing piece of information regarding the perceived intention parameter, and update the perceived intention parameter based on a response of the vehicle driver to the question.

8. The vehicle of claim 1, wherein the circuitry is further configured to store the perceived intention parameter in a memory as a user profile associated with the vehicle driver, and the circuitry utilizes the user profile to personalize a subsequent conversation between the virtual character and the vehicle driver.

9. An information processing apparatus comprising: circuitry configured to acquire data regarding an exchange of information data between a vehicle driver and a virtual character hosted on a computer-based system, develop a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver, and control an action of the virtual character based on the perceived intention parameter of the vehicle driver.

10. The information processing apparatus of claim 9, wherein the circuitry is further configured to extract a keyword from the perceived intention parameter, and generate a query based on the keyword to acquire news information.

11. The information processing apparatus of claim 9, wherein the circuitry is further configured to: detect a contradiction between the perceived intention parameter and pre-existing information in a profile of the vehicle driver, and update the profile by adopting the perceived intention parameter as newer information when the contradiction is detected.

12. The information processing apparatus of claim 9, wherein the circuitry is further configured to select a facility to be proposed to the vehicle driver based on the perceived intention parameter and a current location of the vehicle driver, and generate output data for causing the virtual character to propose the facility.

13. The information processing apparatus of claim 9, wherein the circuitry is further configured to: analyze a context of the exchange of information data to determine if the vehicle driver has a specific interest, and initiate a new topic of conversation via the virtual character related to the specific interest.

14. The information processing apparatus of claim 9, wherein the circuitry is further configured to acquire weather information from an external server, and determine a timing for the virtual character to report the weather information based on the perceived intention parameter.

15. The information processing apparatus of claim 9, wherein the circuitry is further configured to generate a question for the vehicle driver when the perceived intention parameter is insufficient to determine a preference of the vehicle driver.

16. The information processing apparatus of claim 9, wherein the circuitry comprises a communication interface configured to communicate with a plurality of edge terminals, and the circuitry is configured to manage the perceived intention parameter as a user profile accessible by the plurality of edge terminals.

17. The information processing apparatus of claim 9, wherein the circuitry is further configured to analyze the exchange of information data according to a sequence of steps including analyzing a background, a feeling, a reason, and a desire of the vehicle driver.

18. The information processing apparatus of claim 9, wherein the circuitry is further configured to receive the data regarding the exchange of information data from a vehicle via a network, and transmit control data for the action of the virtual character to the vehicle via the network.

19. A non-transitory computer-readable medium storing instructions which, when executed by a computer, cause the computer to perform a method comprising: acquiring data regarding an exchange of information data between a vehicle driver and a virtual character; developing a perceived intention parameter of the vehicle driver based on the information data, wherein the perceived intention parameter includes at least one of a habit, an opinion, a desire, or a way of thinking of the vehicle driver; and controlling an action of the virtual character based on the perceived intention parameter of the vehicle driver.

20. The non-transitory computer-readable medium of claim 19, wherein the instructions cause the computer to perform: analyzing a voice input from the vehicle driver to identify a context; and generating a response of the virtual character that aligns with the context and the perceived intention parameter.

Images