Information processing system and information processing method

Abstract

To provide an information processing system and an information processing method capable of performing appropriate communication while using an expression for a user to easily understand.SOLUTION: An information processing system detects an expression mode of a user, determines an expression mode of an agent to be a simulation object, sets a simulation degree of the expression mode of the agent to the detected expression mode of the user about the expression mode, and generates output data corresponding to the simulation degree. In such a case, the simulation degree of the expression mode of the agent is set so as to come close to the expression mode of the user with time as the generated output data is outputted to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system and an information processing method. [Background technology]

[0002] There is known an information processing device that, when using a voice response system to output response information, does not always respond using the official name or typical phrase, but recognizes the abbreviation, nickname, slang name, or other name used in the user's speech as the user's speech characteristics, and outputs response information using a name that corresponds to the user's speech characteristics (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 199486 Brochure Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional information processing device, the response information of the voice response system is generated using the name used in the user's utterance. In a communication device using an agent, the user can easily understand information presented in a manner similar to the expression used by the user, rather than information presented in a manner different from the expression used by the user. However, if the agent excessively imitates the expression used by the user, the user may feel uncomfortable or uncomfortable with the agent's expression, which may hinder appropriate communication. Therefore, in a device using an agent, it is desired to perform appropriate communication while using expressions that are easy for the user to understand.

[0005] An object of the present invention is to provide an information processing system and an information processing method that are capable of carrying out appropriate communication while using expressions that are easily understandable to a user. [Means for solving the problem]

[0006] The present invention solves the above problem by detecting a user's expression style, determining an agent's expression style to be imitated, setting a degree of imitation of the agent's expression style relative to the detected user's expression style for that expression style, and generating output data according to the degree of imitation. In this case, the above problem is solved by setting the degree of imitation of the agent's expression style so as to approach the user's expression style over time as the generated output data is output to the user. Effect of the Invention

[0007] According to the present invention, it is possible to provide an information processing system and an information processing method that can communicate appropriately while using expressions that are easy for the user to understand. In particular, when providing information using an agent, the degree to which the agent imitates the user's expression style is appropriately changed, which is expected to have the effect of making the user feel more attached to the agent. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an embodiment of an information processing system according to the present invention; [Diagram 2] 2(a) and 2(b) are diagrams showing an example of the interior of a vehicle where the agent device of FIG. 1 is installed. [Diagram 3] 1(a) to 1(c) are diagrams showing an example of an agent device including a personified agent according to the present invention. [Figure 4] 2 is a diagram showing an example of the configuration of an expression mode database that has been subjected to information processing by the information processing device of FIG. 1; [Diagram 5]2 is a diagram showing an example of the configuration of an intimacy database that has been subjected to information processing by the information processing device of FIG. 1; FIG. [Figure 6] 1. (a) and (b) are diagrams (part 1) for explaining the intimacy degree information processed by the information processing device of FIG. [Figure 7] 7 is a flowchart showing an example of an information processing procedure for the intimacy level, which is processed based on the conversation content input by the user of FIG. 6. [Figure 8] 1. (a) to (c) are diagrams (part 2) for explaining the intimacy degree information processed by the information processing device of FIG. [Figure 9] 9 is a flowchart showing an example of an information processing procedure of the intimacy level, which is processed based on a reaction from the user in FIG. 8. [Figure 10] 1. FIG. 4 is a diagram for explaining information processing for determining an expression mode of an agent to be imitated, which is executed in the information processing device of FIG. [Figure 11] Each of (a) to (c) is a diagram (part 1) for explaining information processing for determining an object to be imitated by an agent based on the expression manner in FIG. 10 and setting the degree of imitation of the expression manner of the object to be imitated. [Figure 12] Each of (a) to (c) is a diagram (part 2) for explaining information processing for determining an object to be imitated by an agent based on the expression manner in FIG. 10 and setting the degree of imitation of the expression manner of the object to be imitated. [Figure 13] Each of (a) to (c) is a diagram (part 3) for explaining the information processing for determining an object to be imitated by an agent based on the expression manner in FIG. 10 and setting the degree of imitation of the expression manner of the object to be imitated. [Figure 14] 1. FIG. 4 is a diagram for explaining another example of information processing for setting the degree of imitation of the expression mode of an agent, which is executed in the information processing device of FIG. [Figure 15] 10 is a diagram (part 1) for explaining yet another example of information processing for setting an imitation degree of an agent's expression mode, which is executed in the information processing device of FIG. [Figure 16] FIG. 13 is a diagram (part 2) for explaining yet another example of the information processing for setting the degree of imitation of the expression mode of an agent, which is executed in the information processing device of FIG. [Figure 17] 4 is a flowchart showing an example of an information processing procedure executed in the information processing system of FIG. 1. [Figure 18] 18 is a flowchart showing an example of a subroutine executed in step S8 of FIG. 17. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing one embodiment of an information processing system 1 according to the present invention, and Fig. 2(a) and (b) are diagrams of the interior of a vehicle showing an example of the installation location and operation of an agent device 5 shown in Fig. 1. The information processing system 1 according to the present invention uses an agent device 5 that provides information to a user through an agent function by a personified agent 52 (hereinafter, also simply referred to as agent 52), specifically, through a medium of voice, image, character robot operation, or a combination of these, or that communicates with the user.

[0010] The agent device 5 includes a speaker or other audio output unit for outputting voice and sound effects, a display or other display unit for displaying images including text, and outputs communication information by providing the user with voice, sound effects, text, and other images along with the movement of the character robot. Note that, in this embodiment, the agent 52 is a three-dimensional object such as a character robot, but is not limited to this and may be a two-dimensional image displayed on a display.

[0011] 1, 2(a) and 2(b), the agent device 5 of this embodiment is a human-like character robot as an agent 52 that is mounted on a base 51 so as to be able to appear and disappear by an actuator (not shown). When the agent device 5 receives a control command from the output unit 28 and communicates with the user by the agent function, it appears from the base 51 as shown in Fig. 2(b), and when communication with the user is terminated, the character robot is stored in the base 51 as shown in Fig. 2(a).

[0012] 1, 2(a) and 2(b) are examples of anthropomorphized agents 52, and the agent 52 may not be a human but may be an animal, a plant, a specific character, an avatar or an icon. The agent 52 may be provided as a physical individual, or the shape of a human, an animal, a plant, a specific character or the like as the agent 52 may be displayed as an image on a display.

[0013] In addition, although an example in which the agent device 5 is provided in a vehicle will be described below, the form and installation location of the agent device 5 are not limited to this. The agent device 5 may be any electronic device that has a personified agent and is equipped with an agent function, and may be, for example, a portable speaker-type electronic device as shown in Fig. 3(b) or an electronic device with a display. Furthermore, the functions related to audio output and video output of the agent device 5 described below may be installed in a mobile phone such as a smartphone as shown in Fig. 3(c).

[0014] The information processing system 1 of this embodiment detects the expression mode included in the user's actions and conversation, and communicates by having the personified agent 52 imitate the user's expression mode as shown in Fig. 3(a) to Fig. 3(c). By using so-called mirroring, which imitates the expression mode, it is possible to provide information that is easy for the user to understand, and good communication can be performed. On the other hand, if the agent 52 imitates the user's expression mode excessively, the user may feel uncomfortable or uncomfortable with the expression of the agent 52. Therefore, the information processing system 1 sets the degree of imitation so that the expression mode of the agent 52 gradually approaches the expression mode of the user as communication is repeated. In this embodiment, the user refers to a person who uses the information processing system 1, and although an example applied to a driver of a vehicle will be described below, the user may also be a passenger other than the driver (hereinafter, the driver and passenger will also be simply referred to as passengers).

[0015] Returning to FIG. 1, the information processing system 1 of the present embodiment is composed of an information processing device 2 including a ROM (Read Only Memory) storing programs for executing various processes, a CPU (Central Processing Unit) as an operating circuit functioning as an information processing device by executing the programs stored in the ROM, and a RAM (Random Access Memory) functioning as an accessible storage device, vehicle sensors 3, an input device 4, and an agent device 5 as an output device. These devices are connected, for example, by a CAN (Controller Area Network) or other in-vehicle LAN, and can transmit and receive information to and from each other. In terms of the functional configuration exhibited by the execution of the information processing program, the information processing device 2 includes an occupant identification unit 21, an expression mode detection unit 22, an expression mode database 23, an intimacy determination unit 24, an intimacy database 25, an expression mode setting unit 26, a data generation unit 27, and an output unit 28, as shown in FIG. 1.

[0016] The occupant identification unit 21 identifies the driver D based on information acquired from the vehicle sensors 3 and the input device 4, and at least temporarily stores the information. The vehicle sensors 3 are, for example, an in-vehicle camera installed near the rearview mirror to capture images of the interior of the vehicle, and the input device 4 is, for example, a microphone installed near the seat that can receive voice input. The occupant identification unit 21 identifies the driver D based on image information captured by the in-vehicle camera and voice information input to the in-vehicle microphone. The occupant identification unit 21 outputs attribute information (identification information) of the identified driver D, image information of the driver D acquired from the vehicle sensors 3, and voice information of the driver D acquired from the input device 4 to the expression mode detection unit 22 and the intimacy determination unit 24.

[0017] The driver D may be identified using an input signal from a seating sensor provided inside the seat of the seat or a seat belt sensor. Also, the driver D may be identified by storing the driver's identification data in a wireless key for the vehicle, and automatically or semi-automatically reading the driver's identification data when the vehicle is unlocked or started using the wireless key.

[0018] The expression mode detection unit 22 detects the expression mode of the driver D based on the attribute information (identification information), image information, and voice information of the driver D received from the occupant identification unit 21, and accumulates it in the expression mode database 23. The expression mode of the driver D is an expression used when the driver D shows his / her will, emotion, thought, etc. by action or language. The expression mode includes an action mode having visual characteristics used by the driver D and a conversation mode having auditory characteristics used by the driver D. The action mode and conversation mode of the driver D detected by the expression mode detection unit 22 are used to determine the expression mode of the agent 52 that the agent 52 imitates when outputting communication information. This is because the agent 52 imitates the expression used by the driver D, thereby enabling communication to be performed in a mode that is easy for the driver D to understand. In addition, the agent 52 imitating the expression actually used by the driver D can be expected to have an effect that the expression of the agent 52 is favorably received by the driver D.

[0019] The expression mode detection unit 22 may detect, without being limited to any particular type, visual features such as the appearance including the clothes and hairstyle of the driver D, the facial features such as the face and body shape, the posture when driving and operating the vehicle, facial expressions such as joy, anger, sadness, and happiness, and movements such as gestures and hand movements as movement modes as movement modes from image information capturing an image of the driver D. Since the driver D is likely to notice imitation of movement modes detected from these visual features, the agent 52 can gradually imitate the driver D to prevent the driver from getting the impression that the imitation is excessive.

[0020] Furthermore, the expression style detection unit 22 may detect, without any particular limitation, features related to the tone of the driver D's voice, intonation, polite language, use of popular phrases, language, dialect, etc., conversation content including topics and preferences, estimated age, gender, and other auditory features as conversation styles from the voice information input by the driver D. Since the driver D is less likely to notice imitation of conversation styles detected from these auditory features than movement styles, the agent 52 can gradually imitate the driver D to achieve good communication.

[0021] The expression mode database 23 stores attribute information (identification information) of the driver D and the expression mode of the driver D detected by the expression mode detection unit 22 in association with each other. Fig. 4 is a diagram showing an example of the configuration of the expression mode database 23. The expression mode database 23 stores attribute information of the driver D, the detected expression mode, date and time information when the expression mode was detected, and a detection history such as a cumulative number of detections. Note that, although the expression mode database 23 is configured to be included in the information processing device 2, it may be configured to store or acquire various information by communicating with an external server.

[0022] As shown in Fig. 4, the passenger attribute information stores, for example, a father as user 1, and stores a detection history of the expression patterns used by the father of user 1. At 13:00 on March 18, 2020, a conversation pattern of "Okay" was detected from the voice information input by the father of user 1, and a motion pattern of "pointing thumbs up" was detected from image information capturing the father of user 1, and the cumulative number of detections of these expression patterns is stored as 108. Similarly, it is stored that a conversation mode of "I got it" and a motion mode of <making a circle with thumb and index finger> (cumulative detection count 502 times) were detected at 11:11 on January 11, 2020, a conversation mode of "Understood" and a motion mode of <salute> (cumulative detection count 49 times) were detected at 8:06 on December 8, 2019, and a conversation mode of "Roger" and a motion mode of <raising one hand up and pointing the other hand to the side> (cumulative detection count 9 times) were detected at 7:59 on November 28, 2019. Note that the expression mode may be detected and stored as either the conversation mode or the motion mode, or, as shown in FIG. 4, both the conversation mode and the motion mode may be detected and stored in association with each other. In addition, the expression mode detected by the expression mode detection unit 22 may be classified and stored as an item such as "agreement".

[0023] The intimacy determination unit 24 estimates the intimacy between the driver D and the agent 52, and accumulates it in the intimacy database 25. A higher intimacy indicates that the driver D is more accepting of the agent 52 and more accustomed to communicating with the agent 52, and a lower intimacy indicates that the driver D is less accustomed to communicating with the agent 52. The intimacy of the driver D estimated by the intimacy determination unit 24 is used to determine the degree of imitation of the expression mode of the agent 52 when the agent 52 outputs communication information. Note that the intimacy determination unit 24 and the intimacy database 25 are not essential components of the present invention and may be omitted as necessary.

[0024] In the intimacy database 25, the attribute information (identification information) of the driver D and the intimacy of the driver D estimated by the intimacy determination unit 24 are stored in association with each other. FIG. 5 is a diagram showing an example of the configuration of the intimacy database 25. In addition to storing the attribute information and intimacy of the occupant, the intimacy database 25 may store the riding history of the driver D acquired from the vehicle sensors 3, and the use history of various in-vehicle devices such as a driving assistance device, a navigation device, and an agent device 5. As shown in FIG. 5, the attribute information of the occupant includes, for example, a father recorded as the user 1, and for the father of the user 1, the intimacy with the agent 52, the riding history of the vehicle, and the use history of various in-vehicle devices are stored.

[0025] The intimacy determination unit 24 estimates the intimacy between the driver D and the agent 52, for example, based on the attribute information (identification information) and voice information of the driver D received from the occupant identification unit 21. Specifically, the conversation content included in the voice information input by the driver D is classified according to the field and depth of the topic, and is quantified to calculate the intimacy. The deeper the topic depth, the more specific matters and private events are included, and the shallower the topic depth, the more abstract and general matters are included. Therefore, the deeper the conversation content included in the voice information of the driver D, the more it is estimated that the driver D is accepting the agent 52, and it can be determined that the intimacy is high.

[0026] The classification of topic fields and depth is not particularly limited, but as shown in Figure 6(a), for example, fields are classified into [1] hobbies and social views (opinions and attitudes), [2] life and studies, [3] personality, body and appearance, [4] friends and relations with the opposite sex, and [5] financial relations, and depth is classified into (1) general topics such as news, (2) personal topics including positive information about the speaker, and (3) personal topics including negative information about the speaker. Since personal topics have a higher topic depth than general topics such as news, and personal topics including negative information have a higher topic depth than positive information, for example, (1) general topics such as news are scored as 1 point, (2) personal topics including positive information about the speaker are scored as 3 points, and (3) personal topics including negative information about the speaker are scored as 5 points.

[0027] As shown in FIG. 6(b), the intimacy determination unit 24 quantifies the depth of the fields [1] to [5] based on the conversation content included in the voice information input by the driver D, and calculates the average value. In the example of the father of the user 1, the father's conversation is quantified based on the conversation content of the father, and the score is 5 points for [1] Hobbies and social outlook (opinions and attitudes) because it includes personal topics including (3) negative information about the speaker, and 1 point for [2] Life and studies because it includes general topics such as (1) news. The average score of the fields [1] to [5], 3.8, is set as the intimacy of the father of the user 1. In this way, by calculating the intimacy using the conversation content included in the voice information input by the driver D, the actual words and actions of the driver D toward the agent 52 can be reflected in the intimacy, and appropriate communication between the driver D and the agent 52 can be promoted.

[0028] In this embodiment, the topic fields are classified into five categories, and the topic depth is classified into three categories and quantified, but the number of categories is not limited to this. In addition, if the topic depth is the same, the same score (points) is assigned to all fields. However, since the depth of the field increases from [1] to [5], the score may be set to be higher from [1] to [5]. In addition, the average value of the fields [1] to [5] is used to calculate the intimacy level, but the median value may be used, or the calculation may be performed by an addition method or a subtraction method. In addition, the intimacy level may be expressed using stages such as "high", "medium", and "low" instead of a numerical value. The intimacy level determination unit 24 may update the intimacy level once calculated as appropriate based on the content of the conversation included in the newly acquired voice information of the driver D.

[0029] 7 is a flowchart showing an example of an information processing procedure of the intimacy level based on the conversation content input by the driver D. The flowchart shows information processing executed by the intimacy level determination unit 24 to calculate the intimacy level from the conversation content included in the voice information of the driver D. In the following, it is assumed that the intimacy level of the driver D is already stored in the intimacy level database 25.

[0030] 7, when the ignition switch of the vehicle is turned on, the following information processing is executed. In step S102, the occupant identification unit 21 identifies the driver D based on the image information acquired from the vehicle sensors 3. Next, in step S103, the intimacy determination unit 24 acquires the voice information of the driver D inputted via the input device 4.

[0031] In the next step S104, the intimacy determination unit 24 classifies the conversation contents included in the voice information of the driver D according to the topic field and depth as shown in Fig. 6(a), and converts them into numbers as shown in Fig. 6(b) to calculate the intimacy. In step S105, it is determined whether the calculated intimacy score is higher than the intimacy of the driver D recorded in the intimacy database 25.

[0032] If it is determined in step S105 that the calculated intimacy is higher than the intimacy of driver D stored in the intimacy database 25, the process proceeds to step S106, where the intimacy of driver D is increased and updated. On the other hand, if it is determined in step S105 that the calculated intimacy is not higher than the intimacy of driver D recorded in the intimacy database 25, the intimacy of driver D stored in the intimacy database 25 is maintained as it is.

[0033] In step S108, the information processing from step S103 to step S107 is repeatedly executed until the ignition switch of the vehicle is turned off. The calculation of the intimacy degree may be repeated at a predetermined interval while the driver D is in the vehicle, or the information processing may be executed once each time the driver D gets in the vehicle.

[0034] The intimacy determination unit 24 may estimate the intimacy between the driver D and the agent 52 based on the attribute information (identification information), image information, and voice information of the driver D received from the occupant identification unit 21. Specifically, after the agent 52 outputs an expression mode that imitates the driver D, the image information and voice information of the driver D are acquired. Then, the reaction of the driver D contained in the image information and voice information is detected, and the expression mode output by the agent 52 is compared with the expression mode in which the driver D responded. The intimacy is estimated to be higher as the similarity between the expression mode output by the agent 52 and the expression mode in which the driver D responded is higher, and the intimacy is estimated to be lower as the similarity is lower. In this way, by estimating the intimacy based on the reaction of the driver D to the expression mode output by the agent 52, appropriate communication between the driver D and the agent 52 can be further promoted.

[0035] For example, as shown in FIG. 8(a), it is assumed that the agent 52 outputs a conversational manner of "I understand" and a motion manner of <making a circle with thumb and index finger>. In response to this expression manner of the agent 52, as shown in FIG. 8(b), if the driver D responds with a motion manner of <pointing thumb up>, the similarity between the motion manner of <making a circle with thumb and index finger> output by the agent 52 and the motion manner of the driver D's response <pointing thumb up> is low, so the intimacy is determined to be low. In contrast, as shown in FIG. 8(c), if the driver D responds with a motion manner of <making a circle with thumb and index finger>, the similarity between the motion manner output by the agent 52 and the motion manner of the driver D's response is high, so the intimacy is determined to be high. Similarly, the conversational manner output by the agent 52 and the conversational manner of the driver D's response are compared, and the intimacy is determined based on the similarity. The intimacy determination unit 24 may update the intimacy once estimated based on newly acquired image information and voice information of the driver D as appropriate.

[0036] Fig. 9 is a flowchart showing an example of an information processing procedure of the intimacy level based on the reaction of the driver D. This is a flowchart showing information processing executed by the intimacy level determination unit 24 to estimate the intimacy level based on the response of the driver D to the expression mode of the agent 52. In the following, it is assumed that the intimacy level of the driver D has already been stored in the intimacy level database 25. Note that the information processing of the intimacy level based on the conversation content of the driver D in Fig. 7 and the information processing of the intimacy level based on the reaction of the driver D in Fig. 9 may be appropriately combined and processed.

[0037] First, in step S111 of FIG. 9, when the ignition switch of the vehicle is turned on, the following information processing is executed. In step S112, the occupant identification unit 21 identifies the driver D based on the image information acquired from the vehicle sensors 3. Next, in step S113, the intimacy determination unit 24 determines whether or not the agent 52 has output an expression manner that imitates the driver D. If the agent 52 has output, for example, a conversation manner of "I got it" and an action manner of <making a circle with thumb and index finger> as shown in FIG. 8(a), the process proceeds to step S114. On the other hand, if the agent 52 has not output an expression manner that imitates the driver D, step S113 is repeated for a predetermined time.

[0038] If it is determined in step S113 that the agent 52 has output an expression mode that imitates the driver D, in step S114, the intimacy determination unit 24 acquires image information and / or audio information of the driver D to detect the reaction of the driver D after the agent 52 has output the expression mode. In the following step S115, the intimacy determination unit 24 determines whether the driver D has responded to the expression mode of the agent 52 based on the image information and / or audio information of the driver D. If it is determined that the driver D has responded to the expression mode of the agent 52, the process proceeds to step S116. On the other hand, if it is determined that the driver D has not responded to the expression mode of the agent 52, the process proceeds to step S118.

[0039] If it is determined in step S115 that the driver D responded to the expression mode of the agent 52, the expression mode output by the agent 52 is compared with the expression mode in response by the driver D in step S116 to determine the similarity. For example, in response to the conversation mode of "I got it" and the action mode of <making a circle with thumb and index finger> output by the agent 52 in FIG. 8(a), if the driver D responds with the action mode of <making a circle with thumb and index finger> as shown in FIG. 8(c), it is determined that the similarity is high, and the process proceeds to step S117, where the intimacy degree of the driver D is increased and updated. On the other hand, in the case where the driver D responds with the action mode of <pointing thumb up> as shown in FIG. 8(b), it is determined that the similarity between the action mode output by the agent 52 and the action mode in response by the driver D is low, and the process proceeds to step S118.

[0040] If the result of the judgment in step S115 is that driver D is not responding to the expression manner of agent 52, and if the result of the judgment in step S116 is that the similarity between the expression manner output by agent 52 and the expression manner in which driver D responds is low, the intimacy level of driver D is maintained as it is in step S118.

[0041] In step S119, the information processing from step S113 to step S118 is repeatedly executed until the ignition switch of the vehicle is turned off. Note that the estimation of the intimacy level may be configured to be repeated at a predetermined interval while the driver D is in the vehicle, or may be configured to execute the information processing once each time the driver D gets in the vehicle.

[0042] Furthermore, the intimacy determination unit 24 may estimate the intimacy between the driver D and the agent 52 based on the use history of the various in-vehicle devices of the driver D acquired from the vehicle sensors 3. For example, when the frequency of use of the in-vehicle devices such as the driving assistance device, the navigation device, and the agent device 5 is equal to or higher than a predetermined value, it is determined that the receptivity to the in-vehicle devices is high, and therefore the intimacy between the driver D and the agent 52 is estimated to be high. On the other hand, when the frequency of use of the in-vehicle devices is equal to or lower than a predetermined value, it is determined that the receptivity to the in-vehicle devices is low, and therefore the intimacy between the driver D and the agent 52 is estimated to be low. The predetermined value is not particularly limited, but is a value suitable for determining the receptivity to the in-vehicle devices, such as the frequency of use of the various in-vehicle devices being three times for a vehicle ride history of five times. By estimating the intimacy based on the use status of the various in-vehicle devices, the receptivity of the driver D to the in-vehicle devices can be reflected in the communication between the driver D and the agent 52.

[0043] 1, the expression mode setting unit 26 determines the expression mode of the agent 52 to be output to the driver D. Specifically, when a trigger expression is detected in the image information of the driver D acquired from the vehicle sensors 3 or the voice information of the driver D acquired from the input device 4, the expression mode of the driver D stored in the expression mode database 23 is referenced and the expression mode of the agent 52 to be imitated is determined.

[0044] Furthermore, the expression mode setting unit 26 sets a degree of imitation for the determined expression mode of the agent 52. The degree of imitation is the degree of similarity of the expression mode of the agent 52 to the expression mode of the driver D. The higher the degree of imitation, the higher the similarity between the two and the closer the expression mode of the agent 52 is to the expression mode of the driver D. The lower the degree of imitation, the lower the similarity between the two and the farther the expression mode of the agent 52 is from the expression mode of the driver D.

[0045] As described above, when the agent 52 communicates in an expression style similar to that used by the driver D, it is possible to provide information that is easy for the driver D to understand. However, if the agent 52 imitates the expression style of the driver D too much, such as suddenly imitating the expression style of the driver D exactly, the driver D may feel uncomfortable or uncomfortable with the expression of the agent 52. For this reason, the expression style setting unit 26 sets the degree of imitation so that the expression style of the agent 52 gradually approaches the expression style of the user over time, i.e., as communication is repeated.

[0046] In this way, the expression style used by the driver D is set as the target of imitation, and the expression style of the agent 52 is determined, and the degree of imitation is set so that the expression style of the agent 52 gradually approaches that of the driver D as communication is repeated. This makes it possible to suppress the driver D from feeling uncomfortable or uncomfortable about the expression of the agent 52, and to carry out appropriate communication while using expressions that are easy for the driver D to understand.

[0047] After setting the degree of imitation, the expression mode setting unit 26 determines the degree of imitation to be used when the agent 52 outputs communication information. Then, together with the expression mode of the agent 52 determined as the imitation target, information on the degree of imitation output by the agent 52 is output to the data generating unit 27. Note that the process of determining the expression mode of the agent 52 and the process of setting the degree of imitation by the expression mode setting unit 26 will be described later.

[0048] After setting the degree of imitation, the expression mode setting unit 26 may use the intimacy between the driver D and the agent 52 to determine the degree of imitation used when the agent 52 outputs communication information. Specifically, the intimacy between the driver D and the agent 52 stored in the intimacy database 25 is obtained via the intimacy determination unit 24, and when the intimacy is relatively high, it is determined to output an expression mode with a higher degree of imitation than when the intimacy is relatively low.

[0049] The higher the degree of imitation, the closer the expression mode of the agent 52 is to that of the driver D, so the driver D is more likely to notice the imitation. In contrast, the lower the degree of imitation, the further the expression mode is from that of the driver D, so the driver D is less likely to notice the imitation. When the intimacy between the driver D and the agent 52 is low, the driver D is less receptive to the agent 52 and is more likely to feel that the imitation is excessive. Therefore, an expression mode with a low degree of imitation is output, making it more difficult for the driver D to notice the imitation.

[0050] On the other hand, when the intimacy between the driver D and the agent 52 is relatively high, the driver D is highly receptive to the agent 52 and is unlikely to feel that the imitation is excessive, so an expression mode with a high degree of imitation may be output. This is because, as the intimacy increases, the driver D is more likely to feel attached to the agent 52 when he notices that the expression mode of the agent 52 has become similar to his own expression mode. In addition, when the intimacy is particularly high, a unique expression mode used by the driver D (for example, the conversation mode of "Roger" and the action mode of <holding one hand up and the other hand to the side> shown in FIG. 4) may be output. In this way, by appropriately changing the degree to which the agent 52 imitates the expression mode of the driver D, smooth communication can be performed.

[0051] The data generation unit 27 generates output data to be output to the driver D based on the expression mode of the agent 52 to be imitated and the information on the degree of imitation of the expression mode received from the expression mode setting unit 26. When generating output data for a conversation mode, first, text data (character strings) is generated, the text data is converted into voice data by voice synthesis processing, and this is sent to the output unit 28 as output data for the conversation mode output by the agent 52. When generating output data for a movement mode, digital data is generated using a preset movement path or motion capture, and this is sent to the output unit 28 as output data for the movement mode output by the agent 52. Known technologies can be applied to the voice synthesis processing, motion capture, and digital data generation.

[0052] Furthermore, the data generating unit 27 may set the output frequency of the output data by the agent device 5. The higher the output frequency of the output data, the higher the frequency of communication between the driver D and the agent 52, and the communication is promoted. For this reason, the data generating unit 27 may refer to, for example, the intimacy between the driver D and the agent 52, and set the output frequency of the expression mode of the agent 52 to be high when the intimacy is low. By increasing the output frequency, it is possible to expect an effect of giving the impression that the agent 52 is interested in the driver D. Note that the output frequency may be set in consideration of both the intimacy and the degree of imitation, such that when the intimacy is low, an expression mode with a low degree of imitation is output with high frequency, when the intimacy is medium, an expression mode with a medium degree of imitation is output with medium frequency, and when the intimacy is high, an expression mode with a high degree of imitation is output with low frequency. In addition, when multiple expression modes are used, the output frequency may be set for each expression mode, in addition to changing the output frequency of one expression mode output by the agent 52.

[0053] When the output unit 28 receives output data from the data generation unit 27, it outputs a control signal to the operating unit, speaker or other audio output unit, display or other display unit of the agent device 5, and outputs the generated output data by the agent function of the agent device 5.

[0054] Next, a process of determining the expression mode of the agent 52 to be imitated and a process of setting the degree of imitation for the expression mode, performed by the expression mode setting unit 26, will be described. As described above, when the expression mode setting unit 26 detects a trigger expression included in the image information of the driver D acquired from the vehicle sensors 3 or the voice information of the driver D acquired from the input device 4, the expression mode setting unit 26 refers to the expression mode of the driver D stored in the expression mode database 23 and determines the expression mode of the agent 52 to be imitated. Then, the degree of imitation is set for the determined expression mode.

[0055] FIG. 10 is a diagram for explaining information processing for determining the expression mode of the agent 52 to be imitated. As shown in FIG. 10, when the expression mode setting unit 26 detects a trigger expression of "agreement" such as "That's fine" included in the voice information of the driver D acquired via the input device 4, the expression mode setting unit 26 extracts an expression mode related to "agreement" from the expression modes of the driver D stored in the expression mode database 23. The expression mode of the agent 52 to be imitated is determined from the extracted expression modes related to "agreement", for example, a conversation mode of "Okay" and a motion mode of <pointing thumb up>, a conversation mode of "Got it" and a motion mode of <making a circle with thumb and index finger>, a conversation mode of "Understood" and a motion mode of <salute>, and a conversation mode of "Roger" and a motion mode of <raising one hand up and pointing the other hand to the side>. After determining the expression mode of the agent 52 to be imitated, the expression mode setting unit 26 specifies a change target for bringing the expression mode of the agent 52 closer to the expression mode of the driver D and sets the degree of imitation.

[0056] 11 to 13 are diagrams showing an example of a scene in which the imitation degree of the expression mode of the agent 52 is set. Each of Figs. 11(a) to 11(c) is a scene in which the imitation degree is set using a different expression mode. For example, when the action mode and conversation mode of "approval" are determined as the imitation target of the agent 52, the expression mode setting unit 26 specifies the detection count of each expression mode stored in the expression mode database 23 as the change target. The more the detection count of an expression mode is, the closer it is to the expression mode used by the driver D. Therefore, the expression mode setting unit 26 sets the imitation degree higher as the detection count of an action mode and conversation mode is used. Note that, when the detection count is set as the change target, it is possible to suppress an unnatural event in which the agent 52 frequently imitates an expression mode with a low detection count, that is, an expression mode accidentally taken by the driver D.

[0057] In the scenes shown in Fig. 11(a) to (c), as shown in Fig. 11(a), the conversational mode of "Got it" and the motion mode of <Salute>, which are detected less frequently, are set to a low imitation degree. Also, as shown in Fig. 11(b), the conversational mode of "Okay" and the motion mode of <Pointing thumb up>, which are detected more frequently, are set to a medium imitation degree. Similarly, as shown in Fig. 11(c), the conversational mode of "Got it" and the motion mode of <Making a circle with thumb and index finger>, which are detected more frequently, are set to a high imitation degree. Then, the agent 52 determines the imitation degree to be reflected in the output data to be generated this time so that the agent 52 outputs the expression modes of Fig. 11(a) → Fig. 11(b) → Fig. 11(c) over time, and transmits a command signal to the data generating unit 27. Note that in this embodiment, the imitation degree is set to three levels, namely, "low", "medium", and "high", but is not limited thereto, and the imitation degree may be two levels or three levels or more.

[0058] 12(a) to (c) are scenes in which the degree of imitation is set using the same expression mode. For example, when the expression mode setting unit 26 determines that the target of imitation for the agent 52 is the motion mode of <making a circle with thumb and index finger> among the expression modes of "approval", the expression mode setting unit 26 specifies the hand shape as the target of change in order to make the motion mode of the agent 52 closer to the motion mode of the driver D. The more similar the hand shape of the agent 52 is to the motion mode of the driver D of <making a circle with thumb and index finger>, the higher the degree of imitation is set.

[0059] In the scenes shown in Fig. 12(a) to (c), as shown in Fig. 12(a), the shape <raising one hand>, which has a low similarity to the shape <making a circle with thumb and index finger>, is set to a low imitation degree. Also, as shown in Fig. 12(b), the shape <making a circle with one hand>, which has a medium similarity to the shape <making a circle with thumb and index finger>, is set to a medium imitation degree. As shown in Fig. 12(c), the shape <making a circle with thumb and index finger>, which has a high similarity to the shape <making a circle with thumb and index finger>, is set to a high imitation degree. Then, the imitation degree to be reflected in the output data to be generated this time is determined so that the agent 52 outputs the motion modes of Fig. 12(a) → Fig. 12(b) → Fig. 12(c) over time, and a command signal is output to the data generating unit 27.

[0060] Each of Fig. 13(a) to (c) is a scene in which the degree of imitation is set using a plurality of expression modes. The expression mode setting unit 26 may set the degree of imitation using a plurality of movement modes and conversation modes. By using a plurality of expression modes, the expression used by the agent 52 can be varied and changed stepwise. For example, the expression mode setting unit 26 determines one of the imitation targets of the agent 52 to be the conversation mode of "approval" and specifies the number of times the conversation mode of "approval" is detected as the change target. Furthermore, the expression mode setting unit 26 determines the movement mode of "making a circle with thumb and index finger" of another imitation target and specifies the hand shape as the change target. The more the conversation mode with a higher number of detections is used, and the more similar the hand shape of the agent 52 is to the shape of the driver's movement mode of "making a circle with thumb and index finger", the higher the degree of imitation is set.

[0061] In the scenes shown in Fig. 13(a) to (c), as shown in Fig. 13(a), the shape <raising one hand>, which has a low similarity to the conversational style of "got it" that is detected less frequently, is set to a low imitation degree. Also, as shown in Fig. 13(b), the shape <making a circle with one hand>, which has a medium similarity to the conversational style of "okay" that is detected more frequently, is set to a medium imitation degree. Furthermore, as shown in Fig. 13(c), the shape <making a circle with thumb and index finger>, which has a high similarity to the conversational style of "got it" that is detected more frequently, is set to a high imitation degree. Then, the agent 52 determines the imitation degree to be reflected in the output data to be generated this time so that the agent 52 outputs the expression style of Fig. 13(a) → Fig. 13(b) → Fig. 13(c) over time, and outputs a command signal to the data generating unit 27.

[0062] The expression mode setting unit 26 may use a plurality of action modes and conversation modes to increase the number of expression modes to be imitated over time. For example, the degree of imitation may be set low when one expression mode is used, medium when two expression modes are used, and high when three expression modes are used. In this case, the output data is reflected so that the agent 52 imitates one expression mode, two expression modes, and three expression modes over time.

[0063] Furthermore, the expression mode setting unit 26 may determine the voice characteristics, expression characteristics, and conversation content characteristics contained in the voice information of the driver D as the conversation mode to be imitated, and may specify items related to each characteristic as the change target to set the degree of imitation in order to come closer to the voice information of the driver D. Voice characteristics are difficult for the driver D to notice imitation, while expression characteristics are somewhat easy for the driver D to notice imitation. Furthermore, conversation content characteristics are easy for the driver D to notice imitation. The expression mode setting unit 26 sets the degree of imitation higher, assuming that the more characteristics that are easy to notice imitation are set as the imitation target, the closer the driver D's voice information is to the driver D's voice information.

[0064] As shown in FIG. 14, when speech characteristics such as speed, tone, voice quality, and voice volume are to be imitated, the degree of imitation is set to low. When expression characteristics such as intonation, dialect, phrasing (use of polite language, popular phrases), and accent are to be imitated, the degree of imitation is set to medium. When conversation content characteristics such as topics, preferences, and negative information about a specific person are to be imitated, the degree of imitation is set to high. However, for negative information about a specific person, it is preferable to set a predetermined condition and, for example, to set the voice information input by the driver D as the object of imitation only when the voice information includes negative information about a specific person. The expression mode setting unit 26 reflects this in the output data so that the agent 52 imitates the voice characteristics, expression characteristics, and conversation content characteristics over time.

[0065] Furthermore, the expression mode setting unit 26 may specify conditions related to time, place, and situation included in the behavior and conversation modes of the driver D, and set the degree of imitation by changing these conditions. By using these conditions, it is possible to set imitations in stages, from ones that are easy for the driver D to notice to ones that are difficult to notice, and appropriate communication can be performed using a variety of expressions.

[0066] Fig. 15 is a diagram showing an example in which conditions related to time, place, and situation are changed for the behavior of driver D. The higher the degree of imitation, the closer the behavior is to that of driver D, and the settings are made to reproduce the state when the behavior is detected from driver D.

[0067] When the condition related to the time of the behavioral state is to be changed, for example, the timing when the agent 52 outputs the behavioral state or the length of time when the agent 52 outputs the behavioral state is changed. The scene shown in FIG. 15 is a scene in which the agent 52 imitates the behavioral state of the driver D, which is <yawning>, consisting of three actions, <closing the eyes>, <putting one hand to the mouth>, and <stretching>, and the time condition is to be changed. In the scene shown in FIG. 15, when the timing when the behavioral state is output is delayed by one action and the two actions, <closing the eyes> and <putting one hand to the mouth>, are output, the degree of imitation is set to low. When the timing when the behavioral state is output is delayed by one action and the three actions, <closing the eyes>, <putting one hand to the mouth>, and <stretching>, are output, the degree of imitation is set to medium. On the other hand, when the timing when the behavioral state is output is aligned and the three actions, <closing the eyes>, <putting one hand to the mouth>, and <stretching>, are output, the degree of imitation is set to high.

[0068] Similarly, when the condition related to the location of the movement mode is to be changed, for example, the location or position imitated by the agent 52 is changed. The scene shown in FIG. 15 is a scene in which the movement mode of the driver D <stretching the left arm> is to be imitated and the location condition is to be changed. In the scene shown in FIG. 15, when the movement mode of <stretching the left leg> at a location away from the left arm is output, the imitation degree is set to low. Also, when the movement mode of <stretching the right arm> at a location opposite the left arm is output, the imitation degree is set to medium, and when the movement mode of <stretching the left arm> at the same location is output, the imitation degree is set to high.

[0069] When the condition related to the situation of the behavior is to be changed, for example, the situation imitated by the agent 52 is changed. The scene shown in FIG. 15 is a scene in which the behavior of the driver D <yawning> is to be imitated and the condition related to the situation is to be changed. The behavior of the driver D <yawning> is detected in the morning, so the condition related to the situation is morning. In the scene shown in FIG. 15, when the behavior of the driver D <yawning> is output in the evening, which is a different situation from the morning, the degree of imitation is set low. When the behavior of the driver D <yawning> is output in the afternoon, which is a situation somewhat similar to the morning, the degree of imitation is set to medium, and when the behavior of the driver D <yawning> is output in the morning, which is the same situation, the degree of imitation is set high.

[0070] The expression mode setting unit 26 may set the imitation degree by changing one of the conditions related to the time, place, and situation for the operation mode, or may set the imitation degree by changing a plurality of conditions. The expression mode setting unit 26 may also set the imitation degree by associating the conditions related to the time, place, and situation to be changed with the intimacy of the driver D. For example, when the intimacy of the driver D is low, all of the conditions related to the time, place, and situation are changed to make it difficult for the driver D to notice that the imitation is occurring. When the intimacy of the driver D is medium, two of the conditions related to the time, place, and situation are changed. When the intimacy of the driver D is high, the condition related to the situation is a condition with high empathy, so that it is not changed, and either the time or the place may be changed. The expression mode setting unit 26 reflects this in the output data so that the agent 52 uses an expression mode with a low degree of imitation over time, followed by an expression mode with a high degree of imitation.

[0071] FIG. 16 is a diagram showing an example of changing conditions related to time, place, and situation for the conversation style of the driver D. The higher the degree of imitation, the higher the responsiveness to the conversation style of the driver D, and the agent 52 is set to output an empathetic response. As shown in FIG. 16, for example, when a conversation content such as "National Route 129 is jammed today" is detected from the conversation style of the driver D, the words "today" as the time condition, "National Route 129" as the place condition, and "jam jammed" as the situation condition are extracted and changed. Note that the time condition and current time information, the place condition and vehicle position information, the situation condition and vehicle surrounding information, etc. may be changed in association with each other based on the detection value acquired via the vehicle sensors 3.

[0072] When the time-related condition of the conversational behavior is to be changed, for example, a word related to time is changed and the current time information is changed in association with it. In the scene shown in FIG. 16, the word "today" is changed as the time-related condition, and the current time information is changed in association with it to "commute time zone". When a response sentence such as "National Route 129 is congested during the holiday season", which is a concept far removed from the time "today", is used, the degree of imitation is set to low. When a response sentence such as "National Route 129 is congested again this week", which is a concept somewhat far removed from the time "today", is used, the degree of imitation is set to medium. On the other hand, when a response sentence such as "National Route 129 is congested because it is commuting time", which is associated with the current time information within the time "today", is used, the degree of imitation is set to high.

[0073] Similarly, when the location-related condition of the conversational manner is to be changed, for example, the word related to the location is changed and the vehicle location information is changed in association. In the scene shown in FIG. 16, the word "National Route 129" is changed as the location-related condition, and the vehicle location information "near Atsugi Bypass" is changed in association. When a response sentence such as "Main roads are often congested," which is a concept far removed from the location "National Route 129," is used, the degree of imitation is set to low. When a response sentence such as "National highways are often congested," which is a concept somewhat far removed from the location "National Route 129," is used, the degree of imitation is set to medium. On the other hand, when a response sentence such as "Near the Atsugi Bypass on National Route 129," which is associated with the vehicle location information within the location "National Route 129," is used, the degree of imitation is set to high.

[0074] When the condition related to the conversational style situation is to be changed, for example, words related to the situation are changed and the surrounding information of the vehicle is associated and changed. In the case of the scene shown in FIG. 16, the word "traffic jam" is changed as the condition of the situation, and the surrounding information of the vehicle "near lane regulation" is associated and changed. When a response sentence such as "There are times when national highway 129 is not congested," which is a concept far removed from the situation of "traffic jam," is used, the degree of imitation is set to low. When a response sentence such as "National highway 129 is congested," which is the same concept as the situation of "traffic jam," is used, the degree of imitation is set to medium. On the other hand, when a response sentence such as "National highway 129 seems to be congested because of lane regulation ahead," which is associated with the surrounding information of the vehicle in addition to the situation of "traffic jam," is used, the degree of imitation is set to high.

[0075] The expression mode setting unit 26 may set the degree of imitation for the conversation mode by changing one of the conditions related to time, place, and situation, or may set the degree of imitation for a plurality of conditions. The expression mode setting unit 26 may set the degree of imitation by associating the conditions related to time, place, and situation to be changed with the intimacy of the driver D. For example, when the intimacy of the driver D is low, all of the conditions related to time, place, and situation are changed to make it difficult for the driver D to notice that the imitation is occurring. When the intimacy of the driver D is medium, two of the conditions related to time, place, and situation are changed. When the intimacy of the driver D is high, the condition related to the situation is a condition with high empathy, so that it is not changed, and either the time or the place may be changed. The expression mode setting unit 26 reflects this in the output data so that the agent 52 uses an expression mode with a low degree of imitation over time, followed by an expression mode with a high degree of imitation.

[0076] Next, an information processing procedure of the information processing system 1 of this embodiment will be described with reference to Fig. 17 and Fig. 18. Fig. 17 is a flowchart showing an example of information processing for determining the expression mode of the agent 52 and setting the degree of imitation, which is executed by the information processing system 1 of this embodiment. Fig. 18 shows an example of a subroutine of step S8 shown in Fig. 17. This is a flowchart showing an example of information processing for determining the degree of imitation output by the agent 52. The following description will be given with reference to the scenes in Fig. 10 and Figs. 11(a) to (c).

[0077] 17, when the ignition switch of the vehicle is turned ON, the following information processing is executed. In step S2, the occupant identification unit 21 identifies the driver D based on image information acquired from an in-vehicle camera serving as the vehicle sensors 3. Next, in step S3, the expression mode setting unit 26 refers to the expression mode database 23 to acquire the expression mode stored for the driver D, the number of times the expression mode has been detected, and the intimacy between the driver D and the agent 52.

[0078] In step S4, the expression mode setting unit 26 judges whether or not a trigger expression has been detected based on the image information of the driver D acquired from the vehicle sensors 3 and the voice information of the driver D acquired from the input device 4. For example, if the trigger expression "That's fine (agreement)" shown in Fig. 10 is detected, the process proceeds to step S5. In response to this, step S4 is repeated for a predetermined period of time until the trigger expression is detected.

[0079] When it is determined in step S4 that a trigger expression has been detected, in step S5, the expression mode setting unit 26 determines the expression mode of the agent 52 to be imitated. As shown in Fig. 10, when the trigger expression "agreement" of the driver D is detected, the expression mode setting unit 26 refers to the expression mode database 23 and determines the action mode and conversation mode of "agreement" from the expression modes of the driver D as the ones to be imitated.

[0080] In the next step S6, the expression mode setting unit 26 specifies a change target for the action mode and conversation mode of “agreement” determined as the imitation target, in order to bring the expression mode of the agent 52 closer to the expression mode of the driver D. For example, the change target is the detection count of the action mode and conversation mode shown in FIG.

[0081] In the next step S7, the expression mode setting unit 26 sets the degree of imitation of the expression mode of the agent 52. At this time, the expression mode setting unit 26 sets the expression mode of the agent 52 so that it gradually approaches the expression mode of the driver D as communication is repeated. When the number of detections is the object of change, the expression mode that is detected more frequently can be closer to the expression mode used by the driver D. Therefore, for example, as shown in FIG. 11(a), the conversation mode of "Got it" and the action mode of <Salute>, which are detected less frequently, are set to a low degree of imitation. Also, as shown in FIG. 11(b), the conversation mode of "Okay" and the action mode of <Point thumb up>, which are detected a medium number of times, are set to a medium degree of imitation. On the other hand, as shown in FIG. 11(c), the conversation mode of "Got it" and the action mode of <Make a circle with thumb and index finger>, which are detected a large number of times, are set to a high degree of imitation.

[0082] In step S8, the expression mode setting unit 26 determines the degree of imitation of the expression mode currently output by the agent 52, and proceeds to step S9. In the following step S9, the data generation unit 27 generates output data for the driver D based on the information received from the expression mode setting unit 26 about the expression mode of the agent 52 to be imitated and the degree of imitation of the expression mode.

[0083] In step S10, the output unit 28 outputs control signals to the operating unit of the agent device 5, a speaker or other audio output unit, and a display or other display unit, and outputs output data by the agent function of the agent device 5.

[0084] In step S11, the information processing from step S4 to step S10 is repeatedly executed until the ignition switch of the vehicle is turned off.

[0085] In step S8 in Fig. 17, information processing is executed to determine the degree of imitation of the expression mode currently output by the agent 52, as shown in Fig. 18. When the degree of imitation of the expression mode is set in step S7, in step S81 in Fig. 18, the expression mode setting unit 26 acquires the intimacy between the driver D and the agent 52 stored in the intimacy database 25 via the intimacy determination unit 24.

[0086] If the intimacy level of the driver D acquired in step S81 is low, the process proceeds to step S82, where the imitation level to be output this time is determined to be "low". In this case, as shown in FIG. 11(a), the conversational style of "Got it" and the action style of "Salute" which have a low imitation level are output. On the other hand, if the intimacy level of the driver D acquired in step S81 is medium, the process proceeds to step S83, where the imitation level to be output this time is determined to be "medium". In this case, as shown in FIG. 11(b), the conversational style of "Okay" and the action style of "Pointing your thumb up" which have a medium imitation level are output. If the intimacy level of the driver D acquired in step S81 is high, the process proceeds to step S84, where the imitation level to be output this time is determined to be "high", and as shown in FIG. 11(c), the conversational style of "Got it" and the action style of "Making a circle with your thumb and index finger" which have a high imitation level are output.

[0087] As described above, according to the information processing system 1 and the information processing method of the present embodiment, the expression mode of the driver D (user) is detected, the expression mode of the agent 52 to be imitated is determined, and for the expression mode, the degree of imitation of the expression mode of the agent 52 relative to the detected expression mode of the driver D (user) is set, and output data according to the degree of imitation is generated. At that time, the degree of imitation of the expression mode of the agent 52 is set so as to approach the expression mode of the driver D (user) over time as the generated output data is output to the driver D (user). This allows appropriate communication to be performed while using expressions that are easy for the driver D (user) to understand. In addition, it is possible to suppress the driver D (user) from feeling uncomfortable or becoming uncomfortable with the expression of the agent 52.

[0088] Moreover, according to the information processing system 1 and the information processing method of the present embodiment, an intimacy determination unit 24 that determines the intimacy between the driver D (user) and the agent 52 is further provided, and the expression mode setting unit 26 sets the degree of imitation to be closer to the expression mode of the driver D (user) when the intimacy between the driver D (user) and the agent 52 is relatively high, compared to when the intimacy between the driver D (user) and the agent 52 is relatively low. This allows the degree to which the agent 52 imitates the expression mode of the driver D (user) to be appropriately changed, enabling smooth communication. In addition, an effect that the driver D (user) will easily feel attached to the agent 52 can be expected.

[0089] Moreover, according to the information processing system 1 and the information processing method of the present embodiment, the intimacy determination unit 24 estimates the intimacy between the driver D (user) and the agent 52 based on the contents of the conversation input from the driver D (user). This makes it possible to reflect the actual words and actions of the driver D (user) toward the agent 52 in the intimacy, and promotes appropriate communication between the driver D (user) and the agent 52.

[0090] Moreover, according to the information processing system 1 and the information processing method of the present embodiment, the intimacy determination unit 24 acquires the reaction of the driver D (user) to the output expression mode of the agent 52, and estimates the intimacy between the driver D (user) and the agent 52 based on the acquired reaction of the driver D (user). This can further promote appropriate communication between the driver D (user) and the agent 52.

[0091] According to the information processing system 1 and the information processing method of the present embodiment, the agent device 5 is an in-vehicle device mounted in a vehicle, the user is a vehicle occupant, and the intimacy determination unit 24 estimates the intimacy between the driver D (user) and the agent 52 based on the driver D's (user's) usage of the in-vehicle device mounted in the vehicle, including the driving assistance device. This makes it possible to reflect the receptivity to the in-vehicle device in the communication between the driver D (user) and the agent 52.

[0092] According to the information processing system 1 and the information processing method of the present embodiment, the expression mode of the driver D (user) includes an action mode detected from the visual characteristics of the driver D (user) and a conversation mode detected from the auditory characteristics of the driver D (user), and the expression mode setting unit 26 determines the expression mode of the agent 52 to be imitated by using the action mode of the driver D (user) and / or the conversation mode of the user. As a result, the agent 52 imitates the expression actually used by the driver D (user), so that communication can be performed in a mode that is easy for the driver D (user) to understand. In addition, by the agent 52 imitating the expression actually used by the driver D (user), it is expected that the expression of the agent 52 will be favorably received by the driver D (user).

[0093] According to the information processing system 1 and the information processing method of the present embodiment, the visual characteristics of the driver D (user) include at least one of the appearance, figure, posture, facial expression, and movement of the driver D (user) acquired from image information capturing the driver D (user). Since the driver D (user) is likely to notice imitation of the movement mode detected from the visual characteristics, the agent 52 gradually imitates the driver D, thereby preventing the driver D from being given the impression of excessive imitation.

[0094] According to the information processing system 1 and the information processing method of the present embodiment, the auditory characteristics of the driver D (user) include at least one of the voice quality, intonation, expression, language, dialect, conversation content, estimated age, and gender of the driver D (user) acquired from the voice information of the driver D (user). Since the driver D (user) is less likely to notice imitation of a conversation pattern detected from the auditory characteristics than an action pattern, the agent 52 can gradually imitate the driver D to achieve good communication.

[0095] According to the information processing system 1 and the information processing method of the present embodiment, the expression mode setting unit 26 sets the degree of imitation to approach the expression mode of the driver D (user) by using the movement modes and / or conversation modes of the drivers D (users). This makes it possible to change the expression used by the agent 52 in a diverse and stepwise manner.

[0096] According to the information processing system 1 and the information processing method of the present embodiment, the expression mode setting unit 26 identifies conditions including at least one of time, place, and situation included in the behavior mode and / or conversation mode of the driver D (user), and sets the degree of imitation to be close to the expression mode of the driver D (user) using the conditions. By using these conditions, it is possible to set the imitation in stages, from one that is easy for the driver D (user) to notice to one that is difficult to notice, and appropriate communication can be performed using a variety of expressions.

[0097] Furthermore, according to the information processing system 1 and the information processing method of the present embodiment, when the intimacy between the driver D (user) and the agent 52 is relatively low, the output frequency of outputting the expression mode of the agent 52 is increased compared to when the intimacy between the driver D (user) and the agent 52 is relatively high. This can promote communication between the driver D (user) and the agent 52.

[0098] The above-described embodiments are described for the purpose of facilitating understanding of the present invention, and are not described for the purpose of limiting the present invention. Therefore, the elements disclosed in the above embodiments are intended to include all design modifications and equivalents that fall within the technical scope of the present invention. [Explanation of symbols]

[0099] 1. Information processing system 2. Information processing device 21…Occupant Identification Section 22... Expression mode detection unit 23...Expression form database 24…Intimacy level determination section 25…Intimacy Database 26... Expression mode setting section 27…Data generation section 28...Output section 3. Vehicle sensors 4. Input device 5. Agent device

Claims

1. 1. An information processing system comprising: an agent device including a personified agent that imitates an expression manner of a user; and an information processing device that sets the expression manner of the agent and generates output data for the user, the information processing system outputting the generated output data to the user using the agent, The information processing device includes: A detection unit that detects an expression mode of the user; a determination unit for determining a degree of intimacy between the user and the agent; a setting unit that determines an expression mode of the agent to be imitated, and sets a degree of imitation of the expression mode of the agent relative to the detected expression mode of the user for the determined expression mode; a data generating unit that generates output data according to the set degree of imitation, The setting unit is setting a degree of imitation of the agent's expression manner so as to approach the expression manner of the user over time as the generated output data is output to the user; An information processing system that sets the degree of imitation to be closer to the user's expression style when the intimacy between the user and the agent is relatively high, compared to when the intimacy between the user and the agent is relatively low.

2. The information processing system according to claim 1 , wherein the determination unit estimates the degree of intimacy between the user and the agent based on a conversation content input by the user.

3. 3. The information processing system according to claim 1, wherein the determination unit acquires a reaction of the user to the output expression mode of the agent, and estimates a degree of intimacy between the user and the agent based on the acquired reaction of the user.

4. The information processing system according to any one of claims 1 to 3, wherein the agent device is an in-vehicle device mounted in a vehicle, the user is a vehicle occupant, and the determination unit estimates the degree of intimacy between the user and the agent based on the user's usage status of an in-vehicle device mounted in the vehicle including a driving assistance device.

5. The expression manner of the user includes a movement manner detected from a visual characteristic of the user and a conversation manner detected from an auditory characteristic of the user; 5. The information processing system according to claim 1, wherein the setting unit determines an expression manner of the agent to be imitated by using a behavior manner of the user and / or a conversation manner of the user.

6. The information processing system according to claim 5 , wherein the visual characteristics of the user include at least one of the appearance, figure, posture, facial expression, and movement of the user acquired from image information capturing an image of the user.

7. The information processing system according to claim 5 or 6, wherein the user's auditory characteristics include at least one of the user's voice quality, intonation, phrasing, language, dialect, conversation content, estimated age, and gender obtained from the user's voice information.

8. The information processing system according to any one of claims 5 to 7, wherein the setting unit sets the degree of imitation to approximate the expression manner of the users using the movement manner and / or conversation manner of the users of multiple users.

9. The information processing system according to any one of claims 5 to 8, wherein the setting unit identifies conditions including at least one of time, place, and situation contained in the user's behavioral manner and / or conversational manner, and sets the degree of imitation using the conditions so as to approximate the user's expression manner.

10. The information processing system according to any one of claims 1 to 9, wherein when the intimacy between the user and the agent is relatively low, the output frequency of outputting the agent's expression mode is increased compared to when the intimacy between the user and the agent is relatively high.

11. 1. An information processing method for causing a processor to execute a process of setting an expression manner of an anthropomorphized agent that imitates an expression manner of a user, generating output data for the user, and outputting the generated output data to the user using the agent, comprising: The processor, Detecting an expression manner of the user; determining a degree of intimacy between the user and the agent; determining an expression mode of the agent to be imitated, and setting a degree of imitation of the expression mode of the agent relative to the expression mode of the detected user for the determined expression mode; generating output data according to the set degree of imitation; setting a degree of imitation of the agent's expression manner so as to approach the expression manner of the user over time as the generated output data is output to the user; An information processing method in which, when the intimacy between the user and the agent is relatively high, the degree of imitation is set to be closer to the expression style of the user than when the intimacy between the user and the agent is relatively low.

Images