Electronic apparatus

The electronic device addresses the challenge of inappropriate interactions by recognizing user and pet states to provide tailored advice and actions, improving family and pet interactions through emotion and sentence generation models.

JP2024159588APending Publication Date: 2024-11-08SOFTBANK GROUP CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2024066978
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-28
Filing Date
2024-04-17
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Conventional technologies lack the ability to perform appropriate actions based on the conditions of users and pets within a family setting, limiting the effectiveness of electronic devices in interacting with family members and their pets.

Method used

An electronic device that recognizes the states of both users and pets, determining its own behavior to provide appropriate actions and advice through a combination of emotion and sentence generation models, utilizing a control unit to manage interactions and provide advice based on learned user and pet behaviors and emotions.

Benefits of technology

Enables the device to perform appropriate actions and provide tailored advice, enhancing interactions within families and improving relationships between users and their pets by understanding and responding to their emotional and behavioral states.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159588000001_ABST
    Figure 2024159588000001_ABST
Patent Text Reader

Abstract

To provide an electronic apparatus for executing a proper action.SOLUTION: In a robot, a control part includes a sensor part, a sensor module part, a storage part, a state recognition part, a feeling determination part, an action recognition part, an action determination part, a storage control part, an action control part, a control object, and a communication processing part, recognizes at least either states of users constituting a family or a state of a pet kept by the users, determines an own action corresponding to the recognized state, controls the control object on the basis of the determined own action, determines an action for providing advice information about the pet corresponding to an action of the pet as an own action in the case of recognizing the action of the pet as a state of the pet, and determines an action for providing advice information about the pet corresponding to feelings of the pet as an own action in the case of recognizing the feelings of the pet as a state of the pet.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosed embodiments relate to electronic devices. [Background technology]

[0002] Conventionally, a technology has been disclosed for determining the behavior of a robot, which is an electronic device, based on the state of a user (for example, see Patent Document 1). Patent Document 1 discloses that the robot recognizes the user's reaction when the robot executes a specific action, and if the robot's action in response to the recognized user's reaction cannot be determined, the robot's action is updated by receiving information about an action suitable for the recognized state of the user from a server. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6053847 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for improvement in terms of executing appropriate actions in response to the state of the user or pets that make up the family.

[0005] The present invention has been made in view of the above, and has an object to provide an electronic device that can execute appropriate actions. [Means for solving the problem]

[0006] An electronic device in one aspect of an embodiment has a control unit that recognizes at least one of the status of a user who is a member of a family and the status of a pet kept by the user, determines its own behavior corresponding to the recognized status, and controls a control target based on the determined behavior of its own. Effect of the Invention

[0007] According to one aspect of the embodiment, appropriate actions may be taken. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a system according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a schematic functional configuration of the robot. [Diagram 3] FIG. 3 is a diagram showing an outline of the data structure of the character data. [Figure 4] FIG. 4 is a diagram illustrating an example of an operation flow relating to character setting. [Diagram 5] FIG. 5 is a diagram illustrating an example of an operation flow by the robot. [Figure 6] FIG. 6 is a diagram illustrating an example of a hardware configuration of a computer. [Figure 7] FIG. 7 is a diagram showing an emotion map onto which a plurality of emotions are mapped. [Figure 8] FIG. 8 is a diagram showing another example of the emotion map. [Figure 9] FIG. 9 is a diagram illustrating an example of an emotion table. [Figure 10] FIG. 10 is a diagram illustrating an example of the emotion table. [Figure 11] FIG. 11 is an external view and an internal structure view of a stuffed toy according to another embodiment. [Figure 12] FIG. 12 is a rear front view of a stuffed toy according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] FIG. 1 is a diagram illustrating an example of a system 5 according to the present embodiment. The system 5 includes a robot 100, a robot 101, a robot 102, and a server 300. The robots 100, 101, and 102 are examples of electronic devices. The user 10a, the user 10b, and the user 10c are users of the robot 100. The user 11a, the user 11b, and the user 11c are users of the robot 101. The user 12a and the user 12b are users of the robot 102. In the description of the present embodiment, the user 10a, the user 10b, and the user 10c may be collectively referred to as the user 10. The user 11a, the user 11b, and the user 11c may be collectively referred to as the user 11. The user 12a and the user 12b may be collectively referred to as the user 12. The robot 101 and the robot 102 have substantially the same functions as the robot 100. Therefore, the system 5 will be described mainly with reference to the functions of the robot 100.

[0011] Here, user 10a, user 10b, and user 10c constitute a family. In other words, user 10a, user 10b, and user 10c constitute a family. User 10 also keeps a pet 13. Note that, in FIG. 1, an example in which pet 13 is a dog is shown, but this is merely an example and is not limiting, and the pet may be, for example, a cat, a rabbit, a hamster, or another type of animal.

[0012] As described later, the robot 100 provides advice information about the pet 13 to the user 10, but the user 10 at this time does not have to be a member of the family. Also, as described later, the robot 100 provides advice information about the family to the user 10, but the user 10 at this time does not have to be a pet 13.

[0013] The appearance of the robot may be a human-like appearance like robot 100 and robot 101, or may be a stuffed toy like robot 102. Since robot 102 has the appearance of a stuffed toy, it is considered that children in particular will find it easy to relate to it.

[0014] The robot 100 converses with the user 10 and provides images to the user 10. At this time, the robot 100 cooperates with a server 300 or the like with which it can communicate via a communication network 20 to converse with the user 10 and provide images, etc. to the user 10. For example, the robot 100 not only learns appropriate conversation by itself, but also cooperates with the server 300 to learn how to have a more appropriate conversation with the user 10. The robot 100 also causes the server 300 to record captured image data, etc. of the user 10, and requests the server 300 for the image data, etc. as necessary, and provides the data, etc. to the user 10.

[0015] The robot 100 also has an emotion value that indicates the type of its own emotion. For example, the robot 100 has emotion values ​​that indicate the intensity of each of the emotions of "joy", "anger", "sorrow", "pleasure", "discomfort", "relief", "anxiety", "sadness", "excitement", "concern", "relief", "fulfillment", "emptiness", and "neutral". When the robot 100 converses with the user 10 in a state where the emotion value of excitement is high, for example, the robot 100 utters sounds at a fast speed. In this way, the robot 100 can express its own emotion through actions.

[0016] Furthermore, the robot 100 may be configured to match a sentence generation model (so-called an AI (Artificial Intelligence) chat engine) with an emotion engine, thereby determining an action of the robot 100 corresponding to the emotion of the user 10. Specifically, the robot 100 may be configured to recognize the action of the user 10, determine the emotion of the user 10 regarding the action of the user, and determine an action of the robot 100 corresponding to the determined emotion.

[0017] More specifically, when the robot 100 recognizes the behavior of the user 10, the robot 100 automatically generates the behavioral content that the robot 100 should take in response to the behavior of the user 10, using a preset sentence generation model. The sentence generation model may be interpreted as an algorithm and calculation for automatic dialogue processing by text. The sentence generation model is described in, for example, JP 2018-081444 A and chatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), this is well known, and detailed description thereof will be omitted. Such a sentence generation model is configured by a large language model (LLM: Large Language Model). As described above, in this embodiment, by combining a large language model with an emotion engine, it is possible to reflect the emotions of the user 10 and the robot 100, and various linguistic information, in the behavior of the robot 100. In other words, according to this embodiment, a synergistic effect can be obtained by combining a sentence generation model with an emotion engine.

[0018] The robot 100 also has a function of recognizing the behavior (state) of the user 10. The robot 100 recognizes the behavior of the user 10 by analyzing a facial image of the user 10 acquired by a camera function and a voice of the user 10 acquired by a microphone function. The robot 100 determines the behavior to be performed by the robot 100 based on the recognized behavior of the user 10, etc.

[0019] The robot 100 stores rules that define actions to be taken by the robot 100 based on the emotions of the user 10, the emotions of the robot 100, and the actions of the user 10, and performs various actions according to the rules.

[0020] Specifically, the robot 100 has reaction rules for determining the behavior of the robot 100 based on the emotions of the user 10, the emotions of the robot 100, and the behavior of the user 10. For example, the reaction rules define the behavior of the robot 100 as "laughing" when the behavior of the user 10 is "laughing". The reaction rules also define the behavior of the robot 100 as "apologizing" when the behavior of the user 10 is "angry". The reaction rules also define the behavior of the robot 100 as "answering" when the behavior of the user 10 is "asking a question". The reaction rules also define the behavior of the robot 100 as "calling out" when the behavior of the user 10 is "sad".

[0021] When the robot 100 recognizes the behavior of the user 10 as "anger" based on the reaction rules, the robot 100 selects the behavior of "apologizing" defined in the reaction rules as the behavior to be executed by the robot 100. For example, when the robot 100 selects the behavior of "apologizing", the robot 100 performs the motion of "apologizing" and outputs a voice expressing the word "apologize".

[0022] In addition, it is defined that when the emotion of the robot 100 is "normal" (i.e., "happy" = 0, "anger" = 0, "sad" = 0, and "happy" = 0) and the condition that the state of the user 10 is "alone and looks lonely" is satisfied, the emotion of the robot 100 will change to "worried" and the action of "calling out" will be executed.

[0023] When the robot 100 recognizes based on the reaction rule that the current emotion of the robot 100 is "normal" and that the user 10 is alone and seems lonely, the robot 100 increases the emotion value of "sadness" of the robot 100. The robot 100 also selects the action of "calling out" defined in the reaction rule as the action to be performed on the user 10. For example, when the robot 100 selects the action of "calling out", the robot 100 converts the words "What's wrong?", which indicate concern, into a worried voice and outputs it. In this way, the behavior of the robot 100 is determined according to, for example, the emotions and actions of the user 10, the emotions of the robot 100, and the emotion values ​​thereof.

[0024] The robot 100 also transmits to the server 300 user reaction information indicating whether this action has elicited a positive or negative reaction from the user 10. The user reaction information includes, for example, the user action of "getting angry," the robot 100 action of "apologizing," the fact that the reaction of the user 10 was positive, and the attributes of the user 10.

[0025] The server 300 stores the user reaction information received from the robot 100. The server 300 receives and stores user reaction information not only from the robot 100, but also from each of the robots 101 and 102. The server 300 analyzes the user reaction information from the robots 100, 101, and 102, and updates the reaction rules.

[0026] The robot 100 receives the updated reaction rules from the server 300 by inquiring about the updated reaction rules from the server 300. The robot 100 incorporates the updated reaction rules into the reaction rules stored in the robot 100. This allows the robot 100 to incorporate the reaction rules acquired by the robot 101, the robot 102, etc. into its own reaction rules.

[0027] The robot 100 also has a function of recognizing the state (behavior and emotion) of the pet 13. The robot 100 recognizes the behavior, etc. of the pet 13 by analyzing the facial image of the pet 13 acquired by the camera function and the voice of the pet 13 acquired by the microphone function. The robot 100 determines the behavior to be performed by the robot 100 based on the recognized behavior, etc. of the pet 13.

[0028] The robot 100 according to this embodiment can provide advice information regarding the pet 13. For example, the robot 100 recognizes the state of the pet 13. The state of the pet 13 includes the behavior and emotions of the pet 13. The robot 100 provides advice information regarding the pet 13 according to the recognized state of the pet 13.

[0029] As an example, when the robot 100 recognizes a behavior (appearance) or emotion indicating that the pet 13 wants to play with the user 10, the robot 100 executes an action to start a conversation with the user 10. Specifically, the robot 100 makes an utterance indicating that advice information is about to be provided, such as "I have some advice about your pet." Note that "pet" in the utterance may be the name of the pet 13.

[0030] Next, the robot 100 generates advice information regarding the pet 13 based on the recognized state of the pet 13. The advice information includes, but is not limited to, information regarding the relationship between the pet 13 and the user 10 and information regarding the care of the pet 13. Here, the robot 100 provides advice information by speech in accordance with the state of the pet 13, such as "Your pet wants to play. If you play with your pet, you will become friends with it."

[0031] In this manner, in this embodiment, the robot 100 recognizes the state of the pet 13 and executes an action corresponding to the recognized state of the pet 13, thereby being able to provide the user 10 with appropriate advice regarding the pet 13. That is, the robot 100 according to this embodiment is able to execute an appropriate action for the user 10. Furthermore, the robot 100 is able to support the user 10 (pet owner) in building a better relationship with the pet 13 by providing advice based on a deeper understanding of the state (e.g., feelings and actions) of the pet 13.

[0032] Furthermore, the robot 100 according to this embodiment can provide advice information about the family. For example, the robot 100 recognizes the state of the user 10 who constitutes the family. The state of the user 10 includes the behavior and emotions of the user 10. The robot 100 provides advice information about the family according to the recognized state of the user 10.

[0033] As an example, when the robot 100 recognizes a behavior (appearance) or emotion indicating that the user 10a is lonely, it executes an action of starting a conversation with users 10b and 10c other than the user 10a among the users 10 who constitute the family. Specifically, the robot 100 makes an utterance indicating that advice information is about to be provided, such as "I have some advice about the user 10a." Note that "user 10a" in the utterance content may be the name of the user 10a.

[0034] Next, the robot 100 generates advice information about the user 10a based on the recognized state of the user 10a. The advice information includes, but is not limited to, information about communication between family members. Here, the robot 100 provides advice information suited to the state of the user 10a by speaking to the users 10b and 10c, such as "The user 10a feels lonely. You should talk to him / her and try to communicate with him / her."

[0035] In this manner, in this embodiment, the robot 100 recognizes the state of a user 10 (e.g., user 10a) who constitutes a family, and performs an action corresponding to the state of the recognized user 10 (user 10a), thereby being able to provide appropriate advice regarding the family to users (e.g., users 10b, 10c) other than the recognized user 10. That is, the robot 100 according to this embodiment can perform an appropriate action for the user 10. Furthermore, the robot 100 can facilitate communication between family members by appropriately conveying the state (e.g., opinions and feelings) of the family members and providing advice.

[0036] 2 is a diagram illustrating a schematic functional configuration of the robot 100. The robot 100 is configured by a control unit having a sensor unit 200, a sensor module unit 210, a storage unit (memory unit) 220, a state recognition unit 230, an emotion determination unit 232, a behavior recognition unit 234, a behavior determination unit 236, a memory control unit 238, a behavior control unit 250, a control target 252, and a communication processing unit 280.

[0037] The control target 252 includes a display device 2521, a speaker 2522, a lamp 2523 (e.g., an LED in the eye portion), and a motor 2524 for driving the arms, hands, legs, etc. The posture and gestures of the robot 100 are controlled by controlling the motors 2524 of the arms, hands, legs, etc. Some of the emotions of the robot 100 can be expressed by controlling these motors 2524. In addition, the facial expressions of the robot 100 can also be expressed by controlling the light emission state of the LED in the eye portion of the robot 100. The posture, gestures, and facial expressions of the robot 100 are examples of the attitude of the robot 100.

[0038] The sensor unit 200 includes a microphone 201, a 3D depth sensor 202, a 2D camera 203, and a distance sensor 204. The microphone 201 continuously detects sound and outputs sound data. The microphone 201 may be provided on the head of the robot 100 and may have a function of performing binaural recording. The 3D depth sensor 202 detects the contour of an object by continuously irradiating an infrared pattern and analyzing the infrared pattern from infrared images continuously captured by the infrared camera. The 2D camera 203 is an example of an image sensor. The 2D camera 203 captures an image using visible light and generates visible light video information. The distance sensor 204 detects the distance to an object by irradiating, for example, a laser or ultrasonic waves. The sensor unit 200 may also include a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, and the like.

[0039] 2, the components other than the control target 252 and the sensor unit 200 are examples of components included in a behavior control system of the robot 100. The behavior control system of the robot 100 controls the control target 252.

[0040] The storage unit 220 includes reaction rules 221, history data 222, and character data 223. The history data 222 includes the user 10's past emotion values ​​and behavior history. The emotion values ​​and behavior history are recorded for each user 10, for example, by being associated with the user 10's identification information. The history data 222 may also include user information for each of the multiple users 10 associated with the user 10's identification information. The user information includes information indicating the characteristics of the user 10, such as the user 10's personality, interests, interests, and inclinations. Such user information may be estimated from the user 10's behavior history, or may be registered by the user 10 himself. The history data 222 includes the pet 13's past emotion values ​​and behavior history. The emotion values ​​and behavior history may be recorded for each pet 13, for example, by being associated with the pet 13's identification information. At least a part of the storage unit 220 is implemented by a storage medium such as a memory. It may also include a person DB that stores a facial image of the user 10, attribute information of the user 10, etc., and a pet DB that stores a facial image of the pet 13, attribute information of the pet 13, etc. Among the components of the robot 100 shown in Fig. 2, the functions of the components other than the control target 252, the sensor unit 200, and the storage unit 220 can be realized by a CPU operating based on a program. For example, the functions of these components can be implemented as the operation of a CPU by an operating system (OS) and a program that operates on the OS.

[0041] The character data 223 is data that associates a character with an age. For example, the character is a person who appears in existing content such as animation, video games, comics, movies, etc. Also, the character may be an animal or plant with a personality, or an inanimate object (such as a robot).

[0042] For example, the age (user age) associated with a character in the character data 223 is determined based on the age group of viewers who are assumed to be the target audience of the content in which the character appears.

[0043] Fig. 3 is a diagram showing a schematic data structure of the character data 223. For example, character "A" appears in an animation for kindergarteners. In this case, as shown in Fig. 3, character "A" is associated with a user age of "3 to 7 years old."

[0044] For example, if a movie in which character "C" appears contains violent scenes and is not suitable for viewing by young children, the user age of "12 years and older" is associated with character "C" as shown in FIG.

[0045] The age in the character data 223 may be determined based on an age rating by a rating organization such as PEGI (Pan European Game Information), the Motion Picture Ethics Organization, or CERO (Computer Entertainment Rating Organization). The age of use may be determined by a range such as "3 to 5 years old" or "12 years old or older," or may be determined by a single value such as "10 years old" or "15 years old."

[0046] The sensor module unit 210 includes a voice emotion recognition unit 211, a speech understanding unit 212, a facial expression recognition unit 213, and a face recognition unit 214. Information detected by the sensor unit 200 is input to the sensor module unit 210. The sensor module unit 210 analyzes the information detected by the sensor unit 200, and outputs the analysis result to the state recognition unit 230.

[0047] The voice emotion recognition unit 211 of the sensor module unit 210 analyzes the voice of the user 10 detected by the microphone 201 to recognize the emotion of the user 10. For example, the voice emotion recognition unit 211 extracts feature quantities such as frequency components of the voice, and recognizes the emotion of the user 10 based on the extracted feature quantities. The voice emotion recognition unit 211 also analyzes the voice (cry) of the pet 13 detected by the microphone 201 to recognize the emotion of the pet 13. For example, the voice emotion recognition unit 211 extracts feature quantities such as frequency components of the voice, and recognizes the emotion of the pet 13 based on the extracted feature quantities. The speech understanding unit 212 analyzes the voice of the user 10 detected by the microphone 201, and outputs text information representing the content of the user 10's speech.

[0048] The facial expression recognition unit 213 recognizes the facial expression and emotion of the user 10 from an image of the user 10 captured by the 2D camera 203. For example, the facial expression recognition unit 213 recognizes the facial expression and emotion of the user 10 based on the shape, positional relationship, etc. of the eyes and mouth. The facial expression recognition unit 213 also recognizes the facial expression and emotion of the pet 13 from an image of the pet 13 captured by the 2D camera 203. For example, the facial expression recognition unit 213 recognizes the facial expression and emotion of the pet 13 based on the shape, positional relationship, etc. of the eyes and mouth.

[0049] The face recognition unit 214 recognizes the face of the user 10. The face recognition unit 214 recognizes the user 10 by matching a face image stored in a person DB (not shown) with a face image of the user 10 captured by the 2D camera 203. The face recognition unit 214 also recognizes the face of the pet 13. The face recognition unit 214 recognizes the pet 13 by matching a face image stored in a pet DB (not shown) with a face image of the pet 13 captured by the 2D camera 203.

[0050] The state recognition unit 230 recognizes the state of the user 10 based on the information analyzed by the sensor module unit 210. For example, the state recognition unit 230 mainly performs processing related to perception using the analysis result of the sensor module unit 210. For example, the state recognition unit 230 generates perceptual information such as "Daddy is alone" and "There is a 90% chance that Daddy is not smiling." The state recognition unit 230 performs processing to understand the meaning of the generated perceptual information. For example, the state recognition unit 230 generates semantic information such as "Daddy is alone and looks lonely." In addition, the state recognition unit 230 recognizes the state of the pet 13 based on the information analyzed by the sensor module unit 210. For example, the state recognition unit 230 generates perceptual information such as "The pet is wagging its tail" and "There is an 80% chance that the pet is not angry." The state recognition unit 230 performs processing to understand the meaning of the generated perceptual information. For example, the state recognition unit 230 generates semantic information such as "The pet wants to play."

[0051] The emotion determination unit 232 determines an emotion value indicating the emotion of the user 10 based on the information analyzed by the sensor module unit 210 and the state of the user 10 recognized by the state recognition unit 230. For example, the information analyzed by the sensor module unit 210 and the recognized state of the user 10 are input to a pre-trained neural network to obtain an emotion value indicating the emotion of the user 10. The emotion determination unit 232 also determines an emotion value indicating the emotion of the pet 13 based on the information analyzed by the sensor module unit 210 and the state of the pet 13 recognized by the state recognition unit 230. For example, the information analyzed by the sensor module unit 210 and the recognized state of the pet 13 are input to a pre-trained neural network to obtain an emotion value indicating the emotion of the pet 13.

[0052] Here, the emotion value indicating the emotion of the user 10 or the pet 13 (hereinafter sometimes referred to as "user, etc.") is a value indicating the positive or negative emotion of the user, etc. For example, if the emotion of the user, etc. is a cheerful emotion accompanied by a pleasant feeling or comfort, such as "joy," "pleasure," "comfort," "relief," "excitement," "relief," and "fulfillment," the emotion value indicates a positive value, and the brighter the emotion, the larger the value. If the emotion of the user, etc. is an unpleasant emotion, such as "anger," "sorrow," "discomfort," "anxiety," "sorrow," "worry," and "emptiness," the emotion value indicates a negative value, and the more unpleasant the emotion, the larger the absolute value of the negative value. If the emotion of the user, etc. is not any of the above ("normal"), the value of 0 is indicated. Note that the type of emotion of the user 10 and the type of emotion of the pet 13 may be different or the same. In addition, the emotion value of the emotion of the user 10 may be different or the same as the emotion value of the emotion of the pet 13.

[0053] Furthermore, the emotion determining unit 232 determines an emotion value indicating the emotion of the robot 100 based on the information analyzed by the sensor module unit 210 and the state of the user or the like recognized by the state recognizing unit 230 .

[0054] The emotion value of the robot 100 includes emotion values ​​for each of a plurality of emotion categories, and is, for example, a value (0 to 5) indicating the strength of each of "happiness," "anger," "sorrow," and "pleasure."

[0055] Specifically, the emotion determination unit 232 determines an emotion value indicating the emotion of the robot 100 in accordance with a rule for updating the emotion value of the robot 100, which rule is determined in association with the information analyzed by the sensor module unit 210 and the state of the user, etc., recognized by the state recognition unit 230.

[0056] For example, when the state recognition unit 230 recognizes that the user, etc., looks lonely, the emotion determination unit 232 increases the emotion value of "sadness" of the robot 100. Also, when the state recognition unit 230 recognizes that the user, etc., is smiling, the emotion determination unit 232 increases the emotion value of "joy" of the robot 100.

[0057] The emotion determining unit 232 may determine the emotion value indicating the emotion of the robot 100 by further considering the state of the robot 100. For example, when the battery level of the robot 100 is low or when the surrounding environment of the robot 100 is pitch black, the emotion value of "sadness" of the robot 100 may be increased. Furthermore, when the user 10 continues to talk to the robot 100 despite the battery level being low, the emotion value of "anger" may be increased.

[0058] The behavior recognition unit 234 recognizes the behavior of the user, etc., based on the information analyzed by the sensor module unit 210 and the state of the user, etc., recognized by the state recognition unit 230. For example, the information analyzed by the sensor module unit 210 and the recognized state of the user, etc., are input to a pre-trained neural network, the probability of each of a plurality of predetermined behavior classifications (for example, "laughing," "getting angry," "asking a question," "getting sad") is obtained, and the behavior classification with the highest probability is recognized as the behavior of the user, etc.

[0059] As described above, in this embodiment, the robot 100 identifies the user 10 and then acquires the contents of the user's utterance. When acquiring and using the contents of the utterance, the robot 100 obtains the necessary consent in accordance with laws and regulations from the user 10. The behavior control system of the robot 100 in this embodiment takes into consideration the protection of the personal information and privacy of the user 10.

[0060] The behavior determining unit 236 determines a behavior corresponding to the behavior of the user, etc., recognized by the behavior recognizing unit 234, based on the current emotion value of the user, etc., determined by the emotion determining unit 232, the history data 222 of past emotion values ​​determined by the emotion determining unit 232 before the current emotion value of the user, etc., was determined, and the emotion value of the robot 100. In this embodiment, the behavior determining unit 236 uses one most recent emotion value included in the history data 222 as the past emotion value of the user, etc., but the disclosed technology is not limited to this aspect. For example, the behavior determining unit 236 may use a plurality of most recent emotion values ​​as the past emotion value of the user, etc., or may use an emotion value from a unit period ago, such as one day ago. In addition, the behavior determining unit 236 may determine a behavior corresponding to the behavior of the user, etc., taking into consideration not only the current emotion value of the robot 100 but also the history of the past emotion values ​​of the robot 100. The behavior determined by the behavior determining unit 236 includes gestures performed by the robot 100 or the contents of the speech of the robot 100.

[0061] The behavior decision unit 236 according to this embodiment decides the behavior of the robot 100 as a behavior corresponding to the behavior of the user, etc., based on a combination of the past emotion value and the current emotion value of the user, etc., the emotion value of the robot 100, the behavior of the user, etc., and the reaction rules 221. For example, when the past emotion value of the user, etc. is a positive value and the current emotion value is a negative value, the behavior decision unit 236 decides on an action for changing the emotion value of the user, etc. to a positive value as a behavior corresponding to the behavior of the user, etc.

[0062] The behavior determining unit 236 may determine a behavior corresponding to the behavior of the user 10 based on the emotion of the robot 100. For example, when the robot 100 is insulted by the user 10 or is arrogantly treated by the user 10 (i.e., when the user's reaction is poor), when the surrounding noise is loud and the voice of the user 10 cannot be detected, when the battery level of the robot 100 is low, etc., and the emotion value of "anger" or "sorrow" of the robot 100 increases, the behavior determining unit 236 may determine a behavior corresponding to the behavior of the user 10 according to the increase in the emotion value of "anger" or "sorrow". Also, when the user's reaction is good or the battery level of the robot 100 is high, etc., and the emotion value of "joy" or "pleasure" of the robot 100 increases, the behavior determining unit 236 may determine a behavior corresponding to the behavior of the user 10 according to the increase in the emotion value of "joy" or "pleasure". Furthermore, the behavior determining unit 236 may determine a behavior different from the behavior of the robot 100 toward the user 10 who has increased the emotional value of "anger" or "sorrow" of the robot 100 as the behavior of the robot 100 toward the user 10 who has increased the emotional value of "joy" or "pleasure" of the robot 100. In this manner, the behavior determining unit 236 may determine a different behavior depending on the emotion of the robot 100 itself or how the user 10 has changed the emotion of the robot 100 due to the behavior of the user 10.

[0063] The reaction rules 221 prescribe the behavior of the robot 100 according to a combination of the past emotion value and the current emotion value of the user 10, the emotion value of the robot 100, and the behavior of the user 10. For example, when the past emotion value of the user 10 is a positive value and the current emotion value is a negative value, and the behavior of the user 10 is sad, a combination of gestures and speech content when asking a question to encourage the user 10 with gestures is prescribe as the behavior of the robot 100.

[0064] For example, the reaction rule 221 defines behaviors of the robot 100 for all combinations of patterns of the emotion values ​​of the robot 100 (1296 patterns, which are the fourth power of six values ​​of "joy", "anger", "sorrow", and "pleasure", from "0" to "5"); combination patterns of the past emotion values ​​and the current emotion values ​​of the user 10; and behavior patterns of the user 10. That is, for each pattern of the emotion values ​​of the robot 100, behaviors of the robot 100 are defined according to the behavior patterns of the user 10 for each of a plurality of combinations of the past emotion values ​​and the current emotion values ​​of the user 10, such as negative values ​​and negative values, negative values ​​and positive values, positive values ​​and negative values, positive values ​​and positive values, negative values ​​and normal values, and normal values ​​and normal values. Note that the behavior determination unit 236 may transition to an operation mode in which the behavior of the robot 100 is determined using the history data 222, when the user 10 makes an utterance intending to continue a conversation from a past topic, such as "I want to talk about that topic we talked about last time."

[0065] The reaction rules 221 may prescribe at least one of a gesture and a statement as the behavior of the robot 100 for each of the patterns (1296 patterns) of the emotion value of the robot 100. Alternatively, the reaction rules 221 may prescribe at least one of a gesture and a statement as the behavior of the robot 100 for each group of patterns of the emotion value of the robot 100.

[0066] The strength of each gesture included in the behavior of the robot 100 defined in the reaction rules 221 is predefined. The strength of each speech included in the behavior of the robot 100 defined in the reaction rules 221 is predefined. The reaction rules 221 may include rules for determining the behavior of the robot 100 based on the emotions of not only the user 10 but also the pet 13, the emotions of the robot 100, and the behavior of the pet 13. For example, the reaction rules 221 may determine the behavior of the robot 100 according to a combination of the past emotion value and the current emotion value of the pet 13, the emotion value of the robot 100, and the behavior of the pet 13.

[0067] For example, the reaction rules 221 prescribe behaviors of the robot 100 corresponding to behavioral patterns when the state of the pet 13 requires advice to the user 10, when the user 10 reacts to the provided advice information, etc. As an example, when the behavior decision unit 236 recognizes the behavior or emotion of the pet 13 as the state of the pet 13 based on the reaction rules 221, it decides that its own behavior is to provide the user 10 with advice information about the pet 13 according to the state of the pet 13.

[0068] In addition, for example, the reaction rule 221 defines the behavior of the robot 100 corresponding to a behavior pattern when the state of the user 10a constituting the family requires advice to users 10b and 10c other than the user 10a, when there is a reaction from the user 10 to the provided advice information, etc. As an example, when the behavior determining unit 236 recognizes the behavior and emotion of the user 10a as the state of the user 10a based on the reaction rule 221, it determines, as its own behavior, an action to provide advice information about the family corresponding to the state of the user 10a to the users 10b and 10c.

[0069] The memory control unit 238 determines whether or not to store data including the behavior of the user or the like in the history data 222, based on the predetermined behavior strength for the behavior determined by the behavior determination unit 236 and the emotion value of the robot 100 determined by the emotion determination unit 232.

[0070] Specifically, when a total intensity value, which is the sum of the sum of the emotion values ​​for each of the multiple emotion classifications of the robot 100, the predetermined intensity for the gesture included in the behavior determined by the behavior determination unit 236, and the predetermined intensity for the speech content included in the behavior determined by the behavior determination unit 236, is equal to or greater than a threshold value, it is determined that data including the behavior of the user, etc. is to be stored in the history data 222.

[0071] When the memory control unit 238 decides to store data including the behavior of the user or the like in the history data 222, it stores in the history data 222 the behavior determined by the behavior determination unit 236, information analyzed by the sensor module unit 210 from the present time up to a certain period of time ago (e.g., all peripheral information such as data on the sound, image, smell, etc. of the scene), and the state of the user or the like recognized by the state recognition unit 230 (e.g., the facial expression, emotions of the user 10, the facial expression, emotions of the pet 13, etc.).

[0072] The behavior control unit 250 controls the control target 252 based on the behavior determined by the behavior determination unit 236. For example, when the behavior determination unit 236 determines an behavior including speaking, the behavior control unit 250 outputs a sound from a speaker included in the control target 252. At this time, the behavior control unit 250 may determine the speaking speed of the sound based on the emotion value of the robot 100. For example, the behavior control unit 250 determines a faster speaking speed as the emotion value of the robot 100 increases. In this way, the behavior control unit 250 determines the execution form of the behavior determined by the behavior determination unit 236 based on the emotion value determined by the emotion determination unit 232.

[0073] As described above, in the robot 100 according to this embodiment, the state recognition unit 230 and the behavior recognition unit 234 recognize the states (behavior and emotion) of the user 10 and the pet 13 that constitute the family. The behavior determination unit 236 determines its own behavior corresponding to the recognized state of the user 10 and the pet 13. The behavior control unit 250 controls the control target 252 based on its own determined behavior.

[0074] Specifically, when the behavior control unit 250 recognizes the behavior of the pet 13 as the state of the pet 13, it determines its own behavior to be a behavior that provides advice information about the pet 13 in accordance with the behavior of the pet 13, and controls the control target 252.

[0075] The advice information includes information on the relationship between the pet 13 and the user 10 who keeps the pet 13 (hereinafter, may be referred to as "relationship information"), and information on the care of the pet 13 (hereinafter, may be referred to as "care information"). The relationship information includes information for improving the relationship between the pet 13 and the user 10, but these are examples and are not limited to the above. The care information includes information on how to deal with emotional concerns regarding the care of the pet 13, information on the daily care of the pet 13, etc., but these are examples and are not limited to the above. The advice information may include both the relationship information and the care information, or may include one of them. That is, the advice information includes at least one of the relationship information and the care information. The advice information is not limited to the relationship information and the care information, and may include other information such as, for example, characteristic information of the pet 13.

[0076] For example, when the behavior control unit 250 recognizes a behavior (state) of the pet 13 indicating that it wants to play with the user 10, such as wagging its tail or making a whining noise, it executes a behavior of starting a conversation with the user 10. Specifically, the robot 100 makes an utterance indicating that advice information is about to be provided, such as "I have some advice about your pet."

[0077] Next, the behavior control unit 250 generates advice information regarding the pet 13 based on the recognized behavior of the pet 13, and provides the generated advice information by speech. For example, the behavior control unit 250 provides advice information (here, relationship information) in accordance with the state of the pet 13 by speech, such as "Your pet wants to play. If you play with your pet, you will become friends with it."

[0078] In the above, the behavior control unit 250 recognizes the behavior of the pet 13 as the state of the pet 13, but the behavior control unit 250 may recognize the emotion of the pet 13. For example, when the behavior control unit 250 recognizes that the emotion of the pet 13 is "unpleasant" or "angry" such as the pet 13 barking or baring its teeth, the behavior control unit 250 generates advice information about the pet 13 based on the recognized emotion of the pet 13 and provides the generated advice information by speaking. For example, the behavior control unit 250 provides advice information (here, care information) according to the state of the pet 13 by speaking, such as "Your pet seems to be unhappy. You will be happy if you feed it" or "Your pet seems to be unhappy. You will be happy if you take it for a walk." In addition, since the emotion value of the robot 100 is determined by the emotion determination unit 232, the behavior control unit 250 may provide advice information by speaking that is in accordance with the feelings (concerns) of the user 10, such as "Walking is hard, but I recommend walking to maintain your health." based on the emotion value, etc.

[0079] In this way, the behavior control unit 250 according to this embodiment can provide the user 10 with appropriate advice regarding the pet 13. Furthermore, the behavior control unit 250 can provide advice based on a deeper understanding of the state (e.g., feelings and behavior) of the pet 13, thereby supporting the user 10 (pet owner) in building a better relationship with the pet 13.

[0080] Furthermore, when the behavior control unit 250 recognizes the state of a user 10 who is a member of a family, it determines, as its own behavior, an action to provide advice information about the family according to the recognized state of the user 10, and controls the control target 252.

[0081] Specifically, when the behavior control unit 250 recognizes the behavior of a user 10 who is a member of a family as the state of the user 10, it determines that its own behavior is to provide advice information about the family in accordance with the behavior of the user 10, and controls the control object 252.

[0082] For example, when the behavior control unit 250 receives a voice such as "I feel lonely" or "I feel bored" from the user 10a and recognizes a behavior (state) indicating that the user 10a is lonely, the behavior control unit 250 executes a behavior of starting a conversation with users 10b and 10c other than the user 10a among the multiple users 10 constituting the family. Specifically, the robot 100 makes an utterance indicating that advice information will be provided, such as "I have some advice for the user 10a."

[0083] Next, the behavior control unit 250 generates advice information about the family based on the recognized behavior of the user 10a, and provides the generated advice information by speech. For example, the behavior control unit 250 provides advice information suited to the state of the user 10a, such as "The user 10a feels lonely. You should talk to him / her and try to communicate with him / her" or "The user 10a feels lonely. How about going out together?" to the users 10b and 10c by speech.

[0084] Here, the behavior control unit 250 may generate advice information based on user information indicating the characteristics of the user 10. For example, the behavior control unit 250 generates advice information based on at least one of the personality, interest, hobbies, and inclinations of each of the multiple users 10 contained in the user information. As an example, if the interest or preoccupation of the user 10a is "shopping," the behavior control unit 250 provides advice information in accordance with the interest or preoccupation of the user 10a by speaking, such as "The user 10a feels lonely. How about going shopping together?"

[0085] The user information is not limited to the personality, interests, concerns, and inclinations of the user 10 described above, but may also include other characteristics such as hobbies and preferences.

[0086] In the above, the behavior control unit 250 recognizes the behavior of the user 10 as the state of the user 10, but the behavior control unit 250 may recognize the feelings (opinions) of the user 10. For example, when the behavior control unit 250 receives a voice such as "The user 10b gets angry easily" or "The user 10b is noisy" from the user 10a and recognizes that the feelings of the user 10a toward the user 10b are "unpleasant" or "anxious", the behavior control unit 250 generates advice information about the family based on the recognized feelings of the user 10 and provides the generated advice information to the user 10b by speaking. For example, the behavior control unit 250 provides advice information in accordance with the state of the user 10a by speaking to the user 10b, such as "It seems that what you said makes the user 10a unhappy. Why don't you change the way you say it?". In addition, since the emotion value of the robot 100 is determined by the emotion determination unit 232, the behavior control unit 250 may provide, based on the emotion value, advice information that is sympathetic to the feelings (concerns) of the user 10b, such as, "I understand that you are angry, but if you change the way you say it, your feelings will be conveyed more effectively."

[0087] In this way, the behavior control unit 250 according to this embodiment can provide appropriate advice regarding family members to the user 10. Furthermore, the behavior control unit 250 can facilitate communication between family members by appropriately conveying the state of family members (e.g., opinions and feelings) and providing advice, in other words, can support deepening mutual understanding between family members.

[0088] In addition, the provision of advice information to the user 10 is not limited to being provided by speech using the speaker 2522, which is the control object 252, but may also be provided through the display of the display device 2521, which is the control object 252, the posture of the robot 100 using control of the motors 2524 that drive the arms, hands, legs, etc.

[0089] The behavior control unit 250 may recognize a change in the user 10's emotion in response to the execution of the behavior determined by the behavior determination unit 236. For example, the change in emotion may be recognized based on the voice or facial expression of the user 10. Alternatively, the change in emotion may be recognized based on the detection of an impact by a touch sensor included in the sensor unit 200. When an impact is detected by the touch sensor included in the sensor unit 200, the emotion of the user 10 may be recognized as having worsened, or when the reaction of the user 10 is determined to be laughing or happy based on the detection result of the touch sensor included in the sensor unit 200, the emotion of the user 10 may be recognized as having improved. Information indicating the reaction of the user 10 is output to the communication processing unit 280.

[0090] In addition, after the behavior control unit 250 executes the behavior determined by the behavior determination unit 236 in the execution form determined according to the emotion of the robot 100, the emotion determination unit 232 further changes the emotion value of the robot 100 based on the user's reaction to the execution of the behavior. Specifically, the emotion determination unit 232 increases the emotion value of "happiness" of the robot 100 when the user's reaction to the behavior determined by the behavior determination unit 236 being performed on the user in the execution form determined by the behavior control unit 250 is not bad. In addition, the emotion determination unit 232 increases the emotion value of "sadness" of the robot 100 when the user's reaction to the behavior determined by the behavior determination unit 236 being performed on the user in the execution form determined by the behavior control unit 250 is bad.

[0091] Furthermore, the behavior control unit 250 expresses the emotion of the robot 100 based on the determined emotion value of the robot 100. For example, when the behavior control unit 250 increases the emotion value of "happiness" of the robot 100, the behavior control unit 250 controls the control object 252 to make the robot 100 perform a happy gesture. When the behavior control unit 250 increases the emotion value of "sadness" of the robot 100, the behavior control unit 250 controls the control object 252 to make the robot 100 assume a droopy posture.

[0092] Furthermore, the behavior control unit 250 changes the content of the advice information according to the emotion of the robot 100. For example, the behavior control unit 250 changes the content of the advice information to be generated according to the emotion value of the robot 100. Specifically, when the behavior control unit 250 receives a voice indicating an arrogant attitude from the user, the emotion value of "anger" or "sorrow" of the robot 100 increases, and the behavior control unit 250 generates and provides advice information according to the increased emotion value. The content of such advice information is different from that of advice information provided to a user who does not receive a voice indicating abusive language or an arrogant attitude (i.e., a user who makes good remarks and has a good attitude). As an example, the advice information provided to a user who receives a voice indicating abusive language, etc., is less in the content of the advice proposed, or a part of the advice content is omitted, compared to the advice information provided to a user who does not receive a voice indicating abusive language, etc.

[0093] Furthermore, when the user's voice cannot be detected due to loud surrounding noise, when the robot 100's emotional value of "anger" or "sadness" increases due to low battery power, etc., the behavior control unit 250 generates and provides advice information according to the increased emotional value. The advice information differs in content from advice information provided to the user when the user's voice can be detected or when the battery power is high. As an example, advice information provided to the user when the user's voice cannot be detected, etc., provides less advice or omits part of the advice content compared to advice information provided to the user when the user's voice can be detected, etc.

[0094] Furthermore, when the behavior control unit 250 receives a voice from the user indicating a favorable attitude toward the robot 100, such as "Thank you for your advice," the robot 100's emotional value of "happiness" or "pleasure" increases, and generates and provides advice information according to the increased emotional value. The advice information is different in content from advice information provided to a user who does not receive a voice indicating a favorable attitude (i.e., a user with a normal attitude). As an example, advice information provided to a user who receives a voice indicating a favorable attitude has more extensive advice suggestions or additional advice contents compared to advice information provided to a user who does not receive a voice indicating a favorable attitude.

[0095] The behavior control unit 250 may change the frequency of providing advice information to the user based on the emotion value of the robot 100. For example, when the user's reaction to the provided advice information is not bad (specifically, the user responds with a positive response such as "like") and the emotion value of "happiness" of the robot 100 increases, the behavior control unit 250 may increase the frequency of providing advice information to the user. Also, when the user's reaction to the provided advice information is bad (specifically, the user responds with a negative response such as "that's wrong") and the emotion value of "sadness" of the robot 100 increases, the behavior control unit 250 may decrease the frequency of providing advice information to the user.

[0096] The communication processing unit 280 is responsible for communication with the server 300. As described above, the communication processing unit 280 transmits user reaction information to the server 300. In addition, the communication processing unit 280 receives updated reaction rules from the server 300. When the communication processing unit 280 receives the updated reaction rules from the server 300, it updates the reaction rules 221.

[0097] The server 300 communicates between the robot 100, the robot 101, and the robot 102 and the server 300, receives user reaction information transmitted from the robot 100, and updates the reaction rules based on reaction rules that include actions that have received positive reactions.

[0098] (Character-based action decisions) So far, a case has been described in which the behavior determining unit 236 determines the behavior of the robot 100 based on the state recognized by the state recognizing unit 230. On the other hand, the behavior determining unit 236 may determine the behavior of the robot 100 based on not only the state of the user but also the set character. In this case, the behavior determining unit 236 may obtain an age (user age) associated with the character from the character data 223, and determine the behavior of the robot 100 based on the obtained user age.

[0099] That is, the behavior determining unit 236 determines the behavior of the robot 100 based on the state recognized by the state recognizing unit 230 and the set character or the age associated with the character. This makes it possible to cause the robot 100 to perform an appropriate behavior according to the age of the user. In particular, it becomes possible to restrict the robot 100 from performing actions that are inappropriate for young users (for example, outputting violent content).

[0100] Characters are set in advance in the system 5. The character settings are input as a prompt (command sentence). The prompt may be input via an input device provided in the robot 100, or via an external device such as a server communicably connected to the robot 100. In addition, the prompt may specify the name of the character, or may specify an ID defined for each character.

[0101] For example, the behavior determining unit 236 determines a behavior to output a screen showing the appearance of the character or a color corresponding to the character on a display device 2521 (an example of an output device) provided on the robot. The color corresponding to the character is a theme color or the like that is associated with the character. This allows the user to get a sense of having a conversation with the character.

[0102] Also, for example, the behavior determining unit 236 determines the behavior of outputting information to the display device 2521 or the speaker 2522 (examples of an output device) provided in the robot 100 in a manner according to the age of the user. For example, the behavior determining unit 236 changes the voice of the robot 100 emitted from the speaker 2522 to the character's voice tone.

[0103] Also, for example, the behavior determining unit 236 determines the behavior of outputting a voice or a message by using text composed of words according to the user's age. Here, it is assumed that usable words for each age are set in advance. The behavior determining unit 236 obtains the user's age from the character data 223.

[0104] For example, it is assumed that the words "What's wrong?" and "How is it going?" are stored in advance in the storage unit 220 as words to be output when the robot 100 selects the action of "calling out." In addition, it is assumed that the age "under 12 years old" is associated with "What's wrong?" and the age "12 years old or older" is associated with "How is it going?". For example, if the user age is "18 years old or older," the behavior decision unit 236 decides to output the words "How is it going?". In addition, for example, if the user age is "3 to 7 years old," the behavior decision unit 236 decides to output the words "What's wrong?".

[0105] In this way, by changing the tone of voice and the words output depending on the age of the user, it is possible to increase familiarity with younger users while restricting behavior that is particularly inappropriate for younger users.

[0106] Furthermore, the behavior determining unit 236 determines a behavior for outputting content corresponding to the character to an output device (such as the display device 2521) provided in the robot 100. For example, the behavior determining unit 236 determines a behavior for displaying video content (such as a movie or animation) in which the character appears on the display device 2521.

[0107] The behavior determining unit 236 may also determine an action to output educational content according to the age of use. Here, the educational content is text, video, audio, etc. related to study subjects such as English, arithmetic, Japanese, science, and social studies. The educational content may also be interactive content in which the user inputs answers to questions. For example, the behavior determining unit 236 determines an action to display on the display device 2521 a text of a calculation problem corresponding to the grade according to the age of use. For example, if the age of use is "under 8 years old", the behavior determining unit 236 determines to display an addition problem, and if the age of use is "8 years old or older", the behavior determining unit 236 determines to display a multiplication problem.

[0108] The behavior determining unit 236 may determine a behavior to output content according to the age of use, rather than the character, to an output device provided in the robot 100. In this case, the content may be content in which a character appears, or content that does not depend on a character, such as a generally known folk tale or fairy tale.

[0109] In addition, content corresponding to the character and the grade and educational content corresponding to the age of the user may be pre-stored in the storage unit 220, or may be obtained from an external device such as a server connected to the robot 100 so as to be able to communicate with the robot 100.

[0110] 4 is a diagram showing an example of an operation flow for character setting. Note that "S" in the operation flow indicates a step to be executed.

[0111] In step S50, the robot 100 accepts the setting of the character. Then, in step S51, the robot 100 outputs a screen corresponding to the character (for example, a screen displaying the character's appearance).

[0112] Furthermore, in step S52, the behavior determining unit 236 acquires from the character data 223 a usage age corresponding to the set character.

[0113] Fig. 5 is a diagram showing an example of an operation flow relating to an operation for determining an action in the robot 100. The operation flow shown in Fig. 5 is repeatedly executed. At this time, it is assumed that information analyzed by the sensor module unit 210 is input. Note that "S" in the operation flow indicates a step to be executed.

[0114] First, in step S100, the state recognition unit 230 recognizes the state of the user, etc., based on the information analyzed by the sensor module unit 210. For example, the state recognition unit 230 generates perceptual information such as "Daddy is alone," and "There is a 90% chance that Daddy is not smiling," and performs processing to understand the meaning of the generated perceptual information. For example, the state recognition unit 230 generates semantic information such as "Daddy is alone and looks lonely." In addition, the state recognition unit 230 generates perceptual information such as "Your pet is wagging its tail," and "There is an 80% chance that your pet is not angry," and generates semantic information such as "Your pet wants to play."

[0115] In step S102, the emotion determination unit 232 determines an emotion value indicating the emotion of the user, etc., based on the information analyzed by the sensor module unit 210 and the state of the user, etc., recognized by the state recognition unit 230.

[0116] In step S103, the emotion determination unit 232 determines an emotion value indicating the emotion of the robot 100 based on the information analyzed by the sensor module unit 210 and the state of the user, etc. recognized by the state recognition unit 230. The emotion determination unit 232 adds the determined emotion value of the user, etc. to the history data 222.

[0117] In step S104 , the behavior recognition unit 234 recognizes a behavior classification of the user, etc., based on the information analyzed by the sensor module unit 210 and the state of the user, etc., recognized by the state recognition unit 230 .

[0118] In step S106, the behavior determination unit 236 determines the behavior of the robot 100 based on the usage age acquired in step S52 of FIG. 4, a combination of the current emotion value of the user 10 determined in step S102 of FIG. 5 and the past emotion value included in the history data 222, the emotion value of the robot 100, the behavior of the user 10 recognized by the behavior recognition unit 234, and the reaction rules 221.

[0119] In step S108, the behavior control unit 250 controls the control target 252 based on the behavior determined by the behavior determination unit 236. For example, the behavior control unit 250 controls the control target 252 based on the behavior determined according to at least one of the state of the user 10 constituting the family and the state of the pet 13.

[0120] In step S110, the memory control unit 238 calculates a total intensity value based on the predetermined action intensities for the actions determined by the action determination unit 236 and the emotion value of the robot 100 determined by the emotion determination unit 232.

[0121] In step S112, the storage control unit 238 determines whether the total intensity value is equal to or greater than the threshold. If the total intensity value is less than the threshold, the process ends without storing data including the behavior of the user 10 in the history data 222. On the other hand, if the total intensity value is equal to or greater than the threshold, the process proceeds to step S114.

[0122] In step S114, the behavior determined by the behavior determination unit 236, the information analyzed by the sensor module unit 210 from the present time up to a certain period of time ago, and the state of the user 10 recognized by the state recognition unit 230 are stored in the history data 222.

[0123] As described above, the robot (electronic device) 100 is provided with a control unit that recognizes at least one of the state of the user 10 who constitutes the family and the state of the pet 13 kept by the user 10, determines its own behavior corresponding to the recognized state, and controls the control target based on its own determined behavior. This allows the robot 100 to perform appropriate behavior, such as giving appropriate advice to the user 10 about the family and the pet 13.

[0124] Furthermore, when the control unit of the robot 100 recognizes the behavior of the pet 13 as the state of the pet 13, the control unit determines, as its own behavior, a behavior that provides advice information about the pet 13 in accordance with the behavior of the pet 13. This allows the robot 100 to provide appropriate advice information about the pet 13 in accordance with the behavior of the pet 13.

[0125] Furthermore, when the control unit of the robot 100 recognizes the emotion of the pet 13 as the state of the pet 13, the control unit determines, as its own behavior, an action to provide advice information about the pet 13 according to the emotion of the pet 13. This allows the robot 100 to provide appropriate advice information about the pet 13 according to the emotion of the pet 13.

[0126] Moreover, the advice information regarding the pet 13 includes at least one of information regarding the relationship between the pet 13 and the user 10 who keeps the pet 13, and information regarding the care of the pet 13. This enables the robot 100 to provide, as advice information, for example, information for improving the relationship between the pet 13 and the user 10, and a method for dealing with emotional concerns regarding the care of the pet 13.

[0127] Furthermore, when the control unit of the robot 100 recognizes the behavior of a user 10 who is a member of a family as the state of the user 10, the control unit of the robot 100 determines, as its own behavior, a behavior that provides advice information about the family in accordance with the behavior of the user 10. This enables the robot 100 to provide appropriate advice information about the family in accordance with the behavior of the user 10 who is a member of the family.

[0128] Furthermore, when the control unit of the robot 100 recognizes the emotion of a user 10 who is a member of a family as the state of the user 10, the control unit of the robot 100 determines, as its own behavior, an action to provide advice information about the family according to the emotion of the user 10. This enables the robot 100 to provide appropriate advice information about the family that is in line with the emotion of the user 10 who is a member of the family.

[0129] In addition, the control unit of the robot 100 generates advice information based on at least one of the personality, interest, hobbies, and inclinations of each of the multiple users 10 who make up a family. This allows the robot 100 to provide appropriate advice information about the family that is in line with the personality, etc. of each user 10.

[0130] Although the robot 100 has been given above as an example of an electronic device, the electronic device is not limited to this and may be other types of electronic devices such as a robot having an appearance of a stuffed toy, a smartphone, or a smart speaker.

[0131] In the above embodiment, the robot 100 recognizes the user 10 using a facial image of the user 10, but the disclosed technology is not limited to this aspect. For example, the robot 100 may recognize the user 10 using a voice emitted by the user 10, an email address of the user 10, an SNS ID of the user 10, an ID card with a built-in wireless IC tag that the user 10 possesses, or the like.

[0132] The robot 100 is an example of an electronic device equipped with a behavior control system. The application of the behavior control system is not limited to the robot 100, and the behavior control system can be applied to various electronic devices. The functions of the server 300 may be implemented by one or more computers. At least a part of the functions of the server 300 may be implemented by a virtual machine. At least a part of the functions of the server 300 may be implemented in the cloud.

[0133] 6 is a diagram illustrating an example of a hardware configuration of a computer 1200 functioning as the robot 100 and the server 300. A program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the apparatus according to the present embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0134] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0135] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 retrieves image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0136] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides the programs or data to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0137] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0138] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0139] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0140] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0141] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0142] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0143] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0144] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0145] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, JAVA, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0146] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0147] (Other embodiments) The above-mentioned robot (electronic device) 100 may be mounted on a stuffed toy, or may be applied to a control device connected wirelessly or by wire to a control target device (speaker or camera) mounted on a stuffed toy.

[0148] The emotion determining unit 232 may determine the emotion of the user according to a specific mapping. Specifically, the emotion determining unit 232 may determine the emotion of the user according to an emotion map (see FIG. 7), which is a specific mapping.

[0149] FIG. 7 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0150] (1) For example, if the emotion engine, which is the emotion determination unit 232 of the robot 100, detects emotions in about 100 msec, the frequency of the determination of the reaction action (e.g., a backchannel) of the robot 100 may be set at least at the same timing as the detection frequency (100 msec) of the emotion engine, or may be set at a timing earlier than this. The detection frequency of the emotion engine may be interpreted as a sampling rate.

[0151] By detecting emotions in about 100 msec and immediately performing a corresponding reaction (e.g., a backchannel), unnatural backchannels can be avoided, and a natural dialogue that reads the atmosphere can be realized. The reaction (backchannel, etc.) is performed according to the directionality and the degree (strength) of the mandala in the robot 100 emotion map 400. Note that the detection frequency (sampling rate) of the emotion engine is not limited to 100 ms, and may be changed according to the situation (e.g., when playing sports), the age of the user, etc.

[0152] (2) The direction of emotion and the strength of the emotion may be preset in reference to the emotion map 400, and the movement of the backchannel and the strength of the backchannel may be set. For example, if the robot 100 feels a sense of stability and security, the robot 100 may nod and continue to listen. If the robot 100 feels anxious, confused, or suspicious, the robot 100 may tilt its head or stop shaking its head.

[0153] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0154] (3) If the robot 100 feels good after being praised, the filler "ah" may be inserted before the line, and if the robot 100 feels pain after receiving harsh words, the filler "ugh!" may be inserted before the line. Also, a physical reaction such as the robot 100 crouching down while saying "ugh!" may be included. These emotions are distributed around 9 o'clock on the emotion map 400.

[0155] (4) In the left half of the emotion map 400, internal sensations (reactions) are more important than situational awareness. This can give the impression that the person is reacting unconsciously.

[0156] When the robot 100 feels a favorable feeling in its situation recognition while feeling an internal sensation (reaction) of a sense of satisfaction, the robot 100 may nod deeply while looking at the other person, or may say "uh-huh." In this way, the robot 100 may generate a behavior that shows a balanced favorable feeling toward the other person, that is, a tolerance or tolerance toward the other person. Such emotions are distributed around 12 o'clock on the emotion map 400.

[0157] Conversely, even when the robot 100 is aware of a situation while experiencing an internal sensation (reaction) of discomfort, the robot 100 may shake its head when it feels disgust, or turn the LEDs in its eyes red and glare at the other person when it feels hatred. These emotions are distributed around the 6 o'clock position on the emotion map 400.

[0158] (5) The inside of emotion map 400 represents what is going on in one's mind, while the outside of emotion map 400 represents behavior. Therefore, the further out on the outside of emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0159] (6) When listening to someone while feeling a sense of security, which is distributed around the 3 o'clock area of ​​the emotion map 400, the robot 100 may lightly nod its head and say "Hmmm," but when listening to someone while feeling a sense of love, which is distributed around 12 o'clock, the robot 100 may nod vigorously, nodding its head deeply.

[0160] The emotion determination unit 232 inputs the information analyzed by the sensor module unit 210 and the recognized state of the user 10 into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user 10. This neural network is pre-trained based on a plurality of learning data that are combinations of the information analyzed by the sensor module unit 210 and the recognized state of the user 10, and emotion values ​​indicating each emotion shown in the emotion map 400. In addition, this neural network is trained so that emotions that are arranged close to each other have similar values, as in the emotion map 900 shown in FIG. 8. FIG. 8 is a diagram showing another example of an emotion map. FIG. 8 shows an example in which multiple emotions, such as "peace of mind," "calm," and "encouraging," have similar emotion values.

[0161] Furthermore, the emotion determining unit 232 may determine the emotion of the robot 100 according to a specific mapping. Specifically, the emotion determining unit 232 inputs the information analyzed by the sensor module unit 210, the state of the user 10 recognized by the state recognizing unit 230, and the state of the robot 100 to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the robot 100. This neural network is pre-trained based on a plurality of learning data that are combinations of the information analyzed by the sensor module unit 210, the recognized state of the user 10, and the state of the robot 100, and emotion values ​​indicating each emotion shown in the emotion map 400. For example, the neural network is trained based on learning data that indicates that when it is recognized from the output of a touch sensor (not shown) that the robot 100 is being stroked by the user 10, the emotion value becomes "3" for "happiness," and learning data that indicates that when it is recognized from the output of an acceleration sensor (not shown) that the robot 100 is being hit by the user 10, the emotion value becomes "3" for "anger." Furthermore, this neural network is trained so that emotions that are placed close to each other have similar values, as in the emotion map 900 shown in FIG.

[0162] The emotion determination unit 232 may determine the emotion of the robot 100 based on the behavioral content of the robot 100 generated by the sentence generation model. Specifically, the emotion determination unit 232 inputs the behavioral content of the robot 100 generated by the sentence generation model to a pre-trained neural network, acquires emotion values ​​indicating each emotion shown in the emotion map 400, and integrates the acquired emotion values ​​indicating each emotion with the emotion values ​​indicating each emotion of the current robot 100 to update the emotion of the robot 100. For example, the emotion values ​​indicating each acquired emotion and the emotion values ​​indicating each emotion of the current robot 100 are averaged and integrated. This neural network is pre-trained based on multiple learning data that are combinations of texts indicating the behavioral content of the robot 100 generated by the sentence generation model and emotion values ​​indicating each emotion shown in the emotion map 400.

[0163] For example, if the utterance of the robot 100, "That's great. You're lucky," is obtained as the behavioral content of the robot 100 generated by the sentence generation model, then when the text representing this utterance is input to the neural network, a high emotional value is obtained for the emotion "happy," and the emotions of the robot 100 are updated so that the emotional value of the emotion "happy" becomes higher.

[0164] The behavior determination unit 236 generates the robot's behavior content by adding fixed sentences for asking about the robot's behavior corresponding to the user's behavior to text representing the user's behavior, the user's emotions, and the robot's emotions, and inputting the added sentences into a sentence generation model with an interactive function.

[0165] For example, the behavior determining unit 236 acquires text representing the state of the robot 100 from the emotion of the robot 100 determined by the emotion determining unit 232, using an emotion table as shown in Fig. 9. Fig. 9 is a diagram showing an example of an emotion table. Here, in the emotion table, an index number is assigned to each emotion value for each type of emotion, and text representing the state of the robot 100 is stored for each index number.

[0166] When the emotion of the robot 100 determined by the emotion determination unit 232 corresponds to the index number "2", the text "very happy state" is obtained. When the emotions of the robot 100 correspond to multiple index numbers, multiple texts representing the state of the robot 100 are obtained.

[0167] Also, for the emotions of the user 10, an emotion table as shown in FIG. 10 is prepared. FIG. 10 is a diagram showing an example of the emotion table. Here, if the user's behavior is speaking "I'm AAA," the emotion of the robot 100 is index number "2," and the emotion of the user 10 is index number "3," then "The robot is in a very happy state. The user is in a normal happy state. The user spoke to me saying, "I'm AAA. How would you respond as the robot?" is input to the sentence generation model, and the content of the robot's behavior is acquired. The behavior decision unit 236 decides the robot's behavior from this content of the behavior.

[0168] In this way, the robot 100 can change its behavior according to the index number corresponding to the robot's emotion, so that the user gets the impression that the robot 100 has a heart, and is encouraged to take actions such as talking to the robot.

[0169] The behavior determining unit 236 may generate the behavioral content of the robot by adding not only texts representing the user's behavior, the user's emotions, and the robot's emotions, but also texts representing the contents of the history data 222, adding a fixed sentence for asking the user about the behavioral content of the robot corresponding to the user's behavior, and inputting the result into the dialogue function. This allows the robot 100 to change its behavior according to the history data representing the user's emotions and behavior, so that the user has the impression that the robot has individuality, and is encouraged to take actions such as talking to the robot. The history data may further include the emotions and behavior of the robot.

[0170] Specifically, the stuffed toy is configured as follows. For example, the robot 100 may be applied as a cohabitant (specifically, a stuffed toy 100N shown in FIG. 11 and FIG. 12) that spends daily life with the user 10 and advances a dialogue with the user 10 based on information about the daily life, and provides information tailored to the hobbies and tastes of the user 10. In this embodiment (other embodiments), an example in which the control part of the robot 100 is applied to a smartphone 50 will be described.

[0171] A stuffed animal 100N equipped with the function of an input / output device for the robot 100 has a detachable smartphone 50 that functions as a control part for the robot 100, and the input / output device and the housed smartphone 50 are connected inside the stuffed animal 100N.

[0172] As shown in FIG. 11(A), in this embodiment (other embodiments), the stuffed toy 100N has the shape of a bear covered with soft fabric, and as shown in FIG. 11(B), in the space 52 formed inside, a microphone 201 (see FIG. 2) of the sensor unit 200 is arranged in a portion corresponding to the ear 54, a 2D camera 203 (see FIG. 2) of the sensor unit 200 is arranged in a portion corresponding to the eye 56, and a speaker 60 constituting a part of the control target 252 (see FIG. 2) is arranged in a portion corresponding to the mouth 58 as input / output devices. The microphone 201 and the speaker 60 do not necessarily have to be separate bodies, and may be an integrated unit. In the case of a unit, it is preferable to place them at a position where speech can be heard naturally, such as the nose position of the stuffed toy 100N. Although the stuffed toy 100N has been described as having the shape of an animal, the present invention is not limited to this. The stuffed toy 100N may have the shape of a specific character.

[0173] The smartphone 50 has a function as a sensor module unit 210, a function as a storage unit 220, a function as a state recognition unit 230, a function as an emotion determination unit 232, a function as a behavior recognition unit 234, a function as a behavior determination unit 236, a function as a memory control unit 238, a function as a behavior control unit 250, and a function as a communication processing unit 280, as shown in FIG.

[0174] As shown in FIG. 12, a fastener 62 is attached to a part of the stuffed animal 100N (for example, the back), and by opening the fastener 62, the space 52 communicates with the outside.

[0175] Here, the smartphone 50 is accommodated in the space 52 from the outside and connected to each input / output device via a USB hub 64 (see FIG. 11(B)) to give it functionality equivalent to that of the robot 100 shown in FIG. 1.

[0176] A non-contact type power receiving plate 66 is connected to the USB hub 64. A power receiving coil 66A is built into the power receiving plate 66. The power receiving plate 66 is an example of a wireless power receiving unit that receives power wirelessly.

[0177] The power receiving plate 66 is disposed near the bases 68 of both feet of the stuffed toy 100N, and is located closest to the mounting base 70 when the stuffed toy 100N is placed on the mounting base 70. The mounting base 70 is an example of an external wireless power transmitting unit.

[0178] The stuffed toy 100N placed on this mounting base 70 can be appreciated as an ornament in its natural state.

[0179] In addition, this base portion is formed to be thinner than the surface thickness of other portions of the stuffed animal 100N, so that it is held closer to the mounting base 70.

[0180] The mounting base 70 is equipped with a charging pad 72. The charging pad 72 incorporates a power transmission coil 72A, which sends a signal to search for the power reception coil 66A of the power reception plate 66. When the power reception coil 66A is found, a current flows through the power transmission coil 72A, generating a magnetic field, and the power reception coil 66A reacts to the magnetic field to start electromagnetic induction. As a result, a current flows through the power reception coil 66A, and power is stored in the battery (not shown) of the smartphone 50 via the USB hub 64.

[0181] That is, by placing the stuffed toy 100N on the mounting base 70 as an ornament, the smartphone 50 is automatically charged, so there is no need to take the smartphone 50 out of the space 52 of the stuffed toy 100N for charging.

[0182] In this embodiment (other embodiments), the smartphone 50 is housed in the space 52 of the stuffed toy 100N and connected by wire (USB connection), but this is not limited thereto. For example, a control device having a wireless function (for example, "Bluetooth (registered trademark)") may be housed in the space 52 of the stuffed toy 100N and the control device may be connected to the USB hub 64. In this case, the smartphone 50 and the control device communicate wirelessly without putting the smartphone 50 in the space 52, and the external smartphone 50 connects to each input / output device via the control device, thereby giving the robot 100 the same functions as those of the robot 100 shown in FIG. 1. Also, the control device housed in the space 52 of the stuffed toy 100N may be connected to the external smartphone 50 by wire.

[0183] In this embodiment (and other embodiments), the teddy bear 100N is exemplified, but it may be another animal, a doll, or a specific character. It may also be dressable. Furthermore, the material of the skin is not limited to cloth, and may be other materials such as soft vinyl, but a soft material is preferable.

[0184] Furthermore, a monitor may be attached to the surface of the stuffed toy 100N to add a control target 252 that provides visual information to the user 10. For example, the eyes 56 may be used as a monitor to express joy, anger, sadness, and happiness by the image reflected in the eyes, or a window may be provided in the abdomen through which the monitor of the built-in smartphone 50 can be seen. Furthermore, the eyes 56 may be used as a projector to express joy, anger, sadness, and happiness by the image projected on a wall.

[0185] According to another embodiment, an existing smartphone 50 is placed inside the stuffed animal 100N, and the camera 203, microphone 201, speaker 60, etc. are extended from the smartphone 50 at appropriate positions via a USB connection.

[0186] Furthermore, for wireless charging, the smartphone 50 and the power receiving plate 66 are connected via USB, and the power receiving plate 66 is positioned as far outward as possible when viewed from the inside of the stuffed animal 100N.

[0187] When trying to use wireless charging of the smartphone 50, the smartphone 50 must be placed as far outward as possible when viewed from the inside of the stuffed toy 100N, which makes the stuffed toy 100N rough to the touch from the outside.

[0188] Therefore, the smartphone 50 is placed as close to the center of the stuffed animal 100N as possible, and the wireless charging function (power receiving plate 66) is placed as far outside as possible when viewed from the inside of the stuffed animal 100N. The camera 203, the microphone 201, the speaker 60, and the smartphone 50 receive wireless power via the power receiving plate 66.

[0189] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0190] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0191] 5 system, 10, 11, 12 user, 20 communication network, 100, 101, 102 robot, 200 sensor unit, 201 microphone, 202 3D depth sensor, 203 2D camera, 204 distance sensor, 210 sensor module unit, 211 voice emotion recognition unit, 212 speech understanding unit, 213 facial expression recognition unit, 214 face recognition unit, 220 storage unit, 221 reaction rule, 222 history data, 223 character data, 230 state recognition unit, 232 emotion determination unit, 234 behavior recognition unit, 236 behavior determination unit, 238 memory control unit, 250 behavior control unit, 252 control target, 280 communication processing unit, 300 server, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 Graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 input / output chip

Claims

1. A control unit that recognizes at least one of the state of a user who constitutes a family and the state of a pet kept by the user, determines its own behavior corresponding to the recognized state, and controls a control target based on the determined own behavior. An electronic device comprising:

2. The control unit is When the behavior of the pet is recognized as the state of the pet, an action of providing advice information about the pet in accordance with the behavior of the pet is determined as the behavior of the user's own.

2. The electronic device according to claim 1.

3. The control unit is When the emotion of the pet is recognized as the state of the pet, the behavior of the vehicle is determined to be an action of providing advice information about the pet in accordance with the emotion of the pet.

2. The electronic device according to claim 1.

4. The advice information is The information includes at least one of information regarding a relationship between the pet and the user who keeps the pet, and information regarding care of the pet.

4. The electronic device according to claim 2 or 3.

5. The control unit is When the behavior of a user constituting the family is recognized as a state of the user, the behavior of providing advice information about the family in accordance with the behavior of the user is determined as the behavior of the user.

2. The electronic device according to claim 1.

6. The control unit is when recognizing the emotions of the users constituting the family as the state of the user, determining, as the behavior of the device itself, an action of providing advice information about the family in accordance with the emotions of the user; 2. The electronic device according to claim 1.

7. The control unit is generating the advice information based on at least one of the personality, interest, hobbies, and inclinations of each of the plurality of users constituting the family; 7. The electronic device according to claim 5 or 6.

8. The electronic device includes: It is installed in the stuffed toy, or connected wirelessly or by wire to the controlled device installed in the stuffed toy.

2. The electronic device according to claim 1.

Citation Information

Patent Citations

  • Determination of analite in medium containing particle

    JP1985053847A