Autonomous behavior-based robot for recording daily life
An autonomous robot with emotional intelligence and privacy-focused reporting capabilities addresses the lack of emotional presence in existing robots, enabling users to understand its daily life and interactions while safeguarding privacy.
Patent Information
- Application Number
- JP2024141636
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2038-08-31
AI Technical Summary
Existing robots lack the ability to autonomously interact with their environment and users in a way that mimics the emotional presence of a pet, leading to privacy concerns when monitoring user activities and failing to provide a meaningful companionship experience.
An autonomous robot equipped with motion control, emotional parameters, and event recognition capabilities that generate reaction reports to inform users about its daily life while considering privacy, using sensors to detect events and emotional states, and generating reports through a server system.
The robot effectively informs users about its daily life while protecting privacy, allowing users to empathize with its emotional states and understand its interactions, enhancing companionship and reducing the risk of privacy breaches.
Smart Images

Figure 2024161583000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a robot that autonomously selects an action depending on an internal state or an external environment. [Background technology]
[0002] Humans keep pets to find comfort. At the same time, they often find it difficult to find the time to care for their pets. They may be unable to care for their pet, may not be able to keep a pet in their home, may have allergies, or may be suffering the loss of a loved one. Many people give up on having pets for various reasons. With robots, people who can't keep pets can have the same comfort that pets give. This may be possible (see Patent Documents 1 and 2). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2000-323219 A [Patent Document 2] International Publication No. 2017 / 169826 [Patent Document 3] JP 2005-267375 A [Patent Document 4] JP 2010-21921 A [Patent Document 5] JP 2008-172815 A Summary of the Invention [Problem to be solved by the invention]
[0004] In recent years, robot technology has been advancing rapidly, but robots have yet to have the presence of a companion like a pet. This has not yet been realized. It is because we do not believe that robots have free will. By observing the behavior of pets that seems to be of their own free will, You can feel Yui's presence, empathize with your pet, and be comforted by it.
[0005] There are currently many cameras on the market that can be used to keep an eye on your pets while you are away. According to La, knowing what your pet is doing when you're not around is key. If robots could have the same presence as pets, it would be possible to keep them at home while you are away. It is thought that there will be a demand to know about the daily lives of people in Japan.
[0006] However, if a camera is equipped on the robot, the captured images can be broadcast live to a smartphone or other device. For example, if user A and the robot When User B is at home, the robot sends the captured image to User B's smartphone while he is out. If the image were to be transmitted as is, there is a risk that user A's privacy would be compromised.
[0007] The present invention was completed based on the above problem recognition, and its main object is to To provide a technology for informing a user of a robot's daily life while taking into consideration the user's concerns. [Means for solving the problem]
[0008] In one embodiment of the present invention, an autonomous robot is provided that selects a motion of the robot. A motion control unit, a drive mechanism for executing the motion selected by the motion control unit, and an event A recognition unit that recognizes occurrences and an emotion control unit that changes one or more emotion parameters according to the event. The event occurrence history and the emotion parameter change history are associated and recorded as history information. and a history management unit for recording the history. The action control unit changes the behavior characteristics of the robot based on the emotion parameters. The history management unit records events and emotions when the emotion parameters satisfy a predetermined psychological condition. The emotion parameters when the conditions are met are extracted from the history information, a reaction report is generated, and the user is informed of the emotion. Provide the user with a reaction report. Effect of the Invention
[0009] According to the present invention, it is possible to easily inform a user of a robot's daily life while respecting privacy. It becomes. [Brief description of the drawings]
[0010] [Figure 1] FIG. 1 is a conceptual diagram for explaining how a robot can communicate its daily life to a user. [Diagram 2] Figure 2(a) is a front view of the robot, and Figure 2(b) is a side view of the robot. [Diagram 3] FIG. 2 is a cross-sectional view that illustrates a schematic structure of the robot. [Figure 4] FIG. 2 is a diagram showing the hardware configuration of the robot in the basic configuration. [Diagram 5] FIG. 2 is a functional block diagram of the robot system. [Figure 6] FIG. 2 is a schematic diagram showing a reaction report generation process in this embodiment. [Figure 7] FIG. 2 is a functional block diagram of the robot according to the present embodiment. [Figure 8] FIG. 13 is a schematic diagram showing a change history of emotion parameters. [Figure 9] FIG. 4 is a data structure diagram of history information. [Figure 10] FIG. 13 is a data structure diagram of a motion selection table. [Figure 11] FIG. 13 is a screen diagram of a first reaction report screen. [Figure 12] FIG. 13 is a screen diagram of a second reaction report screen. [Figure 13]FIG. 13 is a data structure diagram of a user authority table. [Figure 14] FIG. 13 is a data structure diagram of an event condition table. [Figure 15] FIG. 13 is a sequence diagram showing the process of displaying a reaction report. [Figure 16] FIG. 11 is a sequence diagram showing a process of displaying details. [Figure 17] FIG. 13 is a screen diagram of a detailed information screen. [Figure 18] FIG. 13 is a screen diagram of the third reaction report screen. [Figure 19] FIG. 13 is a screen diagram of a rest time setting screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] FIG. 1 is a conceptual diagram for explaining how a robot 100 communicates its daily life to a user. It is. The robot 100 in this embodiment has a plurality of kinds of "emotions." Bot100 uses three different emotion patterns to measure the intensity of three different emotions: loneliness, curiosity, and the desire for approval. As will be described later, the emotion parameters are expressed numerically. The robot 100 may express emotions according to external emotion parameters.
[0012] The robot 100 is equipped with a number of types of sensors, and recognizes the external environment using each sensor. The robot 100 recognizes some of the changes in the external environment as "events." Arbitrarily defined as a "notable event" such as seeing the user or detecting a loud sound. The robot 100 can autonomously respond according to the events and emotion parameters. The event determines both the behavior selection method and the emotion parameters of the robot 100. The emotion parameter also affects the action selection method. The action selection method will be described in detail later.
[0013] The robot 100 records events and other "daily occurrences" one by one as "history information." The robot 100 generates a "reaction report" to show to the user based on the history information. Users can generate the data using smartphones, laptops, tablet computers, etc. The reaction report is acquired by the user terminal 400. By displaying the message, the user can feel the robot's presence even when the user is not in contact with the robot 100. You can get a glimpse into the daily life of To100.
[0014] Even when the user is out, the robot 100 can check how the user is spending their time at home. You can find out how your day went by reading the reaction report. The reaction report in this embodiment functions as a "diary" of the robot 100. This is "history information for showing" and is the same as the history information that the robot 100 records one by one. The history information may include information related to privacy. In this situation, we will improve the response report to be more user-friendly and considerate of privacy. Generate it first. The basic configuration of the robot 100 will be described below with reference to FIGS. 2 to 5. The explanation will focus on how to generate reports.
[0015] [Basic configuration] FIG. 2(a) is a front view of the robot 100. FIG. 2(b) is a front view of the robot 100. FIG. The robot 100 in this embodiment determines its behavior based on the external environment and its internal state. The external environment is monitored by various sensors such as cameras and thermo sensors. The internal state of the robot 100 is recognized by various parameters that express the emotions of the robot 100. The robot 100's movement range is within the owner's home. Humans involved with the robot 100 are called "users."
[0016] The body 104 of the robot 100 has a generally rounded shape and is made of urethane or rubber. The robot includes an outer skin made of soft and elastic materials such as resin and fiber. The total weight of the robot 100 is about 5 to 15 kilograms, and the height is 0 It is about 1.5 to 1.2 meters long. It has a moderate weight, roundness, softness, and a pleasant feel. The attributes of the robot 100 are designed to make it easier for the user to hold the robot 100 and to make the user feel comfortable holding the robot 100. The effect is realized.
[0017] The robot 100 has a pair of front wheels 102 (left wheel 102a and right wheel 102b) and one rear wheel. The front wheels 102 are driving wheels, and the rear wheels 103 are driven wheels. The front wheels 102 are Although it does not have a steering mechanism, the rotation speed and direction can be controlled individually. are casters that can rotate freely to move the robot 100 forward, backward, left and right. The rear wheel 103 may be an omni wheel.
[0018] The front wheels 102 and the rear wheels 103 are driven by a drive mechanism (rotation mechanism, link mechanism) to move the body 10 4. Even when driving, most of the wheels are hidden by the body 104, When each wheel is completely retracted into the body 104, the robot 100 is unable to move. That is, as the wheels are retracted, the body 104 descends and rests on the floor F. In the seated state, a flat seating surface 108 (ground contact surface) formed on the bottom of the body 104 abuts against floor surface F.
[0019] The robot 100 has two hands 106. The hands 106 do not have the function of grasping objects. The hand 106 can perform simple actions such as raising, shaking, and vibrating. Can be controlled individually.
[0020] The eyes 110 are capable of displaying images using liquid crystal elements or organic EL elements. 0 uses various sensors, such as microphone arrays and ultrasonic sensors, to identify the direction of a sound source. It also has a built-in speaker and can emit simple sounds.
[0021] The robot 100 has a horn 112 attached to its head. Because the horns 112 are lightweight, the user can lift the robot 100 by grasping the horns 112. A spherical camera is attached to the horn 112, and the entire upper area of the robot 100 is captured. It is possible to capture all of the images at once.
[0022] FIG. 3 is a cross-sectional view that illustrates a schematic structure of the robot 100. As shown in FIG. As shown in FIG. 3, the body 104 of the robot 100 includes a base frame 308, a main body frame 309, and a The base frame 310 includes a pair of resin wheel covers 312 and an outer skin 314. The frame 308 is made of metal, and constitutes the axis of the body 104 and supports the internal mechanism. The base frame 308 includes an upper plate 332 and a lower plate 334. The side plates 336 are connected vertically. A sufficient gap is provided between the base frame 308 to allow ventilation. The battery 118, the control circuit 342 and various actuators are housed in the housing 342.
[0023] The main body frame 310 is made of a resin material, and the head frame 316 and the torso frame 31 8. The head frame 316 is hollow and semi-spherical, and forms the head skeleton of the robot 100. The torso frame 318 has a stepped cylindrical shape and forms the torso skeleton of the robot 100. The body frame 318 is fixed integrally to the base frame 308. 6 is attached to the upper end of the body frame 318 so as to be capable of relative displacement.
[0024] The head frame 316 has three axes: a yaw axis 320, a pitch axis 322, and a roll axis 324. An actuator 326 is provided to rotate each of the axes. 6 includes multiple servo motors for driving each axis individually. The axis 320 is driven for a nodding motion, the pitch axis 322 is driven for a tilting motion, To achieve this, the roll shaft 324 is driven.
[0025] A plate 325 supporting the yaw axis 320 is fixed to the upper part of the head frame 316. The plate 325 is provided with a plurality of ventilation holes 327 for ensuring ventilation between the top and bottom. can be.
[0026] A metal base plate is provided to support the head frame 316 and its internal mechanism from below. The base plate 328 is provided with a cross link mechanism 329 (pantograph). The plate 325 is connected to the upper plate 325 via a joint 330. It is connected to the per plate 332 (base frame 308).
[0027] The torso frame 318 houses the base frame 308 and the wheel drive mechanism 370. The drive mechanism 370 includes a rotation shaft 378 and an actuator 379. The lower half of the wheel cover 312 is provided with a storage space S for the front wheel 102. Therefore, it is considered to be small.
[0028] The outer cover 314 is made of urethane rubber and is connected to the main body frame 310 and the wheel cover 31. The hand 106 is integrally formed with the outer shell 314. The upper end of the outer shell 314 is An opening 390 is provided for introducing outside air.
[0029] FIG. 4 is a diagram showing the hardware configuration of the robot 100. The robot 100 includes an internal sensor 128, a communication device 126, a memory device 124, a processor 1 22, a drive mechanism 120 and a battery 118. 4 is included in the control circuit 342. Each unit is connected to a power supply line 130 and a signal line 132. The battery 118 supplies power to each unit via a power line 130. Each unit transmits and receives a control signal through a signal line 132. The battery is an ion secondary battery and is the power source for the robot 100.
[0030] The internal sensor 128 is a collection of various sensors built into the robot 100. The system is composed of a camera (spherical camera), a microphone array, a distance sensor (infrared sensor), These include motion sensors, touch sensors, acceleration sensors, and odor sensors. Touch sensors include It is installed between the outer skin 314 and the main body frame 310 and detects the user's touch. The sensor is a known sensor that applies the principle that electrical resistance changes due to the adsorption of odor-causing molecules. It is a .
[0031] The communicator 126 is a communication module that performs wireless communication with various external devices. The storage device 124 is composed of a non-volatile memory and a volatile memory, and is used for storing data for a computer program. The processor 122 stores the computer program and various setting information. The drive mechanism 120 is a means for carrying out the above-mentioned functions. 70. In addition, a display and a speaker are also installed.
[0032] The drive mechanism 120 mainly controls the wheels (front wheels 102) and the head (head frame 316). The driving mechanism 120 controls the direction and speed of the movement of the robot 100, as well as the vehicle. The wheels (front wheels 102 and rear wheels 103) can also be raised and lowered. When the wheels are raised, is completely housed in the body 104, and the robot 100 abuts on the floor surface F with the seating surface 108. The robot is in a seated position. The driving mechanism 120 also controls the hand 106 via a wire 134.
[0033] FIG. 5 is a functional block diagram of the robot system 300. The robot system 300 includes a robot 100, a server 200, and a plurality of external sensors 1 Each component of the robot 100 and the server 200 includes a CPU (Central Processor). computing units such as the computing unit (ESS) and various coprocessors, and memory and storage The hardware includes the devices, the wired or wireless communication lines connecting them, and the data stored in the storage device. It is realized by software that provides processing instructions to the arithmetic unit. The program is a system that integrates device drivers, operating systems, and the various layers above them. application programs, and live Each block described below may be configured as a hardware unit. 1 shows functional blocks rather than a single block. A part of the functions of the robot 100 may be realized by the server 200. Some or all of the functions may be realized by the robot 100.
[0034] A plurality of external sensors 114 are installed in the house in advance. The position coordinates of the sensor 114 are registered. The internal sensor 128 of the robot 100 and the multiple Based on the information obtained from the external sensor 114, the server 200 controls the basic movement of the robot 100. The external sensor 114 is used to augment the sensory organs of the robot 100. The server 200 is intended to augment the intelligence of the robot 100. The communicator 126 periodically communicates with the external sensor 114, and the server 200 communicates with the external sensor 114. The position of the robot 100 is identified by this (see also Patent Document 2).
[0035] (Server 200) The server 200 includes a communication unit 204 , a data processing unit 202 , and a data storage unit 206 . The communication unit 204 is responsible for communication with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. Various processes are performed based on the acquired data and the data stored in the data storage unit 206. The data processing unit 202 is an interface between the communication unit 204 and the data storage unit 206. It also functions as a
[0036] The data store 206 includes a motion store 232 and a personal data store 218 . The robot 100 has a number of motion patterns. , meandering towards the owner, staring at the owner with his head tilted to one side, etc. A motion is defined.
[0037] The motion storage unit 232 stores a "motion file" that defines the control contents of the motion. Each motion is identified by a motion ID. The motions are also downloaded to the motion storage unit 160 of the robot 100. The decision as to whether to execute the command may be made by the server 200 or by the robot 100. It also says:
[0038] Many of the motions of the robot 100 are compound motions that include multiple unit motions. For example, when the robot 100 approaches the owner, the robot 100 turns toward the owner. Unit motion of turning around, unit motion of approaching while raising hands, unit motion of approaching while shaking body It is expressed as a combination of unit motions, sitting down with both hands raised, By combining these four motions, The motion is that the robot raises its hands in the air, then shakes its body and sits down. The application file includes the rotation angle and angular velocity of the actuators provided in the robot 100. These are defined in relation to the time axis. Motion file (actuator control information) By controlling each actuator over time, various motions can be achieved. It is expressed.
[0039] The transition time from the previous unit motion to the next unit motion is set as "interval". The interval is determined according to the time required to change the unit motion and the content of the motion. The length of the interval can be adjusted. Below, we will explain when and which motion to choose, and the actuators involved in achieving the motion. The settings related to the behavior control of the robot 100, such as the output adjustment, are collectively referred to as "behavior characteristics." The behavioral characteristics of the robot 100 are determined by the motion selection algorithm, the motion selection probability, , motion files, etc.
[0040] The motion storage unit 232 stores motion files as well as various events. It stores a motion selection table that defines the motion to be executed when In the selection table, one or more motions and their selection probabilities are associated with each event. can be done.
[0041] The personal data storage unit 218 stores information about the user. Stores master information that indicates density and physical and behavioral characteristics of users, such as age and gender. Other attribute information may also be stored.
[0042] The robot 100 has an internal parameter called intimacy for each user. 0 recognizes actions that show affection towards him, such as picking him up or talking to him. When the robot 100 is not connected to the user, the robot 100 becomes more intimate with the user. The intimacy level will be low for users who are violent or who are encountered infrequently.
[0043] The data processing unit 202 includes a position management unit 208, a recognition unit 212, a motion control unit 222, and an intimacy level control unit. The control unit 220 and the state control unit 244 are included. The position management unit 208 identifies the position coordinates of the robot 100. Various internal parameters such as the electrical conductivity, internal temperature, and various physical conditions such as the processing load of the processor 122 The state management unit 244 also manages the emotions of the robot 100 (loneliness, curiosity, acceptance, etc.). These emotion parameters are always The movement target point of the robot 100 changes according to the emotion parameter. For example, when the user feels lonely, the robot 100 moves around the user. Set as a benchmark.
[0044] Emotion parameters change over time. In addition, various emotions are also generated by the actions taken, as described below. Emotion parameters change. For example, if the owner holds the robot, it will feel lonely. The emotion parameter decreases, and when the owner is not visible for a long time, the owner feels lonely. The affection parameter increases little by little.
[0045] The recognition unit 212 recognizes the external environment. The recognition of the external environment includes the weather based on temperature and humidity. This includes a variety of recognition such as recognizing the season and recognizing shade (safety zones) based on the amount of light and temperature. The recognition unit 156 of the robot 100 acquires various kinds of environmental information using the internal sensor 128. This is subjected to primary processing and then transferred to the recognition unit 212 of the server 200.
[0046] Specifically, the recognition unit 156 of the robot 100 recognizes moving objects, particularly people and animals, from an image. The image region corresponding to the moving object is extracted, and the physical and behavioral features of the moving object are extracted from the extracted image region. A "feature vector" is extracted as a set of features that indicate the following: It is a number that quantifies various physical and behavioral characteristics. For example, the width of a human eye is a number between 0 and 1. The feature vector is quantified in the range of 1 to 100, forming one feature vector component. The method of extracting the torrent is an application of known face recognition technology. The signature vector is sent to the server 200 .
[0047] The recognition unit 212 of the server 200 extracts the image captured by the built-in camera of the robot 100. The feature vector obtained by the above process and the user (class) information registered in advance in the personal data storage unit 218 are stored. By comparing the feature vectors of the captured image with those of the person, In addition, the recognition unit 212 performs image recognition of the user's facial expression. The recognition unit 212 estimates the user's emotion by detecting a moving object other than a person, for example The user identification process is also performed on pets such as cats and dogs.
[0048] The recognition unit 212 recognizes various behaviors performed on the robot 100 and determines whether the behaviors are pleasant or unpleasant. The recognition unit 212 also classifies the owner's response to the behavior of the robot 100. By recognizing the cause, reactions can be classified as positive or negative. Pleasant / unpleasant behavior refers to whether the user's response is pleasant or unpleasant for the living body. For example, being held is a pleasurable behavior for the robot 100. Being kicked is an unpleasant act for the robot 100. The user's response behavior is judged based on whether it indicates a pleasant or unpleasant emotion. Being hugged is a positive reaction that indicates the user's positive emotion, while being kicked is a positive reaction that indicates the user's positive emotion. It is a negative response that indicates the person's discomfort.
[0049] The operation control unit 222 of the server 200 cooperates with the operation control unit 150 of the robot 100 to: The motion control unit 222 of the server 200 determines the motion of the robot 100. The movement control unit 222 creates 100 movement destinations and a movement route for the destination. A travel route may be created and then any one of the travel routes may be selected.
[0050] The motion control unit 222 selects a motion for the robot 100 from a plurality of motions in the motion storage unit 232. Each motion is associated with a selection probability for each situation. For example, when the owner performs a pleasant action, motion A will be executed with a 20% probability. When the temperature reaches 30 degrees or higher, motion B will be executed with a 5% probability. A selection method is defined.
[0051] The intimacy management unit 220 manages the intimacy for each user. The personal data is registered as part of the personal data in the personal data storage unit 218. When an unpleasant behavior is detected, the intimacy management unit 220 increases the intimacy level with the owner. If you do not look at the owner for a long period of time, the intimacy level will gradually decrease. It decreases to.
[0052] (Robot 100) The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 148, and an internal The actuator 120 includes a sensor 128 and a drive mechanism 120. The communication unit 142 corresponds to the communication device 126 (see FIG. 4 ) and communicates with the external sensor 114, the server 20, 0 and other robots 100. The data storage unit 148 stores various data The data storage unit 148 corresponds to the storage device 124 (see FIG. 4). The management unit 136 receives the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 executes various processes based on the data stored in the processor 12. 2 and a computer program executed by the processor 122. The processing unit 136 includes a communication unit 142, an internal sensor 128, a drive mechanism 120, and a data storage unit. It also functions as a 148 interface.
[0053] The data storage unit 148 is a motion storage unit that defines various motions of the robot 100. Includes 160. The motion storage unit 160 of the robot 100 includes the motion storage unit 23 of the server 200. Various motion files are downloaded from 2. The motions are stored in the motion ID. The driver sits down with the front wheels 102 folded up, hands 106 are raised, and the two front wheels By rotating the front wheels 102 in the opposite direction, or by rotating only one of the front wheels 102, the robot Rotating the front wheels 102 while the front wheels 102 are retracted. It has various motions, such as shaking when you press the button, stopping and turning around when it moves away from the user, etc. In order to express the motion, the operation timing and operation of various actuators (drive mechanism 120) are The time, direction of movement, etc. are defined chronologically in the motion file. Various data are also downloaded from the personal data storage unit 218 to the data storage unit 148. This may also be the case.
[0054] The data processing unit 136 includes a recognition unit 156 and an operation control unit 150 . The motion control unit 150 of the robot 100 cooperates with the motion control unit 222 of the server 200 to Decide the motion of the bot 100. Some of the motions are decided by the server 200. Other motions may be determined by the robot 100. However, when the processing load of the robot 100 is high, the server 200 determines the motion. The server 200 may determine a base motion. In addition, the robot 100 may determine an additional motion. How the server 200 and the robot 100 share the work is determined by the robot system 3. Just design it according to the specifications of 00.
[0055] The motion control unit 150 of the robot 100 instructs the driving mechanism 120 to execute the selected motion. The driving mechanism 120 controls each actuator according to the motion file. do.
[0056] The operation control unit 150 is configured to ask for a hug when a user with a high degree of intimacy is nearby. It can also perform the motion of lifting both hands 106 as grass, or "hug" When you get tired of it, you can alternate between reverse rotation and stopping while keeping the left and right front wheels 102 in place. The driving mechanism 120 can also express motions that indicate dislike for being held. 50, the front wheels 102, hands 106, and neck (head frame 316) are driven. This allows the robot 100 to perform a variety of motions.
[0057] The perception unit 156 of the robot 100 interprets external information obtained from the internal sensors 128. The recognition unit 156 is capable of visual recognition (visual unit), smell recognition (olfactory unit), and sound recognition (auditory unit). ), and tactile recognition (tactile part) is possible.
[0058] The recognition unit 156 extracts a feature vector from the captured image of the moving object. A feature vector is a set of parameters (feature values) that indicate the physical and behavioral characteristics of a moving object. When a moving object is detected, the odor sensor, built-in sound collecting microphone, temperature sensor, etc. Physical and behavioral features are also extracted from the image. These features are also quantified and a feature vector is created. The recognition unit 156 recognizes the feature vector based on the known technology described in Patent Document 2, etc. Identify the user from the route.
[0059] Among the series of recognition processes including detection, analysis, and judgment, the recognition unit 156 of the robot 100 performs the recognition The server 200 selects and extracts the necessary information, and the server 200 interprets and determines the interpretation. The recognition process may be performed by the recognition unit 212 of the server 200 alone, or may be performed by a The recognition unit 156 of the bot 100 may perform the recognition process alone, or both units may share the same role as described above. The above recognition process may be performed while the image is being viewed.
[0060] When a strong impact is applied to the robot 100, the recognition unit 156 detects the touch sensor and The recognition unit 212 of the server 200 recognizes the user's movement by the acceleration sensor and the user's movement. The user grasps the horn 112 and the robot 10 is recognized as having committed a "violent act." When the robot 100 is lifted, it may be recognized as a violent act. When a user speaks in a specific volume range and a specific frequency band, the server 200 recognizes the The unit 212 may recognize that a "calling action" has been performed on the user. When the temperature of the device is detected, it is recognized that the user has made a "touching action" and that contact has been recognized. When the device detects upward acceleration in this state, it recognizes that the user is being held. It is also possible to sense the physical contact when lifting the steering wheel 104, or to sense the contact with the front wheel 102. The baby may become aware that it is being held due to a reduction in the weight it is carrying. In summary, the robot 100 detects the user's actions as physical information using the internal sensor 128. The recognition unit 212 of the server 200 judges whether the user feels comfortable or uncomfortable. The identification unit 212 executes a user identification process based on the feature vector.
[0061] The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Among various types of response behavior, some typical response behaviors are pleasant or unpleasant, positive or negative. Generally speaking, most of the responses that are considered pleasant are positive reactions, while most of the responses that are considered unpleasant are negative reactions. Most of the responses that are positive or negative are negative. The constant response influences the robot's 100 choice of actions.
[0062] In response to the response action recognized by the recognition unit 156, the intimacy management unit 220 of the server 200 changes the intimacy level with the user. In principle, The degree of intimacy with the user who committed the unpleasant act increases, and the degree of intimacy with the user who committed the unpleasant act decreases.
[0063] Based on the above basic configuration, the implementation of the robot 100 in this embodiment will now be described. In particular, the description will focus on the features and purpose of this implementation and the differences from the basic configuration.
[0064] [Implementation of reaction report generation function] FIG. 6 is a schematic diagram showing the reaction report generation process in this embodiment. The robot 100 records events, emotion parameters, etc. as history information (S10). The history information is a time series of the feelings (emotion parameters) and events (events) of the robot 100. The user sends a message to the server 200 from the user terminal 400. A reaction report is requested (S12). The server 200 receives the reaction report request. When the reaction is received, a reaction report is generated from the history information (S14). Therefore, if you use the history information as it is as a reaction report, the reaction report may be difficult to read. Also, from the viewpoint of privacy protection, the history information itself is not used as a reaction report. In this embodiment, the history information is not provided to the user. By selecting and discarding various information and changing the way the information is presented, The user terminal 400 generates a reaction report for the robot (described in detail later). The information is obtained from the system 300 and displayed on the screen (S16).
[0065] FIG. 7 is a functional block diagram of the robot 100 in this embodiment. The robot system 300 in this embodiment has a reaction report generation function. Therefore, the function block of the robot system 300 shown in FIG. 5 is a block diagram of a robot system 300 in which a plurality of function blocks are added. In FIG. 7, the added functional blocks will be mainly described.
[0066] Note that the robot 100 determines its location based on key frames, not on the external sensor 114. A key frame is distribution information of feature points (feature amounts) in a captured image. The robot 100 uses a graph-based SLAM (Simultaneous Local Algorithm) that uses image features. ization and Mapping) technology, more specifically, ORB (Oriented FAST and Rotated BRI) EF) Keyframes may be generated using feature-based SLAM technology (see Patent Application No. 20 for details). (See No. 18-014365).
[0067] (Server 200) The data processing unit 202 further includes a history management unit 252, an icon selection unit 254, and an atmosphere determination unit. The state management unit 244 includes a mood management unit 256 and a mode selection unit 258. Including 0. The emotion management unit 250 calculates an emotion parameter indicating the intensity of loneliness (hereinafter, the “loneliness parameter”). "), and an emotion parameter that indicates the strength of curiosity (hereafter referred to as the "curiosity parameter"). and the emotional parameter indicating the strength of the desire for recognition (hereinafter referred to as the "desire for recognition parameter"). Manage three types of emotion parameters.
[0068] The history management unit 252 manages the history information of the robot 100. The event occurrence history and the emotion parameter change history can be associated. The history management unit 252 also generates a reaction report based on the history information. Generate.
[0069] The icon selection unit 254 selects an icon according to an event. In the reaction report of this embodiment, events are represented graphically by icons. The icon selection unit 254 selects an icon to be associated with an event. 54 also selects various icons outside of events.
[0070] The atmosphere determination unit 256 determines the atmosphere of the external environment. The method of determining the atmosphere will be described later. The mode selection unit 258 selects the mode of the robot 100. The modes are normal mode, The setting information is related to the operating state of the robot 100, such as the power saving mode, the power saving mode, etc. The selection method will be described in detail later.
[0071] The data storage unit 206 further includes an event condition storage unit 260 and a history information storage unit 262. The event condition storage unit 260 defines an event condition. For example, you can define the circumstances under which an event occurred, such as "occurred in a child's room" or "occurred at the entrance." This is a condition for determining whether or not the history information is to be stored.
[0072] (Robot 100) The internal sensor 128 of the robot 100 includes an imaging unit 158. The imaging unit 158 captures an image of the entire celestial sphere. In this embodiment, the image capturing unit 158 captures surrounding images at all times. The recognition unit 156 of the robot 100 receives an image from an internal sensor 128 such as an imaging unit 158. The external information (raw data) is analyzed, and the analysis result (processed data) is transmitted to the server 200. The recognition unit 212 of the server 200 judges the success or failure of various events based on the analysis results. The success or failure of the event may be determined only on the robot 100 side or only on the server 200 side. Alternatively, the recognition unit 156 of the robot 100 may determine simple events and the recognition unit 156 may determine complex events. The event may be determined by the recognition unit 212 of the server 200 . In the following, it is assumed that the recognition unit 212 of the server 200 determines whether an event is successful or not. The recognition unit 212 of the server 200 is to be read as the recognition unit 156 of the robot 100. It is possible.
[0073] The personal data storage unit 218 of the server 200 stores a terminal ID for identifying the user terminal 400 and a user ID. The user ID is registered in association with the user's physical characteristics and and behavioral features are matched.
[0074] (User terminal 400) The user terminal 400 displays a reaction report, which will be described later, on a web browser. The terminal 400 displays a reaction report and inputs instructions to the robot 100 (see FIG. 19). Dedicated application software for the above-mentioned operations (described in detail below) may be installed.
[0075] FIG. 8 is a schematic diagram showing a change history of emotion parameters. The emotion management unit 250 changes the three types of emotion parameters over time. The vertical axis represents time, and the vertical axis represents the magnitude of the emotion parameter. The meter varies in the range of 0 to 100. In FIG. 8, the loneliness parameter 170 is The emotion management unit 250 determines whether the robot 100 is The longer the period of time that the owner does not see the Gnar, the more the loneliness parameter will increase by 170. When you see the owner, the loneliness parameter will decrease by 170. In the same way, listening to the owner's voice contributes to a decrease in the loneliness parameter 170. When such an event occurs, the emotion management unit 250 also reduces the loneliness parameter 170. This is also fine.
[0076] The need for approval parameter 174 in Figure 8 gradually increases over time. The unit 250 increases the recognition need parameter 174 over time. "I want to be recognized by others" is one of the basic human desires. The emotion management unit 250 expresses the strength of the robot 100's need for approval by the user. Pleasant behavior (attention to Robot 100), such as talking to Robot 100 or holding it. When an unpleasant act is committed, the need for recognition parameter 174 is lowered (satisfied), and when an unpleasant act is committed, the need for recognition parameter 174 is lowered (satisfied). This increases the need for approval parameter by 174 (dissatisfaction).
[0077] When the robot 100 discovers something new, the emotion management unit 250 172. Also, when no event occurs, the emotion management unit 250 increases curiosity. The parameter 172 is gradually decreased.
[0078] The emotion management unit 250 determines whether the three types of emotion parameters satisfy various "psychological conditions." The psychological condition is determined as a timing to indicate a "psychological change" of the robot 100. For example, the approval desire parameter 174 may be set to a predetermined threshold (e.g., 80). When the above is true, the first psychological condition may be satisfied, and the recognition desire parameter 174 When becomes dominant over the other two emotional parameters, even if the second psychological condition is met, In FIG. 8, during the period t0 to t1, the loneliness parameter 170 is Parameter 172 is greater than the need for approval parameter 174. Therefore, at time t1, the emotion management unit 250 It is determined that the second psychological condition is satisfied. In other words, at time t1, the robot 100 The desire for recognition (the desire to be recognized) is more prevalent than loneliness.
[0079] Of the three types of emotion parameters, the first emotion parameter is dominant, and then the second emotion parameter is dominant. When a parameter changes to a dominant state, in other words, the superiority or inferiority of multiple emotion parameters When the situation reversed, it is believed that Robot 100's feelings underwent a major change.
[0080] In addition, the emotional parameters can be set to four or more types, not limited to the three types of loneliness, curiosity, and desire for approval. For example, various emotional parameters such as joy, anger, anxiety, impatience, and shame can be set. You may define events that affect the parameters of the character and their emotional parameters. The emotion parameters change with the passage of time and the occurrence of events.
[0081] The change in the emotion parameter may be random or periodic. A change in the parameter may affect a second emotion parameter, for example a loneliness parameter. When the value of the desire for approval parameter is 50 or more, the emotion management unit 250 may increase the rate of increase of the desire for approval parameter. good.
[0082] Emotion parameters do not necessarily correspond to typical human psychological states such as "curiosity." For example, "Emotion parameter Q1 is Event (E1), Event (E2) ) occurs as a trigger, and it falls as a trigger for the occurrence of event (E3)." By defining the conditions for change, it is possible to define emotion parameters with various personalities. It is okay to define "emotions" specific to the robot 100 that do not mimic human emotions. good.
[0083] FIG. 9 is a diagram showing the data structure of the history information 270. As shown in FIG. The history information 270 is stored in the history information storage unit 262 of the server 200. In the system 300, multiple types of events are defined in advance. Events are classified into one of the event types. For example, an event of event type = En ( Hereinafter, an event (hereinafter referred to as "Event (En)") may be visually recognizing a user. The event (Em) may be the detection of a sound above a certain threshold, or it may be another event. The Ep may be that the robot 100 is wearing clothes. Various events occur when the device is turned on, the user goes out, or finds something new in the room. A vent can be defined.
[0084] The history management unit 252 periodically, for example, every minute, The history information 270 includes the event occurrence history and emotion parameters. The recognition unit 212 recognizes the event and the emotion management unit 25 The motion control unit 150 changes the emotion parameters according to the event. A motion is selected according to the emotion parameter.
[0085] In the history information 270 of FIG. 9, an event (E1) occurs at 10:03. After the vent (E1) occurred, the loneliness parameter dropped sharply from 72 to 14. (E1) proves to be an important event that helped Robot 100 to overcome his loneliness. In addition, the history information 270 also includes an atmosphere type indicating the atmosphere of the external environment (described in detail later). , the motion selected by the robot 100, the location information of the robot 100, room temperature, etc. Such information may be registered.
[0086] FIG. 10 is a data structure diagram of the motion selection table 280. The motion selection table 280 is stored in the motion storage unit 232. In addition, the motion is identified by a motion ID. When an event is recognized, a motion is selected by referring to the motion selection table 280. The robot 100 executes the motion specified by the motion control unit 222. The application selection table 280 is also downloaded from the server 200 to the robot 100. For this reason, the motion control unit 150 of the robot 100 may use the server You can select the motion that corresponds to the event without going through 200. In the following, when a motion is selected by the motion control unit 222 (server 200), The operation control unit 222 (server 200) is a control unit for the operation control unit 150 (robot 10). 0).
[0087] In the motion selection table 280, one or more motions and For example, when an event (E1) is detected, the motion The motion control unit 222 selects the motion (M1) with a selection probability of 5(%) and a selection probability of 15(%). Select motion (M2) with probability.
[0088] In this embodiment, the motion control unit 222 selects a motion according to the emotion parameter. For example, when the loneliness parameter among the emotion parameters is greater than or equal to a predetermined threshold, When the motion control unit 222 is on, the motion control unit 222 When the curiosity parameter is equal to or greater than a predetermined threshold, the operation control unit 222 reduces the probability of selecting a motion (M1) for an event (E1).
[0089] As a concrete example, when an event (En) occurs, the selectable motion (M1), The selection probability of motion (M2) and motion (M3) is set to 30(%), 20(%), Let's assume that the value is 50(%). The following motions are selected when an event (En) occurs: (M1) is written as "motion (En|M1)".
[0090] When loneliness parameter is 80 or more, motion (En|M1) is increased from 30(%) to 40 The increased 10% is the motion (En|M2) and the motion The selection probability of each of the two options (En|M3) is adjusted by decreasing it by 5%. As a result, when the loneliness parameter is 80 or more, the selection probability of motion (En|M2) The probability of selecting motion (En|M3) is reduced from 20(%) to 15(%). ) to 45(%). According to this control method, the motion (En|M1) The robot 100 is more likely to be selected when it is "lonely," and "emotions" influence "action selection." It will have an impact.
[0091] FIG. 11 is a screen view of the first reaction report screen 180. The user requests the server 200 to obtain a reaction report from the user terminal 400. The user can freely switch the display method of the reaction report. The event screen 180 shows a reaction report that associates events with the feelings (emotion parameters) of the robot. Indicates the port.
[0092] When a reaction report acquisition request is received from the user terminal 400, the history management unit 252 A reaction report is generated based on the history information 270. The history management unit 252 Extract emotion parameters when the conditions are met.
[0093] First, at 9:00, one of the multiple psychological conditions is met. Server 2 In 00, emotions are determined in advance for each psychological condition or for each state of emotional parameters. The icon selection unit 254 is associated with the emotion icon 184. An emotion icon 184 is selected according to the emotion parameter at the time. Icon 184a is selected.
[0094] The icon selection section 254 selects an event icon corresponding to an event that occurs when a psychological condition is satisfied. Select the event icon 182. For each of the multiple events, select the event icon 18 2 is associated with the "event when the psychological condition is met" It may be an event that occurred immediately before (the most recent) the point, or a predetermined time after the psychological condition is established. For example, if a psychological condition is satisfied at time t1, Then, the events that occurred during the 7 seconds from time (t1-10 seconds) to time (t1-3 seconds) are reflected. It may also be displayed in a response report.
[0095] In the first reaction report screen 180 of FIG. 11, the robot's reaction when the psychological condition is met is displayed. Emotion icons 184a representing 100 "feelings" are displayed, and events that occur when psychological conditions are met An event icon 182a corresponding to the event (event type) is also displayed. The event icon 182a shown corresponds to an event in which the robot 100 met the owner. The emotion icon 184a indicates the happy feeling of the robot 100, for example, the loneliness Corresponds to a psychological state in which both the meter and the approval desire parameter are below a certain value. By referring to the first reaction report screen 180, the user can determine "I was happy to meet the owner at 9:00," said the robot's owner. You can check the feelings of 100 Toki no Robots.
[0096] The user can check the feelings and events of the robot 100 through the first reaction report screen 180. Events can be known by emotion icons 184 and event icons 182. By symbolically expressing the robot 100 with icons, the user can easily understand the robot's daily life. You can enjoy it like a picture diary.
[0097] In this embodiment, the recognition unit 212 recognizes an event based on information about the external environment. Then, the icon selection unit 254 identifies the event type corresponding to the recognized event. Select the event icon 182 that corresponds to the event type. Event details are not shown, but "What happened?" is displayed in event icon 182. By simply conveying the information, the first reaction report screen 180 does not excessively infringe on privacy. It prevents it from being disclosed.
[0098] The emotion parameters of the robot 100 are not represented by the emotion icon 184 but by the parameter The value may be displayed as it is, or the feelings of the robot 100 may be expressed by a color corresponding to the emotion. The emotion parameters of the robot 100 may be expressed as a user's privacy policy. This is because it is considered unlikely that this will happen.
[0099] The robot 100 selects an action while recognizing a large number of events. Showing events in reaction reports is not only important from the perspective of privacy protection, but also from the perspective of visibility. Therefore, in this embodiment, the event when the psychological condition is satisfied is According to this control method, the robot 100's heart rate is This makes it easier to identify events that may have triggered a change in the robot's Among the many different events taking place around Robot 100, the one that touched Robot 100's heart You can check the situation by focusing on "Events". Among them, events that are important to the robot 100 (or its feelings) are registered in the reaction report. This allows the robot 100 to create a diary-like reaction report that focuses on its feelings.
[0100] At 9:30, the event where the owner is leaving (event icon 182b) and The event where the owner returns home (event icon 182c) occurs in rapid succession. When multiple events occur in succession within a short period of time, the history management unit 252 Reaction report with icon 182b and event icon 182c shifted slightly left and right By such a display method, the event on the first reaction report screen 180 is displayed. This can improve the visibility of the icon 182.
[0101] The history management unit 252 further manages a specific category (a specific event type) that is predefined. Attention marks 186 are set next to events that belong to the group. The highlighting of events that require attention among events of the same type. The 186 rule makes it easier to recognize important events. Important events include the owner Possible reasons for this include the sighting of someone leaving or seeing someone off, something falling, or the discovery of an unregistered person (suspicious person). can be.
[0102] It is also possible that multiple robots 100 are installed in one home. Each has its own personality due to differences in initial settings or environments. For example, There is also the "Robot 100 that is weak to loneliness" setting, which tends to increase the loneliness parameter, and One possible setting would be "Calm Robot 100," which has a setting that makes it difficult for the eccentricity parameter to increase. In addition, the robot 100A has a high degree of intimacy with the user (P01), while the robot 100B may have a lower affinity level. In this case, the user (P01) Being held greatly relieves the "loneliness" of robot 100A, but In this case, even if the user (P01) holds the baby, the loneliness may not be relieved. In addition, the robot 100C, which has never seen the user (P01), Being held may actually increase the curiosity parameter. For the common event of "being held by the user (P01)," each robot Therefore, the psychological conditions of each robot 100 are different. The timing at which you stand up, or in other words, the timing at which you are moved, is likely to be different.
[0103] The history information generated by each of the multiple robots 100 that have the same experience is substantially identical. However, the timing when the psychological condition of the robot 100A is satisfied and the timing when the psychological condition of the robot 100A is satisfied may be different. Since the timing at which the psychological condition of robot 100A and robot 100B is satisfied is different, For example, robot 100A may generate different response reports for the event (E1) in the reaction report, but robot 100B records event (E1) in the reaction report. The user may not record the robot 100A and the robot 100B in the By referring to the reaction report, the robot 100A reacts to the event (E1). However, robot 100B is not able to recognize that it is not interested in the event (E1). The user can compare the reaction reports from the multiple robots 100 to determine the You can enjoy the individuality and feelings of each of the 100 robots. By providing an opportunity for children to feel the individuality of the robot 100, it is possible to develop multiple robots 100. You can enjoy the relationship even more.
[0104] FIG. 12 is a screen view of the second reaction report screen 190. The reaction report includes the motion selected by the robot 100 when the psychological condition is satisfied. The second reaction report screen 190 in FIG. A reaction report with corresponding compounds is shown.
[0105] The motion ID is associated with a motion icon 192. The motion icon 192 may be formed as an animation that reproduces the motion of the robot 100. The history management unit 252 identifies the motion selected when the psychological condition is satisfied. The motion selection unit 254 selects a motion icon 192 corresponding to the identified motion. "The motion selected when the psychological condition is met" refers to the motion that is selected first after the psychological condition is met. The motion may be a motion selected during a predetermined period based on the establishment of the psychological condition. For example, when the psychological condition is satisfied at time t1, the motion The selected motion will be reacted to for 7 seconds from time (t1+0.5 seconds) to time (t1+7.5 seconds). You may also register for a report.
[0106] In the second reaction report screen 190 of FIG. 12, the robot's reaction when the psychological condition is met is displayed. When the 100 motion icons 192a representing the "actions" are displayed, the psychological conditions are met. Event icons 182a corresponding to the current event are also displayed. 182a corresponds to an event where the robot 100 meets its owner. The control 192a represents the movement of the robot. By doing so, "Robot 100 met its owner at 9:00 and started moving." You can see the daily life of the robot 100 and the behavior of the robot 100 at that time.
[0107] The user operates the user terminal 400 to display the first reaction report screen 180 and the second reaction report screen 190. The robot 100 can switch between two reaction report screens 190. The robot system 300 selects a motion according to the emotion parameter at the time. In this case, events, emotion parameters (feelings), and motions (actions) are all intricately related to each other. The behavior of the robot 100 is determined by the interaction between the robots. The behavior of the robot 100 is unpredictable. The user may select a first reaction report screen 180 and a second reaction report screen 190. By referring to the How will Robot 100 feel and what will Robot 100 do as a result of the event? It is possible to know what action was selected by the user. According to the second reaction report screen 190, the robot 100 was happy to meet the owner. It can be inferred that he approached the owner.
[0108] FIG. 13 is a diagram showing the data structure of the user authority table 290. As shown in FIG. The user authority table 290 is stored in the personal data storage unit 218 of the server 200 . In this embodiment, the amount of information provided by the reaction report is controlled according to the user. In this embodiment, the viewing attribute is set for each user. It indicates how much information can be extracted from the history information, in other words, the viewing authority level. The user is identified by a user ID. The user's physical characteristics and user ID are pre-assigned. The viewing attributes are set in advance for each user ID. The attribute A1 has the strongest viewing authority, and the attribute A4 has the weakest viewing authority. The reaction report provided to the user (P01:A1) is the same as the reaction report provided to the user (P04:A4). It provides more information than a port.
[0109] FIG. 14 is a data structure diagram of the event condition table 292. As shown in FIG. The event condition table 292 is stored in the event condition storage unit 260 of the server 200. The event condition table 292 defines the viewing conditions for each event condition. The event conditions are primarily defined to determine privacy-related events. There are conditions related to the user's appearance and voice, such as angry voice or crying voice, and conditions related to the location, such as a child's room or a study. Conditions for doing this, time-related conditions such as late at night or evening, angry faces, crying faces, laughing faces, etc. Event conditions are defined as conditions related to the facial expression of the user who viewed the event. The vent conditions are events that occur in open spaces such as the living room, kitchen, and entrance. You can also classify events that occur in private spaces such as children's rooms, studies, and bedrooms. stomach.
[0110] For example, when an event X1 satisfies an event condition (C1), the viewing attribute A1 Event X1 will only be registered in reaction reports for the above users. User (P01:A1) can check the event X1 in the reaction report, but user (P02 :A2) cannot see event X1 in the reaction report. For example, event X1 (Event of visual recognition of user) visually recognizes user (P06) in his / her room (study, etc.) When the event condition (C1) is satisfied, the user (P01:A1) 6) can confirm from the reaction report that he was in his room, but user (P02:A2) It is not possible to know this. Depending on the user's viewing attributes (viewing authority level), By controlling which events can and cannot be notified, , further strengthening privacy protection.
[0111] FIG. 15 is a sequence diagram showing the process of displaying a reaction report. A reaction report acquisition request is sent from the user terminal 400 to the server 200 (S20 At this time, the user terminal 400 also transmits a terminal ID. The history management unit 252 identifies the user ID corresponding to the last ID. 0 and check the user's browsing attributes (S22). The events that can be displayed on the port are narrowed down. Next, the history management unit 252 refers to the history information. The history management unit 252 checks the time when the psychological condition was established (S24). Generate a reaction report based on the event, emotion parameters, motion, etc. when the event is established ( S26). The icon selection unit 254 selects an icon as appropriate. The communication unit 204 is formed as a web page including the The reaction report is transmitted to the user terminal 400 (S28). The route is displayed on the screen (S30).
[0112] <Show detailed information> FIG. 16 is a sequence diagram showing the process of displaying details. The history management unit 252 acquires detailed information when an event occurs. Image (still image), short movie, text, audio information, temperature by thermo sensor Any information that indicates the content or status of the event, such as distribution information, is sufficient. At the very least, detailed information Any information other than the icon may be provided. For example, the recognition unit 21 of the server 200 2 is the image captured when the event X2 of detecting a user (P01) is recognized. The image Y2 is stored. The robot 100 transmits the captured image to the server 200 at any time. History The management unit 252 stores the event X2 and the captured image Y2 in association with each other. This is the "detailed information" about event X2. It includes the time when the user (P01) was detected. A detailed image may be stored for a predetermined period of time, for example, a short movie of about 5 seconds. Alternatively, the recognition unit 212 may recognize text information such as "The user (P01) was found at the entrance." The history management unit 252 may then store the text information as detailed information. Good too.
[0113] When generating a reaction report, the history management unit 252 records events and The icon ID of the event icon 182 is stored in association with the event ID. This is a number that uniquely identifies the event icon 182. When the user taps any of the event icons 182, the user terminal 400 The history management unit 2 transmits the icon ID corresponding to the icon 182 to the server 200 (S40). 52 identifies the event associated with the icon ID and The detailed information to be provided is identified (S42).
[0114] The emotion management unit 250 transmits the detailed information to the user terminal 400 via the communication unit 204 ( S44). The user terminal 400 displays the detailed information (S46).
[0115] FIG. 17 is a diagram showing the detailed information screen 402. When the event icon 182 is tapped, the user terminal 400 receives a The user terminal 400 then displays a detail window on the reaction report. Detailed information is displayed in the detail window 404. The information may be still images, video, or text. The information may be displayed simultaneously.
[0116] You can also set viewing permissions for detailed information. For example, if the event condition (C2) is met, Event X3, which is set up, is a response report for users with a browsing attribute of A2 or higher. Event icon 182 is displayed, but to see the detailed information, the viewing attribute A1 or higher is required. Alternatively, the event X4 that satisfies the event condition (C4) may be reflected. In order to be visible in the response report, the viewing attribute A2 or higher is required, but For 4, the display of detailed information may be prohibited. Detailed information is displayed to show the occurrence status of the event. As this is raw data showing specifics, it may be related to privacy. It seems that the display of detailed information should be controlled more strictly than that of Event Icon 182.
[0117] The detailed information may be a video without audio or a still image with some parts obscured. The history management unit 252 may also convert still images into images with a known image conversion method. The image is then transformed into a painting-like image or a blurred image, and detailed information is provided to the user terminal 400. The history management unit 252 may provide detailed information close to raw data to users with strong viewing authority. and the more highly edited detailed information is provided to a user with a weaker viewing authority.
[0118] <Display of atmosphere> FIG. 18 is a screen view of the third reaction report screen 410. In addition to events, emotions, and motions, the history management unit 252 also adds moods to the reaction reports. The atmosphere determination unit 256 can record the type (atmosphere information) of the external environment of the robot 100. The types of atmosphere are classified into several types. Specifically, they are "fun," "lively," "quiet," and "dark." Depending on the mood type, the icon selection section 254 displays the mood icon. The history management unit 252 selects the mood type when the psychological condition is satisfied. The corresponding mood icon 194 (image) is recorded in the reaction report.
[0119] The atmosphere determination unit 256 determines the atmosphere type based on the external information obtained from the robot 100. The robot 100 uses the internal sensor 128 to detect images, sounds, smells, and touch. The atmosphere determination unit 256 obtains information related to the atmosphere condition and transmits it to the server 200. The mood type is determined based on the following criteria: sound pressure, smile detection, loud laughter, and The success or failure of the atmospheric conditions is judged based on the number of touches, the number of users around the robot 100, etc. For example, if multiple voices and multiple smiling faces are detected within a given period, The atmosphere condition regarding "is this atmosphere lively?" is satisfied, and the atmosphere determination unit 256 determines that the atmosphere type is "lively." Also, when an angry voice is detected during a predetermined period, the mood determination unit 256 determines that the mood is The atmospheric type is determined to be "tingly."
[0120] In the third reaction report screen 410 of FIG. 18, the “atmosphere” when the psychological condition is met is displayed. The mood icon 194 representing the "type" of the robot 100 is displayed. The user can also view the third reaction report screen 410. By referring to the above, the atmosphere around the robot 100 and the robot 100 at that time can be For example, the user can see whether the robot was in a lively environment or not. The third reaction report screen 410 lets you know whether you had a quiet day. Similarly to the event, the viewing attribute may be set for the event type.
[0121] In this embodiment, the server 200 is described as being equipped with the atmosphere determination unit 256. As will be explained, the robot 100 may have an atmosphere determination function using the atmosphere determination unit 256. In this case, the atmosphere determination unit 256 of the robot 100 supports the atmosphere type (atmosphere information). The server 200 is notified, and the icon selection unit 254 selects an atmosphere icon 1 corresponding to the atmosphere type. 94, and the history management unit 252 generates a reaction report with the atmosphere icon 194 added. Just do that.
[0122] <Setting rest time> FIG. 19 is a screen diagram of the rest time setting screen 420. The user can check the reaction report not only on the user terminal 400 but also on the user terminal 400 can also issue instructions to the robot system 300. In this case, the user can set the rest time (sleep time) of the robot 100 on the user terminal 400. In the rest time setting screen 420 of FIG. 19, the rest time is set to 23:00 to 5:00. The user terminal 400 transmits the rest time designation information to the server 200. The operation control unit 222 of the server 200 controls the robot 100 to be fully engaged during the rest period. The vehicle is then placed on standby at a power station (not shown).
[0123] The rest time can be set in 10-minute increments for a continuous period of at least one hour. You may not want your pet to move around at night. Also, you may want to use your pet when you are not feeling well. It is thought that the user does not want the robot 100 to move around. By setting the above, the robot 100 can be prevented from disrupting the user's life.
[0124] <Summary> The robot system 300 has been described above based on the embodiment. According to this embodiment, the user can control the robot 100 even when the user is not near the robot 100. The user can check the daily life of the robot 100 through the user terminal 400. Write a reaction report like a picture diary about the experience, how you felt, and what actions you took. So you can know.
[0125] When the psychological condition is met, in other words, when the feelings of the robot 100 change. This effectively reduces the amount of information in the reaction report by generating a reaction report that includes only events that are related to the reaction. Not only can you control the robots, but you can also send diary-like reaction reports that emphasize the feelings (sensitivity) of the 100 robots. Narrowing down the information to be provided in the reaction report is a This not only increases the visibility of the robot 100, but also protects the privacy of users who are near the robot 100. It also contributes to protection.
[0126] By referring to the reaction report, the user can understand how the robot 100 is feeling. For example, if Robot 100 is lonely, you can find out how it feels. When a user is having a hard day, they will want to go home early to see Robot 100. Or maybe the father goes out and the mother and children spend time at home with the robot 100. Even when the child is not using the robot, the father can get an overview of the robot's activities through reaction reports. It is believed that it will be easier to talk about life with the Robot 100 after returning home. Users can learn about the lives of their elderly parents through reaction reports. (details below).
[0127] The reaction report does not directly display events, but rather uses various icons to indicate The icons are caricatures of the daily lives of 100 people. You can enjoy the reaction report like a picture diary. The icon display allows you to In addition to making it easier to understand, it also improves the aesthetics of reaction reports and protects privacy. It also contributes to protection.
[0128] Furthermore, the information that can be provided to each user can be precisely controlled based on the viewing attributes. For example, parents can see many of the events in the home, but grandparents can only see a few of the events in the home. The history management unit 252 may be configured to check only a part of the information from an unregistered user terminal 400. The request to obtain a reaction report from such a person may be rejected.
[0129] By checking the detailed information, the user can check the robot 100 even when he or she is not near the robot. For example, you can tell the robot 100 what is happening around it while you are away from home. When there is a "great event" on the app, the details of the event will be displayed, so you can feel more comfortable going out. The user in question can easily share "happy events" with users nearby the robot 100. Nowadays, it is common to share still images and videos with close friends on social networking services (SNS). However, according to the present embodiment, the robot 100 transmits the information to the robot 100. This makes it possible to convey the detailed information you want to share to the people you want to reach.
[0130] In addition, the reaction report includes an atmosphere icon 194 that indicates the atmosphere around the robot 100. Users who are out can learn about the atmosphere at home through reaction reports. Users who are out and about may want to go home if the atmosphere at home looks fun. The father, who is at work, can check the reaction report on his smartphone (user terminal 400). This allows you to get a sense of the "fun" atmosphere of a home, even if you don't know the details of what goes on inside the home. If he could, he would be able to work with peace of mind. Or, when he gets home, You can ask your family if they have had any fun. It is thought that this will serve as a tool to gently connect fathers and their families by allowing them to share information about their families.
[0131] The user can set the rest time of the robot 100 through the user terminal 400. By setting the above, for example, the robot 100 can move around early in the morning to prevent the user from disturbing his / her sleep. Or, you can avoid getting sick because you have to get up early tomorrow. There may be cases where you want the robot 100 to move and wake you up. Although it is simple to set it up from the user terminal 400, it is difficult to match the user's life pattern with the robot 1. In other words, to coordinate the life patterns of the user and the robot 100 This is thought to be a very effective means of achieving this.
[0132] Conventionally, robots that select actions based on emotion parameters are known. The purpose of setting emotion parameters for a robot is to ensure that the robot acts according to its own emotions. The purpose of this is to realize biological behavioral expression such as emotion. It is thought that these are often treated as internal parameters that are not visible from the In contrast, in this embodiment, the user reports the emotion of the robot 100 in a reaction report. The user can know how the robot 100 feels as it goes about its daily life. By knowing the feelings of the robot, the person will be able to consider the feelings of the robot. By considering each other, the robot 100 and the user can have a psychological interaction with each other. You can go even deeper.
[0133] The present invention is not limited to the above-described embodiment and modifications, and may be modified without departing from the spirit and scope of the present invention. The components disclosed in the above embodiment and modified examples can be embodied in the following manner. Various inventions may be formed by appropriately combining a plurality of the above components. Some of the components shown in the above embodiment and modified examples may be deleted.
[0134] One robot 100 and one server 200, and multiple external sensors 114 control the robot Although the system 300 has been described as being configured, part of the functions of the robot 100 is implemented by the server 2. 00, or a part or all of the functions of the server 200 may be realized by the robot 100 One server 200 may control multiple robots 100. Alternatively, multiple servers 200 may cooperate to control one or more robots 100. Good too.
[0135] A third device other than the robot 100 and the server 200 may take on part of the functions. The functions of the robot 100 and the server 200 described above are generally It is also possible to think of it as one "robot". On the other hand, how to allocate the multiple functions required to realize the present invention is up to each hardware. It should be determined in consideration of the processing capacity of the software and the specifications required for the robot system 300. stomach.
[0136] As mentioned above, the term "robot in the narrow sense" refers to the robot 10 that does not include the server 200. 0, while "robot in the broad sense" refers to robot system 300. It is considered that many of the functions of the server 200 may be integrated into the robot 100 in the future. can be done.
[0137] [Variations] <Rejection of reaction report> The history management unit 252 collects reaction records from the user terminal 400 located near the robot 100. For example, the history management unit 252 may reject the port acquisition request. When the image unit 158 visually recognizes the user (P01), The reaction report acquisition request from the user terminal 400 may be rejected. When the user (P01) is recognized from the captured image, the history management unit 252 A terminal ID corresponding to the terminal number is identified, and a response from the user terminal 400 having the terminal ID is received. The report acquisition request is rejected. This control is performed by the user (P01) of the robot. In order to prevent the user from checking changes in the reaction report on the user terminal 400 while interacting with the net 100, If the user (P01) hits or hugs the robot 100, You can check the emotional changes of the robot 100 in real time while it is performing an action. If that were to happen, Robot 100 would be treated as an experimental animal. In order to prevent the robot 100 from being treated like an experimental animal, the history management unit 252 visually records the robot 100. In other words, a user who is not within the range of the robot 100 and cannot directly affect the robot 100. A reaction report may be provided only to users who are too far away to be able to communicate with the user.
[0138] A user (P01) who is the owner of the user terminal 400 (P01) It is also possible to obtain the reaction report not from the user terminal 400 (P01) but from another user terminal 400 (P02). In this case, the robot 100 is the true owner of the user terminal 400 (P02). If a certain user (P02) is not nearby, a reaction report is sent to the user terminal 400 (P02). If the user sends a response report, it may be seen by a nearby user (P01). It is also possible that
[0139] As a countermeasure against such a loophole, first, the application installed in the user terminal 400 (P02) The application software uses the camera function to capture the reaction report. P01), and when a reaction report acquisition request is made, the captured image Q1 is sent to the server 20. In the above example, the captured image Q1 contains the face image of the user (P01). Included.
[0140] When the server 200 receives a reaction report acquisition request, the history management unit 252 00 also acquires a photographed image Q2. In the above example, the photographed image Q2 also has the user (P0 The recognition unit 212 recognizes the face image Q1 (user terminal 400) contained in the captured image Q1. Whether the face image of the same person as the face image in the photograph Q2 (robot 100) is also included in the photographed image Q2 When the same person is photographed, the robot 100 detects the reflection from the user near the robot 100. Since a response report is requested, the history management unit 252 sends the response report to the user terminal 400 (P 02) Refuse to provide a reaction report to the same person, provided that the same person is not photographed. The history management unit 252 may provide a reaction report. The reaction report acquisition request from the user terminal 400 located near the host 100 is more accurate. You can actually refuse.
[0141] In addition, the server 200 is equipped with a distance measuring unit (not shown) and measures the distance to the user terminal 400. The history management unit 252 may measure the distance by a known method such as infrared communication. Alternatively, the reaction report acquisition request from the user terminal 400 may be rejected. The server 200 receives the position coordinates of the robot 100 and the position coordinates of the user terminal 400 from a GPS (Global Positioning System). The robot 100 may detect the user by a known means such as a GPS positioning system. If the user terminal 400 is located at a distance greater than a predetermined distance, The system may also respond to requests for reaction reports from these users.
[0142] When the robot 100 is visually acknowledging the user (P01), When the reaction report acquisition request is received from the terminal 400, the robot 100 01) A specified period based on the time when it was first seen, for example, from the time when it was first seen A reaction report for only one minute may be provided. The user (P01) interacts with the robot 100 and learns the behavior of the robot 100 depending on how the user interacts with it. Opportunities for identifying emotional changes through reaction reports may be limited.
[0143] Conversely, the reaction report can be shown only to users at home or near the robot 100. According to this control method, a reaction report can be sent to a user at a remote location. The user at home can prevent the robot 100 or the server 2 from seeing the contents. 00 may be set to reject remote requests to obtain reaction reports.
[0144] When a reaction report acquisition request is received from the user (P05), the server 200 checks the authority The section (not shown) grants viewing permission to a pre-set administrative user (P07), The authority confirmation unit may check whether the reaction report can be provided. For the terminal 400, information about the user (P05) (user ID, name, affiliation, etc.) The authority confirmation unit sends the permission command to the administrator (P07). Then, the history management unit 252 may transmit a reaction report to the user (P05).
[0145] In addition to whether or not reaction reports can be provided, the period covered by the reaction reports may be restricted. For example, assume that a user (P01) sends a reaction report acquisition request to the server 200. At this time, the history management unit 252 detects that the user (P01) was not near the robot 100. A response report covering the time period (absence period) may be provided to the user (P01). For example, if the user (P01) leaves the robot 100 from time t6 to time t7, Assume that the user (P01) is in the location and is near the robot 100 from time t8 onwards. When the user terminal 400 transmits a reaction report acquisition request to the server 200 after time t8, In this case, the history management unit 252 performs a response record for the absence period from time t7 to time t8. On the other hand, the target period is from time t8 onwards. According to this control method, the user who has returned home from outside does not receive a reaction report. Even so, I can look back on how my family was when I was not there, and You can get a glimpse of the life of Robot 100, but if you are close to Robot 100, The reaction report cannot be seen when the user is far enough away from the robot 100. When the user is notified of the current time, the history management unit 252 sends a reaction report including the current time to the user. In addition, a real-time reaction report may be provided.
[0146] The history management unit 252 determines that the period during which the robot 100 is resting (charging) is the subject of the reaction report. For example, the robot 100 is resting from time t9 to time t10. In this case, the history management unit 252 generates a response report for the time period from time t9 to time t10. The robot 100 may not generate a report on its life during rest. Not only is this irrational, but the psychological conditions do not exist during rest, so the reaction report This is because it is considered that the robot 100 cannot provide meaningful information. Even if the user is resting, the server 200 can provide the user with a reaction report from before the rest. It is also possible to use the following.
[0147] The history management unit 252 records the emotions of the robot 100 in real time in the reaction report. For example, when a user requests a reaction report, The reaction report will only include emotions from a certain time before, for example, 10 minutes before. The committee may also provide information on the
[0148] <Record of movement history> The history management unit 252 can manage not only the time and event, but also the location where the event occurred or the robot. The movement history of the 100 may be recorded in the history information and the reaction report. 08, when the robot 100 moves, based on the captured image obtained by the imaging unit 158 The history management unit 252 may create a map of the location of the robot 100. It may be provided as.
[0149] <Mode Record> The mode selection unit 258 selects the operation mode of the robot 100. When the amount of activity of the robot 100 per unit time becomes equal to or greater than a predetermined value, the robot The operation mode of 100 is changed from normal mode to power saving mode. The amount of activity is The power consumption can be determined based on the number of motion selections or the power consumption of the device. In this case, the types of motions that the robot 100 can select are narrowed down. By limiting the range of motion of the Eta and lengthening the interval, the movement of the robot 100 In the power saving mode, the activity of the robot 100 per unit time may be slowed down. 258 suppresses the amount of power consumed when the power saving mode continues for a predetermined period of time or more. Change to normal mode.
[0150] The mode selection unit 258 is configured to select a mode when the charging rate of the secondary battery mounted on the robot 100 is equal to or lower than a predetermined threshold. When this happens, the operation mode is changed to charging mode (rest mode). The charging unit 222 places the robot 100 in the charging station. At this time, the mode selection unit 258 changes the operation mode from the charging mode to the normal mode.
[0151] The robot system 300 may further have a shooting mode and a shooting non-shooting mode. The horn 112 of the robot 100 may be equipped with a rotary switch (not shown). The user can turn the switch on the horn 112 to switch between the shooting mode and The shooting-disabled mode can be manually switched. In the shooting-enabled mode, the imaging unit 1 58 can capture images of the outside world and store the captured images as part of the history information. In the enable mode, the robot 100 takes pictures for user recognition, etc., but does not take pictures. Alternatively, the robot 100 may store the captured image for a predetermined period of time, for example, 3 The user may discard the image automatically after 3 seconds. When you don't want to take a picture, you can simply rotate the switch on the horn 112 to turn it into the non-photo mode. can be changed to.
[0152] When the switch provided on the horn 112 is operated, the communication unit 142 services the set value. The mode selection unit 258 transmits the setting value received from the robot 100 to the robot 200. It is possible to recognize whether the camera is in a shooting mode or a shooting-disabled mode. In this case, the recognition unit 212 of the server 200 receives the photographed image from the robot 100 and recognizes the image. It performs image recognition processing but does not permanently store the captured images.
[0153] The history management unit 252 records the mode of the robot 100 in the history information and the reaction report. The user may refer to the reaction report to determine the operation mode of the robot 100. You can check the history of changes in the settings. Note that during periods when photography is not possible, no images will be saved. Therefore, detailed information in the form of images cannot be provided.
[0154] <Record of routine activities> The recognition unit 212 may detect a regular behavior of the user. Exercises, saying "good morning" and "good night" to the robot 100 (greetings), The user's daily routines, such as brushing, washing, bathing, sitting in the study, cleaning, eating, and taking a nap, The regular behavior may be registered in advance in the server 200, or the recognition unit 212 may The system observes users over a specified period of time, and when the same behavior is detected repeatedly in the same time period, The user may register the behavior as a regular behavior. The regular behavior is recognized for each user. Recognition unit 2 When 12 detects a regular behavior, it recognizes that a “regular event” has occurred.
[0155] When a regular event occurs, the history management unit 252 records it in the history information. 252 may record an event icon 182 corresponding to a regular event in the reaction report. Alternatively, rather than recording routine events in the reaction report, If the substance is not detected during the regular time period, this fact may be recorded in the reaction report. According to the control method, the user C at a remote location can know the behavior of the robot 100 through a reaction report. It is possible to check whether user D in the area is performing regular actions.
[0156] Currently, the number of single-person households in Japan is said to be over 13 million, and this number is sure to continue to increase in the future. In particular, the number of elderly (over 65 years old) single-person households has reached 6.2 million. Patrolling (checking safety) is an important issue. Currently, children and grandchildren living alone in the neighborhood This can involve regular visits to the parents who are caring for them, or regular phone calls, emails, or social media. By contacting them, they may also check on the situation of parents who live alone.
[0157] However, such safety checks are not only unpleasant for the checkers, but also for those being checked. Children may be concerned about the lives of their elderly parents who live alone in distant places. However, checking social media every day is a big burden. I often refrain from contacting them out of consideration for this reason.
[0158] If single elderly people (elderly parents) were to live with 100 robots, their children would live with 1 robot. It is thought that it will be easier to check the safety of elderly parents through 00. However, the privacy of elderly parents is also a concern. From the viewpoint of protection, the robot 100 will monitor the daily life of the elderly parent and other users (children, etc.) If the privacy of the person being watched is not taken into consideration, "R" becomes the same as "surveillance."
[0159] The history management unit 252 in this embodiment is configured to record the preferences of users who are near the robot 100. Although the system generates a response report that takes into account the elderly parent's behavior, the child does not know the details of the elderly parent's daily life. Elderly parents do not want their children to see all of their daily lives. It is believed that children are satisfied as long as they can see that their elderly parents are living safely as usual. It may be minutes.
[0160] One solution to this problem is to make the report less appealing to readers (children). It is possible to set viewing authority. With this control method, viewers (children) This makes it easier to protect the privacy of elderly parents by limiting the information provided to them.
[0161] Another solution is to record the success or failure of regular events in a reaction report and share it with the viewers. For example, an elderly parent may say "Good morning" to the robot 100. The viewer will see the regular activities (morning and evening) of an elderly parent living with the robot 100. In this case, the reaction report is a regular event. The viewer may be informed of the success or failure of the event. The elderly parent may say "Good morning" to the robot 100. By simply calling, you can let children living far away know that "Today is the same as always. Children can also send regular messages via the user terminal 400. By knowing whether or not an event is taking place, you can easily check that your elderly parents are okay. For those receiving care, simply living their lives as usual can reassure their children. So it's not a burden.
[0162] The viewer (child) can only call if he / she cannot confirm the scheduled event in the response report. You can simply check on the safety of your elderly parents using existing means of communication such as: There is no need to notify the success or failure of a regular event by a reaction report. When a regular event is detected, a known means of contact, such as a regular event (usually The user watching over the user may be notified that a certain action (a different action) has occurred.
[0163] In addition, the above-mentioned viewing restrictions may not be imposed on regular events. For example, when the user says "good night" to the resting robot 100, However, the robot 100 may provide a reaction report that a routine event has occurred.
[0164] <Management of history information> In this embodiment, the history management unit 252 records all events in the history information. He explained that history information may contain a wealth of privacy-related information. Therefore, the history management unit 252 may limit the storage period of the history information. The history management unit 252 stores only the old history information, and the history information older than one day is stored as history information. Alternatively, it may be removed entirely from section 262.
[0165] <Prohibited attributes> You may set a prohibition attribute for some event conditions. The test may not be recorded in the history information, may not be recorded in the reaction report, Detailed information may not be provided. Depending on the level of prohibition, multiple types of prohibition attributes may be provided. For example, an event that satisfies the event condition of the prohibition attribute B1 may be a history The event that satisfies the event condition of the prohibition condition B2 may not even be recorded. The prohibition condition B3 may be recorded in the history information but not provided as detailed information. Events may be recorded in the history information but may be erased after a short period of time, such as 30 minutes.
[0166] Examples of event conditions that can be set as prohibited attributes include crying and yelling (such as during a marital dispute). Recognition), images of a laughing face, images of a sleeping face, etc. are possible.
[0167] <Viewing attributes> In the present embodiment, the viewing authority level is expressed by the viewing attribute. Viewing attributes are not limited to privacy rights. For example, if it occurs in the morning, Events that can only be confirmed in the reaction report when they occur on a rainy day. The system determines whether or not an event can be registered in the reaction report based on external conditions such as events that can be recognized. This is also fine.
[0168] In this embodiment, a viewing attribute is set for a user, and privacy protection is performed based on the viewing attribute. In addition, the level of the signal can be controlled by the user terminal 40 at a remote location. For reaction report requests from 0, the event viewing authority level may be lowered. For example, detailed reaction reports may not be provided to users who are far away, or It may be possible not to provide detailed information to users who are located beyond the distance. In this way, the privacy level may be controlled based on the distance.
[0169] The privacy level may also be controlled based on time. For example, the reaction rate during the day may be It provides detailed response reports to users who send port acquisition requests, but For users who have sent a reaction report request to It can be modest.
[0170] The robot 100 is alone, in other words, is at home alone with no one around. Sometimes, the history manager 252 may provide all events in a reaction report. When the robot 100 is in the same room as other users, the reaction report can provide information. According to this control method, the information about the robot 100 being near the robot 100 can be suppressed. While protecting the user's privacy, when the robot 100 is left alone at home, You can communicate the latest situation in a reaction report.
[0171] When the robot 100 detects a suspicious person, in other words, a person it has never seen before, Then, the history management unit 252 cancels privacy protection and transmits all history information to the user terminal 40. 0. According to this control method, only when a suspicious person breaks into a house, Therefore, the robot 100 can function as a monitoring device. When the user (owner) and an unregistered person are recognized at the same time, the unregistered person is a visitor. At this time, the history management unit 252 will notify the user of the reaction record in order to protect privacy. The information provided on the port may be suppressed.
[0172] <Emphasis> In this embodiment, the history management unit 252 uses an attention mark In addition, the history management unit 252 also assigns a special number to each of the events that are of interest. By making the event icon 182 of the event blink, changing the color, changing the size, etc. It is acceptable to use emphasis expressions such as speech bubbles to draw attention to important events. good.
[0173] In the first reaction report screen 180, etc., a red line (first line) and a blue line It is also possible to display multiple types of lines, such as the red line and the in line. In the game, the time when a noteworthy event occurs is displayed as the attention mark 186, and the blue Lines may be used to indicate the occurrence of all events. Blue lines indicate when psychological conditions are met. Regardless of the event at the time of the event, all events registered in the history information are The points may be displayed.
[0174] <Icon display> In this embodiment, the amount of information in the detailed information is changed according to the browsing attribute. In the same manner, the display of icons such as the event icon 182 is changed according to the viewing attribute. For example, for the event "A customer has come," Event icon 182A uses a photo of the customer's face, and event icon 182B uses a portrait of the customer. Con 182B, an event icon that uses pictures of faces to represent customers without identifying them. Con 182C, an event that does not even identify visitors, such as exclamation marks Have Icon 182D ready.
[0175] The history management unit 252 does not include events in the reaction report for users with a browsing attribute of A1 or more. For users with a viewing attribute A2, an event icon 182B is used, and for users with a viewing attribute A3, an event icon 182C is used. Event icon 182C is displayed for users with the viewing attribute A3, and event icon 182C is displayed for users with the viewing attribute A4. According to this control method, the user can view the By controlling the amount of information provided by each icon according to the attribute, privacy can be improved. This will make it easier to protect the sea.
[0176] <Instructions to Robot 100> In this embodiment, a rest time is set for the robot 100 from the user terminal 400. The method for issuing various commands, not limited to rest time, from the user terminal 400 has been described. For example, from 8:00 to 9:00, the living room I want you to stay in your room, I don't want you to go to the kids' room, I want you to come to my bedroom at 7:00, etc. Any command may be sent from the user terminal 400 to the server 200. The control unit 222 controls the operation of the robot 100 according to the command. The father must check on the child once in the afternoon and once in the afternoon. The father must return home between 22:00 and 23:30. So it may be a good idea to pay attention to noises at the entrance (by increasing the microphone's sound pickup sensitivity).
[0177] However, if the user terminal 400 controls the robot 100 too much, Since the robot 100 will lose its autonomy, the commands that can be issued from the user terminal 400 to the robot 100 will be It is desirable to limit the number of users to a certain extent. The action control unit 222 may randomly select whether to follow the command or not. The higher the user's command, the higher the probability that it will be accepted.
[0178] The user can talk to the robot 100 from the user terminal 400 even when he / she is away from home. The user may read out a message to the robot 100 from the user terminal 400. The server 200 receives the voice message from the user terminal 400 and plays it back. The PC 100 recognizes voice messages through a microphone array (internal sensor 128). The emotion management unit 25 recognizes that an event "a user calls out to me" has occurred. 0 may cause this audio event to lower the loneliness parameter. According to the method, a user speaks to the robot 100 via a user terminal 400 from outside the home. By doing so, the robot 100 can ease its loneliness. Even when traveling, the user can talk to the robot 100 from the user terminal 400. You can show consideration by saying:
[0179] <Other> The reaction report shows emotion parameters instead of emotion icons 184 corresponding to emotion parameters. The history management unit 252 may display the emotion parameter value or the The history management unit 2 may record a color corresponding to the value of the emotion parameter in the reaction report. 52 records the history of changes in the emotional parameters in a reaction report regardless of whether the psychological conditions are met. This is also fine.
[0180] The event may be represented by an event icon 182 or as text information. You may also include a photo of the event in the response report as explained in the More Information section. It may be attached.
[0181] In this embodiment, the server 200 transmits a reaction report to the user terminal 400, The reaction report is displayed on the screen of the user terminal 400. Alternatively, a printer may be installed or connected to the server 200, and a reaction report may be printed out from the printer. Alternatively, the robot 100 or the robot system 300 may be equipped with a monitor. Alternatively, a monitor may be connected to the system and the reaction report may be displayed on the screen.
[0182] In this embodiment, events, motions, atmospheres, etc., when psychological conditions are met are It has been described that the results are displayed in a reaction report. As a variation, the results may be displayed based on the success or failure of psychological conditions. In addition, the history management unit 252 manages events related to a predetermined event condition, important events, and regular events. All events that match the user's browsing attributes, in other words, psychological conditions All events that meet the specified registration conditions other than the above may be registered in the reaction report. The history management unit 252 is configured to manage motions with a low selection probability (actions that are rarely performed) and actions that are not performed for a predetermined period of time. In the case of events whose detection frequency is less than a certain number (events that occur rarely), Regardless, the reaction may be registered in the reaction report.
[0183] The history management unit 252 selects one representative event per predetermined time, for example, per hour. You can also set the importance of each event type. The history management unit 252 manages one or more events that occur in a unit period as representative events. An event that corresponds to the most important event type may be selected. Alternatively, the history management unit 252 randomly selects a representative event from one or more events that occurred during a unit period. Venting may be selected.
[0184] The various icons may be color coded according to category. For example, Events related to discoveries have a blue event icon 182, while discovery events have a red event icon 182, and events related to sound or light may be represented by yellow event icons 182. Events may be color coded according to event type, or may be color coded according to the occurrence of certain event conditions. The event icon 182 may be color coded depending on whether or not the event has occurred.
[0185] In this embodiment, the feelings of the robot 100, events, actions of the robot 100, The atmosphere and other information are registered in the reaction report. The user may display only the feelings (emotion parameters) of the robot 100 in the reaction report. Even with such a reaction report, the user can easily see how the robot 100 spends its day. If only the emotions of the robot 100 are registered in the reaction report, For the same reason, the privacy of users can be protected almost completely. Alternatively, only the atmosphere type may be displayed, or only the event may be displayed.
[0186] In this embodiment, the atmosphere is expressed by the atmosphere icon 194. The atmosphere can be expressed by text information such as "fun" or "lively" or by color. Alternatively, the atmosphere may be expressed by a numerical value or a graph showing the goodness of the atmosphere. It may also represent a change in
[0187] The history management unit 252 may record the amount of activity of the robot 100 in the history information. The history management unit 252 may record the amount of activity of the robot 100 in a reaction report. According to this control method, the user can report whether the robot 100 was active that day. You can find out by clicking on the link.
[0188] The user may notify the robot 100 of events that are scheduled to occur in the near future. For example, the user terminal 400 may transmit information about a visitor, a party, or other events to the server 20. The emotion management unit 250 changes the emotion parameters as the scheduled time approaches. According to such a control method, the robot 100 can be made to have expectations based on future predictions. The robot 100 can also predict the future and create a sense of anxiety. In order to change the robot's behavior, the user can express to the robot 100 what expectations or anxieties the robot has. You can sense it from their actions.
[0189] In the present embodiment, the event is assumed to be an external event such as seeing the user or being held. However, the event may be an internal event such as the temperature inside the body 104 being equal to or higher than a predetermined value or the loneliness parameter being equal to or lower than 20. may also be defined as an event.
[0190] The robot 100 may have a plurality of types of emotion parameters, or may have a single emotion parameter. As described above, the behavioral characteristic may be changed based on the value of the first emotion parameter. When the first emotion value is equal to or greater than the second emotion value, the psychological condition Alternatively, the difference between the first emotion value and the second emotion value may be equal to or greater than a predetermined threshold value. It may be said that the psychological condition is met when this happens.
[0191] When the emotion value of a certain emotion parameter becomes equal to or exceeds a first threshold value, or when the emotion value becomes equal to or exceeds a second threshold value, The psychological condition may be satisfied when the emotional value is below the threshold. When the rate of change reaches or exceeds the third threshold, or falls below the fourth threshold, the psychological condition For example, the curiosity parameter may be set to be the curiosity parameter that increases suddenly when a person discovers something novel. While the need for approval parameter increases, it may change only slowly. In addition, the way of change may be given individuality according to the type of emotion parameter. Depending on the characteristics of the emotional parameters, various psychological conditions can be defined.
[0192] When providing detailed information as a short movie, it is important to match it to the period when the psychological conditions are established. The start and end of the short movie may be determined. For example, if the first psychological condition is satisfied at time t5, At time t6, the first psychological condition is satisfied, and at time t7, the first psychological condition is not satisfied. is a moving image from time t5 to time t6 among the moving images obtained by the camera of the robot 100. A short movie may be generated by cutting out the images.
[0193] The reaction report may record an abnormality that occurred in the robot 100. For example, The processor 122 is overheating, has been stopped in an emergency, and the actuator is not working. abnormal information such as the battery not working properly or the battery 118 (secondary battery) becoming too hot The reaction may be reported to the user terminal 400 via a reaction report.
[0194] Multiple robots 100 may share history information or reaction reports. Assume that (P01), robot 100A, and robot 100B exist in the same household. At time t12, a user (P01) holds the robot 100A in his arms. Assume that the psychological condition is met in 00A. The "hug" at this time is called an event (Ex). The communication unit 142 of the robot 100A transmits a command to the robot 100B including the time t12. Requests acquisition of images (still images or video images) taken for a predetermined period of time. The reception unit 142 selects the captured images during the relevant period from the captured images stored in the server 200. After downloading the image Q3 from the server 200 to the robot 100B, the robot 100A The robot 100A transmits the captured image Q3 to the event (Ex). According to this control method, the robot 100A The robot captures an image of itself being held from the viewpoint of another robot 100B. The robot 100B can obtain the celestial image for the corresponding period and provide it as detailed information. In this case, the robot 100A may transmit a celestial image ( A two-dimensional captured image Q3 may be generated from the three-dimensional image Q1 by a known image processing method.
[0195] The robot 100A sends a request to the server 200, not to the robot 100B, at time t12. The server 200 may request the robot 100B to acquire a captured image. When both the robot 100A and the robot 100B are managed together, the history of the server 200 The history management unit 252 stores the captured image Q3 by the robot 100B as detailed information about the robot 100A. You can save a copy as .
[0196] The robot 100A specifies the start and end points of a video to the robot 100B. For example, the robot 1 may capture a captured image Q3 as a short movie. The communication unit 142 of 00A starts 5 seconds before time t12 and ends 5 seconds after time t12. By specifying the end of the event as the end point, the robot can capture 10 seconds of video before and after the event (Ex). It may be obtained from 100B.
[0197] When the robot 100A executes a certain motion, the robot 10 For example, the communication unit 142 of the robot 100A may transmit a motion (M When executing x), a request for photographing is sent to the robot 100B before the execution. When the robot 100B (the motion control unit 150) receives a request to take a picture, the robot 100B moves to the side of the robot 100A. The robot 100A (the recognition unit 156) recognizes that the robot 100B is nearby. According to this control method, the robot 10 The state of robot 100A performing a motion (Mx) is captured from the viewpoint of robot 100B. The captured image at this time is provided as a detailed image of the robot 100A. An image of the motion executed by the robot 100A taken by the robot 100A itself may be used. So, it may be difficult to understand how the robot 100A is moving. With the mobile phone 100B, it is possible to provide the user with a good-angle shot image. If the motion (Mx) is a rare motion that is rarely executed, it will be executed by robot B. The captured images are valuable. By having multiple robots 100 capture each other's images, It is possible to acquire a variety of captured images (detailed images) that cannot be obtained by a single robot 100 alone. [Explanation of symbols]
[0198] 100 robot, 102 front wheel, 102a left wheel, 102b right wheel, 103 rear wheel, 104 body, 106 hands, 108 seating surface, 110 eyes, 112 horns, 114 exterior Internal sensor, 118 battery, 120 drive mechanism, 122 processor, 124 memory Device, 126, communication device, 128, internal sensor, 130, power line, 132, signal line, 134 Wire, 136 data processing unit, 142 communication unit, 148 data storage unit, 150 movement A motion control unit, 156, a recognition unit, 158, an imaging unit, 160, a motion storage unit, 170, Parameter of, 172 Curiosity parameter, 174 Desire for approval parameter, 180 First Reaction report screen, 182 Event icons, 184 Emotion icons, 186 Featured , 190 2nd reaction report screen, 192 Motion icon, 194 Atmosphere icon Icon, 200 server, 202 data processing unit, 204 communication unit, 206 data storage unit, 208 location management unit, 212 recognition unit, 218 personal data storage unit, 220 intimacy Management unit, 222, motion control unit, 232, motion storage unit, 244, state management unit, 250 Emotion management unit, 252 history management unit, 254 icon selection unit, 256 mood determination unit, 258 mode selection unit, 260 event condition storage unit, 262 history information storage unit, 270 History information, 280 Motion selection table, 290 User authority table, 292 Event condition table, 300 Robot system, 308 Base frame, 310 Body frame, 312 Wheel cover, 314 Skin, 316 Head frame, 318 Fuselage frame, 320 yaw axis, 322 pitch axis, 324 roll axis, 325 pre 326 Actuator, 327 Vent, 328 Base plate, 329 Clamp Ross link mechanism, 330 joint, 332 upper plate, 334 lower plate 336 side plate, 342 control circuit, 370 wheel drive mechanism, 378 rotating Axis, 379 Actuator, 390 Opening, 400 User terminal, 402 More information screen, 404 detail window, 410 third reaction report screen, 420 rest time setting screen
Claims
1. A motion control unit that selects the motion of the robot; A drive mechanism that executes the motion selected by the motion control unit; A recognition unit that recognizes the occurrence of an event; an emotion management unit that changes one or more emotion parameters in response to an event; Record the event occurrence history and the emotion parameter change history as historical information. A history management unit, The action control unit changes a behavior characteristic of the robot based on the emotion parameter, The history management unit records events when emotion parameters satisfy a predetermined psychological condition and and extracting emotion parameters at the time when the psychological condition is established from the history information to produce a reaction report. and providing the reaction report to a user.
2. The emotion management unit is configured to:
2. The autonomous behavior type robot according to claim 1, wherein the psychological condition is determined to be satisfied when the robot.
3. The history management unit further stores a model selected by the operation control unit after the psychological condition is satisfied. and providing the reaction report to a user after recording the reaction information in the reaction report.
3. The autonomous robot according to claim 1 or 2,
4. An imaging unit that images the surroundings of the robot; A communication unit that executes communication processing with the communication terminal, The recognition unit identifies a person from a captured image, The history management unit receives a request to acquire the reaction report from the communication terminal of a first user. When the first user is identified in the captured image, the communication and instructing a reception unit to transmit the reaction report to the communication terminal of the first user.
2. The autonomously acting robot according to claim 1 .
5. An icon selection unit that selects an icon according to an event, The history management unit records an event when the psychological condition is satisfied as a corresponding event. By recording the event in the reaction report as an icon, The autonomous operating system according to claim 1, further comprising an icon representing the occurrence history of the event. Moving robot.
6. The history management unit is configured to store the items of events that belong to a specific category among a plurality of categories. The reaction report further includes adding an emphasis to the con.
5. An autonomous robot according to claim 5.
7. The history management unit records detailed information of the event and one or more events recorded in the reaction report. When any of the above icons is selected by the user, the icon corresponding to the selected icon is 7. The method according to claim 5, wherein the detailed information of the event is provided to the user. An autonomous robot.
8. The history management unit identifies a browsing attribute that is associated with a user in advance, and Among the events when the condition is met, only the events that match the specified browsing attribute are displayed. The autonomous behavior according to claim 1, characterized in that the reaction is recorded in a reaction report to be provided to the user. Moving robot.
9. The history management unit is configured to manage events that have not been notified to the user in advance among events when the psychological condition is satisfied. Only events that meet the viewing authority level set for the user should be provided to the user.
2. The autonomous robot according to claim 1, characterized in that the autonomous robot records the information in a memory.
10. Based on external information, the success or failure of each of multiple predetermined atmospheric conditions is judged. An atmosphere determination unit for determining whether or not the temperature is within the predetermined range, The history management unit further, when any one of the atmosphere conditions is satisfied, detects the satisfied atmosphere. and recording atmospheric information corresponding to the conditions in the reaction report.
1. An autonomous robot as described in claim 1.
11. A mode selection unit for selecting a mode of the robot, The history management unit further records the history of changes in the mode of the robot in the reaction report.
2. The autonomous robot according to claim 1,
12. The recognition unit recognizes a regular behavior of a user as a regular event, The history management unit further records the success or failure of the regular event in the reaction report.
2. The autonomous robot according to claim 1,
13. A communication unit that executes communication processing with the communication terminal, The operation control unit, when an instruction to set a rest period is received from the communication terminal, 2. The method according to claim 1, wherein the robot is placed on an external charger during a period of time when the battery is being charged. An autonomous robot.
14. The ability to select robot motions, performing the selected motion; A function for recognizing the occurrence of an event; The ability to change one or more emotion parameters depending on the event, Record the event occurrence history and the emotion parameter change history as historical information. Functions and A function for changing the behavioral characteristics of the robot based on the emotion parameters; An event when the emotion parameter satisfies a predetermined psychological condition and the satisfaction of the psychological condition Extract emotion parameters at the time of standing from the history information to generate a reaction report and provide the user with and a function of providing the reaction report. A behavior control program for a mobile robot.
Citation Information
Patent Citations
Robot device and its controlling method
JP2001191283A
Device and method for providing information
JP2001222626A
Robot device, information display system and method, robot system, and recording medium
JP2002192485A
Robot system and control method for robot device
JP2003205483A
Remote control system, server and computer program
JP2005184304A
