An autonomous robot that acts based on experience

Abstract

To provide a technique that allows a robot to select its behaviors while making memory associated with places or objects.SOLUTION: An autonomous behavior robot includes a memory unit, a processor, and a driving mechanism. The processor of the robot detects an object, recognizes impression memory of the object, and determines a motion to the object on the basis of the impression memory. The driving mechanism executes a motion determined by the processor.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present invention relates to a robot that autonomously selects an action depending on its internal state or external environment. [Background technology]

[0002] People keep pets to find comfort. However, there are many reasons why people may not have enough time to care for their pets, may not have a suitable living environment for pets, or may have allergies. People are giving up on their pets for various reasons, such as the loss of their pet and the pain of losing it. There are many people. If there was a robot that could act as a pet, they would not be able to keep a pet. It may be possible to provide the same healing that pets provide to people who are in a difficult situation (especially (See Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-323219 [Patent Document 2] International Publication No. 2017 / 169826 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, robot technology has been advancing rapidly, but it has not yet reached the point where robots can function as companions like pets. Humans tend to believe that pets have free will. By observing their behavior, you can sense the existence of free will in your pet and empathize with them. And pets are soothing.

[0005] One of the characteristics of living things is "memory." Some memories are associative memories that are linked to places or things. For example, by experiencing the warmth of a window, The impression of "warmth" is associated with "windows." Or, next to a red coat. When a person has a pleasant experience in the outside world, a red coat can be associated with a positive image. Living organisms form various memories based on the external environment and They make choices about their actions while stimulating their memories.

[0006] The present invention was completed based on the above-mentioned problem recognition, and its main purpose is to make a robot select its behavior while creating memories associated with places or objects. The purpose of this project is to provide technology that enables [Means for solving the problem]

[0007] An autonomously acting robot according to one aspect of the present invention includes a memory unit, a processor, and a drive mechanism that executes a motion selected by the processor. The processor recognizes an impression memory for an object and determines a motion for the object based on the impression memory. In another aspect of the present invention, an autonomously acting robot includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, a captured image acquisition unit that acquires a captured image, and a feature point acquisition unit that extracts a feature point from the captured image. An image feature acquisition unit acquires image feature information by extracting the image feature information from the captured image. an environmental information acquisition unit that acquires environmental information about the location where the image was acquired; a map management unit that generates an environmental memory by associating the environmental information with the map; Equipped with. The operation control unit refers to the environmental memory and selects the environmental information that satisfies the predetermined destination condition. The associated point is set as a movement destination point.

[0008] According to another aspect of the present invention, there is provided an autonomously acting robot, comprising: a motion control unit that selects a motion of the robot; and a motion control unit that executes the motion selected by the motion control unit. a driving mechanism for driving the image pickup device, a captured image acquisition unit for acquiring captured images, and a feature quantity of the captured images. an image feature acquisition unit that acquires image feature information by extracting the image feature information from the captured image; At the time when the image is acquired, the first environmental information is acquired, and the image feature information and the first environmental information are The primary environmental information acquisition unit registers the environmental information in association with the newly acquired information. The image feature information corresponding to the image feature information having a similarity to the first image feature information equal to or greater than a predetermined threshold value. The first environmental information is then treated as second environmental information corresponding to the first image feature information. and a second environmental information acquisition unit that identifies the environmental information. The operation control unit controls the robot based on both the first environmental information and the second environmental information. Change the behavioral characteristics of the bot.

[0009] According to another aspect of the present invention, there is provided an autonomously acting robot, comprising: a motion control unit that selects a motion of the robot; a driving mechanism that executes the motion selected by the motion control unit; a captured image acquisition unit that acquires a captured image; and a control unit that calculates a feature quantity of the captured image. The image feature acquisition unit to extract and the primary environment at the point where the captured image was acquired The primary environmental information acquisition unit acquires information, and the secondary environmental information acquisition unit receives feature values ​​as input and outputs environmental information. The feature quantities of the newly acquired captured image are input to a predetermined estimation model. By using the environmental information output from the estimation model as the second environmental information, and a second environmental information acquisition unit that acquires the second environmental information. The operation control unit controls the robot based on both the first environmental information and the second environmental information. Change the behavioral characteristics of the bot.

[0010] According to another aspect of the present invention, there is provided an autonomously acting robot, comprising: a motion control unit that selects a motion of the robot; and a motion control unit that executes the motion selected by the motion control unit. a drive mechanism for detecting an object; an object detection unit for detecting an object; an environmental information acquisition unit that acquires environmental information at the time when the object is detected; Object management that generates environmental memory by associating objects with environmental information It comprises a section. The movement control unit changes the behavioral characteristics based on the environmental memory.

[0011] According to another aspect of the present invention, there is provided an autonomously acting robot, comprising: a motion control unit that selects a motion of the robot; and a motion control unit that executes the motion selected by the motion control unit. a drive mechanism for detecting an object; an object detection unit for detecting an object; and a state management unit that changes the emotion parameter when a change in emotion is detected. The motion control unit changes the behavioral characteristics of the robot according to the emotion parameters. Can.

[0012] According to another aspect of the present invention, there is provided an autonomously acting robot, comprising: a motion control unit that selects a motion of the robot; and a motion control unit that executes the motion selected by the motion control unit. a drive mechanism for detecting a plurality of objects; an object detection unit for detecting a plurality of objects; an environmental information acquisition unit that acquires environmental information at the time when the object is detected; A state management unit that sets impression memories for multiple objects based on environmental information. And, it is equipped with. The state management unit detects multiple objects again. Changes emotional parameters based on the impression memory associated with the object, The motion control unit changes the behavioral characteristics of the robot according to the emotion parameters. do. [Effects of the Invention]

[0013] According to the present invention, it becomes easier to realize memory generation in a robot and behavior selection accompanying the memory. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a conceptual diagram illustrating how a robot forms memories based on a place. [Figure 2] Figure 2(a) is a front view of the robot, and Figure 2(b) is a side view of the robot. [Figure 3] FIG. 2 is a cross-sectional view schematically illustrating the structure of the robot. [Figure 4] FIG. 2 is a hardware configuration diagram of a robot in a basic configuration. [Figure 5] FIG. 2 is a functional block diagram of the robot system. [Figure 6] FIG. 2 is a functional block diagram of the robot according to the present embodiment. [Figure 7] FIG. 10 is a schematic diagram showing a method for creating a key frame. [Figure 8] FIG. 10 is a diagram illustrating a data structure of environment map information. [Figure 9] FIG. 10 is a data structure diagram of object information. [Figure 10] FIG. 10 is a schematic diagram showing a chain of key frames. [Figure 11] 10 is a flowchart showing an environment map information update process. [Figure 12] FIG. 10 is a schematic diagram for explaining a method for a robot to select a destination in response to the occurrence of an event. [Figure 13] 10 is a flowchart showing the process of processing an event. [Figure 14] 12 is a flowchart showing details of the object determination process shown in S28 of FIG. 11. [Figure 15] FIG. 10 is a diagram showing the data structure of object environment information in a modified example. [Figure 16] FIG. 10 is a diagram showing a data structure of room impression information in a modified example. [Figure 17]10 is a flowchart showing a processing procedure when an object is detected in a modified example. [Figure 18] FIG. 10 is a data structure diagram of behavior experience information in another modified example. DETAILED DESCRIPTION OF THE INVENTION

[0015] FIG. 1 illustrates how a robot 100 forms memories based on locations. This is a conceptual diagram for The robot 100 of this embodiment is equipped with a small camera. By periodically capturing images of the surrounding area using this camera, The robot 100 also acquires a still image (a still image). Environmental information is acquired by various sensors such as sound, Oi, radio wave reception strength, temperature, presence or absence of user, etc., detected values ​​by various sensors The robot 100 stores information based on the captured image (hereinafter referred to as "image"). The two types of memory are called "environmental memory" and "environmental memory." By associating these two types of memories, the robot 100 can A memory of the place (hereafter referred to as "place memory") is formed.

[0016] The image memory layer 400 in FIG. 1 is a memory layer that conceptually shows image memory, and The memory layer 402 is a memory layer that conceptually shows environmental memory. The key frame 404 is a collection of key frames 404. The robot 100 of this embodiment uses the image Graph-based Simultaneous Localization and Analysis (SLAM) using image features Mapping) technology, more specifically, ORB (Oriented FAST and Rotated BRI) EF) Keyframes 404 are generated using feature-based SLAM technology (see details) Details will be provided later).

[0017] The robot 100 periodically forms key frames 404 as it moves. By doing so, a collection of key frames 404, in other words, an image feature distribution, The robot 100 forms an image storage layer 400. The acquired key frame 404 and the many key frames 404 already held (image memory layer 400), the robot 100 understands where it is located in the image memory layer 400. In other words, the robot 100 performs "spatial recognition" by comparing the captured image it is actually viewing with the captured image (memory) it previously viewed and aligning its current situation with its past memory.

[0018] When capturing an image, the robot 100 also acquires environmental information. The memory is a collection of environmental information at each location. The direction of movement is selected based on the information (environmental memory layer 402). When the PC 100 is at the high temperature point P1, it is placed in a cool area where the average temperature is below 20 degrees. If the robot 100 selects to move to a point that satisfies the above conditions, The environmental information is searched from the environmental memory layer 402, and the environmental information of the point P2 satisfies the above conditions. Next, the robot 100 determines the location of the point P2. The robot 100 identifies the image storage layer 400 (key The route to point P2 is identified by referring to the distribution of frame 404. By the above control method, the robot 100 can operate under the condition that the average temperature is 20 degrees or less. Move towards point P2 that satisfies the condition.

[0019] The basic configuration of the robot 100 will be described below with reference to FIGS. 2 to 5. After that, the method of forming and using memories in this embodiment will be mainly explained. do. The robot 100 in the basic configuration uses an external sensor instead of the key frame 404. It is assumed that the position is recognized by a sensor. An external sensor may also be used.

[0020] [Basic configuration] FIG. 2(a) is a front view of the robot 100. FIG. 2(b) is a front view of the robot 100. FIG. 1 is a side external view of the printer 100. The robot 100 in this embodiment is configured to: The external environment is controlled by cameras and sensors. The internal state of the robot 100 is recognized by various sensors such as the Emotions are quantified as various parameters. The robot's range of movement is within the owner's home. A person who uses the service is called a "user."

[0021] The body 104 of the robot 100 has a generally rounded shape. It is made of soft and elastic materials such as polyethylene, rubber, resin, and fiber. The robot 100 may be dressed in clothing. The mass is about 5 to 15 kilograms and the height is about 0.5 to 1.2 meters. With its appropriate weight, roundness, softness, and pleasant feel, it is a product that is loved by users. The robot 100 is easy to hold and makes people want to hold it. The effect is realized.

[0022] The robot 100 has a pair of front wheels 102 (left wheel 102a and right wheel 102b) and , and one rear wheel 103. The front wheel 102 is a driving wheel, and the rear wheel 103 is a driven wheel. The front wheels 102 do not have a steering mechanism, but the rotation speed and direction can be individually controlled. The rear wheels 103 are casters and can be controlled separately. The rear wheel 103 is rotatable so that the 00 can move forward, backward, left and right. It may also be a muniwheel.

[0023] The front wheels 102 and the rear wheels 103 are driven by a drive mechanism (rotation mechanism, link mechanism). The wheels can be completely stored in the body 104. Even when driving, most of the wheels are in contact with the body. When each wheel is completely stored in the body 104, The vehicle 100 is now in an immovable state. The body 104 descends and rests on the floor F. In this resting state, The flat seating surface 108 (ground contact surface) formed at the bottom of the seat 104 is placed on the floor F abuts on.

[0024] The robot 100 has two hands 106. The hands 106 have a function to grasp an object. The hand 106 can perform simple actions such as raising, shaking, and vibrating. The two hands 106 are also individually controllable.

[0025] The eye 110 is capable of displaying images using a liquid crystal element or an organic EL element. The robot 100 is equipped with a microphone array or an ultrasonic sensor that can identify the direction of a sound source. It is equipped with various sensors, including a built-in speaker, and can emit simple sounds.

[0026] The robot 100 has horns 112 attached to its head. Because the bot 100 is lightweight, the user can grab the bot by grabbing the horns 112. It is also possible to lift the 100. The horn 112 is equipped with a panoramic camera. The camera is attached to the robot 100, and can capture an image of the entire upper area of ​​the robot 100 at once.

[0027] FIG. 3 is a cross-sectional view that schematically illustrates the structure of the robot 100. As shown in FIG. As shown in FIG. 3, the body 104 of the robot 100 is mounted on a base frame 3 08, a main body frame 310, a pair of resin wheel covers 312 and an outer The base frame 308 is made of metal and includes a skin 314. The base frame 308 forms the shaft core and supports the internal mechanism. The upper plate 332 and the lower plate 334 are connected to a plurality of side plates 33 6 are connected vertically. The base frame 308 is spaced apart so that it can be ventilated. The battery 118, the control circuit 342 and various actuators are housed on one side. It is tolerated.

[0028] The main body frame 310 is made of a resin material, and the head frame 316 and the body frame The head frame 316 is hollow and hemispherical. The body frame 318 forms the head skeleton of the robot 100. The body frame 318 has a stepped cylindrical shape. The torso frame 318 forms the torso skeleton of the robot 100. The head frame 316 is fixed integrally with the body frame 308. 18 so as to be relatively displaceable.

[0029] The head frame 316 has a yaw axis 320, a pitch axis 322, and a roll axis 324 and an actuator 326 for rotating each axis. The actuator 326 is configured with multiple servos for individually driving each axis. The motor drives the yaw axis 320 for the head swinging motion and the motor drives the nod motion. The pitch axis 322 is driven for tilting the head, and the roll axis 324 is driven for tilting the head. Driven.

[0030] A plate 325 supporting the yaw axis 320 is provided on the top of the head frame 316. The plate 325 has multiple holes to ensure ventilation between the top and bottom. A number of vent holes 327 are formed.

[0031] A metal frame is provided to support the head frame 316 and its internal mechanism from below. The base plate 328 is made of a cross-section. It is connected to the plate 325 via a link mechanism 329 (pantograph mechanism). On the other hand, an upper plate 332 (base frame) is connected to the upper plate 332 via a joint 330. It is connected to the 308.

[0032] The body frame 318 houses the base frame 308 and the wheel drive mechanism 370. The wheel drive mechanism 370 includes a rotating shaft 378 and an actuator 379. The lower half of the body frame 318 is connected to the wheel cover 312. The width is narrow to form a storage space S for the wheel 102.

[0033] The outer skin 314 is made of urethane rubber and is attached to the main body frame 310 and the wheels. The hand 106 is integrally molded with the outer shell 314. An opening 390 is provided at the top end of the outer shell 314 to allow outside air to enter. do.

[0034] FIG. 4 is a diagram showing the hardware configuration of the robot 100. The robot 100 includes an internal sensor 128, a communication device 126, a memory device 124, It includes a processor 122, a drive mechanism 120, and a battery 118. The sensor 122 and the memory device 124 are included in the control circuit 342. The battery 11 is connected to the power supply line 130 and the signal line 132. 8 supplies power to each unit via a power line 130. Each unit is The control signal is transmitted and received via line 132. The battery 118 is a lithium-ion battery. The battery is a rechargeable battery and is the power source for the robot 100.

[0035] The internal sensor 128 is a collection of various sensors built into the robot 100. Specifically, the camera (omnidirectional camera), microphone array, and distance measurement sensor sensor (infrared sensor), thermo sensor, touch sensor, acceleration sensor, The touch sensor is a sensor between the outer cover 314 and the main frame 310. The odor sensor is a known sensor that applies the principle that electrical resistance changes when molecules that cause odors are adsorbed.

[0036] The communication device 126 is a communication module that performs wireless communication with various external devices. The storage device 124 is a non-volatile memory and a volatile memory. It stores computer programs and various setting information. 122 is a computer program execution means. It includes a plurality of actuators and the wheel drive mechanism 370 described above. It will also be equipped with a display and speaker.

[0037] The drive mechanism 120 mainly includes wheels (front wheels 102) and a head (head frame The drive mechanism 120 controls the direction and movement of the robot 100. In addition to changing the speed, the wheels (front wheels 102 and rear wheels 103) are raised and lowered. When the wheels are raised, they are fully retracted into the body 104. The robot 100 comes into contact with the floor F at the seating surface 108 and assumes a seated state. , drive mechanism 120 controls hand 106 via wire 134 .

[0038] FIG. 5 is a functional block diagram of the robot system 300. The robot system 300 includes a robot 100, a server 200, and a plurality of The robot 100 and the server 200 include an external sensor 114. is the calculation of the CPU (Central Processing Unit) and various coprocessors. devices, storage devices such as memory and storage, and wired or wireless connections that connect them The hardware includes a communication line, and the memory device stores the processing instructions to the processor. The computer program is implemented by software that provides Device drivers, operating systems, and the layers above them Various application programs and common functions for these programs The blocks described below may be configured using libraries provided by the company. indicates functional blocks rather than a hardware configuration. Some of the functions of the robot 100 may be realized by the server 200. Some or all of the functions of the server 200 may be realized by the robot 100. good.

[0039] A plurality of external sensors 114 are installed in advance inside the house. The position coordinates of the external sensor 114 are registered in the field 0. Based on information obtained from the sensor 128 and the plurality of external sensors 114, The server 200 determines the basic behavior of the robot 100. The external sensor 114 The server 200 is used to reinforce the sensory organs of the robot 100. The purpose of this is to reinforce the brain of the robot 100. 26 periodically communicates with the external sensor 114, and the server 200 4 to identify the position of the robot 100 (see also Patent Document 2).

[0040] (Server 200) The server 200 includes a communication unit 204, a data processing unit 202, and a data storage unit Including 206. The communication unit 204 performs communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. 2 is the data acquired by the communication unit 204 and stored in the data storage unit 206. The data processing unit 202 executes various processes based on the stored data. It also functions as an interface between the receiver 204 and the data storage 206 .

[0041] The data storage unit 206 includes a motion storage unit 232 and a personal data storage unit 21. Includes 8. The robot 100 has a plurality of motion patterns. Shaking 6, snakes around to approach the owner, tilts his head to the side Various motions are defined, such as staring at something.

[0042] The motion storage unit 232 stores a "motion control" that defines the control content of the motion. Each motion is identified by a motion ID. The motion file is also stored in the motion storage unit 160 of the robot 100. The motion to be executed is determined by the server 200. It may be determined by the robot 100.

[0043] Many of the motions of the robot 100 are complex motions that include multiple unit motions. For example, when the robot 100 approaches the owner, it may have a unit motion of turning towards the owner, a unit motion of approaching while raising its hand, and so on. Position motion, approaching while shaking body Unit motion, raising both hands It may be expressed as a combination of unit motions of sitting down. The combination of four motions allows the robot to approach its owner and hold its hand. The motion of "lifting the body, then shaking the body and finally sitting down" is realized. The motion file contains the motion data of the actuators installed in the robot 100. The rotation angle and angular velocity are defined in relation to the time axis. Each actuator is controlled according to the actuator control information. By controlling the Eta, various motions can be expressed.

[0044] The transition time from the previous unit motion to the next unit motion This is called an "interval." The interval is the time required to change the unit motion. The length of the interval can be adjusted according to the duration and the motion. It is possible to adjust it. Below are some tips on when to choose which motion and what to do to achieve it. Settings related to the behavior control of the robot 100, such as adjusting the actuator output, The behavioral characteristics of the robot 100 are collectively referred to as "behavioral characteristics." Defined by selection algorithm, motion selection probability, motion file, etc. will be done.

[0045] The motion storage unit 232 stores various event files in addition to motion files. A motion selection table that defines the motion to be executed when In the motion selection table, 1 or more events are stored. The motions are associated with their selection probabilities.

[0046] The personal data storage unit 218 stores information about the user. The master information showing the user's intimacy with the user and their physical and behavioral characteristics is stored. Other attribute information such as age and gender may also be stored.

[0047] The robot 100 has an internal parameter called intimacy for each user. Robot 100 picks you up, talks to you, and does other things for you. When a user recognizes a behavior that shows affection, the user's intimacy with that user increases. Users who are not involved with Robot 100, users who are violent, and users who are rarely encountered The level of intimacy with users who are not familiar with the service will be low.

[0048] The data processing unit 202 includes a location management unit 208, a recognition unit 212, and an operation control unit 2 22, includes an intimacy management unit 220 and a status management unit 244. The position management unit 208 identifies the position coordinates of the robot 100. 244 is a device that stores various physical information such as the charge rate, internal temperature, and the processing load of the processor 122. The state management unit 244 manages various internal parameters such as the robot's state. A variety of emotions that represent 100 emotions (loneliness, curiosity, desire for approval, etc.) Manage your parameters. These emotional parameters are constantly fluctuating. The movement target point of the robot 100 changes depending on the parameter. For example, When the user feels lonely, the robot 100 moves from the user's location. Set it as a moving target point.

[0049] Emotion parameters change over time. Also, various emotion parameters change depending on the interaction described below. For example, when the owner "holds" the When the owner is given a gift, the emotional parameter indicating loneliness decreases, and the owner feels lonely for a long time. When the person is not visible, the emotion parameter indicating loneliness increases little by little.

[0050] The recognition unit 212 recognizes the external environment. The recognition of the external environment includes temperature and humidity. Recognition of weather and season based on the amount of light and temperature, recognition of shade (safe zones), etc. The recognition unit 156 of the robot 100 uses the internal sensor 1 28 acquires various environmental information, performs primary processing on it, and then transmits it to the server 200 The received data is transferred to the recognition unit 212.

[0051] Specifically, the recognition unit 156 of the robot 100 recognizes moving objects, particularly Extract image regions corresponding to people and animals, and then extract moving objects from the extracted image regions. We extract a "feature vector" as a set of features that indicate the physical and behavioral characteristics of the The feature vector components (feature values) quantify various physical and behavioral characteristics. For example, the width of a human eye is expressed as a number in the range of 0 to 1, A feature vector is extracted from a captured image of a person. The method for extracting the feature vector is an application of known face recognition technology. The robot 100 transmits the feature vector to the server 200.

[0052] The recognition unit 212 of the server 200 receives an image captured by the built-in camera of the robot 100. The feature vector extracted from the image and the personal data stored in advance in the personal data storage unit 218 are stored. By comparing with the feature vectors of registered users (clusters), It is determined which person the photographed user corresponds to (user identification process). Furthermore, the recognition unit 212 recognizes the user's facial expression by image recognition. The recognition unit 212 recognizes moving objects other than people, such as pets, The user identification process is also performed on cats and dogs.

[0053] The recognition unit 212 recognizes various behaviors performed by the robot 100. The recognition unit 212 also classifies the behavior of the robot 100 into pleasant and unpleasant behaviors. By recognizing the owner's response to the do. Pleasant and unpleasant behaviors are determined by whether the user's behavior is pleasant to the living body. For example, being hugged is considered unpleasant by the robot. For Robot 100, being kicked is a pleasurable act, and for Robot 100, being kicked is a pleasurable act. Positive and negative reactions are those that indicate whether the user's response behavior leads to a positive feeling. It is judged by whether it is a sign of joy or a sign of discomfort. Being kicked is a positive reaction that indicates the user's positive feelings, and being kicked is a negative reaction that indicates the user's negative feelings. It is a negative reaction that shows emotion.

[0054] The operation control unit 222 of the server 200 controls the operation control unit 150 of the robot 100. The server 200 cooperates with the robot 100 to determine the motion of the robot 100. The control unit 222 determines the movement destination of the robot 100 and the movement route therefor. The movement control unit 222 creates multiple movement routes and then You may choose either route.

[0055] The motion control unit 222 selects a motion from a plurality of motions stored in the motion storage unit 232. Select 100 bot motions. Each motion has a selection confirmation for each situation. For example, when the owner gives a pleasant gesture, defines a selection method such as executing motion A with a 20% probability, and when the temperature is 30 degrees or higher, executing motion B with a 5% probability.

[0056] The intimacy management unit 220 manages the intimacy for each user. The degree of intimacy is registered as part of the personal data in the personal data storage unit 218. When a pleasurable behavior is detected, the intimacy management unit 220 Increases intimacy. When unpleasant behavior is detected, intimacy decreases. Additionally, the intimacy level of an owner who has not been seen for a long period of time will gradually decrease.

[0057] (Robot 100) The robot 100 includes a communication unit 142, a data processing unit 136, and a data storage unit 1 48, an internal sensor 128 and a drive mechanism 120. The communication unit 142 corresponds to the communication device 126 (see FIG. 4) and communicates with the external sensor 114 , and is responsible for communication processing with the server 200 and other robots 100. The storage unit 148 stores various data. 24 (see FIG. 4). Based on the acquired data and the data stored in the data storage unit 148 The data processing unit 136 executes various processes using the processor 122 and This corresponds to a computer program executed by the processor 122. The data processing unit 136 is connected to the communication unit 142, the internal sensor 128, the drive mechanism 120, and It also functions as an interface to the data storage unit 148.

[0058] The data storage unit 148 stores motion data that defines various motions of the robot 100. The storage unit 160 includes: The motion storage unit 160 of the robot 100 stores the motion data of the server 200. Various motion files are downloaded from the motion storage unit 232. The motion is identified by a motion ID. By lifting the hand 106 and rotating the two front wheels 102 in opposite directions, Alternatively, the robot 100 can be made to rotate by rotating only one of the front wheels 102. The front wheels 102 are rotated in a retracted state to vibrate. In order to express various motions, such as stopping and looking back when moving away from the user, various actuators (drive mechanism 120) are operated. Timing, movement duration, movement direction, etc. are recorded in the motion file in chronological order. be defined. The data storage unit 148 also stores various data downloaded from the personal data storage unit 218. It may be downloaded.

[0059] The data processing unit 136 includes a recognition unit 156 and an operation control unit 150 . The operation control unit 150 of the robot 100 is connected to the operation control unit 222 of the server 200. The robot 100 decides the motion of the robot 100 in cooperation with the The server 200 determines the motion, and the robot 100 determines the other motions. Alternatively, the robot 100 may determine the motion. When the processing load of server 100 is high, server 200 decides the motion. The server 200 may determine the base motion and Additional motions may be determined in step 100. How the processing is shared between the server 200 and the robot 100 is determined by the It can be designed according to the specifications of the bot system 300.

[0060] The motion control unit 150 of the robot 100 executes the selected motion to the driving mechanism 12. The drive mechanism 120 executes the motion file. Controls each actuator.

[0061] The operation control unit 150 may be configured to select "hold" when a user with a high degree of intimacy is nearby. Execute a motion of lifting both hands 106 as a gesture of asking for When you get tired of holding the baby, you can store the left and right front wheels 102. By alternating between reverse rotation and stopping, it expresses the motion of not wanting to be picked up. The driving mechanism 120 can also perform the following operations according to the instructions of the operation control unit 150. By driving the front wheels 102, hands 106, and neck (head frame 316), Make Bot 100 perform various motions.

[0062] The recognition unit 156 of the robot 100 recognizes the external information obtained from the internal sensor 128. The recognition unit 156 is a unit for visual recognition (visual unit), smell recognition (smelling unit), and the like. It is possible for the sense of sight (sensory area), sound perception (auditory area), and tactile perception (tactile area).

[0063] The recognition unit 156 extracts a feature vector from the captured image of the moving object. As mentioned above, the feature vector is a pattern that shows the physical and behavioral characteristics of a moving object. When a moving object is detected, the odor Physical and behavioral characteristics can also be extracted from sensors, built-in sound collection microphones, and temperature sensors. These features are also quantified and become feature vector components. 56 is based on the known technology described in Patent Document 2, etc., and Identify the user.

[0064] Among the series of recognition processes including detection, analysis, and judgment, the recognition part of the robot 100 156 selects and extracts the information necessary for recognition, and the interpretation process such as judgment is handled by the The recognition process is executed by the recognition unit 212 of the server 200. This may be done by the recognition unit 212 alone, or by the recognition unit 156 of the robot 100 alone. Alternatively, both parties may share roles and perform the above recognition process. This may be done.

[0065] When a strong impact is given to the robot 100, the recognition unit 156 The sensor and the acceleration sensor recognize this, and the recognition unit 2 of the server 200 12 recognizes that a "violent act" has been committed by a nearby user. When the robot grabbed the horns 112 and lifted the robot 100, it was also violent. When a user facing the robot 100 hears a specific volume, When a voice is uttered in a specific region and a specific frequency band, the recognition unit 212 of the server 200 may recognize that a "calling action" has been made to them. When a temperature of this level is detected, it is recognized that a "touch" has been made by the user. However, when the device detects upward acceleration while contact is recognized, it is not "holding" the baby. The physical contact when the user lifts the body 104 is recognized. Alternatively, the load on the front wheels 102 may be decreased to detect the load. Parentheses may be recognized. In summary, the robot 100 acquires the user's actions as physical information using the internal sensor 128, and the recognition unit 212 of the server 200 judges whether the user feels comfortable or uncomfortable. The recognition unit 212 of the server 200 also performs user identification processing based on the feature vector.

[0066] The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Some typical response actions include pleasant or It is associated with unpleasantness, positiveness or negativeness. Generally, it is a pleasant response. Most of the responses that cause discomfort are positive, and most of the responses that cause discomfort are negative. Pleasant and unpleasant behaviors are related to the degree of intimacy, and positive and negative reactions are related to the degree of intimacy with the robot. Affects 0's choice of action.

[0067] The intimacy level of the server 200 is determined according to the interaction recognized by the recognition unit 156. The management unit 220 changes the intimacy level with the user. The intimacy level with the user who committed the offense increases, and the parental level with the user who committed the offense increases. The density decreases.

[0068] Based on the above basic configuration, the robot 100 in this embodiment will now be described. Regarding the implementation of this, we will focus on the features and purpose of this implementation and the differences from the basic configuration. Explain it as a heart.

[0069] [Implementation of memory of places and objects] FIG. 6 is a functional block diagram of the robot 100 in this embodiment. The robot 100 of this embodiment uses a key frame instead of an external sensor 114. The robot 100 recognizes the location based on the key frame 404. The location may be recognized based on both the sensor 4 and the external sensor 114. In this embodiment, the location is recognized based only on the keyframe 404. and explain.

[0070] (Server 200) The position management unit 208 of the data processing unit 202 includes a map management unit 168 and an object management unit 174. The map management unit 168 manages information associating the image memory layer 400 with the environment memory layer 402 (hereinafter referred to as "environment map" or "environment map information"). The object management unit 174 manages object information that associates events with objects. An event is an occurrence that is recognized as occurring when environment information satisfies a predetermined event condition (described below).

[0071] The data store 206 further includes a map store 170 and an object information store. The map storage unit 170 stores the environment map information. The object information storage unit 172 stores object information. Details will be described later. However, the state management unit 244 manages the emotional performance based on the environmental information and events. Change the parameter.

[0072] (Robot 100) In addition to the recognition unit 156 and the operation control unit 150, the data processing unit 136 The captured image acquisition unit 162 includes a captured image acquisition unit 162. The captured image acquisition unit 162 periodically acquires captured images from the camera. For example, captured images are acquired twice per second.

[0073] The recognition unit 156 includes an image feature acquisition unit 164 and an environmental information acquisition unit 166 . The image feature acquisition unit 164 extracts image feature amounts from the captured image. The image feature extraction method is shown in Figure 7. The environmental information acquisition unit 166 acquires the captured image. Obtaining environmental information: Keyframe 404 (image feature information) and environmental information is transmitted to the server 200 by the communication unit 142. The management unit 168 associates the key frame 404 with the environmental information. Create a boundary map.

[0074] FIG. 7 is a schematic diagram showing a method for creating the key frame 404. As shown in FIG. As described above, the captured image acquisition unit 162 periodically acquires captured images. The captured image 412 in FIG. 7 shows a chair 406 and a shelf 408. The image feature acquisition unit 164 identifies the edge points of these objects as feature points 410. The key frame 404 is a set of feature points 410. The number and distribution of feature points 410 included in the image 404, in other words, the image The image feature characterizes the key frame 404. The captured image 412 itself Instead, image storage based on key frames 404 (image feature information) This has the advantage of reducing the amount of data.

[0075] FIG. 8 is a diagram showing the data structure of the environment map information 420. The environment map information 420 is stored in the map storage unit 170 of the server 200. The key frame 404 is stored in a key frame ID (hereinafter referred to as "FID"). The image feature acquisition unit 164 extracts a key frame from the captured image. The environmental information acquisition unit 166 assigns an FID to each formed frame 404. - Environmental information is acquired at the same time as frame 404 (captured image) is acquired. (Hereinafter, the environment sensed at the same time as the key frame 404 is acquired) The map management unit 168 manages the key The frame 404 and the environmental information are associated and registered in the environmental map information 420 . The environment map information 420 connects the image memory with the environment memory.

[0076] The environmental information in this embodiment includes reception strength, a user detection rate, an average volume, and an average temperature. and changes depending on the distance between the robot 100 and the server 200. The acquisition unit 166 measures the reception strength and ranks it into A, B, and C in descending order. According to FIG. 8, the key frame 404 (hereinafter referred to as the "key frame") of FID=F01 The reception strength of the signal is "A" and is written as "F01 (F04)". Therefore, the point corresponding to the key frame 404 (F01) (hereinafter referred to as "point (F01)") ) is a location with good communication. - Frame 404 (F04) has a communication strength of "C", so point (F04 ) is a point where communication is poor. In this embodiment, the point is between 0.1 and 1 meters. It is defined as an area with a radius of about 1 / 4 of a meter.

[0077] The user detection rate indicates how often a user is detected at each location, or in other words, the likelihood of encountering a user at each location. , defined for each user. User X1 detection rate indicates the detection rate of user X1. For example, if robot 100 has been at point (F01) 10 times, If user X1 is detected twice in 10 stay occasions, The group management unit 168 calculates the detection rate of user X1 as 20(%). For example, at location (F04), users X1 and X2 are likely to be encountered, and at location (F05), user X2 can be encountered with a high probability of nearly 50%. On the other hand, at location (F07), users X1 and X2 have never been encountered.

[0078] The environmental information acquisition unit 166 detects the volume at each point using a microphone. The average volume is then ranked A, B, or C in descending order. Quiet and noisy places are identified. Similarly, the environmental information acquisition unit 166 Detect the temperature at each point using a temperature sensor and calculate the average temperature. Average temperatures identify hot and cool areas. may be further classified by other parameters such as time of day or season.

[0079] Environmental information can also include various other sensor information. For example, the smell detected by an odor sensor can be included in the environmental information. Or, you can include in the environmental information whether there is a "ceiling" such as under the table. Good. Not only the user detection rate (visibility), but also the "probability of someone touching" The interaction experience with the user, such as "probability of being hugged" or "probability of being hit" The environmental information may be defined as at least image feature information. Any external information is sufficient, and it can be identified as touch, hearing, or smell, such as temperature or wind direction. It may also be information that

[0080] The server 200 refers to the environment map information 420 and determines the predetermined destination conditions. The point where the robot 100 satisfies the condition is set as the target point of movement. The emotional parameter value (hereafter referred to as "emotion value (loneliness)") indicates ) becomes equal to or greater than the threshold, the operation control unit 222 A point where the value is equal to or greater than a predetermined threshold, for example, point (F05), is set as the movement target point. The destination conditions can be set arbitrarily. For example, When the temperature is above 25 degrees, the location with environmental information that the average temperature is below 20 degrees The destination condition may be satisfied in the

[0081] The state management unit 244 receives the first information acquired at the current location of the robot 100. Next, the emotion parameters are changed based on the environmental information (hereinafter referred to as "environmental information"). The state management unit 244 also manages the state of the robot 100. The emotion parameters are also changed (hereinafter referred to as "secondary reflection (based on environmental information)") based on environmental information previously acquired at the current location (hereinafter, the environmental information sensed at the time keyframe 404 was acquired and before is also referred to as "secondary environmental information"). In other words, state management unit 244 changes the emotion parameters not only based on current primary environmental information, but also on stored past secondary environmental information. Changes in emotion parameters based on environmental information will be described in detail with reference to FIG. 11.

[0082] FIG. 9 is a diagram showing the data structure of the object information 430. The object information 430 is stored in the object information storage unit 17 of the server 200. 2. Various events occur around the robot 100. The recognition unit 212 of the server 200 determines whether the primary environmental information satisfies a predetermined event condition. When the conditions are met, it is determined that an event has occurred. the user has recognized the device, been held by the user, heard a sound above a certain volume, or been splashed with water The recognition unit 212 can be configured to recognize a plurality of images in advance. The vent conditions are set. The event is identified by an event ID.

[0083] When the recognition unit 212 detects the occurrence of an event, the communication unit 204 The image feature acquisition unit 164 transmits an event detection signal to the image feature acquisition unit 100. When the detection signal is received, the object is detected from the latest captured image. The objects can be any type, such as a coffee cup, chair, shelf, or doll. In this embodiment, the image feature acquisition unit 164 acquires a plurality of types of objects. For each object, multiple objects are created with predefined shape and color characteristics. The object pattern is used to generate an object. The image feature acquisition unit 164 performs image recognition on the image. When the object is received, it is assigned an object ID and sent to the server 200. The object management unit 174 associates the event ID with the object ID. The object recognition can be realized by applying known image recognition techniques.

[0084] The object management unit 174 manages the object information 430. In the object information 430, the strength of the association between the event and the object is The relevance between an event (E01) and an object (B01) is an index that expresses the strength of the association and causality between the event (E01) and the object (B01), based on the likelihood that the object (B01) will be recognized when the event (E01) occurs.

[0085] When an object (B01) is detected during an event (E01) The object management unit 174 manages events (E01) and objects (B01) On the other hand, when the event (E01) occurs, the object If the object (B01) is not detected, the object management unit 174 The degree of association between the instance (E01) and the object (B01) is reduced. In this case, the relevance is calculated by the detection rate of the object when the event occurs. For example, in the last 10 occurrences of event (E01), When object (B01) is detected 3 times out of 10 times, the object management The management unit 174 sets the relevance between the event (E01) and the object (B01) to 30. Set to (%).

[0086] When event (E01) occurs, there is a high probability that object (B01) If an event (E0 1) is likely to occur. For example, the degree of confidence that the baby will be held Assume that a sofa (object) is detected with a high probability at this point. In that case, if you go near the sofa, you are more likely to get a hug. This makes it possible to make such predictions.

[0087] The state management unit 244 changes the emotion parameters based on the event ( Hereafter, this will be referred to as "primary reflection (by events)." Events are the first environmental information Since it is also a type of information, the primary reflection by an event is usually the primary environmental information. The state management unit 244 manages the current state of the robot 100. Based on events associated with detected objects at the current location (Hereafter, this will be referred to as "secondary reflection (due to an event)") The state management unit 244 changes the emotion parameters not only according to events that are actually occurring, but also according to events associated with the object. Regarding the change of emotion parameters based on events, see Figures 13 and 14. This will be explained in detail.

[0088] FIG. 10 is a schematic diagram showing a chain of key frames 404. The image feature acquisition unit 164 periodically acquires key frames 404. In 0, when the key frame 404 (F01) is acquired, the environmental information The acquisition unit 166 acquires the environmental information (e01). The map management unit 168 manages the environmental information (e02). In the environment map information 420, the key frame 404 (F01) and the environment information ( e01) and register them. After a certain time has passed, the robot 100 -Get frame 404 (F02) and environment information (e02). Map The management unit 168 associates the key frame 404 (F02) with the environmental information (e02). In addition, if the location (F01) and the location (F02) are adjacent to each other, This information is also recorded.

[0089] After acquiring the key frame 404 (F02), the robot 100 further acquires the key frame 404 (F03). The map management unit 168 acquires the frame 404 (F03). 2) and the point (F03) are adjacent to each other. By the above process, the robot system 300 can move from the point (F01) to , and if you go through point (F02), you can reach point (F03). The positional relationship of frame 404, in other words, the spatial configuration, is learned.

[0090] After keyframe 404(F03), keyframe 404(F04), Keyframe 404 (F05) and keyframe 404 (F06) are acquired. The map management unit 168 finds a key frame similar to key frame 404 (F06). Specifically, key frame 404 (F02) is detected. 04 (F06) and keyframe 404 (F02) feature points 410 The similarity of the number and distribution is calculated, and if the similarity is equal to or greater than a predetermined threshold, the The map management unit 168 determines that the key frame 404 (F02) and the key Frame 404 (F06) is determined to be a key frame 404 at the same point. The map management unit 168 updates the key frame 404(F02) with the key frame 404(F06) using a method described below. The map management unit 168 also recognizes that the key frame 404(F05) and the key frame 404(F02) are adjacent to each other. In this way, the positional relationships of the multiple key frames 404 are registered in the map storage unit 170 as a graph-type database.

[0091] FIG. 11 is a flowchart showing the process of updating the environment map information 420. be. The update process shown in FIG. 11 is periodically executed in the robot system 300. First, the captured image acquisition unit 162 acquires a captured image (S10). The image feature acquisition unit 164 extracts image feature amounts from the captured image and generates key frames. The environmental information acquisition unit 166 generates the (first) environmental information 404 (S12). The state management unit 244 acquires the emotional parameters according to the environmental information (S14). The meter is changed (S16). This is the "primary reaction based on environmental information" mentioned above. How to change the emotion parameters according to the environmental information is a key issue. For example, if the user is not detected in the captured image, If the undetected state continues for a certain period of time, the emotional value (loneliness) will increase. In this case, the recognition unit 212 may recognize the environment information that relieves loneliness. The movement control unit 150 sets the location as a movement destination. When the robot 100 becomes more sensitive, the frequency of the robot's behavior selection may be suppressed, or a specific motion may be selected. The environmental information affects the emotion parameters, and the emotion Emotional parameters affect behavioral characteristics.

[0092] When the user holds the robot, the emotional value (loneliness) may be decreased. When a volume within a predetermined range is detected, the emotional value (curiosity) may be increased. As mentioned above, the intimacy management unit 220 changes not only the emotion value but also the intimacy level. It is also acceptable to do so.

[0093] The recognition unit 212 performs a similarity check on the key frame 404 extracted in S12. Is there a key frame 404 where the value is greater than or equal to a predetermined value? Whether the location is a location that has already been visited (hereinafter referred to as a "revisited location") If it is a revisited point (Y in S18), the map management unit 168 updates the key frame 404 (image feature information) of the revisited point. (S20).

[0094] Some of the captured images are stable, such as bookshelves and sofas, while others are (hereinafter referred to as "stable features"), which exist temporarily, such as coffee cups and documents. (hereinafter referred to as "unstable features"). Map Management Unit 1 68 is the first keyframe acquired at a revisited location (F05). 404(F05-1) and the second acquired key frame 404(F05- 2) Compare keyframe 404 (F05-1) and keyframe 404 (F05-2) is almost the same overall, but the unstable feature part For example, in keyframe 404 (F05-1), contains the image features of the note (unstable features), and keyframe 404 (F05 -2) may not contain image features of the note. This is because the notebook was removed between P5's first and second visits. At this time, the map management unit 168 stores the key frame 404 of the point (F05). The feature point 410 showing the unstable feature is excluded from (F05). The method selects keyframes 404 that contain only stable features, in other words, spatial Keyframes 404 containing only image features that are easy to use as clues for recognition can be formed. As a simpler method, the map management unit 168 may As the latest acquired keyframe 404, the previous keyframe It may overwrite the group 404.

[0095] The state management unit 244 manages the (second) environment previously associated with the revisited point. The emotion parameters are changed based on the boundary information (S22). For example, if environmental information is added to point (F05), The information (e05-1) is associated with the robot 100 at the location (F05). When you revisit the page, the environment information (e05-2) is acquired. The section 244 not only stores the first environmental information (e05-2) but also the previous The emotion parameters are also changed based on the second environmental information (e05-1). This control method allows the user to change the impression they had in the past at a revisited location to the current impression. The status management unit 244 can reflect the new information acquired by revisiting the The environment map information 420 is updated based on the obtained environment information (e05-2). S24).

[0096] For example, point (F07) is a "noisy point" with a high average volume. (See Figure 8). When the robot 100 revisits the location (F07), In this case, the state management unit 244 determines that the sound is not heard. For example, by increasing the emotional value (curiosity) based on past information such as "points" At the same time, the map management unit 168 manages the location (F07), which is a kind of environmental information. Lowers the average volume of the

[0097] On the other hand, if it is not a revisited point (N in S18), the recognition unit 212 The state management unit 244 determines that the newly acquired key frame 404 and the environmental information The information is registered in the environment map information 420 (S26). The object determination process is executed (S28). Details of the object determination process are shown in FIG. This will be discussed in detail later.

[0098] For example, at time t1, there is a sofa at point (F04), and The sofa is moved, and at time t2, the sofa is at point (F04). In this case, the key frame 404( F04-1) and key frame 404 (F04- 2) may be so different that they cannot be considered identical. The 100th person arrives at location (F04) based on the image memory that a sofa exists. I moved, but the sofa wasn't there, so I found the location (F04) that I had memorized the image of. Even though you have revisited point (F04), you will not be able to Since it cannot be found, the recognition unit 212 has now reached the point of new discovery. (N in S18), and the key frame 404 where the sofa does not exist is Register a new account in the top information 420.

[0099] The map management unit 168 detects key frames 40 that have been acquired for a predetermined time or more. 4 may be deleted to erase the memory of a place that has not been visited for a while. This control method is said to be able to express the forgetting of the robot 100. Both are effective in reducing the amount of data stored in the map storage unit 170. Also, at point (F04) there is an old and useless statement that "there is a sofa" This allows for efficient removal of obsolete information.

[0100] FIG. 12 shows the process in which the robot 100 selects a destination in response to the occurrence of an event. FIG. 10 is a schematic diagram for explaining a method for In FIG. 12, when the robot 100 is at point P5, Event E3 occurs at point P. The recognition unit 212 of the server 200 recognizes the user If you don't remember any users with similar physical characteristics to X6, At this time, the operation control unit 2 of the server 200 determines that the user X6 is an unknown user. 22 sets a location that satisfies the evacuation destination conditions as a movement destination. The condition can be set arbitrarily. Specifically, the probability of appearance of users whose intimacy level is equal to or greater than a predetermined value can be set. It may be a point where the ratio is equal to or greater than a predetermined value, or a point where there is an obstruction such as a ceiling or wall. The operation control unit 222 may be configured to set all the events and the evacuation destination conditions. A location that satisfies the evacuation destination conditions when an event occurs, which is associated in advance. is detected from the environment map information 420. In FIG. Adding points P7 and P8 have been identified.

[0101] The operation control unit 222 sets either the point P7 or the point P8 as the movement target point. In FIG. 12, the operation control section 222 selects the point P5 as the point P6. For example, the point P7 is selected as the destination point. Assume that the detection rate of user X2 is high at this location. The robot 100 moves to point P7 while tracing the image memory. 100 moves to point P7, and if user X2 is detected as expected, Stay at point P7 for a while. User X2 cannot be detected even after moving to point P7. If not, the operation control unit 222 sets the point P7 to the point P8 as the movement destination point. According to this control method, "event E3 is surprising, and the user The robot 100 can be made to express a confused behavior such as "moving to point P7 in search of user X2, but then moving to another safe point P3 when user X2 is not there."

[0102] When a specific event occurs, the operation control unit 222 The state management unit 24 sets a point that satisfies the evacuation destination condition as a movement destination point. 4 changes the emotion parameters depending on the event. For example, high frequency When the voice is detected for a predetermined time or longer, the event E4 is established, and the state management unit 2 44 may increase the emotional value (curiosity) in response to event E4. The information is a numerical representation of the external environment based on the detected values ​​of various sensors. An event is an event that satisfies a predetermined event condition in environmental information. For example, it is a special case of environmental information.

[0103] FIG. 13 is a flowchart showing the process of processing an event. In S14 of FIG. 11, each time the (first) environmental information is detected, the recognition unit 212 determines whether the event condition is met. Events, in other words, events for which (primary) environmental information is set in advance It is executed when one of the following conditions is met:

[0104] When an event is detected, the captured image acquisition unit 162 acquires a surrounding image. In this embodiment, the peripheral image is an image that covers two or more directions. The captured image acquisition unit 162 normally targets one direction. A single image is captured by the camera, and when an event occurs, the robot The surrounding image is acquired by an all-around camera that can take wide-angle shots of the area around the NET 100. That's fine.

[0105] The image feature acquisition unit 164 acquires the surrounding images, in other words, the plurality of captured images. The state management unit 244 extracts image features from each of the events (S32). The emotion parameter is changed in response to the change (S34). The first-order reflection of emotion is the process of determining how emotion parameters are calculated for each event. Whether or not to change the value is set in advance.

[0106] Next, the image feature acquisition unit 164 acquires the object from the image features of the surrounding images. If the object exists (Y in S36), the map management unit 168 changes the degree of association between the event and the object (S38). When an event (E01) is detected, an object (B01) is detected. The relevance of an event (E01) and an object (B01) is determined by the frequency of occurrence. When the event (E01) is detected, the object is frequently If an object (B01) is detected, an event (E0 1) is more likely to be associated. If the object does not exist (N in S36) , S38 is skipped.

[0107] When the event is an evacuation event, in other words, a specific event that requires the user to escape from the current location (Y in S40), the operation control unit 222 A point that satisfies the conditions is set as a movement destination point (S42). If not, the process of S42 is skipped.

[0108] FIG. 14 is a flowchart showing details of the object determination process shown in S28 of FIG. -Chart. The image feature acquisition unit 164 extracts an object from the captured image acquired in S10. The state management unit 244 refers to the image storage and detects the past It is determined whether the same object has been detected before (S52). It is the same as an object previously detected, or more precisely, it is similar to the object When the number of objects detected is equal to or greater than the predetermined value (Y in S52), the state management unit 244 changes the emotion parameters according to the object (S54). This is the "secondary reflection by the event" mentioned above. When a new object is detected (N in S52), S54 skips It will be uploaded.

[0109] The state management unit 244 manages events associated with objects, and Emotion parameters are changed depending on the relevance of the object and the event. For example, if an object (B01) is connected to an event (E03) and an event ( E04) and is associated with an object (B01) and an event ( E03) has a relevance of 20% and the object (B01) and event (E0 Assume that the relevance of event 4) is 3%. In this case, the state management unit 244 changes the emotion value in accordance with the event (E03) with the highest relevance. For example, if event (E03) is an event for which a setting is made to "increase the emotion value (approval request) by 30%, the state management unit 244 increases the emotion value (approval request) by 6% (=30%×20%), taking into account the relevance of 20%. According to this control method, even if the event (E03) has not actually occurred, the emotional parameter is affected because the memory of the event (E03) is stimulated by the object (B01) associated with the event (E03), and the change in the emotional parameter can be expressed as a change in behavioral characteristics.

[0110] As described above, the robot 100 and the robot including the robot 100 are The bot system 300 has been described. Living organisms store environmental information obtained through their sensory organs, especially their eyes, as memories. According to this embodiment, the method is similar to that of living organisms. The external sensor 114 shown as an example of the basic configuration is an essential component. Therefore, the external sensor 114 is installed in the planned range of the robot 100. This eliminates the need for pre-installation work.

[0111] The robot 100 behaves based on the key frame 404. When the internal state of the space to which the robot 100 belongs changes, for example, When a change in the position of an object such as a chair occurs, the robot 100 temporarily On the other hand, it may be possible to develop a new sense of spatial awareness in a new environment. The image memory is updated by acquiring a new frame 404. It is similar to living organisms that are temporarily confused by changes in the environment but eventually adapt to them. Behavioral characteristics can be realized.

[0112] When Robot 100 enters a new space, it moves around and They accumulate memories of the place (sense of territory) and form memories of the place. The frame 404 is stored in the map storage unit 170 of the server 200. The storage unit 170 is formed in a non-volatile recording medium such as a hard disk. It is preferable that the robot 100 does not have to walk around the space and recreate its place memory every time it is powered on.

[0113] The robot 100 changes the emotion parameters according to the environmental information, and The robot 100 changes its behavior characteristics according to the emotional parameters. The (primary) environmental information acquired at the "current location / current time" is not enough. Emotions are also influenced by (secondary) environmental information at the "current location / past point in time." This control method allows the system to control the robot by changing the parameters. Because behavioral characteristics change, changes in emotions based on place memory can be expressed through behavior. In addition, environmental information is stored in association with the feature values ​​obtained from the captured images. Therefore, the robot 100 can move to a place different from the place where it formed its place memory. However, they can also take action by utilizing existing place memories.

[0114] The robot 100 changes the emotion parameters depending on the event. In addition, the robot 100 not only acts as an event itself, but also as a The emotion parameters also change depending on the object being played. If you set an event that has a strong effect on the meter, only the event itself Rather, emotions are influenced by objects that are associated with the event. You can express your situation through actions.

[0115] The present invention is not limited to the above-described embodiment and modifications, and the components can be modified and embodied without departing from the spirit and scope of the present invention. By appropriately combining multiple components disclosed in the embodiments and modifications, Various inventions may be formed. Some components may be removed from the composition element.

[0116] One robot 100, one server 200, and multiple external sensors 114 The robot system 300 is configured as follows. A part of the functions of the server 200 may be realized by the server 200. Some or all of the functions may be assigned to the robot 100. A server 200 may control multiple robots 100, or multiple servers may The robots 200 may cooperate to control one or more robots 100 .

[0117] A third device other than the robot 100 and the server 200 may also be responsible for some of the functions. The collection of the functions of the robot 100 and the functions of the server 200 described in Figures 5 and 6 can be understood as a single "robot" from a broader perspective. How to allocate the multiple functions required to realize the present invention to one or more pieces of hardware can be determined in consideration of the processing capacity of each piece of hardware, the specifications required for the robot system 300, etc.

[0118] As mentioned above, the term "robot in the narrow sense" does not include the server 200. This refers to Robot 100, but "robots in the broad sense" are robot systems. Many of the functions of the server 200 will be handled by robots in the future. It is also possible that they will be integrated into the Top 100.

[0119] The state management unit 244 has been described as being installed in the server 200. The function of the management unit 244 may be installed in the robot 100. The functions of the pop management unit 168 and the object management unit 174 are also implemented in the server 200. It may be implemented in the robot 100 without any need for a sensor.

[0120] The environmental information acquisition unit 166 of the robot 100 acquires the first environmental information. It functions as a "primary environment information acquisition unit" and is a state management unit for the robot system 300. 244 may include a "secondary environmental information acquisition unit" that acquires secondary environmental information. Of course, the function of the secondary environmental information acquisition unit is installed in the robot 100. The first environmental information acquisition unit may acquire the following at the current position P1 of the robot 100: (It is assumed that image feature information R1 is also acquired at the same location.) Acquire information E1 and match image feature information R1 with primary environmental information E1 This image feature information R1 and the primary environment information E1 can be registered. The location information to be added is the location information stored in the robot 100 or the server 200. The secondary environmental information acquisition unit then stores the information in a storage unit (not shown). Image feature information R1 acquired at point P1 and a group of acquired image feature information and detects image feature information R2 whose image similarity is equal to or greater than a predetermined threshold. The secondary environmental information acquisition unit acquires the environmental information associated with the image feature information R2. The information E2 is identified as the second environmental information E2 at the current point P1. The operation control unit 222 may change the behavioral characteristics of the robot 100 located at the point P1 based on both the first environmental information E1 and the second environmental information E2, as described above.

[0121] If image feature information R1 and image feature information R1 are similar, image feature information R1 There is a high possibility that the acquisition point P1 of the image feature information R1 is the same as the acquisition point P2 of the image feature information R2. On the other hand, the image feature information (scenery) of points P1 and P2, which are originally separate points, It is possible that the scenery at the current point P1 happened to look similar. The robot 100 recognizes the location P2 (the location P1) and moves to the location P2 (the location P2) which has a similar landscape to the location P1. Recalling the second environmental information E2 recognized at the other location visited previously, It is possible to realize a process in which the emotion parameters are influenced by the secondary environmental information E2. For example, at point P1 where there is a chair C2 that resembles Dad's chair C1 , recorded using the image of "point P2 at dad's chair C1" that we saw previously. The robot 100 can realize complex behavioral characteristics that stimulate the robot's memory. When the user has a positive image of point P2, the operation control unit 222 By making the robot 100 stay in the robot 100 for a predetermined time or more in the robot 100 0's attachment to chair C1 may be expressed indirectly.

[0122] The second environmental information acquisition unit acquires image features similar to the image feature information R1 of the point P1. If the characteristic information cannot be detected, the captured image acquisition unit 162 is instructed to take a photograph. For example, the robot 100 takes a picture of the front and changes the direction of the key frame. When the camera was capturing the image, the camera angle was shifted 30 degrees to the right. The image feature information R3 may be acquired first. may search for previously acquired image feature information similar to the image feature information R3. The robot 100 changes the direction of the built-in camera or the direction of its body. Alternatively, the key frame 404 may be extracted from the panoramic image captured by the panoramic camera. In this way, by changing the shooting direction as needed, Then, image feature information similar to the current location is searched from the image feature information already acquired. It becomes easier to get it out.

[0123] An estimation model may be prepared that takes the feature values ​​of the captured image as input and outputs the environmental information. , and a neural network with environmental information e1, e2, , eu as output. For example, the feature f1 is The number of included feature points 410, the feature value f2 is the feature value in the key frame 404. The number of lines formed by the points 410 in the substantially vertical direction may be used. , the environmental information e1 is the reception strength, and the environmental information e2 is the detection rate of user X1. The second environmental information acquisition unit acquires information on the current position P1 of the robot 100. The obtained features are input into the estimation model, and the output is taken as the second environmental information. The secondary environmental information output by the estimation model may be obtained from the feature quantity. This is virtual environmental information that is inferred. The robot 100 recognizes the scenery at the point P1 and then performs the following based on the image feature of the point P1. The second environmental information is used to imagine the emotion parameters. This allows for the process of being affected.

[0124] The primary environmental information acquisition unit extracts features from the captured image at the current point P1. When the primary environmental information is acquired, these are used as input and output values. In other words, the prediction model may be updated using the training data.

[0125] When capturing the key frame 404, a TOF (Time-of-Flight) A depth camera such as a stereo camera may also be used. The image captured by the camera includes depth information. The primary environmental information acquisition unit may extract features for each depth range. The image feature information for each image may be registered in association with the first environmental information E1. The depth range may be determined as an absolute distance range from the camera, or It may be determined as a relative distance range based on a certain feature point. For example, if five depth ranges are determined, the image feature information for each of the five depth ranges is The common primary environmental information E1 is registered in association with the image feature acquisition. The unit 164 treats a combination of feature points resulting from the same object as an image feature. The information may be extracted and registered in association with the primary environmental information E1. Therefore, the second environmental information acquisition unit may acquire the second environmental information E2 using the feature amount for each depth range. This makes it easier to identify the visual features that create a certain environment. This increases the probability of removing noise feature points, and the shape of the image is estimated using a prediction model. This will increase the learning effect.

[0126] For example, let's assume that there is a person near the TV T1. The boundary information acquisition unit acquires feature amounts for each depth range and registers multiple point information. This location information mainly includes image feature information consisting of feature points of TV T1. This includes information that corresponds to environmental information indicating the presence of people. At that time, a key frame 404 including TV T2 is captured and a characteristic When the amount is obtained, image feature information consisting mainly of feature points of "TV" is Assume that TV T2 is extracted. There is no one near TV T2. Second environment When the information acquisition unit acquires location information similar to the image feature information, the TV Based on the image feature information of the first image, the second environmental information that "there is a person" is obtained. This means that the place where you experience "there are people near the TV" is actually Even if the location is different, image feature information containing many feature points originating from the TV is If you can get it, you can remember the experience of "there are people there" and They can choose learned behaviors such as staying longer without them.

[0127] In the present embodiment, it has been described that one captured image is periodically acquired, and multiple captured images are acquired when an event occurs. Normally, n images are acquired, and when an event occurs, m images are acquired. (m>n) captured images may be acquired. This is an important event that can have a large impact on the behavioral characteristics of the In such cases, it is desirable to obtain more detailed image information about the surrounding area. By capturing a number of captured images, the object at the time of the event can be identified. When an event occurs, the robot 100 can detect multiple Multiple images may be acquired by pointing the onboard camera in several directions. In some cases, the behavior can also express being surprised by an event and checking the surroundings.

[0128] When an event occurs, a captured image may be acquired with a higher image quality than normal. In this case, the external environment can be recognized in more detail. By enlarging the captured image, you can easily identify small objects from the enlarged image. You may recognize it.

[0129] When an event occurs, feature values ​​may be extracted for each depth to detect the object. Specifically, a depth camera may be used to detect the object in multiple depth directions. The robot 100 may be separated from the robot 100 within a predetermined range, for example, 30 centimeters. The first range is within 1 meter, and the second range is within 30 centimeters to 1 meter. The second range is 1 meter or more, and the third range is 1 meter or more. Even if the similarity between the feature information and the existing image feature information for the first range is determined, In this case, even if they are not similar at a distance, they are similar at a close distance. Since the system judges similarity based on similar image feature information, Even when high-quality captured images are acquired during live capture, image processing can be performed at high speed.

[0130] When an event occurs, the detection sensitivity of various sensors such as a microphone may be increased. Some sensors will not be used and will be used only when an event occurs. By doing so, environmental information at the time of the event occurrence may be enriched. Normally, they act unconsciously, but only when they recognize danger or become interested in it. When recognizing an event, it is important to accurately recognize the surrounding environment in an emergency. When an event occurs, in other words, in an emergency, By improving the detection power of the sensors, we can better recognize the external environment when an event occurs. It becomes easier.

[0131] The captured image acquisition unit 162 captures moving images at all times and periodically extracts images from the moving images. The image feature acquisition unit 164 may extract a still image from the moving image. The recognition unit 212 may extract a key frame 404. In this method, objects are extracted from video images taken over a predetermined period before and after the event occurrence. The project may be detected.

[0132] The object determination process shown in Fig. 14 may be executed at a timing different from that of the update process shown in Fig. 11. When detecting an object, the emotion parameters change. As a way to suppress excessive changes in emotion parameters, The object determination process may be executed less frequently than the process.

[0133] In this embodiment, the causality between an event and an object is defined as the relevance, and the emotion parameter is changed based on the object. In addition to visible objects, other environmental information such as specific sounds and smells was also detected. For example, the scent of perfume and female user X4 By linking the event of going out, when the scent of perfume is detected, The state management unit 244 associates the female user X4 with going out, and the emotional value (loneliness) ) may be increased.

[0134] The object management unit 174 determines the strength of association between an event and an object. The expressed relevance may be decreased over time. According to the law, the association between events and objects weakens over time. This "forgetting" can be expressed in the robot 100. 74 reduces the time decay rate of the relevance when the relevance exceeds a predetermined threshold. According to this control method, the relationship between the event and the object can be When a strong association (degree of relevance) is established, such association is quite In other words, the robot 1 expresses emotions similar to "strong prejudices." It can be expressed in 00.

[0135] Some events may be set as special events. For example, the occurrence of a sound exceeding a predetermined value, such as thunder or an explosion, may be defined as a special event. The object management unit 174 manages the special event and the object that is viewed at the same time. For example, set a certain relevance level for the object, for example, 90(%), and You can give Robot 100 a strong impression of "trauma" In addition, when user X1 hits the device hard enough to cause a malfunction, The object management unit 174 manages the special event of being hit and the user X1( The relevance of the object may be set to 90%. According to this control method, the "dislike" can be expressed by a parameter other than the degree of intimacy.

[0136] The event condition can be set arbitrarily. It is not limited to an external event, but may be an emotional value (loneliness) exceeding a predetermined value. Alternatively, the charge rate may be increased to a predetermined value. Mechanical events such as the following may be recognized as events: When the emotion value (loneliness) reaches a predetermined value or more, the operation control unit 222 The destination of the bot 100 is determined based on the probability of a user having a degree of intimacy of a predetermined value or more appearing. It is also possible to have the robot wander around the room searching for this user, selecting a suitable location as a candidate. stomach.

[0137] The image feature acquisition unit 164 acquires a total of 10 key frames 4 at the point P6. At this time, the location of the 10 key frames 404 is A certain number of key frames 404, for example, seven or more, have common image features. The feature may be registered as a key frame 404 (image feature information) of the point P6. Even with this control method, it is difficult to recognize the external environment, such as documents and clothes. Unstable features that are difficult to use as landmarks, in other words, noise information in position recognition, are effectively used. can be effectively removed.

[0138] The environmental information may be categorized. For example, environmental information about temperature may be classified into two or more categories such as "warm" and "cold." Environmental information is categorized into multiple categories for each location, such as "safe," "dangerous," "fun," and "lonely." According to this control method, key frame 4 Since it is possible to compress the environmental information linked to 04, it is possible to reduce the memory capacity and This reduces the processing load associated with detecting boundary information.

[0139] The strength of the received wireless signal from the server 200 determines the territory of the robot 100. The robot 100 may be located within a communication range with the server 200. If you leave the server, you will no longer be able to receive support from the server 200. The control unit 222 refers to the object information 430 and determines whether the reception strength is B or higher. A set of locations may be set as the actionable range. This can prevent the robot 100 from going out of the communication range.

[0140] The robot 100 may set its territory based on the strength of the received signal from other devices, not limited to the server 200. For example, when the external sensor 114 is installed, In this case, the operation control unit 222 may perform wireless communication with any of the external sensors 114. The range in which the robot 100 can move may be set as the range in which the robot 100 can move. periodically communicates with the server 200, and when the reception strength falls below a predetermined value, The user may restrict his / her own behavior so as not to leave the server 200 any further. On the other hand, when a specific event occurs, such as welcoming a user, the operation control unit 222 may set a movement target point even if the reception strength is low.

[0141] The robot 100 is periodically charged by a charging device (not shown). During the power outage, the server 200 may suppress communication with the robot 100. When the robot 100 starts charging with the charging device, the communication unit 142 of the robot 100 starts charging. When charging is completed, a charging completion signal is sent to the server 200. When the server 200 receives the charging start signal, the server 200 You may stop communication with the robot 100 until a charging completion signal is received from the robot 100. Alternatively, while the robot 100 is being charged, the server 200 and the robot The frequency of periodic communication with the server 100 may be reduced. According to the report, by suppressing communication while the robot 100 is charging and not in motion, The power consumption associated with communication between the robot 100 and the server 200 can be reduced.

[0142] The present invention can be applied to other than the robot 100. For example, In the motion system, the idea of ​​location memory using key frames 404 is It is possible to memorize the scenery by periodically capturing images of the outside world. Create a map using the navigation system. The system uses environmental map information to create walking paths that the user has never been to before. You can suggest a walking route that includes a spot where you have had a pleasant experience. You can suggest walking routes that avoid areas where you have had dangerous experiences. You may also suggest a solution.

[0143] The environment map or the environment map information is what is called a "map" The environmental map information only needs to be information that associates geographic information with environmental information, and does not need to be expressed visually. The information may be a key frame 404 or various geographic landmarks. It may be an object.

[0144] In this embodiment, the key frame 404 is associated with the environmental information, and the key Environmental information associated with the frame 404 (records related to the key frame 404) Based on the memory, the emotional parameters are changed (secondary reflection), and It has been explained that the behavioral characteristics of the robot 100 are changed based on the above.

[0145] Alternatively, an environmental memory may be formed by associating an object with environmental information. Based on the environmental information (memory about the object) associated with the object, The behavioral characteristics may be changed by changing the

[0146] The robot 100 changes the emotion parameters based on the type of object. The behavioral characteristics may be changed based on the change in the emotion parameter.

[0147] In addition, the robot 100's behavior in a room (space) containing multiple objects The robot will memorize the impression and adjust the room based on the impression memory (described later). 100 emotional parameters can be changed, and behavioral characteristics can be adjusted based on the emotional parameters. It may be changed.

[0148] In the following, we will consider the change of emotion parameters based on the memory of the room, the change of emotion parameters based on the environmental information associated with the object, and The change in the behavioral characteristics of the robot 100 that accompanies this will be described as a modified example.

[0149] The image feature acquisition unit 164 of the robot 100 acts as an "object detection unit." As described above, the image feature acquisition unit 164 may acquire a plurality of types of objects. For each object, multiple objects are created that define the shape and color characteristics in advance. It has an object pattern and generates an object based on the object pattern. For example, the image feature acquisition unit 164 may perform image recognition on the object. Based on the feature point distribution of the object (B01), The image feature acquisition unit 164 may determine that the object (B01) is a "television" based on the feature point distribution of another object (B04). The image feature acquisition unit 164 or the recognition unit 212 can recognize that the object (B01) and the object (B04) belong to the same category, "television", even though they are different objects. The image feature acquisition unit 164 is a neural network that receives the feature point distribution as input. The object category may be determined by a model.

[0150] Below, we will refer to objects as unique objects such as object (B01). When referring to a specific object, use the term "unique object" and the term "type of object" such as a television. When referring to a specific object, it is called an "object category." Objects are identified by their object IDs, and object categories are identified by their category IDs. When there is no particular distinction, they are simply called "objects." Boo.

[0151] The robot 100 detects the unique object by a method other than image recognition. For example, the robot 100 may ,Object detection by communicating with unique objects. The category ID can be transmitted by the home appliance. In this case, the object detector detects the category ID sent from the home appliance. By doing so, the object category of home appliances (unique objects) is It may be judged. In the following modified example, the object detection unit of the robot 100 detects the object. Described as detecting objects and identifying object categories do.

[0152] FIG. 15 is a diagram showing the data structure of the object environment information 440 in the modified example. be. The object environment information 440 is stored in the object information storage unit 172. The object environment information 440 includes an object ID, a category ID, The object is associated with the environment information and the room ID. The Object ID identifies the unique object, and the Category ID indicates the object category. For example, if a unique object (B01) and a unique object (B04) are both "televisions," the Category IDs of these two unique objects will be the same.

[0153] When the object detection unit detects a unique object, it identifies the object ID and the category ID. The object management unit 174 acquires the environment information when the object is detected. ,Based on the environment information when detecting unique objects, object environment information 4 40. The updating of the object environment information 440 is explained in relation to FIG. The method is basically the same as the one described above.

[0154] The recognition unit 212 detects the room as a closed space by detecting the walls and doors. Each room is identified by a room ID. The unique objects detected by the room ID are managed by the room ID. Room ID=R01 indicates the "living room" and Room ID=R02 indicates the "study." According to Figure 15, the unique object (B01: TV) is It is installed in a room (R01: living room), and the unique object (B04: TV) It is installed in a room (R02: Study). For example, in a room (R01: Living room) There is a 70% chance that one of the TVs (B01) installed near the TV The user is detected, and the TV (B0 4) It is remembered that almost no users have been detected near do. Below, the object category (C01) is television, Object Category (C02) is a printer, and Object Category (C03) is a table. In this example, we will assume that the object category (C04) is a sofa.

[0155] FIG. 16 is a diagram showing the data structure of room impression information 450 in a modified example. In this modification, the data storage unit 206 of the server 200 further The room impression information 450 stores the room impression information 450. The room impression information 450 stores the room (room ID ), category ID and impression memory are associated. The "impression" is information that represents the impression of the robot 100 that the user has, and is expressed as a numerical value (hereinafter referred to as "impression memory value"). It is an index that increases or decreases depending on the external and internal state of the robot. Parameters, the amount of movement of each drive unit, speaker volume, etc. can be managed as numerical values. It is determined based on available parameters.

[0156] For example, in the living room, Robot 100 has many opportunities to interact with its owner. The space available for movement is also large, so you can move around freely. The space is smaller than a living room, and there are fewer opportunities for interaction. If we set an index based on the amount of movement of the moving parts of the bot, The exponent of the amount of movement in the living room is larger than the exponent of the amount of movement in the bedroom.

[0157] If we set an index in terms of speaker output, the owner and There are many opportunities to interact and speak, so the speaker index is It will be larger than the speaker index in the bedroom.

[0158] If we set an index based on the pleasure that 100 robots receive from their owners, In the living room, babies are more likely to be held, and in the bedroom, babies are more likely to be held than in the living room. Therefore, the pleasure index is higher in the living room than in the bedroom. The index is defined and the actual values ​​of the parameters that form the basis of the defined index are calculated as parts. By compiling data for each room under specific conditions over a certain period of time, trends for each room can be analyzed. These indices are managed as impression memories.

[0159] For example, the robot 100 is active in the living room and has many opportunities to speak. When you experience a lot of pleasure and a lot of pleasure, The values ​​of "activity index," "speaker index," and "pleasure index" are high. Based on these results, it can be said that "in a room like a living room, it is okay to move around actively." The impression that "it is okay to make noise and that they often receive pleasurable acts" is formed. can be.

[0160] These indices also change depending on the time of day. For example, during the daytime on weekdays, On weekday nights, you may receive less pleasure than on weekends. In this case, the time period is weekday afternoon (for example, 11:00 to 13:00). As a result, the impression memory value changes between weekday afternoons and weekday evenings. Figure 16 shows the index of pleasurable actions among these impression memories.

[0161] When an event that occurs in a room (R01: living room) is a pleasant action, state management The section 244 increases the impression memory value of the room (R01: living room). When the behavior is an unpleasant behavior, the state management unit 244 changes the impression of the room (R01: living room). This control method reduces the memory value. The robot 100's impression of the room (RO) is expressed numerically. 1: Living room) has an impression memory value of "+10", so the robot 100 R01: They have a positive impression of the living room. For example, when Robot 100 is in the living room, it gives a good impression to the owner. Based on the experience of often receiving pleasurable acts such as being held, To be nurtured.

[0162] When the owner scolded him in the room (R02: Study) (disturbing behavior), Robo The impression memory value for Room 100 (R02: Study) decreases. For example, when the robot 100 is in a room (R02: study), the management unit 244 If no owner can be found for a certain period of time, for example, 10 minutes, When the room was not in use, the impression memory value for the room (R02: study) decreased. In this way, you can not only see events that occur in the room, but also events that occur in the room. The impression memory value is calculated based on the frequency of events and the absence of events. It may be changed.

[0163] In this modification, the robot 100 recognizes that the room (R01) is a "living room." The recognition unit 212 of the robot 100 does not recognize the room. The group is identified by multiple object categories contained in the room, and Assign a room ID to the

[0164] According to FIG. 16, in the room (R01: living room), there is a TV (C01), Table (C03) and sofa (C04) are detected. Robot 100 The room (R01: A high impression memory value for the room (R01: living room) means that the robot 100 has a positive impression of the room, which satisfies the conditions of including a television (C01), a table (C03), and a sofa (C04). The robot 100 uses environmental information obtained in the room, In particular, we aim to change the impression memory value of a room depending on the events experienced in the room. The robot 100 selects a room as a target of the impression memory value from a plurality of rooms included in the room. Identify by object category.

[0165] Assuming that the room impression information 450 shown in FIG. 16 is used, the robot 100 In a room containing a TV (C01), a table (C03), and a sofa (C04) The state management unit 244 manages the state of the robot 100 when it first enters the room. (Hereinafter, the room where the robot 100 is present will be referred to as the “occupied room”) The system analyzes the room based on multiple object categories contained in the room. For example, if the occupied room is a known room (R0 If the room is similar to 1), the state management unit 244 stores the impression of the room (R01). The emotion parameters of the robot 100 are changed based on the memory value. It may be the same as the room (R01), or it may happen to have a TV (C01), Another common feature is that it includes a table (C03) and a sofa (C04). This control method can be used to control a TV, a table, or any other device. The robot 100 has a positive impression of the room (R01: living room), Even when visiting an unknown home for the first time, You can feel happy when you find it. Based on the impression it makes, the robot 100 changes its emotion when it enters an unknown room. When you are in a room similar to a known room, The emotion management unit 244 may decrease the emotion parameter value indicating loneliness. When you are in a room that does not resemble any of the known rooms, the state manager The logic unit 244 increases the emotional parameter indicating loneliness and the emotional parameter indicating curiosity. The parameters may also be increased.

[0166] For example, the state management unit 244 of the robot 100 may When entering a room similar to or identical to one's own, a feeling of loneliness is felt. You can also lower the emotion parameter. The higher the impression memory value, the more "loneliness" you feel. The emotional parameter may be significantly reduced.

[0167] For example, the index of the amount of movement is defined as impression memory, and the index of the amount of movement in a room (R01: living room) is the index of the amount of movement in a room (R02: study). If the index of the amount of activity in the study is larger than the index of the amount of activity in the living room, In this case, the robot 100 moves in a room corresponding to a study. This can be suppressed more effectively than when you are in the corresponding room.

[0168] In summary, the robot 100 is able to adapt to the influence of various external environments in each room. While receiving the influence, the emotional parameters and movement amount change in a complex manner. Such behavioral results are recorded as various indices. Based on this, impression memories such as "being able to move actively" and "being likely to receive pleasurable behavior" are formed. Then, the robot 100 recognizes the room when it is similar to a known room. The robot changes its behavioral characteristics based on the impression memory of the known room. The behavioral performance based on experience influences actual behavioral characteristics. "Dynamic characteristics" are realized.

[0169] As mentioned above, the room is a combination of multiple object categories. The state management unit 244 identifies all unique information detected in the occupied room. There is no need to identify rooms based on objects.

[0170] The object management unit 174 manages each unique object based on the environment information. Specifically, the object management unit 174 The environment information when a unique object is detected is the event condition (described above). When this condition is satisfied, the impression memory value of the unique object is changed according to the event. For example, when a pleasurable activity is detected near a television (B01), the object Even if the impression memory value for the television (B01) is increased, the project management unit 174 In other words, when a specific event occurs, the object manager 174 identifies unique objects within a predetermined range from the robot 100. The impression memory value for that unique object is changed. The determination of whether or not the object is within the predetermined range may be made by measuring the distance of each object, or may be made based on the size of the object included in the captured image. It may be estimated based on the above.

[0171] The object manager 174 also manages the objects that are in front of the robot 100. At the same time, unique objects within a predetermined range are identified and the unique objects are You can also change the impression memory value for the object. Part 174 is not a unique object (B01) but an object category. For example, an impression memory value may be set for the TV surroundings. Since there are people around, events such as being held will occur more frequently. By setting an impression memory value for the category "television," Even if the location is different, the behavior of moving closer to the TV based on memory is You can choose.

[0172] The object management unit 174 manages a plurality of unique objects detected in the occupied room. The occupied room may be identified based on some unique objects among the objects. For example, the object management unit 174 may select objects with particularly high impression memory values. Lowest, for example, the top 3 unique objects and the bottom 6 Then, these six unique objects Identify the object category for each object and · Occupied rooms may be identified as a combination of categories.

[0173] The state management unit 244 manages the object categories detected in the occupied room and , corresponding to each of the multiple rooms registered in the room impression information 450 The state management unit 244 compares the object categories assigned to the object. Similarity may be determined according to the number of matching object categories. 2 out of 3 object categories included in the room (R01: living room) When one of the rooms is detected in the occupied room, the state management unit 244 detects the occupied room and the room (R The similarity of the living room (01) may be determined to be 66% (= 2 ÷ 3). This means that the room occupied is likely to be the same as the known room (R01: living room). The state management unit 244 manages the object categories detected in the occupied room. The room impression information 450 may be compared with the room impression information 450, and the room with the highest similarity, ie, a similarity of 70(%) or more, may be determined to be the same type of room as the occupied room.

[0174] FIG. 17 shows the process when an object is detected in the modified example. This is a flowchart. The process shown in FIG. 17 is periodically executed in the robot system 300. The environmental information acquisition unit 166 first acquires environmental information about the room where the user is present (S6 0). The state management unit 244 changes the emotion parameters based on the environmental information. This is the same as the "first-order reaction based on environmental information" explained in relation to FIG. The object detection unit detects unique objects ( At this time, the object detection unit detects the object of the unique object. Identify the project category.

[0175] Compare the detected object categories with the room impression information450 By doing so, the status management unit 244 identifies the type of room in which the user is present (S66). As described above, the state management unit 244 manages the objects detected in the occupied room. The room category group and the object for each room registered in the room impression information 450 The rooms with similarity to the object category group above a certain value are classified as "similar to known rooms." The room is identified as a "known room" (hereinafter simply referred to as a "known room").

[0176] When the room is a "known room," the state management unit 244 The emotional parameters are changed based on the impression memory values ​​that are assigned in advance. For example, the impression memory value for the room (R01: living room) is set to "+1" (S68). When "0" is assigned, and the room resembles a living room, The state management unit 244 calculates the emotion parameter value indicating loneliness and the emotion parameter value indicating curiosity. The parameter value may be decreased. In this way, a known room, more precisely For example, we can create a system that corresponds to a set of object categories in a known room. Based on the impression memory value, the feeling (emotion parameter) of the robot 100 is calculated. This is the same as the "environment data" described in relation to FIG. This is similar to "secondary reflection based on boundary information."

[0177] On the other hand, if the room is unknown (N in S66), secondary reflection is not performed. Thereafter, the state management unit 244 executes the rule based on the environmental information acquired in S62. The room impression information 450 is updated (S70).

[0178] The robot 100 senses the room based on the impression memory value of a known room similar to the room in which the robot is currently located. Change the emotion parameter and change the behavioral characteristics based on the change in the emotion parameter. In addition, the robot 100 also calculates emotion parameters based on the impression memory value. Behavioral characteristics may be changed without using a meter. For example, impression memory In a room with a low value, the movement control unit 150 of the robot 100 100 movement speed, movement amount (distance that can be traveled in a unit time), momentum ( The amount or range of movement of the various actuators may be restricted. In a room with a low memory value, the operation control unit 150 According to this control method, the impression can be improved. In a room with a low memory value, the robot 100 becomes quiet, and the impression memory value The robot 100 behaves actively in a high-rise room. Based on this, the robot 100 can express its feelings through actions. The impression storage value may be initially set.

[0179] When the robot 100 enters a room for the first time, it The room impression information 450 is then referred to for the judgment. Then, the object environment information 44 is used to determine the behavior of the object, such as its movement speed. By referring to 0, you can identify objects in the room based on past experience. Explore and set it as your destination.

[0180] Also, when the user is in a house for the first time or feels lonely, the user searches for a "fun room (a room that gives a good impression)." Set a room with a high memory value as the target location.

[0181] The motion control unit 222 controls the room including the object category with a high impression memory value. For example, the robot 100 may search for a room containing a television, a table, and a sofa. In this case, the robot 100 Imagine a room with a TV, a table, and a sofa as a "fun room." Find a room that meets these conditions.

[0182] The movement control unit 222 of the robot 100 refers to the room impression information 450, Next, the operation control unit 222 identifies the room ID of the room with the highest impression memory value. Identify multiple object categories contained in the identified room. The production control unit 222 controls the plurality of objects included in this "fun room". Contains a predetermined number of object categories. Select the room as the destination point.

[0183] The robot 100 can select a room as a destination point based on experience. For example, Owner A takes the robot 100A to User B's house. The house of user B is unknown to the robot 100A. First, the set 100A receives room impression information 450 formed in the owner A's house. Based on this, we identified object categories contained in rooms with high impression memory values. Then, the robot 100 determines the object categories. For example, search for a room similar to room (R01: living room) in user B's house. When the robot 100 finds the same room RB in the user B's house, it Enter B and calculate the emotion parameters and memory values ​​based on the impression memory value of the room (R01: living room). This control method changes the experience at home. Based on this, the robot 100 will behave in the same way as at home even when visiting a new house for the first time. It is possible.

[0184] The movement control unit 222 of the robot 100 further determines whether the robot 100 should approach the object in the room. The operation control unit 222 selects a unique object. Based on the associated environmental information, the approach is performed in the same manner as in this embodiment. For example, the robot 100 may select a unique object that indicates loneliness. When the emotion parameter value is high, the user's detection rate is high. The state management unit 244 manages the environment of the unique object. Emotion parameters may be changed based on the environment information. When you are near a sofa where the event of being hit by a robot is likely to occur, Even if you are not actually hugged, the emotional parameter that indicates loneliness will decrease. In addition to changing the emotion parameters and behavioral characteristics of the robot 100 based on the environmental information associated with the key frame 404, the emotion parameters and behavioral characteristics of the robot 100 may be changed based on the environmental information associated with an object (unique object or object category). An impression memory value may be set in advance for the object category. When the robot 100 detects a unique object, the robot 100 may change the behavioral characteristics based on the impression memory value associated with the object category.

[0185] The room with a good impression includes a television, a table, and a sofa. After entering the room, the operation control unit 222 moves the robot 100 closer to the television. According to this control method, the robot 100 can control the robot based on impression memory. Find a "fun room" and after entering the room, choose an object that makes a good impression. As a result, people tend to like the living room or Robot 100's preferences for spaces and objects, such as liking television This can be expressed through actions.

[0186] Similarly, the robot 100 may avoid rooms with low impression memory values. For example, when Robot 100 enters the bathroom, it is chased away by its owner. When any of the unpleasant behaviors is detected, the robot 100 For example, the impression memory value for the combination of towels, washstands, washing machines, etc. As a result, some people have trouble using the bathroom or washing machine. It can express behavioral characteristics.

[0187] Impression memories are not limited to pleasant / unpleasant. For example, lively / quiet, hot / cold The impression memory value may be set based on other evaluation axes such as the room (R 01: When the impression memory of "bustling" is associated with the living room When the emotional parameter indicating "loneliness" increases, the robot may set the room (R01: living room) as the destination point. 1: Living room) is a place where people tend to gather, so this control method will help prevent loneliness. When they were in a room, they approached the room (R01: living room) as if searching for someone. The unique object does not need to be a fixed object, but may be a portable object such as a towel, or may be an object that can move autonomously such as a pet or another robot 100.

[0188] FIG. 18 is a data structure diagram of behavior experience information 460 in another modified example. . The object information storage unit 172 may store the behavior experience information 460. The behavior experience information 460 associates a category ID with a motion ID and an impression memory. For example, suppose that the robot 100 is praised by the owner when it selects a motion with a motion ID=M01 within a predetermined range from an object (B01: television). Since being praised is a pleasant behavior, the object management unit 174 increases the impression memory value for the combination of the object category (C01: television) and the motion (M01). By doing so, the behavioral experience information 460 is updated. When I selected motion (M03) near the Levi, I was scolded by the owner. Since being scolded is an unpleasant behavior, the object management unit 174 Object Category (C01: Television) and Motion (M03) This reduces the impression memory value of the combination.

[0189] The action control unit 222 refers to the action experience information 460 and determines whether the motion control unit 222 is near the television. Increases the selection probability of Motion (M01) and Motion (M03). This control method allows the user to give praise or scold the child. The behavioral characteristics of the robot 100 can be changed by the actions taken. For example, by scolding Robot 100 for sitting near the cupboard, This can stop the robot 100 from sitting near the cupboard. As a result, the user can "discipline" the robot 100. do.

[0190] The state of the object can also be taken into account when changing the impression memory. For example, when the vacuum cleaner is off, nothing happens when you approach the vacuum cleaner. If the cat approaches the vacuum cleaner while it is sleeping, it will be chased away by the owner. Being chased away by the owner is an unpleasant behavior, so the robot 100 learns a behavioral norm that "do not approach the vacuum cleaner when it is on." Whether the vacuum cleaner is on or off may be determined based on whether or not the vacuum cleaner makes a sound. In addition to the state of the object, the time of day may also be taken into consideration when changing the impression memory. For example, if the robot 100 enters the kitchen while the owner is preparing dinner, the owner may chase the robot 100 away (an unpleasant behavior). As a result, the robot 100 can learn a behavioral norm that "do not approach the kitchen where the refrigerator and cupboards are located in the evening (do not disturb the owner in the kitchen)."

[0191] Even if you associate impression memories with the combination of room (room ID) and motion, In this case, for example, it is permissible to make loud noises in the living room, but not in the study. Robot 100 learns the code of conduct that says it should not make loud noises. Similarly, you can move around quickly in the living room, but not in the study. You may also learn that you need to limit your activity level.

[0192] When the robot 100 recognizes the unpleasant behavior, the robot 100 notifies the owner who performed the unpleasant behavior. However, if the intimacy level is above a certain value, When the intimacy management unit 220 recognizes the unpleasant behavior of the user, the intimacy management unit 220 does not decrease the intimacy. Alternatively, the decrease in the degree of intimacy may be smaller than that in normal times. According to this control method, the operator can build a strong relationship of trust with the robot 100. Nurse can now scold Robot 100 without worry.

[0193] As described above, an object may be associated with environmental information, or a space (room ID) may be associated with environmental information. The impression memory may be changed based on information, especially events. Alternatively, the impression memory for the space may be initialized. If a high impression memory value is set for the living room at the beginning, the robot 100 will This can express the behavioral characteristics of people who prefer living rooms. The impression memory value for the living room is transmitted to the server 200 via a mobile terminal such as a smartphone. The object management unit 174 then stores the specified impression storage value as room impression information. You can also set it to 450.

[0194] It is also possible to associate environmental information with a combination of object categories. For example, "The TV is by the window, and the sofa is placed a little away from the window. The room may be identified as a "room where the

[0195] The plurality of robots 100 acquires the environment map information 420, the object information 430, room impression information 450, and behavioral experience information 460 are shared. According to this control method, when the robot 100A prefers the living room, , robot 100B also prefers the living room, and the behavior of multiple robots 100 Characteristics can be linked.

[0196] Robot 100A only selects rooms with high impression memory values ​​as fun rooms. For example, the impression memory of a plurality of rooms may be shared with the bot 100B. Rooms with higher than average values ​​or rooms with impression memory values ​​in the top three Alternatively, the room impression information 450 may be shared. The target is rooms with lower than average values ​​or rooms with impression memory values ​​in the bottom three. The room impression information 450 may be shared as a group.

[0197] Robot 100A and robot 100B target all rooms. There is no need to share impression information 450, and room information can be shared only for some rooms. The impression information 450 may be shared. According to such a control method, for example, The robot 100A transmits information about some rooms to the robot 100B. To prevent this, they create a secret room that only robot 100A likes. It is possible.

[0198] The room where Robot 100A was first held by Owner A, Owner A For rooms with special conditions (memories), such as a room where the owner holds the baby for a long time, In some cases, the robot 100A may not share information with the robot 100B.

[0199] The server 200 may visualize the object arrangement and the object. Impression map generation generates an image (impression map) that shows the impression memory of the object. The communication unit 204 of the server 200 may transmit the impression map to the owner's mobile terminal. For example, the area around an object with a high impression memory value may be displayed in blue, and the area around an object with a low impression memory value may be displayed in red. By doing so, the owner can determine what object or You can visually see what kind of room you like.

[0200] The impression map generating unit may display a change in the impression memory value for an object or a room on the impression map. For example, if a user initially dislikes an object, the user may change the impression memory value. The way in which the user gradually comes to like the product may be shown by displaying a moving impression map.

Claims

1. A memory unit; a processor; a drive mechanism that performs the motion selected by the processor; The processor: Recognizing an impression memory for an object detected within or near the object and in response to event information related to the user; The robot determines a motion for the object based on the impression memory.

2. A memory unit; a processor; a drive mechanism that performs the motion selected by the processor; The processor: Recognize impression memories of objects, when a first object is detected, storing a first impression memory for the first object in the storage unit; When a second object is detected, a second impression memory that is an impression memory of the second object is recognized based on the similarity between the second object and the first object and the first impression memory; The robot is configured to determine a motion relative to the second object based on the second impression memory.

3. A memory unit; a processor; a drive mechanism that performs the motion selected by the processor; The processor: Recognizing an impression memory value based on event information within or near the object; determining a motion for the object based on the impression memory value; Displaying a map showing the space and displaying objects present in the space on the map; When the impression memory value is relatively high, the robot determines a format for displaying each object to be different from a format when the impression memory value is relatively low.

Images