Autonomous mobile body, control method, information processing device, and information processing method

Abstract

This technology pertains an autonomous moving body, a control method, an information processing device, and an information processing method capable of improving the level of satisfaction of a user with respect to an interaction with an autonomous mobile body. The autonomous moving body comprises: a recognition unit that recognizes a physical contact operation of a user on the basis of first information relating to the state of the autonomous moving body and / or a surrounding state; and an operation control unit that controls execution of a promotion reaction, which is a reaction for further prompting the user to perform the contact operation, or a suppression reaction, which is a reaction for preventing the user from performing the contact operation. The present technology can be applied to, for example, a robot for entertainment.

Description

Autonomous moving body, control method, information processing device, and information processing method

[0001] The present technology relates to an autonomous moving body, a control method, an information processing device, and an information processing method, and particularly to an autonomous moving body, a control method, an information processing device, and an information processing method that are configured to improve user satisfaction with interactions with the autonomous moving body.

[0002] Conventionally, robots have been proposed that execute interactions according to the location, strength, and manner of touch, the person touching, the robot's emotions, and the frequency of touch (see, for example, Patent Document 1).

[0003] International Publication No. 2020 / 158642

[0004] In the case of an autonomous mobile body such as the robot described in Patent Document 1, it is desirable to improve the user's satisfaction with the interaction with the autonomous mobile body.

[0005] The present technology has been made in light of such circumstances, and aims to improve user satisfaction with interactions with autonomous moving bodies.

[0006] An autonomous moving body according to a first aspect of the present technology includes a recognition unit that recognizes a physical contact action of a user based on first information relating to at least one of its own state and the state of the surrounding area, and an operation control unit that controls the execution of a promotion reaction, which is a reaction that further encourages the user to make the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action.

[0007] The control method of the first aspect of the present technology is such that an autonomous moving body recognizes a user's physical contact action based on information regarding at least one of its own state and the state of the surrounding area, and controls the execution of a promotion reaction, which is a reaction that further encourages the user to make the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action.

[0008] An information processing device according to a second aspect of the present technology includes a recognition unit that recognizes a physical contact action of a user based on information regarding at least one of the state of an autonomous moving body and the state of the surrounding area of ​​the autonomous moving body, and an operation control unit that controls the autonomous moving body to execute a promotion reaction, which is a reaction that further encourages the user to perform the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action.

[0009] In an information processing method according to a second aspect of the present technology, an information processing device recognizes a user's physical contact action based on information regarding at least one of the state of an autonomous moving body and the state of the area surrounding the autonomous moving body, and controls the autonomous moving body to execute a promotion reaction, which is a reaction that further encourages the user to perform the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action.

[0010] In a first aspect of the present technology, a user's physical contact action is recognized based on information regarding at least one of the user's own condition and the condition of the surroundings, and the execution of a promotion reaction, which is a reaction that further encourages the user to make the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action, is controlled.

[0011] In a second aspect of the present technology, a user's physical contact action is recognized based on information regarding at least one of the state of the autonomous moving body and the state of the surrounding area of ​​the autonomous moving body, and the execution by the autonomous moving body of a promotion reaction, which is a reaction that further encourages the user to perform the contact action, or a suppression reaction, which is a reaction that causes the user to suppress the contact action, is controlled.

[0012] 1 is a block diagram showing an embodiment of an information processing system to which the present technology is applied. FIG. 1 is a front view of an autonomous moving body. FIG. 2 is a rear view of the autonomous moving body. FIG. 3 is a perspective view of the autonomous moving body. FIG. 4 is a side view of the autonomous moving body. FIG. 5 is a top view of the autonomous moving body. FIG. 6 is a bottom view of the autonomous moving body. FIG. 7 is a schematic diagram for describing the internal structure of the autonomous moving body. FIG. 8 is a schematic diagram for describing the internal structure of the autonomous moving body. FIG. 9 is a block diagram showing an example of the functional configuration of the autonomous moving body. FIG. 10 is a block diagram showing an example of the functional configuration implemented by a processing unit of the autonomous moving body. FIG. 11 is a block diagram showing an example of the functional configuration of an information processing terminal. FIG. 12 is a block diagram showing an example of the functional configuration of an information processing server. FIG. 13 is a flowchart for describing interaction control processing. FIG. 14 is a flowchart for describing details of contact recognition processing. FIG. 15 is a diagram schematically showing a relationship between the waveform of IMU time series data and the content of utterances of the autonomous moving body. FIG. 16 is a flowchart for describing details of contact recognition model learning processing. FIG. 17 is a diagram schematically showing an example of the waveform of IMU time series data during contact movements of each user. FIG. 18 is a diagram showing an example of the configuration of a computer.

[0013] Hereinafter, embodiments of the present technology will be described. The description will be made in the following order: 1. Embodiment 2. Modification 3. Other

[0014] <<1. Embodiment>> An embodiment of the present technology will be described with reference to FIGS. 1 to 19 .

[0015] <Configuration Example of Information Processing System 1> FIG. 1 is a block diagram showing an embodiment of an information processing system 1 to which the present technology is applied.

[0016] The information processing system 1 includes autonomous mobile bodies 11-1 to 11-n, information processing terminals 12-1 to 12-n, and an information processing server 13.

[0017] Hereinafter, when there is no need to distinguish between the autonomous mobile bodies 11-1 to 11-n, they will simply be referred to as the autonomous mobile body 11. Hereinafter, when there is no need to distinguish between the information processing terminals 12-1 to 12-n, they will simply be referred to as the information processing terminals 12.

[0018] Communication is possible between each autonomous mobile body 11 and the information processing server 13, between each information processing terminal 12 and the information processing server 13, between each autonomous mobile body 11 and each information processing terminal 12, between each autonomous mobile body 11, and between each information processing terminal 12 via the network 21. In addition, direct communication is also possible between each autonomous mobile body 11 and each information processing terminal 12, between each autonomous mobile body 11, and between each information processing terminal 12 without going through the network 21.

[0019] The autonomous mobile object 11 is an information processing device that performs autonomous operation without being controlled by the information processing terminal 12 and the information processing server 13, or by being controlled by the information processing terminal 12 or the information processing server 13. The autonomous mobile object 11 is also an agent device that enables interaction with a user (e.g., conversation, physical contact, etc.) to be realized more naturally and effectively.

[0020] The autonomous mobile body 11 is an information processing device that recognizes its own and its surrounding situations based on collected sensing data, etc., and autonomously selects and executes various actions according to the situation. Unlike a robot that simply acts in accordance with user instructions, one of the features of the autonomous mobile body 11 is that it autonomously executes appropriate actions according to the situation.

[0021] The autonomous mobile body 11 can, for example, perform user recognition, object recognition, etc. based on captured images, and perform various autonomous actions according to the recognized user, object, etc. The autonomous mobile body 11 can also, for example, perform voice recognition based on the user's speech, and perform actions based on the user's instructions, etc.

[0022] Furthermore, the autonomous mobile body 11 performs pattern recognition learning to acquire the ability to recognize users and objects. In this case, the autonomous mobile body 11 can perform pattern recognition learning related to objects, etc., not only by supervised learning based on given learning data, but also by dynamically collecting learning data based on instructions from a user, etc.

[0023] Furthermore, the autonomous moving body 11 can be disciplined by a user. Here, the discipline of the autonomous moving body 11 is broader than, for example, general discipline in which the autonomous moving body 11 is taught rules and prohibited actions and made to memorize them, and refers to changes in the autonomous moving body 11 that the user can sense as a result of the user's interaction with the autonomous moving body 11.

[0024] Furthermore, the autonomous mobile body 11 can express its own state and communicate with a user or other autonomous mobile bodies by outputting output sounds. The output sounds of the autonomous mobile body 11 include operation sounds that are output in response to the state of the autonomous mobile body 11, and speech sounds for communicating with a user, other autonomous mobile bodies, etc.

[0025] The shape, capabilities, desires, and other levels of the autonomous mobile body 11 can be designed as appropriate depending on the purpose and role. For example, the autonomous mobile body 11 is configured as an autonomous mobile robot that autonomously moves within a space and performs various actions.

[0026] Specifically, for example, the autonomous mobile body 11 may be composed of various types of robots, such as a running type, a walking type, a flying type, a swimming type, etc. For example, the autonomous mobile body 11 may be composed of an autonomous mobile robot such as an entertainment robot that has a shape and behavioral capabilities that imitate those of an animal such as a human or a dog. For example, the autonomous mobile body 11 may be composed of a vehicle or other device that has the ability to interact with a user.

[0027] The information processing terminal 12 is, for example, a smartphone, a tablet terminal, a PC (personal computer), or the like, and is used by the user of the autonomous mobile body 11. The information processing terminal 12 realizes various functions by executing a predetermined application program (hereinafter simply referred to as an application). For example, the information processing terminal 12 manages and customizes the autonomous mobile body 11 by executing the predetermined application.

[0028] For example, the information processing terminal 12 communicates with the information processing server 13 via the network 21 or directly with the autonomous mobile body 11 to collect various data related to the autonomous mobile body 11, present the data to the user, or give instructions to the autonomous mobile body 11.

[0029] The information processing server 13, for example, collects various types of data from each autonomous mobile body 11 and each information processing terminal 12, provides various types of data to each autonomous mobile body 11 and each information processing terminal 12, and controls the behavior of each autonomous mobile body 11. Furthermore, for example, the information processing server 13 can perform pattern recognition learning and processing corresponding to user discipline, similar to the autonomous mobile body 11, based on the data collected from each autonomous mobile body 11 and each information processing terminal 12. Furthermore, for example, the information processing server 13 supplies each information processing terminal 12 with the above-mentioned applications and various types of data related to each autonomous mobile body 11.

[0030] The network 21 may be composed of, for example, public networks such as the Internet, telephone networks, and satellite communication networks, various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks). The network 21 may also include dedicated network such as an IP-VPN (Internet Protocol-Virtual Private Network). The network 21 may also include wireless communication networks such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0031] The configuration of the information processing system 1 can be flexibly changed depending on the specifications, operation, etc. For example, the autonomous mobile body 11 may communicate information with various external devices in addition to the information processing terminal 12 and the information processing server 13. The above external devices can include, for example, servers that transmit weather, news, and other service information, and various home appliances owned by the user.

[0032] Furthermore, for example, the autonomous mobile bodies 11 and the information processing terminals 12 do not necessarily have to have a one-to-one relationship, and may have, for example, a many-to-many, many-to-one, or one-to-many relationship. For example, one user can use one information processing terminal 12 to check data related to multiple autonomous mobile bodies 11, or can use multiple information processing terminals to check data related to one autonomous mobile body 11.

[0033] <Configuration Example of Autonomous Mobile Body 11> Next, a configuration example of the autonomous mobile body 11 will be described with reference to FIGS. 2 to 11. The autonomous mobile body 11 can be various devices that perform autonomous operation based on environmental recognition. In the following, a case will be described in which the autonomous mobile body 11 is an agent-type robot device with an oblong shape that travels autonomously on wheels. The autonomous mobile body 11 performs autonomous operation according to, for example, the user, the surroundings, and its own situation, thereby realizing various interactions including information presentation. The autonomous mobile body 11 is, for example, a small robot that is large and heavy enough to be easily lifted by a user with one hand.

[0034] <Examples of Exterior of Autonomous Moving Body 11> First, examples of exterior of the autonomous moving body 11 will be described with reference to FIGS. 2 to 7 .

[0035] Fig. 2 is a front view of the autonomous mobile body 11, and Fig. 3 is a rear view of the autonomous mobile body 11. Figs. 4A and 4B are perspective views of the autonomous mobile body 11. Fig. 5 is a side view of the autonomous mobile body 11. Fig. 6 is a top view of the autonomous mobile body 11. Fig. 7 is a bottom view of the autonomous mobile body 11.

[0036] 2 to 6, the autonomous moving body 11 has eye units 101L and 101R corresponding to the left and right eyes on the upper part of the main body. The eye units 101L and 101R are realized by, for example, a single or two independent OLEDs (Organic Light Emitting Diodes), LEDs, etc., and can express gaze, blinking, etc.

[0037] The autonomous moving body 11 also includes cameras 102L and 102R above the eye units 101L and 101R. The cameras 102L and 102R have the function of capturing images of the user and the surrounding environment. In this case, the autonomous moving body 11 may implement simultaneous localization and mapping (SLAM) based on the images captured by the cameras 102L and 102R.

[0038] The eye 101L, the eye 101R, the camera 102L, and the camera 102R are disposed on a substrate (not shown) disposed inside the exterior surface. The exterior surface of the autonomous mobile body 11 is basically formed using an opaque material, but a head cover 104 made of a transparent or translucent material is provided in the portion corresponding to the substrate on which the eye 101L, the eye 101R, the camera 102L, and the camera 102R are disposed. This allows the user to recognize the eye 101L and the eye 101R of the autonomous mobile body 11, and the autonomous mobile body 11 can capture images of the outside world.

[0039] 2, 4, and 7, the autonomous moving body 11 is equipped with a ToF (Time of Flight) sensor 103 at the bottom of the front face. The ToF sensor 103 is configured, for example, by 1D-ToF, and has the function of detecting the distance to an object or the ground in front of the autonomous moving body 11. The ToF sensor 103 can, for example, accurately detect the distance to various objects, detect steps, etc., and prevent the autonomous moving body 11 from falling or tipping over.

[0040] 3, 5, etc., the autonomous moving body 11 is provided on its rear surface with a connection terminal 105 for an external device and a power switch 106. The autonomous moving body 11 can connect to an external device via the connection terminal 105 and perform information communication, for example.

[0041] 7, the autonomous mobile body 11 is provided with wheels 107L and 107R on its bottom surface. The wheels 107L and 107R are driven by different motors (not shown). This allows the autonomous mobile body 11 to perform moving operations such as moving forward, backward, turning, and rotating.

[0042] The wheels 107L and 107R can be stored inside the main body and can be protruded to the outside. For example, the autonomous mobile body 11 can perform a jumping motion by forcefully protruding the wheels 107L and 107R to the outside. Note that FIG. 7 shows a state in which the wheels 107L and 107R are stored inside the main body.

[0043] In the following, when there is no need to distinguish between the eye unit 101L and the eye unit 101R, they will simply be referred to as the eye unit 101. In the following, when there is no need to distinguish between the camera 102L and the camera 102R, they will simply be referred to as the camera 102. In the following, when there is no need to distinguish between the wheel 107L and the wheel 107R, they will simply be referred to as the wheel 107.

[0044] <Example of Internal Structure of Autonomous Moving Body 11> FIGS. 8 and 9 are schematic diagrams showing the internal structure of the autonomous moving body 11. FIG.

[0045] 8, the autonomous mobile body 11 includes an IMU 121 and a communication device 122 arranged on an electronic board. The IMU 121 detects three-dimensional acceleration and angular velocity of the autonomous mobile body 11. The communication device 122 is configured to realize wireless communication with the outside, and includes, for example, a Bluetooth or Wi-Fi antenna.

[0046] The autonomous moving body 11 also includes a speaker 123, for example, inside the side surface of the main body. The autonomous moving body 11 can output various sounds using the speaker 123.

[0047] 9 , the autonomous mobile body 11 is provided with a microphone 124L, a microphone 124M, and a microphone 124R inside the upper part of the main body. The microphones 124L, 124M, and 124R collect the user's speech and surrounding environmental sounds. By providing the autonomous mobile body 11 with multiple microphones 124L, 124M, and 124R, it is possible to collect sounds generated in the surrounding area with high sensitivity and detect the position of the sound source.

[0048] 8 and 9, the autonomous mobile body 11 is equipped with motors 125A to 125E (however, motor 125E is not shown). Motor 125A and motor 125B, for example, drive a substrate on which the eye unit 101 and camera 102 are disposed in the vertical and horizontal directions. Motor 125C realizes a forward tilt posture of the autonomous mobile body 11. Motor 125D drives wheel 107L. Motor 125E drives wheel 107R. Motors 125A to 125E enable the autonomous mobile body 11 to express a wide range of movements.

[0049] In the following description, when it is not necessary to distinguish between the microphones 124L to 124R, they will be simply referred to as microphones 124. In the following description, when it is not necessary to distinguish between the motors 125A to 125E, they will be simply referred to as motors 125.

[0050] 10 shows an example of the functional configuration of the autonomous mobile body 11. The autonomous mobile body 11 includes an input unit 201, a processing unit 202, an output unit 203, and a communication unit 204.

[0051] The input unit 201 includes, for example, buttons and switches such as the power switch 106 described above, and detects physical input operations by the user.

[0052] The input unit 201 includes various sensors that collect various data related to the autonomous mobile body 11 and the situation around the autonomous mobile body 11. For example, the input unit 201 includes the above-described camera 102, ToF sensor 103, IMU 121, and microphone 124. The input unit 201 may also include various sensors such as various optical sensors such as a touch sensor 211, a thermosensor 212, a geomagnetic sensor, and a humidity sensor. The input unit 201 supplies sensor data output from each sensor to the processing unit 202.

[0053] The processing unit 202 has a function of controlling each component included in the autonomous mobile body 11 and causing the autonomous mobile body 11 to execute various processes. The processing unit 202 has a central processing unit (CPU) 221, a graphics processing unit (GPU) 222, a dynamic random access memory (DRAM) 223, and a flash memory 224.

[0054] The CPU 221 mainly executes general-purpose processing among various types of processing of the autonomous moving body 11 .

[0055] The GPU 222 mainly executes image processing and machine learning-related processes among various processes of the autonomous moving body 11 .

[0056] The DRAM 223 temporarily stores various programs and data required for processing by the autonomous moving body 11 .

[0057] The flash memory 224 stores various programs and data necessary for the processing of the autonomous moving body 11 .

[0058] The output unit 203 has a function of outputting various expressions of the autonomous moving body 11. The output unit 203 includes an OLED 231 and a drive unit 232 in addition to the speaker 123 described above.

[0059] The OLED 231 constitutes the eye unit 101 described above and represents the eye movement of the autonomous moving body 11 .

[0060] The drive unit 232 includes, for example, the wheels 107 and the motor 125 described above, and is used to express the body movement of the autonomous mobile body 11 .

[0061] The communication unit 204 includes, for example, the above-mentioned connection terminal 105 and communication device 122, and communicates with the information processing terminal 12, the information processing server 13, and other external devices.

[0062] <Example of Functional Configuration of Processing Unit 202> FIG. 11 shows an example of a functional configuration realized by the processing unit 202 (for example, the CPU 221 and the GPU 222) of the autonomous moving body 11 executing a predetermined control program.

[0063] The information processing unit 251 includes a recognition unit 261 , an action planning unit 262 , an action control unit 263 , and a learning unit 264 .

[0064] The recognition unit 261 has a function of recognizing the user and environment around the autonomous mobile body 11 and various information related to the autonomous mobile body 11 based on sensor data supplied from each sensor of the input unit 201.

[0065] For example, the recognition unit 261 performs user identification, recognition of the user's facial expression and gaze, recognition of the user's state and actions, object recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, recognition of stimuli to the autonomous mobile body 11, etc.

[0066] The user's actions to be recognized include, for example, actions (hereinafter referred to as contact actions) of physically contacting the autonomous moving body 11. The contact actions include, for example, a poke action, a wipe action, a pitch action, and a roll action.

[0067] The poke action is, for example, an action of poking the main body of the autonomous moving body 11 with a finger or the like.

[0068] The wiping operation is, for example, an operation of wiping the head cover 104 of the autonomous moving body 11 with a cloth or the like.

[0069] The pitch movement is, for example, a movement of holding the autonomous moving body 11 and swinging it up and down (shaking it up and down).

[0070] The roll motion is, for example, a motion in which the autonomous moving body 11 is held and swung left and right around an axis (roll axis) in the front-to-back direction (shaking in the roll direction).

[0071] For example, the recognition unit 261 recognizes emotions related to the user's voice, understands words, recognizes the position of a sound source, etc. For example, the recognition unit 261 recognizes the ambient temperature, the presence of moving objects, the state and movement of the autonomous moving body 11, etc.

[0072] For example, the recognition unit 261 recognizes a device that is combined with the autonomous moving body 11 (hereinafter referred to as a combined device).

[0073] Examples of the autonomous mobile body 11 being combined with a combination device include when one of the autonomous mobile body 11 and the combination device is attached to the other, when one of the autonomous mobile body 11 and the combination device rides on the other, when the autonomous mobile body 11 and the combination device are combined, etc. Examples of the combination device include parts that can be attached to and detached from the autonomous mobile body 11 (hereinafter referred to as optional parts), a vehicle on which the autonomous mobile body 11 can ride (hereinafter referred to as a boarding vehicle), and a device to which the autonomous mobile body 11 can be attached and detached (hereinafter referred to as an attachment device).

[0074] Possible optional parts include, for example, parts that resemble parts of an animal's body (e.g., eyes, ears, nose, mouth, beak, horns, tail, wings, etc.), costumes, mascot costumes, parts that extend the functions and capabilities of the autonomous mobile body 11 (e.g., medals, weapons, etc.), wheels, caterpillar tracks, etc. Possible ride-on mobile bodies include, for example, cars, drones, robot vacuum cleaners, etc. Possible attachment devices include, for example, combined robots composed of multiple parts including the autonomous mobile body 11.

[0075] The combined device does not necessarily have to be a device dedicated to the autonomous moving body 11, but may be, for example, a general-purpose device.

[0076] The recognition unit 261 also has a function of estimating and understanding the environment and situation in which the autonomous moving body 11 is placed, based on the recognized information. In this case, the recognition unit 261 may perform comprehensive situation estimation using environmental knowledge stored in advance.

[0077] The recognition unit 261 supplies data indicating the recognition result to the action planning unit 262 , the operation control unit 263 , and the learning unit 264 .

[0078] The behavior planning unit 262 sets an operation mode that defines the operation of the autonomous mobile body 11 based on the recognition result by the recognition unit 261, for example, the recognition result of the combined device by the recognition unit 261. The behavior planning unit 262 also has a function of planning an action to be taken by the autonomous mobile body 11 based on, for example, the recognition result by the recognition unit 261, the operation mode, and learning knowledge. Furthermore, the behavior planning unit 262 executes the behavior plan using, for example, a machine learning algorithm such as deep learning. The behavior planning unit 262 supplies data indicating the operation mode and the behavior plan to the operation control unit 263.

[0079] The operation control unit 263 controls the operation of the autonomous mobile body 11 by controlling the speaker 123, the OLED 231, and the drive unit 232 based on the recognition result by the recognition unit 261, the action plan by the action planning unit 262, and the operation mode. For example, the operation control unit 263 causes the autonomous mobile body 11 to move while maintaining a forward-leaning posture, or to perform forward and backward movement, turning movement, rotational movement, etc. For example, the operation control unit 263 causes the autonomous mobile body 11 to actively perform an inducement action that induces interaction between the user and the autonomous mobile body 11. For example, the operation control unit 263 controls the output of various output sounds. The operation control unit 263 supplies control information related to the operation being performed by the autonomous mobile body 11 to the recognition unit 261.

[0080] The operation control unit 263 controls the output sound from the speaker 123 based on the recognition result by the recognition unit 261, the action plan by the action planning unit 262, and the operation mode. For example, the operation control unit 263 sets a control method for the output sound based on the operation mode, etc., and controls the output sound (for example, control of the content of the output sound to be generated and the output timing of the output sound, etc.) based on the set control method. Then, the operation control unit 263 generates output sound data for outputting the output sound and supplies it to the speaker 123.

[0081] In addition, the operation control unit 263 transmits information regarding the operation of the autonomous mobile body 11 (e.g., the operation history of the autonomous mobile body 11) to the information processing terminal 12 and the information processing server 13 via the communication unit 204 and, if necessary, the network 21.

[0082] The learning unit 264 generates learning data for a learning model (hereinafter referred to as the contact recognition model) used to recognize the user's contact action based on the sensor data supplied from each sensor of the input unit 201 and the recognition results of the user's contact action by the recognition unit 261.

[0083] The contact recognition model is a learning model that recognizes a user's contact action based on, for example, information about at least one of the state of the autonomous mobile body 11 and the state of the surroundings of the autonomous mobile body 11. Below, an example will be described in which the contact recognition model recognizes a user's contact action based on sensor data from the IMU 121 (hereinafter referred to as IMU data).

[0084] As will be described later, after recognizing the contact action using the contact recognition model, the recognition unit 261 corrects the recognition result based on information regarding at least one of the state of the autonomous mobile body 11 and the state of the surroundings of the autonomous mobile body 11, which information is different from the information used in the contact recognition model.

[0085] The learning unit 264 stores the generated learning data in the learning data storage unit 254 .

[0086] The learning unit 264 performs machine learning such as deep learning using the learning data stored in the learning data storage unit 254, and generates a contact recognition model formed by a DNN (deep neural network) or the like. The learning unit 264 supplies the generated contact recognition model to the recognition unit 261.

[0087] The contact history DB 252 accumulates a history of the user's contact actions recognized by the recognition unit 261 (hereinafter referred to as contact history).

[0088] The contact history includes, for example, the date and time when the contact action was recognized (i.e., the date and time when the contact action was performed), the type of the contact action, the manner of the contact action, and information regarding the state of the autonomous moving body 11 in response to the contact action. The manner of the contact action includes, for example, at least one of the duration, strength, and pattern of the contact action.

[0089] The recognition unit 261 transmits the contact history stored in the contact history DB 252 to the information processing server 13 via the communication unit 204 and the network 21 as necessary.

[0090] The dirt DB 253 stores dirt level determination criteria, which are data that serve as standards for determining the dirt level indicating the dirtiness of the head cover 104. The dirt level determination criteria include, for example, data indicating the dirtiness of the head cover 104 corresponding to each dirt level.

[0091] The learning data storage unit 254 stores learning data used for learning the contact recognition model.

[0092] The learning unit 264 transmits the learning data stored in the learning data storage unit 254 to the information processing server 13 via the communication unit 204 and the network 21 as necessary.

[0093] <Example of Functional Configuration of Information Processing Terminal 12> FIG. 12 shows an example of the functional configuration of the information processing terminal 12. As shown in FIG.

[0094] The information processing terminal 12 includes an input unit 301 , a communication unit 302 , an information processing unit 303 , an output unit 304 , and a storage unit 305 .

[0095] The input unit 301 includes an input device for a user to input data, instructions, etc. For example, the input unit 301 includes a touch panel, buttons, switches, etc.

[0096] The communication unit 302 communicates with the autonomous mobile body 11 and the information processing server 13 via the network 21 , and also communicates directly with the autonomous mobile body 11 without going through the network 21 .

[0097] The information processing unit 303 includes a mobile object control unit 311 , an output control unit 312 , and an information generation unit 313 .

[0098] The mobile object control unit 311 includes a recognition unit 321, a behavior planning unit 322, and a motion control unit 323. The recognition unit 321, the behavior planning unit 322, and the motion control unit 323 have the same functions as the recognition unit 261, the behavior planning unit 262, and the motion control unit 263 of the autonomous mobile object 11. In other words, the recognition unit 321, the behavior planning unit 322, and the motion control unit 323 can perform various processes in place of the recognition unit 261, the behavior planning unit 262, and the motion control unit 263 of the autonomous mobile object 11.

[0099] This allows the information processing terminal 12 to remotely control the autonomous moving body 11 , and the autonomous moving body 11 can perform various operations under the control of the information processing terminal 12 .

[0100] The output control unit 312 controls the output of various types of information (for example, visual information, auditory information, tactile information, etc.) from the output unit 304.

[0101] The information generation unit 313 generates various types of information and transmits the information to the autonomous moving body 11 or the information processing server 13 as necessary.

[0102] The output unit 304 includes an output device that outputs various types of information (for example, visual information, auditory information, tactile information, etc.) For example, the output unit 304 includes a display device, a speaker, a haptic element, etc.

[0103] The storage unit 305 stores program data and the like necessary for processing by the information processing terminal 12 .

[0104] <Example of Functional Configuration of Information Processing Server 13> FIG. 13 shows an example of the functional configuration of the information processing server 13. As shown in FIG.

[0105] The information processing server 13 includes a communication unit 401 , an information processing unit 402 , a contact history DB 403 , a stain DB 404 , a learning data accumulation unit 405 , and a storage unit 406 .

[0106] The communication unit 401 communicates with the autonomous moving body 11 and the information processing terminal 12 via the network 21 .

[0107] The information processing unit 402 includes a mobile object control unit 411 and a learning unit 412 .

[0108] The mobile object control unit 411 includes a recognition unit 421, a behavior planning unit 422, and a motion control unit 423. The recognition unit 421, the behavior planning unit 422, and the motion control unit 423 have the same functions as the recognition unit 261, the behavior planning unit 262, and the motion control unit 263 of the autonomous mobile object 11. In other words, the recognition unit 421, the behavior planning unit 422, and the motion control unit 423 can perform various processes in place of the recognition unit 261, the behavior planning unit 262, and the motion control unit 263 of the autonomous mobile object 11.

[0109] This allows the information processing server 13 to remotely control the autonomous mobile body 11, and the autonomous mobile body 11 can perform various operations under the control of the information processing server 13.

[0110] The learning unit 412 receives learning data from each autonomous mobile body 11 via the network 21 and the communication unit 401, and stores the data in the learning data storage unit 405. The learning unit 412 generates a contact recognition model using the learning data stored in the learning data storage unit 405. The learning unit 412 transmits the generated contact recognition model to each autonomous mobile body 11 via the communication unit 401 and the network 21.

[0111] The contact history DB 403 accumulates the contact history received from each autonomous moving body 11 .

[0112] The dirt DB 404 stores criteria for determining the degree of dirtiness of the head cover 104 of the autonomous moving body 11 .

[0113] The learning data storage unit 405 stores the learning data received from each autonomous moving body 11 .

[0114] The storage unit 406 stores various programs and data required for the processing of the information processing server 13 .

[0115] <Interaction Control Processing> Next, the interaction control processing executed by the autonomous moving body 11 will be described with reference to the flowchart of FIG.

[0116] This process starts, for example, when the power supply of the autonomous moving body 11 is turned on, and ends when the power supply is turned off.

[0117] In step S1, the autonomous moving body 11 executes contact recognition processing.

[0118] The contact recognition process will now be described in detail with reference to the flowcharts of FIGS.

[0119] In step S21, the recognition unit 261 recognizes the state of the autonomous moving body 11 based on the time-series data of the IMU 121.

[0120] For example, the DARM 223 temporarily stores the IMU data supplied from the IMU 121 for a predetermined period (for example, 2 seconds) and erases old data as appropriate.

[0121] For example, the recognition unit 261 recognizes the state of the autonomous mobile body 11 based on a predetermined rule, based on the IMU data for a predetermined most recent period (hereinafter referred to as IMU time-series data) stored in the DRAM 223. For example, the recognition unit 261 recognizes whether the state of the autonomous mobile body 11 is a laydown state, an upside down state, or another state.

[0122] The laid-down state is, for example, a state in which the autonomous moving body 11 is laid down on an installation surface (for example, a floor surface, the ground, etc.).

[0123] The upside down state is, for example, a state in which the autonomous moving body 11 is upside down.

[0124] In step S22, the recognition unit 261 determines whether the recognition result is in the Laydown state. If it is determined that the recognition result is not in the Laydown state, the process proceeds to step S23.

[0125] In step S23, the recognition unit 261 determines whether the recognition result indicates an upside down state. If it is determined that the recognition result does not indicate an upside down state, the process proceeds to step S24.

[0126] In step S24, the recognition unit 261 executes a contact action recognition process using a contact recognition model. Specifically, the recognition unit 261 inputs the IMU time-series data to the contact recognition model, and recognizes the user's contact action with respect to the autonomous moving body 11 based on the output value output from the contact recognition model. The recognition result of the contact action is classified into, for example, a poke action, a wipe action, a roll action, a pitch action, and no contact action.

[0127] In step S25, the recognition unit 261 determines whether the image from the camera 102 is dark. Specifically, the recognition unit 261 acquires image data from the camera 102L and the camera 102R. For example, if the average value of the luminance of any of the image data is less than a predetermined threshold, the recognition unit 261 determines that the image from the camera 102 is dark, and the process proceeds to step S26.

[0128] In step S26, the recognition unit 261 determines whether the recognition result is a Roll motion. If it is determined that the recognition result is a Roll motion, the process proceeds to step S27.

[0129] In step S27, the recognition unit 261 corrects the recognition result to the Wipe operation.

[0130] For example, the characteristics of the IMU time-series data for a roll operation and a wipe operation may be similar, and the roll operation and the wipe operation may be erroneously recognized based on the IMU time-series data alone.

[0131] In contrast, when a wipe operation is being performed, that is, when the head cover 104 is being wiped, there is a high possibility that the image from the camera 102 will become dark. On the other hand, when a roll operation is being performed, there is a low possibility that the head of the autonomous moving body 11 will be grasped, and therefore there is a high possibility that the image from the camera 102 will not become dark.

[0132] Therefore, if the image from the camera 102 is dark and the recognition result is a Roll operation, it is corrected to a Wipe operation.

[0133] Thereafter, the process proceeds to step S28.

[0134] On the other hand, if it is determined in step S26 that the recognition result is not a Roll motion, the process of step S27 is skipped and the process proceeds to step S28.

[0135] Also, in step S25, for example, if the average brightness value of all image data is greater than or equal to a predetermined threshold, the recognition unit 261 determines that the image from the camera 102 is not dark, and the processing of steps S26 and S27 is skipped, and the processing proceeds to step S28.

[0136] In step S28, the recognition unit 261 determines whether or not the autonomous mobile body 11 is in the inverted pendulum mode. For example, the recognition unit 261 determines whether or not the autonomous mobile body 11 is in contact with the ground, based on sensor data from the ToF sensor 103. If the autonomous mobile body 11 is in contact with the ground, the recognition unit 261 determines whether or not balance control is being performed to maintain the autonomous mobile body 11 in an upright state like an inverted pendulum, based on control information from the operation control unit 263. If the autonomous mobile body 11 is performing balance control, the recognition unit 261 determines that the autonomous mobile body 11 is in the inverted pendulum mode, and the process proceeds to step S29.

[0137] In step S29, the recognition unit 261 determines whether the recognition result is a wipe operation, a roll operation, or a pitch operation. If it is determined that the recognition result is a wipe operation, a roll operation, or a pitch operation, the process proceeds to step S30.

[0138] In step S30, the recognition unit 261 corrects the recognition result without a contact action. For example, when a wipe action, a roll action, or a pitch action is being performed, it is highly likely that the user is holding the autonomous mobile body 11, and it is unlikely that the autonomous mobile body 11 will be in inverted pendulum mode. Therefore, if the recognition result is a wipe action, a roll action, or a pitch action and the autonomous mobile body 11 is in inverted pendulum mode, it is highly likely that the recognition result is incorrect. Therefore, the recognition result is corrected without a contact action.

[0139] Thereafter, the process proceeds to step S33.

[0140] On the other hand, if it is determined in step S28 that the recognition result is not any of the wipe, roll, and pitch operations, the process of step S30 is skipped and the process proceeds to step S33.

[0141] Also, in step S28, if the autonomous mobile body 11 is not in contact with the ground or if the autonomous mobile body 11 is not performing balance control, the recognition unit 261 determines that it is not in inverted pendulum mode, and the processing proceeds to step S31.

[0142] In step S31, the recognition unit 261 determines whether the recognition result is a poke gesture. If it is determined that the recognition result is a poke gesture, the process proceeds to step S30.

[0143] In step S32, the recognition unit 261 corrects the recognition result without a contact motion. For example, if a poke motion is being performed, there is a high possibility that the autonomous mobile body 11 will be in inverted pendulum mode. Therefore, if the recognition result indicates a poke motion but is not in inverted pendulum mode, there is a high possibility that the recognition result is incorrect. Therefore, the recognition result is corrected without a contact motion.

[0144] Thereafter, the process proceeds to step S33.

[0145] On the other hand, if it is determined in step S31 that the recognition result is not a poke motion, the process of step S32 is skipped and the process proceeds to step S33.

[0146] In step S33, the recognition unit 261 determines whether the wiping operation has been completed. For example, if a predetermined time has elapsed since the recognition of the wiping operation ended, the recognition unit 261 determines that the wiping operation has been completed, and the process proceeds to step S34.

[0147] In step S34, the recognition unit 261 detects the degree of soiling of the head cover 104. For example, the recognition unit 261 detects the degree of soiling of the head cover 104 based on the image data of the camera 102 and the soiling degree determination criteria stored in the soiling DB 253.

[0148] Thereafter, the process proceeds to step S35.

[0149] On the other hand, in step S33, for example, if the recognition unit 261 recognizes a wiping operation, or if a predetermined time has not elapsed since the recognition of the wiping operation ended, it determines that the wiping operation has not been completed, the process of step S34 is skipped, and the process proceeds to step S35. Also, in step S33, for example, if a predetermined time has passed since the recognition unit 261 last recognized a wiping operation, it determines that a wiping operation has not been performed, the process of step S34 is skipped, and the process proceeds to step S35.

[0150] In step S35, the recognition unit 261 determines whether the recognition result is a poke gesture. If it is determined that the recognition result is a poke gesture, the process proceeds to step S36.

[0151] In step S36, the recognition unit 261 recognizes the risk of falling. For example, the recognition unit 261 detects the distance to the installation surface (e.g., the ground, floor, etc.) on which the autonomous mobile body 11 is placed, based on sensor data from the ToF sensor 103. If the distance to the installation surface is equal to or greater than a predetermined threshold, for example, if the installation surface cannot be recognized, the recognition unit 261 determines that the autonomous mobile body 11 is approaching a cliff or the like, and that there is a high risk of the autonomous mobile body 11 falling if the user pokes it. On the other hand, if the distance to the installation surface is less than the predetermined threshold, the recognition unit 261 determines that there is a low risk of the autonomous mobile body 11 falling if the user pokes it.

[0152] Thereafter, the process proceeds to step S37.

[0153] On the other hand, if it is determined in step S35 that the recognition result is not a poke motion, the process of step S36 is skipped and the process proceeds to step S37.

[0154] In step S37, the recognition unit 261 updates the contact history as necessary. For example, if the currently recognized contact action has not been recognized within a predetermined period of time immediately before, the recognition unit 261 adds information about the currently recognized contact action to the contact history.

[0155] On the other hand, for example, if the currently recognized contact action has been recognized within a predetermined period immediately before, the recognition unit 261 does not add information about the currently recognized contact action to the contact history. That is, for example, even if the same contact action is recognized within a predetermined period after a certain contact action has been recognized, information about that contact action is not added to the contact history. This prevents information about the same contact action from being added multiple times to the contact history when the same contact action occurs consecutively or intermittently.

[0156] In step S38, the learning unit 264 accumulates learning data as necessary. For example, when the contact movement has ended and the reliability of the recognition result of the contact movement is high, the learning unit 264 generates learning data. The learning data includes, for example, IMU data while the contact movement is being recognized and a label indicating the type of the contact movement.

[0157] A case where the reliability of the contact action recognition result is high is, for example, a case where the user clearly indicates the type of contact action to be performed before performing the contact action. Specifically, for example, this is the case where the user says, "I'll wipe it for you," before wiping the head cover 104 of the autonomous moving body 11. In this case, the reliability of the recognition result that the contact action is a wipe action is high.

[0158] The learning unit 264 stores the generated learning data in the learning data storage unit 254 .

[0159] Then, the contact recognition process ends.

[0160] 14 , in step S2, the autonomous mobile body 11 determines whether or not the conditions for executing a reaction are satisfied. Specifically, the recognition unit 261 supplies information indicating the recognition result of the contact action on the autonomous mobile body 11 to the action planning unit 262.

[0161] The behavior planning unit 262 determines whether the conditions for executing a reaction are met based on, for example, the type of recognized contact action, the manner in which the contact action was performed, the state of the autonomous moving body 11 in response to the contact action, and the history of the contact action.

[0162] The manner of the contact action includes, for example, at least one of the duration, strength, and pattern of the contact action.

[0163] The history of the contacting action includes, for example, at least one of the cumulative number of times and frequency of the contacting action.

[0164] If it is determined that the conditions for executing the reaction are not met, the process returns to step S1.

[0165] Thereafter, in step S2, the processes of steps S1 and S2 are repeatedly executed until it is determined that the conditions for executing a reaction are met.

[0166] On the other hand, if it is determined in step S2 that the conditions for executing the reaction are met, the process proceeds to step S3.

[0167] In step S3, the autonomous moving body 11 executes a reaction.

[0168] Specifically, the behavior planning unit 262 plans the type and manner of reaction to be executed based on at least one of the type of contact motion, the manner of the contact motion, the state of the autonomous moving body 11 in response to the contact motion, and the history of the contact motion. The behavior planning unit 262 supplies data indicating the type and manner of reaction to the operation control unit 263.

[0169] The operation control unit 263 controls at least one of the speaker 123, the OLED 231, and the drive unit 232 based on data indicating the type and manner of reaction, thereby causing the autonomous moving body 11 to execute a reaction.

[0170] Examples of reactions are described below. The reactions include, for example, reactions that express the feelings or thoughts of the autonomous mobile body 11 (hereinafter referred to as emotional reactions), reactions that encourage the user to make a further contact action (hereinafter referred to as promotion reactions), and reactions that encourage the user to suppress a contact action (hereinafter referred to as suppression reactions).

[0171] For example, when the autonomous moving body 11 recognizes a poke action, that is, when it is being poked with a finger or the like, it makes a sound like "Hya!" This is one type of emotional reaction.

[0172] At this time, for example, the autonomous moving body 11 may change its reaction depending on the degree of risk of falling.

[0173] For example, when the autonomous moving body 11 is close to a cliff and there is a high risk of falling (for example, when the risk is equal to or greater than a predetermined threshold), the autonomous moving body 11 may emit a voice such as "It's dangerous" to urge the user to refrain from making a poke action. This is a type of inhibition reaction.

[0174] On the other hand, for example, when the autonomous mobile body 11 is far from a cliff and the risk of falling is low (for example, when the risk is less than a predetermined threshold), the autonomous mobile body 11 may emit a voice prompting the user to perform a poke action, such as "play more." This is a type of emotional reaction and prompting reaction.

[0175] For example, when the autonomous moving body 11 recognizes a wipe operation, that is, when the head cover 104 is being wiped with a cloth or the like, it emits a voice such as "It feels good! Wipe more!" to encourage the user to further wipe the head cover 104. This is a type of prompting reaction.

[0176] At this time, for example, the autonomous moving body 11 may change its reaction depending on the strength of wiping the head cover 104. For example, Fig. 17 schematically shows the relationship between the waveform of the IMU time-series data and the content of the speech of the autonomous moving body 11.

[0177] For example, when the strength with which the autonomous moving body 11 wipes the head cover 104 is appropriate, for example, when the amplitude of the waveform of the IMU time-series data is less than a predetermined threshold, the autonomous moving body 11 utters a voice saying, "It feels good!" This is a type of emotional reaction.

[0178] On the other hand, for example, if the strength of wiping the head cover 104 is too strong, for example, if the amplitude of the waveform of the IMU time-series data is equal to or greater than a predetermined threshold, the autonomous mobile body 11 emits a voice prompting the user to reduce the strength of wiping, saying, "Isn't that a bit too strong?" This is a type of emotional reaction and suppression reaction.

[0179] Furthermore, for example, the autonomous moving body 11 may change its reaction depending on the time it takes to wipe the head cover 104 .

[0180] For example, when the head cover 104 continues to be wiped, the autonomous moving body 11 utters a voice saying, "Thank you!" This is a type of emotional reaction.

[0181] On the other hand, if the autonomous moving body 11 takes too long to wipe the head cover 104, it utters a voice saying, "Isn't that enough?" to encourage the user to stop wiping the head cover 104. This is a type of emotional reaction and suppression reaction.

[0182] Furthermore, for example, the autonomous moving body 11 may change its reaction based on the degree of dirt on the head cover 104 after the wiping operation.

[0183] For example, if the head cover 104 is still dirty (for example, if the degree of dirt is equal to or greater than a predetermined threshold), the autonomous moving body 11 emits a voice prompting the user to further wipe the head cover 104, such as "Wipe more" or "It's still dirty." This is a type of prompting reaction.

[0184] On the other hand, for example, when the head cover 104 is clean (for example, when the degree of dirt is less than a predetermined threshold), the autonomous moving body 11 emits voices such as "It's clean now," "Thank you!", "That's enough," etc., to encourage the user to refrain from wiping the head cover 104. This is a type of emotional reaction and inhibition reaction.

[0185] For example, when the autonomous moving body 11 recognizes a pitch motion, that is, when the autonomous moving body 11 is being swung up and down, it utters a voice saying, "Salt and pepper!" This is a type of emotional reaction.

[0186] For example, when the autonomous moving body 11 recognizes a roll motion, that is, when the autonomous moving body 11 is being swung in the roll direction, it makes a sound saying "shake shake." This is a type of emotional reaction.

[0187] For example, the autonomous moving body 11 may be configured to emit voices such as "This is the XXth time!" or "You've wiped me XX times!" based on the cumulative number of contact actions. For example, the autonomous moving body 11 may be configured to emit voices such as "You wipe me well" or "It's been a while," based on the frequency of each contact action. These are types of emotional reactions.

[0188] Thereafter, the process returns to step S1, and the processes from step S1 onwards are executed.

[0189] In this way, the user's satisfaction with the interaction with the autonomous moving body 11 is improved. That is, the autonomous moving body 11 reacts appropriately to the user's physical contact, making the interaction with the user more natural, that is, allowing the user to interact with and have physical contact with the autonomous moving body 11 more naturally, thereby improving the user's satisfaction.

[0190] <Contact Recognition Model Learning Process> Next, the contact recognition model learning process executed by the information processing server 13 will be described with reference to the flowchart of FIG.

[0191] In step S101, the learning unit 412 collects learning data.

[0192] For example, the learning unit 264 of each autonomous mobile body 11 transmits the learning data stored in the learning data storage unit 254 to the information processing server 13 via the communication unit 204 and the network 21 .

[0193] In response to this, the learning unit 412 of the information processing server 13 receives the learning data from each autonomous mobile body 11 via the network 21 and the communication unit 401. The learning unit 412 stores the received learning data in the learning data storage unit 405.

[0194] In step S102, the learning unit 412 generates a contact recognition model for each user.

[0195] Specifically, the characteristics (e.g., duration, strength, pattern, etc.) of each user's contact action with the autonomous moving body 11 differ from user to user. Therefore, for example, as schematically shown in Fig. 19 , when each user contacts the autonomous moving body 11, the characteristics of the waveform of the IMU data output from the IMU 121 differ from user to user.

[0196] In response to this, the learning unit 412 performs learning processing using the learning data from each autonomous mobile body 11 individually, and generates a contact recognition model for each user of each autonomous mobile body 11. In this way, a contact recognition model that is appropriately tuned for each user is generated.

[0197] The learning unit 412 transmits the contact recognition model for each user to the autonomous moving body 11 of each user via the communication unit 401 and the network 21 .

[0198] This improves the accuracy of recognizing the contact action of each user of each autonomous moving body 11. As a result, the satisfaction of each user with respect to the interaction with the autonomous moving body 11 improves.

[0199] In step S103, the learning unit 412 generates a general-purpose contact recognition model.

[0200] Specifically, the learning unit 412 performs a learning process using the learning data of each user and learning data obtained by merging existing learning data, and generates a contact recognition model for the autonomous moving body 11. In this way, a general-purpose contact recognition model that is not specialized for an individual user is generated.

[0201] This general-purpose contact recognition model is installed in the autonomous moving body 11 when it is shipped, for example.

[0202] This improves the accuracy of recognizing the user's contact action from the start of use of the autonomous moving body 11. As a result, each user's satisfaction with the interaction with the autonomous moving body 11 improves.

[0203] Then, the contact recognition model learning process ends.

[0204] <<2. Modifications>> Modifications of the above-described embodiments of the present technology will now be described.

[0205] <Modifications Regarding Contact Action> For example, the method of recognizing a contact action is not limited to the above-described example.

[0206] For example, the contact recognition model may recognize a contact action based on information other than IMU data (e.g., other sensor data), or on IMU data and other information (e.g., other sensor data).

[0207] For example, the contact recognition model may be configured to recognize a contact action based on one of the acceleration and angular velocity of the autonomous moving body 11 .

[0208] For example, the recognition unit 261 may recognize a contact action without using a contact recognition model.

[0209] For example, the recognition unit 261 may correct the recognition result of the contact operation based on information other than the above-mentioned information (e.g., the image from the camera 102, the sensor data from the ToF sensor 103, and the control information from the operation control unit 263), or based on the above-mentioned information and other information.

[0210] The types of contact actions to be recognized are not limited to the examples described above. For example, other types of contact actions may be added, or the types of contact actions may be reduced.

[0211] Furthermore, the type of contact action may vary depending on the type and specifications of the autonomous moving body. For example, in the case of an autonomous moving body modeled after an animal, the type of contact action may vary depending on the type of animal.

[0212] <Modifications Related to Reactions> In the above description, an example has been shown in which the autonomous mobile body 11 mainly speaks as a reaction to a contact action, but the autonomous mobile body 11 may react by a method of expression other than speech, for example, a change in facial expression (for example, the eye portion 101), a movement of the whole body, etc. Furthermore, for example, the autonomous mobile body 11 may react by combining two or more types of expression methods.

[0213] <Modifications Related to the Configuration of the Autonomous Mobile Body 11> The configuration of the autonomous mobile body 11 described above may be modified as appropriate. For example, the number and positions of the components that make up the autonomous mobile body 11 may be modified as appropriate.

[0214] For example, one camera 102 may be provided on the front surface (front face) of the autonomous moving body 11 near the center between the eye portion 101L and the eye portion 101R.

[0215] For example, four ToF sensors 103 may be provided on the front surface of the autonomous mobile body 11 and two on the back surface. In this case, the four ToF sensors 104 on the front surface are arranged symmetrically with a gap between them, for example, near the lower end of the front surface of the head cover 104. The two ToF sensors 104 on the back surface are arranged symmetrically with a gap between them, for example, near the height of the connection terminals 105 on the back surface of the autonomous mobile body 11. This makes it possible to more accurately recognize the risk of falling, for example.

[0216] For example, one microphone 124 may be provided on each of the front, left side, right side, and back of the autonomous mobile body 11. In this case, the front microphone 124 is disposed, for example, slightly below the head cover 104 on the front of the autonomous mobile body 11, near the center in the left-right direction. The left and right side microphones 124 are disposed, for example, on the left and right side of the autonomous mobile body 11, respectively, at approximately the same height as the power switch 106. The back microphone 124 is disposed, for example, near the connection terminal 105 on the back of the autonomous mobile body 11.

[0217] <Other Modifications> For example, it is possible to appropriately change the allocation of processing in the information processing system 1. For example, part of the processing of the autonomous mobile body 11, the information processing terminal 12, or the information processing server 13 may be executed by another device.

[0218] For example, the autonomous mobile body 11 may transmit various sensor data to the information processing server 13, and the learning unit 412 of the information processing server 13 may generate learning data.

[0219] For example, the learning unit 264 of the autonomous moving body 11 may perform learning processing to generate a contact recognition model corresponding to the user.

[0220] The present technology is not limited to the above-mentioned examples, but can be applied to various autonomous moving bodies that are capable of interacting with a user.

[0221] <<3. Others>> <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, that can execute various functions by installing various programs.

[0222] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0223] In the computer 1000 , a CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0224] An input / output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

[0225] The input unit 1006 includes input switches, buttons, a microphone, an image sensor, etc. The output unit 1007 includes a display, a speaker, etc. The storage unit 1008 includes a hard disk, a non-volatile memory, etc. The communication unit 1009 includes a network interface, etc. The drive 1010 drives removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0226] In the computer 1000 configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program recorded in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0227] The program executed by the computer 1000 (CPU 1001) can be provided by being recorded on a removable medium 1011 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0228] In the computer 1000, the program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Alternatively, the program can be installed in the ROM 1002 or the storage unit 1008 in advance.

[0229] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0230] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0231] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0232] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0233] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0234] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0235] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0236] (1) An autonomous moving body comprising: a recognition unit that recognizes a physical contact action of a user based on first information related to at least one of its own state and a surrounding state; and an operation control unit that controls the execution of a promotion reaction that is a reaction that further encourages the user to make the contact action, or a suppression reaction that is a reaction that causes the user to suppress the contact action. (2) The autonomous moving body described in (1), in which the recognition unit recognizes the state of the autonomous moving body in response to the contact action, and the operation control unit controls the execution of the promotion reaction or the suppression reaction based on the state of the autonomous moving body. (3) The autonomous moving body described in (2), in which the recognition unit recognizes the user's action of wiping the autonomous moving body and the degree of dirt on the autonomous moving body after wiping, and the operation control unit controls the execution of the promotion reaction or the suppression reaction based on the degree of dirt on the autonomous moving body. (4) The autonomous mobile body according to (2) or (3), wherein the recognition unit recognizes a poking action of the user on the autonomous mobile body and a risk level of the autonomous mobile body falling due to the poking by the user, and the operation control unit controls the execution of the promotion reaction or the suppression reaction based on the risk level. (5) The autonomous mobile body according to any of (1) to (4), wherein the recognition unit recognizes a manner of the contact action, and the operation control unit controls the execution of the promotion reaction or the suppression reaction based on the manner of the contact action. (6) The autonomous mobile body according to (5), wherein the manner of the contact action includes at least one of a duration, an intensity, and a pattern of the contact action. (7) The autonomous mobile body according to (6), wherein the manner of the contact action controls the execution of the suppression reaction so as to suppress the intensity of the contact action when the intensity of the contact action is equal to or greater than a threshold. (8) The autonomous mobile body according to any of (1) to (7), wherein the operation control unit controls the execution of a reaction to the contact action based on a history of the contact action. (9) The autonomous moving body according to (8), wherein the history of the contact motion includes at least one of a cumulative number of times and a frequency of the contact motion.(10) The autonomous mobile body according to any one of (1) to (9), wherein the recognition unit corrects the recognition result of the contact action based on second information related to a state of the autonomous mobile body or a surrounding state. (11) The autonomous mobile body according to (10), wherein the first information includes at least one of acceleration and angular velocity of the autonomous mobile body, and the second information includes at least one of image data of the surroundings of the autonomous mobile body and control information of the autonomous mobile body. (12) The autonomous mobile body according to any one of (1) to (11), wherein the recognition unit recognizes the contact action using a contact recognition model that recognizes the contact action based on the first information. (13) The autonomous mobile body according to (12), further comprising a learning unit that learns the contact recognition model using learning data including the first information. (14) A control method in which an autonomous moving body recognizes a physical contact action of a user based on information regarding at least one of its own state and a surrounding state, and controls the execution of a promotion reaction that is a reaction that further urges the user to make the contact action, or a suppression reaction that is a reaction that causes the user to refrain from making the contact action. (15) A recognition unit that recognizes a physical contact action of a user based on information regarding at least one of a state of an autonomous moving body and a surrounding state of the autonomous moving body, an operation control unit that controls the execution, by the autonomous moving body, of a promotion reaction that is a reaction that further urges the user to make the contact action, or a suppression reaction that is a reaction that causes the user to refrain from making the contact action. (16) An information processing method in which an information processing device recognizes a physical contact action of a user based on information regarding at least one of a state of the autonomous moving body and a surrounding state of the autonomous moving body, and controls the execution, by the autonomous moving body, of a promotion reaction that is a reaction that further urges the user to make the contact action, or a suppression reaction that is a reaction that causes the user to refrain from making the contact action.

[0237] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0238] 1 Information processing system, 11-1 to 11-n Autonomous moving body, 12-1 to 12-n Information processing terminal, 113 Information processing server, 101L, 101R Eye unit, 102L, 102R Camera, 103 ToF sensor, 104 Head cover, 121 IMU, 123 Speaker, 201 Input unit, 202 Processing unit, 203 Output unit, 221 CPU, 222 GPU, 231 OLED, 232 Drive unit, 251 Information processing unit, 261 Recognition unit, 262 Action planning unit, 263 Action control unit, 264 Learning unit, 303 Information processing unit, 311 Moving body control unit, 321 Recognition unit, 322 Action planning unit, 323 Action control unit, 402 Information processing unit, 411 Mobile object control unit, 412 Learning unit, 421 Recognition unit, 422 Action planning unit, 423 Operation control unit

Claims

1. A self-mobile robot comprising: a recognition unit that recognizes a physical contact operation of a user based on first information regarding at least one of its own state and the surrounding state; and an operation control unit that controls execution of a promotion reaction, which is a reaction that further prompts the user to perform the contact operation, or a suppression reaction, which is a reaction that suppresses the user from performing the contact operation.

2. The self-mobile robot according to claim 1, wherein the recognition unit recognizes the state of the self-mobile robot with respect to the contact operation, and the operation control unit controls execution of the promotion reaction or the suppression reaction based on the state of the self-mobile robot.

3. The self-mobile robot according to claim 2, wherein the recognition unit recognizes an operation of the user wiping the self-mobile robot and the degree of soiling of the self-mobile robot after wiping, and the operation control unit controls execution of the promotion reaction or the suppression reaction based on the degree of soiling of the self-mobile robot.

4. The self-mobile robot according to claim 2, wherein the recognition unit recognizes an operation of the user poking the self-mobile robot and the risk of the self-mobile robot falling due to the user's poking, and the operation control unit controls execution of the promotion reaction or the suppression reaction based on the risk.

5. The self-mobile robot according to claim 1, wherein the recognition unit recognizes the manner of the contact operation, and the operation control unit controls execution of the promotion reaction or the suppression reaction based on the manner of the contact operation.

6. The self-mobile robot according to claim 5, wherein the manner of the contact operation includes at least one of the duration, intensity, and pattern of the contact operation.

7. The self-mobile robot according to claim 6, wherein when the intensity of the contact operation is equal to or greater than a threshold value, the operation control unit controls execution of the suppression reaction so as to suppress the intensity of the contact operation.

8. The self-mobile robot according to claim 1, wherein the operation control unit controls execution of a reaction to the contact operation based on the history of the contact operation.

9. The self-mobile robot according to claim 8, wherein the history of the contact operation includes at least one of the cumulative number and frequency of the contact operation.

10. The self-mobile robot according to claim 1, wherein the recognition unit corrects the recognition result of the contact operation based on second information regarding its own or the surrounding state.

11. The first information includes at least one of the acceleration and angular velocity of the autonomous mobile body, and the second information includes at least one of the image data around the autonomous mobile body and the control information of the autonomous mobile body. The autonomous mobile body according to claim 10.

12. The recognition unit recognizes the contact operation using a learning model that recognizes the contact operation based on the first information. The autonomous mobile body according to claim 1.

13. The autonomous mobile body according to claim 12, further comprising a learning unit that learns the learning model using learning data including the first information.

14. A control method for an autonomous mobile body to recognize a physical contact operation of a user based on information regarding at least one of its own state and the state of its surroundings, and to control the execution of a promotion reaction, which is a reaction to further prompt the user for the contact operation, or a suppression reaction, which is a reaction to suppress the contact operation by the user.

15. An information processing apparatus including a recognition unit that recognizes a physical contact operation of a user based on information regarding at least one of the state of the autonomous mobile body and the state of the surroundings of the autonomous mobile body, and an operation control unit that controls the autonomous mobile body to execute a promotion reaction, which is a reaction to further prompt the user for the contact operation, or a suppression reaction, which is a reaction to suppress the contact operation by the user.

16. An information processing method for an information processing apparatus to recognize a physical contact operation of a user based on information regarding at least one of the state of the autonomous mobile body and the state of the surroundings of the autonomous mobile body, and to control the autonomous mobile body to execute a promotion reaction, which is a reaction to further prompt the user for the contact operation, or a suppression reaction, which is a reaction to suppress the contact operation by the user.

Images