Self-mobile body, information processing method, program, and information processing apparatus

Abstract

The present technology relates to an autonomous mobile body, an information processing method, a program, and an information processing device which enable user experience to be improved by an output sound from the autonomous mobile body. The autonomous mobile body comprises a recognition unit that performs recognition on a combined instrument, which is an instrument combined with a host device, and a sound control unit that changes a method for controlling an output sound outputted from the host device and performs control on the output sound on the basis of the changed control method. The present technology can be applied to, for example, a robot.

Description

Technical Field

[0001] The present technology relates to an autonomous mobile body, an information processing method, a program, and an information processing apparatus, and more particularly, to an autonomous mobile body, an information processing method, a program, and an information processing apparatus that improve the user experience by the output sound of the autonomous mobile body.

Background Art

[0002] Conventionally, it has been proposed to determine the emotional state of a robot according to a user's initiative, and to select the action content and voice content corresponding to the determined emotion from the performance information corresponding to the exterior unit mounted on the robot, and to autonomously operate the robot with the selected action content and voice content (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the invention described in Patent Document 1, only the fixed sound registered in advance is switched according to the exterior unit mounted on the robot, and there is little variation.

[0005] The present technology has been made in view of such a situation, and aims to improve the user experience by the output sound of an autonomous mobile body such as a robot.

Means for Solving the Problems

[0006] The autonomous mobile body according to one aspect of the present technology includes a recognition unit that recognizes a combined device, which is a device combined with the self-device, and a sound control unit that changes a control method of output sound output from the self-device based on the recognition result of the combined device and controls the output sound based on the changed control method.

[0007] The information processing method according to one aspect of the present technology recognizes a combined device, which is a device combined with an autonomous mobile body, changes a control method of output sound output from the autonomous mobile body based on the recognition result of the combined device, and controls the output sound based on the changed control method.

[0008] The program according to one aspect of the present technology recognizes a combined device, which is a device combined with an autonomous mobile body, changes a control method of output sound output from the autonomous mobile body based on the recognition result of the combined device, and controls the output sound based on the changed control method.

[0009] The information processing apparatus according to one aspect of the present technology includes a recognition unit that recognizes a combined device, which is a device combined with an autonomous mobile body, and a sound control unit that changes a control method of output sound output from the autonomous mobile body based on the recognition result of the combined device and controls the output sound based on the changed control method.

[0010] According to one aspect of the present technology, a combined device, which is a device combined with an autonomous mobile body, is recognized, a control method of output sound output from the autonomous mobile body is changed based on the recognition result of the combined device, and the output sound is controlled based on the changed control method.

Brief Description of Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments for implementing the present technology will be described. The description will be carried out in the following order. 1. Embodiment 2. Variation 3. Others

[0013] <<1. Embodiment>> With reference to FIGS. 1 to 23, embodiments of the present technology will be described.

[0014] <Configuration Example of Information Processing System 1> FIG. 1 shows an embodiment of an information processing system 1 to which the present technology is applied.

[0015] The information processing system 1 includes an autonomous mobile body 11, an information processing server 12, and an operated device 13. The autonomous mobile body 11, the information processing server 12, and the operated device 13 are connected via a network 14.

[0016] The autonomous mobile body 11 is an information processing device that performs autonomous operations either without being controlled by the information processing server 12 or under the control of the information processing server 12. For example, the autonomous mobile body 11 is composed of various robots such as running type, walking type, flying type, and swimming type.

[0017] In addition, the autonomous mobile body 11 is an agent device that enables more natural and effective communication with the user. One of the characteristics of the autonomous mobile body 11 is that it actively executes various operations (hereinafter also referred to as inducing operations) that induce communication with the user.

[0018] For example, the autonomous mobile body 11 can actively present information to the user based on environmental recognition. Also, for example, the autonomous mobile body 11 actively executes various incentive actions to prompt the user to perform predetermined actions.

[0019] Moreover, the incentive actions by the autonomous mobile body 11 can be said to be an active and positive interference with the physical space. The autonomous mobile body 11 can move in the physical space and execute various physical actions on users, living organisms, articles, etc. According to the above characteristics of the autonomous mobile body 11, the user can comprehensively recognize the actions of the autonomous mobile body through vision, hearing, and touch, and can achieve a high level of communication compared to the case of simply interacting with the user using voice.

[0020] Furthermore, the autonomous mobile body 11 can express its own state or communicate with the user or other autonomous mobile bodies by outputting an output sound. The output sound of the autonomous mobile body 11 includes an operation sound output corresponding to the situation of the autonomous mobile body 11 and a speech sound for communicating with the user or other autonomous mobile bodies, etc.

[0021] The operation sound includes a sound output corresponding to the movement of the autonomous mobile body 11 and a sound output corresponding to a stimulus to the autonomous mobile body 11. The sound output corresponding to the movement of the autonomous mobile body 11 includes not only the sound output when the autonomous mobile body 11 moves actively but also the sound output when the autonomous mobile body 11 is moved passively. The stimulus to the autonomous mobile body 11 is, for example, a stimulus to any of the five senses (vision, hearing, smell, taste, touch) of the autonomous mobile body 11. Note that the autonomous mobile body 11 does not necessarily recognize all intervals of the five senses.

[0022] The speech sound does not necessarily have to be a sound representing a language that humans can understand, and may be, for example, a sound representing non-language imitating the cries of animals, etc.

[0023] The information processing server 12 is an information processing device that controls the operation of the autonomous mobile body 11. For example, the information processing server 12 has a function of causing the autonomous mobile body 11 to execute various incentive operations for inducing communication with the user.

[0024] The device to be operated 13 is various devices operated by the autonomous mobile body 11 and the information processing server 12. The autonomous mobile body 11 can operate various devices to be operated 13 either without being controlled by the information processing server 12 or under the control of the information processing server 12. The device to be operated 13 is composed of, for example, home appliances such as lighting devices, game devices, and television devices.

[0025] The network 14 has a function of connecting each component included in the information processing system 1. For example, the network 14 may include a public line network such as the Internet, a telephone line network, or a satellite communication network, and various LANs (Local Area Networks) including Ethernet (registered trademark), WANs (Wide Area Networks), etc. For example, the network 14 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network). For example, the network 14 may include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0026] <Configuration example of the autonomous mobile body 11> Next, with reference to FIGS. 2 to 13, a configuration example of the autonomous mobile body 11 will be described. The autonomous mobile body 11 can be various devices that perform autonomous operations based on environmental recognition. In the following, the case where the autonomous mobile body 11 is an agent-type robot device in an oblong shape that performs autonomous driving by wheels will be described as an example. The autonomous mobile body 11 realizes various communications including information presentation by performing autonomous operations according to, for example, the user, the surroundings, and its own situation. The autonomous mobile body 11 is, for example, a small robot having a size and weight such that it can be easily lifted by the user with one hand.

[0027] <Example of the exterior of the autonomous mobile body 11> First, with reference to FIGS. 2 to 7, an example of the exterior of the autonomous mobile body 11 will be described.

[0028] FIG. 2 is a front view of the autonomous mobile body 11, and FIG. 3 is a rear view of the autonomous mobile body 11. A and B in FIG. 4 are perspective views of the autonomous mobile body 11. FIG. 5 is a side view of the autonomous mobile body 11. FIG. 6 is a top view of the autonomous mobile body 11. FIG. 7 is a bottom view of the autonomous mobile body 11.

[0029] As shown in FIGS. 2 to 6, the autonomous mobile body 11 includes eye parts 101L and 101R corresponding to the left eye and the right eye at the upper part of the main body. The eye parts 101L and 101R are realized by, for example, LEDs or the like, and can express a line of sight, a blink, and the like. Note that the eye parts 101L and 101R are not limited to the above example, and may be realized by, for example, a single or two independent OLEDs (Organic Light Emitting Diodes).

[0030] In addition, the autonomous mobile body 11 includes cameras 102L and 102R above the eye parts 101L and 101R. The cameras 102L and 102R have a function of imaging a user and the surrounding environment. At this time, the autonomous mobile body 11 may realize SLAM (Simultaneous Localization and Mapping) based on the images captured by the cameras 102L and 102R.

[0031] Note that the eye parts 101L, 101R, cameras 102L, and cameras 102R are arranged on a substrate (not shown) disposed inside the exterior surface. In addition, the exterior surface of the autonomous mobile body 11 is basically formed using an opaque material, but a head cover 104 using a transparent or translucent material is provided at a portion corresponding to the substrate on which the eye parts 101L, 101R, cameras 102L, and cameras 102R are arranged. Thereby, the user can recognize the eye parts 101L and 101R of the autonomous mobile body 11, and the autonomous mobile body 11 can image the outside world.

[0032] Also, as shown in FIGS. 2, 4, and 7, the autonomous mobile body 11 is provided with a ToF (Time of Flight) sensor 103 at the lower front part. The ToF sensor 103 has a function of detecting the distance to an object existing in the front. The autonomous mobile body 11 can accurately detect the distance to various objects, for example, by the ToF sensor 103, and can also detect steps or the like to prevent falling or toppling.

[0033] Also, as shown in FIGS. 3, 5, etc., the autonomous mobile body 11 is provided with a connection terminal 105 for an external device and a power switch 106 on the back surface. The autonomous mobile body 11 can be connected to an external device via the connection terminal 105, for example, and perform information communication.

[0034] Also, as shown in FIG. 7, the autonomous mobile body 11 is provided with wheels 107L and 107R on the bottom surface. The wheels 107L and 107R are driven by different motors (not shown), respectively. Thereby, the autonomous mobile body 11 can realize movement operations such as forward movement, backward movement, turning, and rotation.

[0035] Also, the wheels 107L and 107R can be stored inside the main body and protrude outside. For example, the autonomous mobile body 11 can perform a jump operation by vigorously protruding the wheels 107L and 107R outside. Note that FIG. 7 shows a state in which the wheels 107L and 107R are stored inside the main body.

[0036] In the following, when it is not necessary to distinguish the left eye part 101L and the right eye part 101R individually, they are simply referred to as the eye part 101. In the following, when it is not necessary to distinguish the camera 102L and the camera 102R individually, they are simply referred to as the camera 102. In the following, when it is not necessary to distinguish the wheels 107L and 107R individually, they are simply referred to as the wheel 107.

[0037] <Example of the internal structure of the autonomous mobile body 11> FIGS. 8 and 9 are schematic diagrams showing the internal structure of the autonomous mobile body 11.

[0038] As shown in FIG. 8, the autonomous mobile body 11 includes an inertial sensor 121 and a communication device 122 disposed on an electronic substrate. The inertial sensor 121 detects the acceleration and angular velocity of the autonomous mobile body 11. The communication device 122 is a configuration for realizing wireless communication with the outside, and includes, for example, a Bluetooth or Wi-Fi antenna.

[0039] In addition, the autonomous mobile body 11 includes, for example, a speaker 123 inside the side surface of the main body. The autonomous mobile body 11 can output various sounds through the speaker 123.

[0040] As shown in FIG. 9, the autonomous mobile body 11 includes a microphone 124L, a microphone 124M, and a microphone 124R inside the upper part of the main body. The microphones 124L, 124M, and 124R collect the user's speech and ambient environmental sounds. In addition, by including a plurality of microphones 124L, 124M, and 124R, the autonomous mobile body 11 can collect sounds generated in the surroundings with high sensitivity and detect the position of the sound source.

[0041] As shown in FIGS. 8 and 9, the autonomous mobile body 11 includes motors 125A to 125E (however, the motor 125E is not shown). The motors 125A and 125B drive the substrate on which the eyes 101 and the camera 102 are disposed in the vertical and horizontal directions, for example. The motor 125C realizes the forward tilt posture of the autonomous mobile body 11. The motor 125D drives the wheel 107L. The motor 125E drives the wheel 107R. The autonomous mobile body 11 can express rich motions by the motors 125A to 125E.

[0042] Hereinafter, when it is not necessary to distinguish the microphones 124L to 124R individually, they are simply referred to as the microphone 124. Hereinafter, when it is not necessary to distinguish the motors 125A to 125E individually, they are simply referred to as the motor 125.

[0043] <Configuration Example of Functions of Autonomous Mobile Body 11> FIG. 10 shows a configuration example of the functions of the autonomous mobile body 11. The autonomous mobile body 11 includes a control unit 201, a sensor unit 202, an input unit 203, a light source 204, a sound output unit 205, a drive unit 206, and a communication unit 207.

[0044] The control unit 201 has a function of controlling each component included in the autonomous mobile body 11. The control unit 201 controls, for example, the activation and stop of each component. Further, the control unit 201 supplies a control signal or the like received from the information processing server 12 to the light source 204, the sound output unit 205, and the drive unit 206.

[0045] The sensor unit 202 has a function of collecting various data related to the user and the surrounding situation. For example, the sensor unit 202 includes the above-described camera 102, ToF sensor 103, inertial sensor 121, microphone 124, and the like. In addition to the above sensors, the sensor unit 202 may include various sensors such as a geomagnetic sensor, a touch sensor, various optical sensors such as an IR (infrared) sensor, a temperature sensor, a humidity sensor, and the like. The sensor unit 202 supplies sensor data output from each sensor to the control unit 201.

[0046] The input unit 203 includes, for example, buttons and switches such as the above-described power switch 106, and detects a physical input operation by the user.

[0047] The light source 204 includes, for example, the above-described eye part 101, and expresses the eye movement of the autonomous mobile body 11.

[0048] The sound output unit 205 includes, for example, the above-described speaker 123 and an amplifier, and outputs an output sound based on the output sound data supplied from the control unit 201.

[0049] The drive unit 206 includes, for example, the above-described wheels 107 and motor 125, and is used to express the body movement of the autonomous mobile body 11.

[0050] The communication unit 207 includes, for example, the connection terminal 105 and the communication device 122 described above, and communicates with the information processing server 12, the device to be operated 13, and other external devices. For example, the communication unit 207 transmits the sensor data supplied from the sensor unit 202 to the information processing server 12, and receives a control signal for controlling the operation of the autonomous mobile body 11 and output sound data for causing the autonomous mobile body 11 to output an output sound from the information processing server 12.

[0051] <Configuration example of information processing unit 241> FIG. 11 shows a configuration example of an information processing unit 241 realized by the control unit 201 of the autonomous mobile body 11 executing a predetermined control program.

[0052] The information processing unit 241 includes a recognition unit 251, an action planning unit 252, an operation control unit 253, and a sound control unit 254.

[0053] The recognition unit 251 has a function of recognizing various information about the user and the environment around the autonomous mobile body 11 and the autonomous mobile body 11 based on the sensor data supplied from the sensor unit 202.

[0054] For example, the recognition unit 251 performs user identification, recognition of the user's expression and gaze, object recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, recognition of stimuli to the autonomous mobile body 11, etc. For example, the recognition unit 251 performs emotion recognition related to the user's voice, word understanding, recognition of the position of the sound source, etc. For example, the recognition unit 251 recognizes the surrounding temperature, the presence of moving objects, the posture and movement of the autonomous mobile body 11, etc. For example, the recognition unit 251 recognizes a device (hereinafter referred to as a combined device) combined with the autonomous mobile body 11.

[0055] As an example of the autonomous mobile body 11 being combined with a combined device, for example, when one of the autonomous mobile body 11 and the combined device is attached to the other, when one of the autonomous mobile body 11 and the combined device rides on the other, when the autonomous mobile body 11 and the combined device are combined, etc. are assumed. Further, as an example of the combined device, for example, parts detachable from the autonomous mobile body 11 (hereinafter referred to as option parts), a mobile body on which the autonomous mobile body 11 can ride (hereinafter referred to as a riding mobile body), a device to which the autonomous mobile body 11 can be attached or detached (hereinafter referred to as an attachment destination device), etc. are assumed.

[0056] As option parts, for example, parts imitating a part of an animal's body (e.g., eyes, ears, nose, mouth, beak, horns, tail, feathers, etc.), costumes, stuffed toys, parts for expanding the functions and capabilities of the autonomous mobile body 11 (e.g., medals, weapons, etc.), wheels, caterpillars, etc. are assumed. As riding mobile bodies, for example, cars, drones, robot vacuum cleaners, etc. are assumed. As attachment destination devices, for example, combined robots composed of a plurality of parts including the autonomous mobile body 11 are assumed.

[0057] Note that the combined device does not necessarily have to be a device dedicated to the autonomous mobile body 11, and may be a general-purpose device, for example.

[0058] Further, the recognition unit 251 has a function of estimating and understanding the environment and situation in which the autonomous mobile body 11 is placed based on the recognized information. At this time, the recognition unit 251 may comprehensively perform situation estimation using pre-stored environmental knowledge.

[0059] The recognition unit 251 supplies data indicating the recognition result to the action plan unit 252, the motion control unit 253, and the sound control unit 254.

[0060] The action planning unit 252 sets an operation mode that defines the operation of the autonomous mobile body 11 based on the recognition result by the recognition unit 251, for example, the recognition result of the combined device by the recognition unit 251. Further, the action planning unit 252 has a function of planning the actions to be performed by the autonomous mobile body 11 based on, for example, the recognition result by the recognition unit 251, the operation mode, and the learned knowledge. Furthermore, the action planning unit 252 executes the action plan using a machine learning algorithm such as deep learning. The action planning unit 252 supplies operation data and data indicating the action plan to the motion control unit 253 and the sound control unit 254.

[0061] The motion control unit 253 controls the operation of the autonomous mobile body 11 by controlling the light source 204 and the drive unit 206 based on the recognition result by the recognition unit 251, the action plan by the action planning unit 252, and the operation mode. The motion control unit 253, for example, moves the autonomous mobile body 11 while maintaining a forward-leaning posture, or causes it to perform forward and backward movements, turning movements, rotational movements, etc. In addition, the motion control unit 253 actively causes the autonomous mobile body 11 to execute an inducing action that induces communication between the user and the autonomous mobile body 11. Further, the motion control unit 253 supplies information regarding the action being performed by the autonomous mobile body 11 to the sound control unit 254.

[0062] The sound control unit 254 controls the output sound by controlling the sound output unit 205 based on the recognition result by the recognition unit 251, the action plan by the action planning unit 252, and the operation mode. For example, the sound control unit 254 sets a control method for the output sound based on the operation mode and the like, and controls the output sound (for example, controls the content of the output sound to be generated and the output timing of the output sound) based on the set control method. Then, the sound control unit 254 generates output sound data for outputting the output sound and supplies it to the sound output unit 205. In addition, the sound control unit 254 supplies information regarding the output sound output by the autonomous mobile body 11 to the motion control unit 253.

[0063] <Method for generating synthesized sound> Next, the method for generating the synthesized sound in the sound control unit 254 will be described.

[0064] The sound control unit 254 generates an output sound composed of synthesized sounds using, for example, an FM sound source. At this time, the sound control unit 254 dynamically and continuously changes various parameters related to the synthesis of the FM sound source, and changes the waveform of the synthesized sound, that is, the pitch (pitch, height of the sound), volume, timbre, speed, etc. of the synthesized sound, so that the impression and emotional meaning of the synthesized sound can be expressed in various ways.

[0065] FIG. 12 is a diagram for explaining parameters related to synthesized sounds. FIG. 12 shows the relationship between the configuration of a synthesizer that performs synthesis of an FM sound source and the output mode expressed by the synthesized sound due to changes in the parameters related to each configuration.

[0066] The sound control unit 254 can change the basic texture of the sound by changing, for example, the parameters related to the oscillator. As an example, the sound control unit 254 can express a soft impression by setting the waveform of the sound to a sine wave, and can express a sharp impression by setting it to a sawtooth shape.

[0067] Also, the sound control unit 254 can express differences in gender, intonation, emotional ups and downs, etc. by controlling, for example, the parameters of the pitch controller, that is, the pitch.

[0068] FIG. 13 is a diagram showing an example of emotions that can be expressed by controlling the pitch and speed of sound. Note that the size (area) of the hatched region in FIG. 13 indicates the volume. It is known that the pitch and speed of sound strongly affect the evocation of emotions expressed by the sound. The sound control unit 254 can, for example, express the degree of joy or anger by setting the pitch and speed relatively high. Conversely, the sound control unit 254 can also express sadness by setting the pitch and speed relatively low. In this way, the sound control unit 254 can express various emotions and their degrees by controlling the pitch and speed of the sound.

[0069] Returning to FIG. 12, the sound control unit 254 can express the clarity of sound (how the mouth is opened) by controlling the parameters of the filter. For example, the sound control unit 254 can express a muffled sound or an open sound by raising or lowering the frequency of the high-cut filter.

[0070] Also, the sound control unit 254 can change the accent of the volume and the impression of the rising or ending manner by the temporal change of the amplifier.

[0071] Also, the sound control unit 254 can express the tremor or smoothness of the voice by controlling the parameters of the modulator.

[0072] In this way, the sound control unit 254 can variously express impressions and emotional meanings by changing the parameters related to the oscillator, modulator, pitch controller, filter, or amplifier.

[0073] <Functional Configuration Example of Information Processing Server 12> FIG. 14 shows a functional configuration example of the information processing server 12.

[0074] The information processing server 12 includes a communication unit 301, a recognition unit 302, an action planning unit 303, an operation control unit 304, and a sound control unit 305.

[0075] The communication unit 301 communicates with the autonomous mobile body 11 and the operated device 13 via the network 14. For example, the communication unit 301 receives sensor data from the autonomous mobile body 11 and transmits a control signal for controlling the operation of the autonomous mobile body 11 and output sound data for causing the autonomous mobile body 11 to output an output sound to the autonomous mobile body 11.

[0076] The recognition unit 302, the action plan unit 303, the motion control unit 304, and the sound control unit 305 have the same functions as the recognition unit 251, the action plan unit 252, the motion control unit 253, and the sound control unit 254 of the autonomous mobile body 11. That is, the recognition unit 302, the action plan unit 303, the motion control unit 304, and the sound control unit 305 can perform various processes in place of the recognition unit 251, the action plan unit 252, the motion control unit 253, and the sound control unit 254 of the autonomous mobile body 11.

[0077] Thereby, the information processing server 12 can remotely control the autonomous mobile body 11, and the autonomous mobile body 11 can perform various operations and output various output sounds under the control of the information processing server 12.

[0078] <Processing of the autonomous mobile body 11> Next, with reference to FIGS. 15 to 23, the processing of the autonomous mobile body 11 will be described.

[0079] Hereinafter, an example in which the autonomous mobile body 11 independently performs various operations and outputs various output sounds without being controlled by the information processing server 12 will be described.

[0080] <Operation mode setting process> First, with reference to the flowchart of FIG. 15, the operation mode setting process executed by the autonomous mobile body 11 will be described.

[0081] This process is started, for example, when the power of the autonomous mobile body 11 is turned on and ends when it is turned off.

[0082] In step S1, the recognition unit 251 determines whether the combination of the combined devices has changed. The recognition unit 251 detects the addition and deletion of the combined devices combined with the autonomous mobile body 11 based on the sensor data and the like supplied from the sensor unit 202. When the recognition unit 251 does not detect the addition and deletion of the combined devices, it determines that the combination of the combined devices has not changed, and repeats this determination process at a predetermined timing until it determines that the combination of the combined devices has changed.

[0083] On the other hand, when the recognition unit 251 detects the addition or deletion of the combined device, it determines that the combination of the combined devices has changed, and the process proceeds to step S2.

[0084] Note that the method for recognizing the combined device is not particularly limited. Hereinafter, examples of the method for recognizing the combined device will be described.

[0085] First, an example of a method for directly recognizing the combined device will be described.

[0086] For example, a method of electrically recognizing the combined device can be considered. For example, the combined device is recognized by passing an electrical signal between the autonomous mobile body 11 and the combined device.

[0087] For example, a method of recognizing the combined device using a physical switch can be considered. For example, when the combined device is combined with the autonomous mobile body 11, the combined device is recognized by the contact switch provided on the autonomous mobile body 11 being pressed by the combined device. For example, when the combined device is combined with the autonomous mobile body 11, the combined device is recognized by the optical switch provided on the autonomous mobile body 11 being blocked by the combined device.

[0088] For example, a method of optically recognizing the combined device using visual information such as color and barcode can be considered. For example, based on the images captured by the camera 102L and the camera 102R, the combined device and the features of the combined device (such as color and shape) are recognized.

[0089] For example, a method of recognizing the combined device by magnetic force can be considered. For example, the combined device is recognized based on the magnetic force of the magnet provided on the combined device.

[0090] For example, a method of recognizing a combined device by radio waves can be considered. For example, the recognition unit 251 recognizes the combined device based on the result of the communication device 122 of the autonomous mobile body 11 reading information from an RFID (Radio Frequency Identifier) provided in the combined device, or performing short-range wireless communication such as Bluetooth or Wi-Fi with the combined device.

[0091] Next, an example of a method of indirectly recognizing a combined device based on a change in the movement of the autonomous mobile body 11 due to the combination of the combined devices will be described.

[0092] For example, the combined device is recognized by applying a predetermined rule to the detection value based on the sensor data from the sensor unit 202.

[0093] For example, when the autonomous mobile body 11 is on wheels and when it is on a turntable, the ratio of the vibration amount of the autonomous mobile body 11 to the movement amount (odometry) of the autonomous mobile body 11 changes. For example, when the autonomous mobile body 11 is on wheels, the vibration amount of the autonomous mobile body 11 decreases while the movement amount of the autonomous mobile body 11 increases. On the other hand, when the autonomous mobile body 11 is on a turntable, the vibration amount of the autonomous mobile body 11 increases while the movement amount of the autonomous mobile body 11 decreases. Therefore, for example, based on the ratio of the vibration amount to the movement amount of the autonomous mobile body 11, it is recognized that the wheels or the turntable has been combined with the autonomous mobile body 11.

[0094] For example, when wheels larger than the wheels 107L and 107R or a caterpillar are attached to the autonomous mobile body 11, the rolling resistance increases. Therefore, based on the detected value of the rolling resistance of the autonomous mobile body 11, it is recognized that the wheels or the caterpillar has been combined with the autonomous mobile body 11.

[0095] For example, when the autonomous mobile body 11 is attached to the combined device or integrated with the combined device, the movement of the autonomous mobile body 11 may be restricted. For example, the recognition unit 251 recognizes the combined device by detecting the restriction of the movement of the autonomous mobile body 11 based on the sensor data from the sensor unit 202.

[0096] Note that the recognition of the combined device may be performed by combining the above-described plurality of methods.

[0097] For example, based on the vibration pattern of the autonomous mobile body 11 detected by using the inertial sensor 121, it is possible to recognize that the autonomous mobile body 11 has mounted on the wheels. Further, for example, by detecting the magnetic force of the magnet provided on the wheels by the magnetic sensor, it is possible to recognize that the autonomous mobile body 11 has mounted on the wheels.

[0098] Here, in the recognition method using the inertial sensor 121, while the time required for wheel recognition becomes long, wheel recognition becomes possible even if the wheels are not correctly mounted. On the other hand, in the recognition method using the magnetic sensor, while the time required for wheel recognition becomes short, wheel recognition becomes difficult if the wheels are not correctly mounted. Therefore, by combining the two recognition methods, the disadvantages of both can be compensated, and the recognition accuracy and recognition speed of the wheels are improved.

[0099] Further, for example, it is also possible to recognize the combined device by using the discriminator generated by performing machine learning using the sensor data from the sensor unit 202.

[0100] In step S2, the autonomous mobile body 11 changes the operation mode.

[0101] Specifically, the recognition unit 251 supplies the action plan unit 252 with data indicating the presence and type of the combined device combined with the autonomous mobile body 11.

[0102] When the combined device is not combined with the autonomous mobile body 11, the action plan unit 252 sets the operation mode to the normal mode.

[0103] On the other hand, when the combined device is combined with the autonomous mobile body 11, the action plan unit 252 sets the operation mode, for example, based on the type of the combined device combined.

[0104] For example, when a cat ear-shaped option part (hereinafter referred to as an ear-shaped part) is attached to the head of the autonomous mobile body 11, the action planning unit 252 sets the operation mode to the cat mode. For example, when the autonomous mobile body 11 is in a vehicle, the action planning unit 252 sets the operation mode to the vehicle mode.

[0105] When a plurality of combined devices are combined with the autonomous mobile body 11, the action planning unit 252 sets the operation mode based on the combination, for example. Alternatively, the action planning unit 252 sets the operation mode based on the type of the most prioritized combined device based on the priority order of the combined devices.

[0106] Further, for example, the action planning unit 252 may set the operation mode based only on whether or not the combined devices are combined, regardless of the types of the combined devices.

[0107] The action planning unit 252 supplies data indicating the set operation mode to the operation control unit 253 and the sound control unit 254.

[0108] Thereafter, the process returns to step S1, and the processes after step S1 are executed.

[0109] <Basic example of operation sound output control process> Next, with reference to the flowchart of FIG. 16, a basic example of the operation sound output control process executed by the autonomous mobile body 11 will be described.

[0110] In step S51, the recognition unit 251 converts the sensor data into intermediate parameters.

[0111] For example, the sensor data of the acceleration sensor included in the inertial sensor 121 includes a component of gravitational acceleration. Therefore, if the operation sound is output using the sensor data of the acceleration sensor as it is, the operation sound will be output constantly even when the autonomous mobile body 11 is not moving.

[0112] In addition, since the sensor data of the acceleration sensor includes the acceleration in the three-axis directions of the x-axis, y-axis, and z-axis, in addition to the components corresponding to the movement of the autonomous mobile body 11, components corresponding to vibration and noise are also included. Therefore, when the operation sound is output using the sensor data of the acceleration sensor as it is, in addition to the movement of the autonomous mobile body 11, the operation sound will be output in response to vibration and noise.

[0113] On the other hand, the recognition unit 251 converts the sensor data of each sensor included in the sensor unit 202 into intermediate parameters that can be understood by humans, corresponding to the situation of the autonomous mobile body 11 for which the operation sound is to be output.

[0114] Specifically, the recognition unit 251 acquires sensor data from each sensor included in the sensor unit 202, and performs arithmetic and logical operations such as filtering and threshold processing on each sensor data, thereby converting each sensor data into intermediate parameters of a predetermined type.

[0115] FIG. 17 shows a specific example of a method for converting sensor data into intermediate parameters.

[0116] For example, the recognition unit 251 acquires sensor data indicating the rotation speed of the motor 125D or the motor 125E of the autonomous mobile body 11 from the rotation sensor 401 included in the sensor unit 202. The recognition unit 251 calculates the amount of movement of the autonomous mobile body 11 by calculating the odometry based on the rotation speed of the motor 125D or the motor 125E. In addition, the recognition unit 251 calculates the speed (hereinafter referred to as the translational speed) in the translational direction (front-back, left-right direction) of the autonomous mobile body 11 based on the amount of movement of the autonomous mobile body 11. Thereby, the sensor data is converted into the speed (translational speed), which is an intermediate parameter.

[0117] For example, the recognition unit 251 is included in the sensor unit 202 and is provided on the bottom surface of the autonomous mobile body 11. The recognition unit 251 acquires sensor data indicating whether an object (e.g., the floor surface) is close to the bottom surface from an IR sensor 402 (not shown in FIGS. 2 to 9). Further, the recognition unit 251 acquires sensor data indicating the acceleration of the autonomous mobile body 11 from an acceleration sensor 121A included in the inertial sensor 121. The recognition unit 251 recognizes whether the autonomous mobile body 11 is lifted based on whether an object is close to the bottom surface of the autonomous mobile body 11 and the acceleration of the autonomous mobile body 11. Thereby, the sensor data is converted into the presence or absence of lifting, which is an intermediate parameter.

[0118] For example, the recognition unit 251 acquires sensor data indicating the acceleration of the autonomous mobile body 11 from the acceleration sensor 121A. Further, the recognition unit 251 acquires sensor data indicating the angular velocity of the autonomous mobile body 11 from an angular velocity sensor 121B included in the inertial sensor 121. The recognition unit 251 detects the amount of movement after the autonomous mobile body 11 is lifted based on the acceleration and angular velocity of the autonomous mobile body 11. This amount of movement indicates, for example, the amount by which the autonomous mobile body 11 is swayed after being lifted. Thereby, the sensor data is converted into the amount of movement after the autonomous mobile body 11 is lifted, which is an intermediate parameter.

[0119] For example, the recognition unit 251 acquires sensor data indicating the angular velocity of the autonomous mobile body 11 from the angular velocity sensor 121B. The recognition unit 251 detects the rotation (lateral rotation) in the yaw direction about the vertical axis of the autonomous mobile body based on the angular velocity of the autonomous mobile body 11. Thereby, the sensor data is converted into the lateral rotation of the autonomous mobile body 11, which is an intermediate parameter.

[0120] For example, the recognition unit 251 is included in the sensor unit 202 and acquires sensor data indicating the presence or absence of contact with the autonomous mobile body 11 from a touch sensor 403 provided at one or more sites where the user is likely to touch. The touch sensor 403 is constituted by, for example, a capacitance type or pressure sensitive type touch sensor. The recognition unit 251 recognizes contact actions such as the user touching, stroking, tapping, pressing, etc. based on the presence or absence of contact with the autonomous mobile body 11. As a result, the sensor data is converted into the presence or absence of a contact action with the autonomous mobile body 11, which is an intermediate parameter.

[0121] In step S52, the sound control unit 254 generates an operation sound based on the intermediate parameter and the operation mode.

[0122] For example, when the speed of the autonomous mobile body 11 is equal to or higher than a predetermined threshold value, the sound control unit 254 generates a translational sound, which is an operation sound corresponding to the translation of the autonomous mobile body 11. At this time, the sound control unit 254 changes some of the parameters such as the pitch (for example, frequency), volume, timbre (for example, frequency component, modulation degree, etc.), speed, etc. of the translational sound based on, for example, the speed of the autonomous mobile body 11 and the operation mode.

[0123] For example, when the operation mode is set to the normal mode, a continuous sound corresponding to the speed of the autonomous mobile body 11 and simulating the rotation sound of the wheels is generated as the translational sound.

[0124] For example, when the operation mode is set to the cat mode described above, a sound simulating the footsteps of a cat is generated as the translational sound.

[0125] For example, when the operation mode is set to the vehicle mode described above, a sound whose pitch changes according to the speed of the autonomous mobile body 11 and simulates the running sound of a vehicle is generated as the translational sound.

[0126] For example, when the autonomous mobile body 11 is lifted, the sound control unit 254 generates a lifting sound, which is an operation sound corresponding to the lifting of the autonomous mobile body 11. At this time, the sound control unit 254 changes some of the parameters such as the pitch, volume, timbre, and speed of the lifting sound based on, for example, the change in the amount of movement after the autonomous mobile body 11 is lifted and the operation mode.

[0127] For example, when the operation mode is set to the normal mode, a sound like a person being surprised is generated as the lifting sound.

[0128] For example, when the operation mode is set to the cat mode, a sound containing a low component like making a cat angry is generated as the lifting sound.

[0129] Note that, for example, when the operation mode is set to the vehicle mode, the lifting sound is not generated and not output.

[0130] For example, when the rotation speed of the lateral rotation of the autonomous mobile body 11 is equal to or higher than a predetermined threshold value, the sound control unit 254 generates a rotation sound, which is an operation sound corresponding to the lateral rotation of the autonomous mobile body 11. At this time, the sound control unit 254 changes some of the parameters such as the pitch, volume, timbre, and speed of the rotation sound based on, for example, the change in the rotation speed in the lateral direction of the autonomous mobile body 11 and the operation mode.

[0131] For example, when the operation mode is set to the normal mode, a rotation sound whose pitch changes according to the rotation speed of the autonomous mobile body 11 is generated.

[0132] For example, when the operation mode is set to the cat mode, the pitch changes according to the rotation speed of the autonomous mobile body 11, and a rotation sound with a timbre different from that in the normal mode is generated.

[0133] For example, when the operation mode is set to the vehicle mode, the pitch changes according to the rotation speed of the autonomous mobile body 11, and a rotation sound with a tone color different from that in the normal mode and the cat mode is generated. For example, a translational sound imitating the rotation sound of the motor is generated.

[0134] For example, when a contact action on the autonomous mobile body 11 is recognized, the sound control unit 254 generates a contact sound, which is an operation sound indicating the reaction of the autonomous mobile body 11 to the contact action. At this time, the sound control unit 254 changes some of the parameters such as the height, volume, tone color, and speed of the contact sound based on, for example, the type, duration, and intensity of the contact action on the autonomous mobile body 11, as well as the operation mode and the like.

[0135] For example, when the operation mode is set to the cat mode, a sound imitating the meowing of a cat is generated as the contact sound.

[0136] Note that, for example, when the operation mode is set to the normal mode or the vehicle mode, the contact sound is not generated and not output.

[0137] In this way, the content of the operation sound is set to the content corresponding to the type of the combined device.

[0138] In step S53, the autonomous mobile body 11 outputs an operation sound. Specifically, the sound control unit 254 generates output sound data for outputting the generated operation sound and supplies it to the sound output unit 205. The sound output unit 205 outputs an operation sound based on the acquired output sound data.

[0139] Note that, for example, the sound control unit 254 makes the reaction speed of the operation sound when the recognition of the situation of the autonomous mobile body 11 (for example, the movement of the autonomous mobile body 11 or the stimulus to the autonomous mobile body 11) that triggers the output of the output sound faster than the reaction speed of the operation sound when the recognition of the situation ends. For example, the sound control unit 254 controls the output of the operation sound so that the operation sound quickly rises at the start of the recognition of the situation and the operation sound slowly stops at the end of the recognition of the situation.

[0140] For example, A in FIG. 18 is a graph showing the waveform of the sensor data of the touch sensor 403. The horizontal axis represents time, and the vertical axis represents the value of the sensor data. B in FIG. 18 is a graph showing the waveform of the contact sound. The horizontal axis represents time, and the vertical axis represents the volume of the contact sound.

[0141] For example, at time t1, when the user starts the contact action on the autonomous mobile body 11, the touch sensor 403 starts outputting sensor data. Thereby, the recognition of the contact action by the recognition unit 251 is started. At this time, the sound control unit 254 sharply raises the contact sound. That is, the sound control unit 254 starts outputting the contact sound substantially simultaneously with the start of the recognition of the contact action, and sharply increases the volume of the contact sound.

[0142] On the other hand, at time t2, when the user finishes the contact action on the autonomous mobile body 11, the touch sensor 403 stops outputting sensor data. Thereby, the recognition of the contact action by the recognition unit 251 ends. At this time, the sound control unit 254 gently stops the contact sound. That is, after the recognition of the contact action ends, the sound control unit 254 gently decreases the volume of the contact sound and continues to output the contact sound for a while.

[0143] Thereby, a more natural contact sound is output. For example, since the contact sound is output substantially simultaneously with the start of the user's contact action, even if the user's contact action is short, it is prevented that the output of the contact sound starts unnaturally after the end of the contact action. Also, after the end of the user's contact action, the aftertaste of the contact sound remains, and it is prevented that the contact sound suddenly stops unnaturally.

[0144] Also, for example, the translation sound may be controlled in the same way as the contact sound. For example, the translation sound may sharply rise substantially simultaneously with the start of the recognition of the movement in the translation direction of the autonomous mobile body 11, and may gently stop when the recognition of the movement in the translation direction of the autonomous mobile body 11 ends.

[0145] <Specific Example of Translation Sound Output Control Process> Next, with reference to FIGS. 19 to 22, a specific example of the process for controlling the output of the translational sound will be described. Specifically, a specific example of the translational sound output control process will be described for the case where the ear-shaped part, which is one of the optional parts, is not attached to the autonomous mobile body 11 and the case where it is attached.

[0146] <Translational Sound Output Control Process in Normal Mode> First, with reference to the flowchart of FIG. 19, the translational sound output control process will be described for the case where the ear-shaped part is not attached to the autonomous mobile body 11 and the operation mode is set to the normal mode.

[0147] This process is started, for example, when the power of the autonomous mobile body 11 is turned on and ends when it is turned off.

[0148] In step S101, the recognition unit 251 detects the rotation speed r of the motor. Specifically, the recognition unit 251 acquires sensor data indicating the rotation speed of the motor 125D or the motor 125E of the autonomous mobile body 11 from the rotation sensor 401 included in the sensor unit 202. The recognition unit 251 detects the rotation speed r of the motor 125D or the motor 125E based on the acquired sensor data.

[0149] In step S102, the recognition unit 251 determines whether the rotation speed r > the threshold value Rth. If it is determined that the rotation speed r ≤ the threshold value Rth, the translational sound is not output and the process returns to step S101. Since the rotation speed r is substantially proportional to the translational speed of the autonomous mobile body 11, when the translational speed of the autonomous mobile body 11 is below a predetermined threshold value, the translational sound is not output.

[0150] Thereafter, in step S102, the processes of step S101 and step S102 are repeatedly executed until it is determined that the rotation speed r > the threshold value Rth.

[0151] On the other hand, in step S102, if it is determined that the rotation speed r > the threshold value Rth, that is, if the translational speed of the autonomous mobile body 11 exceeds a predetermined threshold value, the process proceeds to step S103.

[0152] In step S103, the recognition unit 251 sets the variable v to the rotational speed r - threshold value Rth. The variable v is proportional to the rotational speed r and approximately proportional to the translational speed of the autonomous mobile body 11. The recognition unit 251 supplies the data indicating the variable v to the sound control unit 254.

[0153] In step S104, the sound control unit 254 sets the volume of the translational sound to min(A * v, VOLmax). Here, A is a predetermined coefficient. Also, the volume VOLmax is the maximum volume of the translational sound. Thereby, within the range where the volume of the translational sound is below the maximum volume VOLmax, the volume is set to be approximately proportional to the translational speed of the autonomous mobile body 11.

[0154] In step S105, the sound control unit 254 sets the frequency of the translational sound to min(f0 * exp(B * v), FQmax). Here, B is a predetermined coefficient. Also, the frequency FQmax is the maximum frequency of the translational sound.

[0155] For humans, the frequency of pleasant - sounding sounds is in the range of about 200 - 2000 Hz. Also, the resolution of human hearing becomes finer as the frequency decreases and coarser as the frequency increases. Therefore, within the range where the frequency (pitch) of the translational sound is below the maximum frequency FQmax (for example, 2000 Hz), it is set to change exponentially with respect to the translational speed of the autonomous mobile body 11.

[0156] In step S106, the autonomous mobile body 11 outputs the translational sound. Specifically, the sound control unit 254 generates output sound data for outputting the translational sound with the set volume and frequency, and supplies it to the sound output unit 205. The sound output unit 205 outputs the translational sound based on the acquired output sound data.

[0157] After that, the process returns to step S101, and the processes after step S101 are executed.

[0158] As a result, for example, when the translational speed of the autonomous mobile body 11 is equal to or lower than a predetermined threshold value, as shown in A of FIG. 20, the translational sound is not output. On the other hand, when the translational speed of the autonomous mobile body 11 exceeds the predetermined threshold value, as shown in B and C of FIG. 20, the higher the translational speed, the higher the frequency (pitch) of the translational sound and the larger the amplitude (volume).

[0159] <Translational Sound Output Control Process in Cat Mode> Next, with reference to the flowchart of FIG. 21, the translational sound output control process when an ear-shaped part is attached to the autonomous mobile body 11 and the operation mode is set to the cat mode will be described.

[0160] This process is started, for example, when the power supply of the autonomous mobile body 11 is turned on and ends when it is turned off.

[0161] In step S151, in the same manner as the process of step S101 in FIG. 19, the rotation speed r of the motor is detected.

[0162] In step S152, in the same manner as the process of step S102 in FIG. 19, it is determined whether the rotation speed r > the threshold value Rth. If it is determined that the rotation speed r > the threshold value Rth, the process proceeds to step S153.

[0163] In step S153, the recognition unit 251 adds the rotation speed r to the movement amount Δd. The movement amount Δd represents the integrated value of the rotation speed of the motor since the autonomous mobile body 11 started moving in the translational direction or since the previous translational sound was output, and is substantially proportional to the movement amount of the autonomous mobile body 11 in the translational direction.

[0164] In step S154, the recognition unit 251 determines whether the movement amount Δd > the threshold value Dth. If it is determined that the movement amount Δd ≤ the threshold value Dth, the translational sound is not output and the process returns to step S151. That is, when the movement amount in the translational direction since the autonomous mobile body 11 started moving in the translational direction or since the previous translational sound was output is equal to or lower than a predetermined threshold value, the translational sound is not output.

[0165] After that, in step S152, the processings of steps S151 to S154 are repeatedly executed until it is determined that the rotation speed r ≤ the threshold value Rth or until it is determined in step S154 that the movement amount Δd > the threshold value Dth.

[0166] On the other hand, in step S154, when it is determined that the movement amount Δd > the threshold value Dth, that is, when the movement amount in the translational direction after the autonomous mobile body 11 starts moving in the translational direction or the movement amount in the translational direction after the previous translational sound is output exceeds a predetermined threshold value, the process proceeds to step S155.

[0167] In step S155, similar to the processing of step S103 in FIG. 19, the rotation speed r - the threshold value Rth is set to the variable v.

[0168] In step S156, the sound control unit 254 sets the volume of the translational sound to min(C * v, VOLmax). Here, C is a predetermined coefficient. Thereby, within the range where the volume of the translational sound is below the maximum volume VOLmax, the volume is set to be substantially proportional to the translational speed of the autonomous mobile body 11.

[0169] Note that the coefficient C is set to a value smaller than the coefficient A used in the processing of step S104 in FIG. 19, for example. Therefore, in the cat mode, compared with the normal mode, the change amount of the volume of the translational sound with respect to the translational speed of the autonomous mobile body 11 becomes smaller.

[0170] In step S157, the sound control unit 254 sets the harmonic component according to the variable v. Specifically, the sound control unit 254 sets the harmonic component of the translational sound so that the harmonic component increases as the variable v increases, that is, as the translational speed of the autonomous mobile body 11 increases.

[0171] In step S158, the autonomous mobile body 11 outputs a translational sound. Specifically, the sound control unit 254 generates output sound data for outputting a translational sound including the set harmonic components at the set volume, and supplies it to the sound output unit 205. The sound output unit 205 outputs a translational sound based on the acquired output sound data.

[0172] After that, the process proceeds to step S159.

[0173] On the other hand, in step S152, when it is determined that the rotation speed r ≤ the threshold value Rth, that is, when the translational speed of the autonomous mobile body 11 is equal to or lower than a predetermined threshold value, the processes from step S153 to step S158 are skipped, and the process proceeds to step S159.

[0174] In step S159, the recognition unit 251 sets the movement amount Δd to 0. That is, after the output of the translational sound or when the translational speed of the autonomous mobile body 11 becomes equal to or lower than a predetermined threshold value, the movement amount Δd is reset to 0.

[0175] After that, the process returns to step S151, and the processes after step S151 are executed.

[0176] Thereby, for example, when the translational speed of the autonomous mobile body 11 is equal to or lower than a predetermined threshold value, as shown in A of FIG. 22, the translational sound is not output. On the other hand, when the translational speed of the autonomous mobile body 11 exceeds a predetermined threshold value, as shown in B and C of FIG. 22, the translational sound is intermittently output while sandwiching a silent period. Also, as the speed increases, the harmonic components of the translational sound increase, and the output timing of the translational sound becomes denser.

[0177] In this way, the control method of the translational sound is changed depending on whether the ear-shaped parts are attached to the autonomous mobile body 11.

[0178] For example, when an ear-shaped part is attached to the autonomous mobile body 11, the content of the operating sound is changed to a sound imitating the movement sound of a cat. For example, when the autonomous mobile body 11 moves in the translational direction, the translational sound is not continuously output, but is intermittently output like the footsteps of a cat. Also, it is assumed that an actual cat kicks the ground harder as its moving speed increases, and the footsteps become harder sounds. Therefore, as the translational speed of the autonomous mobile body 11 increases, the harmonic components of the translational sound increase so that the sound becomes harder.

[0179] As described above, the user can strongly feel that the character of the autonomous mobile body 11 changes depending on whether the ear-shaped part is attached to the autonomous mobile body 11, improving the user's satisfaction.

[0180] Note that the timbre of the translational sound may be set to, for example, a timbre obtained by applying a variable v, an integer multiple of the variable v, or a value obtained by applying an exponential function to the variable v to a predetermined filter.

[0181] Also, for example, a sound of a predetermined waveform may be created or recorded, and the pitch and volume may be dynamically changed based on the variable v to generate the translational sound. Also, for example, translational sounds of a plurality of waveforms may be created or recorded, and the sound to be used may be switched based on the variable v. For example, two types of sounds, a soft ground-kicking sound and a hard ground-kicking sound, may be created in advance, and the translational sound may be generated by changing the ratio of synthesizing these sounds based on the variable v.

[0182] Furthermore, the rotation sound may be controlled in the same manner as the translational sound. For example, when the absolute value a of the angular velocity detected by the angular velocity sensor 121B exceeds a predetermined threshold value Ath, the rotation sound is output, and the variable v is set to the absolute value a - threshold value Ath of the angular velocity and used for controlling the rotation sound.

[0183] Also, the lifting sound may be controlled in the same manner as the translational sound and the rotational sound. In this case, for example, based on the difference between frames of acceleration detected by the acceleration sensor 121A, the lifting sound is modulated to represent the intensity of the lifting of the autonomous mobile body 11.

[0184] <Specific Example of Lifting Sound Output Control Process> When the situation of the autonomous mobile body 11 is recognized using a plurality of types of sensors respectively, the recognition characteristics such as the recognition speed and recognition accuracy may differ depending on the characteristics of each sensor.

[0185] For example, as described above with reference to FIG. 17, the lifting of the autonomous mobile body 11 is recognized using the IR sensor 402 and the acceleration sensor 121A. And as will be described later, there is a difference in the recognition characteristics of the lifting of the autonomous mobile body 11 between the case of using the IR sensor 402 and the case of using the acceleration sensor 121A.

[0186] On the other hand, by controlling the output sound by a control method according to the characteristics of each sensor, it is possible to improve the response performance and the range of expression of the output sound.

[0187] Here, with reference to FIG. 23, a specific example of the process for controlling the output of the lifting sound will be described.

[0188] This process is started, for example, when the power of the autonomous mobile body 11 is turned on and ends when it is turned off.

[0189] In step S201, the recognition unit 251 determines whether the lifting is recognized by the acceleration sensor 121A. If the recognition unit 251 does not recognize the lifting of the autonomous mobile body 11 based on the sensor data from the acceleration sensor 121A, it determines that the lifting is not recognized by the acceleration sensor 121A, and the process proceeds to step S202.

[0190] In step S202, the recognition unit 251 determines whether the lifting is recognized by the IR sensor 402. If the recognition unit 251 does not recognize the lifting of the autonomous mobile body 11 based on the sensor data from the IR sensor 402, it determines that the lifting is not recognized by the IR sensor 402, and the process returns to step S201.

[0191] After that, in step S201, until it is determined that the lifting is recognized by the acceleration sensor 121A or in step S202, until it is determined that the lifting is recognized by the IR sensor 402, the processes of step S201 and step S202 are repeatedly executed.

[0192] On the other hand, in step S202, when the recognition unit 251 recognizes the lifting of the autonomous mobile body 11 based on the sensor data from the IR sensor 402, it determines that the lifting is recognized by the IR sensor 402, and the process proceeds to step S203.

[0193] For example, when the IR sensor 402 is used, regardless of the way the autonomous mobile body 11 is lifted, the recognition accuracy is high. On the other hand, when the acceleration sensor 121A is used, the recognition accuracy is high when the autonomous mobile body 11 is quickly lifted, but the recognition accuracy is low when the autonomous mobile body 11 is slowly lifted. Also, when the acceleration sensor 121A is used, it is difficult to distinguish the lifting of the autonomous mobile body 11 from other movements.

[0194] Furthermore, generally, the sampling rate of the IR sensor 402 is lower than that of the acceleration sensor 121A. Therefore, when the IR sensor 402 is used, the recognition speed (response speed) of the lifting of the autonomous mobile body 11 may be slower compared to the case where the acceleration sensor 121A is used.

[0195] Therefore, the process proceeds to step S203 when the lifting of the autonomous mobile body 11 is recognized by the IR sensor 402 before the acceleration sensor 121A, and for example, it is assumed that the autonomous mobile body 11 is slowly lifted.

[0196] In step S203, the autonomous mobile body 11 outputs a predetermined lifting sound. Specifically, the recognition unit 251 notifies the sound control unit 254 that the autonomous mobile body 11 has been lifted. The sound control unit 254 generates output sound data for outputting a lifting sound with a predetermined pitch, volume, timbre, and speed, and supplies it to the sound output unit 205. The sound output unit 205 outputs a lifting sound based on the acquired output sound data.

[0197] Here, as described above with reference to FIG. 17, the amount of movement after the autonomous mobile body 11 is lifted is detected using the acceleration sensor 121A and the angular velocity sensor 121B. On the other hand, the IR sensor 402 cannot detect the amount of movement after the autonomous mobile body 11 is lifted. Therefore, when the lifting of the autonomous mobile body 11 is recognized by the IR sensor 402 before the acceleration sensor 121A, it is difficult to detect the amount of movement after the autonomous mobile body 11 is lifted.

[0198] Therefore, when the lifting of the autonomous mobile body 11 is recognized by the IR sensor 402 before the acceleration sensor 121A, regardless of how the autonomous mobile body 11 is lifted, a fixed lifting sound is output first.

[0199] After that, the process proceeds to step S204.

[0200] On the other hand, in step S201, when the recognition unit 251 recognizes the lifting of the autonomous mobile body 11 based on the sensor data from the acceleration sensor 121A, it determines that the lifting has been recognized by the acceleration sensor 121A, and the processes of step S202 and step S203 are skipped, and the process proceeds to step S204.

[0201] This is the case where the lifting of the autonomous mobile body 11 is recognized by the acceleration sensor 121A before or substantially simultaneously with the IR sensor 402. For example, it is assumed that the autonomous mobile body 11 is lifted quickly.

[0202] In step S204, the autonomous mobile body 11 outputs a lifting sound according to the lifting method.

[0203] Specifically, the recognition unit 251 detects the amount of movement after the autonomous mobile body 11 is lifted based on the sensor data from the acceleration sensor 121A and the angular velocity sensor 121B. The recognition unit 251 supplies the data indicating the detected amount of movement to the sound control unit 254.

[0204] The sound control unit 254 generates a lifting sound. At this time, the sound control unit 254 changes some of the parameters such as the pitch, volume, timbre, and speed of the lifting sound based on, for example, the change in the amount of movement after the autonomous mobile body 11 is lifted and the operation mode.

[0205] If a fixed lifting sound has been output previously by the process of step S203, the parameters of the lifting sound are set so that the connection with the fixed lifting sound is natural.

[0206] The sound control unit 254 generates output sound data for outputting the generated lifting sound and supplies it to the sound output unit 205.

[0207] The sound output unit 205 outputs a lifting sound based on the acquired output sound data.

[0208] Thereafter, the process returns to step S201, and the processes after step S201 are executed.

[0209] In this way, regardless of the lifting method of the autonomous mobile body 11, the lifting sound is output quickly. Also, a lifting sound according to the lifting method of the autonomous mobile body 11 is output.

[0210] As described above, according to the situation of the autonomous mobile body 11, particularly according to the combination with the combined device, the operation sound of appropriate content is output at an appropriate timing. Also, the responsiveness and the range of expression of the operation sound of the autonomous mobile body 11 are expanded. As a result, the user experience by the operation sound of the autonomous mobile body 11 is improved.

[0211] <<2. Modification Example>> Hereinafter, a modification example of the above-described embodiment of the present technology will be described.

[0212] The types of the above-described operation sounds, the types of the combined devices, and the method of controlling the operation sounds are merely examples and can be arbitrarily changed. For example, when the autonomous mobile body 11 rides on a robot-type vacuum cleaner, it may output an operation sound as if it is cleaning.

[0213] In the above description, an example of controlling the operation sound among the output sounds based on the combination with the combined device has been shown, but the speech sound may be controlled in the same manner.

[0214] For example, the user may be able to set a method for controlling the output sound when the autonomous mobile body 11 is combined with each combined device.

[0215] For example, the autonomous mobile body 11 may set a method for controlling the output sound based on the combined device and other conditions. For example, when the autonomous mobile body 11 is combined with the combined device, the method for controlling the output sound may be set according to conditions such as time (e.g., time zone, season, etc.) and location.

[0216] When the autonomous mobile body 11 is combined with other combined devices to form another autonomous mobile body (e.g., a robot, etc.), for example, the output sound may be controlled not by the autonomous mobile body 11 alone but by the entire newly formed autonomous mobile body.

[0217] For example, not only when the autonomous mobile body 11 comes into contact with the combined device, but also in a non-contact state, it may be recognized that the two are combined, and the method for controlling the output sound may be changed. For example, when the autonomous mobile body 11 approaches another combined device such as a robot, it may be recognized that the autonomous mobile body 11 and the combined device are combined, and the method for controlling the output sound may be changed.

[0218] In this case, for example, based on the images captured by cameras 102L and 102R, the approach of other combined devices is recognized. However, when using images, if the combined device is in the blind spot of the autonomous mobile body 11, the autonomous mobile body 11 cannot recognize the approach of the combined device. In contrast, as described above, short-range wireless communication may be further used to recognize the approach of the combined device.

[0219] For example, when the user wears the combined device and the autonomous mobile body 11 approaches the user wearing the combined device, the control method of the output sound may be changed. As a result, for example, when the user returns home, the autonomous mobile body 11 can greet the user and output a happy output sound.

[0220] For example, the autonomous mobile body 11 uses images captured by cameras 102L and 102R, a human presence sensor, etc., to recognize the user himself / herself regardless of the presence of the combined device, and based on the combination with the user, the control method of the output sound may be set.

[0221] For example, as the shape of the autonomous mobile body 11 changes, the control method of the output sound may be changed. For example, the control method of the output sound may be changed so as to output an output sound corresponding to a living thing or a character close to the shape after deformation of the autonomous mobile body 11.

[0222] For example, when an electronic device such as a smartphone different from the autonomous mobile body is combined with the combined device, the control method of the output sound of the electronic device may be changed. For example, when the smartphone is mounted on a turntable and moves by the turntable, the smartphone may output an operation sound corresponding to the rotation of the wheels of the turntable.

[0223] For example, when the autonomous mobile body 11 is combined with the combined device, the sensors of the combined device may be used to recognize the situation of the autonomous mobile body 11.

[0224] For example, as described above, the information processing server 12 can receive sensor data from the autonomous mobile body 11 and control the output sound of the autonomous mobile body 11 based on the received sensor data. Further, when the information processing server 12 controls the output sound of the autonomous mobile body 11, the information processing server 12 may generate the output sound, or the autonomous mobile body 11 may generate the output sound under the control of the information processing server 12.

[0225] <<3. Others>> <Configuration Example of Computer> The above-described series of processes can be executed by hardware or by software. When the series of processes is executed by software, the program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, and, for example, a general-purpose personal computer that can execute various functions by installing various programs.

[0226] FIG. 24 is a block diagram showing a configuration example of the hardware of a computer that executes the above-described series of processes by a program.

[0227] In the computer 1000, a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0228] Further connected to the bus 1004 is an input / output interface 1005. Connected to the input / output interface 1005 are an input unit 1006, an output unit 1007, a recording unit 1008, a communication unit 1009, and a drive 1010.

[0229] The input unit 1006 includes an input switch, buttons, a microphone, an imaging device, etc. The output unit 1007 includes a display, a speaker, etc. The recording unit 1008 includes a hard disk, a non-volatile memory, etc. The communication unit 1009 includes a network interface, etc. The drive 1010 drives a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0230] In the computer 1000 configured as described above, the CPU 1001 loads and executes, for example, a program recorded in the recording unit 1008 via the input / output interface 1005 and the bus 1004 into the RAM 1003, whereby the above-described series of processes are performed.

[0231] The program executed by the computer 1000 (CPU 1001) can be recorded and provided, for example, on a removable medium 1011 as a package medium or the like. Further, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0232] In the computer 1000, the program can be installed in the recording unit 1008 via the input / output interface 1005 by mounting the removable medium 1011 on the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the recording unit 1008. Additionally, the program can be installed in advance in the ROM 1002 or the recording unit 1008.

[0233] Note that the program executed by the computer may be a program in which processing is performed in time series in accordance with the order described in this specification, or may be a program in which processing is performed in parallel or at a necessary timing such as when a call is made.

[0234] In addition, in this specification, a system means a collection of a plurality of components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device in which a plurality of modules are housed in one housing are both systems.

[0235] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

[0236] For example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

[0237] In addition, each step described in the above flowchart can be executed by one device or can be shared and executed by a plurality of devices.

[0238] Furthermore, when a plurality of processes are included in one step, the plurality of processes included in that one step can be executed by one device or can be shared and executed by a plurality of devices.

[0239] <Example of configuration combination> The present technology can also adopt the following configuration.

[0240] (1) A recognition unit that recognizes a combined device that is a device combined with the own device, A sound control unit that changes a control method of an output sound output from the own device based on the recognition result of the combined device, and controls the output sound based on the changed control method An autonomous mobile body comprising. (2) The recognition unit further recognizes the situation of the own device, The sound control unit controls the output sound based on the situation of the own device. The autonomous mobile body described in the above (1). (3) When the recognition of a predetermined situation starts, the sound control unit makes the reaction speed of the output sound faster than the reaction speed of the output sound when the recognition of the predetermined situation ends. The autonomous mobile body described in the above (2). (4) When the recognition of the predetermined situation starts, the sound control unit quickly starts the output sound, and when the recognition of the predetermined situation ends, the sound control unit gently stops the output sound. The autonomous mobile body described in the above (3). (5) The predetermined situation is the movement of the self-device or a stimulus to the self-device. The autonomous mobile body described in the above (3) or (4). (6) The recognition unit recognizes the situation of the self-device by using a plurality of types of sensors respectively. Based on the type of the sensor used for the recognition of the situation of the self-device, the sound control unit changes the control method of the output sound. The autonomous mobile body described in any one of the above (2) to (5). (7) The sound control unit controls the output sound by a control method according to the characteristics of the sensor used for the recognition of the situation of the self-device. The autonomous mobile body described in the above (6). (8) Based on the recognition result of the combined device, the sound control unit changes at least one of the content of the output sound to be generated and the output timing of the output sound. The autonomous mobile body described in any one of the above (1) to (7). (9) The sound control unit changes the content of the output sound to the content corresponding to the type of the combined device combined with the self-device. The autonomous mobile body described in the above (8). (10) The sound control unit changes at least one of the pitch, volume, tone color, and speed of the output sound. The autonomous mobile body according to the above (8) or (9). (11) The sound control unit changes the control method of the output sound based on the recognized type of the combined device. The autonomous mobile body according to any one of the above (1) to (10). (12) The output sound includes a sound output corresponding to the movement of the self-device or a sound output corresponding to a stimulus to the self-device. The autonomous mobile body according to any one of the above (1) to (11). (13) The recognition unit recognizes the combined device based on sensor data from one or more sensors. The autonomous mobile body according to any one of the above (1) to (12). (14) The recognition unit recognizes the combined device based on a change in the movement of the self-device recognized based on the sensor data. The autonomous mobile body according to the above (13). (15) The combined device includes at least one of a part detachable from the self-device, a device detachable from the self-device, and a mobile body on which the self-device can ride. The autonomous mobile body according to any one of the above (1) to (14). (16) Recognize a combined device that is a device combined with an autonomous mobile body, Based on the recognition result of the combined device, change the control method of the output sound output from the autonomous mobile body, Control the output sound based on the changed control method. Information processing method. (17) Recognize a combined device that is a device combined with an autonomous mobile body, Based on the recognition result of the combined device, change the control method of the output sound output from the autonomous mobile body, Control the output sound based on the changed control method. A program for causing a computer to execute the processing. (18) A recognition unit that recognizes a combined device that is a device combined with an autonomous mobile body, a sound control unit that changes a control method of an output sound output from the autonomous mobile body based on the recognition result of the combined device and controls the output sound based on the changed control method An information processing apparatus comprising:

[0241] Note that the effects described in this specification are merely examples and are not limited, and there may be other effects.

Explanation of Signs

[0242] 1 Information processing system, 11 Autonomous mobile body, 12 Information processing server, 201 Control unit, 202 Sensor unit, 205 Sound output unit, 241 Information processing unit, 251 Recognition unit, 252 Action planning unit, 253 Motion control unit, 254 Sound control unit, 302 Recognition unit, 303 Action planning unit, 304 Motion control unit, 305 Sound control unit

Claims

1. A recognition unit that recognizes the status of the self-device and a combined device that is a device combined with the self-device, based on the recognition result of the combined device, changes the control method of the output sound output from the self-device, and based on the changed control method and the status of the self-device, a sound control unit that controls the output sound comprising: When the recognition of the movement of the self-device or the stimulus to the self-device is started, the sound control unit makes the reaction speed of the output sound faster than the reaction speed of the output sound when the recognition of the movement of the self-device or the stimulus to the self-device is completed An autonomous mobile body.

2. When the recognition of the movement of the self-device or the stimulus to the self-device is started, the sound control unit quickly starts the output sound, and when the recognition of the situation of the movement of the self-device or the stimulus to the self-device is completed, the sound control unit gently stops the output sound The autonomous mobile body according to Claim 1.

3. The recognition unit recognizes the status of the self-device by using a plurality of types of sensors respectively, based on the type of the sensor used for the recognition of the status of the self-device, the sound control unit changes the control method of the output sound The autonomous mobile body according to Claim 1.

4. The sound control unit controls the output sound by a control method according to the characteristics of the sensor used for the recognition of the status of the self-device The autonomous mobile body according to Claim 3.

5. Based on the recognition result of the combined device, the sound control unit changes at least one of the content of the output sound to be generated and the output timing of the output sound The autonomous mobile body according to Claim 1.

6. The sound control unit changes the content of the output sound to the content corresponding to the type of the combined device combined with the self-device The autonomous mobile body according to Claim 5.

7. The sound control unit changes at least one of the pitch, volume, timbre, and speed of the output sound The autonomous mobile body according to Claim 5.

8. Based on the recognized type of the combined device, the sound control unit changes the control method of the output sound The autonomous mobile body according to Claim 1.

9. The output sound includes a sound output corresponding to the movement of the self-device or a sound output corresponding to a stimulus to the self-device The autonomous mobile body according to Claim 1.

10. The recognition unit recognizes the combined device based on sensor data from one or more types of sensors The autonomous mobile body according to Claim 1.

11. The recognition unit recognizes the combined device based on a change in the movement of the own device recognized based on the sensor data. The autonomous mobile body according to claim 10.

12. The combined device includes at least one of a part detachable from the own device, a device detachable from the own device, and a moving body on which the own device can ride. The autonomous mobile body according to claim 1.

13. Recognize the situation of the autonomous mobile body and the combined device that is a device combined with the autonomous mobile body, Based on the recognition result of the combined device, change the control method of the output sound output from the autonomous mobile body, Based on the changed control method and the situation of the autonomous mobile body, control the output sound, Make the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body starts faster than the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body ends. Information processing method.

14. Recognize the situation of the autonomous mobile body and the combined device that is a device combined with the autonomous mobile body, Based on the recognition result of the combined device, change the control method of the output sound output from the autonomous mobile body, Based on the changed control method and the situation of the autonomous mobile body, control the output sound, Make the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body starts faster than the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body ends. A program for causing a computer to execute the process.

15. A recognition unit that recognizes the situation of the autonomous mobile body and the combined device that is a device combined with the autonomous mobile body, A sound control unit that changes the control method of the output sound output from the autonomous mobile body based on the recognition result of the combined device, and controls the output sound based on the changed control method and the situation of the autonomous mobile body Comprising The sound control unit makes the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body starts faster than the reaction speed of the output sound when the recognition of the movement of the autonomous mobile body or the stimulus to the autonomous mobile body ends. Information processing device.

Images