Autonomous mobile vehicle, information processing method, and program

Abstract

The present art relates to an autonomous mobile body, an information processing method, and a program, for making it possible to enhance power of expression by voice by the autonomous mobile body. The autonomous mobile body comprises: an operation control unit for controlling operation of the autonomous mobile body; a recognition unit for recognizing the state of the autonomous mobile body in operation; and a voice control unit for generating or processing voice data corresponding to a voice outputted in accordance with operation of the autonomous mobile body and controlling a characteristic and the output timing of the voice, on the basis of the result of recognition of the state of the autonomous mobile body. The present art is applicable to a pet-type robot, for example.

Description

Technical Field

[0001] The present technology relates to an autonomous mobile body, an information processing method, and a program, and particularly to an autonomous mobile body, an information processing method, and a program capable of communicating with a user.

Background Art

[0002] Conventionally, electronic devices that recognize a user's gesture and can be operated by the gesture have become widespread (see, for example, Patent Document 1).

[0003] In recent years, the popularity of pet-type robots that recognize a user's gesture or the like and can communicate with the user has been increasing.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, a pet-type robot performs not only movements necessary for the operation of the robot but also movements and performances that express intentions and emotions to the user. In this case, it is desired to improve the expressiveness of the pet-type robot by operation sounds, effect sounds, or the like.

[0006] The present technology has been made in view of such a situation, and aims to improve the expressive power of an autonomous mobile body such as a pet-type robot by voice.

Means for Solving the Problems

[0007] One aspect of this technology is an autonomous mobile body that moves autonomously, comprising: an operation control unit that controls the operation of the autonomous mobile body; a recognition unit that recognizes the state of the autonomous mobile body while it is in operation; and an audio control unit that generates or processes audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile body based on the recognition result of the state of the autonomous mobile body, and controls the characteristics and output timing of the audio.

[0008] One aspect of this technology is an information processing method that controls the operation of an autonomous mobile device, recognizes the state of the autonomous mobile device while it is in operation, generates or processes audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device based on the recognition result of the state of the autonomous mobile device, and controls the characteristics and output timing of the audio.

[0009] One aspect of this technology involves a program that controls the operation of an autonomous mobile device, recognizes the state of the autonomous mobile device while it is in operation, generates or processes audio data corresponding to the audio output in accordance with the operation of the autonomous mobile device based on the recognition result of the state of the autonomous mobile device, and causes a computer to execute a process that controls the characteristics and output timing of the audio.

[0010] In one aspect of this technology, the operation of an autonomous mobile device is controlled, the state of the autonomous mobile device during operation is recognized, and based on the recognition result of the state of the autonomous mobile device, audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device is generated or processed, and the characteristics and output timing of the audio are controlled. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing one embodiment of an information processing system to which this technology is applied. [Figure 2] This figure shows an example of the hardware configuration of an autonomous mobile device. [Figure 3] This is an example of the configuration of actuators used in an autonomous mobile vehicle. [Figure 4] This is a diagram illustrating the functions of the display on an autonomous mobile device. [Figure 5] It is a diagram showing an operation example of an autonomous mobile body. [Figure 6] It is a block diagram showing a functional configuration example of an autonomous mobile body. [Figure 7] It is a block diagram showing a functional configuration example of the information processing unit of an autonomous mobile body. [Figure 8] It is a diagram for explaining an example of a method for outputting voice of a conventional autonomous mobile body. [Figure 9] It is a diagram for explaining an example of a method for outputting voice of an autonomous mobile body. [Figure 10] It is a diagram for explaining an example of a method for outputting voice of an autonomous mobile body. [Figure 11] It is a diagram for explaining an example of a method for recognizing a petting method of an autonomous mobile body. [Figure 12] It is a diagram for explaining an example of a method for recognizing a petting method of an autonomous mobile body. [Figure 13] It is a diagram for explaining an example of a control algorithm for voice parameters. [Figure 14] It is a diagram showing an example of an opening angle. [Figure 15] It is a diagram showing an example of a transition diagram of the internal state of an autonomous mobile body. [Figure 16] It is a flowchart for explaining touch reaction sound control processing. [Figure 17] It is a flowchart for explaining a first embodiment of cry control processing. [Figure 18] It is a diagram for explaining an example of the relationship between the way of touch and the cry output. [Figure 19] It is a diagram for explaining an example of the relationship between the way of touch and the cry output. [Figure 20] It is a flowchart for explaining a second embodiment of cry control processing. [Figure 21] It is a flowchart for explaining a third embodiment of cry control processing. [Figure 22]It is a diagram showing an example of voice parameters of a chirping sound when an operation of touching and leaving the back of the autonomous mobile body is repeated at short intervals. [Figure 23] It is a diagram showing an example of voice parameters of a chirping sound when the hand is moved back and forth and stroking on the back of the autonomous mobile body. [Figure 24] It is a diagram showing an example of voice parameters of a chirping sound when the hand is continuously placed on the back of the autonomous mobile body without moving. [Figure 25] It is a flowchart for explaining the fourth embodiment of the chirping sound control process. [Figure 26] It is a flowchart for explaining the details of the reaction intensity detection process. [Figure 27] It is a diagram for explaining a method of detecting the reaction intensity. [Figure 28] It is a diagram showing an example of how to raise the pitch of a chirping sound. [Figure 29] It is a diagram showing an example of parameters used for setting a favorite place, etc. [Figure 30] It is a flowchart for explaining the details of the sleep talk control process. [Figure 31] It is a flowchart for explaining the motion sound control process. [Figure 32] It is a graph showing the relationship between the opening angle and voice parameters. [Figure 33] It is a diagram showing the relationship between the detection result of the sole button and the volume of the footsteps. [Figure 34] It is a diagram for explaining an example of a method of calculating voice parameters. [Figure 35] It is a graph showing the relationship between the opening angle and voice parameters. [Figure 36] It is a diagram showing an example of the configuration of a computer.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments for implementing the present technology will be described. The description will be made in the following order. 1. Embodiment 2. Variation 3. Others

[0013] <<1. Embodiment>> Embodiments of this technology will be described with reference to Figures 1 to 35.

[0014] <Example Configuration of Information Processing System 1> Figure 1 is a block diagram showing one embodiment of an information processing system 1 to which this technology is applied.

[0015] The information processing system 1 comprises autonomous mobile units 11-1 to 11-n, information processing terminals 12-1 to 12-n, and an information processing server 13.

[0016] Hereafter, when it is not necessary to distinguish between autonomous mobile units 11-1 through 11-n individually, they will simply be referred to as autonomous mobile unit 11. Hereafter, when it is not necessary to distinguish between information processing terminals 12-1 through 12-n individually, they will simply be referred to as information processing terminal 12.

[0017] Communication is possible via the network 21 between each autonomous mobile unit 11 and the information processing server 13, between each information processing terminal 12 and the information processing server 13, between each autonomous mobile unit 11 and each information processing terminal 12, between each autonomous mobile unit 11, and between each information processing terminal 12. Furthermore, direct communication is also possible between each autonomous mobile unit 11 and each information processing terminal 12, between each autonomous mobile unit 11, and between each information processing terminal 12 without using the network 21.

[0018] The autonomous mobile unit 11 is an information processing device that recognizes its own and its surroundings based on collected sensor data, and autonomously selects and executes various actions according to the situation. Unlike robots that simply perform actions according to user instructions, one of the features of the autonomous mobile unit 11 is that it autonomously performs appropriate actions according to the situation.

[0019] The autonomous mobile device 11 can, for example, perform user recognition or object recognition based on captured images, and perform various autonomous actions according to the recognized user or object. Furthermore, the autonomous mobile device 11 can, for example, perform speech recognition based on user utterances and take actions based on user instructions.

[0020] Furthermore, the autonomous mobile device 11 performs pattern recognition learning in order to acquire the ability to recognize users and objects. In this process, the autonomous mobile device 11 can not only perform supervised learning based on given learning data, but also dynamically collect learning data based on instructions from users, etc., and perform pattern recognition learning related to objects, etc.

[0021] Furthermore, the autonomous mobile device 11 can be trained by the user. Here, training the autonomous mobile device 11 is broader than general training, such as teaching and having it memorize rules and prohibitions, and refers to the changes that the user can perceive in the autonomous mobile device 11 as a result of the user's interaction with the autonomous mobile device 11.

[0022] The shape, capabilities, and needs of the autonomous mobile unit 11 can be appropriately designed according to its purpose and role. For example, the autonomous mobile unit 11 may consist of an autonomous mobile robot that moves autonomously within space and performs various actions. Specifically, for example, the autonomous mobile unit 11 may consist of an autonomous mobile robot that has a shape and movement capabilities that mimic a human or an animal such as a dog. Alternatively, for example, the autonomous mobile unit 11 may consist of a vehicle or other device that has the ability to communicate with a user.

[0023] The information processing terminal 12 consists of, for example, a smartphone, tablet, or PC (personal computer), and is used by the user of the autonomous mobile unit 11. The information processing terminal 12 performs various functions by executing a predetermined application program (hereinafter simply referred to as "application"). For example, the information processing terminal 12 manages and customizes the autonomous mobile unit 11 by executing a predetermined application.

[0024] For example, the information processing terminal 12 communicates with the information processing server 13 via the network 21 or communicates directly with the autonomous mobile unit 11 to collect various data about the autonomous mobile unit 11, present it to the user, or give instructions to the autonomous mobile unit 11.

[0025] The information processing server 13, for example, collects various data from each autonomous mobile unit 11 and each information processing terminal 12, provides various data to each autonomous mobile unit 11 and each information processing terminal 12, and controls the operation of each autonomous mobile unit 11. Furthermore, for example, based on the data collected from each autonomous mobile unit 11 and each information processing terminal 12, the information processing server 13 can perform pattern recognition learning and processing corresponding to user training, similar to the autonomous mobile units 11. In addition, for example, the information processing server 13 supplies various data related to the aforementioned applications and each autonomous mobile unit 11 to each information processing terminal 12.

[0026] Network 21 consists of several public network networks, such as the Internet, telephone networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks), including Ethernet®. Network 21 may also include dedicated network networks such as IP-VPN (Internet Protocol-Virtual Private Network). Network 21 may also include wireless communication networks such as Wi-Fi® and Bluetooth®.

[0027] The configuration of the information processing system 1 can be flexibly changed depending on the specifications and operation. For example, the autonomous mobile unit 11 may communicate with various external devices in addition to the information processing terminal 12 and the information processing server 13. These external devices may include, for example, servers that transmit weather, news, and other service information, as well as various home appliances owned by the user.

[0028] Furthermore, for example, the relationship between the autonomous mobile unit 11 and the information processing terminal 12 does not necessarily have to be one-to-one; for example, it may be many-to-many, many-to-one, or one-to-many. For example, one user can use one information processing terminal 12 to check data related to multiple autonomous mobile units 11, or use multiple information processing terminals to check data related to one autonomous mobile unit 11.

[0029] <Example hardware configuration of autonomous mobile unit 11> Next, we will describe an example of the hardware configuration of the autonomous mobile unit 11. In the following explanation, we will use the case where the autonomous mobile unit 11 is a dog-type quadruped robot as an example.

[0030] Figure 2 shows an example of the hardware configuration of the autonomous mobile robot 11. The autonomous mobile robot 11 is a dog-type quadrupedal robot equipped with a head, torso, four legs, and a tail.

[0031] The autonomous mobile unit 11 is equipped with two displays 51L and 51R on its head. Hereafter, when it is not necessary to distinguish between displays 51L and 51R individually, they will simply be referred to as "display 51".

[0032] Furthermore, the autonomous mobile unit 11 is equipped with various sensors. For example, the autonomous mobile unit 11 includes a microphone 52, a camera 53, a ToF (Time Of Flight) sensor 54, a human presence sensor 55, a distance measuring sensor 56, a touch sensor 57, an illuminance sensor 58, a sole button 59, and an inertial sensor 60.

[0033] The autonomous mobile device 11 is equipped with, for example, four microphones 52 on its head. Each microphone 52 collects ambient sounds, such as the user's speech and surrounding environmental sounds. By having multiple microphones 52, it is possible to collect sounds occurring in the surroundings with high sensitivity and to localize sound sources.

[0034] The autonomous mobile robot 11 is equipped with two wide-angle cameras 53, for example, on its nose and waist, to capture images of its surroundings. For example, the camera 53 positioned on its nose captures images within the autonomous mobile robot 11's forward field of view (i.e., the dog's field of view). The camera 53 positioned on its waist captures images of the area around the autonomous mobile robot 11, primarily above it. Based on the images captured by the camera 53 positioned on its waist, the autonomous mobile robot 11 can extract feature points from the ceiling and other elements to achieve SLAM (Simultaneous Localization and Mapping).

[0035] The ToF sensor 54 is, for example, located at the tip of the nose and detects the distance to an object located in front of the head. The autonomous mobile body 11 can accurately detect the distance to various objects using the ToF sensor 54, and can perform actions according to its relative position to objects such as the user or obstacles.

[0036] The human presence sensor 55 is positioned, for example, on the chest to detect the location of the user or a pet owned by the user. The autonomous mobile device 11 can detect an animal in front of it using the human presence sensor 55, and can perform various actions toward that animal, such as actions corresponding to emotions like interest, fear, or surprise.

[0037] The distance measuring sensor 56 is positioned, for example, on the chest, and detects the condition of the floor surface in front of the autonomous mobile body 11. The autonomous mobile body 11 can accurately detect the distance to an object on the floor surface in front of it using the distance measuring sensor 56, and can perform actions according to its relative position to the object.

[0038] The touch sensors 57 are positioned in areas where the user is likely to touch the autonomous mobile device 11, such as the top of the head, under the chin, or on the back, to detect user contact (touch). The touch sensors 57 are composed of, for example, capacitive or pressure-sensitive touch sensors. The autonomous mobile device 11 can detect user contact actions such as touching, stroking, tapping, or pressing using the touch sensors 57, and can perform actions corresponding to those contact actions. Furthermore, by arranging the touch sensors 57 linearly or in a planar manner on each part, it becomes possible to detect the location of touch within each part.

[0039] The illuminance sensor 58 is positioned, for example, on the back of the head, at the base of the tail, and detects the illuminance of the space in which the autonomous mobile body 11 is located. The autonomous mobile body 11 can detect the ambient brightness using the illuminance sensor 58 and perform actions according to that brightness.

[0040] The foot buttons 59 are positioned, for example, on the parts corresponding to the paw pads of each of the four legs, and detect whether or not the bottom surface of the legs of the autonomous mobile unit 11 is in contact with the floor. The autonomous mobile unit 11 can detect contact or non-contact with the floor surface using the foot buttons 59, and can, for example, understand that it has been picked up by a user.

[0041] The inertial sensors 60 are, for example, positioned in the head and torso, respectively, and detect physical quantities such as velocity, acceleration, and rotation of the head and torso. For example, the inertial sensors 60 are composed of a 6-axis sensor that detects acceleration and angular velocity in the X, Y, and Z axes. The autonomous mobile body 11 can accurately detect the movement of its head and torso using the inertial sensors 60, enabling situation-appropriate motion control.

[0042] The configuration of sensors equipped in the autonomous mobile unit 11 can be flexibly changed depending on the specifications and operation. For example, in addition to the above configuration, the autonomous mobile unit 11 may be further equipped with various communication devices, such as a temperature sensor, a geomagnetic sensor, and a GNSS (Global Navigation Satellite System) signal receiver.

[0043] Next, with reference to Figure 3, an example of the configuration of the joints of the autonomous mobile body 11 will be described. Figure 3 shows an example of the configuration of the actuators 71 provided by the autonomous mobile body 11. In addition to the rotation points shown in Figure 3, the autonomous mobile body 11 has a total of 22 degrees of freedom of rotation: two each in the ears and tail, and one in the mouth.

[0044] For example, the autonomous mobile robot 11 has three degrees of freedom in its head, enabling it to perform actions such as nodding and tilting its head. Furthermore, the autonomous mobile robot 11 can replicate the swinging motion of its waist using an actuator 71 located in its waist, thereby achieving more natural and flexible movements that are closer to those of a real dog.

[0045] The autonomous mobile body 11 may achieve the 22 rotational degrees of freedom described above by, for example, combining a single-axis actuator and a two-axis actuator. For example, single-axis actuators may be used in the elbow and knee areas of the legs, and two-axis actuators may be used in the shoulders and thigh joints.

[0046] Next, referring to Figure 4, the functions of the display 51 provided by the autonomous mobile unit 11 will be explained.

[0047] The autonomous mobile robot 11 is equipped with two displays 51R and 51L, corresponding to the right and left eyes, respectively. Each display 51 has the function of visually representing the eye movements and emotions of the autonomous mobile robot 11. For example, each display 51 can represent the movements of the eyeballs, pupils, and eyelids in accordance with emotions and actions, thereby creating natural movements similar to those of real animals such as dogs, and expressing the gaze and emotions of the autonomous mobile robot 11 with high precision and flexibility. In addition, users can intuitively grasp the state of the autonomous mobile robot 11 from the eye movements displayed on the displays 51.

[0048] Furthermore, each display 51 is realized, for example, by two independent OLEDs (Organic Light Emitting Diodes). By using OLEDs, it becomes possible to reproduce the curved surface of the eyeball. As a result, a more natural appearance can be achieved compared to representing a pair of eyeballs with a single flat display or representing two eyeballs with two independent flat displays.

[0049] With the above configuration, the autonomous mobile unit 11 can reproduce movements and emotional expressions that are closer to those of real living organisms by precisely and flexibly controlling the movements of its joints and eyeballs, as shown in Figure 5.

[0050] Figure 5 shows an example of the operation of the autonomous mobile unit 11. In Figure 5, the external structure of the autonomous mobile unit 11 is simplified in order to focus on the operation of the joints and eyeballs of the autonomous mobile unit 11.

[0051] <Example of functional configuration of autonomous mobile unit 11> Next, with reference to Figure 6, an example of the functional configuration of the autonomous mobile unit 11 will be described. The autonomous mobile unit 11 comprises an input unit 101, a communication unit 102, an information processing unit 103, a drive unit 104, an output unit 105, and a storage unit 106.

[0052] The input unit 101 includes the aforementioned microphone 52, camera 53, ToF sensor 54, human presence sensor 55, distance sensor 56, touch sensor 57, illuminance sensor 58, foot button 59, and inertial sensor 60, and has the function of collecting various sensor data about the user and surrounding conditions. The input unit 101 also includes input devices 121 such as switches and buttons. The input unit 101 supplies the collected sensor data and input data input via the input devices to the information processing unit 103.

[0053] The communication unit 102 communicates with other autonomous mobile units 11, information processing terminals 12, and information processing servers 13 via the network 21 or without the network 21, and sends and receives various types of data. The communication unit 102 supplies received data to the information processing unit 103 and obtains data to be transmitted from the information processing unit 103.

[0054] Furthermore, the communication method of the communication unit 102 is not particularly limited and can be flexibly changed according to the specifications and operation.

[0055] The information processing unit 103 is equipped with a processor such as a CPU (Central Processing Unit) and performs various information processing and controls various parts of the autonomous mobile unit 11.

[0056] The drive unit 104 flexes and extends multiple joints of the autonomous mobile body 11 based on control by the information processing unit 103. More specifically, the drive unit 104 drives the actuators 71 provided in each joint based on control by the information processing unit 103. The drive unit 104 also supplies drive data indicating the operating angle of each actuator to the information processing unit 103.

[0057] The output unit 105 includes, for example, a display 51, a speaker, a haptic device, etc., and outputs visual information, auditory information, tactile information, etc., based on control by the information processing unit 103.

[0058] The memory unit 106 includes, for example, non-volatile memory and volatile memory, and stores various programs and data.

[0059] In the following, when each part of the autonomous mobile unit 11 communicates with the information processing server 13, etc., via the communication unit 102 and the network 21, the phrase "via the communication unit 102 and the network 21" will be omitted as appropriate. For example, when the recognition unit 151 communicates with the information processing server 13 via the communication unit 102 and the network 21, it will simply be stated that the recognition unit 151 communicates with the information processing server 13.

[0060] <Example configuration of information processing unit 103> Figure 7 shows an example of the functional configuration of the information processing unit 103 in Figure 6. The information processing unit 103 comprises a recognition unit 151, a learning unit 152, and an action control unit 153.

[0061] The recognition unit 151 recognizes the situation in which the autonomous mobile body 11 is located, based on the sensor data and input data supplied from the input unit 101, the received data supplied from the communication unit 102, and the drive data supplied from the drive unit 104.

[0062] The situation in which the autonomous mobile unit 11 is located includes, for example, the situation of itself and its surroundings. The situation of itself includes, for example, the state and movement of the autonomous mobile unit 11. The surrounding situation includes, for example, the state, movement and instructions of people around it such as users, the state and movement of living things around it such as pets, the state and movement of objects around it, time, place, and the surrounding environment. Objects around it include, for example, other autonomous mobile units. In addition, the recognition unit 151 performs, for example, person identification, facial expression and gaze recognition, emotion recognition, object recognition, motion recognition, spatial area recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, temperature recognition, speech recognition, word comprehension, position estimation, posture estimation, etc., in order to recognize the situation.

[0063] Furthermore, the recognition unit 151 has the function of estimating and understanding the situation based on the various types of information it has recognized. For example, the recognition unit 151 recognizes stimuli given to the autonomous mobile body 11 from the outside, and the person who gave the stimuli. The stimuli to be recognized include, for example, visual stimuli, auditory stimuli, and tactile stimuli. In this case, the recognition unit 151 may make an overall estimation of the situation using knowledge that has been stored in advance.

[0064] The recognition unit 151 supplies data indicating the recognition or estimation result of the situation (hereinafter referred to as "situation data") to the learning unit 152 and the behavior control unit 153. The recognition unit 151 also registers the situation data into the behavior history data stored in the storage unit 106.

[0065] The behavioral history data is data that shows the history of the actions of the autonomous mobile unit 11. The behavioral history data includes items such as the date and time when the action started, the date and time when the action ended, the trigger for performing the action, the location where the action was instructed (if a location was specified), the circumstances when the action was performed, and whether or not the action was completed (whether the action was performed to the end).

[0066] The triggers for an action include, for example, if the action is triggered by a user instruction, the content of that instruction is registered. Also, for example, if the action is triggered by a predetermined situation, the details of that situation are registered. Furthermore, for example, if the action is triggered by an object indicated by the user or an object that has been recognized, the type of that object is registered.

[0067] The learning unit 152 learns the situation and actions, and the effects of those actions on the environment, based on one or more of the following: sensor data and input data supplied from the input unit 101, received data supplied from the communication unit 102, drive data supplied from the drive unit 104, situation data supplied from the recognition unit 151, and data on the actions of the autonomous mobile body 11 supplied from the action control unit 153. For example, the learning unit 152 may perform the pattern recognition learning described above, or learn behavioral patterns that correspond to user training.

[0068] For example, the learning unit 152 achieves the above learning using machine learning algorithms such as deep learning. Note that the learning algorithm employed by the learning unit 152 is not limited to the above example and can be designed as appropriate.

[0069] The learning unit 152 supplies data indicating the learning results (hereinafter referred to as learning result data) to the action control unit 153 or stores it in the memory unit 106.

[0070] The behavior control unit 153 controls the behavior of the autonomous mobile unit 11 based on the recognized or estimated situation and the learning result data. The behavior control unit 153 supplies data related to the behavior of the autonomous mobile unit 11 to the learning unit 152 and registers it in the behavior history data stored in the storage unit 106. The behavior control unit 153 includes an internal state control unit 161, an operation control unit 162, and a voice control unit 163.

[0071] The internal state control unit 161 controls the internal state of the autonomous mobile device 11 based on the recognized or estimated situation and the learned data. For example, the internal state control unit 161 controls the state transitions of the internal state of the autonomous mobile device 11.

[0072] The internal state of the autonomous mobile device 11 is an internal state that is not visible externally, and is set based on at least one of the following: the behavior, physical condition, emotions, age, and battery level of the autonomous mobile device 11. The physical condition of the autonomous mobile device 11 includes, for example, hunger level. Hunger level is set based on, for example, the time elapsed since the autonomous mobile device 11 performed the action of eating food. The age of the autonomous mobile device 11 is set based on, for example, the time elapsed since the date of purchase of the autonomous mobile device 11 or the date it was first powered on, or the total operating time of the autonomous mobile device 11.

[0073] The motion control unit 162 controls the drive unit 104 and the output unit 105 based on at least one of the recognized or estimated situation, learned result data, and the internal state of the autonomous mobile unit 11, thereby controlling the actions necessary for the autonomous mobile unit 11 to perform various actions. For example, the motion control unit 162 performs rotation control of the actuator 71 and display control of the display 51.

[0074] Furthermore, the actions of the autonomous mobile unit 11 include, for example, actions necessary for the operation of the autonomous mobile unit 11, actions that express intentions or emotions to the user, and performances. Hereinafter, the latter actions will be referred to as motion.

[0075] The data necessary to realize motion (hereinafter referred to as motion data) is, for example, created in advance using an authoring tool and stored in the memory unit 106 during the manufacturing of the autonomous mobile unit 11. Alternatively, for example, the motion data is downloaded to the autonomous mobile unit 11 from an information processing terminal 12 or an information processing server 13.

[0076] The motion control unit 162 controls the drive unit 104 and the output unit 105 based on motion data, thereby causing the autonomous mobile body 11 to perform motion.

[0077] The voice control unit 163 generates and processes voice data corresponding to the voice output by the autonomous mobile unit 11, and controls the characteristics and output timing of the voice.

[0078] Here, the characteristics of the sound to be controlled include, for example, the type of sound (e.g., animal sounds, conversation sounds, etc.), content, features (e.g., pitch, volume, timbre, etc.), and sound quality. The content of the sound includes, for example, the content of the conversation in the case of conversation sounds. Controlling the timing of sound output also includes controlling whether or not sound is output.

[0079] The audio control unit 163 includes an audio output control unit 171 and an audio processing unit 172.

[0080] The voice output control unit 171 controls the generation and processing of voice data by the voice processing unit 172. For example, the voice output control unit 171 controls the transition of the voice processing unit 172's pronunciation mode based on at least one of the following: the behavior of the autonomous mobile unit 11, the state of the autonomous mobile unit 11, the person who gave the stimulus, the surrounding circumstances, and the content of the stimulus.

[0081] The pronunciation mode is, for example, a mode for switching the algorithm and parameters used by the speech processing unit 172 to generate or process speech data. By switching the algorithm and parameters used to generate speech data, for example, the characteristics of the speech and the output timing based on the speech data generated by the speech processing unit 172 change.

[0082] The voice output control unit 171 controls the output of voice from the output unit 105 based on one or more of the recognized or estimated situation, the learned result data, and the internal state set by the internal state control unit 161. For example, the voice output control unit 171 supplies voice data generated or processed by the voice processing unit 172 to the speaker provided in the output unit 105 and controls the output of voice from the speaker.

[0083] The voice processing unit 172 holds, for example, programs and control scripts for generating or processing voice data. Based on one or more of the recognized or estimated situation, the learning result data, and the internal state of the autonomous mobile unit 11, the voice processing unit 172 controls the characteristics of the voice by generating or processing voice data corresponding to the voice output by the autonomous mobile unit 11.

[0084] The sounds output by the autonomous mobile device 11 include, for example, sounds used by the autonomous mobile device 11 to communicate with the user or to express its state or emotions, sounds associated with the autonomous mobile device 11's movements, and sounds used to enhance the performance of the autonomous mobile device 11. The sounds used by the autonomous mobile device 11 to communicate with the user or to express its state or emotions include, for example, animal sounds, conversation sounds, sleep talking, etc. Sounds of movement include, for example, animal sounds, footsteps, etc. Sounds used for performance include, for example, sound effects, music, etc.

[0085] Furthermore, the sounds output by the autonomous mobile unit 11 include, for example, sounds that are output or change in response to external stimuli (hereinafter referred to as stimulus response sounds), and sounds that are output or change in accordance with (in conjunction with) the movements of the autonomous mobile unit 11. Stimulus response sounds include, for example, animal sounds, conversation sounds, sleep talking, etc. Sounds that are output or change in accordance with the movements of the autonomous mobile unit 11 include, for example, operation sounds and performance sounds.

[0086] Hereinafter, among the sounds that are output or change in accordance with the movements of the autonomous mobile unit 11, sounds that are output or change in accordance with motion will be referred to as motion sounds.

[0087] <Example of how to output audio> Next, with reference to Figures 8 to 14, an example of how the autonomous mobile unit 11 outputs sound will be described.

[0088] First, with reference to Figure 8, we will explain an example of a conventional method for outputting voice from an autonomous mobile device.

[0089] For example, an external computer 201 generates and processes audio data corresponding to the voice of the autonomous mobile device 202, and stores it in the autonomous mobile device 202. The autonomous mobile device 202 then outputs audio based on the stored audio data.

[0090] Therefore, the autonomous mobile unit 202 can only output sounds that correspond to pre-stored sound data. To increase the types of sounds that can be output and enhance the expressiveness of the autonomous mobile unit 11, it is necessary to generate many types of sound data in advance and store them in the autonomous mobile unit 11.

[0091] Next, with reference to Figure 9, an example of how the autonomous mobile unit 11 outputs sound will be described.

[0092] For example, there are two methods for outputting voice from the autonomous mobile device 11, as shown in Figures 9A and 9B.

[0093] For example, as shown in Figure 9A, an external computer 201 generates and processes audio data corresponding to the voice of the autonomous mobile unit 202, and stores it in the storage unit 106 of the autonomous mobile unit 11. The audio processing unit 172 processes the audio data stored in the storage unit 106 as needed (for example, modulating the audio data). The audio output control unit 171 outputs the audio based on the audio data processed by the audio processing unit 172 from the output unit 105.

[0094] Furthermore, as shown in Figure 9B, for example, the voice processing unit 172 of the autonomous mobile unit 11 generates and processes voice data corresponding to the voice to be output (for example, by performing numerical calculations on the voice data). The voice output control unit 171 outputs voice based on the voice data generated or processed by the voice processing unit 172 from the output unit 105.

[0095] Therefore, the autonomous mobile device 11 can output various types of audio depending on the situation, without having to prepare many types of audio data in advance.

[0096] For example, the autonomous mobile device 11 can generate and output sounds that respond to user actions (e.g., petting the autonomous mobile device 11), as shown in Figure 10A.

[0097] Specifically, for example, the input unit 101 of the autonomous mobile device 11 supplies sensor data corresponding to the user's actions to the recognition unit 151. The recognition unit 151 recognizes the user's actions based on the sensor data. For example, the recognition unit 151 recognizes how the user strokes the autonomous mobile device 11 based on the sensor data from the touch sensor 57.

[0098] Here, with reference to Figures 11 and 12, an example of a method for recognizing how to stroke the autonomous mobile body 11 will be explained.

[0099] For example, as shown in Figure 11, touch sensors 57-1 to 57-4 are mounted on the back of the autonomous mobile body 11, arranged in a front-to-back direction. Touch sensor 57-1 is positioned at the very front, and touch sensor 57-4 is positioned at the very back.

[0100] Figure 12 shows examples of the waveforms of sensor data S1 to S4 output from touch sensors 57-1 to 57-4 when a user strokes the back of the autonomous mobile body 11 from front to back, as shown in Figure 11. In the graph of Figure 12, the horizontal axis represents time, and the vertical axis represents the value of the sensor data.

[0101] As shown in this example, the frontmost touch sensor 57-1 reacts first, and sensor data S1 changes. Next, the second touch sensor from the front, 57-2, reacts, and sensor data S2 changes. Then, the third touch sensor from the front, 57-3, reacts, and sensor data S3 changes. Finally, the rearmost touch sensor 57-4 reacts, and sensor data S4 changes.

[0102] The recognition unit 151 can recognize the position, strength, speed, and direction of a stroke by analyzing changes in the sensor data of each touch sensor 57. For example, the recognition unit 151 can recognize patterns (stroking styles) such as whether the stroke was fast, slow, light, or thorough.

[0103] The recognition unit 151 then supplies situation data, including the action recognition result, to the action control unit 153.

[0104] The voice processing unit 172 calculates voice parameters used to control the output voice based on the recognized action.

[0105] Audio parameters are parameters used to control the characteristics of sound. Specifically, examples of audio parameters include frequency, volume, modulation degree, harmonic components, the degree of application of a low-pass filter, and the degree of application of an effect.

[0106] The voice processing unit 172 can generate various types of voice (corresponding voice data) in response to recognized actions by controlling each voice parameter.

[0107] For example, a dog's bark (woof) can be reproduced by raising and then lowering the frequency of the sound. For example, the tone of the bark changes depending on the fundamental frequency. For example, a loud dog bark can be represented by lowering the fundamental frequency of the bark. For example, a soft dog bark can be represented by raising the fundamental frequency of the bark. For example, an excited emotion can be represented by raising the fundamental frequency of the bark. For example, a strong emotional fluctuation can be represented by increasing the rate of change of the bark's frequency (amount of change per unit time).

[0108] For example, the degree to which a low-pass filter is applied controls the vowels contained in the speech. Specifically, the degree to which the low-pass filter is applied allows for the reproduction of sounds similar to "a" (pronounced with the mouth wide open) and sounds similar to "u" (pronounced with the mouth slightly open).

[0109] Furthermore, each audio parameter has a direction that expresses a strong response to external stimuli and a direction that expresses a weak response. For example, the louder the volume, the stronger the response expressed, and the quieter the volume, the weaker the response expressed. For example, the higher the frequency, the stronger the response expressed, and the lower the frequency, the weaker the response expressed.

[0110] The voice processing unit 172 generates voice data based on the calculated voice parameters. The voice output control unit 171 controls the timing of voice output from the output unit 105 based on the recognized action.

[0111] Furthermore, if the autonomous mobile device 11 directly controls the output timing and audio parameters based on sensor data, there is a risk that audio may be output at unnatural timings or that audio may be output in response to noise components in the sensor data. In contrast, the autonomous mobile device 11 controls the output timing and audio parameters based on actions recognized based on sensor data, so that it can output appropriate audio at the appropriate timing.

[0112] Furthermore, for example, the autonomous mobile unit 11 can generate and output audio corresponding to its internal state, as shown in Figure 10B.

[0113] For example, the internal state control unit 161 sets the internal state of the autonomous mobile device 11 based on the situation recognized or estimated by the recognition unit 151. The voice processing unit 172 calculates voice parameters used to control the output voice based on the internal state of the autonomous mobile device 11 set by the internal state control unit 161. The voice processing unit 172 generates voice data based on the calculated voice parameters. The voice output control unit 171 controls the timing of voice output from the output unit 105 based on the internal state of the autonomous mobile device 11 set by the internal state control unit 161.

[0114] This allows the autonomous mobile device 11 to output the appropriate sound at the appropriate time based on its internal state.

[0115] <Example of a control algorithm for audio parameters> Next, with reference to Figure 13, an example of a control algorithm for the voice parameters of the autonomous mobile unit 11 will be described. Here, the control algorithm for the voice parameters of the autonomous mobile unit 11's vocalizations will be explained as a specific example.

[0116] For example, the recognition unit 151 detects the angle at which the autonomous mobile body 11 opens its mouth (hereinafter referred to as the opening angle) based on the drive data from the drive unit 104. For example, as shown in Figure 14, the angle θ centered on the corner of the mouth of the autonomous mobile body 11 is detected as the opening angle.

[0117] For example, if the mouth of the autonomous mobile unit 11 is displayed on the screen and does not physically open, the angle at which the displayed mouth is open is used as the opening angle.

[0118] For example, the recognition unit 151 recognizes the amount and location of a touch on the head based on the sensor value of the touch sensor 57 on the head of the autonomous mobile body 11. For example, the recognition unit 151 recognizes the amount and location of a touch on the torso based on the sensor value of the touch sensor 57 on the torso of the autonomous mobile body 11.

[0119] Here, the amount of touch is detected based on the sensor value of the touch sensor 57, and represents, for example, the intensity of the touch.

[0120] Furthermore, the amount of touch may be calculated based on the intensity and duration of the touch (for example, based on the product of the intensity and duration of the touch). This introduces the concept of time into the amount of touch.

[0121] Alternatively, the amount of touch may be calculated based on the intensity of the touch and the area of ​​the touched region (for example, based on the product of the intensity of the touch and the area of ​​the touched region). This introduces the concept of area into the amount of touch.

[0122] Furthermore, for example, the amount of touch may be calculated based on the intensity of the touch, the duration of the touch, and the area being touched.

[0123] For example, the recognition unit 151 recognizes whether or not the autonomous mobile body 11 touches a location that elicits a strong positive response (hereinafter referred to as a "favorite location"), and how it is being stroked, based on the amount and location of touches on the head and body of the autonomous mobile body 11. The stroking method is represented, for example, by the strength, location, speed, and direction of the strokes.

[0124] For example, the sound output control unit 171 controls the timing of the sound output (including whether or not a sound is output) based on the opening angle and the way the animal is being stroked.

[0125] For example, the audio processing unit 172 controls the vowel of the bark based on the mouth opening angle. For example, when a real dog barks, the sound corresponding to the vowel of the bark changes depending on how wide the mouth is opened. For example, when a dog barks, if the mouth is opened wide, the sound is closer to 'a', and when the mouth is opened narrowly, the sound is closer to 'u'. This vowel control is achieved by adjusting the degree to which the low-pass filter is applied, as described above.

[0126] For example, the voice processing unit 172 controls the pitch and volume of the chirp based on whether or not a favorite spot is touched. For example, if a favorite spot is touched, the pitch and volume of the chirp are increased.

[0127] For example, the sound processing unit 172 controls the pitch of the chirp based on how it is being petted. For instance, the pitch (frequency) of the chirp changes in accordance with changes in the strength and position of the petting.

[0128] For example, the sound output control unit 171 detects the elapsed time since the last vocalization. The elapsed time since the last vocalization is, for example, the time elapsed since the last vocalization ended.

[0129] For example, the audio processing unit 172 controls the velocity of the cry (e.g., the attack of the cry) based on the elapsed time since the previous cry output.

[0130] <Example of state transitions in the internal state of the autonomous mobile device 11> Next, with reference to Figure 15, an example of a state transition in the internal state of the autonomous mobile unit 11 will be explained.

[0131] For example, the internal state of the autonomous mobile device 11 transitions between a normal state, a relaxed state, a drowsy state, and an unhappy state in response to external stimuli, etc., under the control of the internal state control unit 161.

[0132] For example, when the autonomous mobile unit 11 is powered on, its internal state is set to the normal state as the initial state. The normal state is a neutral state in which the emotions of the autonomous mobile unit 11 are neither positive nor negative.

[0133] For example, in the normal state, if the user provides a certain amount of positive stimulation, the internal state will transition in a positive direction and transition to a relaxed state. The relaxed state is a state in which the autonomous mobile device 11 is relaxed and at ease.

[0134] Positive stimuli are, for example, stimuli that the autonomous mobile device 11 likes. For example, stroking the autonomous mobile device 11 or talking to the autonomous mobile device 11 would be considered positive stimuli.

[0135] For example, if the autonomous mobile device 11 is performing a predetermined action such as walking or eating, even if a positive stimulus is applied, its internal state remains in its normal state and does not transition to a relaxed state.

[0136] For example, if the internal state is relaxed, and the user provides a certain amount of positive stimulation, it will transition in a positive direction and transition to a drowsy state. The drowsy state is a state in which the autonomous mobile device 11 is satisfied and sleepy, or is drowsy.

[0137] For example, if the internal state is relaxed or drowsy and no stimulation is applied for an extended period, it will transition in a negative direction and then back to a normal state.

[0138] For example, in its normal state, if no stimulus is applied for an extended period, the internal state will transition in a negative direction, transitioning to a grumpy state. A grumpy state is a state in which the emotions of the autonomous mobile entity are negative. For example, a grumpy state is when autonomous mobile entity 11 is angry, sulking, or dissatisfied.

[0139] For example, if the autonomous mobile device 11 is performing a predetermined action such as walking or eating, even if it continues without receiving any stimulation, its internal state will remain normal and will not transition to an unhappy state.

[0140] For example, if the internal state is in an unhappy state, then if the user provides a certain amount of positive stimulation, it will transition in a positive direction and return to a normal state.

[0141] Furthermore, the detailed transition conditions between internal states change depending on, for example, the user providing the stimulus and the method of stimulation.

[0142] For example, when petted gently, the internal state shifts to a positive direction in a shorter time compared to when petted normally. For example, when petted roughly, the internal state either does not shift or shifts to a negative direction.

[0143] For example, if the recognition unit 151 recognizes that a user who frequently interacts with the autonomous mobile body 11 is stroking it, the time required for the internal state to transition to a positive state will be shortened. For example, if the recognition unit 151 fails to recognize the user stroking it, the time required for the internal state to transition to a positive state will be lengthened.

[0144] Figure 15 shows an example of internal states and state transitions, and these can be modified. For example, the number of internal states can be increased or decreased, or the conditions and destinations of state transitions can be changed. For instance, internal states could be configured to transition based on negative stimuli from the user.

[0145] <Processing by autonomous mobile unit 11> Next, the processing of the autonomous mobile unit 11 will be explained with reference to Figures 16 to 35.

[0146] <Touch response sound control processing> First, we will explain the touch response sound control process performed by the autonomous mobile unit 11, referring to the flowchart in Figure 16.

[0147] A touch-response sound is a type of stimulus-response sound, which is an audio output in response to touching the autonomous mobile device 11, or an audio that changes in response to touching the autonomous mobile device 11. Here, we will explain examples in which animal sounds and sleep talk are used as touch-response sounds.

[0148] This process starts, for example, when the power to the autonomous mobile device 11 is turned on, and ends when the power to the autonomous mobile device 11 is turned off.

[0149] In step S1, the audio output control unit 171 is set to the no-response mode. That is, the audio output control unit 171 sets the pronunciation mode of the audio processing unit 172 to the no-response mode as the initial mode.

[0150] In step S2, the voice output control unit 171 determines whether the conditions for entering the petting-to-sing mode have been met. For example, if the internal state of the autonomous mobile body 11 is relaxed, the voice output control unit 171 determines that the conditions for entering the petting-to-sing mode have been met, and the process proceeds to step S3.

[0151] In step S3, the autonomous mobile unit 11 enters the mode of making noise when petted. Specifically, the voice output control unit 171 sets the sound production mode of the voice processing unit 172 to the mode of making noise when petted. This causes the sound production mode to transition from the unresponsive mode to the mode of making noise when petted.

[0152] Furthermore, while the autonomous mobile unit 11 is set to the mode that makes noise when petted, it will not output any sounds that express its state or emotions other than the noise it makes in response to touch. However, it is still possible to output operational sounds and sound effects.

[0153] In step S4, the autonomous mobile unit 11 performs a sound control process. The details of the sound control process will be described later, but this process causes the autonomous mobile unit 11 to emit a sound in response to a touch.

[0154] In step S5, the voice output control unit 171 determines whether the conditions for exiting the petting-and-meow mode have been met. If it is determined that the conditions for exiting the petting-and-meow mode have not been met, the process returns to step S4.

[0155] Subsequently, in step S5, the processes of steps S4 and S5 are repeatedly executed until it is determined that the conditions for exiting the petting-and-meowing mode are met. During this time, the autonomous mobile unit 11 emits a meow in response to touch.

[0156] On the other hand, in step S5, for example, if the internal state of the autonomous mobile body 11 is anything other than the relaxed state, the voice output control unit 171 determines that the condition for exiting the petting-and-singing mode has been met, and the process proceeds to step S6.

[0157] In step S6, the autonomous mobile unit 11 exits the mode of making noise when petted. Specifically, the voice output control unit 171 sets the sound generation mode of the voice processing unit 172 to the no-response mode. This causes the sound generation mode to transition from the mode of making noise when petted to the no-response mode.

[0158] The process then proceeds to step S7.

[0159] On the other hand, if it is determined in step S2 that the conditions for entering the "petting makes noise" mode are not met, the processing in steps S3 through S6 is skipped, and the process proceeds to step S7.

[0160] In step S7, the voice output control unit 171 determines whether the conditions for entering sleep-talking mode have been met. For example, if the internal state of the autonomous mobile unit 11 is in a drowsy state, the voice output control unit 171 determines that the conditions for entering sleep-talking mode have been met, and the process proceeds to step S8.

[0161] In step S8, the autonomous mobile unit 11 enters sleep-talking mode. Specifically, the voice output control unit 171 sets the pronunciation mode of the voice processing unit 172 to sleep-talking mode. This causes the pronunciation mode to transition from unresponsive mode to sleep-talking mode.

[0162] Furthermore, while the autonomous mobile unit 11 is set to sleep-talking mode, it will not output any sounds other than sleep-talking that express its state or emotions. However, it is still possible to output operational sounds and sound effects.

[0163] In step S9, the autonomous mobile unit 11 performs sleep-talking control processing. Details of the sleep-talking control processing will be described later, but this process causes the autonomous mobile unit 11 to output sleep-talking. After the autonomous mobile unit 11 outputs sleep-talking, the pronunciation mode transitions from sleep-talking mode to unresponsive mode.

[0164] After that, the process returns to step S2, and the processes from step S2 onward are executed.

[0165] On the other hand, in step S7, for example, if the internal state of the autonomous mobile body 11 is anything other than a drowsy state, the voice output control unit 171 determines that the conditions for entering sleep-talking mode are not met, and the process returns to step S2, and the processes from step S2 onward are executed.

[0166] <First embodiment of the sound control processing> Next, with reference to the flowchart in Figure 17, a first embodiment of the sound control process in step S4 of Figure 16 will be described.

[0167] In step S21, the audio output control unit 171 determines, based on the status data from the recognition unit 151, whether or not a touch exceeding a predetermined amount has been detected. If it is determined that a touch exceeding a predetermined amount has been detected, the process proceeds to step S22.

[0168] In step S22, the sound output control unit 171 determines whether a time equal to or greater than the shortest output interval has elapsed since the last sound was output. If the elapsed time since the last sound output ended is equal to or greater than the shortest output interval which is a predetermined threshold, the sound output control unit 171 determines that a time equal to or greater than the shortest output interval has elapsed since the last sound was output, and the process proceeds to step S23.

[0169] The minimum output interval may be either a fixed value or a variable value. If the minimum output interval is a variable value, for example, it will fluctuate almost randomly within a predetermined time range. This will cause the interval at which the autonomous mobile unit 11 emits a bark when touched to fluctuate randomly, resulting in a response that is closer to that of a real dog.

[0170] In step S23, the autonomous mobile unit 11 outputs a sound depending on the touch state.

[0171] Specifically, the voice output control unit 171 instructs the voice processing unit 172 to generate a sound.

[0172] The voice processing unit 172 recognizes the touch state (e.g., touch amount, touch position, etc.) based on the status data from the recognition unit 151. Based on the touch state, the voice processing unit 172 calculates the voice parameters of the sound to be output. Based on the calculated voice parameters, the voice processing unit 172 generates voice data corresponding to the sound.

[0173] The audio output control unit 171 supplies the audio data generated by the audio processing unit 172 to the output unit 105.

[0174] The output unit 105 outputs a sound based on the audio data. At this time, the motion control unit 162 may control the drive unit 104 to move the mouth and body of the autonomous mobile unit 11 in accordance with the output of the sound.

[0175] Furthermore, if the autonomous mobile unit 11 is not in a state suitable for outputting vocalizations, it may be prevented from outputting vocalizations. A state unsuitable for outputting vocalizations includes, for example, when performing a predetermined action such as walking or eating, when outputting sounds other than vocalizations (e.g., conversation sounds), or when the mouth is physically unable to open due to being covered. However, if the unit is outputting sounds that can be output simultaneously with vocalizations, such as operation sounds or sound effects, it will not be determined that the unit is unsuitable for outputting vocalizations.

[0176] After that, the sound control process ends.

[0177] Here, with reference to Figures 18 and 19, we will explain an example of the relationship between how to touch the device and the resulting sound.

[0178] Figures 18 and 19 show the relationship between the amount of touch and the frequency and volume of the outputted sound.

[0179] Figure 18 shows an example of a vocalization when the duration of the touch is less than or equal to the maximum vocalization response time Tmax.

[0180] For example, at time t1a when the touch begins, the sound output starts, and the frequency and volume of the sound gradually increase until time t2a when the touch ends, reaching a maximum at time t2a. The maximum values ​​of the sound frequency and volume change depending on the amount of touch. At time t2a, after the touch ends, the frequency and volume of the sound decrease, and at time t3a, the sound output ends.

[0181] In this way, the autonomous mobile device 11 continuously emits a sound while being touched, and changes the frequency and volume of the sound according to the amount of touch. Furthermore, the autonomous mobile device 11 fades out the sound after the touch ends. This achieves a natural sound that responds to touch.

[0182] Figure 19 shows an example of a vocalization when the duration of the touch exceeds the maximum vocalization response time Tmax.

[0183] For example, at time t1b when the touch is initiated, the sound output begins, and the frequency and volume of the sound gradually increase until time t2b, when the maximum sound response time Tmax has elapsed, reaching a maximum at time t2b. The maximum values ​​of the sound frequency and volume change depending on the amount of touch. After time t2b, regardless of the amount of touch, the sound frequency and volume decrease, and the sound output ends at time t3b, before time t4b when the touch ends.

[0184] This prevents unnatural, continuous sound output when the device is touched for an extended period, resulting in more natural-sounding sounds.

[0185] The maximum barking response time Tmax may be either a fixed value or a variable value. If the maximum barking response time Tmax is a variable value, for example, it will fluctuate almost randomly within a predetermined time range. This will cause the time it takes for the autonomous mobile unit 11 to emit a bark in response to a prolonged touch to fluctuate randomly, resulting in a response that is closer to that of a real dog.

[0186] Returning to Figure 17, if it is determined in step S22 that less than the minimum output interval has elapsed since the last vocalization, the process in step S23 is skipped, and the vocalization control process ends. In other words, if less than the minimum output interval has elapsed since the last vocalization, no vocalization will be output even if a touch exceeding a predetermined touch amount is detected.

[0187] This prevents repeated short-interval vocalizations, for example, when touches are made repeatedly at short intervals or when touches are made continuously for a long period of time, resulting in more natural-sounding vocalizations.

[0188] Furthermore, if it is determined in step S21 that no touch exceeding a predetermined amount has been detected, steps S22 and S23 are skipped, and the sound control process ends. In other words, if no touch exceeding a predetermined amount has been detected, no sound is output.

[0189] <Second embodiment of the sound control processing> Next, with reference to the flowchart in Figure 20, a second embodiment of the sound control process in step S4 of Figure 16 will be described.

[0190] In step S41, similar to the process in step S21 in Figure 17, it is determined whether a touch exceeding a predetermined amount has been detected. If it is determined that a touch exceeding a predetermined amount has been detected, the process proceeds to step S42.

[0191] In step S42, the audio output control unit 171 sets the minimum output interval based on the touch amount. For example, if the touch amount is less than a predetermined threshold, the audio output control unit 171 sets the minimum output interval to a predetermined time. On the other hand, if the touch amount is greater than or equal to a predetermined threshold, the audio output control unit 171 shortens the minimum output interval to less than a predetermined time. For example, the audio output control unit 171 shortens the minimum output interval as the touch amount increases.

[0192] In step S43, similar to the process in step S22 in Figure 17, it is determined whether a time equal to or greater than the shortest output interval has elapsed since the last vocalization. If it is determined that a time equal to or greater than the shortest output interval has elapsed since the last vocalization, the process proceeds to step S44.

[0193] In step S44, a sound is output depending on the touch state, similar to the process in step S23 in Figure 17.

[0194] On the other hand, if it is determined in step S43 that the time elapsed since the last vocalization has not been equal to or greater than the shortest output interval, the process in step S44 is skipped and the vocalization control process ends.

[0195] Furthermore, if it is determined in step S41 that no touch exceeding a predetermined amount has been detected, the processes in steps S42 to S44 are skipped, and the sound control process ends.

[0196] In this way, the interval between vocalizations changes based on the amount of touch. For example, if the autonomous mobile device 11 is touched or stroked forcefully, it will start emitting vocalizations at shorter intervals. This results in a more natural response to touch.

[0197] <Third embodiment of the sound control processing> Next, with reference to the flowchart in Figure 21, a third embodiment of the sound control process in step S4 of Figure 16 will be described.

[0198] In step S61, similar to the process in step S21 in Figure 17, it is determined whether a touch exceeding a predetermined amount has been detected. If it is determined that a touch exceeding a predetermined amount has been detected, the process proceeds to step S62.

[0199] In step S62, similar to the process in step S22 in Figure 17, it is determined whether a time equal to or greater than the shortest output interval has elapsed since the last vocalization. If it is determined that a time equal to or greater than the shortest output interval has elapsed since the last vocalization, the process proceeds to step S63.

[0200] In step S63, the voice output control unit 171 determines whether a touch has continued at the same location since the last time the sound output was stopped. For example, the voice output control unit 171 recognizes the touch location based on the status data from the recognition unit 151 and detects the amount of movement of the touch location since the last time the sound output was stopped, based on the recognized touch location. If the detected amount of movement of the touch location is greater than or equal to a predetermined threshold, the voice output control unit 171 determines that a touch has not continued at the same location since the last time the sound output was stopped, and the process proceeds to step S64. This also includes cases where there was a period of time when no touch was detected since the last time the sound output was stopped.

[0201] In step S64, the sound output control unit 171 determines whether the sound is continuous with the previous sound. For example, if the sound output control unit 171 determines that the sound is not continuous with the previous sound if a time threshold has elapsed since the end of the previous sound output, the process proceeds to step S65.

[0202] The continuous detection threshold is set to a time longer than the shortest output interval.

[0203] In step S65, the autonomous mobile unit 11 begins outputting its first call. The first call is either a single, non-consecutive call, or the first call in a sequence of calls.

[0204] Specifically, the voice output control unit 171 instructs the voice processing unit 172 to generate the first sound.

[0205] The voice processing unit 172 begins generating voice data corresponding to the first sound. Specifically, the voice processing unit 172 recognizes the touch state based on the situation data from the recognition unit 151, and calculates the voice parameters of the sound to be output based on the touch state. The voice processing unit 172 generates voice data corresponding to the sound based on the calculated voice parameters.

[0206] The audio output control unit 171 starts supplying the audio data generated by the audio processing unit 172 to the output unit 105.

[0207] The output unit 105 starts outputting a sound based on the audio data.

[0208] The process then proceeds to step S67.

[0209] On the other hand, in step S64, for example, if the audio output control unit 171 determines that the sound is continuous with the previous sound if a time equal to or greater than the continuity determination threshold has not elapsed since the end of the previous sound output, the process proceeds to step S66.

[0210] Furthermore, if three or more sounds, including the sound in question, are emitted at intervals shorter than the consecutive detection threshold, those three or more sounds will be judged to be consecutive.

[0211] In step S66, the autonomous mobile unit 11 begins outputting the second and subsequent sounds.

[0212] Specifically, the audio output control unit 171 instructs the audio processing unit 172 to generate the second and subsequent calls. Here, the second and subsequent calls refer to the calls that are output the second and subsequent times among the calls that have been determined to be consecutive.

[0213] The voice processing unit 172 starts generating voice data. Specifically, the voice processing unit 172 recognizes the touch state based on the status data from the recognition unit 151 and calculates the voice parameters of the sound to be output based on the touch state. At this time, for example, the voice processing unit 172 sets the velocity to a lower value than the first sound. The voice processing unit 172 generates voice data corresponding to the sound based on the calculated voice parameters.

[0214] The audio output control unit 171 starts supplying the audio data generated by the audio processing unit 172 to the output unit 105.

[0215] The output unit 105 starts outputting a sound based on the audio data.

[0216] As a result, the first call in a sequence will have a more suppressed attack.

[0217] The process then proceeds to step S67.

[0218] In step S67, the audio output control unit 171 determines whether the touch amount is above a certain level based on the status data from the recognition unit 151. If it is determined that the touch amount is above a certain level, the process proceeds to step S68.

[0219] In step S68, the voice output control unit 171 determines whether or not the autonomous mobile body 11 is being petted based on the status data from the recognition unit 151. If it is determined that the autonomous mobile body 11 is being petted, the process proceeds to step S69.

[0220] In step S69, the sound output control unit 171 determines whether or not a time equal to or greater than the first maximum output time has elapsed since the start of the sound output. If it is determined that a time equal to or greater than the first maximum output time has not elapsed since the start of the sound output, the process returns to step S67.

[0221] Subsequently, steps S67 to S70 are repeatedly executed until it is determined in step S67 that the touch amount is not above a certain level, or in step S69 that a time equal to or greater than the first maximum output time has elapsed since the start of the sound output, or in step S70 that a time equal to or greater than the second maximum output time has elapsed since the start of the sound output.

[0222] On the other hand, if it is determined in step S68 that the autonomous mobile body 11 is not being stroked, for example, if the touch position is fixed and does not move, the process proceeds to step S70.

[0223] In step S70, the sound output control unit 171 determines whether a time equal to or greater than the second maximum output time has elapsed since the start of the sound output. The second maximum output time is set to a time shorter than the first maximum output time in step S69. If it is determined that a time equal to or greater than the second maximum output time has elapsed since the start of the sound output, the process returns to step S67.

[0224] Subsequently, steps S67 to S70 are repeatedly executed until it is determined in step S67 that the touch amount is not above a certain level, or in step S69 that a time equal to or greater than the first maximum output time has elapsed since the start of the sound output, or in step S70 that a time equal to or greater than the second maximum output time has elapsed since the start of the sound output.

[0225] On the other hand, if it is determined in step S70 that a time equal to or greater than the second maximum output time has elapsed since the start of the sound output, the process proceeds to step S71. This occurs, for example, when the same position has been touched for a period of time equal to or greater than the second maximum time.

[0226] Furthermore, if it is determined in step S69 that a period of time equal to or greater than the first maximum output time has elapsed since the start of vocalization, the process proceeds to step S71. This occurs, for example, when the petting continues for longer than the first maximum time.

[0227] Furthermore, if it is determined in step S67 that the amount of touch is less than a certain level, the process proceeds to step S71. This occurs, for example, when the touch to the autonomous mobile body 11 is completed.

[0228] In step S71, the autonomous mobile unit 11 stops outputting sounds. Specifically, the voice output control unit 171 stops supplying voice data to the output unit 105. The voice output control unit 171 also instructs the voice processing unit 172 to stop generating sounds. The voice processing unit 172 stops generating voice data.

[0229] After that, the sound control process ends.

[0230] On the other hand, in step S63, if the amount of movement of the detected touch position is less than a predetermined threshold, the voice output control unit 171 determines that a touch has continued at the same position since the last time the sound output was stopped, and the processing in steps S64 to S71 is skipped, and the sound control processing ends.

[0231] Furthermore, if it is determined in step S62 that a time equal to or less than the shortest output interval has not elapsed since the last vocalization, the processing in steps S63 to S71 is skipped, and the vocalization control process ends.

[0232] Furthermore, if it is determined in step S61 that no touch exceeding a predetermined amount has been detected, the processes in steps S62 to S71 are skipped, and the sound control process ends.

[0233] Here, with reference to Figures 22 to 24, an example of a cry output in the third embodiment of the cry control processing will be described. Figures 20B to 22B show the time series transitions of the touch amount, the touch position in the front-to-back direction on the back of the autonomous mobile body 11, and the velocity and pitch of the cry.

[0234] Figure 22 shows an example of the sounds produced when the user repeatedly touches and removes the back of the autonomous mobile unit 11 at short intervals, as shown in Figure 22A.

[0235] Specifically, the first touch occurs between time t1c and time t2c, the second touch occurs between time t3c and time t4c, and the third touch occurs between time t5c and time t6c. The amount and location of the first through third touches are almost the same.

[0236] For example, if the interval between time t2c and time t3c, and the interval between time t4c and time t5c are less than the continuity determination threshold, the first to third sounds will be determined to be consecutive.

[0237] In this case, for example, the velocity of the second and third calls is made weaker than the velocity of the first call. Also, the pitch of the first through third calls is set to be almost the same. As a result, the attack of the second and subsequent calls is suppressed compared to the attack of the first call. That is, the change in volume and pitch at the beginning of the second and subsequent calls becomes more gradual compared to the first call.

[0238] As a result, for example, the user perceives the second and subsequent calls as less aggressive than the first call. Furthermore, the user gets the impression that the first through third calls are continuous, with temporary interruptions due to breathing, allowing them to perceive the autonomous mobile unit's calls as more natural.

[0239] Figure 23 shows an example of a sound produced when a user moves their hand back and forth on the back of the autonomous mobile unit 11 and strokes it, as shown in Figure 23A.

[0240] Specifically, touches occurred continuously from time t1d to time t4d. The amount of touches during this period was almost the same, except at the start and end, and the touch position moved back and forth.

[0241] In this case, for example, the first sound is emitted from time t1d until time t2d, when the first longest output time has elapsed, and the first sound stops being emitted at time t2d. Next, the second sound starts being emitted at time t3d, when the shortest output interval has elapsed from time t2d, and the second sound stops being emitted at time t4d.

[0242] In this case, for example, the velocity of the second call is made weaker than the velocity of the first call. As a result, the attack of the second call is suppressed compared to the attack of the first call. In addition, the pitch of the calls changes depending on the touch position in both the first and second calls.

[0243] As a result, even if the back of the autonomous mobile unit 11 is continuously stroked, the sound will not be emitted for an unnaturally long time, and short pauses will be provided. In addition, the user will get the impression that the first and second sounds are consecutive and temporarily interrupted by a breath, allowing them to perceive the sounds of the autonomous mobile unit 11 as natural.

[0244] Figure 24 shows an example of a sound produced when the user keeps their hand on the back of the autonomous mobile unit 11 without moving it, as shown in Figure 24A.

[0245] Specifically, touches occur continuously from time t1e to time t3e. The amount of touches during this period is almost the same, except for the start and end of each touch. Also, the touch locations during this period are almost the same.

[0246] In this case, for example, the sound is emitted from time t1e until time t2e, when the second longest output time has elapsed, and then the sound stops being emitted from time t2e onward.

[0247] As a result, when the touch position does not move, the duration of the vocalization is shorter compared to when the touch position moves (when being petted). In other words, when the touch position moves (when being petted), the duration of the vocalization is longer compared to when the touch position does not move.

[0248] As a result, even if a hand remains on the back of the autonomous mobile unit 11, the vocalization will stop without unnaturally prolonging the output. Also, since the touch state (stimulus) does not change except for the hand remaining on the back, the vocalization will not resume.

[0249] In addition, in the third embodiment of the sound control processing, the shortest output interval may be changed based on the touch amount, similar to the second embodiment.

[0250] <Fourth embodiment of the sound control processing> Next, with reference to the flowchart in Figure 25, a fourth embodiment of the sound control process in step S4 of Figure 16 will be described.

[0251] In step S81, similar to the process in step S21 in Figure 17, it is determined whether a touch exceeding a predetermined amount has been detected. If it is determined that a touch exceeding a predetermined amount has been detected, the process proceeds to step S82.

[0252] In step S82, similar to the process in step S22 in Figure 17, it is determined whether a time equal to or greater than the shortest output interval has elapsed since the last vocalization. If it is determined that a time equal to or greater than the shortest output interval has elapsed since the last vocalization, the process proceeds to step S83.

[0253] In step S83, the recognition unit 151 performs a reaction intensity detection process. The details of the reaction intensity detection process will be described later, but this process detects the reaction intensity based on the touched position, etc.

[0254] In step S84, the autonomous mobile unit 11 outputs a sound according to the response intensity. For example, the autonomous mobile unit 11 uses the response intensity instead of the touch amount to perform a process similar to the process in step S23 of Figure 17. As a result, for example in Figures 18 and 19, the frequency and volume of the sound are controlled based on the response intensity instead of the touch amount.

[0255] After that, the sound control process ends.

[0256] On the other hand, in step S82, if it is determined that the time elapsed since the previous sound output is less than the shortest output interval, the processes of steps S83 and S84 are skipped, and the sound control process ends.

[0257] Also, in step S81, if it is determined that no touch equal to or greater than a predetermined touch amount is detected, the processes of steps S82 to S84 are skipped, and the sound control process ends.

[0258] Note that in the fourth embodiment of the sound control process, similar to the second embodiment, the shortest output interval may be changed based on the touch amount.

[0259] <Details of Reaction Strength Detection Process> Next, referring to the flowchart of FIG. 26, the details of the reaction strength detection process in step S83 of FIG. 25 will be described.

[0260] In step S101, the recognition unit 151 acquires individual parameters. For example, the recognition unit 151 acquires individual parameters related to the individual of the autonomous mobile body 11 stored in the storage unit 106. Alternatively, for example, the recognition unit 151 receives the individual parameters of the autonomous mobile body 11 from the information processing terminal 12 or the information processing server 13.

[0261] Here, examples of individual parameters will be described.

[0262] For example, parameters unique to each autonomous mobile body 11 are used as individual parameters. For example, the product number or the like of the autonomous mobile body 11 is used as an individual parameter.

[0263] For example, parameters representing the characteristics of the autonomous mobile body 11 are used as individual parameters. For example, the color of the autonomous mobile body 11, the type of organism represented by the autonomous mobile body 11, the type of dog represented by the autonomous mobile body 11, etc. are used as individual parameters.

[0264] For example, parameters representing the attributes given by the user to the autonomous mobile body 11 are used as individual parameters. For example, the name, birthday, etc. of the autonomous mobile body 11 are used as individual parameters. For example, the age based on the birthday of the autonomous mobile body 11 is used as an individual parameter.

[0265] In step S102, the recognition unit 151 sets a reaction reference point based on the individual parameters. For example, the recognition unit 151 converts the character string representing the individual parameters into a numerical value using a hash function or the like, and sets a reaction reference point based on the obtained numerical value.

[0266] The reaction reference point is a point that serves as a reference for detecting the intensity of the reaction (reaction intensity) of the autonomous mobile body 11 to a touch. For example, the closer the touch position is to the reaction reference point, the stronger the reaction intensity, and the farther the touch position is from the reaction reference point, the weaker the reaction intensity. Based on this reaction reference point, the reaction pattern of the autonomous mobile body 11 to a touch is set.

[0267] For example, as shown in FIG. 27, based on the individual parameters, a reaction reference point P0 is set within the reaction area A1 of the autonomous mobile body 11. The reaction area A1 is, for example, the range in which the touch sensor 57 provided on the back of the autonomous mobile body 11 can react (the range in which a touch can be detected).

[0268] For example, when the individual parameter is a parameter unique to the autonomous mobile body 11, the reaction reference point is set at a position unique to the autonomous mobile body 11. For example, when the individual parameter is a parameter representing the characteristics or attributes of the autonomous mobile body 11, in the autonomous mobile body 11 having the same characteristics or attributes, the reaction reference point is set at the same position.

[0269] In step S103, the recognition unit 151 detects the touch position based on the sensor data from the touch sensor 57.

[0270] In step S104, the recognition unit 151 detects the distance between the reaction reference point and the touch position.

[0271] For example, as shown in Figure 27, when a touch position P1 is detected, the distance d between the reaction reference point P0 and the touch position P1 is detected.

[0272] In step S105, the recognition unit 151 calculates the reaction intensity based on the detected distance.

[0273] For example, the recognition unit 151 normalizes the distance between the touch position and the reaction reference point to a range of 0 to 1 by dividing the distance d by the size of the reaction area A1 (for example, the radius of the reaction area A1). The recognition unit 151 sets the reciprocal of the normalized distance as the reaction intensity.

[0274] Alternatively, for example, the recognition unit 151 converts the distance d into a reaction intensity using a predetermined function.

[0275] Furthermore, the reaction intensity increases as the distance d decreases, and decreases as the distance d increases. In other words, the closer the touch position P1 is to the reaction reference point P0, the greater the reaction intensity, and the further the touch position P1 is from the reaction reference point P0, the smaller the reaction intensity.

[0276] The recognition unit 151 supplies situation data, including the detected reaction intensity, to the behavior control unit 153.

[0277] After that, the reaction intensity detection process ends.

[0278] As a result, a different reaction reference point is set for each autonomous mobile unit 11, or for each autonomous mobile unit 11 that has the same characteristics or attributes. This causes the reaction pattern to change for each autonomous mobile unit 11 in response to the same touch. For example, the characteristics and timing of the sound output in response to the same touch will change for each autonomous mobile unit 11. This expresses the individuality of each autonomous mobile unit 11 and allows users to feel more attached to them.

[0279] Furthermore, since it is not necessary to prepare many types of voice data to express the individuality of each autonomous mobile unit 11, the production cost and amount of voice data can be reduced.

[0280] In the above explanation, an example was shown in which a reaction reference point is set on the back of the autonomous mobile body 11. However, the reaction reference point can be set at any position on the surface of the autonomous mobile body 11 where touch detection is possible.

[0281] Furthermore, based on the individual parameters of each autonomous mobile unit 11, a different reaction reference point may be set for each autonomous mobile unit 11 (for example, the head, jaw, abdomen, back, etc.).

[0282] <Modified version of the fourth embodiment of the sound control processing> Here, a modified example of the fourth embodiment of the above-described sound control process will be explained with reference to Figures 25 and 26.

[0283] For example, the recognition unit 151 may set strong reaction locations based on reaction reference points. Strong reaction locations are places where the autonomous mobile unit 11 reacts more strongly than other locations. For example, strong reaction locations are divided into favorite locations where the autonomous mobile unit 11 shows a stronger positive reaction than other locations, and locations where the autonomous mobile unit 11 shows a stronger negative reaction than other locations (hereinafter referred to as averse locations). The response pattern of the autonomous mobile unit 11 to touch is set based on these strong reaction locations.

[0284] Alternatively, for example, the learning unit 152 may perform learning processing based on behavioral history data stored in the memory unit 106 and set strong response locations to set the response pattern of the autonomous mobile unit 11 to touch.

[0285] Specifically, as described above, the action history data registers data indicating the recognition result or estimation result of the situation by the recognition unit 151. For example, the learning unit 152 detects the site and the position within the site that have been stroked in the past, and the time of stroking based on the action history data. Then, the learning unit 152 sets the site with the largest amount of stroking in the past as the favorite location. For example, among the head, chin, abdomen, and back, if the amount of stroking of the head is the largest, the head is set as the favorite location.

[0286] Note that, for example, the cumulative value of the stroking time is used as the amount of stroking. However, the time when the hand is just placed without moving is not included in the stroking time.

[0287] Also, for example, the recognition unit 151 sets the location where the autonomous mobile body 11 is damaged as the disliked location, thereby setting the reaction pattern of the autonomous mobile body 11 to touch. Note that a physically defective location may be set as the disliked location, or a location where a virtual injury is assumed to have occurred may be set as the disliked location.

[0288] For example, the voice processing unit 172 changes the cry when the strongly reactive location is touched and when a location other than the strongly reactive location is touched. Specifically, for example, when the favorite location is touched, the voice processing unit 172 raises the pitch of the cry more than when a location other than the favorite location is touched.

[0289] FIG. 28 shows an example of how to raise the pitch of the cry. The horizontal axis of the graphs A and B in FIG. 28 indicates time, and the vertical axis indicates pitch. The dotted curve C1a in FIG. 28A shows the pitch characteristics of the cry when a location other than the favorite location is touched, and the dashed-dotted curve C1b shows the pitch characteristics of the cry when the favorite location is touched. The dotted curve C2a in FIG. 28B shows the pitch characteristics of the cry when a location other than the favorite location is touched, and the dashed-dotted curve C2b shows the pitch characteristics of the cry when the favorite location is touched.

[0290] In example A of Figure 28, the pitch is raised mainly around the peak of the waveform of the bird's call. In example B of Figure 28, the entire waveform of the bird's call's pitch is raised. Comparing example A of Figure 28 with example B of Figure 28, it is easier for the user to notice the change in the pitch of the bird's call in example B of Figure 28.

[0291] Furthermore, the volume may be increased along with the pitch of the calls.

[0292] Furthermore, for example, the autonomous mobile unit 11 may, conversely, lower the pitch of its vocalizations when an aversion location is touched compared to when a location other than an aversion location is touched.

[0293] Furthermore, for example, the voice output control unit 171 changes the frequency of outputting sounds depending on whether a favorite spot is stroked or a part other than the favorite spot is stroked. For example, when a favorite spot is stroked, the voice output control unit 171 shortens the minimum output interval compared to when a part other than the favorite spot is stroked. Alternatively, for example, when a favorite spot is touched, the voice output control unit 171 reduces the amount of touch used to determine the output of a sound in step S21 of Figure 17, etc., compared to when a part other than the favorite spot is touched. As a result, when a favorite spot is stroked, the frequency of sound output is higher than when a part other than the favorite spot is stroked.

[0294] Alternatively, instead of setting a favorite location, the characteristics and timing of the vocalizations may change for each body part based on the amount of time it has been petted in the past.

[0295] For example, the voice processing unit 172 controls voice parameters for each body part based on the amount of time it has been petted in the past. For instance, if a body part that has been petted a lot in the past is petted, the voice processing unit 172 increases the volume or frequency of the meow. Conversely, for example, if a body part that has been petted little in the past is petted, the voice output control unit 171 decreases the volume or frequency of the meow.

[0296] For example, the voice output control unit 171 shortens the minimum output interval or reduces the amount of touch used to determine the output of a sound for areas that have been petted a lot in the past. Conversely, the voice output control unit 171 lengthens the minimum output interval or increases the amount of touch used to determine the output of a sound for areas that have been petted a little in the past.

[0297] Figure 29 shows an example of parameters used for setting favorite locations, etc.

[0298] `individual_voice_param` is a parameter that indicates individual differences in the voice quality of each autonomous mobile unit 11. `individual_voice_param` is set to a value within the range of 0.0 to 1.0.

[0299] `chin_weight` is a parameter that indicates the degree to which the autonomous mobile device 11 prefers to be touched on its chin. `chin_weight` is set to a value within the range of 0.0 to 1.0. For example, the closer `chin_weight` is to 1.0, the higher the autonomous mobile device 11's preference for being touched on its chin. That is, the more positive the autonomous mobile device 11 will react to being touched on its chin. The closer `chin_weight` is to 0.0, the lower the autonomous mobile device 11's preference for being touched on its chin. That is, the less positive the autonomous mobile device 11 will react to being touched on its chin. Alternatively, the more negative the autonomous mobile device 11 will react to being touched on its chin.

[0300] head_weight, like chin_weight, is a parameter that indicates the degree of preference for touching the head of the autonomous mobile unit 11.

[0301] body_forward_weight, like chin_weight and head_weight, is a parameter that indicates the degree of preference for touching the front part of the autonomous mobile body 11 (e.g., the abdomen).

[0302] body_back_weight, like chin_weight, head_weight, and body_forward_weight, is a parameter that indicates the degree of preference for touching the rear part of the autonomous mobile body 11 (e.g., the back).

[0303] Furthermore, the recognition unit 151 may set strong reaction locations not on a whole-body basis, but on a regional basis within a body part. For example, the recognition unit 151 may set the region that has been stroked the most within the body part that has been stroked the most in the past as a strong reaction location.

[0304] <Sleep-talking control processing> Next, referring to the flowchart in Figure 30, we will explain the details of the sleep-talking control process in step S8 of Figure 16.

[0305] In step S201, the autonomous mobile unit 11 begins to output sleep-talking.

[0306] Specifically, the voice output control unit 171 instructs the voice processing unit 172 to generate sleep-talking.

[0307] The voice processing unit 172 begins generating and supplying voice data corresponding to sleep talking. Specifically, the voice processing unit 172 calculates the voice parameters of the sleep talking to be output based on the situation data from the recognition unit 151. Based on the calculated voice parameters, the voice processing unit 172 generates voice data corresponding to the sleep talking.

[0308] The voice processing unit 172 may, for example, detect the day's activities based on the activity history data stored in the memory unit 106, and set the content of the sleep talk based on the day's activities.

[0309] The audio output control unit 171 starts supplying the audio data generated by the audio processing unit 172 to the output unit 105.

[0310] The output unit 105 starts outputting sleep talk based on the audio data.

[0311] Furthermore, sleep-talking is output only for a predetermined period of time. The output time for sleep-talking can be either a fixed value or a variable value. If the output time for sleep-talking is a variable value, for example, the output time will fluctuate almost randomly within a predetermined range. This will result in more natural-sounding sleep-talking.

[0312] In step S202, the voice processing unit 172 determines whether or not an external stimulus has been applied based on the situation data from the recognition unit 151. If it is determined that an external stimulus has been applied, the process proceeds to step S203.

[0313] In step S203, the voice processing unit 172 modulates the sleep speech in response to external stimuli. For example, if the autonomous mobile body 11 is stroked, shaken, or spoken to, the voice processing unit 172 modulates the sleep speech by changing the voice parameters of the sleep speech in response to these stimuli.

[0314] The process then proceeds to step S204.

[0315] On the other hand, if it is determined in step S202 that no external stimulus is being applied, the process in step S203 is skipped, and the process proceeds to step S204.

[0316] In step S204, the voice output control unit 171 determines whether the sleep-talking output has ended. If it is determined that the sleep-talking output has not ended, the process returns to step S202.

[0317] Subsequently, in step S204, the processes in steps S202 to S204 are repeatedly executed until it is determined that the sleep-talking output has ended. This allows the sleep-talking output to continue, and the sleep-talking to change in response to external stimuli.

[0318] On the other hand, if it is determined in step S204 that the output of sleep-talking has ended, the process proceeds to step S205.

[0319] In step S205, the autonomous mobile unit 11 exits sleep-talking mode. Specifically, the voice output control unit 171 sets the pronunciation mode of the voice processing unit 172 to the unresponsive mode. This causes the pronunciation mode to transition from sleep-talking mode to the unresponsive mode.

[0320] After that, the sleep-talking control process ends.

[0321] As described above, the expressive capabilities of the autonomous mobile unit 11 through sound are improved. For example, the autonomous mobile unit 11 will be able to output more natural and richer stimulus response sounds in response to various external stimuli. In addition, each autonomous mobile unit 11 will exhibit individuality in its response to stimuli. For example, the characteristics and output timing of the stimulus response sound in response to the same stimulus will change for each autonomous mobile unit 11.

[0322] As a result, users will feel a sense of connection to the autonomous mobile device 11 that is closer to that of a real pet or other living creature, and their attachment to and satisfaction with the autonomous mobile device 11 will improve.

[0323] Furthermore, since there is no need to prepare audio data for each stimulus response sound, the production cost and data volume of stimulus response sounds can be reduced.

[0324] <Motion sound control processing> Next, the motion sound control process performed by the autonomous mobile unit 11 will be explained with reference to the flowchart in Figure 31.

[0325] This process is initiated, for example, when the conditions for the autonomous mobile unit 11 to perform a predetermined motion are met.

[0326] In step S301, the autonomous mobile unit 11 starts motion. Specifically, the motion control unit 162 controls the drive unit 104 and the output unit 105 based on the motion data stored in the memory unit 106 to start the execution of motion for which the execution conditions have been met.

[0327] In step S302, the recognition unit 151 recognizes the external state of the autonomous mobile body 11 based on the sensor data supplied from the input unit 101 and the drive data supplied from the drive unit 104. The external state of the autonomous mobile body 11 refers to the state that appears outside the autonomous mobile body 11. For example, the recognition unit 151 detects state variables that represent the external state of each part of the autonomous mobile body 11, such as the position and angle of the head, torso, legs, tail, and mouth. The recognition unit 151 supplies situation data, including the recognition result of the external state of the autonomous mobile body 11 (for example, the detection result of state variables that represent the external state), to the action control unit 153.

[0328] In step S303, the audio output control unit 171 determines whether or not it is necessary to output motion sound based on the recognition result of the external state of the autonomous mobile body 11. If it is determined that it is necessary to output motion sound, the process proceeds to step S304.

[0329] In step S304, the audio processing unit 172 calculates the audio parameters of the motion sound and generates audio data.

[0330] Specifically, the audio output control unit 171 instructs the generation of audio data corresponding to motion sounds that correspond to the external state of the autonomous mobile unit 11.

[0331] The audio processing unit 172 calculates the audio parameters of the motion sound to be output based on the detection results of state variables representing the external state of the autonomous mobile body 11.

[0332] Figure 32 shows the relationship between the mouth opening angle and the audio parameters when the autonomous mobile unit 11 performs a mouth opening and closing motion and outputs a sound as a motion sound in accordance with the mouth movement. Specifically, Figure 32 shows the time series transitions of the mouth opening angle, which is a state variable of the autonomous mobile unit 11, and the volume, low-pass filter application degree, and frequency, which are audio parameters of the sound.

[0333] The audio processing unit 172 sets the volume of the bird's cry, the degree of application of the low-pass filter, and the frequency based on the aperture angle, as shown in the graph in Figure 32.

[0334] In this example, the volume of the cry is proportional to the opening angle until the opening angle reaches a predetermined value, that is, until the mouth of the autonomous mobile body 11 is opened to a certain extent. On the other hand, the degree of application and frequency of the low-pass filter on the cry remain constant until the opening angle reaches a predetermined value.

[0335] Furthermore, when the aperture angle is greater than or equal to a predetermined value, that is, when the mouth of the autonomous mobile body 11 is opened beyond a certain point, the volume of the cry becomes constant. On the other hand, when the aperture angle is greater than or equal to a predetermined value, the degree of application of the low-pass filter is inversely proportional to the aperture angle, and the frequency is proportional to the aperture angle.

[0336] As a result, when the opening angle is below a predetermined value, the sound quality of the cry remains fixed, while the volume changes in accordance with the change in the opening angle.

[0337] On the other hand, when the mouth opening angle exceeds a predetermined value, the volume of the sound remains fixed, while the sound quality changes in accordance with the change in mouth opening angle. For example, as mentioned above, when an animal has its mouth closed, the sound is closer to "u," and when its mouth is open, the sound is closer to "a." This change in vowel is achieved by changing the degree to which the low-pass filter is applied according to the mouth opening angle.

[0338] This allows for realistic vocalizations that correspond to mouth movements. For example, the volume and tone of the vocalizations are adjusted according to how wide the mouth is opened, and the rate at which the vocalizations change varies depending on how quickly the mouth is moved.

[0339] Furthermore, for example, if the autonomous mobile unit 11 keeps its mouth open for an extended period while performing a motion of opening and closing its mouth, the continuous emission of a sound may be perceived as unnatural by the user.

[0340] In contrast, as described above with reference to Figure 23, for example, a pause for breathing may be inserted in between. For example, similar to the case where the autonomous mobile unit 11 is continuously being petted, when the time the mouth is open exceeds the maximum output time, the vocalization stops or is attenuated. Subsequently, if the mouth of the autonomous mobile unit 11 remains open until the minimum output interval has elapsed, a vocalization with a weaker velocity than the initial vocalization is output again.

[0341] The audio processing unit 172 generates audio data corresponding to motion sounds based on the calculated audio parameters.

[0342] In step S305, the autonomous mobile unit 11 outputs motion sound. Specifically, the audio output control unit 171 supplies audio data generated by the audio processing unit 172 to the output unit 105. The output unit 105 outputs motion sound based on the audio data.

[0343] The process then proceeds to step S306.

[0344] On the other hand, if it is determined in step S303 that there is no need to output motion sound, the processing in steps S304 and S305 is skipped, and the process proceeds to step S306.

[0345] In step S306, the motion control unit 162 determines whether the motion has finished. If it is determined that the motion has not finished, the process returns to step S302.

[0346] Subsequently, in step S306, the processes in steps S302 to S306 are repeatedly executed until it is determined that the motion has ended. This ensures that motion sounds are output as needed during the execution of the motion.

[0347] On the other hand, if it is determined in step S306 that the motion has ended, the motion sound control process terminates.

[0348] Furthermore, the autonomous mobile unit 11 can output motion sounds other than cries based on its external state using a similar method.

[0349] For example, during the execution of a motion such as running or walking, the recognition unit 151 detects the vertical position of the center of gravity of the autonomous mobile body 11 and the angles of the joints of each leg as state variables, based on sensor data from the inertial sensor 60 and drive data from the drive unit 104. Based on the detection results of the vertical position of the center of gravity of the autonomous mobile body 11 and the angles of the joints of each leg, the voice processing unit 172 generates voice data corresponding to the sound of a sigh or breathing to be output in accordance with the movement of the autonomous mobile body 11. Based on the voice data generated by the voice processing unit 172, the voice output control unit 171 outputs the sound of a sigh or breathing from the output unit 105 in accordance with the movement of the autonomous mobile body 11.

[0350] This allows the autonomous mobile device 11 to emit breaths and breathing sounds in accordance with its movement, just like a real dog.

[0351] For example, during the execution of a movement motion, the recognition unit 151 detects the presence or absence and magnitude of an impact on the autonomous mobile body 11 as state variables based on sensor data from the inertial sensor 60. Based on the detection results of the presence or absence and magnitude of an impact on the autonomous mobile body 11, the audio processing unit 172 generates audio data corresponding to an impact sound to be output in accordance with the magnitude of the impact. Based on the audio data generated by the audio processing unit 172, the audio output control unit 171 outputs an impact sound from the output unit 105 in accordance with the impact on the autonomous mobile body 11.

[0352] This allows the autonomous mobile unit 11 to output an impact sound corresponding to the magnitude of the impact. Furthermore, if an unexpected impact occurs during motion execution, for example, the autonomous mobile unit 11 can quickly respond and output an impact sound.

[0353] Impacts to the autonomous mobile unit 11 could include, for example, collisions with walls, tripping, and falls. Impact sounds could include, for example, collision sounds, tripping sounds, and falling sounds.

[0354] For example, during the execution of a movement motion, the recognition unit 151 detects whether each leg of the autonomous mobile body 11 is landing and the magnitude of the impact at the time of landing as state variables, based on sensor data from the sole buttons 59 and the inertial sensor 60. Based on the detection results of whether each leg of the autonomous mobile body 11 is landing and the magnitude of the impact at the time of landing, the audio processing unit 172 generates audio data corresponding to footsteps according to the magnitude of the impact at the time of landing. Based on the audio data generated by the audio processing unit 172, the audio output control unit 171 outputs footsteps from the output unit 105 in accordance with the landing of each leg.

[0355] As a result, the autonomous mobile unit 11 can output footstep sounds in accordance with the landing of each leg, and can also output footstep sounds of a magnitude corresponding to the impact of the landing.

[0356] For example, during the execution of a movement motion, the recognition unit 151 detects the swaying of the torso of the autonomous mobile body 11 as a state variable based on sensor data from the inertial sensor 60. If the autonomous mobile body 11 is assumed to be wearing clothes, the audio processing unit 172 generates audio data corresponding to the sound of clothing rustling caused by the swaying of the torso of the autonomous mobile body 11, based on the detection result of the swaying of the torso of the autonomous mobile body 11. Based on the audio data generated by the audio processing unit 172, the audio output control unit 171 outputs the sound of clothing rustling from the output unit 105 in accordance with the swaying of the torso of the autonomous mobile body 11.

[0357] This allows the autonomous mobile unit 11 to output a rustling sound in accordance with the shaking of its body.

[0358] Furthermore, the autonomous mobile unit 11 may, for example, be configured to output the sound of bells along with the sound of clothing rustling, assuming that it is wearing clothing with bells attached during the Christmas season.

[0359] For example, during the execution of a motion to turn around, the recognition unit 151 detects the angle of the neck joint of the autonomous mobile body 11 as a state variable based on drive data from the drive unit 104. Based on the detection result of the angle of the neck joint of the autonomous mobile body 11, the audio processing unit 172 generates audio data corresponding to a turning sound that represents the turning of the autonomous mobile body 11. Based on the audio data generated by the audio processing unit 172, the audio output control unit 171 outputs the turning sound from the output unit 105 in accordance with the movement of the neck (head) of the autonomous mobile body 11.

[0360] This allows the autonomous mobile unit 11 to output a turning sound in conjunction with the autonomous mobile unit 11 turning its head.

[0361] Furthermore, the audio output control unit 171 switches the algorithm and parameters used by the audio processing unit 172 as needed, depending on the type of motion being performed. This makes it possible to change the characteristics and output timing of the motion sound depending on the type of motion being performed, even if the external state of the autonomous mobile body 11 is the same.

[0362] For example, even if the mouth movements of the autonomous mobile device 11 are the same, different motion sounds will be output depending on the motion being performed. Specifically, for example, when the autonomous mobile device 11 is performing a driving motion, a breathy cry will be output in accordance with the mouth movements. On the other hand, for example, when the autonomous mobile device 11 is performing a motion of trying to get the user's attention, a sweet cry will be output in accordance with the mouth movements.

[0363] For example, if contact with an object on the sole of the foot is detected by the foot button 59, the output of motion sounds will be switched depending on the difference in the motion being performed.

[0364] Figure 33 shows the relationship between the detection results of the foot sole button 59 and the volume of the motion sound, which is the sound of footsteps.

[0365] For example, if the autonomous mobile unit 11 is sleeping, that is, if it is performing a sleeping motion, even if the sole button 59 detects contact, no footstep sound will be output.

[0366] On the other hand, when the autonomous mobile unit 11 is standing and walking, that is, when it is performing a walking motion, footstep sounds are output in accordance with the detection of contact with the sole button 59.

[0367] This ensures that appropriate motion sounds are output in accordance with the motion and external conditions of the autonomous mobile unit 11.

[0368] <Specific example of how to calculate audio parameters for motion sounds> Next, with reference to Figures 34 and 35, we will explain a specific example of how to calculate the audio parameters of motion sounds.

[0369] For example, a motion sound designer adjusts the audio parameters while listening to each motion sound actually output from the autonomous mobile unit 11 to design an appropriate motion sound. Alternatively, the designer may use an authoring tool to create algorithms for generating each motion sound and set the audio parameters.

[0370] At this time, the designer sets the audio parameters for each motion sound to cover the movements of the autonomous mobile unit 11, for example, based on the maximum value and maximum range of change of movement of each part of the autonomous mobile unit 11.

[0371] For example, data showing the relationship between state variables and audio parameters in the keyframes of each motion is stored in the audio processing unit 172 of the autonomous mobile unit 11. More specifically, for example, data showing the relationship between state variables and the values ​​and changes in audio parameters in the keyframes of each motion is stored in the audio processing unit 172.

[0372] The audio processing unit 172 calculates the audio parameters of the motion sound in real time based on data showing the relationship between the state variables and the values ​​and changes of the audio parameters in the keyframes. Based on the calculated audio parameters, the audio processing unit 172 generates audio data corresponding to the motion sound.

[0373] Figure 34 shows an example of keyframes. Specifically, Figure 34 is the graph from Figure 32 with auxiliary lines added to represent the time points t1f through t6f corresponding to the keyframes.

[0374] Time t1f is the time when the aperture angle starts to rise (starts rising from 0 degrees). Time t2f is the time when the aperture angle reaches the angle at which the sound volume becomes constant. Time t3f is the time when the aperture angle reaches its maximum value. Time t4f is the time when the aperture angle starts to decrease from its maximum value. Time t5f is the time when the aperture angle reaches the angle at which the sound volume becomes constant. Time t6f is the time when the aperture angle becomes 0 degrees.

[0375] For example, the audio processing unit 172 interpolates the audio parameters at aperture angles other than keyframes based on the values ​​and changes in the aperture angle and audio parameters at each keyframe. This allows the audio parameters at each aperture angle to be calculated in real time.

[0376] For example, Figure 35 is a graph showing the relationship between the aperture angle and sound parameters (e.g., volume). The horizontal axis represents the aperture angle, and the vertical axis represents the sound parameters.

[0377] For example, the values ​​and changes in the audio parameters at keyframes with aperture angles of 0 degrees, 30 degrees, and 60 degrees are stored in the audio processing unit 172. The audio processing unit 172 calculates the audio parameters at other aperture angles in real time by linear interpolation based on the audio parameters at aperture angles of 0 degrees, 30 degrees, and 60 degrees.

[0378] The audio processing unit 172 then generates audio data corresponding to the motion sound in real time based on the calculated audio parameters.

[0379] For example, the designer may create a function that converts state variables representing the external state of the autonomous mobile unit 11 into audio parameters for each motion of the autonomous mobile unit 11. The audio processing unit 172 may then use the created function to calculate the audio parameters for each motion sound in real time based on the state variables of the autonomous mobile unit 11.

[0380] As described above, based on the external state of the autonomous mobile unit 11, audio data corresponding to motion sounds is generated or processed in real time, and motion sounds are output based on the generated or processed audio data.

[0381] This makes it possible to output appropriate motion sounds at the appropriate timing in conjunction with the movements of the autonomous mobile unit 11 while it is performing various motions.

[0382] Furthermore, even if, for example, the autonomous mobile unit 11 performs various motions, and user intervention or disturbances cause the autonomous mobile unit 11 to move in a way that differs from the expected motion (e.g., a failed landing), it will be possible to output the appropriate motion sound at the appropriate time.

[0383] As a result, the expressive power of the autonomous mobile device 11 through voice is improved, leading to increased user satisfaction.

[0384] Furthermore, it becomes possible to output various types of motion sounds without having to prepare all the corresponding audio data in advance. This reduces the production cost and data volume of motion sounds.

[0385] <<2. Variant>> The following describes some modifications of the embodiments of the present technology described above.

[0386] <Variations regarding the types of stimulus-response sounds> For example, it is possible to use stimuli other than the touch stimuli mentioned above as triggers for the stimulus response sound.

[0387] For example, actions such as lifting, shaking, or hugging the autonomous mobile body 11 can be used to trigger the stimulus response sound.

[0388] For example, stimuli given to the autonomous mobile body 11 without physical contact, such as visual or auditory stimuli, can be used to trigger the stimulus response sound.

[0389] Specifically, for example, when the autonomous mobile unit 11 is shown something it likes or dislikes, a stimulus response sound may be output. Examples of things it likes include its favorite animals (including humans), its favorite plants, its companions (for example, other friendly autonomous mobile units), its favorite toys, and its favorite food. Examples of things it dislikes include its disliked animals (including humans), its disliked plants, its enemies (for example, other autonomous mobile units it does not get along with), and food it dislikes.

[0390] For example, when the autonomous mobile unit 11 is exposed to sounds it likes or dislikes, it may be configured to output a stimulus response sound. Examples of sounds it likes include voices from the user or companions, or its favorite music. Examples of sounds it dislikes include reprimands, voices from enemies, or music it dislikes.

[0391] Furthermore, for stimuli other than touch, the output stimulus response sound may be made to change according to the content of the stimulus. In other words, the output timing (e.g., minimum output interval), sound parameters, etc. may be made to change according to the content of various stimuli.

[0392] The content of the stimulus can be described by, for example, the type of stimulus, the method of stimulus, the intensity of the stimulus, the timing of the stimulus, the duration of the stimulus, the location of the stimulus, or a combination of these.

[0393] Furthermore, the stimulus response sound output may change depending on the combination of multiple types of stimuli. For example, the stimulus response sound output may change depending on whether the autonomous mobile body 11 is stroked while being shown something the user likes, or whether the autonomous mobile body 11 is stroked while being played a sound the user likes.

[0394] For example, sounds other than the aforementioned barking and sleep-talking can be used as stimulus response sounds. For instance, barking, growling, and the sound of a hungry stomach can be used as stimulus response sounds.

[0395] Furthermore, while the above explanation showed an example of changing the velocity between the call before and after taking a breath, other vocal parameters besides velocity may also be changed.

[0396] <Variations concerning factors that change stimulus response sounds> For example, the characteristics and timing of the output stimulus response sound may be changed by factors other than the content of the stimulus. Such factors may include, for example, the behavior of the autonomous mobile unit 11, the state of the autonomous mobile unit 11, the recipient of the stimulus, and the surrounding circumstances.

[0397] First, we will explain a specific example of how the stimulus response sound output changes due to the actions of the autonomous mobile unit 11.

[0398] For example, even when the same part of the autonomous mobile device 11 is stroked, it changes the stimulus response sound it outputs depending on its actions at that time. Specifically, for example, if the autonomous mobile device 11 is not doing anything and is stroked, it will emit a sound, but if it is performing some action, it will not emit a sound.

[0399] For example, the autonomous mobile device 11 changes the stimulus response sound it outputs based on its previous action. For instance, when the autonomous mobile device 11 is petted after it has been moving, it emits a more labored breathing sound compared to when it is petted after it has not been moving.

[0400] Next, we will explain a specific example in which the stimulus response sound output changes depending on the state of the autonomous mobile device 11.

[0401] For example, even when the same body part is stroked, the autonomous mobile robot 11 changes the stimulus response sound it outputs depending on the state of that body part. Specifically, for example, if a body part that is not experiencing any activity is stroked, the autonomous mobile robot 11 will emit a sound that suggests it is pleased. On the other hand, if a body part that is set as an aversion area is stroked, the autonomous mobile robot 11 will emit a sound that expresses aversion, or it will perform an action that suggests it is unhappy without emitting a sound. Furthermore, if a body part that is virtually itchy is stroked, the autonomous mobile robot 11 will emit a sound that suggests it is enjoying itself.

[0402] For example, even when the same part of the body is stroked, the autonomous mobile robot 11 changes the stimulus sound it outputs according to its emotions. Specifically, for example, when the autonomous mobile robot 11 is in a good mood and is stroked, it will emit a sound that suggests it is happy. On the other hand, when the autonomous mobile robot 11 is in a bad mood and is stroked, it will emit a sound that suggests it is unhappy.

[0403] For example, when the autonomous mobile robot 11 is shown food, it changes the stimulus response sound it outputs according to its hunger level. For instance, when the autonomous mobile robot 11 is shown food while highly hungry, it may emit a joyful cry or a rumbling stomach sound. On the other hand, when the autonomous mobile robot 11 is shown food while less hungry, it does not react in particular and does not emit any stimulus response sound.

[0404] For example, the autonomous mobile device 11 changes its stimulus response sound based on its age. For instance, as the autonomous mobile device 11 gets older, it lowers the pitch of its calls to make it sound older.

[0405] Next, we will explain a specific example where the output stimulus response sound changes depending on the recipient of the stimulus. Here, the recipient of the stimulus can be other than the user, such as other autonomous mobile devices or animals like pets.

[0406] For example, the autonomous mobile device 11 changes the stimulus response sound it outputs depending on its relationship with the person it stimulates (e.g., intimacy level). The intimacy level with the person it stimulates is set based on, for example, the number of times it has met the person it stimulated in the past and the total time spent together. For example, the more intimate the autonomous mobile device 11 is with the person it touches, the more affectionate the sound it makes (e.g., a sweet, pleading sound). For example, if the autonomous mobile device 11 cannot recognize the person it touches (e.g., if it has not met the person it touched before), it will output a default sound.

[0407] For example, since children generally have less force and touch area, it is expected that the response to a child's touch will be reduced. To address this, for example, the autonomous mobile device 11 may temporarily increase the sensitivity to detect touches if the touch amount remains small for a predetermined period of time or longer.

[0408] Next, we will explain specific examples of how the stimulus response sound produced changes depending on the surrounding environment.

[0409] For example, the autonomous mobile device 11 changes its stimulus response sound in response to ambient noise. For example, the autonomous mobile device 11 changes the volume of its calls in accordance with the level of ambient noise. This ensures that, for example, the calls are clearly audible regardless of the level of ambient noise.

[0410] For example, the autonomous mobile device 11 changes its stimulus response sound depending on the location and time. For instance, the autonomous mobile device 11 may change its vocalizations depending on whether it is indoors or outdoors for a walk, or it may change its vocalizations depending on the time of day.

[0411] For example, when the autonomous mobile device 11 emits a sound, it changes the sound in response to the surrounding reactions. For example, when the autonomous mobile device 11 emits a sound, if the other party (for example, a nearby user or another autonomous mobile device) shows a positive reaction, it changes the sound in response to that reaction. On the other hand, when the autonomous mobile device 11 emits a sound, if the other party is unresponsive or shows a negative reaction, it stops emitting the sound and becomes quiet.

[0412] Positive reactions include, for example, smiling, striking up a conversation, or approaching. Negative reactions include, for example, making a displeased face or running away.

[0413] For example, the autonomous mobile device 11 may predict the owner's return time based on the owner's location information or the learning results of the learning unit 152 regarding the owner's return time, and change its sound when the predicted return time approaches.

[0414] Furthermore, for example, the output stimulus response sound may change depending on two or more combinations of the actions of the autonomous mobile unit 11, the state of the autonomous mobile unit 11, the person who gave the stimulus, the surrounding circumstances, and the content of the stimulus.

[0415] Furthermore, for example, the settings for the audio parameters of the stimulus response sound may differ for each autonomous mobile unit 11. For example, the audio processing unit 172 of each autonomous mobile unit 11 may change the base frequency and timbre of the stimulus response sound based on the individual parameters of each autonomous mobile unit 11.

[0416] <Variations in reaction patterns> The above explanation shows an example of setting the response pattern of the autonomous mobile unit 11 to touch by setting a reaction reference point or a strong reaction location. In contrast, the response pattern of the autonomous mobile unit 11 to touch may also be set by a method other than a reaction reference point and a strong reaction location.

[0417] For example, the learning unit 152 may learn the user's stimulation method and set the response pattern of the autonomous mobile unit 11 based on the results of learning the stimulation method.

[0418] For example, the learning unit 152 performs learning processing based on behavioral history data and the like stored in the memory unit 106 to learn the owner's petting patterns for the autonomous mobile body 11. The petting patterns are represented, for example, by the petting position, intensity, speed, etc. The learning unit 152 then sets the owner's petting patterns and similar patterns as strong-response touch patterns.

[0419] The voice processing unit 172 then changes the sound produced depending on whether the animal is being stroked with a strong-response touch pattern or with any other pattern. For example, when the animal is being stroked with a strong-response touch pattern, the voice processing unit 172 increases the volume of the sound or raises the pitch of the sound to produce a more positive response than when it is being stroked with any other pattern.

[0420] As a result, for example, each time the owner strokes the autonomous mobile device 11, the autonomous mobile device 11 becomes more responsive. Also, for example, when the owner strokes the autonomous mobile device 11, the autonomous mobile device 11 becomes more responsive compared to when another user strokes it. As a result, the owner can experience, for example, the feeling that the autonomous mobile device 11 is becoming attached to them, and their affection for the autonomous mobile device 11 increases.

[0421] Furthermore, for example, the learning unit 152 may set response patterns to stimuli other than touch by learning the content of stimuli other than touch that have been given to the autonomous mobile body 11 in the past. For example, the learning unit 152 may set response patterns to what the user says by learning the content of what the user has said in the past.

[0422] <Regarding the transfer of features of autonomous mobile unit 11> For example, when replacing the autonomous mobile unit 11 due to a malfunction or other reason, data relating to the characteristics of the old autonomous mobile unit 11 may be transferred to the new autonomous mobile unit 11 so that the new autonomous mobile unit 11 can inherit the characteristics of the old autonomous mobile unit 11. Specifically, for example, the new autonomous mobile unit 11 may be able to inherit the reaction patterns from the old autonomous mobile unit 11.

[0423] This can, for example, prevent users from experiencing feelings of loss or eliminate the need for users to train the new autonomous mobile device 11.

[0424] By a similar method, for example, in a system where an autonomous mobile unit 11 produces offspring, it is possible to ensure that the offspring autonomous mobile unit 11 inherits all or part of the characteristics of the parent autonomous mobile unit 11.

[0425] <Variations related to motion sounds> For example, the autonomous mobile unit 11 can change the characteristics and output timing of motion sounds according to the surrounding conditions. For instance, the autonomous mobile unit 11 can change the volume and quality of motion sounds according to the ambient noise around it.

[0426] For example, when the autonomous mobile unit 11 performs a motion of eating food, it may change the sound of the eating motion depending on the type of food being eaten.

[0427] For example, the autonomous mobile unit 11 performs preliminary actions to output motion sounds as needed. For example, when the autonomous mobile unit 11 performs a motion to output a sound, if it is holding a bone in its mouth, it first performs the action of putting the bone down.

[0428] <Other variations> For example, while the information processing terminal 12 is running an application that manages the autonomous mobile unit 11, it may output motion sounds in sync with the motion of the autonomous mobile unit 11 on the screen. In this case, for example, the audio parameters of the motion sounds may be changed from the audio parameters for the autonomous mobile unit 11 depending on the difference in sound systems such as speakers between the information processing terminal 12 and the autonomous mobile unit 11.

[0429] If the information processing terminal 12 cannot output motion sounds, the user may be made aware that motion sounds are being output by, for example, changing the color of the autonomous mobile object 11 on the screen or displaying a waveform representing the motion sound.

[0430] For example, the information processing terminal 12 or information processing server 13 may perform some of the processing of the autonomous mobile unit 11 as described above. For example, the information processing terminal 12 or information processing server 13 may perform all or part of the processing of the information processing unit 103 of the autonomous mobile unit 11 and remotely control the autonomous mobile unit 11. Specifically, for example, the information processing terminal 12 or information processing server 13 may remotely control the output of stimulus response sounds and motion sounds of the autonomous mobile unit 11.

[0431] This technology can be applied not only to the aforementioned dog-type quadruped robot, but also to autonomous mobile devices that emit stimulus response sounds or motion sounds.

[0432] <<3.B>> <Example of computer configuration> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up that software are installed on a computer. Here, a computer includes computers built into dedicated hardware, as well as general-purpose personal computers that can perform various functions by installing various programs.

[0433] Figure 36 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program.

[0434] In computer 1000, the CPU (Central Processing Unit) 1001, ROM (Read Only Memory) 1002, and RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0435] An input / output interface 1005 is further connected to the bus 1004. An input / output interface 1005 is connected to an input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010.

[0436] The input section 1006 consists of input switches, buttons, a microphone, an image sensor, etc. The output section 1007 consists of a display, a speaker, etc. The storage section 1008 consists of a hard disk or non-volatile memory, etc. The communication section 1009 consists of a network interface, etc. The drive 1010 drives removable media 1011 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0437] In the computer 1000 configured as described above, the CPU 1001 loads, for example, a program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes it, thereby performing the series of processes described above.

[0438] The program executed by computer 1000 (CPU 1001) can be provided by recording it on removable media 1011, such as a packaged media. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0439] In computer 1000, programs can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable media 1011 into the drive 1010. Alternatively, programs can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Furthermore, programs can be pre-installed in the ROM 1002 or the storage unit 1008.

[0440] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0441] Furthermore, in this specification, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure or not. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device in which multiple modules are housed in one enclosure, are both considered systems.

[0442] Furthermore, the embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0443] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0444] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0445] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0446] <Examples of configuration combinations> This technology can also be configured as follows:

[0447] (1) In an autonomous mobile vehicle that moves autonomously, The motion control unit controls the operation of the autonomous mobile unit, A recognition unit that recognizes the state of the autonomous mobile unit while it is in operation, Based on the recognition result of the state of the autonomous mobile device, an audio control unit generates or processes audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device, and controls the characteristics and output timing of the audio. An autonomous mobile vehicle equipped with the following features. (2) The recognition unit detects a state variable representing the state of the autonomous mobile body, The audio control unit controls the characteristics and output timing of the audio based on the state variables. The autonomous mobile body described in (1) above. (3) The audio control unit calculates audio parameters used to control the characteristics of the audio based on the state variables, and generates or processes the audio data based on the calculated audio parameters. The autonomous mobile body described in (2) above. (4) The voice control unit calculates the voice parameters based on data showing the relationship between the state variables and the voice parameters. The autonomous mobile body described in (3) above. (5) The voice control unit calculates the voice parameters using a function that converts the state variables into voice parameters. The autonomous mobile body described in (3) above. (6) The aforementioned audio parameters include one or more of the following: frequency, volume, modulation degree, harmonic components, degree of low-pass filter application, and degree of effect application. An autonomous mobile body as described in any of (3) to (5) above. (7) The recognition unit detects the opening angle, which is the angle at which the autonomous mobile body opens its mouth. The sound control unit generates or processes the sound data corresponding to the vocalizations to be output in accordance with the mouth movements of the autonomous mobile body, based on the opening angle, and controls the characteristics and output timing of the vocalizations. An autonomous mobile body as described in any of (2) to (6) above. (8) The sound control unit continues to output the sound until a first time has elapsed while the autonomous mobile body has its mouth open, and after the first time has elapsed, it stops outputting the sound or attenuates the sound. The autonomous mobile body described in (7) above. (9) The sound control unit shall output the sound again if the autonomous mobile body continues to open its mouth after a second period of time has elapsed since the output of the sound was stopped. The autonomous mobile body described in (8) above. (10) The aforementioned sound control unit changes the characteristics of the sound to be output again from the sound output previously. The autonomous mobile body described in (9) above. (11) The recognition unit detects whether the autonomous mobile body has landed and the magnitude of the impact at the time of landing. Based on the detection results of whether the autonomous mobile body has landed and the magnitude of the impact at the time of landing, the sound control unit generates or processes the sound data corresponding to the footsteps to be output in accordance with the movement of the autonomous mobile body's legs, and controls the characteristics and output timing of the footsteps. An autonomous mobile body as described in any of (2) to (10) above. (12) The recognition unit detects the center of gravity position and the angle of the leg joints of the autonomous mobile body. The voice control unit generates or processes voice data corresponding to the breath or breathing sounds that are output in accordance with the movement of the autonomous mobile body, based on the detection results of the center of gravity position and the angles of the leg joints of the autonomous mobile body, and controls the characteristics and output timing of the voice. An autonomous mobile body as described in any of (2) to (11) above. (13) The recognition unit detects the angle of the neck joint of the autonomous mobile body, Based on the detection result of the angle of the neck joint of the autonomous mobile body, the voice control unit generates or processes the voice data corresponding to the voice output in accordance with the movement of the autonomous mobile body's neck, and controls the characteristics and output timing of the voice. An autonomous mobile body as described in any of (2) to (12) above. (14) The recognition unit detects whether or not an impact occurred to the autonomous mobile body and the magnitude of the impact. The audio control unit generates or processes audio data corresponding to the audio to be output in accordance with the impact, based on the detection results of whether or not an impact occurred on the autonomous mobile body and the magnitude of the impact, and controls the characteristics and output timing of the audio. An autonomous mobile body as described in any of (2) to (13) above. (15) The audio control unit further controls the characteristics and output timing of the audio based on the type of operation. An autonomous mobile body as described in any of (1) to (14) above. (16) The audio control unit controls whether or not to output the audio for the same state of the autonomous mobile unit, based on the type of operation. The autonomous mobile body described in (15) above. (17) The recognition unit recognizes the external state that appears outside the autonomous mobile body, The audio control unit generates or processes the audio data and controls the characteristics and output timing of the audio based on the external state of the autonomous mobile unit. An autonomous mobile body as described in any of (1) to (16) above. (18) The actions of the autonomous mobile body are actions or performances that express the will or emotions of the autonomous mobile body. The autonomous mobile body described in (1) above. (19) Controlling the movement of autonomous mobile devices, The state of the autonomous mobile unit while it is in operation is recognized, Based on the recognition result of the autonomous mobile device's state, audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device is generated or processed, and the characteristics and output timing of the audio are controlled. Information processing methods. (20) Controlling the movement of autonomous mobile devices, The state of the autonomous mobile unit while it is in operation is recognized, Based on the recognition result of the autonomous mobile device's state, audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device is generated or processed, and the characteristics and output timing of the audio are controlled. A program that causes a computer to perform a process.

[0448] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur. [Explanation of symbols]

[0449] 1 Information processing system, 11-1 to 11-n Autonomous mobile unit, 12-1 to 12-n Information processing terminal, 13 Information processing server, 101 Input unit, 103 Information processing unit, 104 Drive unit, 105 Output unit, 151 Recognition unit, 152 Learning unit, 153 Action control unit, 161 Internal state control unit, 162 Motion control unit, 163 Voice control unit, 171 Voice processing unit, 172 Voice output control unit

Claims

1. In an autonomous mobile vehicle that moves autonomously, The motion control unit controls the operation of the autonomous mobile unit, A recognition unit that recognizes the state of the autonomous mobile unit while it is in operation, Based on the recognition result of the state of the autonomous mobile device, an audio control unit generates or processes audio data corresponding to the audio to be output in accordance with the operation of the autonomous mobile device, and controls the characteristics and output timing of the audio. Equipped with, The recognition unit detects the center of gravity position and the angle of the leg joints of the autonomous mobile body. The voice control unit generates or processes voice data corresponding to the breath or breathing sounds that are output in accordance with the movement of the autonomous mobile body, based on the detection results of the center of gravity position and the angles of the leg joints of the autonomous mobile body, and controls the characteristics and output timing of the voice. Autonomous mobile device.

2. The recognition unit detects a state variable representing the state of the autonomous mobile body, The audio control unit controls the characteristics and output timing of the audio based on the state variables. The autonomous mobile body according to claim 1.

3. The audio control unit calculates audio parameters used to control the characteristics of the audio based on the state variables, and generates or processes the audio data based on the calculated audio parameters. The autonomous mobile body according to claim 2.

4. The voice control unit calculates the voice parameters based on data showing the relationship between the state variables and the voice parameters. The autonomous mobile body according to claim 3.

5. The voice control unit calculates the voice parameters using a function that converts the state variables into voice parameters. The autonomous mobile body according to claim 3.

6. The aforementioned audio parameters include one or more of the following: frequency, volume, modulation degree, harmonic components, degree of low-pass filter application, and degree of effector application. The autonomous mobile body according to claim 3.

7. The recognition unit detects the opening angle, which is the angle at which the mouth of the autonomous mobile body opens, The sound control unit generates or processes the sound data corresponding to the vocalizations to be output in accordance with the mouth movements of the autonomous mobile body, based on the opening angle, and controls the characteristics and output timing of the vocalizations. The autonomous mobile body according to claim 2.

8. The sound control unit continues to output the sound until a first time has elapsed while the autonomous mobile body has its mouth open, and after the first time has elapsed, stops outputting the sound or attenuates the sound. The autonomous mobile body according to claim 7.

9. The sound control unit shall output the sound again if the autonomous mobile body continues to open its mouth after a second time has elapsed since the output of the sound was stopped. The autonomous mobile body according to claim 8.

10. The aforementioned sound control unit changes the characteristics of the sound to be output again from the sound output previously. The autonomous mobile body according to claim 9.

11. The recognition unit detects whether the autonomous mobile body has landed and the magnitude of the impact at the time of landing. Based on the detection results of whether the autonomous mobile body has landed and the magnitude of the impact at the time of landing, the sound control unit generates or processes the sound data corresponding to the footsteps to be output in accordance with the movement of the autonomous mobile body's legs, and controls the characteristics and output timing of the footsteps. The autonomous mobile body according to claim 2.

12. The recognition unit detects the angle of the neck joint of the autonomous mobile body, Based on the detection result of the angle of the neck joint of the autonomous mobile body, the voice control unit generates or processes the voice data corresponding to the voice output in accordance with the movement of the autonomous mobile body's neck, and controls the characteristics and output timing of the voice. The autonomous mobile body according to claim 2.

13. The recognition unit detects whether or not an impact occurred to the autonomous mobile body and the magnitude of the impact. The audio control unit generates or processes audio data corresponding to the audio to be output in accordance with the impact, based on the detection results of whether or not an impact occurred on the autonomous mobile body and the magnitude of the impact, and controls the characteristics and output timing of the audio. The autonomous mobile body according to claim 2.

14. The audio control unit further controls the characteristics and output timing of the audio based on the type of operation. The autonomous mobile body according to claim 1.

15. The audio control unit controls whether or not to output the audio for the same state of the autonomous mobile unit, based on the type of operation. The autonomous mobile body according to claim 14.

16. The recognition unit recognizes the external state that appears outside the autonomous mobile body, The audio control unit generates or processes the audio data and controls the characteristics and output timing of the audio based on the external state of the autonomous mobile unit. The autonomous mobile body according to claim 1.

17. The actions of the autonomous mobile body are actions or performances that express the will or emotions of the autonomous mobile body. The autonomous mobile body according to claim 1.

18. Information processing device, Controlling the operation of autonomous mobile devices, The center of gravity position and the angle of the leg joints of the autonomous mobile body are detected, Based on the detection results of the center of gravity position and the angle of the leg joints of the autonomous mobile body, audio data corresponding to the sound representing a breath or breathing that is output in accordance with the movement of the autonomous mobile body is generated or processed, and the characteristics and output timing of the sound are controlled. Information processing methods including

19. Controlling the operation of autonomous mobile devices, The center of gravity position and the angle of the leg joints of the autonomous mobile body are detected, Based on the detection results of the center of gravity position and the angle of the leg joints of the autonomous mobile body, audio data corresponding to the sound representing a breath or breathing that is output in accordance with the movement of the autonomous mobile body is generated or processed, and the characteristics and output timing of the sound are controlled. A program that causes a computer to perform a process that includes [a specific action].

Images