Mobile device, method of controlling audio output of mobile device, and program

Abstract

To carry out audio output in response to various situations without damaging safety or convenience of a mobile device.SOLUTION: The present invention is directed to a mobile device which can carry out autonomous travel. The mobile device has: a first audio data group storing unit and a second audio data group storing unit which respectively store first audio data group and second audio data group which are different from each other in kind; and an audio output control unit for, in response to a situation of the mobile device, outputting sound using audio data extracted from either the first audio data group storing unit or the second audio data group storing unit. The first audio data group can be set by a user with an external input during operation of the mobile device, and the second audio data group is previously set before operation of the mobile device and cannot be changed in settings by the user.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a mobile device, and a method and program for controlling audio output of a mobile device. [Background technology]

[0002] Autonomous robots as mobile devices are being used for various purposes, such as patrolling and monitoring buildings and cleaning. Autonomous robots perform various tasks while coexisting with people in areas of human activity, such as office buildings, commercial facilities, government offices, hospitals, and other public facilities. For this reason, communication by voice output is used to ensure human safety and convenience.

[0003] In this regard, for example, Patent Document 1 describes a configuration in which a voice generating device 20 is provided on the head 6 of a moving object 1, and a manager or operator of the moving object 1 uses an environmental information setting device 80 to set various settings such as whether the voice generating device 20 can speak at a predetermined position where the moving object 1 is located and to set the allowable volume (see paragraphs 0012, 0015, etc.). Patent Document 2 also describes a configuration in which a communication robot that executes communication behavior with a human using at least voice is provided with an experience detection means that detects whether the human has heard the voice before, and a speed determination means that determines the speaking speed of the voice based on at least the result of the determination by the movement determination means (claim 1, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7081926 [Patent Document 2] Patent No. 5842245 Summary of the Invention [Problem to be solved by the invention]

[0005] An autonomous robot acts while outputting sounds required in various situations. For example, when patrolling the floors of a building, the autonomous robot is configured to continuously output a specific sound in order to alert people in the vicinity. In addition, depending on the facility in which the autonomous robot is installed, customized sound output is performed depending on the location, time, etc. in the facility. Meanwhile, the autonomous robot may be given a function as a communication terminal when people in the vicinity wish to make some kind of contact with the facility manager. Alternatively, the autonomous robot may be configured to output a warning sound when a person comes into contact with a specific part of the autonomous robot.

[0006] In an autonomous robot that outputs voice in various situations as described above, when conditions corresponding to multiple situations are met at the same time, how to prioritize the voice output has an important meaning that affects the safety and convenience of operating the autonomous robot. For example, if a warning sound is issued and an attention sound that notifies the presence of the autonomous robot is output at the same time, the warning sound may not be sufficiently transmitted to the surroundings, which is a safety problem. Alternatively, if a guidance voice output based on environmental information such as the robot's position and time overlaps with the attention sound, the guidance voice may be difficult to hear and convenience may be impaired. In this way, in an autonomous robot that outputs voice according to the situation, it is necessary to manage and control various voice outputs by prioritizing them from the standpoint of safety and convenience. Although the technologies described in Patent Documents 1 and 2 above relate to robots that output voice, there is no disclosure of such priority control of voice output.

[0007] One object of the present invention is to provide a mobile device, and a method and program for controlling audio output of a mobile device, that can execute audio output according to various situations without compromising the safety and convenience of the mobile device. [Means for solving the problem]

[0008] One aspect of the present invention is a mobile device capable of autonomous movement, comprising a first voice data group storage unit and a second voice data group storage unit in which a first voice data group and a second voice data group of different types are stored, and an audio output control unit that outputs audio using voice data extracted from either the first voice data group storage unit or the second voice data group storage unit depending on the situation in which the mobile device is placed, wherein the first voice data group can be set by a user via external input while the mobile device is in operation, and the second voice data group is set in advance before operation of the mobile device begins, and the user is not allowed to change the settings. Effect of the Invention

[0009] According to the present invention, it is possible to execute audio output according to various situations without compromising the safety and convenience of the mobile device. [Brief description of the drawings]

[0010] [Figure 1] 1 is a system configuration diagram illustrating a configuration of a robot system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating a configuration of hardware and functional blocks of a robot according to an embodiment of the present invention. [Diagram 3] 2 is a block diagram illustrating a configuration of hardware and functional blocks of a management device according to an embodiment of the present invention; FIG. [Figure 4] 2 is a block diagram illustrating an example of a configuration of main functional blocks of a facility manager terminal device according to an embodiment of the present invention; FIG. [Diagram 5] 3 is a block diagram showing an example of the configuration of audio data according to the embodiment of the present invention; FIG. [Figure 6] FIG. 4 is an explanatory diagram of types of sounds output by a robot in one embodiment of the present invention. [Figure 7] 11 is a flowchart illustrating the flow of a voice output control process executed in a robot in one embodiment of the present invention. [Figure 8] 1 is a diagram illustrating state transition control of voice output control executed in a robot in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the present invention will be described based on an embodiment with reference to the accompanying drawings. [System Configuration] 1 shows a system configuration diagram illustrating a system configuration of a robot system 1, which is a mobile device system according to an embodiment of the present invention. As shown in FIG. 1, the robot system 1 includes a management cloud 10, a robot 20, and a facility manager terminal device 30.

[0012] The management cloud 10 and the robot 20 are connected to each other so that they can communicate with each other via a network 40. The network 40 is configured with wireless communication lines such as a mobile communication network, a local area network (LAN), a wide area network (WAN), etc. The robot 20 receives and stores power from a charging device (not shown) and can operate autonomously.

[0013] The management cloud 10 is a system for managing the operating status of the robot system 1, and has functions such as collecting information on the operating status from the robot 20, managing the collected information, and outputting it to the outside as necessary. The management cloud 10 is typically realized by a cloud system in which electronic devices such as server devices and personal computers are connected to each other via a network so that they can communicate with each other. The management cloud 10 can also be a management device configured by a single computer.

[0014] The robot 20 is a mobile device capable of autonomous movement using a motor driven by a battery as a power source, and is preferably used for security patrols in facilities such as office buildings, but is not limited thereto and can be widely applied to cleaning, guidance, equipment inspection, and other tasks. The robot 20 has a function of moving while measuring its surrounding environment in real time and setting a route by a precision distance measurement system using laser light, generally called LiDAR (Light Detection And Ranging). When the battery needs to be charged, the robot 20 approaches a pre-specified charging device and is mechanically and electrically connected to the charging device to charge the battery. The robot system 1 is provided with a required number of charging devices according to the number of robots 20 to be placed, the size of their operating area, and the like.

[0015] The facility equipment 50 indicates facilities and equipment that have a function of operating in cooperation with the autonomously moving robot 20 in a facility where the robot system 1 is introduced and operated. An example of the facility equipment 50 is an elevator. When the robot 20 patrolling the facility moves between different floors, the robot 20 communicates with and calls an API (Application Programming Interface) of an elevator management device, and gives instructions such as specifying the destination floor. As will be described later, at this time, the elevator is operated exclusively for the robot 20 for safety reasons, and in order to inform people around the elevator of this, the robot 20 will board the elevator and an audio notification is issued that the elevator is now exclusively for the robot. The facility equipment 50 also includes a charging device for charging the robot 20.

[0016] [Robot 20 Configuration] Next, the configuration of the robot 20 will be described. FIG. 2 is a block diagram illustrating the configuration of hardware and functional blocks of the robot 20 in one embodiment of the present invention. As shown in FIG. 2, the robot 20 is an information processing device having a function of setting a route based on surrounding environment recognition by a sensor, moving autonomously, and a communication function. Specifically, the robot 20 includes a processor 21, a memory 22, an auxiliary storage unit 23, a network communication unit 24, a data IO unit 25, an input / output unit 26, a drive unit 27, a LiDAR device 28, and a battery 29. Note that the robot 20 does not necessarily need to be configured from the functional blocks illustrated in FIG. 2, and may be realized by another configuration that provides equivalent functions.

[0017] The processor 21 is configured using computing devices such as a CPU and a GPU, and is a computing device that reads various programs and data from the memory 22 and auxiliary storage unit 23 described below, and executes data processing to realize the functions of the robot 20.

[0018] The memory 22 is a storage area for various programs and various data for causing the hardware group to function as the robot 20, and can be configured with storage devices such as ROM, RAM, and flash memory. In this embodiment, the memory 22 stores programs for a management control unit 221, a drive control unit 222, a communication control unit 223, and a sound output control unit 224. The functions of each program will be described later. The memory 22 also stores voice data 225 for generating voice to be output by the robot 20.

[0019] The auxiliary storage unit 23 is a storage device that provides a storage area for storing data and the like used by each program stored in the memory 22, and is configured, for example, by a solid-state drive (SSD), a hard disk drive (HDD), etc. In this embodiment, the auxiliary storage unit 23 stores charging device information 231, robot information 232, map information 233, and audio output control information 234.

[0020] The network communication unit 24 provides a two-way communication function with the management cloud 10 through the network 40, and is configured with hardware such as a mobile communication module and a network interface card (NIC). The network communication unit 24 is provided with a communication management unit 241 for managing the communication function of the communication hardware. The communication management unit 241 includes driver software for operating the communication hardware.

[0021] The data IO unit 25 provides a data input / output function between the processor 21, the network communication unit 24, and the input / output unit 26, and includes various interface circuits.

[0022] The input / output unit 26 is composed of various input devices such as a keyboard, a touch panel, a microphone, etc. that enable data input from the outside to the robot 20, and output devices such as a monitor display and a speaker for outputting output display data, output audio data, etc. generated by the processor 21. The input / output unit 26 includes a touch switch for making a voice call through the facility manager or security guard via the facility manager terminal device 30 to the robot 20. The input / output unit 26 also includes an emergency stop switch for making an emergency stop of the robot 20 for some reason.

[0023] The driving unit 27 includes a power source such as a motor for enabling the robot 20 to move, and hardware such as a reduction / transmission mechanism.

[0024] The LiDAR device 28 is a distance measuring device that uses laser light and has a function of measuring the distance to objects around the robot 20. The obtained measurement data is passed to a program executed by the processor 21 and is used to create a map of the surroundings of the robot 20, set a route for the robot 20 based on the map, and control its movement. Note that other distance measuring devices such as millimeter wave radar can also be used for the distance measuring function. The created map is also used for the voice output control process of the robot 20, which will be described later.

[0025] The battery 29 has the function of supplying power to the drive unit 27 including a motor and the like, and the control system of the robot 20 including the processor 21, and is composed of a rechargeable secondary battery of an appropriate type.

[0026] The management control unit 221, the drive control unit 222, the communication control unit 223, and the audio output control unit 224, which are programs stored in the memory 22, will be described.

[0027] The management control unit 221 has a function of controlling the overall operation of the robot 20 based on information on the operating state of the robot 20. The functions of the management control unit 221 include map creation based on distance measurement data from the LiDAR device 28, collection of information on the surrounding environment by various sensors and path control of the robot 20 based on the collected data, and control of a series of operations for charging the robot 20 based on information on the charging state of the battery 29. The management control unit 221 can be configured as a combination of program modules that execute these individual controls.

[0028] The drive control unit 222 performs drive control of the motors constituting the drive unit 27 and the like in order to control the movement of the robot 20 under the control of the management control unit 221 .

[0029] The communication control unit 223 has a function of controlling data communication between the processor 21 and the network communication unit 24 and the like.

[0030] The audio output control unit 224 controls the audio output by the robot 20 via an audio output means such as a speaker. Specifically, the audio output control unit 224 has a function of outputting an appropriate type of audio at a predetermined volume using the audio data 225 stored in the memory 22 according to conditions such as the position of the robot 20, the current time, the presence or absence of an operation input to the robot 20 from the outside, its operating state, and the presence or absence of a voice call between the robot 20 and the facility manager terminal device 30. When multiple types of audio are output simultaneously, priority control is performed to determine which audio type is to be preferentially output according to the combination of audio types. The control contents by the audio output control unit 224 will be described later based on a specific example. An example of the configuration of the audio data 225 is shown in FIG. 5. There are five types of audio in the audio data 225, which are system sounds SS1, ..., system setting sounds SAS1, ... as the first audio data group, user setting sounds UAS1, ... as the second audio data group, safety measures sounds SFTS1, ..., and call sounds CS1, ... in the example of FIG. 5. The voice data 225 is digital data for generating voice to be output from the robot 20, and is, for example, a data file in MP3 format or WAV format. A unique ID is assigned to each voice data 225, such as SS1, SS2, etc. for system sounds. The voice types will be described later in relation to the voice output control policy.

[0031] The program stored in the memory 22 executes data processing to realize the assigned function using charging device information 231, robot information 232, map information 233, and audio output control information 234 stored in the auxiliary storage unit 23. Here, the charging device information 231 is information that is collected by the charging devices and indicates the operating state of the charging devices, and is sent to the robot 20 by one-way communication broadcast from each charging device. The charging device information 231 of each charging device is sent from each robot 20 to the management cloud 10 and managed by the management cloud 10.

[0032] The robot information 232 is information related to the operating state of the robot 20 collected by the management control unit 221 in each robot 20. The robot information 232 collected in each robot 20 is transmitted to the management cloud 10 and used for management of the robot 20 in the management cloud 10. The robot information 232 in this embodiment includes each of the following items: a robot ID, which is identification information unique to each robot 20; location information, which is information indicating the location of each robot 20; speed information, which is information indicating the moving speed of each robot 20; a battery voltage, which is information indicating the terminal voltage of the battery 29 mounted in each robot 20; and error information, which is information regarding an error occurring in each robot 20. Note that the items included in the robot information 232 are not limited to this example, and may be changed as appropriate.

[0033] The map information 233 quantifies the environment of the operating area of ​​the robot 20 to enable the robot 20 to move autonomously, and is expressed as a set of coordinates of feature points that indicate the route that the robot 20 can move along, and the X and Y coordinates of the feature points are recorded in combination with information about the feature points. The feature points also include the coordinates of the installation positions of the charging devices, and the charging device IDs of the corresponding charging devices are recorded. When charging is required, each robot 20 can determine the nearest charging device based on its own position and the positions of the charging devices recorded in the map information 233, and control the robot 20 to move to that charging device. Note that the items included in the map information 233 are not limited to this example, and may be changed as appropriate.

[0034] The audio output control information 234 is control data used by the audio output control unit 224 to control the audio output of the robot 20. The audio output control information 234 specifies the type and content of audio data to be output according to the position of the robot 20 on the map, the type and content of audio data to be output when a specific operation (touching a specific location, operating an emergency stop switch, etc.) is performed on the robot 20, and a priority output relationship in a situation where multiple types of audio data are output. Specifically, the audio output control information 234 stores a combination of the ID of each data file of the audio data 225 (e.g., system sound SS1, etc.) and the condition for outputting the audio of the data file specified by the ID (e.g., the position of the robot 20, time, specific operation on the robot 20, etc.). Note that the items included in the audio output control information 234 are not limited to this example and may be changed as appropriate.

[0035] [Management Cloud 10 Configuration] Next, the configuration of the management cloud 10 will be described. FIG. 3 is a block diagram illustrating the configuration of hardware and functional blocks of the management cloud 10 in one embodiment of the present invention. As shown in FIG. 3, the management cloud 10 is a cloud system having a function of collecting and recording information on the operating state of each robot 20 transmitted from each robot 20, and managing the robot system 1 based on the information. In the management cloud 10, a plurality of computer hardware resources distributed on a network are managed by management software so as to cooperate with each other, but here, for convenience of explanation, it is shown as a set of computers including a processor 11, a memory 12, an auxiliary storage unit 13, a communication unit 14, a data IO unit 15, and an input / output unit 16. As described above, the management cloud 10 may be a management device configured by a single computer.

[0036] The processor 11 is configured using a calculation device such as a CPU, and is a calculation device that reads various programs and data from the memory 12 and auxiliary storage unit 13 described below, and executes data processing to realize the functions of the management cloud 10.

[0037] The memory 12 is a storage area for various programs and various data for making the hardware group function as the management cloud 10, and can be configured with storage devices such as ROM, RAM, and flash memory. In this embodiment, the memory 12 stores programs for a management control unit 121 for controlling the operation as the management cloud 10, a communication control unit 122 for controlling communication with each robot 20, and a voice output control unit 123 for instructing the robot 20 to output voice from the management cloud 10 side. The memory 12 also stores voice data 124 used for voice output at the robot 20 from the management cloud 10 side. This voice data 124 can be input and stored by the operator of the robot system 1 through various input devices such as a keyboard, a touch panel, and a microphone that constitute the input / output unit 16 described later. The voice data 124 includes unique voice output contents that are defined according to the facility and situation in which the robot 20 is operated. The voice data 124 can be generated by text input or voice input using voice data generation software (not shown) in the management cloud 10. At this time, each generated voice data file is linked to the ID of the corresponding voice data file, and conditions for outputting the voice data are set and stored as voice output control information 134. After the created voice data file and the corresponding voice output control information 134 are generated in the management cloud 10, they may be transferred to the robot 20 and stored therein.

[0038] The audio output control unit 123 has a function of controlling the audio output by the robot 20 through an audio output means such as a speaker from the management cloud 10 side. Specifically, the audio output control unit 123 has a function of instructing the audio output control unit 224 of the robot 20 to output an appropriate type of audio at a predetermined volume according to conditions such as the position of the robot 20, the current time, the presence or absence of an operation input to the robot 20 from the outside, its operating state, and the presence or absence of a voice call between the robot 20 and the facility manager terminal device 30, using the audio data 124 stored in the memory 12. Even when audio output is performed by an instruction from the management cloud 10, if multiple types of audio are output simultaneously, priority control is performed to determine which audio type is to be preferentially output according to the combination of audio types. The control contents by the audio output control unit 123 will be described later based on a specific example. The function of the audio output control unit 123 of the management cloud 10 may be assigned to the audio output control unit 224 of the robot 20, and the audio output control unit 123 may not be provided on the management cloud 10 side.

[0039] The auxiliary storage unit 13 is a storage device that provides a storage area for storing data and the like used by each program stored in the memory 12, and is configured by, for example, a solid-state drive (SSD), a hard disk drive (HDD), and the like. In this embodiment, the auxiliary storage unit 13 stores charging device information 131, robot information 132, map information 133, and audio output control information 134. The charging device information 131, the robot information 132, and the map information 133 are similar to the charging device information 231, the robot information 232, and the map information 233 stored in the auxiliary storage unit 23 of the robot 20. The audio output control information 134 is control data used by the audio output control unit 123 to control the audio output of the robot 20. The audio output control information 134 specifies the type and content of audio data to be output according to the position of the robot 20 on the map, the type and content of audio data to be output when a predetermined operation (contact with a specific location, operation of an emergency stop switch, etc.) is performed on the robot 20, and a priority output relationship in a situation in which multiple types of audio data are output. As described above, the audio output control information 134 on the management cloud 10 side may be transferred to and integrated with the audio output control information 234 of the robot 20 .

[0040] The communication unit 14 provides a communication function with each robot 20 through the network 40, and is configured with hardware such as a mobile communication module, a network interface card (NIC), and the like.

[0041] The data IO unit 15 provides a data input / output function between the processor 11, the communication unit 14, and the input / output unit 16, and includes various interface circuits.

[0042] The input / output unit 16 is composed of various input devices such as a keyboard, touch panel, microphone, etc. that enable data input to the management cloud 10 from outside, and output devices such as a monitor display and speaker for outputting output display data, output audio data, etc. generated by the processor 11.

[0043] [Facility manager terminal device 30] FIG. 4 shows a block diagram illustrating the main configuration of the facility manager terminal device 30 in this embodiment. The facility manager terminal device 30 has a function that enables a security guard or the like, as a facility manager of the facility where the robot 20 is operated, to make a voice call with a facility user via the robot 20. Specifically, the facility manager terminal device 30 provides a voice call function with a user using the input / output unit 26 of the robot 20 via a network and the management cloud 10. The facility manager terminal device 30 illustrated in FIG. 4 includes a communication unit 31 and an input / output unit 32. The communication unit 31 is a communication hardware such as a communication module for mobile communication and a NIC, and enables communication with the management cloud 10 via a network. The input / output unit 32 includes a microphone for voice input and a speaker for voice output. The microphone is provided with a mute switch for suppressing voice input. The on / off of this mute switch is determined by the voice output control unit 224 of the robot 20 monitoring the voice level of the call between the robot 20 and the facility manager terminal device 30. The input / output unit 32 may be a headset that integrates a microphone and earphones. Also, the input / output unit 32 may be provided with a camera and a display so as to enable image communication as well as voice communication. Note that, in this embodiment, the robot 20 and the facility manager terminal device 30 located remotely from the robot 20 are configured to communicate with each other via the management cloud 10, but the robot 20 may be configured to directly communicate with the robot 20 via a network.

[0044] [Robot 20's voice output control policy] Next, regarding the voice output control from the robot 20 executed in this embodiment, the types of voice data implemented in the robot system 1, the conditions for enabling voice output for each type, and the priority control relationship between multiple voice types will be described. Figure 6 shows an example of the voice types to be output from the robot 20 in the robot system 1 of this embodiment.

[0045] In this embodiment, as illustrated in FIG. 6, six types of voices can be output: system sound, system setting sound, user setting sound, safety measure sound, call sound, and conversational voice. To explain in order, the system sound is a sound output as a warning sound such as a buzzer sound that does not use natural language when some action is taken from the outside of the robot 20, for example, when the head of the robot 20, which is rotating during operation to collect environmental information, is touched and the rotation is stopped. Next, the system setting sound is a type of voice that is stored in the system in advance based on the position, current time, and operating state of the robot 20 when the robot 20 acts for patrol, security, etc. in a facility where the robot system 1 is introduced and operated. As an example, when the robot 20 communicates with a management device of an elevator, which is one of the facility equipment, an announcement that "This elevator is only for robots" is output, and when the robot 20 is approaching a charging device installed in the facility for docking, an announcement that "The robot will dock. Please be careful" is output.

[0046] Next, the user-defined sound is a type of sound arbitrarily registered by the user of the robot system 1 via the input / output unit 16 of the management cloud 10. Examples of user-defined sounds include "patrol security in progress", "If you do not have an IC card, please come to the reception desk", and "If you are visiting the robot exhibition, please come to Hall A". The output conditions of the user-defined sound, such as the position of the robot 20 and the current time, can be input and set when the user-defined sound is input. Next, the safety sound is a type of sound for alerting the surroundings of the presence of the robot 20, and is stored in advance as voice data of the robot 20 regardless of the facility where the robot 20 is introduced, and is continuously output for safety reasons, except when the facility manager is making a voice call via the robot 20 or when the voice output is muted under a predetermined condition, as described later.

[0047] Next, the call sound is a type of voice output when a predetermined operation is performed from the robot 20 to call the facility manager terminal device 30, such as touching a specific part of the robot. As an example, the call sound includes an artificial ringback tone and a natural language announcement such as "A security guard is being called." The call sound is stored in advance as voice data of the robot 20. The conversation voice is a type of voice output from the robot 20 regarding the contents of a voice call from the facility manager terminal device 30, and includes a natural language voice such as "What's the matter?"

[0048] [Audio output mode] Next, also with reference to Fig. 6, a description will be given of the voice output mode, which is a policy that defines the priority relationship of the types of voices output from the robot 20. As exemplified in Fig. 6, six types of voice output modes are defined in this embodiment: "Normal", "User-set sound output", "System-set sound output", "Conversation priority", "Conversation", and "Mute".

[0049] The normal mode is a state in which the robot 20 is activated and begins to act autonomously, and the application conditions for other sound output modes are not met. In the normal mode, the robot 20 continuously outputs a safety measure sound at a predetermined volume.

[0050] The user-defined sound output mode is a mode that is applied when a command to output an announcement sound defined by a user is received from the management cloud 10, and is a state in which no other sound output mode is applied. As an example, the user-defined sound output mode corresponds to a case in which a remote operator of the facility manager terminal device 30 issues a command manually, the robot 20 is at a specific position on a map, the current time of the robot 20 is within a predetermined range, the robot 20 receives information such as weather and news through a network, etc. The user-defined sound output mode may be configured to output a plurality of registered sounds in a predetermined order or randomly.

[0051] The system setting sound output mode is a mode that is applied when the robot 20 is placed in a specific situation. Specifically, the system setting sound output mode is a mode that is applied when the robot 20 is acting autonomously with a certain purpose, that is, when the robot 20 is in a situation where its own autonomous behavior takes priority over the presence of people around it. As an example, the system setting sound output mode is applied when the robot 20 gets on and off an elevator, when the robot 20 docks with a charging device for charging, when the robot 20 finishes charging and leaves the charging device, and the like. The system setting sound output mode may be configured to output a predetermined announcement in a situation that is expected in advance, for example, when the robot 20 is surrounded by people and cannot move.

[0052] The conversation priority mode is a mode that is applied when it is determined that the microphone provided in the input / output unit 32 of the facility manager terminal device 30 is not in a muted state. Specifically, the detection of the muted state is when it is determined that there is an audio input signal from the microphone, but the signal level is equal to or lower than a predetermined threshold. In other words, the microphone of the facility manager terminal device 30 is on, but no speech is being made. In this case, since a conversation is about to be started from the facility manager terminal device 30 at a remote location via the management cloud 10, the safety measure sound is suppressed to a predetermined minimum volume so as not to interfere with the conversation. In addition, when it is determined that an action such as touching a specific part has been performed on the robot 20, the conversation priority mode is also applied when a state in which it is recognized that an object of a certain size exists within a specific area in front of the robot 20 (for example, a state in which it is estimated that a person is standing in front of the robot 20) continues. In either case, it is considered that this indicates a state in which a voice conversation is about to be carried out via the robot 20, so the safety measure sound is suppressed to a predetermined minimum volume as described above. Although different from the voice output control, when it is determined that an action such as touching a specific part of the robot 20 has occurred, the motion control of the robot 20 can be either to stop the movement of the robot 20 for a certain period of time or not to restrict the movement of the robot 20. This takes into consideration the case where it is not acceptable for the robot 20 to stop every time it is called.

[0053] The conversation mode is applied when it is determined that the microphone of the facility manager terminal device 30 is on and that a volume level based on speech is detected. The only condition for application of the conversation mode and the conversation priority mode is the difference in the determination threshold of the volume level. The mute mode is applied when the emergency stop switch provided on the robot 20 is operated. In this embodiment, since an alternate type switch is used as the emergency stop switch, the mute mode continues until the emergency stop switch is treated as being returned. In the mute mode, sounds other than those corresponding to voice calls in the conversation mode are muted. This is because it is annoying for other voices to continue to be output when the robot 20 is in an emergency stop state, the emergency stop state means that there is a person nearby and a certain degree of safety is ensured, the robot 20 is no longer performing normal business, and so on.

[0054] FIG. 6 shows a list of the above-described audio output controls. That is, in normal mode, the safety measure sound is output at a volume according to the surrounding sound environment. In user-set sound output mode and system-set sound output mode, the safety measure sound is output at the minimum volume. In conversation priority mode, the system sound, system-set sound, and user-set sound are muted (indicated as "0" in FIG. 6), and the safety measure sound is output at the minimum volume. In conversation mode and mute mode, the system sound, system-set sound, user-set sound, safety measure sound, and call sound are all muted. The relationship between the audio types and audio output modes in FIG. 6 is stored as the audio output control information 234 of the robot 20 and the audio output control information 134 of the management cloud 10, respectively.

[0055] Since such a plurality of voice output modes are defined, appropriate voice output control can be performed for each voice type according to the state of the robot 20, taking into consideration safety and convenience.

[0056] [Priority output control based on audio type] Next, the priority output control based on the above-mentioned audio type will be described. The output priority order among the basic audio types is as follows, in descending order of priority:

[0057] Conversational sound > Call sound > System settings sound > User settings sound

[0058] A conversation voice in which a person, such as a security guard at the facility manager terminal device 30, is talking through the robot 20 has the highest priority because it means that the situation around the robot 20 requires some kind of support from the facility management side. This is followed by a call sound connected to the situation in which the conversation voice is being uttered. Next, a system setting sound having content that informs the surroundings about the behavior of the robot 20 is given priority. This priority is stored as the voice output control information 234 of the robot 20 and the voice output control information 134 of the management cloud 10, respectively.

[0059] The output of the system sound and the safety sound is controlled independently of the above-mentioned priority order. First, the system sound is output when the output condition is satisfied in an audio output mode other than the conversation priority mode, the conversation mode, and the mute mode. The system sound is an artificial sound that is not in a natural language, and is based on the idea that there is no problem even if it is output together with the system setting sound and the user setting sound in a natural language. This priority order is stored as the audio output control information 234 of the robot 20 and the audio output control information 134 of the management cloud 10, respectively.

[0060] The safety measure sound is muted in conversation mode and mute mode. In addition, when the safety measure sound overlaps with the system setting sound and the user setting sound, the safety measure sound is suppressed to a predetermined minimum volume. The safety measure sound is an artificial voice that is not a natural language, and is based on the idea that there is no problem even if it is outputted overlapping with the system setting sound and the user setting sound in natural language. However, the safety measure sound is outputted at a predetermined minimum volume considering safety so that the system setting sound and the user setting sound in natural language are not hindered from being heard. In addition, this minimum volume necessary for safety is set for each map held by the robot 20, taking into consideration the environmental sound level of the surrounding environment. In this case, the robot 20 may be provided with an environmental sound level detection function, and the minimum volume level may be dynamically changed based on the detection result. The reason why the safety measure sound is not muted is that the output time interval of the user setting sound is left to the user, and if the safety measure sound is muted, there is a risk that the robot 20 will operate silently, which may be a safety problem. This priority is stored as the audio output control information 234 of the robot 20 and the audio output control information 134 of the management cloud 10, respectively.

[0061] The above voice output priority control enables the robot 20 to execute user-friendly voice output control that takes into account both safety and convenience, based on weighting that takes into account the safety of each voice type and user convenience.

[0062] [Voice output control of robot 20] Next, a description will be given of the voice output control executed in the robot 20 of the robot system 1 having the above-described configuration. Fig. 7 is a flowchart illustrating the flow of the voice output control process executed by the voice output control unit 224 of the robot 20. The control flow example in Fig. 7 is started when the robot 20 is started and enters an operating state, and is executed continuously thereafter while the robot 20 is operating.

[0063] In step S100, the audio output control unit of the robot 20 sets the audio output mode to "normal." Specifically, the audio output control unit 224 selects and sets "normal" from the audio output modes stored in the audio output control information 234. Thereafter, the robot 20 moves autonomously while outputting a safety measure sound at a volume equal to or higher than the minimum volume, as exemplified in FIG. 6.

[0064] In step S110, the audio output control unit 224 continuously monitors whether the transition condition of the audio output mode other than the normal mode is satisfied. In step S120, the audio output control unit 224 determines whether the transition condition is satisfied, specifically, whether the transition condition to the system setting sound output mode or the user setting sound output mode is satisfied. The audio output control unit 224 determines whether the transition condition is satisfied based on the operation status of the robot 20 and the audio output control information 234. If the audio output control unit 224 determines that the transition condition to the system setting sound output mode or the user setting sound output mode is satisfied (S120, YES), in step S130, the audio output control unit 224 extracts the corresponding audio data from the system setting sound SAS1, the user setting sound UAS1, etc. of the audio data 225 based on whether the audio output mode has been transitioned to the system setting sound output mode or the user setting sound output mode, and outputs the extracted audio data from the speaker as audio. When the transition conditions for both the system-set sound output mode and the user-set sound output mode are met, the audio output control unit 224 outputs the system-set sound in priority over the user-set sound.

[0065] Regardless of whether the mode has transitioned to the system setting sound output mode or the user setting sound output mode in S130, the audio output control for the system sound and the safety measure sound is executed separately by the audio output control unit 224 in step S140. Regardless of whether the audio output mode is the system setting sound output mode or the user setting sound output mode, when the output condition for the system sound is satisfied, the audio output control unit 224 outputs the system sound by the system sound SS1 or the like extracted from the audio data 225 in overlap with the system setting sound or the user setting sound. This is according to a policy that there is no problem even if the system sound, which is an artificial sound, is output in overlap with the system setting sound or the user setting sound, which is a natural language.

[0066] On the other hand, in either the system setting sound output mode or the user setting sound output mode, the sound output control unit 224 outputs the safety measure sound SFTS1 etc. of the sound data 225 at the minimum volume. This prevents the system setting sound or the user setting sound, which is a natural language, from being silent for a certain period of time or more. This is because it is a safety problem for the robot 20 to operate in silence. On the other hand, since the sound is output overlapping with the system setting sound or the user setting sound composed of a natural language, the playback volume is set to the minimum so as not to cause any impediment to hearing the system setting sound or the user setting sound.

[0067] In step S150, the audio output control unit 224 determines whether the transition condition of the system-set sound mode or the user-set sound mode has been released. If it is determined that the transition condition has not been released, the audio output control unit 224 repeats the processes of steps S130 and S140. If it is determined that the transition condition has been released, the audio output control unit 224 proceeds to the process of step S160.

[0068] In step S160, the audio output control unit 224 determines whether the robot 20 stops operating and ends the audio output process. If it is determined that the audio output process should not be ended (step S160, NO), the audio output control unit 224 sets the audio output mode to the normal mode in step S100. If it is determined that the audio output process should be ended (step S160, YES), the audio output control unit 224 ends the audio output process.

[0069] According to the sound output control process described above, the sound output mode can be set to any one of the normal mode, the system setting sound output mode, and the user sound playback mode based on the operating status of the robot 20, and appropriate sound suited to the operating status of the robot 20 can be output while ensuring safety. In particular, the system setting sound is played back with priority over the user setting sound, and the system sound and the safety measure sound are played back overlapping with the system setting sound and the user setting sound, so that both safety and convenience can be achieved.

[0070] [State transition control of audio output] Next, a state transition control is described which determines under what conditions the robot 20 transitions to another sound output mode, that is, mute mode, conversation priority mode, or conversation mode, when the sound output mode is set to the normal mode, the system-set sound output mode, or the user-set sound output mode. FIG. 8 is a state transition diagram showing conditions for transitioning to another sound output mode, that is, mute mode, conversation priority mode, or conversation mode, when the sound output mode is set to the normal mode, the system-set sound output mode, or the user-set sound output mode in the robot 20. For simplicity, the normal mode, the system-set sound output mode, and the user-set sound output mode are collectively referred to as the "normal mode." In the normal mode, the microphone of the facility manager terminal device 30 is in a muted state.

[0071] The arrows connecting the blocks representing the normal mode, mute mode, conversation priority mode, and conversation mode indicate the direction of state transition, and the operation or process that causes the state transition is described next to each arrow. The state transitions between the audio output modes will be explained below in order.

[0072] (1) Between normal mode and mute mode The mute mode is a mode for muting the audio output of the robot 20. When the emergency stop switch of the robot 20 is operated in the normal mode, the robot transitions to the mute mode. When the emergency stop is released, the robot transitions from the mute mode to the normal mode.

[0073] (2) Between stationary mode and conversation priority mode In the normal mode, if a specific location on the robot 20 is touched in order to have a conversation with a security guard or the like of the facility manager terminal device 30, and if the presence of some object is detected, such as a person standing in front of the robot 20, or if the microphone of the facility manager terminal device 30 is unmuted and turned on, the robot 20 transitions to the conversation priority mode. This makes it possible to prepare for a conversation between the robot 20 and the facility manager terminal device 30. In addition, in the conversation priority mode, if the microphone of the facility manager terminal device 30 is muted or no object is detected in front of the robot 20, the robot transitions to the normal mode. (However, the transition to the normal mode without the detection of an object in front occurs only when the conversation priority mode is entered by the robot touch.)

[0074] (3) Between stationary mode and conversation mode The conversation mode is a state in which a conversation is taking place between the robot 20 and the facility manager terminal device 30. The normal mode does not transition directly to the conversation mode, but passes through the conversation priority mode. On the other hand, from the conversation mode, the microphone of the facility manager terminal device 30 is muted, and when a timeout occurs after a certain period of time has elapsed, the mode transitions to the normal mode.

[0075] (4) Between Mute Mode and Conversation Priority Mode In the mute mode, if the microphone of the facility manager terminal device 30 is in the unmuted state when the emergency stop switch is released, the mode transitions to the conversation priority mode. If the emergency stop switch of the robot 20 is operated in the conversation priority mode, the mode transitions to the mute mode.

[0076] (5) Between mute mode and conversation mode The mute mode does not transition directly to the conversation mode, but passes through the conversation priority mode. On the other hand, from the conversation mode, when the emergency stop switch of the robot 20 is operated, the mode transitions to the mute mode.

[0077] (6) Between conversation priority mode and conversation mode In the conversation priority mode, the mode transitions to the conversation mode if a predetermined voice level is detected by the speech of a person on the facility manager terminal device 30 side. In the conversation mode, the mode transitions to the conversation priority mode if a certain time passes without detecting a predetermined voice level while the microphone is unmuted, resulting in a timeout.

[0078] According to the state transition control of the voice output described above, when the emergency stop switch of the robot 20 is operated, the robot waits in a stopped state without emitting any voice. Also, the state transition between a conversation priority mode in which a conversation between the robot 20 and the facility manager terminal device 30 is awaited, and a conversation mode in which a conversation actually takes place, can be appropriately performed through detection of the voice level.

[0079] [Example] Next, the audio output control of this embodiment will be illustrated by an example.

[0080] Case 1: The robot 20 outputs a guidance voice as a user-defined sound while patrolling the facility. The robot 20 autonomously travels to the facility entrance, and when it reaches the destination that has been set in advance as map information, it outputs a user-defined sound saying, "If you do not have an IC card, please come to the reception desk" (user-defined sound output mode). When a visitor touches a specific part of the robot 20 (for example, a sticker indicating the location of the call sensor), the robot transitions to conversation priority mode, the user-set sounds are muted, and a call sound or a voice such as "Security guard is being called" is output. When a remote monitor (security guard) located at the facility manager terminal device 30 starts talking via the microphone, the voice output mode transitions to a conversation mode, and the voice of the remote monitor is output from the robot 20. When the remote observer mutes the microphone, the system will transition back to user-defined sound output mode and output the message "If you do not have an IC card, please come to the reception desk."

[0081] Case 2 While the robot 20 is autonomously traveling and patrolling within the facility, it outputs a user-defined sound saying, "Customers attending the robot exhibition should come to Hall A" (user-defined sound output mode). The robot 20 arrives at the elevator hall to board the elevator and starts the operation for boarding the elevator while communicating with the elevator management device. At this time, the voice output mode transitions from the user-defined sound output mode to the system-defined sound output mode, and the robot 20 calls attention to the surroundings while outputting the system-defined sound "This elevator is for robot-only operation." When the elevator arrives at the destination floor, the robot 20 gets off the elevator. At that time, the robot 20, which is in the system setting sound output mode, alerts the surroundings by saying, "The robot is getting off." After the passenger has disembarked, the robot 20 starts patrolling again. At this time, the audio output mode becomes the user-defined audio output mode again, and the user-defined audio "Customers attending the robot exhibition should come to Hall A" is played.

[0082] The above-mentioned cases 1 and 2 are examples of the voice output control processing of the robot 20, and the robot 20 can autonomously transition to a corresponding voice output mode according to the operating status of each facility in which it is installed, and output a voice appropriate to the operating status.

[0083] As described above, according to the audio output control process of the present embodiment, audio output can be executed according to various situations without compromising the safety and convenience of the mobile device.

[0084] When a state is present in which a voice in a natural language and a voice in a non-natural language can be output simultaneously, the voice output control unit 224 may allow the voice in the natural language and the voice in the non-natural language to be output in an overlapping manner. In this way, information can be transmitted by both natural language speech and non-natural language speech, which do not impair the ease of listening to each other even when output in an overlapping manner, and convenience is not impaired.

[0085] The audio output control unit 224 may stop all audio output from the robot 20 when an emergency stop is performed to forcibly stop the operation of the robot 20. In this way, when the robot 20 is in an emergency stop and there is no need to alert those in the vicinity, no sound is emitted, thereby reducing annoyance to those in the vicinity.

[0086] The audio output control unit 224 is capable of operating in a conversation priority mode in which a voice call can be made between the robot 20 and an external device such as the facility manager terminal device 30, and a conversation mode in which a voice call is actually being made between the robot 20 and an external device. When the audio output control unit 123, 224 operates in the conversation mode, it may transition from an operating state in the conversation priority mode to the conversation mode. In this way, sounds that may interfere with conversation can be suppressed, for example by muting audio output other than the safety sound before an actual conversation begins, thereby improving safety and convenience.

[0087] When the conversation mode is established, the audio output control unit 224 may stop outputting audio other than the voice of the conversation between the robot 20 and the external device. In this way, other audio outputs that may interfere with the conversation are suppressed, improving safety and convenience.

[0088] When the conversation priority mode is established, the audio output control unit 224 may output only a safety measure sound at a minimum volume, the safety measure sound being output to notify surrounding people of the presence of the mobile device. In this way, by muting audio output other than the safety measure sound before an actual conversation begins, sounds that may interfere with the conversation are suppressed, thereby improving convenience while maintaining safety.

[0089] When audio output in a natural language is executed, the audio output control unit 224 may control the volume of a safety measure sound that is output to notify the surroundings of the presence of the robot 20 to a minimum volume. In this way, the safety of the robot 20 being able to alert those around it is maintained, while the voice output in natural language becomes easier to hear, improving convenience.

[0090] The sound output control unit 224 may operate in a state in which only a safety measure sound is output to notify surrounding people of the presence of the robot 20 immediately after the robot 20 is started. In this way, the robot 20 starts to alert those in the vicinity immediately after it is started, ensuring safety.

[0091] The present embodiment also includes a method for controlling audio output of a robot 20 capable of autonomous movement, in which an audio output control unit 224 for controlling the audio output of the robot 20 outputs audio using audio data extracted from either a user-set sound or a system-set sound in which a group of different types of audio data 225 is stored, depending on the situation in which the robot 20 is placed, the user-set sound being set by a user via external input while the robot 20 is in operation, and the system-set sound being set in advance before the robot 20 begins to be operated, and the user is not allowed to change the settings. The present embodiment also includes a method for controlling audio output of a robot 20, and a program for causing the robot 20 to execute the audio output control method.

[0092] The above-mentioned series of processes can be executed by hardware or software. In other words, the functional configurations of Figs. 2 and 3 are merely examples and are not particularly limited. That is, it is sufficient that the management cloud 10 and the robot 20 are provided with a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the examples of Figs. 2 and 3. In addition, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination thereof. The functional configuration in this embodiment is realized by a processor that executes arithmetic processing, and the processors that can be used in this embodiment include those configured as single pieces of various processing devices such as single processors, multiprocessors, and multicore processors, as well as those in which these various processing devices are combined with processing circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays).

[0093] When a series of processes is executed by software, the programs constituting the software are installed into a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a general-purpose personal computer.

[0094] The recording medium including such a program is not only constituted by a removable medium such as a USB memory that is distributed separately from the device main body in order to provide the user with the program, but also constituted by a recording medium provided to the user in a state in which the program is preinstalled in the device main body. The removable medium is, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc, etc. The magneto-optical disk is, for example, an MD (Mini-Disk), etc. Also, the recording medium provided to the user in a state in which the program is preinstalled in the device main body is, for example, a ROM in which the program is recorded, a hard disk, etc.

[0095] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually.

[0096] Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and it is also possible to combine the configurations of the above-mentioned embodiments and the modified examples. Furthermore, various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0097] 1. Robot System 10. Managed Cloud 11,21 Processor 12,22 Memory 20. Robot 24 Network Communication Department 30 Facility manager terminal device 40 Network 224 Audio output control unit 225 Audio Data 234 Audio output control information

Claims

1. A mobile device capable of autonomous movement, a first voice data group storage unit and a second voice data group storage unit in which a first voice data group and a second voice data group of different types are stored; a voice output control unit that outputs voice using voice data extracted from either the first voice data group storage unit or the second voice data group storage unit according to a situation in which the mobile device is placed, the first voice data group can be set by a user through an external input during operation of the mobile device, and the second voice data group is set in advance before starting operation of the mobile device, and the user is not allowed to change the setting; Mobile device.

2. 2. The mobile device according to claim 1, wherein the voice output control unit allows the voice in the natural language and the voice in the non-natural language to be output in an overlapping manner when the voice in the natural language and the voice in the non-natural language can be output simultaneously.

3. 2. The method of claim 1, wherein the audio output control unit stops all audio output from the mobile device when an emergency stop is performed to forcibly stop the operation of the mobile device. The mobile device according to claim 1 .

4. The mobile device of claim 1, wherein the audio output control unit is capable of operating in a conversation priority mode in which a voice call can be made between the mobile device and an external device, and a conversation mode in which a voice call is actually being made between the mobile device and an external device, and when the audio output control unit operates in the conversation mode, it transitions from an operating state in the conversation priority mode to the conversation mode.

5. The mobile device according to claim 4 , wherein the audio output control unit stops outputting audio other than audio for a call between the mobile device and the external device when the conversation mode is established.

6. 5. The mobile device according to claim 4, wherein the audio output control unit, when the conversation priority mode is established, outputs only a safety measure sound that is output to notify surroundings of the presence of the mobile device at a minimum volume.

7. The mobile device according to claim 1 , wherein the audio output control unit controls the volume of a safety sound output for notifying surroundings of the presence of the mobile device to a minimum volume when audio output in a natural language is executed.

8. The mobile device according to claim 1 , wherein the sound output control unit operates in a state in which only a safety measure sound is output for notifying surroundings of the presence of the mobile device immediately after the mobile device is started up.

9. 1. A method for controlling audio output of a mobile device capable of autonomous movement, comprising: a processing unit for controlling audio output of the mobile device, performing voice output using voice data extracted from either a first voice data group storage unit or a second voice data group storage unit in which a first voice data group and a second voice data group of different types are stored according to a situation in which the mobile device is placed; the first voice data group can be set by a user through an external input during operation of the mobile device, and the second voice data group is set in advance before starting operation of the mobile device, and the user is not allowed to change the setting; A method for controlling audio output of a mobile device.

10. A processing unit for controlling audio output of a mobile device capable of autonomous movement, a program for executing voice output using voice data extracted from either a first voice data group storage unit or a second voice data group storage unit, in which a first voice data group and a second voice data group of different types are stored, according to a situation in which the mobile device is placed, the first voice data group can be set by a user through an external input during operation of the mobile device, and the second voice data group is set in advance before starting operation of the mobile device, and the user is not allowed to change the setting; program.

Images