Management system, management method, and information processing device

Abstract

We provide management technology that enables the efficient management of multiple robots. [Solution] A management system 100 for managing multiple robots that are not fixed in place, wherein each robot 20 has an acquisition unit (sensor unit) 21 that acquires behavior-related information related to its own actions and a transmission unit (communication unit) 22 that transmits the behavior-related information to the computer, and the computer 10 comprises a receiving unit (communication unit) 12 that receives the behavior-related information from the robot 20, a generation unit 13 that analyzes the received behavior-related information and generates update information to be passed on to other robots, and a distribution unit 14 that distributes the generated update information to the other robots.

Description

[Technical Field]

[0001] The present invention relates to a management system, a management method, and an information processing apparatus. [Background Art]

[0002] In recent years, self-standing remote monitoring robots (so-called avatar robots) equipped with cameras, monitors, and the like have become widespread.

[0003] For example, the following Patent Document 1 describes a mobile unit with a camera (avatar robot) comprising: a camera that provides a captured image to a remote operator; and a mask processing unit that executes mask processing for hiding at least a part of the captured image from the remote operator, and switches the mask processing in accordance with the application. [Prior Art Literature] [Patent Literature]

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2019-062308 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] Utilizing an avatar robot (hereinafter simply referred to as "robot") allows, for example, robots arranged in airports, large-scale commercial facilities, and the like to perform customer service work, and robots arranged in factories to perform product assembly work. In order to cause a remote robot to perform some work, methods such as autonomously controlling the robot, and remotely controlling the robot by an operator instructing the robot to perform operations are conceivable; however, with any of these methods, when there are a plurality of robots, there has been a problem that it becomes difficult to manage the work status and the like of each robot.

[0006] The present invention has been made in consideration of the circumstances described above, and provides a management technique capable of efficiently managing a plurality of robots. [Means for solving the problem]

[0007] A management system according to one aspect of the present invention is a management system for managing a plurality of robots that are not fixed in place using a computer, wherein each robot has an acquisition unit that acquires behavior-related information relating to its own actions and a transmission unit that transmits behavior-related information to the computer, and the computer comprises a receiving unit that receives behavior-related information from the robots, a generating unit that analyzes the received behavior-related information and generates update information to be passed on to other robots, and a distribution unit that distributes the generated update information to other robots. [Effects of the Invention]

[0008] According to the present invention, it becomes possible to efficiently manage multiple robots. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the network configuration of the management system 100 according to this embodiment. [Figure 2] This is an illustrative diagram to explain the overview of management system 100. [Figure 3] This is a block diagram showing the functional configuration of computer 10. [Figure 4] This is a block diagram showing the functional configuration of robot 20. [Figure 5] This diagram shows the physical configuration of computer 10. [Figure 6] This diagram shows the physical configuration of robot 20. [Figure 7] This is a sequence diagram showing an example of a process performed by computer 10 and robot 20. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the attached drawings. In each drawing, components denoted by the same reference numerals have the same or similar configurations.

[0011] Figure 1 shows the network configuration of the management system 100 according to this embodiment. The management system 100 comprises a plurality of robots 20-N (2≦N) that are not fixed in place, and a computer 10 capable of operating the 20-N.

[0012] Here, robot 20-N and computer 10 can communicate with each other via communication network N. Figure 1 shows one computer 10 as an example, but the number of computers 10 is arbitrary. Also, in the following explanation, unless there is a need to distinguish each robot 20-N, it will simply be referred to as robot 20.

[0013] One or more parts of the communication network N may be wired or wireless. The communication network N may include, but is not limited to, an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), part of the internet, part of a public switched telephone network (PSTN), mobile phone network, ISDNs (Integrated Service Digital Networks), wireless LANs, LTE (Long Term Evolution), CDMA (Code Division Multiple Access), Bluetooth (registered trademark), satellite communications, or two or more combinations thereof.

[0014] <Overview of Management System 100> Figure 2 is an illustrative diagram illustrating the overview of the management system 100. Each robot 20 is a mobile robot deployed, for example, in an airport, and performs customer service (customer service tasks) for customers in airport facilities and shops. Each robot 20 can switch between autonomous control, where it performs customer service tasks while making its own judgments about the situation, and remote control, where it operates according to instructions from a remote user. Each robot transmits behavior-related information related to its actions to the computer 10 at predetermined times (for example, when charging). To give an example, a robot 20 performing customer service tasks would transmit information such as the date and time of the service, the location, and the content of the customer service task to the computer 10 as behavior-related information.

[0015] The computer 10 integrally manages the operations of all robots 20 disposed within the airport. The computer 10 is equipped with an AI (Artificial Intelligence) model (customer service AI model) for managing customer service and the like provided by each robot. By using the customer service AI model, the computer 10 learns action-related information received from each robot 20, generates update information regarding customer service operations that should be handed over to other robots 20, and distributes the update information to the other robots 20.

[0016] When the other robots 20 receive the update information, they control their own actions based on the received update information. As an example, assume a case where update information indicating that a certain robot (referred to as "first robot" for convenience) 20 is serving a customer T is distributed from the computer 10 to the other robots 20.

[0017] In this case, if among the other robots 20 there is a robot 20 that is providing or intends to provide customer service to the customer T similarly to the first robot 20 (in other words, if there is a robot 20 that is performing an action overlapping with the action of the first robot 20), the other robot 20 stops its own action based on the received update information.

[0018] As described above, after analyzing the action-related information transmitted from each robot 20, by generating update information that should be handed over to other robots 20 and simultaneously distributing the update information to the other robots 20, it becomes possible to efficiently manage a plurality of robots. Hereinafter, the configurations of the computer 10 and the robots 20 will be described.

[0019] <Functional Configuration> (Computer 10) The computer 10 is an information processing device that integrally manages the operations of each robot 20. The computer 10 is configured by, for example, a server or a personal computer. The computer 10 may also be configured using a virtual server, a cloud server, or the like.

[0020] Figure 3 is a block diagram showing the functional configuration of computer 10. The computer (information processing device) 10 is composed of an input unit 11, a communication unit 12, a generation unit 13, and a distribution unit 14.

[0021] The input unit 11 receives various instructions regarding the operation of the robot 20. These instructions are entered by a remote operator or the like. If the robot 20 is to perform customer service tasks, the input unit 11 receives various instructions regarding communication with customers, as well as instructions to switch the control method of the robot 20.

[0022] The communication unit (receiving unit) 12 sends and receives various data from the robot 20 and other devices. To give an example, the communication unit 12 receives action-related information acquired by the robot 20, while also transmitting instructions entered by the remote operator to the corresponding robot 20.

[0023] The generation unit 13 generates update information to be passed on to other robots based on the behavior-related information of the robot 20 received by the communication unit 12. As already explained, the generation unit 13 is equipped with a customer service AI model for managing customer service provided by each robot. The customer service AI model can utilize, for example, a machine learning model for analyzing, evaluating, and judging input data based on training data (information necessary for training the model), or a generative AI model for generating new content from various input data.

[0024] The generation unit 13 uses such a customer service AI model to learn behavior-related information received from each robot 20 and generates updated information regarding customer service tasks to be handed over to other robots 20. The updated information may include information representing the current actions of each robot 20, as well as information representing future action plans (for example, guidance to the store, product explanation, etc.).

[0025] The distribution unit 14 distributes the update information for robot 20 generated by the generation unit 13 to other robots 20. The distribution timing can be set arbitrarily; for example, it may be distributed at a pre-set timing (e.g., at regular intervals), or it may be distributed in real time as soon as the update information is generated.

[0026] (Robot 20) Robot 20 is composed of, for example, a telepresence robot (also called an avatar robot) and may have moving parts such as wheels. Robot 20 is an unfixed robot and multiple units are deployed, for example, in a facility within an airport. Here, "unfixed robot 20" includes cases where robot 20 is a mobile type with wheels, etc., and cases where it is a wearable type that can be worn by a person and has a manipulator, etc. In this embodiment, a mobile robot is assumed. Mobile robots include those that run on one, two, or multiple wheels, those that run on caterpillar tracks, those that run on rails, those that move by jumping, those that walk on two legs, four legs, or multiple legs, those that navigate on or underwater using a screw, and those that fly using a propeller, etc. Wearable robots are described, for example, in MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa and Masahiko Inami, "Fusion: full body surrogacy for collaborative communication," Proceedings SIGGRAPH '18 ACM SIGGRAPH 2018 Emerging Technologies Article No. 7. Furthermore, robot 20 includes autonomous or semi-autonomous vehicles and heavy machinery, drones and various flying objects. Robot 20 also includes robots equipped with cameras that can move along rails.

[0027] The robot 20 operates based on commands and instructions from the computer 10, which has successfully authenticated the user. Here, user authentication may be performed using a known method, and the information for user authentication may be registered in advance.

[0028] Figure 4 is a block diagram showing the functional configuration of robot 20. The robot 20 is comprised of a sensor unit 21, a communication unit 22, a drive unit 23, and an action control unit 24.

[0029] The sensor unit (acquisition unit) 21 acquires behavior-related information concerning the robot's own actions. This behavior-related information includes motion image data captured by the camera, audio data representing conversations with customers acquired by the microphone, and ambient environmental information representing the robot's surroundings, such as ambient temperature data acquired by a temperature sensor, location data acquired by a GPS sensor, and date and time data acquired by a timer. Note that these are merely examples, and other data may be acquired depending on the system design.

[0030] The communication unit (transmitter) 22 sends and receives various data with the computer 10 and other devices. To give an example, the communication unit 22 transmits action-related information acquired by the robot 20 (i.e., video data, audio data, location data, date and time data, etc.) to the computer 10, while receiving various instructions transmitted from the computer 10. When the communication unit 22 transmits action-related information to the computer 10, it also transmits an identification ID to the computer 10 to identify the robot 20.

[0031] The drive unit 23 drives various parts of the robot 20 (such as arms and wheels) according to drive instructions and other commands transmitted from the computer 10.

[0032] The behavior control unit 24 controls its own actions based on update information of another robot (i.e., the first robot) 20 transmitted from the computer 10. Specifically, if it is determined from the update information that the actions of the first robot overlap with its own actions, it will stop its own actions. To explain using customer service as an example, if the first robot 20 is performing customer service for customer T, the behavior control unit 24 will stop the customer service for customer T to avoid overlapping tasks. Note that the control of actions is not limited to stopping the current action; it is possible to arbitrarily change the settings, for example, by changing the current action or starting a different action.

[0033] <Physical configuration> (Computer 10) Figure 5 shows the physical configuration of the computer 10 according to this embodiment. Computer 10 includes a CPU (Central Processing Unit) 10a, which corresponds to an arithmetic unit; RAM (Random Access Memory) 10b, which corresponds to a storage device; ROM (Read-only Memory) 10c, which corresponds to a storage device; a communication device 10d; an input device 10e; and an output device 10f. Each of these components is connected to each other via a bus so that data can be sent and received from each other. In this example, the case in which computer 10 is composed of a single computer is described, but computer 10 may be realized by combining multiple computers. Also, the configuration shown in Figure 5 is just one example, and computer 10 may have other configurations, or may not have some of these configurations.

[0034] The CPU 10a is a computing device that controls the execution of programs stored in RAM 10b or ROM 10c, and performs calculations and processing of data. The CPU 10a executes a program (management program) for managing multiple robots 20. The CPU 10a receives various data from the input device 10e and the communication device 10d, and outputs the calculation results of the data to the output device 10f or stores them in RAM 10b. The CPU 10a realizes the above-mentioned functions by executing the management program stored in ROM 10c or the like.

[0035] RAM10b is a data-rewritable memory device and may be composed of semiconductor memory elements, for example. RAM10b may store the program executed by CPU10a, user information such as a user ID that identifies the user, etc. Note that these are examples, and RAM10b may store other data, or some of this data may not be stored.

[0036] ROM10c is a data readable memory device and may be composed of, for example, semiconductor memory elements. ROM10c may store, for example, a management program or data that is not rewritable.

[0037] The communication device 10d may include various communication interfaces for connecting the computer 10 to other devices. The communication device 10d may be connected to a communication network N such as the Internet or a telephone line.

[0038] The input device 10e accepts data input from an external source and may include, for example, a touch panel or a microphone.

[0039] The output device 10f outputs the calculation results from the CPU 10a and may consist of a display panel such as an LCD (Liquid Crystal Display) or a speaker. The output device 10f may output action-related information acquired by each robot 20.

[0040] These physical configurations are illustrative and do not necessarily have to be independent. For example, computer 10 may have a large-scale integration (LSI) that integrates the CPU 10a with RAM 10b and ROM 10c.

[0041] (Robot 20) Figure 6 shows the physical configuration of the robot 20 according to this embodiment. The robot 20 includes a CPU 20a (corresponding to a computing device), a RAM 20b (corresponding to a memory device), a ROM 20c (corresponding to a memory device), a communication device 20d, an output device 20e, a drive device 20f, and a sensor group 20g. These components are connected to each other via a bus, enabling data transmission and reception. Note that the configuration shown in Figure 6 is just an example, and the robot 20 may have other configurations, or may not have some of these configurations.

[0042] The CPU 20a is a processing unit that controls the execution of programs stored in RAM 20b or ROM 20c, and performs calculations and processing of data. The CPU 20a executes programs (action-related programs) for acquiring action-related information. The CPU 20a receives various data from communication devices 20d and the like, and displays the results of data calculations on output devices 20e or stores them in RAM 20b. The CPU 20a also controls the drive device 20f to control the movement of the robot 20.

[0043] RAM20b is a data-rewritable memory device and may be composed of semiconductor memory elements, for example. RAM20b may store programs executed by CPU20a, specific information about the robot 20, etc. Note that these are examples, and RAM20b may store other data, or some of this data may not be stored.

[0044] ROM20c is a memory device that allows data to be read, and may be composed of, for example, semiconductor memory elements.

[0045] The communication device 20d may include various communication interfaces for connecting the robot 20 to other devices. The communication device 20d may be connected to a communication network N such as the Internet.

[0046] The output device 20e outputs the calculation results from the CPU 20a, and may be composed of, for example, a display panel such as an LCD or a speaker.

[0047] The drive unit 20f includes remotely controllable actuators, wheels, manipulators, etc. If the robot 20 is a mobile robot, the drive unit 20f includes at least wheels, etc., but may also include manipulators. If the robot 20 is a wearable robot, the drive unit 20f includes at least manipulators.

[0048] The sensor group 20g acquires behavioral information of the robot 20 and may consist of, for example, a camera, microphone, GPS sensor, timer, thermometer, etc.

[0049] <Processing Flow> Figure 7 is a sequence diagram showing an example of processing performed by the computer 10 and robot 20 according to this embodiment. This flow is realized by the CPU 10a of the computer 10 and the CPU 20a of each robot 20 executing management programs and action-related programs, respectively. In the following description, we will assume that a robot (first robot) 20 is performing customer service tasks.

[0050] When the communication unit 22 of the first robot 20 under management acquires action-related information (such as information representing the date and time of customer service, location, and content of customer service work) from the sensor unit 21, it transmits this information, along with the robot's identification ID, to the computer 10 (step S1). As already explained, the timing of transmission of the acquired action-related information can be arbitrarily set and changed according to the system design, etc.

[0051] When the communication unit 12 of the computer 10 receives action-related information from the first robot 20 (step S2), it sends it to the generation unit 13.

[0052] The generation unit 13 inputs the behavior-related information received from the first robot 20 into the customer service AI model to generate updated information regarding customer service tasks to be handed over to other robots 20 (step S3). Details of the behavior-related information and updated information have already been revealed, so they will not be explained here.

[0053] The distribution unit 14 distributes the update information for robot 20 generated by the generation unit 13 to other robots 20 (step S4). The distribution timing is arbitrary, but for example, it may be distributed in real time when the update information is generated.

[0054] When the communication unit 22 of the other robot 20 receives update information from the first robot 20 (step S5), the action control unit 24 determines, based on the received update information, whether or not the actions of the first robot 20 overlap with its own actions (step S6).

[0055] If the action control unit 24 determines that the action of the first robot 20 does not overlap with its own action (step S6; NO), it continues its own action at the present time (for example, advertising a store) (step S7). On the other hand, if the action control unit 24 determines that the action of the first robot 20 overlaps with its own action, it stops its own action at the present time (step S8) to avoid overlapping work by other robots.

[0056] As described above, according to this embodiment, after analyzing the action-related information transmitted from each robot 20, update information to be passed on to other robots 20 is generated and distributed to the other robots 20 simultaneously. The other robots 20 control their own actions based on the received update information, making it possible to efficiently manage multiple robots.

[0057] B. Variations In the embodiment described above, we explained the case of managing a robot 20 that performs customer service tasks, but this is not the only example. For example, it can be applied to any robot 20 that performs any task, such as a robot 20 that patrols a facility and performs security work, or a robot 20 that assembles products in a factory.

[0058] Furthermore, although this embodiment describes the case in which update information is distributed to other robots 20, it may also be distributed to, for example, the terminal of the operator who remotely controls each robot 20, the terminal of the operator who works together with the robot 20, or even the terminal of the administrator who manages the computer 10.

[0059] Furthermore, other robots 20 may determine whether the actions of robot 20 overlap with their own based on the update information received from robot 1 20, but this is not the only way to explain. To illustrate with an example where each robot 20 performs customer service, they may determine the "level of hospitality" to the customer based on the update information received from robot 1, and then decide on the content of the customer service work to be performed based on the result of that determination.

[0060] The "hospitality level" is set by dividing the level of hospitality provided through customer service into multiple stages (for example, three stages). For example, the number of customers can be determined from the surrounding environment information included in the update information, and the fewer the customers, the higher the "hospitality level" can be set. The surrounding environment information may include, for example, video image data to determine the number of customers within a certain range, ambient sound data representing the surrounding environment sounds, and temperature data representing the surrounding temperature. In this way, the higher the "hospitality level," the more attentive customer service may be provided (for example, accompanying the customer and giving them a tour of the store).

[0061] The embodiments and modifications described above are for the purpose of facilitating understanding of the present invention and are not intended to limit its interpretation. Each element, as well as its arrangement, materials, conditions, shape, and size, etc., are not limited to those exemplified and can be modified as appropriate. Furthermore, each element can be partially substituted or combined with others. Furthermore, in this embodiment and its modifications, "part" and "device" do not merely mean physical means, but also include cases where the functions of the "part" and "device" are realized by software. Also, the functions of one "part" and "device" may be realized by two or more physical means, or the functions of two or more "parts" and "devices" may be realized by one physical means. [Explanation of Symbols]

[0062] 10...Computer, 11...Input unit, 12...Communication unit, 13...Generation unit, 14...Distribution unit, 10a...CPU, 10b...RAM, 10c...ROM, 10d...Communication device, 10e...Input device, 10f...Output device, 20...Robot, 21...Sensor unit, 22...Communication unit, 23...Drive unit, 24...Action control unit, 20a...CPU, 20b...RAM, 20c...ROM, 20d...Communication device, 20e...Output device, 20f...Drive device, 20g...Sensor group, 100...Management system

Claims

1. A management system that uses a computer to manage multiple robots that are not fixed in place, The aforementioned robot, An acquisition unit that acquires behavioral information related to its own actions, It has a transmitting unit that transmits the aforementioned behavior-related information to the computer, The aforementioned computer, A receiving unit that receives the action-related information from the robot, A generation unit that analyzes the received behavior-related information and generates updated information to be passed on to other robots, A distribution unit that distributes the generated update information to the other robots, A management system equipped with the following features.

2. The aforementioned other robots A receiving unit that receives the aforementioned update information, Based on the aforementioned update information, a control unit controls its own actions, The management system according to claim 1, comprising:

3. The control unit of the other robot is The management system according to claim 2, wherein, based on the update information, if it is found that the robot's actions overlap with its own actions, the robot stops its own actions.

4. The transmission unit of the robot is The management system according to claim 1 or 2, wherein the system transmits the action-related information to the computer at a pre-set timing.

5. The distribution unit of the aforementioned computer is The management system according to claim 1 or 2, which distributes the update information, along with the other robots, to the terminal of an operator who takes over the work of the robot and / or the terminal of an operator who remotely controls the robot.

6. The aforementioned behavior-related information includes ambient environment information that represents the conditions surrounding the robot. The management system according to claim 1 or 2, wherein the generation unit changes the content of the update information to be handed over to the other robot based on the surrounding environment information.

7. A management method for managing multiple robots that are not fixed in place, using a computer, The robot acquires behavior-related information concerning its own actions in an acquisition step, The robot transmits the action-related information to the computer in a transmission step, The computer receives the action-related information from the robot in a receiving step, The computer analyzes the received behavior-related information and generates updated information to be passed on to other robots (generation step), A management method comprising a distribution step in which the computer distributes the generated update information to the other robots.

8. An information processing device for managing multiple robots that are not fixed in place, The robot has a receiving unit that receives behavior-related information concerning its own actions, A generation unit that analyzes the received behavior-related information and generates updated information to be passed on to other robots, A distribution unit that distributes the generated update information to the other robots, An information processing device equipped with the following.

Images