Robot control system, method for controlling robot, and program

Abstract

To flexibly control a robot according to the result of executing a predetermined operation.SOLUTION: A robot control system 100 includes a robot 1 and a user terminal 2. The user terminal 2 causes the robot 1 to execute a predetermined operation configured of a first execution unit and a second execution unit. The user terminal 2 instructs the robot 1 to execute the first execution unit, receives the result of execution of the first execution unit by the robot 1 from the robot 1, and instructs the robot 1 to execute the second execution unit according to the result of execution of the first execution unit.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a robot control system, a robot control method, and a program for causing a computer to execute the robot control method. [Background technology]

[0002] In recent years, in order to solve the labor shortage, it has been proposed to have robots take over certain tasks that have been performed manually up until now. For example, a robot has been proposed to guide customers in a store (see Patent Document 1). This robot can obtain information about visiting users so that it can grasp the information about the visiting users and provide customer service, and can handle the customers accordingly. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-144232 A Summary of the Invention [Problem to be solved by the invention]

[0004] When performing a predetermined task, the next step may be determined depending on the result of execution of a certain step in the task. However, the conventional robots described above are programmed to consistently execute all steps required for the predetermined task, and are not able to check information regarding the execution of each step. For this reason, the conventional robots are not able to perform flexible operations such as selecting an optimal step depending on the execution result that is different from the expectation, for example, when the execution result of a certain step is different from the expectation.

[0005] In addition, since conventional robots are programmed to consistently execute all the steps required for a predetermined task, when changing the task to be executed by the robot, it is necessary to rewrite the entire program. As a result, making such changes requires a huge workload and cost.

[0006] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to perform flexible control of a robot in accordance with the results of execution of a predetermined work process. [Means for solving the problem]

[0007] A robot control system according to one aspect of the present invention includes a robot and a first computer. The first computer causes the robot to execute a predetermined task composed of a first execution unit and a second execution unit. In this system, the first computer commands the robot to execute the first execution unit, receives a result of execution of the first execution unit by the robot from the robot, and commands the robot to execute the second execution unit according to the result of execution of the first execution unit.

[0008] According to this configuration, the robot is caused to execute a first execution unit corresponding to a specific operation performed in a predetermined task, and a result of execution of the first execution unit by the robot is received. Next, in accordance with the received execution result of the first execution unit, the robot is caused to execute a second execution unit corresponding to the next operation in the predetermined task. In this way, the execution result of the first execution unit is received, and the second execution unit to be executed next is executed in accordance with the execution result of the first execution unit, thereby realizing flexible control of the robot in accordance with the execution result of the specific operation during the predetermined task.

[0009] Furthermore, according to the above configuration, a predetermined task to be executed by the robot can be realized by combining the first execution unit and the second execution unit. In other words, a user can cause the robot to execute any task simply by programming any combination of multiple execution units. Furthermore, a task to be executed by the robot can be changed simply by changing the combination of execution units. As a result, the workload and cost associated with changing the task to be executed by the robot can be reduced.

[0010] In the above robot control system, the first execution unit and / or the second execution unit may be a program unit including at least one API request for requesting the robot to execute a predetermined function. With this configuration, even a user without advanced programming knowledge can easily create an application for causing the robot to execute the first execution unit and the second execution unit.

[0011] The robot control system may further include a second computer. The second computer has a command storage unit that stores an execution command for the robot corresponding to an execution unit to be executed by the robot. In this case, when the first computer commands the robot to execute the first execution unit or the second execution unit, the first computer may transmit the execution unit to be executed by the robot to the second computer, the second computer may store an execution command corresponding to the execution unit received from the first computer in the command storage unit, and the robot may execute the execution command stored in the command storage unit. According to this configuration, the robot may be caused to execute only the execution command stored in the command storage unit. As a result, the robot may be prevented from executing other actions that have not been commanded.

[0012] In the robot control system, the second computer may acquire the state of the robot, and when the robot is in a state in which the execution command can be executed, store the execution command in the command storage unit. With this configuration, it is possible to reliably cause the robot to execute the first execution unit and / or the second execution unit.

[0013] In the above robot control system, there may be a plurality of robots. In this case, the first computer may specify a robot to execute the first execution unit or the second execution unit when issuing a command to execute the first execution unit or the second execution unit. With this configuration, it is possible to select an optimal robot to execute the first execution unit or the second execution unit.

[0014] A method for controlling a robot according to another aspect of the present invention is a method for causing a robot to execute a predetermined task constituted by a first execution unit and a second execution unit. The method for controlling a robot includes the following steps. A step of instructing the robot to execute a first execution unit. A step of receiving, from the robot, a result of execution of the first execution unit by the robot. In response to receiving the execution result of the first execution unit, instructing the robot to execute a second execution unit.

[0015] According to this configuration, the robot is caused to execute a first execution unit corresponding to a specific operation performed in a predetermined task, and a result of execution of the first execution unit by the robot is received. Next, in accordance with the received execution result of the first execution unit, the robot is caused to execute a second execution unit corresponding to the next operation in the predetermined task. In this way, the execution result of the first execution unit is received, and the second execution unit to be executed next is executed in accordance with the execution result of the first execution unit, thereby realizing flexible control of the robot in accordance with the execution result of the specific operation during the predetermined task.

[0016] Furthermore, according to the above configuration, a predetermined task to be executed by the robot can be realized by combining the first execution unit and the second execution unit. In other words, a user can cause the robot to execute any task simply by programming any combination of multiple execution units. Furthermore, a task to be executed by the robot can be changed simply by changing the combination of execution units. As a result, the workload and cost associated with changing the task to be executed by the robot can be reduced. Effect of the Invention

[0017] According to the present invention, it is possible to control a robot flexibly according to the results of execution of a predetermined task process. Also, by simply programming an arbitrary combination of multiple execution units, it is possible to make the robot perform an arbitrary task. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 illustrates a robot control system. [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of a robot. [Diagram 3] FIG. 2 is a diagram showing the hardware configuration of a user terminal. [Figure 4] FIG. 2 is a diagram showing the hardware configuration of a first server. [Diagram 5] FIG. 4 shows the hardware configuration of a second server. [Figure 6] 11 is a flowchart showing the operation of a user terminal. [Figure 7] 10 is a flowchart showing the operation of the first server. [Figure 8] 4 is a flowchart showing the operation of the robot. [Figure 9] FIG. 2 is a diagram showing transmission and reception of information in a robot control system when a robot is caused to perform a predetermined task. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, a robot control system according to the present embodiment will be described with reference to the drawings.

[0020] <Robot control system> A robot control system 100 will be described below with reference to Fig. 1. Fig. 1 is a diagram showing the robot control system 100. As shown in Fig. 1, the robot control system 100 includes a robot 1, a user terminal 2, and a first server 3. The robot 1, the user terminal 2, and the first server 3 are connected to each other so as to be able to communicate with each other via a network 9. The network 9 is realized by, for example, the Internet and / or a communication network provided by a communication carrier.

[0021] The robot 1 performs a predetermined task. There are a plurality of robots 1 in the robot control system 100. Each of the plurality of robots 1 is assigned identification information (robot ID) for identifying the robot 1. When making a robot 1 perform a predetermined task, the robot ID of the robot 1 to perform the task is specified. The plurality of robots 1 are managed in a robot database DB (FIG. 4) of the first server 3.

[0022] The robot 1 is capable of independently executing a plurality of functions for performing a predetermined task. The functions that the robot 1 can execute include, for example, an autonomous driving function for autonomously driving to a predetermined position, a voice output function for outputting a predetermined voice, a voice input function for converting an input voice into a predetermined format (for example, text data, etc.), an object discrimination function for discriminating an object detected by a camera or the like, a face recognition function for authenticating a face detected by a camera or the like, etc. In addition, the robot 1 may have a remote control function for remotely controlling the robot 1.

[0023] The user terminal 2 (an example of a first computer) is a device configured with an information processing circuit (e.g., a CPU) that executes various information processes, a storage device (e.g., a ROM, a RAM, an SSD, a hard disk, etc.), various interfaces, etc. The user terminal 2 is, for example, a tablet terminal, a computer such as a smartphone or a personal computer, or an IoT device such as a smart speaker.

[0024] The user terminal 2 commands the robot 1 to execute a predetermined task. When the user terminal 2 causes the robot 1 to execute the predetermined task, the user terminal 2 causes the robot 1 to execute a combination of a plurality of execution units in sequence. In other words, the predetermined task executed by the robot 1 is configured as a combination of a plurality of execution units.

[0025] An execution unit is a program unit (e.g., a function) that causes the robot 1 to perform a specific operation. Examples of execution units include an execution unit that causes the robot 1 to autonomously travel to a predetermined position, an execution unit that causes the robot 1 to output a predetermined sound, an execution unit that causes the robot 1 to input sound, an execution unit that acquires an image (still image / video) of a person or the like in front of the robot 1, an execution unit that identifies an object in front of the robot 1, an execution unit that causes the robot 1 to travel behind a person or the like, and the like.

[0026] By simply combining a plurality of execution units as described above, it is possible to make the robot 1 perform any task. For example, by combining an execution unit that makes the robot 1 autonomously travel to a specified table, an execution unit that makes the robot 1 output a voice message such as "Sorry to keep you waiting" to a customer seated at the table indicating that the robot 1 has reached the specified table, an execution unit that makes the robot 1 input a voice response from the customer such as "Thank you," and an execution unit that makes the robot 1 autonomously travel away from the table in response to the voice input, it is possible to make the robot 1 perform a serving task of carrying food to a specified table.

[0027] In addition, for example, by combining an execution unit that causes the robot 1 to travel autonomously along a predetermined route, an execution unit that identifies people present around the robot 1 while it is traveling autonomously along the predetermined route, an execution unit that obtains video (still image / video) of the suspicious person if the identified person is a suspicious person, and an execution unit that outputs an audio warning such as "No entry here" to the suspicious person, it is possible to cause the robot 1 to perform a warning task of warning the suspicious person when it finds a suspicious person.

[0028] In addition, by appropriately combining the above execution units, the robot 1 can be made to perform tasks such as security, object searching, reception, and advertising. In security, when the robot 1 finds a suspicious person, it performs actions such as warning the suspicious person and calling for help from other robots 1. In object searching, the robot 1 autonomously travels to search for a specific object and notifies the location of the object. In reception, when the robot 1 detects a visitor, it greets the visitor with a greeting such as "Hello," guides the visitor to a designated location, counts the number of visitors, and so on. In advertising, the robot 1 displays advertisements according to various situations and / or outputs audio related to the advertisements.

[0029] An execution unit for making the robot 1 perform a specific operation is composed of at least one API (Application Programming Interface) request, a definition of certain parameters, and / or other various commands, etc. The API request is for requesting the first server 3 to make the robot 1 execute a certain function.

[0030] For example, an execution unit for making the robot 1 drive autonomously is composed of an API request for autonomous driving that requests the first server 3 to make the robot 1 drive autonomously to a specified position, definitions of various parameters, and various other commands.

[0031] The execution units are disclosed to the user in advance as source code and / or as executable functions. Also, a sample program (e.g., source code) that combines execution units to perform a predetermined task may be disclosed to the user in advance. Furthermore, an API may be disclosed in advance.

[0032] In the robot control system 100, a combination of multiple published execution units is written in an application program AP, and the user terminal 2 executes this application program AP to cause the robot 1 to execute a predetermined task. Alternatively, the user can stop the execution of the application program AP midway by operating the user terminal 2, and then select the execution unit he or she wants to execute. Furthermore, the user can select multiple execution units each time by operating the input unit 24 of the user terminal 2 to cause the robot 1 to execute a predetermined task.

[0033] The application program AP can be created, for example, in a predetermined program creation GUI by arranging blocks corresponding to each execution unit in order of execution and / or arranging multiple blocks that are executed in parallel in parallel. In this way, by using the program creation GUI, the application program AP can be created without writing the code for the application program AP.

[0034] Alternatively, for example, the application program AP can be created by writing character strings (execution unit names) representing each execution unit in the application program AP in a layout that follows the execution procedure of the execution unit, or by using a spreadsheet program to arrange the execution unit names in accordance with the execution procedure. Furthermore, the application program AP can be created by appropriately using a program creation GUI and code entry.

[0035] In this way, by disclosing to the user in advance the execution units that cause the robot 1 to perform specific operations, even a user without advanced knowledge of programming can easily create an application program AP by combining execution units in the above-mentioned manner. Also, the user can easily incorporate a combination of execution units that cause the robot 1 to perform a specified task into existing software (applications) that the user owns.

[0036] Furthermore, when changing the task to be performed by the robot 1, it is only necessary to change the combination of execution units, which reduces the operational and cost burdens associated with changing the task to be performed by the robot 1.

[0037] When the user terminal 2 makes the robot 1 execute a predetermined task, it transmits an execution unit, which is part of the predetermined task, to the first server 3. The first server 3 interprets the received execution unit and commands the robot 1 to execute an operation corresponding to the execution unit. When the robot 1 executes an operation corresponding to the execution unit, the execution result is transmitted to the user terminal 2 via the first server 3. The user terminal 2 transmits the next execution unit to the first server 3 in accordance with the received execution result. In this way, in the robot control system 100, the robot 1 is controlled in an "interactive format" in which commands and execution results are exchanged between the robot 1 and the user terminal 2.

[0038] The first server 3 (an example of a second computer) is a computer system configured with a CPU, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces (e.g., communication interface), etc. The first server 3 may be a virtual computer in which the CPU, storage device, various interfaces, etc. are realized by software.

[0039] The first server 3 receives an execution unit from the user terminal 2, and causes the robot 1 to execute an operation corresponding to the received execution unit. Specifically, the first server 3 interprets an API request, various commands, etc. included in the execution unit received from the user terminal 2, and generates a command (called an execution command) for causing the robot 1 to execute a specific operation corresponding to the execution unit based on the interpreted contents.

[0040] For this reason, the first server 3 provides an API function that interprets an API request included in an execution unit received from the user terminal 2, commands the robot 1 to execute the operation requested in the API request, and acquires the execution result from the robot 1 as necessary. Examples of the API functions provided by the first server 3 include a status request API, an autonomous driving API, a map request API, a following driving API, an audio output API, an audio input API, an object discrimination API, a video capture API, and a face recognition API.

[0041] The status request API is an API that acquires the status of the robot 1. The autonomous driving API is an API that makes the robot 1 autonomously drive to a specified position. The map request API is an API that acquires map data that indicates the operating area of ​​the robot 1 and the current position of the robot 1. The following driving API is an API that makes the robot 1 drive behind a person or the like.

[0042] The audio output API is an API that converts designated audio content (e.g., audio content written in text) into predetermined audio data (e.g., MP3 data) and outputs it to the robot 1. The audio input API is an API that acquires audio around the robot 1, converts the acquired audio into a predetermined format (text data), and outputs it.

[0043] The object discrimination API is an API that causes the robot 1 to discriminate objects present around the robot 1 and output information about the objects. The video capture API is an API that causes the robot 1 to capture still images and / or videos of people and the like present around the robot 1. The face recognition API is an API that instructs the robot 1 to perform face recognition using a trained model created in advance and outputs information about detected people.

[0044] The robot control system 100 may include a second server 4. The second server 4 is a computer system configured with a CPU, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces (e.g., communication interface), etc. The second server 4 may be a virtual computer in which the CPU, storage device, various interfaces, etc. are realized by software.

[0045] The second server 4 is capable of communicating with the robot 1, the user terminal 2, and the first server 3 via the network 9. The second server 4 provides an application program AP for causing the robot 1 to execute a predetermined task. When the second server 4 is provided in the robot control system 100, the application program AP may be downloaded to the user terminal 2 and executed on the user terminal 2, or the second server 4 may execute the application program AP in response to an operation of the user terminal 2.

[0046] <Robot configuration> A specific configuration of the robot 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing a hardware configuration of the robot 1. As shown in Fig. 2, the robot 1 has a first control device 11, a traveling unit 12, a first communication unit 13, a sensor 14, a camera 15, an audio output unit 16, and an audio input unit 17.

[0047] The first control device 11 is a computer system configured with a CPU, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces, etc., and controls the robot 1. The first control device 11 has a first control unit 111 and a first storage unit 113. The first control unit 111 is configured with the CPU of the first control device 11, etc., and performs various information processing in the first control device 11.

[0048] The first control unit 111 executes each function program stored in the first storage unit 113 to realize various functions of the robot 1.

[0049] The first storage unit 113 is a data storage area of ​​the storage device of the first control device 11, and stores the above-mentioned function programs, various setting parameters, etc. Specifically, the first storage unit 113 stores, for example, an autonomous driving program P1 that realizes an autonomous driving function that autonomously drives the robot 1 to a predetermined position, a voice output program P2 that realizes a voice output function that causes the robot 1 to output a predetermined voice, a voice input program P3 that realizes a voice input function that converts an input voice into a predetermined format, an object discrimination program P4 that realizes an object discrimination function that discriminates an object detected by the camera 15 or the like, and a face recognition program P5 that realizes a face recognition function that authenticates a face detected by the camera 15 or the like. In addition, the first storage unit 113 may store a remote control program that realizes a remote control function that remotely controls the robot 1.

[0050] In this way, the first storage unit 113 of the robot 1 stores programs for executing each function required for the predetermined task, rather than a program for executing the entire predetermined task to be executed by the robot 1. The first control unit 111 executes any one of the above programs to cause the robot 1 to execute the predetermined function.

[0051] The running unit 12 causes the robot 1 to run. The running unit 12 is composed of, for example, wheels, a motor for rotating the wheels, etc. The first control unit 111 controls the rotation of the motor of the running unit 12 to control the running of the robot 1.

[0052] The first communication unit 13 is connected to the network 9 and executes communication with other devices (such as the user terminal 2, the first server 3, and the second server 4). The first communication unit 13 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0053] The sensor 14 is configured to detect the distance to an object around the robot 1. The sensor 14 outputs data related to the detected distance to the first control device 11. The sensor 14 is, for example, a distance measuring sensor such as LiDAR (Light Detection and Ranging). The sensor 14 outputs the detected data (for example, point cloud data) to the first control device 11. The first control unit 111 of the first control device 11 performs self-location estimation and environmental map creation based on the data acquired from the sensor 14. The environmental map represents the positions of walls, obstacles, etc. in the place where the robot 1 operates. The created environmental map is stored in the first storage unit 113. The user terminal 2, the first server 3, and the second server 4 can acquire the environmental map stored in the first storage unit 113.

[0054] The camera 15 is configured to capture an image of the periphery of the robot 1. Video data acquired by the camera 15 is output to the first control device 11. The first control unit 111 stores the acquired video data in the first storage unit 113 as a moving image or a still image.

[0055] The audio output unit 16 is configured to output a predetermined sound. The audio output unit 16 is, for example, a speaker. The first control unit 111 can output a predetermined sound from the audio output unit 16 by executing the audio output program P2 using the content of the sound to be output from the audio output unit 16 as input data.

[0056] The voice input unit 17 is configured to input voice generated around the robot 1. The voice input unit 17 is, for example, a microphone. The first control unit 111 executes the voice input program P3 using the voice input by the voice input unit 17 as input data, thereby obtaining data representing the content of the input voice (for example, text data of the voice content).

[0057] The robot 1 may have a display unit. The display unit displays various information such as advertisements. The display unit is, for example, a display device such as a liquid crystal display, an organic EL display, or a CRT display.

[0058] <User terminal configuration> A specific configuration of the user terminal 2 will be described with reference to Fig. 3. Fig. 3 is a diagram showing a hardware configuration of the user terminal 2. As shown in Fig. 3, the user terminal 2 has a second control unit 21, a second storage unit 22, a second communication unit 23, an input unit 24, and a display unit 25. The second control unit 21 is a CPU or the like provided in the user terminal 2, and performs information processing in the user terminal 2. The second control unit 21 performs information processing in the user terminal 2 by executing a program (for example, an application program AP described below) stored in the second storage unit 22.

[0059] The second storage unit 22 is a data storage area of ​​the storage device of the user terminal 2, and stores various programs, setting parameters, etc. Specifically, the second storage unit 22 stores an application program AP. In addition, an execution unit for causing the robot 1 to execute a specific operation, etc. may be stored in the second storage unit 22.

[0060] The second communication unit 23 is connected to the network 9 and executes communication with other devices (the robot 1, the first server 3, the second server 4, etc.) The second communication unit 23 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0061] The input unit 24 is composed of a touch panel, a keyboard, a pointing device (such as a mouse), and / or an operation button, etc. The input unit 24 accepts operations from a user. The display unit 25 is an image display device such as a liquid crystal display, an organic EL display, a CRT display, etc. The display unit 25 displays various information.

[0062] <First server configuration> A specific configuration of the first server 3 will be described with reference to FIG. 4. FIG. 4 is a diagram showing a hardware configuration of the first server 3. As shown in FIG. 4, the first server 3 has a third control unit 31, a third storage unit 32, and a third communication unit 33. The third control unit 31 is a CPU or the like provided in the first server 3, and performs information processing in the first server 3. The third control unit 31 executes a program stored in the third storage unit 32 to perform information processing in the first server 3. Specifically, the third control unit 31 performs information processing to realize the above-mentioned API function, information processing to interpret the contents of the execution unit, and the like.

[0063] The third storage unit 32 is a data storage area of ​​the storage device of the first server 3, and stores various programs, setting parameters, etc. Specifically, the third storage unit 32 stores an API program IP for implementing API functions, a control program CP for interpreting the contents of execution units, a robot database DB, etc.

[0064] The robot database DB manages the identification information (robot ID) of each of the multiple robots 1 present in the robot control system 100. In the robot database DB, the identification information of the robot 1 may be stored in association with information required to access the robot 1 (e.g., IP address) and / or information related to the robot 1 (e.g., model name, use, etc.).

[0065] The third storage unit 32 has a command storage unit 321. The command storage unit 321 stores an execution command for the robot 1 corresponding to an execution unit received from the user terminal 2. One command storage unit 321 is assigned to each of the multiple robots 1.

[0066] The robot 1, which is in a state capable of executing an execution command, constantly monitors whether an execution command is stored in its own command memory unit 321, and if an execution command is stored in the command memory unit 321, executes the execution command.

[0067] The third communication unit 33 is connected to the network 9 and executes communication with other devices (the robot 1, the user terminal 2, the second server 4, etc.) The third communication unit 33 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0068] <Second server configuration> A specific configuration of the second server 4 will be described with reference to Fig. 5. Fig. 5 is a diagram showing a hardware configuration of the second server 4. As shown in Fig. 5, the second server 4 has a fourth control unit 41, a fourth storage unit 42, and a fourth communication unit 43.

[0069] The fourth control unit 41 is a CPU or the like provided in the second server 4, and performs information processing in the second server 4. The third storage unit 32 is a data storage area of ​​a storage device of the user terminal 2, and stores various programs, setting parameters, and the like. Specifically, the fourth storage unit 42 stores an application program AP to be provided to the user terminal 2. The fourth communication unit 43 connects to the network 9, and executes communication with other devices (the robot 1, the user terminal 2, the first server 3, etc.). The fourth communication unit 43 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0070] <User terminal operation> Next, a description will be given of the control of the robot 1 in the above-mentioned robot control system 100. First, the operation of the user terminal 2 in controlling the robot 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the operation of the user terminal 2.

[0071] First, the second control unit 21 of the user terminal 2 transmits an execution unit to be executed this time to the first server 3 (step S11). The execution unit to be executed this time is determined, for example, depending on which execution unit has been executed in the application program AP, or is determined by user selection.

[0072] When transmitting the execution unit to be executed this time, the second control unit 21 designates which robot among the multiple robots 1 is to execute the execution unit. Specifically, the second control unit 21 transmits identification information (robot ID) of the robot 1 to be executed for the execution unit together with the execution unit to the first server 3. As a result, the operation corresponding to the execution unit transmitted to the first server 3 in step S11 is executed by the robot 1 designated in step S11.

[0073] After transmitting the execution unit, the second control unit 21 of the user terminal 2 determines whether or not the execution result of the execution unit transmitted in step S11 has been received (step S12). The execution result of the execution unit can be, for example, "Completed", which indicates that the execution of the execution unit has been completed normally, or "Abnormal", which indicates that an abnormality has occurred during the execution of the execution unit. The execution result can be "Abnormal", for example, when the specified robot 1 does not exist, when the specified robot 1 cannot execute the execution unit, or when an abnormality such as an error has occurred during the execution of the execution unit.

[0074] Depending on the execution unit executed by the robot 1, an execution result based on information acquired by the robot 1 (for example, the result of identifying an object, the result of authenticating a human face, the contents of an input voice, etc.) is received by the user terminal 2. In addition, the execution result received by the user terminal 2 may include "executing" which indicates that the execution unit is being executed.

[0075] If the execution result has not been received ("No" in step S12), the second control unit 21 waits until the execution result is received. During this wait, the second control unit 21 may execute other information processing.

[0076] On the other hand, if an execution result is received ("Yes" in step S12), the second control unit 21 determines the type of the received execution result (step S13). If the received execution result is "Completed" ("Completed" in step S13), the operation of the user terminal 2 returns to step S11. That is, the second control unit 21 transmits the next execution unit to be executed to the first server 3. When transmitting the next execution unit to be executed, the second control unit 21 specifies the same robot 1 (robot ID) as the robot 1 (robot ID) specified in step S11. This makes it possible to cause a specific robot 1 to execute multiple execution units.

[0077] On the other hand, if the received execution result is "abnormal", the second control unit 21 stops the execution of the application program AP (if the application program AP is being executed). After that, for example, the user may check the state of the robot 1 and perform maintenance as necessary.

[0078] When the robot 1 receives an execution result based on the acquired information, the second control unit 21 causes the display unit 25 of the user terminal 2 to display the execution result. The second control unit 21 may also cause the display unit 25 to display the execution result of "abnormal" and the execution result of "completed". Furthermore, the execution result of "abnormal" and the execution result of "completed" may be notified by other methods, such as by making the user terminal 2 emit a sound.

[0079] The above steps S11 to S13 are repeatedly executed, for example, until all the execution units written in the application program AP have been executed in sequence, or until the user has finished selecting the execution units.

[0080] <Operation of the first server> Next, the operation of the first server 3 in controlling the robot 1 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the operation of the first server 3. First, the third control unit 31 of the first server 3 judges whether or not an execution unit has been received from the user terminal 2 (step S21). If an execution unit has not been received ("No" in step S21), the third control unit 31 waits until an execution unit is received. During this wait, the third control unit 31 may execute other information processing.

[0081] When an execution unit is received ("Yes" in step S21), the third control unit 31 interprets the contents of the received execution unit and determines which robot 1 among the multiple robots 1 is to execute what operation (step S22). The third control unit 31 creates an execution command corresponding to the operation to be executed by the robot 1. The third control unit 31 also determines the robot 1 to execute the execution command.

[0082] Thereafter, the third control unit 31 refers to the robot database DB stored in the third storage unit 32 and determines whether or not the robot 1 determined to execute the execution command exists (step S23). If the identification information (robot ID) of the robot 1 is not stored in the robot database DB and it is determined that the specified robot 1 does not exist ("No" in step S23), the third control unit 31 transmits an "abnormal" message indicating that the specified robot 1 does not exist as an execution result to the user terminal 2 (step S27).

[0083] On the other hand, if the specified robot 1 exists ("Yes" in step S23), the third control unit 31 acquires the state of the specified robot 1 and judges whether the robot 1 is in a state in which it can execute the execution command (step S24). If the robot 1 cannot execute the execution command ("No" in step S24), the third control unit 31 transmits an "abnormal" message indicating that the specified robot 1 cannot execute the execution command to the user terminal 2 as an execution result (step S27).

[0084] The robot 1 cannot execute an execution command when, for example, the robot 1 is executing another execution command, or when the robot 1 is stopped due to an abnormality.

[0085] On the other hand, if the specified robot 1 is capable of executing the execution command ("Yes" in step S24), the third control unit 31 stores the execution command generated from the execution unit received in step S21 in the command storage unit 321 assigned to the specified robot 1 (step S25). As a result, the specified robot 1 executes the execution command stored in its own command storage unit 321, and performs the specific operation indicated in the execution unit.

[0086] While the robot 1 is executing the execution command stored in the command storage unit 321, the third control unit 31 judges whether or not the execution result of the execution command has been received from the robot 1 (step S26). If the execution result has not been received from the robot 1 ("No" in step S26), the third control unit 31 waits until the execution result is received. During this wait, the third control unit 31 may execute other information processing.

[0087] On the other hand, when an execution result is received from the robot 1 ("Yes" in step S26), the third control unit 31 transmits the execution result received from the robot 1 to the user terminal 2 (step S27). After transmitting any execution result to the user terminal 2, the operation of the first server 3 returns to step S21. That is, the third control unit 31 of the first server 3 executes the above steps S21 to S27 every time it receives an execution unit from the user terminal 2.

[0088] <Robot movement> Next, the operation of the robot 1 in controlling the robot 1 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation of the robot 1. The first control unit 111 of the robot 1 constantly outputs the state of the robot 1, and constantly monitors whether or not an execution command is stored in the command storage unit 321 of the first server 3 assigned to the robot 1. The robot 1 operates as follows when executing an execution unit (execution command).

[0089] First, the first control unit 111 of the robot 1 judges whether or not an execution command is stored in the command storage unit 321 assigned to the robot 1 (step S31). If no execution command is stored ("No" in step S31), the first control unit 111 waits. During this wait, the first control unit 111 may execute other information processing.

[0090] If an execution command is stored ("Yes" in step S31), the first control unit 111 executes the execution command stored in the command storage unit 321, and causes the robot 1 to perform the operation indicated in the execution command (step S32). If an abnormality occurs in the robot 1 during execution of the execution command and the robot 1 is unable to execute the execution command ("Yes" in step S33), the first control unit 111 transmits an execution result of "abnormality" indicating that an abnormality has occurred to the first server 3 (step S34).

[0091] On the other hand, if the execution of the execution command is completed without any abnormality occurring in the robot 1 ("No" in step S33) ("Yes" in step S35), the first control unit 111 transmits an execution result of "Completion" indicating that the execution command has been normally completed to the first server 3 (step S36). The execution command that has been completely executed is erased from the command storage unit 321. Depending on the execution command executed by the robot 1, an execution result based on information acquired by the robot 1 (for example, the result of identifying an object, the result of authenticating a human face, the contents of the input voice, etc.) is transmitted together with the execution result of "Completion".

[0092] Thereafter, the operation of the robot 1 returns to step S31. That is, the first control unit 111 of the robot 1 executes the above steps S31 to S36 each time an execution command is stored in its own command storage unit 321. In this way, the robot 1 does not execute a plurality of execution commands consistently to execute a predetermined task, but executes an operation according to the execution command each time the execution command is stored in the command storage unit 321 (that is, for each execution unit).

[0093] <Example of a robot performing a specific task> The above describes the operations of the robot 1, the user terminal 2, and the first server 3 when executing one execution unit (execution command). On the other hand, as described above, when the robot 1 is made to perform a predetermined task (such as serving food, security, searching for items, reception, and advertising), a combination of multiple execution units is executed. Below, an execution example when the robot 1 is made to perform a predetermined task will be described with reference to FIG. 9. FIG. 9 is a diagram showing the transmission and reception of information in the robot control system 100 when the robot 1 is made to perform a predetermined task. Below, an operation example when the robot 1 is made to perform a serving task will be described.

[0094] The food serving task is composed of a combination of an execution unit that causes robot 1 to autonomously navigate to a specified table, an execution unit that causes robot 1 to output a voice message to a customer seated at the table informing them that robot 1 has arrived, an execution unit that causes robot 1 to input the customer's voice response, and an execution unit that causes robot 1 to autonomously navigate away from the table in response to the voice input.

[0095] First, the first control unit 111 of the user terminal 2 transmits an execution unit for causing the robot 1 to autonomously travel to a specified table to the first server 3 (step S101). The first control unit 111 transmits information on the location of the specified table (for example, the coordinate value of the location) ("table position" in FIG. 9) as a target position for autonomous travel, to the first server 3, including the execution unit. In addition, the first control unit 111 transmits identification information (robot ID) of the robot 1 to perform the table setting task to the first server 3 together with the execution unit.

[0096] The third control unit 31 of the first server 3 that has received the execution unit determines whether or not the robot 1 determined to execute the food distribution task is present, and if the robot 1 is present, checks the state of the robot 1 (step S102). If the robot 1 is in a state in which it can execute the execution command, the third control unit 31 stores an execution command instructing the robot 1 to autonomously travel to a target position (table position) in the command storage unit 321 assigned to the robot 1. The first control unit 111 of the robot 1 executes the autonomous travel program P1 according to the execution command stored in its own command storage unit 321, thereby causing the robot 1 to autonomously travel to the specified table (step S103).

[0097] When the robot 1 reaches the specified table, the first control unit 111 of the robot 1 transmits an execution result indicating that the execution command for executing the autonomous traveling has been completed to the first server 3 (step S104). The third control unit 31 of the first server 3, which receives this execution result, transmits an execution result indicating that the execution unit for executing the autonomous traveling has been completed to the user terminal 2 (step S105).

[0098] Upon receiving the execution result indicating that the execution unit for executing the autonomous traveling has been completed, the first control unit 111 of the user terminal 2 transmits an execution unit for causing the robot 1 to output a voice indicating that the robot 1 has arrived (e.g., a voice saying "Sorry to keep you waiting") to the first server 3 (step S106). The first control unit 111 transmits information regarding the content of the voice to be output by the robot 1 (e.g., text data indicating the voice content) ("Voice content" in FIG. 9) to the first server 3, including the execution unit. The first control unit 111 also transmits identification information (robot ID) of the robot 1 to perform the food distribution task together with the execution unit to the first server 3.

[0099] The third control unit 31 of the first server 3 that has received the execution unit determines whether or not there is a robot 1 that has been determined to execute the food distribution task, and if there is such a robot, checks the state of the robot 1 (step S107). If the robot 1 is in a state in which it can execute the execution command, the third control unit 31 stores an execution command to output a specified sound in the command storage unit 321 assigned to the robot 1. The third control unit 31 also generates audio data (e.g., MP3 data) for emitting the sound content based on information about the sound content included in the execution unit, and stores the data in the command storage unit 321 together with the execution command.

[0100] The first control unit 111 of the robot 1 executes the voice output program P2 in accordance with the execution command stored in its own command memory unit 321, and causes the robot 1 to output the specified voice content by playing back the voice data stored in its own command memory unit 321 (step S108).

[0101] When the robot 1 outputs the voice of the designated voice content, the first control unit 111 of the robot 1 transmits an execution result indicating that the execution command to output the voice has been completed to the first server 3 (step S109). The third control unit 31 of the first server 3, which has received this execution result, transmits an execution result indicating that the execution unit for outputting the voice has been completed to the user terminal 2 (step S110).

[0102] Upon receiving the execution result indicating that the execution unit for outputting the voice has been completed, the first control unit 111 of the user terminal 2 transmits an execution unit for inputting voices around the robot 1 (e.g., customers) to the robot 1 and outputting the voice contents to the first server 3 (step S111). The first control unit 111 transmits the identification information (robot ID) of the robot 1 performing the food serving task together with this execution unit to the first server 3.

[0103] The third control unit 31 of the first server 3 that has received the execution unit determines whether or not the robot 1 determined to execute the food distribution task is present, and if the robot 1 is present, checks the state of the robot 1 (step S112). If the robot 1 is in a state in which it can execute the execution command, the third control unit 31 stores an execution command in the command storage unit 321 assigned to the robot 1, the execution command instructing the robot 1 to input surrounding voices and output the voice contents.

[0104] The first control unit 111 of the robot 1 executes the voice input program P3 in accordance with the execution command stored in its own command memory unit 321, thereby inputting voices around the robot 1 (e.g., customers) and generating data in a predetermined format (e.g., text data) indicating the content of the input voice (voice contents) (step S113).

[0105] When the robot 1 inputs the voice and generates data indicating the voice content, the first control unit 111 of the robot 1 transmits an execution result indicating that the execution command to input the voice and output the voice content has been completed to the first server 3 (step S114). The first control unit 111 transmits data indicating the input voice content ("voice data" in FIG. 9) together with this execution result to the first server 3. The third control unit 31 of the first server 3, which receives this execution result and the data indicating the voice content, transmits the received execution result and data indicating the voice content to the user terminal 2 (step S115).

[0106] When an execution result indicating that the execution unit of the voice input is completed is received and the input voice data means a reply such as "thank you", the first control unit 111 of the user terminal 2 transmits an execution unit for autonomously driving the robot 1 from the table to a predetermined position (e.g., the waiting position of the robot 1) to the first server 3 (step S116). The first control unit 111 transmits information about the specified predetermined position (e.g., the coordinate value of the predetermined position) as a target position for autonomous driving to the first server 3, including it in the execution unit. The first control unit 111 also transmits identification information (robot ID) of the robot 1 to perform the food distribution task together with this execution unit to the first server 3.

[0107] The third control unit 31 of the first server 3 that has received the execution unit determines whether or not the robot 1 determined to execute the food distribution task is present, and if the robot 1 is present, checks the state of the robot 1 (step S117). If the robot 1 is in a state in which it can execute the execution command, the third control unit 31 stores an execution command instructing the robot 1 to leave the table and autonomously travel to a target position (predetermined position) in the command storage unit 321 assigned to the robot 1. The first control unit 111 of the robot 1 executes the autonomous travel program P1 according to the execution command stored in its own command storage unit 321, thereby causing the robot 1 to autonomously travel from the table to a specified predetermined position (step S118).

[0108] When the robot 1 reaches the designated position, the first control unit 111 of the robot 1 transmits an execution result indicating that the execution command for executing the autonomous traveling has been completed to the first server 3 (step S119). The third control unit 31 of the first server 3, which receives this execution result, transmits an execution result indicating that the execution unit for executing the autonomous traveling has been completed to the user terminal 2 (step S120).

[0109] In this way, in the robot control system 100, when the robot 1 is made to perform a task consisting of a combination of multiple execution units, the robot 1 is made to perform an execution unit corresponding to a specific operation included in the task, and the execution result of the execution unit by the robot 1 is received by the user terminal 2. Next, the user terminal 2 makes the robot 1 execute an execution unit corresponding to the next operation according to the execution result of the received execution unit. In other words, in the robot control system 100, the robot 1 is controlled in an "interactive format" in which the execution units and the execution results of the execution units are exchanged between the robot 1 and the user terminal 2. Then, the robot 1 is made to perform a predetermined task by repeatedly executing the "interaction" between the execution units and the execution results of the execution units.

[0110] By controlling the robot in an "interactive" manner, it is possible to realize flexible control of the robot 1 according to the results of execution of a specific action during a given task. For example, as described above, it is possible to execute control such that after it is confirmed that execution of a certain execution unit has been completed normally, the next execution unit is executed.

[0111] In addition, by being able to check the execution results of the execution units, it is possible to perform the following operations, for example: When the robot 1 is made to perform a predetermined task by executing an application program AP, if the execution result obtained by executing a certain execution unit is different from that expected by the application program AP (for example, if the presence of a person different from that expected is confirmed, or if there is a voice input different from that expected), the execution of the application program AP can be stopped and an execution unit according to the obtained execution result can be selected on the user terminal 2.

[0112] Specifically, for example, when robot 1 is made to perform a serving task, if the execution results of object identification and face authentication show that it is the birthday of the person to whom the food is to be served, robot 1 can be made to execute an execution unit that causes robot 1 to output a voice message celebrating the birthday of the person before an execution unit that causes robot 1 to autonomously drive from the table to a specified position.

[0113] Also, for example, when robot 1 is made to perform security work by photographing the area around robot 1 and issuing a warning if a suspicious person is included in the photographed image, if the result of photographing the image shows that not only a suspicious person but also a security guard is present around robot 1, instead of making robot 1 execute an execution unit of outputting an audio warning to the suspicious person, robot 1 can be made to execute an execution unit of notifying the security guard of the presence of the suspicious person and requesting him to remove the suspicious person.

[0114] [Variations] Although the embodiments of the present invention have been described above, the present invention is not limited to these, and various modifications are possible without departing from the spirit of the present invention.

[0115] (a) The first server 3 may provide an API other than the APIs described above.

[0116] (b) In the robot control system 100 of the above embodiment, the application program AP is stored in the user terminal 2, and the application program AP is executed by the user terminal 2. Alternatively, the application program AP is stored in the second server 4, and the application program AP is executed on the second server 4 by operating the user terminal 2. However, this is not limited to the above.

[0117] For example, the application program AP may be stored in the first server 3, and the application program AP may be executed on the first server 3. Alternatively, the application program AP may be stored in one of the multiple robots 1, and the application program AP may be executed by that robot 1.

[0118] (c) The robot control system 100 may have a function of converting an application program AP created using a program creation GUI to generate source code for the application program AP. Also, the robot control system 100 may have a function of replacing the application program AP with commands in a specific language.

[0119] (d) In the robot control system 100, a predetermined task can be executed by a plurality of robots 1 working together by appropriately selecting a combination of a plurality of execution units and a robot 1 that executes each execution unit.

[0120] (e) The API function provided by the first server 3 and each function (program that realizes the function) of the robot 1 may be automatically updateable. In addition, when a new device or the like is attached to the robot 1, the API (API program IP) for operating the device or the like and the program on the robot 1 side may be automatically updated / installed.

[0121] (f) A run unit may consist of only one API request, and a run unit may contain no API requests.

[0122] (g) Commands from the user terminal 2 to the first server 3 and commands from the first server 3 to the robot 1 may be generated by means of functions other than the API.

[0123] <Features of the embodiment> The above embodiment can also be described as follows.

[0124] (1) A robot control system (e.g., robot control system 100) includes a robot (e.g., robot 1) and a first computer (e.g., user terminal 2). The first computer causes the robot to execute a predetermined task composed of a first execution unit and a second execution unit. In this system, the first computer commands the robot to execute the first execution unit, receives a result of execution of the first execution unit by the robot from the robot, and commands the robot to execute the second execution unit according to the result of execution of the first execution unit.

[0125] According to this configuration, the robot is caused to execute a first execution unit corresponding to a specific operation performed in a predetermined task, and a result of execution of the first execution unit by the robot is received. Next, in accordance with the received execution result of the first execution unit, the robot is caused to execute a second execution unit corresponding to the next operation in the predetermined task. In this way, the execution result of the first execution unit is received, and the second execution unit to be executed next is executed in accordance with the execution result of the first execution unit, thereby realizing flexible control of the robot in accordance with the execution result of the specific operation during the predetermined task.

[0126] Furthermore, according to the above configuration, a predetermined task to be executed by the robot can be realized by combining the first execution unit and the second execution unit. In other words, a user can cause the robot to execute any task simply by programming any combination of multiple execution units. Furthermore, a task to be executed by the robot can be changed simply by changing the combination of execution units. As a result, the workload and cost associated with changing the task to be executed by the robot can be reduced.

[0127] (2) In the robot control system of (1) above, the first execution unit and / or the second execution unit may be a program unit including at least one API request that requests the robot to execute a predetermined function. With this configuration, even a user without advanced programming knowledge can easily create an application that causes the robot to execute the first execution unit and the second execution unit.

[0128] (3) The robot control system of (1) or (2) above may further include a second computer (e.g., the first server 3). The second computer has a command storage unit (e.g., the command storage unit 321) that stores an execution command for the robot corresponding to an execution unit to be executed by the robot. In this case, when the first computer commands the robot to execute the first execution unit or the second execution unit, the first computer may transmit the execution unit to be executed by the robot to the second computer, the second computer may store the execution command corresponding to the execution unit received from the first computer in the command storage unit, and the robot may execute the execution command stored in the command storage unit. According to this configuration, the robot can be made to execute only the execution command stored in the command storage unit. As a result, the robot can be prevented from executing other actions that have not been commanded.

[0129] (4) In the robot control system of (3) above, the second computer may acquire the state of the robot, and when the robot is in a state in which the execution command can be executed, store the execution command in the command storage unit. With this configuration, it is possible to reliably cause the robot to execute the first execution unit and / or the second execution unit.

[0130] (5) In any of the robot control systems (1) to (4) above, there may be a plurality of robots. In this case, the first computer may specify a robot to execute the first run unit or the second run unit when issuing a command to execute the first run unit or the second run unit. With this configuration, it is possible to select an optimal robot to execute the first run unit or the second run unit.

[0131] (6) A method for controlling a robot, which causes a robot (for example, robot 1) to execute a predetermined task constituted by a first execution unit and a second execution unit, comprises the following steps. A step of instructing the robot to execute a first execution unit (for example, step S11, step S101, S106, S111). A step of receiving, from the robot, a result of execution of the first execution unit by the robot (for example, steps S12 to S13, steps S104 to S105, S109 to S110, and S114 to S115). A step of instructing the robot to execute a second execution unit in response to receiving the execution result of the first execution unit (for example, steps S11, S106, S111, S116).

[0132] According to this configuration, the robot is caused to execute a first execution unit corresponding to a specific operation performed in a predetermined task, and a result of execution of the first execution unit by the robot is received. Next, in accordance with the received execution result of the first execution unit, the robot is caused to execute a second execution unit corresponding to the next operation in the predetermined task. In this way, the execution result of the first execution unit is received, and the second execution unit to be executed next is executed in accordance with the execution result of the first execution unit, thereby realizing flexible control of the robot in accordance with the execution result of the specific operation during the predetermined task.

[0133] Furthermore, according to the above configuration, a predetermined task to be executed by the robot can be realized by combining the first execution unit and the second execution unit. In other words, a user can cause the robot to execute any task simply by programming any combination of multiple execution units. Furthermore, a task to be executed by the robot can be changed simply by changing the combination of execution units. As a result, the workload and cost associated with changing the task to be executed by the robot can be reduced. [Explanation of symbols]

[0134] 100: Robot control system 1:Robot 11: First control device 111: First control section 113: 1st memory section P1: Autonomous driving program P2: Audio output program P3: Voice input program P4: Object discrimination program P5: Face recognition program 12: Running part 13: First Communication Department 14: Sensor 15: Camera 16: Audio output section 17: Audio input section 2: User terminal 21: Second control section 22:Second memory section AP: Application Program 23: Second Communication Department 24: Input section 25:Display section 3: First server 31: Third control section 32:Third memory section CP: Control Program IP:API Program 321: Command storage unit 33: Third Communications Department 4: Second server 41: 4th control section 42: 4th memory section 43: 4th Communications Department 9: Network

Claims

1. Robots and a first computer that causes the robot to execute a predetermined task constituted by a first execution unit and a second execution unit; Equipped with The first computer commanding the robot to execute the first execution unit; receiving, from the robot, a result of execution of the first execution unit by the robot; instructing the robot to execute the second execution unit according to a result of the execution of the first execution unit; Robot control system.

2. The robot control system according to claim 1 , wherein the first execution unit and / or the second execution unit is a program unit including at least one API request requesting the robot to execute a predetermined function.

3. a second computer having a command storage unit that stores execution commands for the robot corresponding to execution units to be executed by the robot; The first computer transmits an execution unit to be executed by the robot to the second computer when instructing the robot to execute the first execution unit or the second execution unit; the second computer stores in the command storage unit an execution command corresponding to the execution unit received from the first computer; The robot executes the execution command stored in the command storage unit. The robot control system of claim 1 .

4. The second computer is Acquire the state of the robot; When the robot is in a state capable of executing the execution command, the execution command is stored in the command storage unit. The robot control system of claim 3.

5. The robot is present in a plurality of locations, The robot control system according to claim 1 , wherein the first computer, when issuing a command to execute the first execution unit or the second execution unit, specifies a robot to be caused to execute the first execution unit or the second execution unit.

6. A method for controlling a robot to execute a predetermined task constituted by a first execution unit and a second execution unit, comprising: commanding the robot to execute the first execution unit; receiving, from the robot, a result of execution of the first execution unit by the robot; instructing the robot to execute a second execution unit in response to receiving a result of the execution of the first execution unit; A control method comprising:

7. A program for causing a computer to execute the control method according to claim 6.

Images