Operation program generation system

Abstract

This operation program generation system 100 comprises: a computer 20 that receives input of task information 32a for execution of a task to be performed by an operation of a robot 10; and a server 30 that generates, for causing the robot 10 to execute the task of the task information 32a input by the computer 20, an operation program 11 for the robot 10 on the basis of setting at least one piece of skill information 32b selected from a plurality of pieces of skill information 32b which are stored in advance and which include elements of the operation program 11.

Description

Operation program generation system

[0001] This disclosure relates to an operating program generation system.

[0002] Conventionally, robot operation program generation systems have been disclosed. For example, Japanese Patent No. 7278246 describes an operation program generation system that includes a robot control device that receives input of information on a task to be performed by a robot via a terminal device, and selects and sets a program from a library that defines a series of operations for the robot to perform the task based on the task information.

[0003] Patent No. 7278246

[0004] In the operation program generation system disclosed in Japanese Patent No. 7278246, a robot control device selects and sets a program from a library that defines a series of operations that cause the robot to perform a task. However, even for the same task, different robots require different programs. Therefore, simply selecting a program from a library that defines a series of operations that cause the robot to perform a task and applying it as is may result in the robot not operating accurately. Therefore, there is a need for a method for easily and accurately generating operation programs for robots.

[0005] This disclosure has been made to solve the above-mentioned problems, and one purpose of this disclosure is to provide an operation program generation system that can generate an operation program for a robot easily and with high accuracy.

[0006] An operation program generation system according to one aspect of the present disclosure includes a task receiving unit that receives input of task information for performing a task by the operation of a robot, and an operation program generation unit that generates an operation program for the robot based on setting at least one piece of skill information selected from a plurality of pieces of skill information that are pre-stored and include elements of an operation program, in order to have the robot perform the task of the task information input by the task receiving unit.

[0007] As described above, an operation program generation system according to one aspect of the present disclosure includes an operation program generation unit that generates an operation program for a robot by setting at least one piece of skill information selected from a plurality of pieces of skill information that are pre-stored and contain elements of an operation program in order to have the robot execute the task of the task information input by the task receiving unit. This allows the robot to operate accurately by selecting, from the plurality of pieces of skill information, skill information containing elements of an operation program suitable for the robot that will execute the task, unlike when an operation program is directly generated from task information for a task to be performed by the robot's operation. Furthermore, even a user who is unable to directly generate a robot operation program can easily generate an operation program because the operation program is generated by inputting a task. These features allow the robot operation program to be generated easily and with high accuracy.

[0008] The operation program generation system of the present disclosure can generate an operation program for a robot easily and with high accuracy.

[0009] FIG. 1 is a block diagram showing an operation program generation system according to a first embodiment. FIG. 2 is a diagram for explaining an example of creation of a new task set by the operation program generation system according to the first embodiment. FIG. 3 is a diagram for explaining a first example of creation of an operation program by the operation program generation system according to the first embodiment. FIG. 4 is a diagram for explaining a second example of creation of an operation program by the operation program generation system according to the first embodiment. FIG. 5 is a diagram showing an example of skills selected when creating an operation program by the operation program generation system according to the first embodiment. FIG. 6 is a diagram showing an example of robots selected when creating an operation program by the operation program generation system according to the first embodiment. FIG. 7 is a diagram showing an example of an actual robot used in the operation program generation system according to the first embodiment. FIG. 8 is a diagram showing an example of a robot used in a simulation of the operation program generation system according to the first embodiment. FIG. 9 is a diagram showing an example of a scenario for a simulation of an operation program by the operation program generation system according to the first embodiment. FIG. 10 is a diagram showing an example of evaluation of a simulation of an operation program by the operation program generation system according to the first embodiment. FIG. 11 is a diagram showing an example of skills selected based on evaluation when creating an operation program by the operation program generation system according to the first embodiment. FIG. 12 is a block diagram showing an operation program generation system according to a second embodiment. FIG. 13 is a diagram showing a third example of creation of an operation program by the operation program generation system according to the second embodiment. Fig. 10 is a diagram for explaining activities in a third example of operation program creation by the operation program generation system according to the second embodiment. Fig. 11 is a diagram for explaining other activities in the third example of operation program creation by the operation program generation system according to the second embodiment. Fig. 12 is a diagram for explaining a fourth example of operation program creation by the operation program generation system according to the second embodiment. Fig. 13 is a diagram showing an example of a setting screen in the fourth example of operation program creation by the operation program generation system according to the second embodiment.10 is a diagram showing examples of skills and robots that are set when an operation program is created by the operation program generation system according to the second embodiment. FIG. 11 is a diagram showing another example setting screen for a fourth example of operation program creation by the operation program generation system according to the second embodiment. FIG. 12 is a diagram showing examples of skills and robots that are set when an operation program is created by the operation program generation system according to the second embodiment. FIG. 13 is a diagram for explaining a fifth example of operation program creation by the operation program generation system according to the second embodiment. FIG. 14 is a diagram for explaining an example of spatial position information used in an operation program by the operation program generation system according to the second embodiment. FIG. 15 is a diagram for explaining a sixth example of operation program creation by the operation program generation system according to the second embodiment. FIG. 16 is a diagram for explaining spatial position information stored in a server of the operation program generation system according to the second embodiment. FIG. 17 is a block diagram showing an operation program generation system according to the third embodiment. FIG. 18 is a diagram for explaining a first example of operation of a robot in the operation program generation system according to the third embodiment. FIG. 19 is a diagram for explaining an operation program in the first example of operation of the operation program generation system according to the third embodiment. FIG. 19 is a diagram for explaining skill definitions used in an operation program in the first example of operation of the operation program generation system according to the third embodiment. Fig. 10 is a diagram for explaining a second operation example of a robot of the operation program generation system according to the third embodiment. Fig. 11 is a diagram for explaining an operation program of the second operation example of the operation program generation system according to the third embodiment. Fig. 12 is a diagram for explaining skill definitions used in the operation program of the second operation example of the operation program generation system according to the third embodiment.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] First Embodiment As shown in FIG. 1, an operation program generation system 100 is a system that generates an operation program 11 for a robot 10 .

[0012] As shown in FIG. 1, the operation program generation system 100 is a system that allows a user of a robot 10 to specify a task to be performed by the operation of the robot 10 and obtain an operation program 11 that operates the robot 10.

[0013] As shown in FIG. 1 , the operation program generation system 100 includes a computer 20 and a server 30 connected to the computer 20 via a network. The operation program generation system 100 also includes a plurality of computers 40 connected to the server 30 via the network. The operation program generation system 100 also includes a plurality of computers 50 connected to the server 30 via the network. The operation program generation system 100 also includes a plurality of computers 60 connected to the server 30 via the network. The computer 20 is an example of a "task receiving unit" in the present disclosure. The server 30 is an example of an "operation program generation unit," "first server," "second server," and "simulation execution unit" in the present disclosure.

[0014] As shown in FIG. 1 , the robot 10 includes a social robot that provides services to people. The robot 10 includes, for example, a robot with one or more articulated arms, a robot that can move autonomously using wheels, a robot that can walk using multiple legs, a drone robot that can fly, and a robot with a head. The robot 10 provides services such as customer service, product delivery, cleaning, security, and guidance in commercial facilities. The robot 10 also provides services such as assisting patients or residents in hospitals and nursing homes.

[0015] 1 , the computer 20 is used by a user who uses the robot 10. The computer 20 is, for example, a personal computer. The computer 20 includes a control unit 21, a display unit 22, and an input unit 23. The computer 20 is used, for example, to control the operation of the robot 10. Note that a computer that controls the operation of the robot 10 may be provided separately from the computer 20.

[0016] The control unit 21 includes a processor such as a CPU (Central Processing Unit) and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 21 also executes programs to perform processing.

[0017] The display unit 22 displays information and includes a display such as an organic EL display or a liquid crystal display.

[0018] The input unit 23 includes, for example, devices such as a keyboard and a mouse. The input unit 23 may also be a touch panel provided on the display unit 22.

[0019] The computer 20 accepts input of task information 32a for operating the robot 10 to execute a task. That is, the computer 20 accepts input of task information 32a for executing a task to be performed by the robot 10 from a user who uses the robot 10. The computer 20 also stores information about the robot 10 used by the user.

[0020] The information about the robot 10 is used to select the robot 10 to be operated and to select skills. In addition, when a task requires multiple robots 10 with skills to achieve subtasks, the information about the robot 10 is used when creating an operation program 11 for the multiple robots 10 to cooperate in executing the task.

[0021] Here, the tasks in the task information input to the computer 20 are expressed in terms that can be understood even by a user unfamiliar with the program of the robot 10. For example, as shown in Fig. 2, the tasks include luggage transport 201, luggage grasping 202, luggage placement 203, nighttime surveillance 211, lights on / off 212, warning people 213, door opening / closing 221, identifier reading 222, curtain opening / closing 223, dialogue instruction 231, luggage transport 232, lights on / off 233, curtain opening / closing 234, etc. Furthermore, a task is a unit of work that indicates what work the robot 10 will perform. Furthermore, a task does not depend on the function of the robot 10.

[0022] As shown in Fig. 2, tasks are combined to create a task set. The created task set is then made public and available to multiple users. As shown in Fig. 2, a user can create a new task set by combining tasks in a published task set. The published task set is stored, for example, in the server 30. A task set may be created by a user who uses the robot 10, or by a developer of the task set.

[0023] 2, a cafe food delivery task set 200 is published as a task set. The cafe food delivery task set 200 includes a baggage carrying task 201, a baggage grasping task 202, and a baggage placement task 203 as tasks.

[0024] 2, a station night security task set 210 is published as a task set. The station night security task set 210 includes night surveillance 211, lighting on / off 212, and warning people 213 as tasks.

[0025] 2, a hospital delivery task set 220 is published as a task set. The hospital delivery task set 220 includes a door opening / closing 221, an identifier reading 222, and a curtain opening / closing 223 as tasks.

[0026] 2, a new task set is created by combining tasks available from publicly available task sets, such as a task set 230 for assisting with getting dressed at a care facility.

[0027] In the example shown in Figure 2, the task set 230 for assisting with changing clothes at a nursing home combines a task for giving dialogue instructions 231, a task for carrying luggage 232 copied from the cafe food delivery task set 200, a task for turning lights on and off 233 copied from the station night security task set 210, and a task for opening and closing curtains 234 copied from the hospital delivery task set 220.

[0028] 1 , the server 30 is connected to the computer 20 via a network. The server 30 generates an operation program 11 for the robot 10. The server 30 includes a control unit 31 and a storage unit 32.

[0029] The control unit 31 includes a processor such as a CPU, and memories such as a ROM and a RAM, and also executes programs to perform processing.

[0030] The storage unit 32 stores information. For example, the storage unit 32 stores information uploaded to the server 30. The storage unit 32 stores information including a plurality of pieces of task information 32a, a plurality of pieces of skill information 32b, and a plurality of simulation programs 32c. The storage unit 32 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0031] As shown in FIG. 1 , the computer 40 is connected to the server 30 via a network. The computer 40 is used by a skill creator who creates skill information 32b. The computer 40 is, for example, a personal computer. The computer 40 creates the skill information 32b through operation by the skill creator. The computer 40 transmits the created skill information 32b to the server 30 through operation by the skill creator. Note that the computer 40 may also be a mobile terminal such as a tablet or smartphone.

[0032] Skill information 32b created in computer 40 and sent to server 30 may be made public, may be made public to a limited audience, or may be kept private, at the option of the skill creator.

[0033] As shown in Fig. 1, a computer 50 is connected to a server 30 via a network. The computer 50 is used by a simulation creator who creates a simulation program 32c that performs a simulation using an operation program 11. The computer 50 is, for example, a personal computer. The computer 50 creates the simulation program 32c through operation by the simulation creator. The computer 50 transmits the created simulation program 32c to the server 30 through operation by the simulation creator.

[0034] The simulation program 32c created in the computer 50 and transmitted to the server 30 may be made public, may be made public to a limited audience, or may be kept private, depending on the choice of the simulation creator.

[0035] As shown in Fig. 1, the computer 60 is connected to the server 30 via a network. The computer 60 is used by a robot creator to create elements of the robot. The computer 60 is, for example, a personal computer. The computer 60 creates the elements of the robot 10 through operation by the robot creator. The computer 60 transmits the created elements of the robot 10 to the server 30 through operation by the robot creator.

[0036] The elements of the robot 10 include, for example, device drivers and mechanical design information of the robot 10 .

[0037] The elements of the robot 10 created in the computer 60 and transmitted to the server 30 may be made public, may be made public to a limited audience, or may be kept private, at the option of the robot creator.

[0038] In the first embodiment, the server 30 generates the operation program 11 for the robot 10 by setting at least one skill information 32b selected from a plurality of skill information 32b that are stored in advance and include elements of the operation program 11, in order to have the robot 10 execute the task of the task information 32a input by the computer 20.

[0039] Specifically, the user operates the computer 20 to select task information 32a of the server 30. At least one piece of skill information 32b is set for the selected task information 32a by the server 30. Then, based on the skill information 32b being set for the task information 32a, the operation program 11 of the robot 10 is generated by the server 30.

[0040] The server 30 generates the operation program 11 based on setting at least one piece of skill information 32b from a plurality of pieces of skill information 32b suited to the type of robot 10 and the type of task.

[0041] A plurality of pieces of skill information 32b created by a plurality of skill creators are stored in the server 30. The server 30 then generates the operation program 11 by setting at least one piece of skill information 32b selected from the plurality of pieces of skill information 32b stored in the server 30.

[0042] The skill information 32b can be set for a plurality of different task information 32a. For example, as shown in Figures 3 and 4, the skill of finding something can be set for subtasks 241a and 241f in task 241 of going from the first floor to the third floor by elevator in Figure 3. Also, the skill of finding something can be set for subtasks 251a and 251c in task 251 of turning off the lights in Figure 4.

[0043] A subtask is a task broken down into its smallest unit. In other words, a task is made up of a combination of multiple subtasks.

[0044] To execute a task, skills are set. Specifically, skills are means for executing subtasks. Skills depend on the robot's functions.

[0045] Modules are reusable abstractions of tasks, subtasks, and skills. For example, the tasks that instantiate the task module "push" are "press button" and "push work."

[0046] 3 , for example, a task 241 for going from the first floor to the third floor by elevator is set with a plurality of pieces of skill information 32b. Specifically, for task 241 for going from the first floor to the third floor by elevator, a plurality of pieces of skill information 32b for executing a subtask 241a for finding the elevator, a subtask 241b for moving in front of the elevator, a subtask 241c for finding an upper floor button on the elevator, a subtask 241d for pressing an upper floor button, and a subtask 241e for moving into the elevator are set with the robot 10 before getting into the elevator. Furthermore, for task 241 for going from the first floor to the third floor by elevator, a plurality of pieces of skill information 32b for executing a subtask 241f for finding the third floor button, a subtask 241g for pressing the third floor button, a subtask 241h for waiting when the elevator doors are closing, and a subtask 241i for moving out of the elevator when the elevator doors are opening are set with the robot 10 before getting into the elevator.

[0047] 4, for example, a plurality of pieces of skill information 32b are set for task 251 of turning off the lights. Specifically, for task 251 of turning off the lights, a plurality of pieces of skill information 32b are set for executing subtask 251a of finding the light switch, subtask 251b of moving in front of the light switch, subtask 251c of finding the switch toggle of the light switch, and subtask 251d of stretching the arm from top to bottom.

[0048] It is possible to set multiple skills for the same action. For example, as shown in Figure 5, the skill for executing subtask 241d of pressing the upstairs button can be selected from skills 241da, 241db, 241dc, and 241dd.

[0049] Skill 241da includes elements of an operation program related to arm position control and a force sensor.

[0050] The skill 241db includes elements of an operation program related to arm torque control.

[0051] The skill 241dc includes elements of an operation program related to arm position control.

[0052] Skill 241dd includes elements of a movement program related to leg movement.

[0053] It is possible to select multiple robots 10 to execute a task. For example, as shown in Fig. 6, it is possible to select a robot 10a as robot A, a robot 10b as robot B, and a robot 10c as robot C.

[0054] The robot 10a includes two position-controllable arms, a camera, a force sensor, and legs for movement. The robot 10a includes an arm position control module, a camera module, a force sensor module, and a leg movement module.

[0055] The robot 10b includes two torque-controllable and position-controllable arms, a camera, and wheels for movement. The robot 10b includes an arm position control module, an arm torque control module, a camera module, and a wheeled movement module.

[0056] The robot 10c includes one arm whose position can be controlled, a camera, and wheels for movement. The robot 10c includes an arm position control module, a camera module, and a module for movement by the wheels.

[0057] The server 30 generates an operation program 11 for the robot 10 by combining and setting the skill information 32b and the robot 10 from the selected task information 32a. That is, the server 30 generates the operation program 11 based on setting the skill information 32b selected from a plurality of pieces of skill information 32b stored in advance. The server 30 also generates the operation program 11 based on setting the robot 10 selected from a plurality of robots 10 stored in advance.

[0058] The server 30 uses the operation program 11 generated with the skill information 32b set to perform a simulation of the robot 10 executing a task. The server 30 evaluates the set skill information 32b from the simulation results.

[0059] The server 30 performs a simulation of the robot executing a task by setting the interface for applying skill information to the robot 10 to be the same as the interface of the actual robot 10. For example, the interface of the actual robot C shown in Fig. 7 is set to be the same as the interface of the robot C used in the simulation shown in Fig. 8.

[0060] Specifically, the server 30 stores a plurality of simulation programs 32c created by a plurality of simulation creators. The server 30 then performs a simulation in which the robot 10 executes a task using the operation program 11 generated by setting the skill information 32b and the simulation program 32c stored in the server 30. The server 30 displays an evaluation value of the skill information 32b evaluated by the simulation on the display unit 22 of the computer 20.

[0061] When the evaluation values ​​of the skill information 32b are displayed on the display unit 22, the user inputs a selection of the skill information 32b to be set for the task information 32a to the computer 20. Furthermore, the computer 20 may select the skill information 32b with the highest evaluation value based on the evaluation values ​​of the skill information 32b.

[0062] The server 30 performs a simulation of the robot 10 executing the task, and evaluates the robot 10 with respect to the task information 32a. The server 30 displays the evaluation value of the robot 10 evaluated by the simulation on the display unit 22 of the computer 20.

[0063] When the evaluation values ​​of the robots 10 are displayed on the display unit 22, the user inputs the selection of the robot 10 to be set for the task information 32a to the computer 20. Furthermore, the computer 20 may select the robot 10 with the highest evaluation value based on the evaluation values ​​of the robots 10. Furthermore, if the robot 10 is predetermined, the computer 20 may obtain the robot 10 with the highest evaluation value from the server 30 and automatically set it.

[0064] For example, the server 30 simulates the operation program 11 using a scenario such as that shown in FIG.

[0065] 9, a hospital with an elevator is set as the environment. In the simulation scenario, robots A, B, and C are set as robots for which operation programs 11 are to be generated. In the simulation scenario, going from the first floor to the third floor is set as the task. In the simulation scenario, success or failure of the task, working time, etc. are set as evaluation indices.

[0066] In a simulation performed using the example scenario shown in Fig. 9, an evaluation is performed as shown in Fig. 10. Fig. 10 shows an example of evaluation of the skill of executing the subtask 241d of pressing the upstairs button.

[0067] In scenario (1), skill 241dc, which includes elements of an operation program related to arm position control, is selected as the skill for executing the pressing subtask 241d. Robot C is also selected. In scenario (1), the task is evaluated as failed because the button is broken. In this case, the evaluation is "X."

[0068] In scenario (2), skill 241da, which includes elements of an operation program related to arm position control and a force sensor, is selected as the skill that executes the pressing subtask 241d. Robot A is also selected. In scenario (2), it is evaluated that pressing the button is successful. In scenario (2), it is also evaluated that the task time is 100 seconds. In this case, it is evaluated as "△".

[0069] In scenario (3), skill 241db, which includes elements of an operation program related to arm torque control, is selected as the skill for executing pressing subtask 241d. Robot B is also selected. In scenario (3), it is evaluated that pressing the button is successful. In scenario (3), it is also evaluated that the task time is 50 seconds. In this case, it is evaluated as "Good."

[0070] As shown in the example of Fig. 11, subtask 241d is selected based on the evaluation. For example, in the example of Fig. 11, among skills 241da, 241db, and 241dc, skill 241db, which includes elements of an operation program related to arm torque control, is highly evaluated, and therefore skill 241db is selected as the skill to execute pushing subtask 241d.

[0071] Based on the evaluation of the simulation, the developer developing the modules of the robot 10 may consider improving the robot C by adding an arm torque control module, as in the example shown in FIG.

[0072] Furthermore, based on the evaluation of the simulation, the skill creator who creates the skill may consider creating a skill that will speed up the task.

[0073] Effect of the First Embodiment The operation program generation system 100 includes a server 30 that generates an operation program 11 for the robot 10 by setting at least one piece of skill information 32b selected from a plurality of pieces of skill information 32b that are pre-stored and contain elements of the operation program 11, in order to have the robot 10 execute a task represented by task information 32a input by the computer 20. This allows the operation program 11 to be generated by selecting, from the plurality of pieces of skill information 32b, skill information 32b containing elements of the operation program 11 suitable for the robot 10 that will execute the task, unlike when the operation program 11 is generated directly from the task information 32a of a task that the robot 10 will perform. This allows the robot 10 to operate with high accuracy. Furthermore, even a user who is unable to directly generate the operation program 11 for the robot 10 can easily generate the operation program 11 because the operation program 11 is generated by inputting a task. These features allow the operation program 11 for the robot 10 to be generated easily and with high accuracy.

[0074] The server 30 generates the operation program 11 by setting at least one piece of skill information 32b from a plurality of pieces of skill information 32b suited to the type of robot 10 and the type of task. This allows the skill information 32b suited to the robot 10 that is to execute a task to be set according to the type of robot 10 and the type of task, making it possible to generate with high accuracy operation programs 11 suited to the type of robot 10 for a variety of tasks.

[0075] The operation program generation system 100 includes a server 30 that stores a plurality of pieces of skill information 32b created by a plurality of skill creators. The server 30 generates an operation program 11 based on setting at least one piece of skill information 32b selected from the plurality of pieces of skill information 32b stored in the server 30. This allows the plurality of pieces of skill information 32b created by a plurality of skill creators to be set when generating the operation program 11, making it possible to easily set skill information 32b corresponding to a variety of tasks and a variety of robots 10 and generate the operation program 11.

[0076] The skill information 32b can be set for a plurality of different task information 32a, so that common skill information 32b can be used for a plurality of tasks, and therefore there is no need to prepare dedicated skill information 32b for each type of task.

[0077] The operation program generation system 100 includes a server 30 that performs a simulation of the robot 10 executing a task using an operation program 11 generated with skill information 32b set therein, and evaluates the set skill information 32b from the simulation results. As a result, the skill information 32b is evaluated from the results of the generated operation program 11, and it is possible to select highly evaluated skill information 32b from the plurality of pieces of skill information 32b and generate an operation program 11 that causes the robot 10 to operate with higher accuracy.

[0078] The operation program generation system 100 includes a server 30 that stores a plurality of simulation programs 32c created by a plurality of simulation creators. The server 30 performs a simulation of a robot 10 executing a task using an operation program 11 generated with skill information 32b set therein and the simulation program 32c stored in the server 30. This allows verification of the generated operation program 11 using a simulation program 32c selected from the plurality of simulation programs 32c created by the plurality of simulation creators.

[0079] The evaluation value of the skill information 32b evaluated by the server 30 is displayed on the display unit 22 of the computer 20. The computer 20 accepts an input of a selection of skill information 32b to be set for the task information 32a. This allows the user to easily select skill information 32b with a high evaluation value, making it possible to easily generate an operation program 11 that causes the robot 10 to operate with high precision.

[0080] The server 30 generates an operation program 11 by setting a robot 10 selected from a plurality of robots 10 stored in advance. The operation program generation system 100 includes the server 30 that performs a simulation of the robot 10 executing a task using the operation program 11 generated by setting the robot 10, and evaluates the robot 10 with respect to the task information 32a. As a result, the robot 10 is evaluated with respect to the task, making it possible to easily select a robot 10 suitable for executing the task from the plurality of robots 10.

[0081] The evaluation value of the robot 10 evaluated by the server 30 is displayed on the display unit 22 of the computer 20. The computer 20 receives an input of the selection of the robot 10 to be set for the task information 32a. This allows the user to easily select the robot 10 with the high evaluation value.

[0082] The server 30 sets the interface for applying the skill information 32b to the robot 10 to be the same as the interface of the actual robot 10, and performs a simulation of the robot 10 executing a task. This makes it possible to align the interfaces between the simulation and the actual robot 10, so that the operation program 11 can be executed without distinguishing between the skills of the simulation and the actual robot 10.

[0083] The robot 10 includes a social robot that provides services to people. This allows the operation program 11 for the social robot that provides services to people to be generated easily and with high accuracy.

[0084] Second Embodiment The configuration of an operation program generation system 300 according to a second embodiment of the present invention will be described with reference to Figures 13 to 26. Note that components similar to those in the first embodiment are given the same reference numerals and descriptions thereof will be omitted. As shown in Figure 13, the operation program generation system 300 is a system that generates an operation program 11 for a robot 10.

[0085] As shown in Figure 13, the operation program generation system 300 is a system that allows a user of the robot 10 to specify a task to be performed by the operation of the robot 10 and obtain an operation program 11 that operates the robot 10.

[0086] As shown in FIG. 13 , the operation program generation system 300 includes a computer 20 and a server 301 connected to the computer 20 via a network. The operation program generation system 300 also includes a plurality of computers 40 connected to the server 301 via the network. The operation program generation system 300 also includes a plurality of computers 50 connected to the server 301 via the network. The operation program generation system 300 also includes a plurality of computers 60 connected to the server 301 via the network. The operation program generation system 300 also includes a plurality of computers 70 connected to the server 301 via the network. The computer 20 is an example of a "task receiving unit" in the present disclosure. The server 30 is an example of an "operation program generation unit," "first server," "second server," "third server," and "simulation execution unit" in the present disclosure.

[0087] In the second embodiment, skill information 32b that enables the robot 10 to execute the task of the task information 32a is associated with the task information 32a and stored in the server 301. For example, the skill information 32b has a list of tasks that can be executed by the skill, and is associated with the task information 32a of the listed tasks.

[0088] 13 , the server 301 is connected to the computer 20 via a network. The server 301 generates the operation program 11 for the robot 10. The server 301 includes a control unit 31 and a storage unit 32.

[0089] The control unit 31 includes a processor such as a CPU, and memories such as a ROM and a RAM, and also executes programs to perform processing.

[0090] The storage unit 32 stores information. For example, the storage unit 32 stores information uploaded to the server 30. The storage unit 32 stores information including a plurality of pieces of task information 32a, a plurality of pieces of skill information 32b, a plurality of simulation programs 32c, and spatial position information 32d. The storage unit 32 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0091] As shown in FIG. 13 , the computer 70 is connected to the server 301 via a network. The computer 70 is used by a user at the site who uses the robot 10. The computer 70 is, for example, a personal computer. The computer 70 generates spatial position information 32d, which is three-dimensional position information of the area where the robot 10 works, through operation by the user at the site who uses the robot 10. The computer 70 transmits the generated spatial position information 32d to the server 301 through operation by the user at the site. The computer 70 may be a mobile terminal such as a tablet or smartphone. The spatial position information 32d is generated from blueprints, CAD data, etc. of the location where the robot 10 operates. The spatial position information 32d may be generated by someone other than the user at the site, as long as it can acquire three-dimensional information of the location where the robot 10 operates.

[0092] The spatial position information 32d created in the computer 70 and transmitted to the server 301 may be made public, may be made public to a limited audience, or may be made private, at the discretion of the creator. For example, the spatial position information 32d stored in the server 301 may be made public so that it can be used by anyone other than the person who uploaded it.

[0093] The spatial position information 32d is used when the robot 10 performs work. Specifically, the robot 10 operates based on the generated operation program 11 and the spatial position information 32d of the server 301.

[0094] The spatial position information 32d includes, for example, as shown in FIG. 25, at least one of spatial shape information 32e of the area where the robot 10 works, position information 32f of objects placed in the area where the robot 10 works, people flow information 32g of the area where the robot 10 works, regulation information 32h of the area where the robot 10 works, and position information 32i of people in the area where the robot 10 works.

[0095] The spatial shape information 32e of the area where the robot 10 works includes information on the planar shape of the site and information on the three-dimensional shape of the site. For example, the shape information 32e includes a floor map of the site for each floor level. The shape information 32e also includes three-dimensional CAD data of the site.

[0096] The position information 32f of objects placed in the area where the robot 10 works includes information indicating the positions of objects placed on the site, such as desks, chairs, shelves, electrical appliances, various devices, etc. For example, the object position information 32f includes information on the two-dimensional position coordinates of the object and information on the three-dimensional position coordinates of the object.

[0097] The people flow information 32g of the area where the robot 10 works includes information on the movement and concentration of people at the site. The people flow information 32g may be obtained by analyzing surveillance video or by aggregating data from smartphones carried by people. The people flow information 32g may also be obtained based on past data.

[0098] The restriction information 32h of the area where the robot 10 works includes information on locations where passage is prohibited due to construction or other work, and information on locations where the robot has difficulty passing due to steps, slopes, etc. The restriction information 32h is set, for example, using an app on a mobile device. The restriction information 32h may also be set based on the spatial shape information 32e.

[0099] The position information 32i of people in the area where the robot 10 is working includes information indicating the positions of people at the work site. The position information 32i of people may be obtained by analyzing surveillance video or the like, or by aggregating data from smartphones or the like carried by people.

[0100] In the second embodiment, the server 301 generates the operation program 11 for the robot 10 by setting at least one skill information 32b selected from a plurality of skill information 32b that are pre-stored and include elements of the operation program 11, in order to have the robot 10 execute the task of the task information 32a input by the computer 20.

[0101] In the second embodiment, a task is broken down into smaller parts, which are called subtasks. A subtask is broken down into units that can be executed using skills, which are called activities. A subtask may be executed by multiple activities or by a single activity.

[0102] In other words, skills are set to perform tasks, and skills are the means to perform activities, and skills depend on the functionality of the robot.

[0103] 14, for example, a task 310 of going from the first floor to the third floor by elevator is set with a plurality of pieces of skill information 32b. Specifically, for the task 310 of going from the first floor to the third floor by elevator, a plurality of pieces of skill information 32b are set with which to execute a subtask 311 of getting on the elevator at the first floor and a subtask 312 of getting out of the elevator at the third floor.

[0104] 15, a plurality of activities are set for the subtask 311 of getting on the elevator on the first floor, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of the following activities is set for the subtask 311 of getting on the elevator on the first floor: activity 311a of finding the elevator, activity 311b of moving in front of the elevator, activity 311c of finding the button for an upper floor, activity 311d of pressing the button for an upper floor, and activity 311e of moving into the elevator.

[0105] Furthermore, the subtask 311 of getting on the elevator on the first floor may be set as a single activity. In other words, the subtask 311 may become the activity as it is. Then, skill information 32b for executing the set activity is set. For example, as shown in Fig. 16, the skill for executing the subtask 311 of getting on the elevator on the first floor is set as skill 311f of calling someone by speaker and waiting until they get on the elevator. In this case, a speaker-equipped robot 10d capable of speaker output can be selected as the robot 10 to be operated.

[0106] 17 , a plurality of pieces of skill information 32b are set for task 320 of checking whether there is a person in room A. Specifically, a plurality of pieces of skill information 32b are set for task 320 of checking whether there is a person in room A, which executes subtask 321 of moving to the front of the room and subtask 322 of opening the door.

[0107] 18 , a plurality of activities are set for the subtask 322 of opening the door, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 322a of finding an identification code, an activity 322b of extending an arm, and an activity 322c of pulling to open the door is set for the subtask 322 of opening the door.

[0108] In the second embodiment, executable skills are displayed for activities for executing tasks and subtasks on an operation screen for creating the operation program 11. For example, as shown in Fig. 18 , skills associated with an activity 322a for finding an identification code are displayed as skills required for the activity. In the example of Fig. 18 , a skill 322aa for scanning a code with a robot camera, a skill 322ab for scanning a code with an environmental camera, a skill 322ac for taking photographs with a camera, a skill 322ad for reading a code, and a skill 322ae for scanning a code are displayed as skills required for the activity 322a for finding an identification code.

[0109] Then, when a skill 322aa for scanning a code with a robot camera is selected to perform an activity 322a for finding an identification code, a camera-equipped robot 10e having a camera module and a code reading module can be selected as the robot 10, as shown in FIG. 19 (A).

[0110] Also, if a skill 322ab for scanning a code with an environmental camera is selected to perform an activity 322a for finding an identification code, an environmental camera 10f having a camera module and a code reading module, as shown in FIG. 19(B), can be selected as the device for execution.

[0111] Furthermore, when the camera photography skill 322ac and the code reading skill 322ad are selected to perform the activity 322a of finding an identification code, as shown in FIG. 19(C), a camera 10g having a camera module and a processing unit 10h having a code reading module can be selected as the equipment for execution.

[0112] Furthermore, when a code scanning skill 322ae is selected to execute an identification code finding activity 322a, a camera 10g having a camera module and a processing unit 10h having a code reading module can be selected as devices for execution, as shown in FIG. 19(D). In other words, the code scanning skill 322ae is a skill that executes processing using multiple devices (such as robots and cameras) for one skill. In this case, multiple devices (robots and cameras) are called from one skill 322ae.

[0113] Furthermore, the subtask 322 of opening the door may be set as a single activity. In other words, the subtask 322 may become the activity as it is. Then, skill information 32b for executing the set activity is set. For example, as shown in FIG. 20 , for the activity of the subtask 332 of opening the door, a skill 332a of calling someone using a speaker and waiting until the door opens, a skill 332b of sending a command to open the automatic door, and a skill 332c of pulling the door with an arm are displayed as skills required to execute the activity of the subtask 332 of opening the door.

[0114] In this case, as shown in FIG. 21, a speaker-equipped robot 10i having a speaker output module that can execute a skill 332a of calling someone by speaker and waiting until the door opens can be selected as the robot 10.

[0115] In the second embodiment, the robot 10 operates based on the generated operation program 11 and the spatial position information 32 d of the server 301 .

[0116] For example, spatial position information 32d is embedded in the operation program 11. The robot 10 operates based on the operation program 11 with the embedded spatial position information 32d. In this case, values ​​such as coordinates are set in the operation program 11 when the program is generated, as position information such as a movement destination and a work position.

[0117] Furthermore, the source of the spatial position information 32d may be embedded in the operation program 11. That is, when the robot 10 uses the spatial position information 32d, the robot 10 may access the spatial position information 32d in the server 301 from which the information is obtained and perform an operation based on the position information. That is, the spatial position information 32d is obtained by the robot 10 when the operation program 11 is executed.

[0118] Furthermore, the robot 10 may acquire the spatial position information 32d itself and perform the operation when executing the operation program 11. For example, the robot 10 acquires the spatial position information 32d from a sensor, a camera, an external camera, or the like.

[0119] The spatial position information 32d is used when setting the movement destination and working position of the robot 10.

[0120] 22, for example, in a task 340 of delivering mail to AA, spatial position information 32d is used to set the location of AA and the location where the mail is to be placed. The task 340 of delivering mail to AA includes a subtask 341 of moving to AA's desk, a subtask 342 of picking up the mail, and a subtask 343 of placing the mail on the desk.

[0121] 23, for example, spatial position information 32d includes coordinate data 32da indicating the position of the room, coordinate data 32db indicating the position of the desk, and coordinate data 32dc indicating the position on the desk. These coordinate data may be represented by a three-dimensional coordinate system of length, width, and height, or may be represented by a two-dimensional coordinate system of length and width.

[0122] When the spatial position information 32d is embedded in the operation program 11, in the subtask 341 of moving to the desk of Mr. AA, the location of Mr. AA is acquired from the spatial position information 32d and embedded. In the subtask 343 of placing mail on the desk, the position where the mail is placed is acquired from the spatial position information 32d and embedded.

[0123] That is, when generating the operation program 11 by embedding the spatial position information 32d, a task is decomposed into subtasks and activities, the spatial position information 32d required to execute the subtasks and activities is embedded, and skills are combined to generate the operation program 11. Then, when the robot 10 operates, it operates based on the operation program 11 with the embedded spatial position information 32d.

[0124] 24, in a task 350 of delivering tea to a conference room KA, spatial position information 32d is used to set the position of the conference room KA to which the tea is to be delivered. The task 350 of delivering tea to a conference room KA includes a subtask 351 of picking up the tea, a subtask 352 of moving to the conference room KA to which the tea is to be delivered, and a subtask 353 of handing over the tea.

[0125] The destination of the tea can be selected as an argument. For example, the operation program 11 is generated so that the destination can be selected from conference room KA, conference room KB, etc. In other words, by specifying the destination when using the operation program 11 for delivering tea, the robot 10 can be made to perform the operation of delivering tea to the specified destination.

[0126] When the acquisition destination of the spatial position information 32d is embedded in the operation program 11, in the subtask 352 for moving to the conference room KA, which is the destination of the tea delivery, the acquisition destination of the position information of the destination is embedded in the location of the tea delivery destination. For example, the address information of the server 301 or the like is embedded in the operation program 11 as the acquisition destination of the position information.

[0127] When the acquisition destination of the spatial position information 32d is embedded in the operation program 11, when the operation program 11 is executed, the spatial position information 32d of the specified delivery destination is acquired each time, and the robot 10 is operated.

[0128] That is, when embedding the acquisition source of the spatial position information 32d in the operation program 11, a task is decomposed into subtasks and activities, the acquisition source of the spatial position information 32d required to execute the subtasks and activities is embedded, and skills are combined to generate the operation program 11. Then, when the robot 10 operates, it acquires the spatial position information 32d from the acquisition source and operates based on the operation program 11.

[0129] The spatial position information 32d may be updated as appropriate. In this case, the spatial position information 32d may be configured by the robot developer or by someone else. For example, a user at the facility where the robot 10 works may update the spatial position information 32d. The spatial position information 32d may also be updated based on the image capture results of a fixed camera or a mobile camera installed in the facility where the robot 10 works. In this case, the spatial position information 32d may be updated by identifying the positions of people and desks through image processing based on the image capture results. The spatial position information 32d may also be updated based on position information acquired by another robot 10 installed in the facility where the robot 10 works. For example, the spatial position information 32d may be updated based on images captured by the other robot 10.

[0130] 26 , when the spatial position information 32d is shared, the spatial position information 32d is made public to the server 301, and the spatial position information 32d made public to the server 301 is used. Specifically, a provider of a people flow prediction service creates a people flow model using a congestion prediction app and registers it in the congestion prediction data of the server 301. Also, a provider of a facility operation service sets a prohibited area using a no-entry area setting app and registers it in the 3D map facility location of the server 301.

[0131] In addition, a service provider for predicting people flow uses the 3D map facility location information of the server 301 to predict people flow information using a congestion prediction application.

[0132] Furthermore, a social robot development service provider generates a movement route for the social robot using a route generation app based on the congestion forecast data, 3D map facility location information, and no-traffic zone setting data from the server 301. That is, an operation program for the movement of the social robot is generated using on-site location information, people flow information, regulation information, and the like. The congestion forecast data, 3D map facility location information, and no-traffic zone setting data from the server 301 may be referenced when the operation program is executed. For example, the operation program for the social robot does not specify a specific movement route. When the social robot moves, an appropriate movement route may be set based on the congestion forecast data, 3D map facility location information, and no-traffic zone setting data from the server 301.

[0133] (Effects of the Second Embodiment) The operation program generation system 300 includes a server 301 that generates an operation program 11 for the robot 10 based on setting at least one piece of skill information 32b selected from a plurality of pieces of skill information 32b that are stored in advance and include elements of the operation program 11, in order to have the robot 10 execute a task of task information 32a input by the computer 20. This makes it possible to easily generate the operation program 11 for the robot 10 with high accuracy.

[0134] Skill information 32b that allows the robot 10 to execute the task of the task information 32a is associated with the task information 32a and stored in the server 301. This makes it easy to select skills that allow the robot 10 to execute the task of the task information 32a, making it easier to generate the operation program 11 for the robot 10.

[0135] A server 301 is provided that stores spatial position information 32d of the area where the robot 10 works, and the robot 10 operates based on the generated operation program 11 and the spatial position information 32d of the server 301. This allows the position information required for the robot 10 to work to be set based on the spatial position information 32d, making it possible to easily operate the robot 10 in a desired position.

[0136] The spatial position information 32d stored in the server 301 is made public so that it can be used by anyone other than the person who uploaded it. This allows the public spatial position information 32d to be used to efficiently acquire position information required for the robot 10 to perform its work. Furthermore, since developers of apps and robots can acquire the public spatial position information 32d from the server 301, they can efficiently develop apps and robots using the position information required for the robot to perform its work.

[0137] 27 to 33, the configuration of an operation program generation system 400 according to a third embodiment of the present invention will be described. Note that the same components as those in the first embodiment are denoted by the same reference numerals and their description will be omitted. As shown in FIG. 27, the operation program generation system 400 is a system that generates an operation program 11 for a robot 10.

[0138] As shown in Figure 27, the operation program generation system 400 is a system that allows a user of a robot 10 to specify a task to be performed by the operation of the robot 10 and obtain an operation program 11 that operates the robot 10.

[0139] 27 , the operation program generation system 400 includes a computer 20 and a server 401 connected to the computer 20 via a network. The operation program generation system 400 also includes a plurality of computers 40 connected to the server 401 via the network. The operation program generation system 400 also includes a plurality of computers 50 connected to the server 401 via the network. The operation program generation system 400 also includes a plurality of computers 60 connected to the server 401 via the network. The computer 20 is an example of a "task receiving unit" in the present disclosure. The server 401 is an example of an "operation program generation unit," "first server," "second server," "third server," "fourth server," and "simulation execution unit" in the present disclosure.

[0140] 27 , the server 401 is connected to the computer 20 via a network. The server 401 generates the operation program 11 for the robot 10. The server 401 includes a control unit 31 and a storage unit 32.

[0141] The control unit 31 includes a processor such as a CPU, and memories such as a ROM and a RAM, and also executes programs to perform processing.

[0142] The storage unit 32 stores information. For example, the storage unit 32 stores information uploaded to the server 401. The storage unit 32 stores information including a plurality of pieces of task information 32a, a plurality of pieces of skill information 32b, a plurality of simulation programs 32c, and a plurality of skill definitions 32j. The storage unit 32 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0143] As shown in FIG. 27 , the computer 40 is connected to the server 401 via a network. The computer 40 is used by a skill creator who creates the skill information 32b. The computer 40 is, for example, a personal computer. The computer 40 creates the skill information 32b through operation by the skill creator. The computer 40 transmits the created skill information 32b to the server 401 through operation by the skill creator. Note that the computer 40 may also be a mobile terminal such as a tablet or smartphone.

[0144] In the third embodiment, the server 401 stores skill definitions 32j that define input information and output information for the skill information 32b and are made publicly available to skill creators. That is, the skill information 32b created by the skill creator has a corresponding skill definition 32j defined. The skill definitions 32j are used by the skill creator when developing the skill information 32b required to execute a task or subtask. The skill definitions 32j are defined, for example, by an administrator who manages the operation program generation system 400. This allows a common definition to be assigned to skills for causing the robot 10 to perform similar operations. The skill definitions 32j define input information and output information, allowing the skill creator to create a program for outputting the output information from the input information. This allows multiple skill creators to create multiple pieces of skill information 32b each having a common skill definition 32j defined therein. This allows tasks or subtasks using multiple pieces of skill information 32b each having a defined skill definition 32j to be easily combined. The skill definition 32j defines input information and output information for a skill for executing a task or subtask, as shown in, for example, FIGS.

[0145] The server 401 generates the operation program 11 for the robot 10 by setting at least one skill information 32b selected from a plurality of skill information 32b that are pre-stored and contain elements of the operation program 11, in order to have the robot 10 execute the task of the task information 32a input by the computer 20.

[0146] For example, as shown in FIG. 28, (1) the robot 10 moves to the position where the transport device 402, which is a mobile body, has moved to, (2) the robot 10 retrieves the specimen from the transport device 402, (3) the robot 10 moves to the inspection device 403 and transports the specimen, (4) the robot 10 sets the specimen in the inspection device 403, (5) the robot 10 operates the inspection device 403 to start the inspection, and (6) the robot 10 moves to a standby position and waits, thereby executing a task 410 (see FIG. 29) to inspect the transported specimen.

[0147] 29, a plurality of pieces of skill information 32b are set for task 410 of inspecting a transported sample. Specifically, task 410 of inspecting a transported sample includes subtask 411 of moving to a transport device, subtask 412 of removing the sample from the transport device, subtask 413 of moving to an inspection device, subtask 414 of setting the sample in the inspection device, subtask 415 of pressing an inspection start button, and subtask 416 of moving to a standby position.

[0148] A plurality of activities are set for the subtask 411 of moving to the transport device, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 411a for acquiring a current location, an activity 411b for acquiring a destination, and a movement activity 411c is set for the subtask 411 of moving to the transport device.

[0149] A plurality of activities are set for the subtask 412 of retrieving the specimen from the transport device, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 412a of acquiring the position of the object, an activity 412b of reaching out, and an activity 412c of grasping the object is set for the subtask 412 of retrieving the specimen from the transport device.

[0150] A plurality of activities are set for the subtask 413 of traveling to the inspection device, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 413a for acquiring a current location, an activity 413b for acquiring a destination, and a travel activity 413c is set for the subtask 413 of traveling to the inspection device.

[0151] A plurality of activities are set for the subtask 414 of setting the sample on the testing device, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 414a of acquiring the position of the object, an activity 414b of reaching out, an activity 414c of pushing in, and an activity 414d of releasing the object is set for the subtask 414 of setting the sample on the testing device.

[0152] A plurality of activities are set for the subtask 415 of pressing the inspection start button, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 415a of acquiring the position of an object, an activity 415b of reaching out, and an activity 415c of pushing is set for the subtask 415 of pressing the inspection start button.

[0153] A plurality of activities are set for the subtask 416 of moving to a standby position, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 416a of acquiring a current location, an activity 416b of acquiring a destination, and an activity 416c of moving is set for the subtask 416 of moving to a standby position.

[0154] 30, for activities 411a, 413a, and 416a that acquire a current location, a skill for acquiring the current location by self-reference can be set as a skill to be executed. The skill for acquiring the current location by self-reference defines, as its skill definition, the target whose location is to be acquired as input information and the current location of the target as output information. In other words, with the skill for acquiring the current location by self-reference, if the target whose location is to be acquired is input, the current location of the target is output.

[0155] In the activities 411b, 413b, and 416b for acquiring a destination, a skill for acquiring the destination based on a marker position using a camera can be set as a skill to be executed. The skill for acquiring the destination based on a marker position using a camera has the name of the destination defined as input information and the coordinate values ​​of the destination defined as output information as a skill definition. In other words, the skill for acquiring the destination based on a marker position using a camera outputs the coordinate values ​​of the destination when the name of the destination is input. Note that the coordinate values ​​of the destination may be two-dimensional coordinate values ​​or three-dimensional coordinate values.

[0156] For the movement activities 411c, 413c, and 416c, a skill for moving by wheel can be set as a skill to be executed. The skill for moving by wheel has a current location and a destination defined as input information, and an action completion defined as output information. In other words, for the skill for moving by wheel, if the current location and destination are input, the action completion is output after moving from the current location to the destination.

[0157] In the activities 412a, 414a, and 415a that acquire the position of a target, a skill for acquiring the position of a target using a camera / image AI can be set as the skill to be executed. The skill for acquiring the position of a target using a camera / image AI has the target item defined as input information and the target coordinate value of the target defined as output information as the skill definition. In other words, in the skill for acquiring the position of a target using a camera / image AI, when the target item is input, the target coordinate value of the target is output. Note that the target coordinate value of the target may be a two-dimensional coordinate value or a three-dimensional coordinate value.

[0158] In the reaching activities 412b, 414b, and 415b, a skill for making the arm reach a target position can be set as a skill to be executed. The skill for making the arm reach a target position is defined as having the target coordinate value of the target as input information and the completion of the operation as output information. In other words, in the skill for making the arm reach a target position, if the target coordinate value of the target is input, the completion of the operation is output after the target has reached the target coordinate value.

[0159] In the activity 412c for grasping an object, a skill for grasping with a hand can be set as a skill to be executed. The skill for grasping with a hand does not have input information as a skill definition, and the output information defines the completion of the operation. In other words, in the skill for grasping with a hand, the completion of the operation is output after grasping with a hand.

[0160] In the push activities 414c and 415c, a skill for pushing in a specified direction with an arm can be set as the skill to be executed. The skill for pushing in a specified direction with an arm is defined as having the push direction and force magnitude as input information, and the operation completion as output information. In other words, for the skill for pushing in a specified direction with an arm, if the push direction and force magnitude are input, the arm is pushed in the specified direction, and then the operation completion is output.

[0161] In the object release activity 414d, a skill for releasing an object grasped by a hand can be set as a skill to be executed. The skill for releasing an object grasped by a hand does not have input information as a skill definition, and the output information defines the completion of the operation. In other words, in the skill for releasing an object grasped by a hand, the completion of the operation is output after the object grasped by the hand is released.

[0162] 31, (1) the robot 10 moves to the shelf 404, (2) the robot 10 picks up an item from the shelf 404, (3) the robot 10 moves to the transport device with basket 406 and transports the item, (4) the robot 10 places the item on the transport device with basket 406, (5) a movement instruction is issued to the transport device with basket so that the robot 10 moves, (6) the robot 10 moves to the shelf 405, (7) the robot 10 picks up an item from the shelf 405, and (8) the robot 10 (9) the robot 10 moves to the basket-equipped conveying device 406 and conveys the item, (10) the robot 10 issues a movement command to move relative to the basket-equipped conveying device, (11) the robot 10 moves to the sorting box 407, (12) the robot 10 takes out the item from the basket-equipped conveying device 406, and (13) the robot 10 sorts the items and places them in the sorting box, thus executing a task 420 (see FIG. 32 ) of sorting the items from the shelf into the sorting box.

[0163] 32, a plurality of pieces of skill information 32b are set for task 420, which involves sorting items from shelves into sorting bins. Specifically, task 420, which involves sorting items from shelves into sorting bins, includes subtasks 421, which involves moving to the shelves, subtask 422, which involves removing items from the shelves, subtask 423, which involves moving to the conveying device with basket, subtask 424, which involves placing the items on the conveying device with basket, subtask 425, which involves instructing the conveying device with basket to move, and subtask 426, which involves moving to the shelves.

[0164] A plurality of activities are set for the subtask 421 of moving to the shelf, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 421a for acquiring a current location, an activity 421b for acquiring a destination, and a movement activity 421c is set for the subtask 421 of moving to the shelf.

[0165] A plurality of activities are set for the subtask 422 of retrieving an item from a shelf, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 422a of acquiring the position of an object, an activity 422b of reaching out, and an activity 422c of grabbing the object is set for the subtask 422 of retrieving an item from a shelf.

[0166] A plurality of activities are set for the subtask 423 of moving to the car-equipped conveyance device, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 423a for acquiring a current location, an activity 423b for acquiring a destination, and a movement activity 423c is set for the subtask 423 of moving to the car-equipped conveyance device.

[0167] A plurality of activities are set for the subtask 424 of placing an article on a conveyance device with a car, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 424a of acquiring the position of an object, an activity 424b of reaching out, and an activity 424c of releasing the object is set for the subtask 424 of placing an article on a conveyance device with a car.

[0168] A plurality of activities are set for the subtask 425 that instructs the car-equipped conveyance device to move, and skill information 32b for executing each of the set activities is set. Specifically, the subtask 425 that instructs the car-equipped conveyance device to move is set with skill information 32b for executing an activity 425a that acquires a destination and an activity 425b that instructs the car-equipped conveyance device to move.

[0169] A plurality of activities are set for the subtask 426 of moving to the shelf, and skill information 32b for executing each of the set activities is set. Specifically, skill information 32b for executing each of an activity 426a of acquiring a current location, an activity 426b of acquiring a destination, and an activity 426c of moving is set for the subtask 426 of moving to the shelf.

[0170] 33, for activities 421a, 423a, and 426a that acquire a current location, a skill for acquiring the current location by self-reference can be set as a skill to be executed. The skill for acquiring the current location by self-reference defines, as its skill definition, the target whose location is to be acquired as input information and the current location of the target as output information. In other words, with the skill for acquiring the current location by self-reference, if the target whose location is to be acquired is input, the current location of the target is output.

[0171] In the activities 421b, 423b, 425a, and 426b that acquire a destination, a skill that acquires the position of an object by querying the management system can be set as the skill to be executed. The skill that acquires the position of an object by querying the management system has the target or destination defined as input information and the coordinate values ​​of the target or destination defined as output information as the skill definition. In other words, in the skill that acquires the position of an object by querying the management system, when the target or destination is input, the coordinate values ​​of the target or destination are output. Note that the coordinate values ​​of the target or destination may be two-dimensional coordinate values ​​or three-dimensional coordinate values.

[0172] Furthermore, in activities 423b and 426b that acquire a destination for executing a subtask of moving to a transport device, a skill for querying the transport device to acquire the current location of the transport device can be set as a skill to be executed. The skill for querying the transport device to acquire the current location of the transport device has the target transport device defined as input information and the coordinate values ​​of the transport device defined as output information. In other words, the skill for querying the transport device to acquire the current location of the transport device outputs the coordinate values ​​of the transport device when the target transport device is input. Note that the coordinate values ​​of the transport device may be two-dimensional coordinate values ​​or three-dimensional coordinate values.

[0173] For the movement activities 421c, 423c, and 426c, a skill for moving by wheel can be set as a skill to be executed. The skill for moving by wheel has a current location and a destination defined as input information, and an action completion defined as output information. In other words, for the skill for moving by wheel, if the current location and destination are input, the action completion is output after moving from the current location to the destination.

[0174] In the activities 422a and 424a that acquire the position of a target, a skill for acquiring the position of a target using camera logic can be set as the skill to be executed. The skill for acquiring the position of a target using camera logic has the target item defined as input information and the target coordinate value defined as output information as the skill definition. In other words, in the skill for acquiring the position of a target using camera logic, when the target item is input, the target coordinate value of the target is output. Note that the target coordinate value of the target may be a two-dimensional coordinate value or a three-dimensional coordinate value.

[0175] Furthermore, in the activities 422a and 424a that acquire the position of an object, a skill for acquiring the position of an object using a camera / image AI can be set as a skill to be executed. The skill for acquiring the position of an object using a camera / image AI has the target item defined as input information and the target coordinate value of the object defined as output information as a skill definition. In other words, in the skill for acquiring the position of an object using a camera / image AI, when the target item is input, the target coordinate value of the object is output. Note that the target coordinate value of the object may be a two-dimensional coordinate value or a three-dimensional coordinate value.

[0176] In the reaching activities 422b and 424b, a skill for making the arm reach a target position can be set as a skill to be executed. The skill for making the arm reach a target position is defined as having the target coordinate value of the target as input information and the completion of the operation as output information. In other words, in the skill for making the arm reach a target position, if the target coordinate value of the target is input, the completion of the operation is output after the target has reached the target coordinate value.

[0177] In the activity 422c for grasping an object, a skill for grasping with a hand can be set as a skill to be executed. The skill for grasping with a hand does not have input information as a skill definition, and the output information defines the completion of the operation. In other words, in the skill for grasping with a hand, the completion of the operation is output after grasping with a hand.

[0178] In the object release activity 424c, a skill for releasing an object grasped by a hand can be set as a skill to be executed. The skill for releasing an object grasped by a hand does not have input information as a skill definition, and the output information defines the completion of the operation. In other words, in the skill for releasing an object grasped by a hand, the completion of the operation is output after the object grasped by the hand is released.

[0179] In the activity 425b for instructing movement, a skill for instructing the movement of a device via a management system can be set as a skill to be executed. The skill for instructing the movement of a device via a management system has a skill definition in which a movement target coordinate value is defined as input information and an operation completion is defined as output information. In other words, the skill for instructing the movement of a device via a management system inputs a movement target coordinate value for the device, instructs the device to move, and then outputs an operation completion.

[0180] Furthermore, for the activity 425b instructing movement, a skill for directly instructing a device to move can be set as a skill to be executed. The skill for directly instructing a device to move has a skill definition in which a movement target coordinate value is defined as input information and an operation completion is defined as output information. In other words, a skill for instructing a device to move via a management system inputs a movement target coordinate value for the device, instructs the device to move, and then outputs an operation completion.

[0181] (Effects of the Third Embodiment) The operation program generation system 400 includes a server 401 that generates an operation program 11 for the robot 10 based on setting at least one piece of skill information 32b selected from a plurality of pieces of skill information 32b that are stored in advance and include elements of the operation program 11, in order to have the robot 10 execute a task of task information 32a input by the computer 20. This makes it possible to easily generate the operation program 11 for the robot 10 with high accuracy.

[0182] A server 401 is provided in which skill definitions 32j, which define input and output information for skill information 32b, are stored and made publicly available to skill creators. This allows skill creators to create skill information 32b with standardized input and output information. This allows multiple pieces of skill information 32b to be easily combined to execute tasks, subtasks, and activities. While skills can be selected as means to accomplish a single activity, defining input and output information in the skill information allows appropriate skill combinations to be made depending on the task and robot configuration, even when multiple means are possible. For example, as shown in FIG. 33 , for the subtask "instructing movement," a skill "instructing device movement via a management system" and a skill "instructing device movement directly to device" can be selected. This allows appropriate skill selection to be made depending on the constraints of the installation location (network configuration).

[0183] The skill information 32b created by a skill creator so as to correspond to the skill definition 32j is stored in the server 401. This allows the skill information 32b, in which the input information and output information are standardized, to be easily used in creating an operation program.

[0184] (Modifications) The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the description of the above-mentioned embodiments, and further includes all modifications (modifications) within the meaning and scope equivalent to the claims.

[0185] In the above embodiment, an example is shown in which at least one piece of skill information is set from a plurality of pieces of skill information suitable for both the type of robot 10 and the type of task, but the present disclosure is not limited to this. For example, at least one piece of skill information 32b may be set from a plurality of pieces of skill information 32b suitable for the type of task, or at least one piece of skill information 32b may be set from a plurality of pieces of skill information 32b suitable for the type of robot 10.

[0186] In the above embodiment, an example is shown in which a simulation is performed using the generated operation program 11 to evaluate the selected skill information 32b, but the present disclosure is not limited to this. For example, the selected skill information 32b may be evaluated by actually operating the robot 10 using the operation program 11.

[0187] In the above embodiment, a simulation is performed using the generated operation program 11 to evaluate the robot 10 with respect to the task information 32 a, but the present disclosure is not limited to this. For example, the robot 10 may be actually operated using the operation program 11 to evaluate the robot 10 with respect to the task information 32 a.

[0188] In the above embodiment, an example has been described in which a simulation program is stored in the server 30 and the operation program generation system 100 uses the simulation program, but the present disclosure is not limited to this. For example, the operation program generation system 100 does not need to use a simulation program. Furthermore, the simulation program does not need to be stored in the server 30.

[0189] In the above embodiment, an example has been shown in which the operation program 11 is generated by the server 30, but the present disclosure is not limited to this. For example, the operation program 11 may be generated by the computer 20 used by the user of the robot 10, or the operation program 11 may be generated by both the server 30 and the computer 20.

[0190] In the above embodiment, the task information 32 a and the skill information 32 b are stored in a common server 30, but the present disclosure is not limited to this. For example, the server that stores the task information 32 a and the server that stores the skill information 32 b may be separate servers.

[0191] In the above embodiment, the skill information 32b and the simulation program 32c are stored in a common server 30. However, the present disclosure is not limited to this. For example, the server that stores the skill information 32b and the server that stores the simulation program 32c may be separate servers.

[0192] In the above embodiment, an example has been described in which the server 30 that generates the operation program 11 and the server 30 that performs the simulation are the same server, but the present disclosure is not limited to this. For example, the server 30 that generates the operation program 11 and the server 30 that performs the simulation may be different servers.

[0193] In the above embodiment, an example has been shown in which the computer 20 for generating the operation program 11 and the robot 10 are provided separately, but the present disclosure is not limited to this. For example, the computer 20 may be provided in the robot 10. Furthermore, when the computer 20 is provided in the robot 10, the robot 10 and the server 30 may communicate information directly with each other.

[0194] In the above embodiment, the robot 10 that generates the operation program 11 is a social robot, but the present disclosure is not limited to this. For example, the robot 10 that generates the operation program 11 may be an industrial robot or a medical robot.

[0195] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0196] Aspects It will be understood by those skilled in the art that the exemplary embodiments described above are specific examples of the following aspects.

[0197] (Aspect 1) An operation program generation system comprising: a task receiving unit that receives input of task information for causing a robot to perform a task by operating the robot; and an operation program generation unit that generates an operation program for the robot based on setting at least one piece of skill information selected from a plurality of pieces of skill information that are pre-stored and include elements of an operation program, in order to have the robot perform the task of the task information input by the task receiving unit.

[0198] (Aspect 2) The operation program generation system according to aspect 1, wherein the operation program generation unit generates the operation program based on setting at least one piece of skill information from a plurality of pieces of skill information suited to the type of the robot and the type of the task.

[0199] (Aspect 3) An operation program generation system according to Aspect 1 or Aspect 2, comprising a first server in which the plurality of pieces of skill information created by a plurality of skill creators are stored, and the operation program generation unit generates the operation program based on setting at least one piece of skill information selected from the plurality of pieces of skill information stored in the first server.

[0200] (Aspect 4) The operation program generation system according to any one of Aspects 1 to 3, wherein the skill information can be set for a plurality of pieces of task information that are different from one another.

[0201] (Aspect 5) The operation program generation system according to any one of Aspects 1 to 4, further comprising a simulation execution unit that performs a simulation of the robot executing the task using the operation program generated with the skill information set, and evaluates the set skill information from the simulation results.

[0202] (Aspect 6) An operation program generation system according to Aspect 5, further comprising a second server in which a plurality of simulation programs created by a plurality of simulation creators are stored, wherein the simulation execution unit performs a simulation of the robot executing the task using the operation program generated with the skill information set and the simulation program stored in the second server.

[0203] (Aspect 7) The operation program generation system according to aspect 5 or aspect 6, wherein the operation program generation unit displays an evaluation value of the skill information evaluated by the simulation execution unit on a display unit of the task reception unit, and the task reception unit receives input of a selection of the skill information to be set for the task information.

[0204] (Aspect 8) The operation program generation system according to any one of Aspects 1 to 7, further comprising a simulation execution unit that generates the operation program based on setting the robot selected from a plurality of the robots stored in advance, and performs a simulation of the robot executing the task using the operation program generated by setting the robot, and evaluates the robot against the task information.

[0205] (Aspect 9) The operation program generation system according to Aspect 8, wherein the operation program generation unit displays an evaluation value of the robot evaluated by the simulation execution unit on a display unit of the task reception unit, and the task reception unit receives an input of a selection of the robot to be set for the task information.

[0206] (Aspect 10) The operation program generation system according to aspect 8 or aspect 9, wherein the simulation execution unit sets an interface for applying the skill information to the robot to be the same as the interface of the actual robot, and performs a simulation of the robot executing the task.

[0207] (Aspect 11) The operation program generation system according to any one of Aspects 1 to 10, wherein the robot includes a social robot that provides services to people.

[0208] (Aspect 12) The operation program generation system according to aspect 3, wherein the skill information that enables the robot to execute the task of the task information is associated with the task information and stored in the first server.

[0209] (Aspect 13) The operation program generation system according to any one of Aspects 1 to 12, further comprising a third server that stores spatial position information of an area in which the robot works, and the robot operates based on the generated operation program and the spatial position information of the third server.

[0210] (Aspect 14) The operation program generation system according to aspect 13, wherein the spatial position information stored in the third server is made public so that it can be used by anyone other than the person who uploaded it.

[0211] (Aspect 15) The operation program generation system according to Aspect 13 or Aspect 14, wherein the spatial position information includes at least one of spatial shape information of an area where the robot works, position information of objects placed in the area where the robot works, people flow information of the area where the robot works, regulation information of the area where the robot works, and position information of people in the area where the robot works.

[0212] (Aspect 16) The operation program generation system according to any one of Aspects 1 to 15, further comprising a fourth server in which a skill definition, in which input information and output information for the skill information are defined, is stored so as to be made public to skill creators.

[0213] (Aspect 17) The operation program generation system according to aspect 16, further comprising a first server that stores the skill information created by the skill creator so as to correspond to the skill definition.

[0214] REFERENCE SIGNS LIST 10 Robot 11 Operation program 20 Computer (task receiving unit) 22 Display unit 30, 301 Server (operation program generating unit, first server, second server, third server, simulation executing unit) 32a Task information 32b Skill information 32c Simulation program 32d Spatial position information 100, 300 Operation program generating system

Claims

1. An operation program generation system comprising: a task receiving unit that receives input of task information for causing a robot to perform a task by operating it; and an operation program generation unit that generates an operation program for the robot based on setting at least one piece of skill information selected from a plurality of pieces of skill information that are pre-stored and include elements of an operation program, in order to have the robot perform the task of the task information input by the task receiving unit.

2. The operation program generation system of claim 1, wherein the operation program generation unit generates the operation program based on setting at least one piece of skill information from a plurality of pieces of skill information suitable for the type of robot and the type of task.

3. An operation program generation system as described in claim 1, comprising a first server in which the plurality of skill information created by a plurality of skill creators is stored, and the operation program generation unit generates the operation program based on setting at least one of the skill information selected from the plurality of skill information stored in the first server.

4. The operation program generation system according to claim 1, wherein the skill information can be set for a plurality of different task information.

5. An operation program generation system as described in claim 1, further comprising a simulation execution unit that performs a simulation of the robot executing the task using the operation program generated by setting the skill information, and evaluates the set skill information from the simulation results.

6. An operation program generation system as described in claim 5, further comprising a second server in which a plurality of simulation programs created by a plurality of simulation creators are stored, wherein the simulation execution unit performs a simulation of the robot executing the task using the operation program generated with the skill information set and the simulation program stored in the second server.

7. The operation program generation system described in claim 5, wherein the operation program generation unit displays the evaluation value of the skill information evaluated by the simulation execution unit on the display unit of the task reception unit, and the task reception unit accepts input of a selection of the skill information to be set for the task information.

8. The operation program generation system according to claim 1, further comprising a simulation execution unit that generates the operation program based on setting the robot selected from a plurality of robots stored in advance, and performs a simulation of the robot executing the task using the operation program generated by setting the robot, and evaluates the robot against the task information.

9. The operation program generation system described in claim 8, wherein the operation program generation unit displays the evaluation value of the robot evaluated by the simulation execution unit on the display unit of the task reception unit, and the task reception unit receives input of the selection of the robot to be set for the task information.

10. The operation program generation system of claim 8, wherein the simulation execution unit sets an interface for applying the skill information to the robot to be the same as the interface of the actual robot, and performs a simulation of the robot executing the task.

11. The operation program generation system according to claim 1, wherein the robot includes a social robot that provides services to people.

12. An operation program generation system as described in claim 3, wherein the skill information that enables the robot to execute the task of the task information is associated with the task information and stored in the first server.

13. The operation program generation system according to claim 1, further comprising a third server in which spatial position information of the area in which the robot works is stored, and the robot operates based on the generated operation program and the spatial position information of the third server.

14. The operation program generation system according to claim 13, wherein the spatial location information stored in the third server is made publicly available to anyone other than the person who uploaded it.

15. The operation program generation system of claim 13, wherein the spatial position information includes at least one of spatial shape information of the area where the robot works, position information of objects placed in the area where the robot works, people flow information of the area where the robot works, regulation information of the area where the robot works, and position information of people in the area where the robot works.

16. The operating program generation system according to claim 1, further comprising a fourth server in which skill definitions, in which input information and output information for said skill information are defined, are stored so as to be made publicly available to skill creators.

17. The operation program generation system according to claim 16, further comprising a first server in which the skill information created by the skill creator so as to correspond to the skill definition is stored.

Images