Robot control system and robot control method

Abstract

The robot control system (1) comprises: a teacher storage unit (211) storing information of a plurality of teachers; a selection unit (214) selecting any one of the plurality of teachers stored in the teacher storage unit (211) when support information is required in order for the robot (20) to continue working; an environment information sending unit (215) sending environment information of the robot (20) to the teacher selected by the selection unit (214) via a network line (50); and a support information receiving unit (216) receiving support information from the teacher selected by the selection unit (214) via the network line (50).

Description

Technical Field

[0001] The present disclosure relates to a robot control system and a robot control method. Background Art

[0002] Patent document 1 discloses a robot guidance method, which consists of the following stages: a first stage in which the robot stops halfway in a reproduction action based on a pre-registered work program, a second stage in which direct operation of the robot is allowed, a third stage in which the robot performs a guidance action through direct operation, a fourth stage in which direct operation of the robot is prohibited, and a fifth stage in which the robot continues to start the reproduction action based on the work program.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2006-82185 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The present disclosure provides a system effective for efficient collaboration between robots and humans.

[0008] Means for solving problems

[0009] A robot control system according to one aspect of the present disclosure comprises: a teacher storage unit which stores information of a plurality of teachers; a selection unit which selects any one of the plurality of teachers stored in the teacher storage unit when support information is required in order for the robot to continue operating; an environment information sending unit which sends environment information of the robot to the teacher selected by the selection unit via a network line; and a support information receiving unit which receives support information from the teacher selected by the selection unit via the network line.

[0010] Another aspect of the robot control method disclosed herein includes: storing information of multiple instructors; selecting any one of the multiple instructors when support information is needed in order for the robot to continue working; sending environmental information of the robot to the selected instructor via a network line; and receiving support information from the selected instructor via the network line.

[0011] Effects of the Invention

[0012] According to the present disclosure, it is possible to provide a system that is effective for efficient collaboration between robots and humans. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram illustrating the overall structure of a robot control system.

[0014] Figure 2 This is a block diagram illustrating the functional structure of an edge controller.

[0015] Figure 3 This is a block diagram illustrating the functional structure of a collaboration controller.

[0016] Figure 4 This is a block diagram illustrating the hardware structure of an edge controller and a collaborative controller.

[0017] Figure 5 This is a flowchart illustrating the output steps of the action command.

[0018] Figure 6 This is a flowchart illustrating the updating procedure of the work model.

[0019] Figure 7 This is a flowchart illustrating the procedure of receiving support information. DETAILED DESCRIPTION

[0020] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the description, the same reference numerals are given to elements having the same elements or the same functions, and repeated descriptions are omitted.

[0021] [Robot control system]

[0022] Figure 1 The robot control system 1 shown is a control system that enables collaboration between a robot and an instructor via a network line. The robot control system 1 is configured to perform the following actions: controlling the robot so that it performs a specified operation; selecting any one of a plurality of instructors when support information is required in order for the robot to continue the operation; sending the robot's environmental information to the selected instructor via the network line; receiving support information from the selected instructor via the network line; and controlling the robot based on the support information so that it continues the operation. The operation performed by the robot is not particularly limited. Specific examples of the operation performed by the robot include processing, assembly, and conveying operations. The operation performed by the robot is not limited to operations in the industrial field. For example, the operation performed by the robot may also be agricultural operations such as sowing, harvesting, leaf scraping, and pesticide spreading.

[0023] For example, the robot control system 1 exists between a plurality of robot devices 10 and a plurality of teaching terminals 40. The robot device 10 has a robot 20 and a robot controller 30. The robot 20 is, for example, a vertical multi-joint robot, having a base 21, a terminal 22, and a multi-joint arm 23. The base 21 is disposed in a work area. The base 21 may also be disposed on a vehicle that can move within the work area. The terminal 22 holds a tool for operation. Specific examples of tools include gripping handles and various tools. The multi-joint arm 23 uses a plurality of actuators (e.g., electric actuators) as power sources to change the position and posture of the terminal 22 relative to the base 21.

[0024] The robot controller 30 controls the robot 20 so that it moves according to the action instructions. The action instructions include, for example, movement instructions for the terminal 22 and movement instructions for the tool corresponding to the movement of the terminal 22. The movement instructions for the terminal 22 are instructions for changing the position and posture of the terminal 22, and include multiple target positions / postures (target positions / postures of the terminal 22) arranged in a time series. The teaching terminal 40 is a terminal used by the instructor, and can be any terminal as long as it can receive and display environmental information, and input and send support information. Specific examples of the teaching terminal 40 include smartphones, tablet computers, and personal computers.

[0025] For example, the robot control system 1 includes an edge controller 100 and a cooperative controller 200. The edge controller 100 is connected to the robot controller 30 by wire or wirelessly. The edge controller 100 generates an action instruction for causing the robot 20 to perform a prescribed operation based on the environmental information of the robot 20, and sends the action instruction to the robot controller 30. For example, the edge controller 100 is configured to perform the following actions: storing an operation model, the operation model responding to the input of the environmental information of the robot 20 and outputting the operation instruction information of the robot 20 corresponding to the environmental information; deriving the operation instruction information for the robot 20 based on the environmental information and the operation model of the robot 20; determining whether the operation of the robot 20 based on the operation instruction information can be performed based on the environmental information and the operation model of the robot 20; and requesting support information when it is determined that the operation of the robot 20 based on the operation instruction information cannot be performed.

[0026] When it is determined that the robot 20 can perform the operation based on the operation instruction information, the edge controller 100 generates an action command based on the operation instruction information and outputs it to the robot controller 30. When it is determined that the robot 20 cannot perform the operation based on the operation instruction information, the edge controller 100 generates an action command based on the support information and outputs it to the robot controller 30.

[0027] The cooperative controller 200 is connected to the plurality of teaching terminals 40 via a network line 50. The cooperative controller 200 may be connected to the edge controller 100 via a wired or wireless dedicated line, or may be connected to the edge controller 100 via a network line 50. The network line 50 may be a local area network or a wide area network (such as the Internet).

[0028] The cooperative controller 200 may also be configured to perform the following actions: store information of a plurality of teachers; select any one of the plurality of teachers in response to a request for support information from the edge controller 100; send environmental information of the robot 20 to the teaching terminal 40 of the selected teacher via the network line 50; receive support information based on the environmental information from the teaching terminal 40 of the selected teacher via the network line 50; and send the received support information to the edge controller 100. The specific structures of the edge controller 100 and the cooperative controller 200 are exemplified below.

[0029] (Edge Controller)

[0030] like Figure 2 As shown, the edge controller 100 has a model storage unit 111, an environment information acquisition unit 112, a work instruction derivation unit 113, a work instruction storage unit 114, a reliability derivation unit 115, a work feasibility determination unit 116, a support request unit 117, a support information receiving unit 118, an action instruction generation unit 121 and an action instruction sending unit 122 as a functional structure (hereinafter referred to as a "functional module").

[0031] The model storage unit 111 stores a plurality of the above-mentioned operation models corresponding to the plurality of robots 20, respectively. As described above, the operation model responds to the input of the environmental information of the robot 20, and outputs the operation instruction information of the robot 20 corresponding to the environmental information. The environmental information is information indicating the state of the surrounding environment of the robot 20. As a specific example of the environmental information, two-dimensional or three-dimensional image information and the like can be cited. The operation instruction information includes information required to determine the operation content, such as the operation object in the surrounding environment of the robot 20. In the case where there are multiple objects that may be operation objects in the surrounding environment of the robot 20, the operation instruction information may also include multiple numerical values ​​that respectively indicate the recommendation degree of the multiple objects as operation objects. The operation instruction information may also be the above-mentioned action instruction.

[0032] The operation model is constructed through machine learning. As a specific example of the operation model, a neural network can be cited in which the above-mentioned environmental information is set as an input vector and the above-mentioned operation instruction information is set as an output vector. The neural network has an input layer, one or more intermediate layers, and an output layer. The input layer outputs the input vector directly to the next intermediate layer. The intermediate layer converts the input from the previous layer through an activation function and outputs it to the next layer. The output layer converts the input from the intermediate layer farthest from the input layer through an activation function, and outputs the conversion result as an output vector. The operation model is not necessarily limited to a neural network as long as it outputs the operation instruction information in response to at least the input of environmental information. For example, the operation model can also be a function derived based on statistical analysis, etc.

[0033] The work model may also be configured to output work instruction information including the above-mentioned motion instructions. In this case, the work model may also include an algorithm that generates movement instructions for the end portion 22 by simulated random trials so that the robot 20 does not collide with surrounding objects.

[0034] The environment information acquisition unit 112 acquires the environment information of the robot 20 from the robot controller 30. The operation instruction deriving unit 113 derives the operation instruction information for the robot 20 based on the environment information of the robot 20 acquired by the environment information acquisition unit 112 and the operation model stored in the model storage unit 111. For example, the operation instruction deriving unit 113 inputs the environment information of the robot 20 into the operation model and obtains the operation instruction information outputted by the operation model in correspondence thereto. The operation instruction storage unit 114 stores the operation instruction information derived by the operation instruction deriving unit 113.

[0035] The reliability deriving unit 115 derives the reliability of the operation instruction information. For example, the reliability deriving unit 115 derives the reliability based on the difference in recommendation between the object with the highest recommendation as the operation object (hereinafter referred to as the "preferred object") and other objects. Specifically, the reliability deriving unit 115 increases the reliability as the difference in recommendation between the preferred object and other objects increases (i.e., as the recommendation of the preferred object increases relatively).

[0036] When the operation instruction information is the above-mentioned motion instruction, the reliability deriving unit 115 may further derive the reliability based on the motion of the robot 20 based on the motion instruction. For example, the reliability deriving unit 115 may derive the reliability based on the simulation of the motion of the robot 20 based on the motion instruction. For example, the reliability deriving unit 115 may reduce the reliability as the distance between the robot 20 and the surrounding objects decreases when the robot 20 in motion is closest to the surrounding objects.

[0037] The operation determination unit 116 determines whether the operation of the robot 20 based on the operation instruction information can be performed based on the environment information of the robot 20 obtained by the environment information acquisition unit 112 and the operation model stored in the model storage unit 111. For example, when the reliability derived by the reliability deriving unit 115 exceeds a predetermined threshold, the operation determination unit 116 sets the operation of the robot 20 based on the operation instruction information to "permitted" (allowed), and when the reliability is lower than the threshold, the operation determination unit 116 sets the operation of the robot 20 based on the operation instruction information to "unpermitted" (prohibited).

[0038] When the operation determination unit 116 sets the operation of the robot 20 based on the operation instruction information as "not possible", the support request unit 117 requests the cooperative controller 200 to send support information. When requesting to send the support information, the support request unit 117 sends the environment information of the robot 20 to the cooperative controller 200. When requesting to send the support information, the support request unit 117 may further send the difficulty of providing the support information to the cooperative controller 200. In this case, the support request unit 117 may also calculate the difficulty based on the reliability. For example, the lower the reliability, the higher the difficulty.

[0039] The support information receiving unit 118 receives support information sent by the cooperation controller 200 in response to a request from the support requesting unit 117. For example, the support information includes information necessary to specify the content of the work, similar to the work instruction information. The support information may also include an action instruction.

[0040] The motion instruction generation unit 121 generates the above-mentioned motion instructions according to the operation instruction information or the support information. For example, the motion instruction generation unit 121 generates a movement instruction of the terminal unit 22 and a motion instruction of a tool corresponding to the movement of the terminal unit 22 to perform the operation content determined by the operation instruction information or the support information. In the case where the operation instruction information and the support information include motion instructions, the motion instruction generation unit 121 selects the motion instruction based on the operation instruction information or the motion instruction based on the support information according to the determination result of the operation determination unit 116. The motion instruction transmission unit 122 transmits the motion instruction generated by the motion instruction generation unit 121 to the robot controller 30.

[0041] The edge controller 100 may also be configured to further perform the following actions: storing the environment information of the robot 20 and the work results of the robot 20 as learning data; and updating the work model of the model storage unit 111 through machine learning based on the stored learning data. In addition, the edge controller 100 may also be configured to: when the work model of any robot 20 is updated, the work models of other robots 20 are also made to reflect the updated content of the work model. For example, the edge controller 100 also has a performance information acquisition unit 123, a learning data storage unit 124, a model update unit 125, an update content reflection unit 126, and a performance information sending unit 127 as functional modules.

[0042] The performance information acquisition unit 123 acquires the work performance information related to the work result of the robot 20. For example, the work performance information includes work instruction information or support information, and success information indicating whether the work of the robot 20 that performed the operation according to the work instruction information or support information was successful. As a specific example of the success information, information indicating whether there is an alarm in the robot controller 30 can be cited.

[0043] The operation performance information may also include efficiency information indicating the operation efficiency of the robot 20 that has performed an operation according to the operation instruction information or the support information. As a specific example of the efficiency information, the operation time of the robot 20 measured by the robot controller 30 may be cited. For example, the performance information acquisition unit 123 acquires the operation instruction information or the support information from the action instruction generation unit 121, and acquires the success information and the efficiency information from the robot controller 30.

[0044] The learning data storage unit 124 stores the environment information of the robot 20 and the operation performance information of the robot 20 corresponding to the environment information as learning data. For example, the learning data storage unit 124 obtains the environment information of the robot 20 from the environment information acquisition unit 112, and obtains and stores the operation performance information of the robot 20 corresponding to the environment information from the performance information acquisition unit 123. The learning data storage unit 124 may also store the learning data for each robot 20. In other words, the learning data storage unit 124 may also store a plurality of learning data sets (a collection of learning data) corresponding to a plurality of robots 20, respectively.

[0045] The model updating unit 125 updates the operation model of the model storage unit 111 by machine learning (e.g., deep learning) based on the learning data stored in the learning data storage unit 124. For example, the model updating unit 125 updates the parameters of the operation model (e.g., the weight of the activation function) according to the learning data so as to improve the suitability of the operation instruction information output by the operation model in response to the input of the environmental information. More specifically, the model updating unit 125 updates the parameters of the operation model by repeatedly applying the environmental information of the robot 20 and the operation performance of the robot 20 corresponding to the environmental information to the operation model.

[0046] When the model updating unit 125 updates the operation model of any robot 20, the update content reflecting unit 126 also causes the operation model of any other robot 20 to reflect the updated content of the operation model. For example, when the model updating unit 125 updates the operation model of any robot 20, the update content reflecting unit 126 overwrites the operation model of other robots 20 that perform the same type of operation as the robot 20 with the updated operation model. The performance information transmitting unit 127 transmits the operation performance information acquired by the performance information acquiring unit 123 to the cooperative controller 200. As described later, the operation performance information transmitted by the performance information transmitting unit 127 is used for evaluation by the instructor.

[0047] (Collaborative Controller)

[0048] like Figure 3 As shown, the cooperative controller 200 has a teaching person storage unit 211, a support request acquisition unit 212, a teaching request unit 213, a selection unit 214, an environment information transmission unit 215, a support information reception unit 216 and a support information transmission unit 217 as functional modules.

[0049] The instructor storage unit 211 stores information of multiple instructors. The instructor information includes, for example, instructor identification information and access information to the instructor. The access information is information indicating the destination of the instructor's information, such as an email address. The instructor information may also include information on the types of jobs that the instructor can teach. The types of jobs that the instructor can teach are the types of jobs that the instructor can determine in which steps the robot 20 should follow to complete the job. The types of jobs that the instructor can teach are determined based on the instructor's professional knowledge or skills. For example, an instructor who has professional knowledge or skills in cutting but does not have professional knowledge or skills in welding can teach cutting but cannot teach welding.

[0050] The support request acquisition unit 212 acquires a request for sending support information from the edge controller 100 (support request unit 117). As described above, the request for sending support information includes environmental information of the robot 20. The request for sending support information may also include the difficulty of providing support information. When support information is required, the teaching request unit 213 requests multiple teachers stored in the teacher storage unit 211 to send support information via the network line 50. The situation where support information is required is the following situation: if there is no support information, the robot 20 cannot continue to work. As a specific example of the situation where support information is required, there can be cited a situation where the edge controller 100 sets the operation of the robot 20 based on the above-mentioned operation instruction information to "not possible". For example, when the support request acquisition unit 212 acquires a request for sending support information, the teaching request unit 213 publishes a message requesting the sending of support information to the teaching terminals 40 of the above-mentioned multiple teachers.

[0051] The selection unit 214 selects any one of the multiple teachers stored in the model storage unit 111 as the sender of the support information. The selection unit 214 may also select any one of the multiple teachers based on the information of the job category that the teacher can teach and the job category of the robot 20. For example, the selection unit 214 may also select a teacher who can teach the job category of the robot 20 from the above-mentioned multiple teachers. The selection unit 214 may also select any one of the multiple teachers based on the speed of response to the request from the teaching request unit 213. For example, the selection unit 214 may also preferentially select a teacher who responds faster to the request from the teaching request unit 213. In addition, the selection unit 214 may also select any one of the multiple teachers based on a combination of multiple selection criteria (for example, a selection criterion based on the above-mentioned job category and a selection criterion based on the response speed).

[0052] The environment information sending unit 215 sends the environment information of the robot 20 to the teacher selected by the selection unit 214 (hereinafter referred to as the "selected teacher") via the network line 50. For example, the environment information sending unit 215 sends the environment information received from the edge controller 100 (support request unit 117) to the teaching terminal 40 of the selected teacher. The teaching terminal 40 of the selected teacher displays the environment information of the robot 20, and sends the support information input by the selected teacher based on the environment information to the cooperation controller 200.

[0053] The support information receiving unit 216 receives support information from the selected teacher via the network line 50. For example, the support information receiving unit 216 receives support information sent by the teaching terminal 40 of the selected teacher. The support information sending unit 217 sends the support information received by the support information receiving unit 216 to the edge controller 100. The support information sent by the support information sending unit 217 is received by the support information receiving unit 118 described above.

[0054] The cooperative controller 200 may also be configured to further perform the following actions: deriving the suitability of the support information based on existing data before the robot 20 performs an operation based on the support information; and determining whether the support information can be adopted based on the suitability. For example, the cooperative controller 200 further includes a priori evaluation unit 221 and a support determination unit 222 as functional modules.

[0055] Before the robot 20 performs an operation based on the support information, the prior evaluation unit 221 derives the suitability of the support information (hereinafter referred to as "prior suitability") based on the existing data. As specific examples of the existing data, the support information itself as the object of the prior suitability derivation and the past performance of the selected instructor in sending support information (for example, the a posteriori suitability described below) can be cited. For example, the prior evaluation unit 221 derives the prior suitability based on whether the work object represented by the support information is an object suitable as the work object. For example, in the case where an object that is fixed in space and obviously cannot be transported is set as the work object to be transported, the prior evaluation unit 221 lowers the prior suitability.

[0056] The support availability determination unit 222 determines whether the support information can be adopted based on the prior suitability of the support information derived by the prior evaluation unit 221. For example, when the prior suitability of the support information exceeds a predetermined adoption threshold, the support availability determination unit 222 sets the support information as "adoptable", and when the prior suitability of the support information is lower than the adoption threshold, the support availability determination unit 222 sets the support information as "unadoptable". When the support availability determination unit 222 sets the support information as "unadoptable", the support information transmission unit 217 is prohibited from transmitting the support information.

[0057] The cooperative controller 200 may also be configured to further perform the following actions: after the robot 20 performs an operation based on the support information, derive the suitability of the support information based on the operation result of the robot 20; and derive the score of the instructor of the support information based on the suitability, and the cooperative controller 200 selects any one of the multiple instructors based on the score. For example, the cooperative controller 200 further includes a performance information receiving unit 231, a posteriori evaluation unit 232, and a score deriving unit 233 as functional modules.

[0058] The performance information receiving unit 231 receives the operation performance information indicating the operation result of the robot 20 based on the support information from the above-mentioned performance information sending unit 127. The a posteriori evaluation unit 232 derives the suitability of the support information (hereinafter referred to as "a posteriori suitability") based on the operation performance information received by the performance information receiving unit 231. For example, in the case where the above-mentioned success information indicates the success of the operation of the robot 20, the a posteriori evaluation unit 232 increases the a posteriori suitability compared to the case where the success information indicates the failure of the operation of the robot 20. In addition, the a posteriori evaluation unit 232 increases the a posteriori suitability as the operation efficiency indicated by the above-mentioned efficiency information increases.

[0059] The score derivation unit 233 derives the score of the instructor of the support information based on the a posteriori suitability derived by the a posteriori evaluation unit 232. For example, the score derivation unit 233 increases the score of the instructor as the a posteriori suitability increases.

[0060] The selection unit 214 may also select any one of the multiple teachers based on the score derived by the score deriving unit 233. For example, the selection unit 214 may also preferentially select a teacher with a high score derived by the score deriving unit 233. In this case, the selection unit 214 may also select any one of the multiple teachers based on a combination of a selection criterion based on the score and other selection criteria (for example, a selection criterion based on the above-mentioned job type and a selection criterion based on the response speed).

[0061] The cooperative controller 200 may also be configured to further perform the following actions: store a teaching performance value in association with a teacher, the teaching performance value being a basis for calculating the remuneration amount of the teacher related to the transmission of the support information; and update the teaching performance value of the teacher of the support information in accordance with the operation whenever the robot 20 based on the support information is executed. In addition, the cooperative controller 200 may also be configured to further perform the following actions: store a utilization performance value in association with the robot 20, the utilization performance value being a basis for calculating the billing amount corresponding to the utilization of the support information; and update the utilization performance value of the robot 20 in accordance with the operation whenever the robot 20 based on the support information is executed. For example, the cooperative controller 200 further has a teaching performance storage unit 241, a teaching performance update unit 242, a utilization performance storage unit 243, and a utilization performance update unit 244 as functional modules.

[0062] The teaching performance storage unit 241 stores the above-mentioned teaching performance value in association with the instructor. For example, the teaching performance value is a value indicating the performance of sending support information for enabling the robot 20 to continue working. The teaching performance value may include the evaluation result of the a posteriori fitness of the a posteriori evaluation unit 232, and may also include the cumulative value of the a posteriori fitness. In this case, for example, the score derivation unit 233 may also refer to the teaching performance storage unit 241 and derive the score of the instructor based on the teaching performance value. The score derivation unit 233 may also use the teaching performance value itself as the instructor's score, or may perform a prescribed operation on the teaching performance value to calculate the instructor's score.

[0063] Whenever a task of the robot 20 based on the support information is performed, the teaching performance update unit 242 updates the teaching performance value of the instructor of the support information in accordance with the task. For example, whenever the a posteriori evaluation unit 232 derives the a posteriori suitability of the support information, the teaching performance update unit 242 adds the value related to the a posteriori suitability (hereinafter referred to as the "added value") and the teaching performance value of the instructor of the support information. Therefore, whenever a task of the robot 20 based on the support information is performed, the teaching performance value of the instructor of the support information increases, and the higher the a posteriori suitability, the greater the increase. The teaching performance update unit 242 may also change the added value corresponding to the teaching performance value of the instructor of the support information according to the difficulty of providing the support information. For example, the higher the difficulty of providing the support information, the more the teaching performance update unit 242 increases the added value.

[0064] The utilization performance storage unit 243 stores the utilization performance value in association with the robot 20. The utilization performance value may also include the evaluation result of the suitability of the a posteriori evaluation unit 232. Whenever the robot 20 performs a task based on the support information, the utilization performance update unit 244 updates the utilization performance value of the robot 20 in accordance with the task. For example, whenever the a posteriori evaluation unit 232 derives the a posteriori suitability of the support information, the utilization performance update unit 244 adds a value related to the a posteriori suitability (hereinafter referred to as an "additional value") and the utilization performance value of the robot 20 that uses the support information. Therefore, whenever the robot 20 performs a task based on the support information, the utilization performance value of the robot 20 increases, and the higher the a posteriori suitability, the greater the increase. The utilization performance update unit 244 may also change the addition value corresponding to the utilization performance value of the robot 20 that uses the support information according to the difficulty of providing the support information. For example, the utilization performance updating unit 244 may increase the added value as the difficulty level of providing the support information increases.

[0065] (Hardware structure of edge controller and collaborative controller)

[0066] Figure 41 is a block diagram illustrating the hardware structure of the edge controller 100 and the cooperative controller 200. The edge controller 100 has a circuit 190. The circuit 190 includes at least one processor 191, a memory 192, a storage 193, and a communication port 194. The storage 193 is, for example, a storage medium such as at least one hard disk or a non-volatile memory. For example, the storage 193 stores a program that causes the edge controller 100 to perform the following actions: storing a work model, the work model responding to the input of the environment information of the robot 20 and outputting the work instruction information of the robot 20 corresponding to the environment information; deriving the work instruction information for the robot 20 based on the environment information and the work model of the robot 20; determining whether the work of the robot 20 based on the work instruction information can be performed based on the environment information and the work model of the robot 20; and requesting support information when it is determined that the work of the robot 20 based on the work instruction information cannot be performed. For example, the storage 193 stores a program for constituting the functional module of the above-mentioned edge controller 100. The memory 192 temporarily stores the program loaded from the storage 193 and the calculation result of the processor 191, etc. The processor 191 executes the above-mentioned program in cooperation with the memory 192 to form each functional module of the edge controller 100. The communication port 194 performs network communication between the robot controller 30 and the cooperative controller 200 according to the instruction from the processor 191.

[0067] The cooperative controller 200 has a circuit 290. The circuit 290 includes at least one processor 291, a memory 292, a storage 293, and a communication port 294. The storage 293 is, for example, a storage medium such as at least one hard disk or a non-volatile memory. The storage 293 stores a program that enables the cooperative controller 200 to perform the following actions: store information of multiple teachers; select any one of the multiple teachers in response to a request for support information from the edge controller 100; send environmental information of the robot 20 to the teaching terminal 40 of the selected teacher via the network line 50; receive support information based on the environmental information from the teaching terminal 40 of the selected teacher via the network line 50; and send the received support information to the edge controller 100. For example, the storage 293 stores a program for constituting the functional module of the above-mentioned cooperative controller 200. The memory 292 temporarily stores the program loaded from the storage 293 and the calculation result of the processor 291. The processor 291 executes the above-mentioned program in cooperation with the memory 292 to form each functional module of the collaborative controller 200. The communication port 294 performs network communication with the teaching terminal 40 and the edge controller 100 in response to the instruction from the processor 291. The storage body 293 may not necessarily be built into the collaborative controller 200. For example, the storage body 293 may also be an external storage body (such as a cloud) connected to the communication port 294 via the network line 50.

[0068] [Robot control steps]

[0069] Next, as an example of a robot control method, a control step performed by the robot control system 1 is described. The control step includes an output step of an action command performed by the edge controller 100, an update step of a work model performed by the edge controller 100, and a receiving step of support information performed by the cooperative controller 200. Each step is exemplified in detail below.

[0070] (Output steps of action instructions)

[0071] The steps of outputting the action instructions include the following: deriving the operation instruction information for the robot 20 based on the environmental information of the robot 20 and the above-mentioned operation model; judging whether the operation of the robot 20 based on the operation instruction information can be performed based on the environmental information and the operation model of the robot 20; if it is judged that the operation of the robot 20 based on the operation instruction information can be performed, generating the action instructions based on the operation instruction information and outputting it to the robot controller 30; if it is judged that the operation of the robot 20 based on the operation instruction information cannot be performed, requesting the support information, generating the action instructions based on the support information and outputting it to the robot controller 30.

[0072] Figure 5 1 is a flowchart illustrating the steps of outputting an action instruction executed by the edge controller 100 to a robot 20. Figure 5 As shown, the edge controller 100 first executes steps S01 and S02. In step S01, the environment information acquisition unit 112 acquires the environment information of the robot 20 from the robot controller 30. In step S02, the operation instruction deriving unit 113 derives the operation instruction information of the robot 20 based on the environment information of the robot 20 acquired by the environment information acquisition unit 112 and the operation model stored in the model storage unit 111. For example, the operation instruction deriving unit 113 inputs the environment information of the robot 20 into the operation model, obtains the operation instruction information outputted by the operation model accordingly, and writes it into the operation instruction storage unit 114.

[0073] Next, the edge controller 100 executes steps S03 and S04. In step S03, the reliability deriving unit 115 derives the reliability of the operation instruction information. In step S04, the operation feasibility determination unit 116 determines whether the operation of the robot 20 based on the operation instruction information can be performed based on the environmental information of the robot 20 obtained by the environmental information obtaining unit 112 and the operation model stored in the model storage unit 111. For example, when the reliability derived by the reliability deriving unit 115 exceeds a predetermined threshold, the operation feasibility determination unit 116 determines that the operation of the robot 20 based on the operation instruction information can be performed, and when the reliability is lower than the threshold, the operation feasibility determination unit 116 determines that the operation of the robot 20 based on the operation instruction information cannot be performed.

[0074] If it is determined in step S04 that the robot operation based on the operation instruction information can be performed, the edge controller 100 executes step S05. In step S05, the motion instruction generation unit 121 generates a motion instruction for the robot 20 based on the operation instruction information. For example, the motion instruction generation unit 121 generates a movement instruction for the end portion 22 and a motion instruction for a tool corresponding to the movement of the end portion 22 so that the operation content determined by the operation instruction information is performed.

[0075] When it is determined in step S04 that the robot operation based on the operation instruction information can be performed, the edge controller 100 executes steps S06, S07, and S08. In step S06, the support request unit 117 requests the cooperative controller 200 to send support information. In addition, the support request unit 117 sends environmental information of the robot 20 for the instructor to refer to to the cooperative controller 200. In step S07, the support information receiving unit 118 receives the support information sent by the cooperative controller 200 in response to the request from the support request unit 117. In step S08, the action instruction generation unit 121 generates an action instruction for the robot 20 based on the support information. For example, the action instruction generation unit 121 generates a movement instruction for the terminal unit 22 and an action instruction for the tool corresponding to the movement of the terminal unit 22, so that the work content determined by the support information is performed.

[0076] After steps S05 and S08, the edge controller 100 executes step S09. In step S09, the motion instruction sending unit 122 outputs the motion instruction generated by the motion instruction generating unit 121 to the robot controller 30. The robot controller 30 controls the robot 20 according to the motion instruction output from the motion instruction sending unit 122. As a result, the robot 20 performs the operation based on the operation instruction information or the support information. At this point, the output step of the motion instruction is completed. The edge controller 100 repeats the above steps for each of the multiple robots 20.

[0077] (Steps to update the job model)

[0078] The updating step of the operation model includes the following steps: storing the environmental information of the robot 20 and the operation results of the robot 20 as learning data; updating the operation model of the model storage unit 111 through machine learning based on the stored learning data; and making the operation models of other robots 20 also reflect the updated content of the operation model of the robot 20.

[0079] Figure 6 2 is a flowchart illustrating the update steps performed by the edge controller 100 on the operation model of a robot 20. Figure 6 As shown, the edge controller 100 first executes steps S11, S12, and S13. In step S11, the performance information acquisition unit 123 waits for the action instruction generation unit 121 to generate the above-mentioned action instruction. In step S12, the performance information acquisition unit 123 acquires the work performance information related to the work result of the robot 20 based on the above-mentioned action instruction. For example, the performance information acquisition unit 123 acquires the work instruction information or support information from the action instruction generation unit 121, and acquires the above-mentioned success information and efficiency information from the robot controller 30. In step S13, the performance information sending unit 127 sends the work performance information acquired by the performance information acquisition unit 123 to the collaborative controller 200.

[0080] Next, the edge controller 100 executes steps S14 and S15. In step S14, the learning data storage unit 124 stores the environment information of the robot 20 and the operation performance information of the robot 20 corresponding to the environment information as the above-mentioned learning data. For example, the learning data storage unit 124 obtains the environment information of the robot 20 from the environment information acquisition unit 112, obtains the operation performance information of the robot 20 corresponding to the environment information from the performance information acquisition unit 123, and stores these information as learning data. In step S15, the model update unit 125 confirms whether the number of data stored in the learning data storage unit 124 (the number of environment information and operation performance information of the robot 20) exceeds the prescribed learnable number. The number of data here is the number of data stored after the latest operation model (the operation model stored in the model storage unit 111) is obtained.

[0081] If it is determined in step S15 that the amount of data does not exceed the prescribed learnable amount, the edge controller 100 returns the process to step S11. If it is determined in step S15 that the amount of data exceeds the prescribed learnable amount, the edge controller 100 executes steps S16 and S17. In step S16, the model updating unit 125 updates the operation model of the model storage unit 111 through machine learning based on the learning data stored in the learning data storage unit 124. In step S17, the update content reflection unit 126 causes the operation model of any other robot 20 to reflect the updated content of the operation model of the robot 20 in step S16. The edge controller 100 repeats the above steps for each operation model of multiple robots 20.

[0082] (Steps for receiving support information)

[0083] The step of receiving support information includes the following steps: in response to a request for support information from the edge controller 100, selecting any one of a plurality of instructors; sending environmental information of the robot 20 to the teaching terminal 40 of the selected instructor via the network line 50; receiving support information based on the environmental information from the teaching terminal 40 of the selected instructor via the network line 50; and sending the received support information to the edge controller 100.

[0084] For example, Figure 7 As shown, the collaborative controller 200 executes steps S21, S22, S23, and S24. In step S21, the support request acquisition unit 212 acquires a request for sending support information from the support request unit 117. In step S22, the teaching request unit 213 extracts a teacher who can teach the operation category of the robot 20 that is the object of the support information from the multiple teachers stored in the teacher storage unit 211. Hereinafter, the teacher extracted by the teaching request unit 213 is referred to as a "candidate teacher". In step S23, the teaching request unit 213 requests the candidate teacher extracted in step S22 to send support information via the network line 50. In step S24, the score derivation unit 233 waits for a response from the candidate teacher. The score derivation unit 233 may also wait for responses from multiple candidate teachers.

[0085] Next, the collaborative controller 200 executes steps S25 and S26. In step S25, the score derivation unit 233 derives the score of the candidate teacher who responded to the teaching request based on the above-mentioned a posteriori suitability previously derived by the a posteriori evaluation unit 232. For example, the score derivation unit 233 refers to the teaching performance storage unit 241 and derives the score of the candidate teacher based on the above-mentioned teaching performance value. In step S26, the selection unit 214 selects any one of the candidate teachers as the sender of the support information. For example, the selection unit 214 can also select any one of the multiple teachers based on the score derived by the score derivation unit 233.

[0086] Next, the cooperative controller 200 executes steps S27 and S28. The selection unit 214 may also select any one of the multiple teachers based on a combination of a selection criterion based on the score and other selection criteria (for example, a selection criterion based on the above-mentioned job category and a selection criterion based on the response speed). In step S27, the environment information sending unit 215 sends the environment information of the robot 20 to the selected teacher (the teacher selected by the selection unit 214) via the network line 50. In step S28, the support information receiving unit 216 receives support information from the selected teacher via the network line 50.

[0087] Next, the cooperative controller 200 executes steps S31 and S32. In step S31, the prior evaluation unit 221 derives the a priori suitability of the support information based on the existing data. In step S32, the support availability determination unit 222 determines whether the support information can be adopted based on the a priori suitability derived by the prior evaluation unit 221. For example, the support availability determination unit 222 determines whether the a priori suitability exceeds a prescribed adoption threshold.

[0088] If it is determined in step S32 that the support information is not adoptable, the cooperation controller 200 returns the process to step S24. Thereafter, until the support availability determination unit 222 determines that the support information is adoptable, the acquisition of support information and the determination of availability based on the a priori suitability are repeated.

[0089] If it is determined in step S32 that the support information is adoptable, the cooperative controller 200 executes steps S33 and S34. In step S33, the support information transmitting unit 217 transmits the support information determined to be adoptable to the edge controller 100. In step S34, the performance information receiving unit 231 receives the work performance information indicating the work results of the robot 20 based on the support information from the performance information transmitting unit 127.

[0090] Next, the collaborative controller 200 executes steps S35 and S36. In step S35, the a posteriori evaluation unit 232 derives the above-mentioned a posteriori suitability of the support information based on the work performance information received by the performance information receiving unit 231. In step S36, the teaching performance update unit 242 adds the value related to the a posteriori suitability of the support information and the teaching performance value of the teacher of the support information, and the utilization performance update unit 244 adds the value related to the a posteriori suitability of the support information and the utilization performance value of the robot 20 that utilizes the support information. At this point, the support information receiving step is completed. The collaborative controller 200 repeats the above steps.

[0091] [Effects of this embodiment]

[0092] As described above, the robot control system 1 has: a teacher storage unit 211, which stores information of multiple teachers; a selection unit 214, which selects any one of the multiple teachers stored in the teacher storage unit 211 when support information is required in order for the robot 20 to continue working; an environment information sending unit 215, which sends the environment information of the robot 20 to the teacher selected by the selection unit 214 via the network line 50; and a support information receiving unit 216, which receives support information from the teacher 50 selected by the selection unit 214 via the network line.

[0093] According to the robot control system 1, it is possible to receive support information from a plurality of instructors via the network line 50. Therefore, even if there is no instructor waiting all the time, support information can be obtained earlier and the operation of the robot 20 can be continued. In addition, since each instructor does not need to wait near the robot 20 all the time, each instructor can complete his or her own operation. Therefore, it is conducive to efficient cooperation between the robot 20 and the human.

[0094] The robot control system 1 may further include: a priori evaluation unit 221 that derives the suitability of the support information based on existing data before the robot 20 performs an operation based on the support information; and a support availability determination unit 222 that determines whether the support information can be adopted based on the suitability of the support information derived by the priori evaluation unit 221. In this case, by widely accepting support information from a plurality of instructors and setting support information with a low suitability as unadoptable, the reliability of the operation of the robot 20 based on the support information can be improved.

[0095] Alternatively, the robot control system 1 further includes: a posteriori evaluation unit 232 that derives the suitability of the support information based on the operation result of the robot 20 after the operation of the robot 20 based on the support information; and a score deriving unit 233 that derives the score of the instructor of the support information based on the suitability of the support information derived by the posteriori evaluation unit 232, and the selection unit 214 selects any one of the plurality of instructors based on the score derived by the score deriving unit 233. In this case, by widely accepting the provision of support information from a plurality of instructors and giving priority to instructors who tend to provide support information with high suitability, the reliability of the operation of the robot 20 based on the support information can be improved.

[0096] Alternatively, the information of the teacher stored in the teacher storage unit 211 includes information of the type of work that can be taught, and the selection unit 214 selects any one of the plurality of teachers based on the information of the type of work that can be taught and the type of work of the robot 20. In this case, support information is widely provided from a plurality of teachers, and a teacher matching the type of work to which the support information is applied is given priority, thereby further improving the reliability of the work of the robot 20 based on the support information.

[0097] Alternatively, the robot control system 1 further includes a teaching request unit 213. When support information is needed, the teaching request unit 213 requests multiple teachers stored in the teaching storage unit 211 to send support information via the network line 50. The selection unit 214 selects any one of the multiple teachers according to the speed of responding to the request from the teaching request unit 213. In this case, by giving priority to the teacher with a faster response, the teacher can be urged to provide support information more quickly. Therefore, the operating efficiency of the robot 20 can be further improved.

[0098] The robot control system 1 may also include: a teaching performance storage unit 241, which stores a teaching performance value in association with a teacher, the teaching performance value being a calculation basis for the amount of remuneration related to the sending of support information; and a teaching performance update unit 242, which updates the teaching performance value of the teacher of the support information in accordance with the operation whenever the robot 20 performs an operation based on the support information. In this case, by associating the teaching performance with the amount of remuneration, the teacher can be encouraged to provide support information more actively. Therefore, the operating efficiency of the robot 20 can be further improved.

[0099] The robot control system 1 may further include: a utilization performance storage unit 243 that stores a utilization performance value in association with the robot 20, the utilization performance value being a basis for calculating a charge amount corresponding to the utilization of the support information; and a utilization performance update unit 244 that updates the utilization performance value of the robot 20 in accordance with the operation whenever the robot 20 performs an operation based on the support information. In this case, effective utilization of the support information can be promoted through reasonable charging corresponding to the utilization performance.

[0100] The robot control system 1 may further include: a model storage unit 111 storing a work model, the work model outputting work instruction information of the robot 20 corresponding to the environment information in response to the input of the environment information of the robot 20; a work instruction deriving unit 113 deriving the work instruction information of the robot 20 based on the environment information of the robot 20 and the work model stored in the model storage unit 111; a work availability determination unit 116 determining whether the work of the robot 20 based on the work instruction information can be performed based on the environment information of the robot 20 and the work model stored in the model storage unit 111; and a support request unit 117 requesting support information when the work availability determination unit 116 determines that the work of the robot 20 based on the work instruction information cannot be performed. In this case, the burden on the instructor can be further reduced by effectively utilizing the autonomous action of the robot 20 and reducing the situation where support information is required.

[0101] The robot control system 1 may further include: a learning data storage unit 124 that stores environmental information of the robot 20 and work results of the robot 20 as learning data; and a model updating unit 125 that updates the work model through machine learning based on the learning data stored in the learning data storage unit 124. In this case, the autonomy of the robot 20 is improved through machine learning, thereby gradually reducing the burden on the instructor.

[0102] The robot control system 1 may further include an update content reflecting unit 126. When the model updating unit 125 updates any one operation model, the update content reflecting unit 126 causes the operation models of other robots 20 of the robot 20 to also reflect the update content of the operation model. In this case, by sharing the latest operation model, the autonomy of each robot 20 can be quickly improved.

[0103] Although the embodiments have been described above, the present invention is not necessarily limited to the above-described embodiments, and various modifications can be made without departing from the gist of the invention.

[0104] Description of symbols

[0105] 1: robot control system; 20: robot; 50: network line; 111: model storage unit; 113: operation instruction derivation unit; 116: operation feasibility determination unit; 117: support request unit; 124: learning data storage unit; 125: model update unit; 126: update content reflection unit; 211: instructor storage unit; 213: teaching request unit; 214: selection unit; 215: environment information sending unit; 216: support information receiving unit; 221: prior evaluation unit; 222: support feasibility determination unit; 232: a posteriori evaluation unit; 233: score derivation unit; 241: teaching performance storage unit; 242: teaching performance update unit; 243: utilization performance storage unit; 244: utilization performance update unit.

Claims

1. A robot control system, comprising: A work instruction deriving unit, which derives work instruction information for the robot to continue working based on the environmental information and the work model, wherein the work model outputs instruction data representing the work instruction information in response to the input of the environmental information, wherein The job instruction information includes a designation of a job object of the job, and the instruction data includes a first recommendation degree of a first object as the job object and a second recommendation degree of a second object as the job object; a reliability deriving unit for evaluating the reliability of the work instruction information based on a difference between the first recommendation degree and the second recommendation degree, a determination unit for determining whether it is necessary to support the robot to continue the operation according to the reliability of the operation instruction information; a selection unit, which selects a remote instructor from a plurality of remote instructor candidate data based on the stored instructor data when it is determined that the robot needs to be supported to continue the operation; an environment information sending unit, which sends the environment information to the selected remote instructor via a network line; a support information receiving unit for receiving support information from the selected remote instructor via a network line; a control unit configured to control the robot to continue the operation based on the support information; as well as A model updating unit updates the operation model by machine learning based on learning data, wherein the learning data includes a combination of the environment information and the operation results of the robot based on the operation instruction information and a combination of the environment information and the operation results of the robot based on the support information.

2. The robot control system according to claim 1, wherein: The robot control system also has: an a priori evaluation unit that derives a priori suitability of the support information before executing the task based on the support information; and The support possibility determination unit does not adopt the support information when the priori suitability is lower than a suitability threshold value.

3. The robot control system according to claim 2, wherein: The selection unit selects another remote teacher from the plurality of remote teacher candidates based on the teacher data when the prior suitability is lower than the suitability threshold. The environment information sending unit sends the environment information to the selected other remote teachers, The support information receiving unit receives other support information based on the environmental information from the selected other remote teachers via the network line. The control unit controls the robot to continue the task based on the other support information.

4. The robot control system according to claim 2, wherein: The selection unit, the environmental information sending unit, the support information receiving unit, and the prior evaluation unit repeatedly perform additional selection processing until it is determined that the prior suitability is greater than the suitability threshold when the prior suitability is lower than the suitability threshold. The additional selection process includes: selecting other remote teachers from the plurality of candidate remote teachers according to the teacher data; Sending the environmental information to the selected other remote teachers; receiving other support information based on the environmental information from the selected other remote instructors via the network line; and Assessing the a priori suitability of the other supporting information, After repeatedly performing the additional selection process, the control unit controls the robot to continue the operation based on the other support information.

5. The robot control system according to claim 2, wherein: The support information includes a designation of a task object of the task, The a priori evaluation unit evaluates the a priori suitability based on suitability of a work object of one of the works.

6. The robot control system according to any one of claims 1 to 5, wherein: The teaching teacher data includes scores of the plurality of candidate remote teaching teachers. The selection unit selects the remote teaching teacher from among the plurality of remote teaching teacher candidates based on at least the scores of the plurality of remote teaching teacher candidates. The robot control system also has: an a posteriori evaluation unit that derives a posteriori suitability of the support information based on a work result of the work based on the support information; and A score deriving unit updates the score of the selected remote teacher based on the a posteriori suitability.

7. The robot control system according to any one of claims 1 to 5, wherein: The teaching teacher data includes the types of tasks taught by each of the plurality of candidate remote teachers. The selection unit selects the remote teacher based on a comparison between the job type of the job and the job type taught by each of the plurality of remote teacher candidates.

8. The robot control system according to any one of claims 1 to 5, wherein: The robot control system further includes a teaching request unit, which sends an instruction request to the plurality of candidate remote teachers via the network line before selecting the remote teacher. The selection unit selects the remote teacher from among two or more remote teachers who have responded to the instruction request via the network line.

9. The robot control system according to claim 8, wherein: The selection unit further selects the remote teacher from among two or more remote teachers based on a response speed to the instruction request.

10. The robot control system according to claim 1, wherein: The teaching data includes the types of tasks taught by multiple remote teachers. The robot control system further comprises a teaching request unit, Before selecting the remote teaching teacher from the plurality of remote teaching teachers, the teaching request unit extracts the plurality of remote teaching teachers from the plurality of remote teaching teachers based on a comparison between the type of the work and the types of the teaching work of each of the plurality of remote teaching teachers.

11. The robot control system according to claim 1, wherein: When it is determined that the reliability of the operation instruction information is higher than a reliability threshold, the control unit controls the robot to continue the operation based on the operation instruction information. The determination unit determines that the robot needs to be supported when the reliability of the operation instruction information is lower than the reliability threshold.

12. A robot control method, comprising: deriving work instruction information for the robot to continue the work based on the environment information and the work model, the work model outputting instruction data representing the work instruction information in response to the input of the environment information, wherein the work instruction information includes a designation of a work object of the work, and the instruction data includes a first recommendation degree of a first object as the work object and a second recommendation degree of a second object as the work object; evaluating the reliability of the operation instruction information according to a difference between the first recommendation degree and the second recommendation degree; Determining whether it is necessary to support the robot to continue working according to the reliability of the working instruction information; Selecting a remote instructor from a plurality of candidate remote instructors according to the stored instructor data; Sending the environmental information to the selected remote instructor via a network line; receiving support information based on the environmental information from the selected remote instructor via the network line; controlling the robot to continue the operation based on the support information; and The operation model is updated by machine learning based on learning data, wherein the learning data includes a combination of the environment information and an operation result of the robot based on the operation instruction information and a combination of the environment information and an operation result of the robot based on the support information.

13. The robot control method according to claim 12, wherein: The robot control method further comprises: Before executing the task based on the support information, evaluating the a priori suitability of the support information; and When the prior suitability is lower than a suitability threshold, the support information is not used.

Images