Robot control system and control device

Abstract

To provide a robot control system that enables an operator to easily execute desired processing, and a control device.SOLUTION: A robot control system 1 comprises a robot 200, at least one camera 300 and a control device 100. The control device identifies a posture of a partial body or a posture of a whole body of an operator on the basis of an image photographed with the at least one camera, and makes the robot execute processing according to the postures.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for controlling a robot. [Background technology]

[0002] A technique for controlling a robot based on an image captured by a camera has been known for some time. For example, Japanese Patent Application Laid-Open No. 2014-104527 (Patent Document 1) discloses a robot system, a program, a production system, and a robot. According to Patent Document 1, the robot system includes a robot that performs production work together with a worker in a production system, an imaging information acquisition unit that acquires imaging information from an imaging unit that images the worker, a robot control unit that controls the robot based on the imaging information, and a display control unit that performs display control on a display unit that displays a display image. First, the robot control unit detects a gesture of the worker based on the acquired imaging information, and identifies a robot control command associated with the detected gesture. Then, the display control unit displays a notification image on the display unit that notifies the worker of the robot control command identified by the robot control unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-104527 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a robot control system and a control device that can easily execute processing desired by an operator. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided a robot control system including a robot, at least one camera, and a control device. The control device identifies a posture of a part or an entire body of a worker based on an image of the at least one camera, and causes the robot to execute a process according to the posture. Effect of the Invention

[0006] As described above, according to the present invention, it is possible to provide a robot control system and a control device that allow an operator to easily execute a process that he or she desires. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing an overall configuration of a robot control system according to a first embodiment. [Diagram 2] FIG. 2 is a conceptual diagram showing correspondence relationship data according to the first embodiment. [Diagram 3] 5 is a flowchart showing information processing for robot control according to the first embodiment. [Figure 4] FIG. 2 is a conceptual diagram showing an image for identifying a posture of a worker according to the first embodiment. [Diagram 5] FIG. 11 is a conceptual diagram showing correspondence relationship data according to the second embodiment. [Figure 6] 10 is a flowchart showing information processing for robot control according to a second embodiment. [Figure 7] FIG. 13 is an image diagram showing a screen of a control device according to a third embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing correspondence relationship data according to the fourth embodiment. [Figure 9] FIG. 13 is an image diagram showing a screen of a control device according to a fourth embodiment. [Figure 10] 13 is a flowchart showing information processing for robot control according to a fourth embodiment. [Figure 11] FIG. 13 is a conceptual diagram showing correspondence relationship data according to the fifth embodiment. [Figure 12] FIG. 13 is a conceptual diagram showing worker information data according to the fifth embodiment. [Figure 13] 13 is a flowchart showing information processing for robot control according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. <First embodiment> <Overall configuration of the robot control system>

[0009] First, the overall configuration of a robot control system 1 according to this embodiment will be described with reference to Fig. 1. The robot control system 1 includes, as main devices, a robot 200, one or more cameras 300, 300..., and a control device 100 for controlling the operation of the robot 200 based on the captured images. Note that multiple robots 200 may also be prepared.

[0010] The robot control system 1 according to this embodiment is applied to, for example, a production site in a factory, and is configured to cause the robot 200 to execute a predetermined task at the production site. Moreover, in the robot control system 1 according to this embodiment, the robot 200 is not separated by a fence or the like, humans can access the working area of ​​the robot 200, and humans and the robot 200 work together to advance the work.

[0011] The one or more cameras 300... may be cameras attached to the robot 200, cameras fixed to a workbench or ceiling, or wearable cameras attached to a worker's body, work clothes, glasses, hat, helmet, etc.

[0012] The control device 100 grasps the position of the part and the current situation based on the images captured by the cameras 300, 300, ..., and causes the robot 200 to execute various tasks. A task may be, for example, a process of moving a workpiece from one location to another location, or a process of handing over a tool appropriate for the workpiece W to a worker.

[0013] The control device 100 mainly includes a CPU 110, a memory 120, a display 130, an operation unit 140, a speaker 150, and a communication interface 160. The CPU 110 controls the robot 200 and each unit of the control device 100 by executing a program stored in the memory 120. For example, the CPU 110 executes a program stored in the memory 120 and performs various information processing, which will be described later, by referring to various types of data.

[0014] The memory 120 is realized by various RAMs, various ROMs, etc. The memory 120 stores programs such as tasks of the robot 200 executed by the CPU 110, and data generated by the execution of the programs by the CPU 110, such as the operating state, current position, posture, and target position of the robot 200.

[0015] Particularly, in this embodiment, the memory 120 stores correspondence data 121 as shown in Fig. 2. The correspondence data 121 stores correspondences between conditions related to the posture of the worker, other associated conditions, and processes to be executed by the robot 200.

[0016] Returning to FIG. 1, the display 130 displays text and images based on signals from the CPU 110 .

[0017] The operation unit 140 receives instructions from an operator and inputs them to the CPU 110 .

[0018] The speaker 150 outputs various sounds based on signals from the CPU 110 .

[0019] The display 130, the operation unit 140, and the speaker 150 may be realized by other terminals.

[0020] The communication interface 160 is realized by a connector, an antenna, etc., and exchanges various data with other devices such as the robot 200 and the cameras 300, 300, . . . via a communication cable, wireless LAN, etc.

[0021] In this way, the CPU 110 of the control device 100, in accordance with the robot control program in the memory 120, causes the robot 200 to perform various operations appropriate to the worker's current posture and movement based on the images acquired from the cameras 300, 300, etc. via the communication interface 160. <Information processing by the control device 100>

[0022] The information processing of the control device 100 in this embodiment will be described in detail below with reference to Fig. 3. The CPU 110 of the control device 100 reads out, for example, a program for causing the robot 200 to execute a task in accordance with the program in the memory 120, and executes the following processes.

[0023] First, the CPU 110 acquires images captured by the cameras 300, 300, . . . via the communication interface 160 (step S102).

[0024] As shown in FIG. 4, the CPU 110 identifies the coordinates of each part of the worker's body based on the captured image (step S104).

[0025] For example, when using a three-dimensional camera such as an RGB-D camera, the CPU 110 calculates the three-dimensional coordinates of each part of the worker's body, the workpiece W, and parts as the coordinates of the camera 300 by adding depth information to the two-dimensional coordinates obtained above.

[0026] When a two-dimensional camera is used, the CPU 110 can detect the same point using multiple cameras 300, 300..., and therefore calculates the three-dimensional coordinates of each part of the worker's body, the workpiece W, and parts by triangulation or the like.

[0027] Then, the CPU 110 identifies the posture of a part or the whole of the worker's body (step S106). For example, the CPU 110 calculates the angle of the worker's spine from the vertical direction, the absolute angle of the working arm, the relative angle between the bones of the upper arm and the bones of the forearm, the relative angle between the bones of the forearm and the back of the hand, the distance between the right arm and the left arm, etc.

[0028] The CPU 110 refers to the correspondence data 121 and determines whether or not a process corresponding to the currently specified posture of a part or the whole of the worker's body is registered (step S108).

[0029] If a process corresponding to the posture of part or all of the worker's body identified this time is registered (YES in step S108), the CPU 110 refers to the correspondence data 121 and determines whether other associated conditions are satisfied based on the images captured by the cameras 300, 300, etc., the content of the task currently being executed by the robot 200, etc. (step S110).

[0030] If the other associated conditions are met (YES in step S110), CPU 110 identifies the corresponding process (step S112).

[0031] The CPU 110 transmits a control command to the robot 200 via the communication interface 160 (step S114).

[0032] In response to an instruction from the control device 100, the robot 200 executes a task. <Second embodiment>

[0033] In the above embodiment, the process is executed based on the posture of a part or the whole of the worker's body. In the present embodiment, the process is specified based on the relative position between the position of a part of the worker's body or the position of the whole of the worker's body and the position of the workpiece and / or component.

[0034] In this embodiment, the memory 120 of the control device 100 stores correspondence data 122 as shown in Fig. 5. The correspondence data 122 stores correspondences between identifying information of a body part of a worker, identifying information of a part, a relative position of the part with respect to the body part of the worker, other associated conditions, and processes to be executed by the robot 200.

[0035] In this embodiment, the CPU 110 of the control device 100 reads out, for example, a program for causing the robot 200 to execute a task in accordance with the program in the memory 120, and executes the processing shown in FIG.

[0036] First, the CPU 110 acquires images captured by the cameras 300, 300, . . . via the communication interface 160 (step S202).

[0037] As shown in FIG. 4, the CPU 110 identifies the type and position of each part of the worker's body, and the type, model number and position of the part, based on the captured image (step S204).

[0038] Then, CPU 110 calculates the relative position of the part with respect to the position of each part of the worker's body (step S206). Note that the relative position of the part of the part of the worker's body may be calculated.

[0039] The CPU 110 refers to the correspondence data 122 and determines whether or not a process corresponding to the relative position of the part with respect to the position of each part of the worker's body is registered (step S208).

[0040] If a process corresponding to the relative position of the part with respect to the positions of each part of the worker's body is registered (YES in step S208), the CPU 110 refers to the correspondence data 122 and determines whether other associated conditions are satisfied based on the content of the task currently being executed by the robot 200, etc. (step S210).

[0041] If the other associated conditions are met (YES in step S210), CPU 110 identifies the corresponding process (step S212).

[0042] The CPU 110 transmits a control command to the robot 200 via the communication interface 160 (step S214). <Third embodiment>

[0043] It is preferable that the correspondence relationship according to the above embodiment can be freely set by an operator. More specifically, the CPU 110 of the control device 100 displays a screen for setting the correspondence relationship as shown in Fig. 7 according to the program in the memory 120. The CPU 110 registers various data input by the operator or the like in the correspondence relationship data 121, 122 in association with each other. <Fourth embodiment>

[0044] Furthermore, it is preferable that the posture conditions, relative position conditions, and other associated conditions for the robot to execute processing can be set for each worker. This is because different workers find different conditions easy to work under. For example, the desired position and timing for handing over a screwdriver may differ.

[0045] In this embodiment, the memory 120 of the control device 100 stores correspondence data 123 as shown in Fig. 8. The correspondence data 123 stores correspondences between identifying information of a worker, a posture of the worker, identifying information of a body part of the worker, identifying information of a part, a relative position of the part with respect to the body part of the worker, other associated conditions, and processes to be executed by the robot 200.

[0046] Then, the CPU 110 of the control device 100 displays information for identifying the worker and a screen for setting the correspondence relationship, as shown in Fig. 9, in accordance with the program in the memory 120. The CPU 110 registers the data input by the worker or the like in the correspondence relationship data 123.

[0047] In this embodiment, the CPU 110 of the control device 100 reads out, for example, a program for causing the robot 200 to execute a task in accordance with the program in the memory 120, and executes the processing shown in FIG.

[0048] First, the CPU 110 acquires images captured by the cameras 300, 300 . . . via the communication interface 160 (step S302).

[0049] The CPU 110 acquires characteristic data of the worker based on the captured image, thereby identifying the worker (step S304).

[0050] As shown in FIG. 4, the CPU 110 identifies the coordinates of each part of the worker's body based on the captured image (step S306).

[0051] CPU 110 identifies the posture of each part based on the coordinates of each part (step S308).

[0052] The CPU 110 refers to the correspondence data 121 and determines whether or not a process corresponding to the posture of the worker is registered in association with the worker (step S310).

[0053] If a process corresponding to the worker's posture is registered (YES in step S310), CPU 110 identifies the type and position of each part of the worker's body, and the type, model number and position of the part based on the captured image, as shown in FIG. 4 (step S312).

[0054] Then, CPU 110 calculates the relative position of the part with respect to the position of each part of the worker's body (step S314).

[0055] CPU 110 refers to correspondence data 122 and determines whether a process corresponding to the relative positions of parts with respect to the positions of each part of the worker is registered in association with the worker (step S316).

[0056] If a process corresponding to the relative position of the part with respect to the positions of each part of the worker's body is registered (YES in step S316), the CPU 110 refers to the correspondence data 122 and determines whether other associated conditions are met in association with the worker based on the content of the task currently being executed by the robot 200, etc. (step S318).

[0057] If other associated conditions are met (YES in step S318), CPU 110 identifies the corresponding process (step S320).

[0058] The CPU 110 transmits a control command to the robot 200 via the communication interface 160 (step S322). <Fifth embodiment>

[0059] Alternatively, it is preferable that the posture conditions, relative position conditions, and other associated conditions for the robot to execute processing can be set for each worker's physique. This is because the conditions that make it easy to work vary depending on the worker's physique. For example, the posture in which a screwdriver should be handed over differs depending on the length of the worker's arms.

[0060] Specifically, the memory 120 of the control device 100 may store correspondence data 124 as shown in Fig. 11. In this embodiment, the correspondence data 124 may store correspondences between the posture of the worker for each height, identifying information of the body part of the worker, identifying information of the part, the relative position of the part with respect to the body part of the worker, other associated conditions, and the process to be executed by the robot 200.

[0061] The memory 120 may store worker information data 125 as shown in Fig. 12. The worker information data 125 stores the correspondence between the characteristic data and the height of each worker.

[0062] In this embodiment, the CPU 110 of the control device 100 reads out, for example, a program for causing the robot 200 to execute a task in accordance with the program in the memory 120, and executes the processing shown in FIG.

[0063] First, the CPU 110 acquires images captured by the cameras 300, 300, . . . via the communication interface 160 (step S402).

[0064] The CPU 110 acquires characteristic data of the worker based on the captured image, thereby identifying the worker (step S404).

[0065] The CPU 110 refers to the worker information data 125 to identify the worker's height (step S406).

[0066] As shown in FIG. 4, the CPU 110 identifies the coordinates of each part of the worker's body based on the captured image (step S408).

[0067] The CPU 110 identifies the posture of each part of the worker's body as shown in FIG. 4 (step S410).

[0068] The CPU 110 refers to the correspondence data 121 and determines whether or not a process corresponding to the posture of the worker that is associated with the height of the worker is registered (step S412).

[0069] If a process corresponding to the worker's posture is registered (YES in step S412), CPU 110 identifies the type and position of each part of the worker's body, and the type, model number and position of the part based on the captured image, as shown in FIG. 4 (step S414).

[0070] Then, CPU 110 calculates the relative position of the part with respect to the position of each part of the worker's body (step S416).

[0071] CPU 110 refers to correspondence data 122 and determines whether a process corresponding to the relative positions of parts with respect to the positions of each part of the worker's body, which is associated with the worker's height, is registered (step S418).

[0072] If a process corresponding to the relative position of the part with respect to the positions of each part of the worker's body is registered (YES in step S418), the CPU 110 refers to the correspondence data 122 and determines whether other associated conditions associated with the worker are satisfied based on the content of the task currently being executed by the robot 200, etc. (step S420).

[0073] If other associated conditions are met (YES in step S420), CPU 110 identifies the corresponding process (step S422).

[0074] The CPU 110 transmits a control command to the robot 200 via the communication interface 160 (step S424). Sixth embodiment

[0075] Some or all of the roles of the devices such as the control device 100 and the robot 200 in the robot control system 1 of the above embodiment may be performed by other devices. For example, some of the roles of the control device 100 may be performed by the robot 200, the role of the control device 100 may be performed by multiple personal computers, or the information processing of the control device 100 may be performed by a server on a cloud. <Summary>

[0076] In the above embodiment, a robot control system is provided that includes a robot, at least one camera, and a control device. The control device identifies a posture of a part or the whole of a body of a worker based on an image of the at least one camera, and causes the robot to execute a process according to the posture.

[0077] Preferably, as the posture, the control device identifies the inclination of the worker's spine based on images from at least one camera.

[0078] Preferably, as the posture, the control device identifies a relative angle between a first bone and a second bone of the worker based on an image from at least one camera.

[0079] Preferably, the control device stores a posture corresponding to a process for each worker.

[0080] In the above embodiment, a control device is provided that includes a robot, at least one camera, a communication interface for communication therewith, a memory, and a processor. The processor identifies a posture of a part or the whole of a body of a worker based on an image of the at least one camera, and causes the robot to execute a process according to the posture.

[0081] In the above embodiment, a robot control system is provided that includes a robot, at least one camera, and a control device. The control device identifies a position of a part or an entirety of a worker's body based on an image of the at least one camera, and causes the robot to execute a process according to the position.

[0082] In the above embodiment, a robot control system is provided that includes a robot, at least one camera, a communication interface for communication, a memory, and a processor. The processor identifies a position of a part or an entirety of a worker's body based on an image of the at least one camera, and causes the robot to execute a process according to the position.

[0083] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0084] 1: Robot control system 100: Control device 110: CPU 120: Memory 121: Correspondence data 122: Correspondence data 123: Correspondence data 124: Correspondence data 125: Worker information data 130: Display 140:Operation unit 150: Speaker 160: Communication interface 200: Robot 300: Camera

Claims

1. Robots and At least one camera; a control device including a memory that stores a correspondence relationship between a plurality of body parts of a worker, a relative angle between the plurality of body parts, and a process to be executed by the robot; The control device includes: receiving, in accordance with an input from an operator, designation of a plurality of body parts of the operator and designation of relative angles of the plurality of body parts, and registering the designation in advance in the memory; A robot control system that, during actual work, identifies the posture of a part or all of a worker's body, including the relative angles of multiple parts of the worker's body, based on images from the at least one camera, and causes the robot to execute processing according to the posture by referring to the memory.

2. A communication interface for communicating with a robot and at least one camera; a memory that stores a correspondence relationship between a plurality of body parts of a worker, relative angles of the plurality of body parts, and a process to be executed by the robot; a processor; The processor, receiving, in accordance with an input from an operator, designation of a plurality of body parts of the operator and designation of relative angles of the plurality of body parts, and registering the designation in advance in the memory; A control device that, during actual work, identifies a posture of a part or all of a worker's body, including relative angles of multiple parts of the worker's body, based on images from the at least one camera, and causes the robot to execute processing corresponding to the posture by referring to the memory.

Images