System for detecting structure of end effector

Abstract

To easily grasp the movement of an end effector which is mounted on a robot arm in use, with no need for the end effector to operate on the robot arm.SOLUTION: A system 1 for detecting a structure of an end effector ef includes: imaging means 2 for imaging the end effector; element recognition means 4 for recognizing a constituent element in the end effector in an image of the end effector imaged by the imaging means; and movable range detection means for detecting a movable range of the constituent element recognized by the element recognition means. The movable range detection means is configured to: detect a position of an image of the constituent element recognized in each of images of the end effector in each of possible states of the end effector imaged by the imaging means; and detect a change range of the detected position as the movable range of the constituent element.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a system for detecting the structure of an end effector attached to the tip of a robotic arm using a computer system, and more particularly to a system for detecting the presence, position, or further movement range of movable elements in the end effector.

Background Art

[0002] In recent years, in the manufacturing industry, the demand for introducing robots has been increasing. In the case of robots used for manufacturing objects, various end effectors are attached to the tip of the robotic arm, and various object processing and operations are performed. Therefore, various techniques for executing control according to the type of the end effector attached to the tip of the robotic arm have been proposed. For example, in Patent Document 1, as a method for controlling a robot to ensure safety when using a replacement hand (end effector) that has never been used before in the robot, when it is determined that the speed of the movable part is equal to or higher than a predetermined speed when the robotic arm equipped with the end effector is driven, a configuration for reducing the operating speed of the robotic arm has been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Robot maintenance and operation require specialized knowledge. In particular, even experienced personnel need time for the operation of robot teaching, which is used to teach the robot a trajectory. In order to eliminate the robot teaching operation, it is necessary to understand the structure of the end effector that the robot is going to use, and to understand the structure for using the end effector. In this case, when the end effector is attached to the robot arm and operated in a situation where it is not known what the end effector is, there are considered to be problems from the viewpoint of safety. In addition, when using an unknown end effector, it is advantageous that how the end effector moves can be easily grasped without operating it on the robot arm.

[0005] In view of the above circumstances, the main problem of the present invention is to enable the movement of the end effector attached to and used by the robot arm to be easily grasped without operating on the robot arm.

Means for Solving the Problem

[0006] According to the present invention, the above problem is solved by a system for detecting the structure of an end effector, imaging means for imaging the end effector, element recognition means for recognizing components in the end effector in the image of the end effector imaged by the imaging means, and movable range detection means for detecting a movable range of the components recognized by the element recognition means, wherein the movable range detection means is configured to detect positions of images of the components recognized in each of the images of the end effector in each possible state of the end effector imaged by the imaging means, and detect a change range of the detected positions as the movable range of the components. This is achieved by

[0007] In the above, the "end effector" is, as already mentioned, a device configured to be mounted at the tip of a robotic arm and perform a certain type of processing or operation on a certain object. The end effector typically has components (such as a processing unit, a movable part, etc.) for performing some treatment on the object to be processed or operated on, and the components can take different states corresponding to the control commands given from the robot side. The "imaging means" may be a camera or the like that captures an image of the end effector. The "element recognition means" and the "movable range detection means" may be realized by operations according to a program in a computer device. Specifically, the element recognition means first uses any image segmentation technique in the image of the end effector captured by the imaging means to recognize the image of each component in the end effector, and is configured to recognize what kind of functional component the recognized component is (for example, a movable claw structure, a servo motor, an arm, etc.) using any classification method. The movable range detection means is configured to detect the displacement (change in position) of the image of the component recognized by the element recognition means from the images of the end effector captured by the imaging means for each state that the end effector can take, between the states that the end effector can take with respect to each other. And when displacement is detected for a certain component in the end effector, the movable range detection means may be configured to detect the range in which the component has displaced as the movable range of the component. Note that the states that an arbitrary end effector can take are the various states of the end effector when various control commands from the robot side are given to the end effector when it is mounted on the robot.

[0008] According to the configuration of the system of the present invention described above, by causing an arbitrary end effector to be imaged by the imaging means in each state when various control commands are given to it, it can be easily grasped without operating on the robotic arm whether there are movable components in the components of the end effector, and when it is movable, its movable range is automatically detected, whereby how the end effector moves can be easily grasped.

[0009] In the above configuration, when multiple components are movable in the end effector, the movable range may be detected for each component. When two or more control commands are given to a certain end effector, the displacement of the component for each change in the control command may be detected, and the movable range for each change in the control command may be detected. When none of the components are displaced even when different control commands are given (for example, switching the ON / OFF of the electromagnetic force at the tip), it may be detected that there is no movable range for the component.

Advantages of the Invention

[0010] Thus, according to the configuration of the present system described above, it becomes possible to grasp or record how the end effector moves without actually moving it on the robot, and it is possible to avoid operating on the robot in a state where the movement is unknown, which is safer. Further, in the system of the present invention, since the movable range of the component is detected from the image obtained by photographing the end effector with the photographing means, it is also advantageous in that it is possible to grasp how the end effector moves with a lower computational load than when grasping the structure using three-dimensional point cloud data, for example.

[0011] Other objects and advantages of the present invention will become apparent from the following description of the preferred embodiments of the present invention.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Explanation of Reference Numerals

[0013] 1... System, 2... Camera (imaging means), 3... Monitor, 4... Computer device, ef... End effector

Best Mode for Carrying Out the Invention

[0014] The present invention will be described in detail below with reference to the accompanying drawings in some preferred embodiments. In the drawings, the same reference numerals indicate the same parts.

[0015] System configuration As schematically depicted in Fig. 1(A), a system 1 for detecting the structure of an end effector of the present embodiment (hereinafter referred to as "the present system") is composed of a camera that images the end effector ef and a computer device 4 that receives the image captured by the camera 2 and executes the processes described below. The computer device 4 may be a normal computer device that has a display monitor 3, is operated through a normal operation terminal, and operates according to a program. It is equipped with a CPU, a storage device, and an input / output device that are interconnected by a bidirectional common bus. The storage device may include a memory that stores each program for executing the arithmetic processing used in the present embodiment, a work memory, and a data memory used during processing. The end effector ef may be an end effector ef of any form attached to a robotic arm. The end effector ef is configured to transition to various states according to a control command given from the robot side. In this regard, in the case of an end effector having a movable part, the position of the movable part (with respect to the immovable part) changes according to the given control command. Also, there may be a case of an end effector having no movable part (in that case, in the system of the present embodiment, it will be detected that there is no movable part).

[0016] As a configuration realized by the computer device of the present system 1, as shown in Fig. 1(B), it may be provided with an image recognition unit, a movable range detection unit, and a recording unit.

[0017] The image recognition unit receives an image of the end effector ef captured by a camera. First, according to the method of an arbitrary segmentation technique (such as SAM), the images of each component in the image of the end effector ef are segmented and recognized. Then, according to the method of an arbitrary classification technique (such as vgg16, res-net), it is recognized what kind of component each segmented and recognized component image is. In such a classification technique, in advance, learning data in which various components in various end effectors, for example, images of a movable claw structure, a servo motor, and an arm, and labels for identifying the types of each component are paired is prepared. Using the learning data, an identification model is constructed that is learned to identify the type of each component for an arbitrary component image according to an arbitrary machine learning algorithm. Each component image recognized in the image of the end effector ef captured by the camera is input into such an identification model, and as a recognition result, the type of the component of the image is output.

[0018] The movable range detection unit, to put it simply, detects the movable range of the movable part among the components in the end effector ef captured by the camera. Specifically, in the movable range detection unit, the positions of each component (in the image) recognized in each image of the end effector ef in the state when different control commands are given (even if no control command is actually given to the end effector, it is sufficient if the end effector takes the state when each control command is given) are detected respectively. For each component, the change in the detected position is detected, and the range of the change is detected as the movable range of the component. Note that when there is no change in the position for each component, it is detected that there is no movable range.

[0019] The recording unit records the movable range of each component of the end effector detected by the movable range detection unit, and may be configured so that the user can refer to the movable range of each component when using the end effector.

[0020] System operation (a) Overview In the system of this embodiment, to put it simply, for any end effector, when each control command that can be given to it is given, the end effector in the state at that time is photographed by a camera. In the image of each state of the end effector, each component in the end effector is recognized, the change in each position is detected, and each movable range is detected.

[0021] Specifically, for example, as schematically depicted in FIGS. 2(A) and (B), in the case of an end effector whose state changes when control command ef_c0 and control command ef_c1 are respectively given (an end effector in which a pair of claw portions n1 and n2 open and close by a control command), first, the end effector in each state is photographed by a camera, and images Im0 and Im1 as depicted in FIG. (i) thereof are obtained. Then, in each of the images Im0 and Im1, by image segmentation technology, as in FIG. (ii) thereof, the images of each component e1 to e4 in the image of the end effector are segmented and recognized. Further, by class classification technology, as in FIG. (iii) thereof, the types of each component e1 to e4 are identified, and the movable parts e3 and e4 (claw portions n1 and n2) are specified. Thereafter, as schematically depicted in FIG. 2(C), the positions of the movable parts e3 and e4 in the state when control command ef_c0 is given to the end effector and the positions of the movable parts e3 and e4 in the state when control command ef_c1 is given to the end effector are respectively detected, the displacements MR(e3) and MR(e4) of those positions are detected, and the change range is specified as the movable range of each of the movable parts e3 and e4 (claw portions n1 and n2). Note that the positions and movable ranges of the movable parts e3 and e4 may be represented by coordinates measured from an arbitrary reference point (an arbitrary fixed point in the end effector). The information obtained here is saved in the recording unit as already mentioned so that the user can refer to it as appropriate.

[0022] Even if different control commands are given to a certain end effector, when there is no part whose position changes in that end effector, it may be recorded that there is no movable range as that end effector.

[0023] As described above, since the movable range of the movable part in an arbitrary end effector is detected and can be referred to in a timely manner, how the end effector moves can be grasped in advance before using the end effector, and thereby, an improvement in the safety when using the end effector is expected.

[0024] (b) Processing procedure The system of the present embodiment may execute processing according to a procedure as described in FIG. 3.

[0025] Referring to the figure, for the end effector to be detected, first, in one state (state 0), the end effector is imaged by a camera (step 2), and in the image, segmentation (step 3) and classification (step 4) are executed, and the processing part (the part that performs some processing on the operation target) or the movable part in the end effector is recognized (step 5). Next, if there is another state in the end effector (step 6), it transitions to that other state (step 7), and in the state after the transition, steps 2 to 6 above are repeated. Thereafter, when there is no other state in the end effector, the position and displacement of the processing part or the movable part in the end effector recognized so far are detected (step 8), and the displacement range is recorded as the movable range of the processing part or the movable part (step 9). Note that the possible states of the end effector may be three or more, and in that case, the displacement between each state, or the displacement between each adjacent state, may be detected and recorded.

[0026] Thus, in the present embodiment described above, before actually moving the end effector on the robot, the possible states or movements of the end effector can be grasped, a situation where the end effector makes an unexpected movement for the user is avoided, and it is advantageous in that an improvement in safety can be expected. Further, in the case of the present embodiment, it is also advantageous in that the movable range can be easily grasped only by photographing the end effector with a camera in each of its possible states.

[0027] The above description has been made in relation to embodiments of the present invention, but many modifications and changes are readily possible for those skilled in the art, and the present invention is not limited only to the embodiments illustrated above, and it will be apparent that it can be applied to various devices without departing from the concept of the present invention.

Claims

【Claim 1】 A system for detecting the structure of an end effector, comprising: imaging means for imaging the end effector; element recognition means for recognizing components in the end effector in an image of the end effector imaged by the imaging means; a movable range detection means for detecting a movable range of the components recognized by the element recognition means; wherein the movable range detection means is configured to detect positions of images of the components recognized in each of the images of the end effector in each possible state of the end effector imaged by the imaging means, and to detect a change range of the detected positions as a movable range of the components.

Images