Robot system and method for correcting the origin of a robot

Abstract

This invention provides a robot system that can automatically correct the control origin of each joint of a robot without setting extraction points on the robot or operating the robot. [Solution] The robot system 100 includes a robot 110, a motion control device 212 for operating the robot 110, a camera 400 for photographing the robot 110, a contour extraction device 214 for extracting the contour of the robot from the image captured by the camera 400, an angle output device 216 for outputting the actual angles of each axis of the robot from the contour of the robot, and a correction value calculation device for calculating a correction value for the origin of the control value of each axis by comparing the control angle of each axis, which is obtained from the control value of the motion control device 212 (position signal of the encoder 134), with the actual angle.

Description

[Technical Field]

[0001] The present invention relates to a robot system that automatically corrects the control origin of each joint of a robot and a robot origin correction method. [Background Art]

[0002] Correction of the control origin for each joint (axis) of a robot is required when a new tool is attached to the robot, or when components such as the motor, reducer, encoder, and robot body constituting the robot are replaced. Although an operator can manually perform origin correction using pins or the like, for example, Patent Document 1, Patent Document 2, and Patent Document 3 disclose robot systems that automatically correct the control origin of a robot.

[0003] In the invention described in Patent Document 1, a detection jig is mounted on a robot (for example, a hand fork), the robot is operated to prepare a predetermined reference posture, then, with the hand fork moved to a transfer position, a camera identifies coordinates corresponding to the detection jig, and a correction value for an encoder for controlling the motor is calculated based on the coordinates.

[0004] In the invention described in Patent Document 2, a marker is provided at the tip of an origin correction tool as the tip point of the robot, a first posture and a second posture are set in advance for each joint that is an origin correction target, and the displacement amount and displacement direction of the marker caused by changing the posture from the first posture to the second posture are measured through imaging by a camera.

[0005] In the invention described in Patent Document 3, a camera is attached to the robot as a tool, the robot is operated to a posture in which a mark on a work holding jig is included in the field of view of the camera, measured coordinate values of the mark are obtained through measurement using the camera, and whether correction of the posture of the robot is necessary is determined based on the deviation between the measured coordinate values and the target coordinate values of the mark. [Related Art Documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2019-141921 [Patent Document 2] Japanese Patent Publication No. 2012-223871 [Patent Document 3] Japanese Patent Publication No. 2018-126857 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, it is conceivable that operating the robot while correcting the control origin of each joint may not always guarantee sufficient safety. Furthermore, detection jigs, markers, and other extraction points must be set on the robot, and in environments where the robot is dirty, the dirt may obscure the extraction points, making them unidentifiable from camera images. Additionally, in situations such as spot welding with a robot, it may not be possible to attach cameras or other tools to the robot.

[0008] In view of these problems, the present invention aims to provide a robot system that can automatically correct the control origin of each joint of a robot without setting extraction points on the robot or operating the robot. [Means for solving the problem]

[0009] To solve the above problems, a typical configuration of the robot system according to the present invention comprises a robot, a motion control device for operating the robot, a camera for photographing the robot, a contour extraction device for extracting the contour of the robot from the image captured by the camera, an angle output device for outputting the actual angles of each axis of the robot from the contour of the robot, and a correction value calculation device for calculating a correction value for the origin of the control value of each axis by comparing the control angle of each axis obtained from the control value of the motion control device with the actual angle.

[0010] The machine learning model described above should ideally be a convolutional neural network.

[0011] The above camera would ideally be mounted on a teaching pendant used to instruct the robot on how to perform actions.

[0012] A typical configuration of the robot origin correction method according to the present invention includes the steps of: extracting the contour of the robot from an image taken by a camera of the robot operated by a motion control device using a contour extraction device; outputting the actual angles of each axis of the robot from the robot's contour using an angle output device; and calculating a correction value for the origin of the control value of each axis using a correction value calculation device by comparing the control angle of each axis obtained from the control value of the motion control device with the actual angle. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide a robot system that can automatically correct the control origin of each joint of a robot without setting extraction points on the robot or operating the robot. [Brief explanation of the drawing]

[0014] [Figure 1] This is a perspective view showing a schematic example of the configuration of the robot system according to this embodiment. [Figure 2] Figure 1 shows a magnified perspective view of the teaching pendant, viewed from both the front and back surfaces. [Figure 3] Figure 1 is a block diagram illustrating the overall configuration of the robot system. [Figure 4] Figure 3 is a flowchart illustrating the operation of the robot system. [Figure 5] This figure illustrates contour extraction of a robot in an example image taken with the camera shown in Figure 1. [Figure 6] This diagram illustrates the rotation direction of each axis of the robot in Figure 1. [Figure 7] Figure 1 is a neural network diagram illustrating a schematic example of a machine learning model that can be used by the robot system. [Figure 8]It is a schematic diagram showing a schematic configuration example of a robot system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Dimensions, materials, and other specific numerical values shown in these embodiments are merely illustrative for facilitating understanding of the present invention, and do not limit the present invention unless otherwise specified. In the present specification and the accompanying drawings, elements having substantially the same functions and configurations are denoted by the same reference numerals to omit redundant description, and elements not directly related to the present invention are omitted from illustration.

[0016] Figure 1 is a perspective diagram schematically showing an outline of a configuration example of the robot system 100 according to the present embodiment. Figure 2 is an enlarged perspective view of the teaching pendant of Figure 1 as seen from the front surface and the back surface, respectively. As shown in Figure 1 and Figure 2, the robot system 100 comprises, as an example, a robot 110, a robot controller 200 connected to the robot 110, a teaching pendant 300 for teaching operations to the robot 110, and a camera 400 provided in the teaching pendant 300.

[0017] The robot 110 comprises a base 120 and a robot arm 130 connected to the base 120. The robot arm 130 is, for example, a 6-axis vertical articulated type, but is not limited thereto; it may be an articulated type with fewer than 6 axes, or may be an articulated type with 7 or more axes. Further, the robot arm 130 may be a horizontal articulated type such as a 4-axis horizontal articulated type. A tool such as a robot hand can be optionally attached to the distal end of the robot arm 130.

[0018] The robot control device 200 is a device that performs various controls including the origin correction method described later, such as controlling the operation of the robot and sensors. The robot control device 200 can control the robot 110 and cause various functions to be executed by executing various programs stored in a storage device (not shown).

[0019] The robot control device 200 may be configured to be connected to a storage device that is a data storage device (not shown) or a drive device that is a recording medium reader / writer (not shown). Alternatively, a personal computer 230 provided with a drive device may be connected, and programs and data stored in a recording medium (not shown) may be installed into the robot control device 200. Further, the robot control device 200 itself may be configured by a general-purpose personal computer, and various functions may be executed by installing various programs therein.

[0020] Furthermore, in the present embodiment, the robot control device 200 is provided outside the robot arm 130 (and the tool), but it may be provided inside the robot arm 130 (and the tool).

[0021] A teaching pendant 300 is a device used for creating programs for the robot 110 and teaching operations. The teaching pendant 300 has a function as a remote controller for causing the robot 110 to perform the same operations as in actual work, and storing the positions, angles, and movements of each joint and the base 120.

[0022] Unlike well-known teaching pendants, the teaching pendant 300 of this embodiment can have a camera 400 capable of capturing images necessary for performing origin correction, for example, on the back of the teaching pendant 300. With this configuration, the operator does not need to prepare a separate camera for origin correction and can efficiently perform origin correction using the teaching pendant 300. In addition, the operator can perform normal operations while facing the front surface of the teaching pendant 300 and, if necessary, activate the origin correction program to capture images of the robot 110 from the camera 400 on the back of the teaching pendant 300.

[0023] As shown in Figure 2, the teaching pendant 300 includes buttons 310 for performing various operations and settings, a display device 320 that displays teaching data, error messages in case of abnormalities, and various data during automatic operation, and a camera 400. It may also include an emergency stop switch and an enable switch. The buttons 310 may include, for example, numeric keys, motion axis keys, and function keys.

[0024] Figure 3 is a block diagram illustrating the overall configuration of the robot system 100 shown in Figure 1. The robot 110 comprises multiple robot arms 130, and each axis (joint) that connects the robot arms 130 is equipped with a motor 132 and an encoder 134 (i.e., it is equipped with multiple motors 132 and encoders 134).

[0025] The robot control device 200 of the robot system 100 is equipped with an input / output unit 220. The input / output unit 220 performs signal input and output and connects the robot control device 200 to external devices such as a camera 400 and a teaching pendant 300, as well as various sensors such as a motor 132 and an encoder 134, via an interface. To this end, it is equipped with a communication device (not shown), a D / A converter, a motor drive circuit, an A / D converter, and so on.

[0026] Specific communication methods in communication devices may include, for example, serial communication standards such as RS232C / 485, data communication compliant with USB standards, general network protocols such as EtherNET®, and industrial network protocols such as EtherCAT®, EtherNet / IP®, PROFINET®, and CC-LinkIE®.

[0027] The input devices 311 include buttons 310 on a teaching pendant 300 operated by the operator, a keyboard or touch panel on a personal computer 230, and a mouse (not shown).

[0028] The camera 400 is preferably mounted on the teaching pendant 300 and outputs to the robot control device 200 via the operation of buttons 310 by the operator. The camera 400 typically acquires and outputs color images, but it may also acquire and output monochrome images.

[0029] Furthermore, the robot control device 200 includes a motion control device 212 that controls the movement of the robot 110 according to a program or operator's operation, a contour extraction device 214 that extracts the contour of the robot 110 from an image captured by the camera 400, an angle output device 216 that outputs the actual angles of each axis (joint) which are the connection points of the robot arm 130 of the robot 110 from the contour of the robot 110, and a correction value calculation device 218 that compares the control angle of each axis obtained from the control value of the motion control device 212 (position signal representing the current position: encoder value that controls the motor 132 that rotates the axis) with the actual angle and calculates a correction value for the origin of the control value of each axis (control value when the actual angle becomes zero). The contour extraction device 214, angle output device 216 and correction value calculation device 218 are provided in the robot control device 200, but may also be provided in control devices not shown in the teaching pendant 300 and the personal computer 230.

[0030] The motion control device 212 can control the operating position of the robot 110 and set the robot 110 to a predetermined posture. Here, the target position of the operation may be the tool center point (TCP) set from the teaching pendant 300, the initial position when a new tool is attached to the robot 110, or the start or end position when the robot 110 is powered on or powered off.

[0031] The memory device of the control device 210 stores link parameters, which are design parameters for the length and positional relationship of each robot arm 130.

[0032] The motion control device 212 calculates the position of each axis. Specifically, the motion control device 212 calculates the position of each axis, starting with the axis closest to the mounting surface of the multi-axis robot robot 110 (i.e., the axis closest to the base 120), using the rotation angle of the axis detected by the encoder 134 and the link parameters read from the storage device, and finally calculates the current position of the robot's end effector or tool. The position of each axis and the current position of the end effector or tool calculated by the motion control device 212 also include the orientation (vertical, horizontal, and torsional angles) of each axis and the end effector or tool.

[0033] Figure 4 is a flowchart illustrating the operation of the robot system 100 in Figure 3. Figure 5 is a diagram illustrating contour extraction of the robot in an example image taken of the robot 110 by the camera 400 in Figure 1. Figure 6 is a diagram illustrating the rotation directions of each axis J1, J2, J3, J4, J5, and J6 of the robot 110 in Figure 1. Figure 7 is a neural network diagram illustrating a schematic example of a machine learning model that can be used with the robot system 100 in Figure 1.

[0034] The method for automatically correcting the control origin of each joint of the robot 110 (origin correction method) will be explained below using Figures 4, 5, 6, and 7. When performing origin correction, such as when attaching a new tool to the robot 110, the operator does not need to move the robot 110 to prepare a predetermined reference posture for origin correction. In other words, the operator can start origin correction at the current position of the robot 110, where safety is ensured.

[0035] Furthermore, the operator does not need to position the camera 400 in a fixed position for origin correction. In other words, the operator can, for example, hold the teaching pendant 300 in their hand and photograph the robot 110 with the camera 400 on the teaching pendant 300 from any position.

[0036] The operator selects the origin correction mode by, for example, operating the buttons 310 on the teaching pendant 300 (S1 in Figure 4). Depending on the selection of the origin correction mode, the display device 320 on the teaching pendant 300 displays the field of view that the camera 400 can capture, and the operator captures the entire robot 110, including all axes J1, J2, J3, J4, J5, J6 (see Figure 6), while checking the field of view or display. The image captured by the camera 400 (see Figure 5) is input to the contour extraction device 214.

[0037] The contour extraction device 214 extracts the contour of the robot 110 from the image (S2 in Figure 4). For example, when the entire robot 110 is photographed as in image P1 shown in Figure 5(a), the contour of its outermost shell is extracted as in image P2 shown in Figure 5(b). Depending on the shooting direction, some arms may overlap, so it is preferable to photograph from a direction in which the rotation angles (actual angles) of axes J1, J2, J3, J4, J5, and J6 can be observed as much as possible. For the method of extracting contours from the image, OpenCV (Canny + contour extraction) or deep learning (Mask R-CNN, etc.) can be used.

[0038] The angle output device 216 then outputs the actual angles of each axis of the robot 110 from the contour of the robot 110 (S3 in Figure 4). With this configuration, even if part of the robot 110 is dirty, it does not affect the extraction of the contour of the robot 110, and it is possible to determine whether or not the actual angles of each axis of the robot 110 match the control angles based on the extracted contour.

[0039] Preferably, the angle output device 216 outputs the actual angle from the contour of the robot 110 using a machine learning model. With this configuration, the posture of the robot 110 when photographed by the camera 400 is not fixed or limited to a predetermined posture, but can be any posture. Similarly, the position of the camera 400 that photographs the robot 110 is not fixed or limited to a predetermined position, but can be any position. In this way, even if the degree of freedom of the image shooting conditions required on the input side is increased, an appropriate output can be obtained with a machine learning model, improving the convenience of the operator.

[0040] More preferably, the machine learning model is a convolutional neural network. With such a configuration, training data can be efficiently learned from a large number of combinations of robot 110 poses and camera positions, improving the accuracy of the calculated actual angles. Note that the neural network diagram in Figure 7 illustrates a schematic example of the machine learning model configuration and is not limited to these input layers. The contents of the hidden layers are arbitrary as long as the angles of axes J1, J2, J3, J4, J5, and J6 of the output layer can be output.

[0041] Convolutional neural networks are network architectures for deep learning that learn directly from image data, and are particularly effective for detecting patterns in images to recognize objects, classes, and categories. Specifically, existing image models such as ResNet (Residual Network) and EfficientNet can be used.

[0042] The neural network layers of a neural network begin with an input layer where data is supplied, followed by a hidden layer, and then an output layer. Each layer is composed of artificial neurons, and in the case of a convolutional layer, it is also called a kernel or filter. Different types of layers that make up a neural network can include, but are not limited to, convolutional layers, fully connected layers, regression layers, activation layers, and batch normalization layers.

[0043] The correction value calculation device 218 compares the control angle of each axis, which is obtained from the control value of the motion control device 212 (position signal representing the current position: encoder value controlling the motor 132 that rotates the axis), with the actual angle, and calculates a correction value for the origin of the control value of each axis (the control value when the actual angle becomes zero) (S4 in Figure 4).

[0044] The operator exits the origin correction mode by, for example, operating the buttons 310 on the teaching pendant 300 (S5 in Figure 4).

[0045] As explained in Figures 1 to 7 above, according to the robot system 100 of this embodiment, even if a part of the robot 110 is dirty, it does not affect the extraction of the robot's contour. Based on the extracted contour, it is possible to determine whether the actual angles of each axis J1, J2, J3, J4, J5, and J6 of the robot match the control angles, and the origin (the control value when the actual angle becomes zero; the control value is the encoder value that controls the motor that rotates the axis) can be corrected. Therefore, it is possible to automatically correct the control origin of each joint of the robot 110 without setting extraction points such as markers on the robot 110, and without operating the robot 110.

[0046] Figure 8 is a schematic diagram showing a schematic configuration example of a robot system 100 according to another embodiment. Unlike the schematic configuration example in Figure 1, in the robot system 100 of Figure 8, the camera 400 is, for example, a handheld camera (see Figure 8(a)). The operator can select the origin correction mode by operating the personal computer 230. The operator can photograph the stationary robot 110 in any posture from any camera position and input the captured image into the robot control device 200.

[0047] As shown in Figure 8(b), for example, three cameras 400 may be fixed around the robot 110, and each of the three cameras 400 may photograph the robot 110 from a different fixed angle. Using multiple cameras 400 and fixing the camera positions improves the accuracy of contour extraction.

[0048] Furthermore, if the camera positions are fixed for multiple cameras 400, the angle output device 216 may output the actual angles of each axis of the robot 110 without using a machine learning model. For example, the posture of the robot 110 at the time of shooting can be determined, and the actual angles can be determined by detecting the image error (deviation) compared to the model image.

[0049] Preferred embodiments of the present invention have been described above with reference to the attached drawings, but it goes without saying that the present invention is not limited to such examples. It will be obvious to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. [Industrial applicability]

[0050] This invention can be used as a robotic system. [Explanation of Symbols]

[0051] 100...Robot system, 110...Robot, 120...Base, 130...Robot arm, 132...Motor, 134...Encoder, 200...Robot control device, 212...Motion control device, 214...Contour extraction device, 216...Angle output device, 218...Correction value calculation device, 220...Input / output unit, 230...Personal computer, 300...Teaching pendant, 310...Buttons, 311...Input device, 320...Display device, 400...Camera

Claims

1. Robots and, A motion control device for operating the robot, A camera for photographing the aforementioned robot, A contour extraction device for extracting the contour of the robot from an image captured by the aforementioned camera, An angle output device that outputs the actual angles of each axis of the robot from the contour of the robot, A robot system characterized by comprising a correction value calculation device that calculates a correction value for the origin of the control value of each axis by comparing the control angle of each axis obtained from the control value of the motion control device with the actual angle.

2. The robot system according to claim 1, characterized in that the angle output device outputs the actual angle from the contour of the robot using a machine learning model.

3. The robot system according to claim 2, characterized in that the machine learning model is a convolutional neural network.

4. The robot system according to any one of claims 1 to 3, characterized in that the camera is provided on a teaching pendant for teaching the robot how to move.

5. The process involves extracting the contour of a robot from an image captured by a camera of the robot being operated by a motion control device using a contour extraction device, The steps include outputting the actual angles of each axis of the robot from the contour of the robot using an angle output device, A robot origin correction method characterized by including the step of comparing the control angles of each axis obtained from the control values ​​of the motion control device with the actual angles and calculating a correction value for the origin of the control values ​​of each axis using a correction value calculation device.

Images