Robotic system

Abstract

A robot system is provided which can easily perform three-dimensional correction using a camera or the like to perform work in the case of positional deviation of a robot. The robot system is provided with: a robot (2); a robot carrying device (3) for mounting the robot and moving to a prescribed work space; at least two target markers (4) provided in the work space; a target marker position acquisition section (5) which performs stereoscopic measurement on the at least two target markers (4) using a vision sensor provided in the robot (2) to find a three-dimensional position; a deviation amount acquisition section (6) which finds a deviation amount of the robot (2) from a desired relative position with the work space based on the found three-dimensional position; and a robot control section (7) which causes the robot (2) to act in accordance with a corrected value based on a prescribed motion amount using the found deviation amount.

Description

Technical Field

[0001] This invention relates to a robot system. Background Technology

[0002] In recent years, for example, the following technical methods have been proposed: placing robots on trolleys or AGVs (Automated Guided Vehicles) and moving them, thereby automating various tasks by placing robots near the workspace of industrial machinery such as machine tools.

[0003] In this case, for example, in a system equipped with robots that use machine tools, trolleys, AGVs, etc. and are configured in designated positions, when the robot performs various operations such as loading / unloading workpieces on the machine tool, the stopping position of the trolley or AGV carrying the robot changes, so the robot cannot adequately handle the necessary operations by only performing the same actions each time.

[0004] Therefore, it is necessary to correct the robot's movements in order to measure the offset of the trolley / AGV relative to the machine tool's stopping position and to perform operations correctly in the workspace.

[0005] As a method for correcting the robot's movements, one proposed method is to install a camera on the robot's fingertip, use the camera to detect target markers set in the workspace, thereby determining the relative positional relationship between the robot and the machine tool or other workspace objects, and correcting the positional offset.

[0006] For example, Patent Document 1 discloses "a coordinate correction method for a mobile robot, characterized in that the mobile robot is a playback-mode operation robot with a vision sensor mounted on its arm. When the operation robot stops entering the work table, before the operation program begins, the vision sensor is in a vertical position to take pictures of two marks set on a predetermined part of the surface of the work table. The horizontal coordinates of the marks are calculated by an image processing device, the offset between the horizontal coordinates and the taught horizontal coordinates is calculated, the horizontal coordinates of the taught operation program are corrected using the offset, and the operation program is executed. The coordinate correction method includes the step of taking pictures of the marks by tilting the vision sensor only by a predetermined angle θ before the operation program begins, calculating the horizontal coordinates of the marks based on the image, taking out the vertical offset σ based on the offset between the horizontal coordinates and the taught horizontal coordinates under the same tilt position, performing a calculation based on the formula: Δh=σ / sinθ, and using the value of Δh to correct the vertical coordinates of the taught operation program."

[0007] Patent document 2 discloses "a method for correcting the three-dimensional position and posture of an autonomous walking robot. The autonomous walking robot has an autonomous walking part and an arm of a teaching and regenerating robot mounted on the walking part. When the robot walks towards the target location of the robot's operation via the walking part and stops at the target location, a correction mark installed at a predetermined position at the target location is photographed by a vision sensor set on the arm. The error between the stopping position and the teaching position at the target location is corrected based on the photographed image. The method is characterized by driving each motion axis of the arm in a manner that the image of the correction mark is photographed at a predetermined position with a predetermined shape and a predetermined size. The correction amount of the three-dimensional position and posture is calculated based on the driving amount of each motion axis. The teaching data of the arm is then three-dimensionally corrected based on the correction amount."

[0008] Existing technical documents

[0009] Patent documents

[0010] Patent Document 1: Japanese Patent Application Publication No. 03-281182

[0011] Patent Document 2: Japanese Patent Application Publication No. 09-070781 Summary of the Invention

[0012] The problem that the invention aims to solve

[0013] However, when robots are mounted on trolleys or AGVs, there is a strong desire to perform 3D corrections easily using cameras or similar devices whenever the robot's position shifts.

[0014] That is, there is a strong expectation that tasks can be performed not only simply, but also quickly and easily without making the user particularly aware of their difficulty.

[0015] Methods for solving problems

[0016] One embodiment of the robot system disclosed herein comprises: a robot; a robot transport device for carrying the robot and moving it toward a designated workspace; at least two target markers disposed in the workspace; a target marker position acquisition unit that uses a vision sensor disposed on the robot to perform stereoscopic measurement on the at least two target markers to determine their three-dimensional positions; an offset acquisition unit that determines, based on the acquired three-dimensional positions, an offset of the robot from the workspace relative to a desired position; and a robot control unit that uses the acquired offset to cause the robot to move with a value corrected according to a predetermined motion amount.

[0017] Invention Effects

[0018] According to one aspect of the robot system disclosed herein, even if the robot's position is shifted due to the movement of robot handling devices such as trolleys or AGVs, the robot can still perform three-dimensional correction and operate in the accurate relative position.

[0019] By performing stereoscopic measurements on target markers of two or more points, three-dimensional correction can be performed, for example, using an inexpensive two-dimensional camera.

[0020] Even if the user is unaware of the concept of coordinate system or scene setting, it can automatically make corrections, enabling the robot to perform tasks with high precision and appropriate actions. Attached Figure Description

[0021] Figure 1 This is a diagram illustrating one aspect of the robot system disclosed herein.

[0022] Figure 2 This is a block diagram representing one aspect of the robot system disclosed herein.

[0023] Figure 3 This diagram is used in the explanation of the method and steps for determining the three-dimensional position of a target marker by using a vision sensor installed on the robot to perform stereo measurement.

[0024] Figure 4 This diagram is used in the explanation of the method and steps for determining the three-dimensional position of a target marker by using a vision sensor installed on the robot to perform stereo measurement.

[0025] Figure 5 This diagram is used in the explanation of the method and steps for determining the three-dimensional position of a target marker by using a vision sensor installed on the robot to perform stereo measurement.

[0026] Figure 6 This diagram is used in the explanation of the method and steps for determining the offset of the robot from the desired relative position in the workspace based on the obtained three-dimensional position and using the obtained offset for correction. Detailed Implementation

[0027] The following is for reference Figures 1 to 6 The robot system according to one embodiment of the present invention will be described.

[0028] like Figure 1 as well as Figure 2As shown, the robot system 1 of this embodiment includes: a robot 2; a robot transport device 3, which carries the robot 2 and moves it to a designated work space (work area) for performing operations on the robot 2 at a designated location; at least two target markers 4, which are set in the work space; a target marker position acquisition unit 5, which uses a vision sensor 51 provided on the robot 2 to perform stereoscopic measurement on the at least two target markers 4 to determine their three-dimensional positions; an offset acquisition unit 6, which determines the offset of the robot 2 from the work space relative to the desired position based on the acquired three-dimensional positions; and a robot control unit 7, which uses the acquired offset to make the robot 2 move with a value corrected according to a predetermined motion amount.

[0029] The vision sensor 51 of the target mark position acquisition unit 5 is provided in the movable parts of the robot 2. Specifically, the vision sensor 51 is provided in the movable parts of the robot 2 such as the fingertips, arms, and limbs. In this embodiment, since stereo measurement is performed, an inexpensive two-dimensional camera can be used as the vision sensor 51.

[0030] Figure 1 The robot 2 shown is a 6-axis robot. In this embodiment, it is preferable to set at least 3 target markers 4 in the workspace. In this case, such as Figure 1 As shown, by placing the vision sensor 51 on the fingertip 21 of the robot 2, the robot control unit 7 is configured to perform three-dimensional corrections in six degrees of freedom to enable the robot 2 to move.

[0031] In the robot system 1 of this embodiment, for example, the motion program of the robot 2, the image processing program including the measurement settings and offset calculation program of the vision sensor 51, and the camera calibration data of the vision sensor 51 are preset and packaged, and stored in the storage unit 8. This will be described in detail later.

[0032] Furthermore, in the robot system 1 of this embodiment, before or during the robot 2's motion program and the measurement operation of the vision sensor 51, a target marker 4 is measured and its position is determined. The determination unit 9 then determines whether the obtained offset exceeds a preset threshold. Moreover, if the determination result indicates that the threshold has been exceeded, all target markers 4 in the workspace at the current time point are measured to obtain the offset again.

[0033] Furthermore, in the robot system 1 of this embodiment, the robot is configured such that, before entering the machine tool 10, which serves as the work space, it is roughly positioned using the target mark 4 provided on the machine tool 10, and then enters the machine tool 10, which serves as the work space, and uses the target mark 4 provided inside the machine tool 10 to determine the accurate offset in the machine tool 10.

[0034] Furthermore, in the robot system 1 of this embodiment, a warning unit 11 is configured to issue an alarm when the distance between the robot 2 and the machine tool 10 is below a preset threshold before entering the machine tool 10.

[0035] In the robot system 1 of this embodiment, which is constructed with the above-described structure, it is configured to affix two or more target markers 4 within the workspace, and to determine the three-dimensional position by performing stereoscopic measurement on each target marker 4. Preferably, three target markers are set. In this case, at least two target markers are set inside the workspace, and at least one target marker 4 is set outside.

[0036] For example, such as Figures 3 to 5 As shown, the same target mark 4 is detected twice by changing the position of the vision sensor 51 (target mark position acquisition unit 5) which is composed of a camera, thereby measuring the three-dimensional position (X, Y, Z) of the target mark 4.

[0037] At this time, a target mark 4 is detected at the positions of the two cameras (target mark position acquisition unit 5 and vision sensor 51). Based on these two detection results, the three-dimensional position of the target mark 4 is calculated through stereo calculation. For example, the line of sight (X, Y, W', P', R') from the camera toward the target mark 4 is detected, and the three-dimensional position of the workpiece is calculated through stereo calculation using the two line of sight data. In addition, W' and P' are the direction vectors representing the line of sight, and R' is the angle around the target.

[0038] Furthermore, in a preferred embodiment, three-dimensional measurements are performed on the three target marks 4 disposed on the surface of the machine tool 10 to measure the three-dimensional position (X, Y, Z) of each target mark 4. A total of six inspections are performed on each of the three target marks 4.

[0039] Next, by synthesizing the three-dimensional positions of the three target markers 4, the three-dimensional position and orientation of the machine tool 10 relative to the robot 2 are determined. That is, three-dimensional measurements are performed on three positions of an object, and these measurement results are synthesized to determine the overall position and orientation of the object. In this embodiment, the overall position and orientation of the machine tool 10 are determined by measuring three locations on the surface of the machine tool 10.

[0040] For example, the overall three-dimensional position (X, Y, Z, W, P, R) of the computer tool is calculated based on the three-dimensional positions (X, Y, Z) of the three target markers 4. In this case, the overall three-dimensional position (X, Y, Z, W, P, R) of the computer tool is calculated by establishing a coordinate system where the position of the first target marker 4 is set as the origin, the position of the second target marker 4 is set as a point on the X-axis, and the position of the third target marker 4 is set as a point on the XY plane.

[0041] Next, as Figure 6 As shown, the offset of robot 2 from the three-dimensional workspace of the machine tool in six degrees of freedom is calculated based on the calculated three-dimensional position of the machine tool, and the movement of robot 2 is corrected.

[0042] In this embodiment, the offset is calculated based on the actual three-dimensional detection position and posture, and the original reference position and posture. The coordinate system itself is moved and rotated such that the machine tool at the actual detection position overlaps with the machine tool at the reference position. The amount of coordinate system movement calculated in this way is set as the offset (correction amount), thereby correcting the prescribed movements of robot 2. Furthermore, Figures 3 to 5 It is represented in two dimensions, but it remains essentially unchanged even in three dimensions.

[0043] Furthermore, in this embodiment, based on the calibration method of robot 2 described above, all settings are completed from the beginning, making it usable as a package. The specific structural elements of the package include the robot 2's motion program, image processing program, and camera calibration data. These are pre-stored in the storage unit 8.

[0044] The storage unit 8 stores calibration data for the camera (vision sensor 51) based on the coordinate system (mechanical interface coordinate system) set on the fingertip 21 of the robot 2, i.e., calibration data in the mechanical interface coordinate system. On the other hand, the robot control unit 7 can determine the position of the fingertip 21 of the robot 2 when the camera (vision sensor 51) takes a picture in the robot coordinate system. Therefore, by using the calibration data stored in the storage unit 8, the two-dimensional points in the sensor coordinate system are mapped to the three-dimensional points in the mechanical interface coordinate system. Furthermore, based on the position of the fingertip 21 of the robot 2 determined by the robot control unit 7, the coordinates in the mechanical interface coordinate system are converted to the robot coordinate system. This allows the mapping of the two-dimensional points in the sensor coordinate system when the camera (vision sensor 51) takes a picture to the three-dimensional points in the robot coordinate system. In other words, the position and orientation of the sensor coordinate system observed from the robot coordinate system can be determined, thereby enabling three-dimensional position measurement.

[0045] In addition, in this embodiment, when the robot 2 is working on the work space or just before it is about to start working, it first measures only one target mark with its vision. If the measurement result is the same as when the above operation was performed, it is determined that the positional relationship between the robot and the work space has not changed after the above operation was performed and the work continues directly. If it is different, the work is interrupted and the above operation is performed again.

[0046] Measuring all target markers 4 each time takes time, but this time can be reduced using the method of this embodiment. Furthermore, the threshold for determining identical positions can be set according to the overall required accuracy of the system.

[0047] Furthermore, in the case where the workspace is the machine tool 10 as in this embodiment (where it is set inside the machine tool 10), the robot 2 is roughly positioned using the target mark 4 set on the outside of the machine tool 10 before entering the machine tool 10, and then accurately positioned using the target mark 4 set inside the machine tool 10 (2-stage positioning).

[0048] Furthermore, when precision is required for positioning relative to the internal worktable or other components of the machine tool 10, but the front width of the machine tool 10 is narrow, the robot 2 may come into contact with the entrance of the machine tool 10 without measurement. In this case, the robot 2 can be moved without contact, and an alarm can be issued when contact is imminent.

[0049] Therefore, according to the robot system 1 of this embodiment, even if the position of the robot 2 is offset due to the movement of the robot handling device 3 such as the trolley or AGV, the robot 2 can still perform operations by performing corrections in a three-dimensional, six-degree-of-freedom manner. Therefore, by performing corrections in a three-dimensional, six-degree-of-freedom manner, corrections that cannot be performed by simple three-dimensional correction in XYZ are possible, for example, corrections can be performed even when the floor is uneven or deformed.

[0050] Furthermore, by performing stereo measurements on target markers of two or more points, three-dimensional correction can be performed using, for example, an inexpensive 2D camera. In particular, by performing stereo measurements on target markers of three or more points, even an inexpensive 2D camera can perform correction for six degrees of freedom. In the case of two points, the amount of rotation about the line segment connecting those two points cannot be determined. However, since this amount of rotation is difficult to change in the system structure, it becomes a sufficiently practical structure.

[0051] Furthermore, even if the user is unaware of the concept of coordinate system or the setting of the scene, it can automatically make corrections, and Robot 2 can perform its work.

[0052] The above describes one implementation of the robot system, but it is not limited to the above implementation and can be appropriately modified without departing from its main idea.

[0053] Explanation of reference numerals in the attached figures

[0054] 1 Robot System

[0055] 2 robots

[0056] 3 Robotic Handling Device

[0057] 4 Target Marking

[0058] 5. Target Marking Location Acquisition Section

[0059] 6. Offset Acquisition Section

[0060] 7. Robot Control Department

[0061] 8 Storage Units

[0062] 9 Judgment Department

[0063] 10 Machine tools (industrial machinery)

[0064] 11 Warning Department

[0065] 21 fingertips

[0066] 51 vision sensors.

Claims

1. A robot system, characterized in that, have: robot; A robotic transport device for carrying the robot and moving it to a designated workspace; At least two target markers are set in the workspace; The target marker position acquisition unit uses a vision sensor installed on the robot to perform stereo measurement on the at least two target markers to determine their three-dimensional positions; The offset acquisition unit calculates the offset of the robot from the desired relative position in the work space based on the acquired three-dimensional position. The robot control unit uses the acquired offset to cause the robot to move with a value corrected according to a predetermined amount of motion. Before or during a task, measure one target marker and determine its position. If the obtained offset exceeds a preset threshold, measure all target markers in the task space at the current time point and obtain the offset again.

2. The robot system according to claim 1, characterized in that, The visual sensor is located in the movable part of the robot.

3. The robot system according to claim 1 or 2, characterized in that, At least three target markers are set within the workspace. The visual sensor is located at the fingertips of the robot. The robot control unit enables the robot to perform actions in three dimensions.

4. The robot system according to any one of claims 1 to 3, characterized in that, The robot's motion program is pre-set and encapsulated, along with an image processing program that includes the measurement settings of the vision sensor and the offset calculation program, and camera calibration data of the vision sensor.

5. The robot system according to any one of claims 1 to 4, characterized in that, Before entering the machine tool that serves as the workspace, or just before entering the machine tool that serves as the workspace, the machine tool is positioned using a target mark set on the machine tool. After entering the machine tool that serves as the workspace, the offset in the machine tool is calculated using a target mark set inside the machine tool.

6. The robot system according to claim 5, characterized in that, An alarm is issued when the distance between the robot and the machine tool falls below a pre-set threshold before the robot enters the machine tool.

Images