Tool coordinate system setting device, setting method, and robot system

Abstract

Provided are a setting device, a robot system, and a setting method with which it is possible to easily teach / set a tool coordinate system of a robot and to easily perform fine adjustment or the like thereafter. This setting device comprises: a display unit that displays a first image representing the three-dimensional shape of a constituent element of a robot, and a second image including information relating to orientation, the first and second images being acquired from a three-dimensional sensor having a known position / orientation relationship with the robot; an operation unit capable of changing the position and orientation of the second image on the display unit; and a setting unit that, on the basis of the relative position / orientation relationship between the first image and the second image, sets a tool coordinate system having the same position and orientation as the position and orientation of the second image.

Description

Tool coordinate system setting device and setting method, and robot system

[0001] The present disclosure relates to a tool coordinate system setting device and setting method, and a robot system.

[0002] When performing work using a robot, there is known a technique for teaching or setting a tool coordinate system to the robot. The tool coordinate system is a coordinate system set for a tool attached to the end of the robot's arm, for example, and the robot's operations can be taught based on this coordinate system.

[0003] There are various known techniques for teaching or setting a tool coordinate system to a robot, such as a method of touching up a representative point of the robot, such as the tool tip point, in multiple orientations relative to a single point in real space.

[0004] Another well-known method is to operate a robot with a target that can be detected by a two-dimensional camera attached to the tip of the tool, detect the position of the target, and input the posture of the target individually. Still another known method is to perform an offline simulation and load the results into a real robot.

[0005] JP 2020-075325 A JP 2018-069361 A

[0006] In the method of touching up the tool tip point to a point in real space, accurate teaching and setting are difficult unless the tool tip point has a shape that allows for accurate specification of a specific position (for example, a sharp point). In the method using a target object, it can be difficult to attach the target object to the tool tip. Furthermore, since the posture must be input by the operator, it can be very time-consuming if the tool tip is tilted. Furthermore, when using offline simulation, separate simulation software is required, and if the simulation software does not work well with the actual robot, it can be difficult to adjust the set tool coordinate system.

[0007] Therefore, there is a need for an apparatus or method that allows the tool coordinate system to be easily taught and set, and that allows subsequent fine adjustments to be easily performed as needed.

[0008] One aspect of the present disclosure is a tool coordinate system setting device having a display unit that displays a first image representing the three-dimensional shape of a component of a robot and a second image including information regarding the posture, both of which are acquired from a three-dimensional sensor whose position and posture relationship with the robot is known; an operation unit that can change the position and posture of the second image on the display unit; and a setting unit that sets a tool coordinate system having the same position and posture as the position and posture of the second image based on the relative position and posture relationship between the first image and the second image.

[0009] Another aspect of the present disclosure is a robot system having a robot, a three-dimensional sensor whose position and orientation relationship with the robot is known, a display unit that displays a first image obtained from the three-dimensional sensor and representing a three-dimensional shape of a component of the robot, and a second image including information regarding the orientation, an operation unit that can change the position and orientation of the second image on the display unit, and a setting unit that sets a tool coordinate system having the same position and orientation as the position and orientation of the second image based on the relative position and orientation relationship between the first image and the second image.

[0010] Yet another aspect of the present disclosure is a method for setting a tool coordinate system, including: displaying, on a display unit, a first image representing a three-dimensional shape of a component of a robot and a second image including information regarding the orientation, the first image being acquired from a three-dimensional sensor whose position and orientation relationship with the robot is known; changing the position and orientation of the second image on the display unit; and setting a tool coordinate system having the same position and orientation as the position and orientation of the second image based on the relative position and orientation relationship between the first image and the second image.

[0011] 5 is a schematic configuration diagram of a robot system including a setting device according to an embodiment; FIG. 6 is a diagram showing an example of the results of detection of a robot tool by a 3D sensor; FIG. 7 is a diagram showing a state in which a tool coordinate system is superimposed on the detection result of FIG. 2; FIG. 8 is a diagram showing an example in which a 3D model of the tool is superimposed on the detection result of FIG. 2; FIG. 9 is a diagram showing an example in which a tool coordinate system is set on and rotated as a 3D model of the tool; FIG. 10 is a diagram showing an example in which the position of the tool coordinate system of FIG. 5 is changed; and FIG. 11 is a flowchart showing an example of a method for setting a tool coordinate system.

[0012] 1 is a schematic diagram of a robot system including a setting device according to an embodiment. The robot system 2 includes an industrial robot 4 such as a vertical articulated robot, a robot control device 6 including a processor, memory, etc., and controlling the robot 4, a teaching pendant 8 including a processor, memory, etc., and connected to the robot control device 6 for wired or wireless communication, and a setting device 10 such as a personal computer (PC) connected to at least one of the robot control device 6 and the teaching pendant 8 for wired or wireless communication. The teaching pendant 8 is, for example, a portable terminal such as a tablet that a user can operate to teach the robot 4.

[0013] In this specification, the term "user" is used to include a person, such as a user or an operator, who operates the setting device 10, etc. Furthermore, the term "position and orientation" is used to include both position and orientation.

[0014] The robot 4 comprises a base 12 installed at a predetermined location such as a factory, a movable part (here, a robot arm) 14 configured to be movable relative to the base 12, a flange 16 attached to the tip of the robot arm 14, and a tool 18 such as a robot hand attached to the flange 16.

[0015] The position and orientation of a component of the robot 4 (here, the tool 18) can be detected by a three-dimensional (3D) sensor 20. The 3D sensor 20 is installed on a fixed location such as a stand 22, and can detect the position and shape of the tool 18. For example, the 3D sensor 20 can output point cloud data, which will be described later, as an example of a first image representing the three-dimensional shape of the tool 18. Various types of 3D sensors can be used, such as a stereo type using two cameras, a TOF type combining a camera and a light source, and a structured illumination type using a camera and a projector.

[0016] A user can use the teaching pendant 8 to input various information to the robot control device 6, or receive information from the robot control device 6 and display the information on a display or the like of the teaching pendant 8. The user can also use the setting device 10 to perform input operations for setting a tool coordinate system, which will be described later, or the like. The setting device 10 has an input unit 24 such as a mouse or a keyboard, a display 26 such as a liquid crystal screen, and electronic components 28 such as a processor and memory, and in this embodiment, the input unit 24, the display 26, and the electronic components 28 correspond to an operation unit, a display unit, and a setting unit, respectively.

[0017] 1 to 7, specific examples of the operations and processing in the system 2 will be described below. First, in step S1, the 3D sensor 20 is used to detect the tool 18 attached to the robot 4. Note that the installation position of the 3D sensor 20 is determined as a position within the robot's workspace (user coordinate system), and therefore the processing in step S1 determines the position and orientation of the tool 18 in the user coordinate system.

[0018] In the next step S2, as shown in Fig. 2, the output obtained by actually detecting the tool 18 with the 3D sensor 20, specifically the point cloud data 30 of the tool 18, is displayed on the display 26 or the like of the setting device 10. Here, the tool 18 is a gripping hand having two claws 54 as shown in Fig. 5 or 6, and the point cloud 32 on the left side in Fig. 2 corresponds to the tip side of the tool 18, and the point cloud 34 on the right side corresponds to the rear end side (the side connected to the flange 16).

[0019] Next, in step S3, a second image (here, a pointer) 36 representing the tool coordinate system is superimposed on the point cloud data 30 (here, the tip-side point cloud 32) as shown in Fig. 3. The pointer 36 can be displayed by a user's operation via the input unit 24, but the pointer 36 may also be automatically displayed at an appropriate position simultaneously with or after the display of the point cloud data 30 based on a previously prepared program or the like.

[0020] Since the pointer 36 is set as the tool coordinate system in a later process, it preferably has a shape that includes information about the attitude of the tool coordinate system and that allows visual recognition of information about not only the position but also the attitude of the tool coordinate system. In the illustrated example, the pointer 36 has an origin 38 that corresponds to a representative point of the tool coordinate system, such as the tool tip point (TCP), and three straight lines 40, 42, and 44 that are orthogonal to each other at the origin 38, and these three straight lines correspond to the X, Y, and Z axes of the tool coordinate system, respectively.

[0021] Next, in step S4, the position and orientation of the pointer 36 is changed and taught. This process can be performed by the user through operation via the input unit 24. The user can visually change the position and orientation of the pointer 36 while viewing the display 26, and can change the pointer 36 to a desired position and orientation. Furthermore, because the position and orientation of the pointer 36 is changed in a virtual space by the setting device 10, the pointer 36 (its origin) does not necessarily have to be on the surface of the tool 18; for example, as shown in FIG. 6 (described later), the pointer 36 can be positioned in a space where the tool 18 does not exist.

[0022] Next, in step S5, the setting device 10 calculates the position and orientation of a representative point of the tool 18 (here, the tool tip point (TCP)) in the user coordinate system based on the changed position and orientation of the pointer. Specifically, the setting device 10 calculates the position and orientation of the origin 38 of the pointer 36 in the user coordinate system based on the positional relationship between the pointer 36 and at least one of the points included in the point cloud data 30 (for example, the point closest to the pointer 36). Because the position of the flange 16 in the user coordinate system is known, the positions of each of the points in the point cloud 30 representing the tool 18 attached to the flange 16 can be easily calculated, and therefore the position and orientation of the TCP can also be easily determined.

[0023] Next, in step S6, the pointer 36, whose origin is the calculated TCP, is set in the robot control device 6 as the tool coordinate system. In other words, the tool coordinate system has the same position and orientation as the pointer 36. Furthermore, as setting information for the tool coordinate system, the point cloud data 30 and the position of the pointer 36 changed or taught by the user can be stored in a memory or the like of the setting device 10. This makes it possible for the setting device 10 to reproduce the set tool coordinate system at a desired timing and to adjust at least one of its position and orientation as appropriate. Furthermore, a user interface configured to allow the user to directly input values ​​corresponding to the user coordinate system in addition to operating the pointer 36 may be provided.

[0024] The robot control device 6 can appropriately control the robot 4, for example, by creating an operation command based on the set tool coordinate system and sending it to the robot 4. Furthermore, TCP can be easily set and taught even for tools with shapes for which it is difficult to set and teach a TCP, and the user can also fine-tune a tool coordinate system that has been set once, for example, by jogging around the tool coordinate system.

[0025] When the above-described offline simulation is used, it is difficult to accurately set the tool coordinate system if there is an error in mounting the tool 18 relative to the flange 16. In contrast, in this embodiment, the tool coordinate system is set and taught using 3D data obtained by detecting the actual robot tool, so there is no adverse effect due to the mounting error.

[0026] FIG. 4 shows an example in which a frame representing the shape of the tool 18 is superimposed on point cloud data 30 representing the tool 18. Based on CAD data of the tool 18 previously stored in the memory of the setting device 10 and the detection results of the tool 18 by the 3D sensor 20, the frame representing the shape of the tool 18 can be superimposed on the displayed point cloud data 30. In the illustrated example, a frame 46 is superimposed on the tip side of the tool 18, and a frame 48 is superimposed on the flange side of the tool 18. This display allows the user to visually and efficiently set the tool coordinate system (operate the pointer). For example, if the user wants to set the origin of the tool coordinate system to the outer surface of the tool 18, displaying the frames 46 and 48 makes the user's operation easier.

[0027] In the example of FIG. 4 , in order to display frames 46 and 48 based on CAD data, a detection process using CAD model matching is performed to detect a shape similar to the CAD data from the acquired point cloud data. However, instead of CAD model matching, detection results detected by various detection tools may be overlaid on the point cloud data. For example, a process for detecting a set of connected points (3D blobs) formed by connecting nearby points in the point cloud data may be used. In this process, at least a distance threshold is set for regarding adjacent point clouds as a set of connected points, and the set of connected points is detected from the entire point cloud. The set of connected points detected by this process may be displayed as part of a first image representing the three-dimensional shape of tool 18. As another detection tool, a segmentation generated using AI (artificial intelligence) may be overlaid on the point cloud data.

[0028] 5 and 6 show an example in which a prepared 3D model (e.g., a CAD model) 50 of the tool 18 is used as a first image representing the three-dimensional shape of the tool 18. Based on the results obtained by actually detecting the tool 18 with the 3D sensor 20, the 3D model 50 corresponding to the tool 18 can be read from a memory or the like, and the 3D model 50 can be displayed in an appropriate position and orientation. By using the detection results of the 3D sensor 20 and the CAD model 50, the 3D model 50 having a rear end 52 connected to the flange 16 and a front end (here, two claws) 54 can be displayed on the display 26, and the user can operate the pointer 36 while visually checking the displayed 3D model 50.

[0029] 5 shows an example of changing the attitude of the pointer 36. For example, a sphere 56 is displayed with its center at the origin of the pointer 36, and the user can rotate the sphere 56 to rotate the pointer 36 to a desired attitude.

[0030] 6 shows an example of changing the position of the pointer 36. For example, the user can change the position of the origin 38 of the pointer 36, which has been set to a desired orientation by the operation in FIG. 5, to set the pointer 36 to a desired position and orientation, and a tool coordinate system can be set based on this pointer 36.

[0031] In this way, the tool coordinate system can be set by detecting the tool 18 corresponding to the CAD data using the 3D sensor 20, displaying a three-dimensional image of the CAD model 50 and the pointer 36 as the detection results on the display unit 26, and determining the position and orientation of the pointer 36 relative to the CAD model 50. Because the 3D model 50 accurately represents the shape of the tool 18, the user can accurately set the tool coordinate system at a desired position.

[0032] In the above embodiment, the PC 10 connected to the robot control device 6 has been described as an example of the setting device 10, but this is not limiting. For example, the teaching pendant 8 may have the function of the setting device, or the robot control device 6 may have the function of the setting device, and the above processing may be performed from the PC 10 connected to the robot control device 6 via a browser or the like.

[0033] According to an embodiment of the present disclosure, a result (first image) obtained by actually detecting a part of a robot with a 3D sensor is displayed, and the user can determine and change the position and orientation of a second image including orientation information while visually checking the first image. Therefore, even if the tool has a complex shape, the tool coordinate system can be set at any position, and even if there is an actual tool installation error or the like, the robot in a state including the error is detected by the 3D sensor, so the error does not affect the positional accuracy of the tool coordinate system.

[0034] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0035] The following additional notes are provided regarding the above-described embodiment and modifications.

[0036] (Supplementary Note 1) A tool coordinate system setting device comprising: a display unit that displays a first image representing a three-dimensional shape of a component of a robot and a second image including information regarding the orientation, the first image being acquired from a three-dimensional sensor whose position and orientation relationship with the robot is known; an operation unit that can change the position and orientation of the second image on the display unit; and a setting unit that sets a tool coordinate system having the same position and orientation as the position and orientation of the second image based on the relative position and orientation relationship between the first image and the second image.

[0037] (Supplementary Note 2) The setting device according to Supplementary Note 1, wherein the component is a tool attached to the robot, and the display unit displays the second image superimposed on the first image.

[0038] (Supplementary Note 3) The setting device according to Supplementary Note 1 or 2, wherein the display unit displays point cloud data representing a three-dimensional shape of the component.

[0039] (Supplementary Note 4) The setting device according to Supplementary Note 1 or 2, wherein the display unit displays a detection result detected by a predetermined detection tool in a manner superimposed on the point cloud data.

[0040] (Supplementary Note 5) The setting device according to Supplementary Note 4, wherein the display unit displays a set of connected points, which are detected by a predetermined detection tool and which connect adjacent points in the point cloud data, superimposed on the point cloud data.

[0041] (Supplementary Note 6) The setting device according to claim 4, wherein the display unit displays a segmentation generated using AI in a manner superimposed on the point cloud data.

[0042] (Supplementary Note 7) The setting device according to Supplementary Note 4, wherein the display unit displays a CAD model representing a three-dimensional shape of the component as the detection result.

[0043] (Supplementary Note 8) A robot system comprising: a robot; a three-dimensional sensor whose position and orientation relationship with the robot is known; a display unit that displays a first image representing a three-dimensional shape of a component of the robot and a second image including information about the orientation, both acquired from the three-dimensional sensor; an operation unit that can change the position and orientation of the second image on the display unit; and a setting unit that sets a tool coordinate system having the same position and orientation as the position and orientation of the second image based on the relative position and orientation relationship between the first image and the second image.

[0044] (Supplementary Note 9) A method for setting a tool coordinate system, comprising: displaying on a display unit a first image representing a three-dimensional shape of a component of a robot and a second image including information regarding the orientation, the first image being acquired from a three-dimensional sensor whose position and orientation relationship with the robot is known; changing the position and orientation of the second image on the display unit; and setting a tool coordinate system having the same position and orientation as the position and orientation of the second image based on the relative position and orientation relationship between the first image and the second image.

[0045] 2 Robot system 4 Robot 6 Robot control device 8 Teaching operation panel 10 Setting device 12 Base 14 Robot arm 16 Flange 18 Tool 20 3D sensor 22 Stand 24 Input unit 26 Display 28 Electronic component 30 Point cloud data 36 Pointer 46, 48 Frame 50 CAD model

Claims

1. A tool coordinate system setting device, comprising: a display unit that displays a first image representing a three-dimensional shape of a component of the robot, acquired from a three-dimensional sensor whose positional and postural relationship with the robot is known, and a second image including information regarding the posture; an operation unit that can change the positional and postural relationship of the second image on the display unit; and a setting unit that sets a tool coordinate system having the same positional and postural relationship as that of the second image, based on the relative positional and postural relationship between the first image and the second image.

2. The setting device according to claim 1, wherein the component is a tool attached to the robot, and the display unit superimposes and displays the second image on the first image.

3. The setting device according to claim 1 or 2, wherein the display unit displays point cloud data representing the three-dimensional shape of the component.

4. The setting device according to claim 1 or 2, wherein the display unit superimposes and displays a detection result detected by a predetermined detection tool on the point cloud data.

5. The setting device according to claim 4, wherein the display unit superimposes and displays a set of connection points obtained by connecting adjacent points in the point cloud data, detected by a predetermined detection tool, on the point cloud data.

6. The setting device according to claim 4, wherein the display unit superimposes and displays a segmentation generated using AI on the point cloud data.

7. The setting device according to claim 4, wherein the display unit displays a CAD model representing the three-dimensional shape of the component as the detection result.

8. A robot system, comprising: a robot; a three-dimensional sensor whose positional and postural relationship with the robot is known; a display unit that displays a first image representing a three-dimensional shape of a component of the robot, acquired from the three-dimensional sensor, and a second image including information regarding the posture; an operation unit that can change the positional and postural relationship of the second image on the display unit; and a setting unit that sets a tool coordinate system having the same positional and postural relationship as that of the second image, based on the relative positional and postural relationship between the first image and the second image.

9. Displaying, on a display unit, a first image representing a three-dimensional shape of a component of the robot, acquired from a three-dimensional sensor whose positional and orientation relationship with the robot is known, and a second image including information regarding orientation; changing a positional and orientation of the second image on the display unit; and setting a tool coordinate system having the same positional and orientation as that of the second image based on a relative positional and orientation relationship between the first image and the second image. A method for setting a tool coordinate system including the above steps.

Images