Apparatus, robot system, and method for teaching a robot the position and posture at which it grips a workpiece

Abstract

Conventionally, there has been a need for technologies enabling highly-accurate teaching of a position at which a robot is made to grasp a workpiece by a hand.　A device 100 includes: an image data acquisition unit 82 that acquires, when a robot 12 is grasping a workpiece by a hand, image data of the workpiece imaged by a visual sensor 14 disposed at a known position on a control coordinate system; a workpiece position acquisition unit 84 that acquires workpiece position data indicating a position and a posture of the workpiece on the basis of the image data; a hand position acquisition position 86 that acquires hand position data indicating a position and a posture of the hand obtained when the visual sensor has imaged the image data; and a teaching position acquisition unit 88 that acquires, on the basis of the workpiece position data and the hand position data, teaching position data indicating a positional relationship between the hand and the workpiece obtained when the visual sensor 14 has imaged the image data.

Description

[Technical Field]

[0001] The present disclosure relates to an apparatus, a robot system, and a method for teaching a position and posture for a robot to grip a workpiece. [Background technology]

[0002] BACKGROUND ART A robot system is known in which a robot performs an operation to grip a workpiece with its hand based on image data captured by a visual sensor (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-209979 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, there has been a demand for technology that can teach a robot the position at which it should grip a workpiece with its hand with high precision. [Means for solving the problem]

[0005] In one aspect of the present disclosure, an apparatus for teaching a position and posture of a robot in a control coordinate system for controlling the robot at which the hand grasps a workpiece includes an image data acquisition unit that acquires image data of the workpiece captured by a visual sensor positioned at a known position in the control coordinate system when the robot is grasping the workpiece with its hand, a work position acquisition unit that acquires, based on the image data, work position data indicating the position and posture of the workpiece in the control coordinate system at the time the visual sensor captured the image data, a hand position acquisition unit that acquires hand position data indicating the position and posture of the hand in the control coordinate system at the time the visual sensor captured the image data, and a teaching position acquisition unit that acquires, based on the work position data and the hand position data, teaching position data indicating the positional relationship between the hand and the workpiece in the control coordinate system at the time the visual sensor captured the image data.

[0006] In another aspect of the present disclosure, a method for teaching a position and posture of a robot in a control coordinate system for controlling the robot at which the hand grasps a workpiece includes a processor acquiring image data of the workpiece captured by a visual sensor positioned at a known position in the control coordinate system when the robot is grasping the workpiece with its hand, acquiring work position data indicating the position and posture of the workpiece in the control coordinate system at the time the visual sensor captured the image data based on the image data, acquiring hand position data indicating the position and posture of the hand in the control coordinate system at the time the visual sensor captured the image data, and acquiring teaching position data indicating the positional relationship between the hand and the workpiece in the control coordinate system at the time the visual sensor captured the image data based on the work position data and the hand position data. [Effects of the Invention]

[0007] By acquiring teaching position data based on image data captured when the operator actually grasps the workpiece with the hand at the gripping position that the operator wants to teach, the robot can be taught the gripping position that the operator wants to teach with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view of a robot system according to an embodiment. [Figure 2] FIG. 2 is a block diagram of the robot system shown in FIG. [Figure 3] FIG. 2 is an enlarged view of the hand shown in FIG. [Figure 4] FIG. 1 is a diagram of a workpiece according to one embodiment. [Figure 5] The hand shown in FIG. 3 is shown holding the workpiece shown in FIG. 4. [Figure 6] This is an example of image data of a workpiece being held by a hand captured by the visual sensor shown in FIG. [Figure 7] 7A and 7B are schematic diagrams illustrating a state in which a workpiece model is applied to the image data shown in FIG. 6. [Figure 8] 4 is a flowchart showing an example of a method for teaching the robot in the robot system shown in FIG. 1 a position and posture for gripping a workpiece with a hand. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In various embodiments described below, similar elements will be given the same reference numerals, and duplicate explanations will be omitted. First, a robot system 10 according to one embodiment will be described with reference to FIGS. 1 and 2. The robot system 10 performs the task of picking up workpieces W piled up in a container A, and includes a robot 12, a visual sensor 14, a control device 16, and a teaching device 18.

[0010] In this embodiment, the robot 12 is a vertical articulated robot and includes a robot base 20, a rotating body 22, a lower arm 24, an upper arm 26, a wrist 28, and a hand 30. The robot base 20 is fixed to the floor of a work cell. The rotating body 22 is attached to the robot base 20 so as to be rotatable about a vertical axis. The lower arm 24 is attached to the rotating body 22 so as to be rotatable about a horizontal axis, and the upper arm 26 is rotatably attached to the distal end of the lower arm 24. The wrist 28 is rotatably attached to the distal end of the upper arm 26.

[0011] The robot base 20, the rotating body 22, the lower arm 24, the upper arm 26, and the wrist 28 are each provided with a plurality of servo motors 40 (FIG. 2). In response to commands from the control device 16, the servo motors 40 rotate the rotating body 22, the lower arm 24, the upper arm 26, and the wrist 28 around their respective drive shafts, thereby operating the robot 12. In this document, the robot base 20, the rotating body 22, the lower arm 24, the upper arm 26, and the wrist 28 are referred to as a mechanical unit 42 of the robot 12.

[0012] The hand 30 is detachably attached to the tip of the wrist 28 (so-called wrist flange) and is moved by a mechanical unit 42 of the robot 12. Specifically, as shown in Fig. 3, the hand 30 has a hand arm 32, claws 34 and 36, and a claw drive unit 38. The base end of the hand arm 32 is connected to the tip of the wrist 28.

[0013] The claws 34 and 36 are provided at the tip of the hand arm 32 so as to be able to open and close. In this embodiment, each of the claws 34 and 36 is a cylindrical rod member that extends linearly. The claw drive unit 38 has, for example, a pneumatic or hydraulic cylinder or a servo motor, and opens and closes the claws 34 and 36 in response to commands from the control device 16. The hand 30 can grip and release the workpiece W by opening and closing the claws 34 and 36.

[0014] 1 and 2, the visual sensor 14 captures an image of the workpiece W. Specifically, the visual sensor 14 is fixed to a holding frame 44. The holding frame 44 is fixed on the floor of the work cell, and positions the visual sensor 14 in a stationary position above the container A.

[0015] In this embodiment, the visual sensor 14 is a three-dimensional visual sensor having an imaging sensor (CMOS, CCD, etc.) and an optical lens (collimator lens, focus lens, etc.) that guides the subject image to the imaging sensor, and is configured to image the subject along the line of sight VL and measure the distance d to the subject image.

[0016] The teaching device 18 teaches the robot 12 an operation to grasp the workpieces W piled up in a container A with the hand 30. Specifically, the teaching device 18 is, for example, a portable computer such as a teaching pendant or a tablet terminal device, and has a processor 50, a memory 52, an I / O interface 54, a display device 56, and an input device 58. The processor 50 has a CPU or a GPU, etc., and is communicatively connected to the memory 52, the I / O interface 54, the display device 56, and the input device 58 via a bus 60, and performs arithmetic processing to realize a teaching function, which will be described later, while communicating with these components.

[0017] The memory 52 has RAM, ROM, or the like, and temporarily or permanently stores various data. The I / O interface 54 has, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices via wired or wireless communication under instructions from the processor 50.

[0018] The display device 56 has a liquid crystal display, an organic EL display, or the like, and visibly displays various data under instructions from the processor 50. The input device 58 has a push button, a keyboard, a mouse, a touch panel, or the like, and receives input data from an operator.

[0019] The teaching device 18 is configured to send commands to the robot 12 via the control device 16 in accordance with input data to the input device 58, and to cause the robot 12 to perform a jog motion in accordance with the commands. The display device 56 and the input device 58 may be integrated into the housing of the teaching device 18, or may be separate from the housing of the teaching device 18 and attached externally to the housing.

[0020] The control device 16 controls the operations of the robot 12 and the visual sensor 14. Specifically, the control device 16 is a computer having a processor 70, a memory 72, an I / O interface 74, a display device 76, and an input device 78. The configurations and functions of the processor 70, the memory 72, the I / O interface 74, the display device 76, and the input device 78 are the same as those of the processor 50, the memory 52, the I / O interface 54, Since it is similar to the display device 56 and the input device 58, a duplicated description will be omitted.

[0021] The processor 70 is communicatively connected to the memory 72, the I / O interface 74, the display device 76, and the input device 78 via a bus 80, and performs arithmetic processing to realize functions for operating the robot 12 and the visual sensor 14 while communicating with these components. The I / O interface 54 of the teaching device 18, the servo motors 40 of the robot 12, and the visual sensor 14 are connected to the I / O interface 74, and the processor 70 communicates with these components through the I / O interface 74.

[0022] 1, a robot coordinate system C1 is set in the mechanical unit 42 of the robot 12. The robot coordinate system C1 is a control coordinate system C for automatically controlling the operation of the robot 12 to grip the workpiece W with the hand 30. In this embodiment, the robot coordinate system C1 is set with respect to the mechanical unit 42 so that its origin is located at the center of the robot base 20 and its z axis coincides with the rotation axis of the rotating body 22.

[0023] 3, a tool coordinate system C2 is set for the hand 30 of the robot 12. The tool coordinate system C2 is a control coordinate system C that defines the position and posture of the hand 30 in the robot coordinate system C1. In this embodiment, the tool coordinate system C2 is set for the hand 30 so that its origin is located at the midpoint between the claws 34 and 36, its y-axis direction is parallel to the opening and closing directions of the claws 34 and 36, and its z-axis direction is parallel to the extension directions of the claws 34 and 36.

[0024] The positional relationship between the tool coordinate system C2 and the robot coordinate system C1 is known, and the coordinates of the tool coordinate system C2 and the coordinates of the robot coordinate system C1 can be converted to each other via a known transformation matrix M1 (for example, a homogeneous transformation matrix). Therefore, the origin position and the directions of each axis of the tool coordinate system C2 in the robot coordinate system C1 can be determined by the coordinates (X RT ,Y RT ,Z RT ,W RT ,P RT ,R RT ) where the coordinates (X RT ,Y RT ,Z RT ) indicates the origin position of the tool coordinate system C2 in the robot coordinate system C1, and the coordinate (W RT ,P RT ,R RT ) indicate the directions of the axes (so-called yaw, pitch, and roll) of the tool coordinate system C2 in the robot coordinate system C1.

[0025] When the mechanical unit 42 of the robot 12 positions the hand 30 at a predetermined position and orientation, the processor 70 of the control device 16 first sets a tool coordinate system C2, which represents the predetermined position and orientation, in the robot coordinate system C1. The processor 70 then generates commands to the servo motors 40 to place the hand 30 at the position and orientation defined by the set tool coordinate system C2, and operates the mechanical unit 42 in accordance with the commands, thereby moving the hand 30. In this way, the processor 70 can position the hand 30 at a predetermined position and orientation in the robot coordinate system C1 by operating the mechanical unit 42.

[0026] 1, a sensor coordinate system C3 is set for the visual sensor 14. The sensor coordinate system C3 is a control coordinate system C that defines the position and orientation of the visual sensor 14 in the robot coordinate system C1 (i.e., the line of sight direction VL), and also defines the coordinates of each pixel of the image data (or the image sensor) captured by the visual sensor 14. In this embodiment, the sensor coordinate system C3 is set for the visual sensor 14 so that its origin is located at the center of the image sensor and its z-axis direction coincides with the line of sight direction VL of the visual sensor 14.

[0027] In this embodiment, the visual sensor 14 is placed at a known position in the robot coordinate system C1 by the holding frame 44. More specifically, the positional relationship between the sensor coordinate system C3 and the robot coordinate system C1 is known through calibration, and the coordinates of the sensor coordinate system C3 and the coordinates of the robot coordinate system C1 are mutually convertible via a known transformation matrix M2 (e.g., a homogeneous transformation matrix). In this way, the position and orientation of the visual sensor 14 in the robot coordinate system C1 (i.e., the origin position and directions of each axis of the sensor coordinate system C3) are known.

[0028] In an actual work line, the processor 70 of the control device 16 captures images of the workpieces W randomly piled in a container A with the visual sensor 14, and, based on the captured image data ID, causes the robot 12 to grasp the workpieces W with the hand 30 and pick them up from the container A. An example of the workpieces W is shown in FIG. 4.

[0029] In the example shown in Figure 4, the workpiece W is, for example, a connecting rod for an automobile, and has a shaft portion W1, a large ring portion W2, and a small ring portion W3. The shaft portion W1 extends straight along the axis B. The large ring portion W2 is fixed to one end of the shaft portion W1 and has a through hole H1. On the other hand, the small ring portion W3 is fixed to the other end of the shaft portion W1 and has a through hole H2.

[0030] In order for the robot 12 to perform an operation of gripping the workpiece W with the hand 30, a workpiece coordinate system C4 is set for the workpiece W. The workpiece coordinate system C4 is a control coordinate system C that defines the position and posture of the workpiece W in the robot coordinate system C1. In this embodiment, the workpiece coordinate system C4 is set with respect to the workpiece W so that its origin is located at the center of the shaft portion W1, its y axis is parallel to (or coincides with) the axis B, and its z axis is parallel to the central axes of the through holes H1 and H2.

[0031] Fig. 5 shows an example of a state in which the robot 12 grips the workpiece W with the hand 30. In the example shown in Fig. 5, the hand 30 opens its claws 34 and 36 while inserted into the through-hole H1, thereby gripping the large ring portion W2 of the workpiece W with the claws 34 and 36. In order to have the robot 12 perform the operation of gripping the workpiece W as shown in Fig. 5, it is necessary to teach the position and posture in which the robot 12 grips the workpiece W with the hand 30.

[0032] A method for teaching the robot 12 in the robot system 10 to grip the workpiece W will be described below. First, the operator causes the hand 30 of the robot 12 to grip the workpiece W at the gripping position that the operator wishes to teach. For example, the operator operates the teaching device 18 to jog the robot 12, thereby causing the hand 30 to grip the workpiece W.

[0033] More specifically, while viewing the display device 56 of the teaching device 18, the operator operates the input device 58 to move the hand 30 using the mechanism unit 42 of the robot 12, and inserts the claws 34 and 36 in the closed state into the through-hole H1 of the workpiece W stored in a predetermined storage location. Next, the operator operates the input device 58 to open the claws 34 and 36, and presses the claws 34 and 36 against the inner wall surface of the through-hole H1 to grip the workpiece W.

[0034] At this time, the operator jogs the robot 12 so that the hand 30 grasps the workpiece W at the gripping position that the operator wants to teach. Below, a case where the operator has the hand 30 grasp the workpiece W at the gripping position shown in Fig. 5 will be described. Next, the operator operates the input device 58 to jog the robot 12, and causes the mechanical unit 42 of the robot 12 to move the hand 30 so that the workpiece W grasped by the hand 30 is within the field of view of the visual sensor 14.

[0035] Next, the operator operates the input device 58 to cause the visual sensor 14 to capture an image of the workpiece W held by the hand 30. The visual sensor 14 receives an image capture command from the teaching device 18 via the control device 16, and captures image data ID1 of the workpiece W. An example of the image data ID1 is shown in FIG.

[0036] In the example shown in FIG. 6, the visual features (edges, holes, vertices, etc.) of the workpiece W and the hand 30 are displayed as a three-dimensional point cloud in the image data ID1. Each point constituting the three-dimensional point cloud has information on the distance d described above, and is represented by three-dimensional coordinates (X S ,Y S ,Z S ) In other words, in this embodiment, the image data ID1 is three-dimensional image data.

[0037] The processor 50 of the teaching device 18 acquires image data ID1 from the visual sensor 14 through the control device 16 and the I / O interface 54. In this manner, in this embodiment, the processor 50 functions as an image data acquisition unit 82 (FIG. 2) that acquires image data ID of the workpiece W captured by the visual sensor 14 when the robot 12 is holding the workpiece W with the hand 30.

[0038] Next, based on the image data ID1, the processor 50 acquires workpiece position data WPD1 that indicates the position and orientation of the workpiece W in the robot coordinate system C1 when the visual sensor 14 captured the image data ID1. Specifically, the processor 50 first acquires a workpiece model WM that is a model of the workpiece W. This workpiece model WM is, for example, a three-dimensional CAD model, and is stored in advance in the memory 52.

[0039] The processor 50 uses predetermined pattern matching parameters to analyze the three-dimensional point cloud image of the workpiece W depicted in the image data ID1, and simulates the placement of the workpiece model WM within the image data ID1 so that it matches the workpiece W depicted in the image data ID1. Figure 7 shows the simulated placement of the workpiece model WM in the image data ID1. Note that, for ease of understanding, an image of the hand 30 is omitted from Figure 7.

[0040] The processor 50 then sets a workpiece coordinate system C4 with respect to the workpiece model WM arranged in the image data ID1 in the positional relationship shown in Fig. 4. This workpiece coordinate system C4 indicates the position and orientation of the workpiece W shown in the image data ID1 in the sensor coordinate system C3, and the coordinate Q of the sensor coordinate system C3 SW_1 (X SW_1 ,Y SW_1 ,Z SW_1 ,W SW_1 ,P SW_1 ,R SW_1 ) (first coordinate).

[0041] Here, the coordinate (X SW_1 ,Y SW_1 ,Z SW_1 ) indicates the origin position of the workpiece coordinate system C4 in the sensor coordinate system C3, and the coordinate (W SW_1 ,P SW_1 ,R SW_1 ) indicate the directions of the axes (so-called yaw, pitch, and roll) of the workpiece coordinate system C4 in the sensor coordinate system C3. The processor 50 calculates the coordinates Q of the workpiece coordinate system C4 in the sensor coordinate system C3 as data indicating the position and orientation of the workpiece W in the sensor coordinate system C3, which are shown in the image data ID1.SW_1 Get.

[0042] Then, the processor 50 converts the obtained coordinates Q SW_1 By converting the coordinates into the robot coordinate system C1, the coordinate Q in the robot coordinate system C1 of the workpiece coordinate system C4 shown in Fig. 7 is RW_1 (X RW_1 ,Y RW_1 ,Z RW_1 ,W RW_1 ,P RW_1 ,R RW_1 ) (second coordinate). This coordinate Q RW_1 is data indicating the position and orientation in the robot coordinate system C1 of the workpiece W (specifically, the workpiece model WM in FIG. 7) shown in the image data ID1.

[0043] The processor 50 calculates this coordinate Q RW_1 As described above, in this embodiment, the processor 50 obtains the workpiece position data WPD1 (coordinates Q RW_1 ) and functions as a work position acquisition unit 84 (FIG. 2).

[0044] Meanwhile, the processor 50 acquires hand position data HPD1 that indicates the position and posture of the hand 30 in the robot coordinate system C1 when the visual sensor 14 captures the image data ID1. Specifically, the processor 50 acquires the coordinates Q of the tool coordinate system C2 in the robot coordinate system C1 when the visual sensor 14 captures the image data ID1. RT_1 (X RT_1 ,Y RT_1 ,Z RT_1 ,W RT_1 ,P RT_1 ,R RT_1 ) (third coordinate) as the hand position data HPD1. In this manner, in the present embodiment, the processor 50 acquires the hand position data HPD1 (coordinate Q RT_1 ) functions as the hand position acquisition unit 86 (FIG. 2).

[0045] Then, based on the acquired work position data WPD1 and hand position data HPD1, the processor 50 acquires teaching position data TPD1 that indicates the positional relationship between the hand 30 and the work W in the control coordinate system C when the visual sensor 14 captured the image data ID1.

[0046] As an example, the processor 50 calculates the coordinate Q obtained as the workpiece position data WPD1. RW_1 and the coordinate Q obtained as hand position data HPD1 RT_1 and the workpiece position data WPD1 as coordinate Q RW_1 The hand position data HPD1 is used as the coordinate Q RT_1 where the coordinates Q RW_1 and Q RT_1 Since the positional relationship between the tool coordinate system C2 and the workpiece coordinate system C4 in the robot coordinate system C1 is known, the coordinate Q of the workpiece coordinate system C4 in the robot coordinate system C1 is RW_1 can be transformed into the tool coordinate system C2.

[0047] By this coordinate transformation, the processor 50 calculates the coordinate Q of the workpiece coordinate system C4 in the tool coordinate system C2 when the visual sensor 14 captured the image data ID1. TW_1 (X TW_1 ,Y TW_1 ,Z TW_1 ,W TW_1 ,P TW_1 ,R TW_1 ) to obtain the coordinate Q TW_1 is data indicating the position and orientation of the workpiece W (i.e., the origin position and directions of each axis of the workpiece coordinate system C4) relative to the hand 30 (i.e., the tool coordinate system C2) when the visual sensor 14 captured the image data ID1. The processor 50 calculates the coordinates Q TW_1 is acquired as the taught position data TPD1.

[0048] As another example, the processor 50 may use the coordinate Q as the workpiece position data WPD1. RW_1 and the coordinate Q as the hand position data HPD1 RT_1Based on the hand position data HPD1, coordinate Q RT_1 coordinate Q as work position data WPD1 RW_1 As a result, the processor 50 converts the coordinates Q of the tool coordinate system C2 in the workpiece coordinate system C4 when the visual sensor 14 captures the image data ID1 into coordinates of the workpiece coordinate system C4 represented by WT_1 (X WT_1 ,Y WT_1 ,Z WT_1 ,W WT_1 ,P WT_1 ,R WT_1 ) to get the

[0049] This coordinate Q WT_1 is data indicating the position and orientation of the hand 30 (i.e., the origin position and directions of each axis of the tool coordinate system C2) relative to the workpiece W (i.e., the workpiece coordinate system C4) when the visual sensor 14 captured the image data ID1. The processor 50 calculates the coordinates Q WT_1 is acquired as the taught position data TPD1.

[0050] As described above, in this embodiment, the processor 50 generates the teaching position data TPD1 (coordinate Q) indicating the positional relationship between the hand 30 and the workpiece W in the control coordinate system C (tool coordinate system C2, workpiece coordinate system C4) when the visual sensor 14 captured the image data ID1 based on the workpiece position data WPD1 and the hand position data HPD1. TW_1 , coordinate Q WT_1 ) is acquired by the teaching position acquisition unit 88 (Fig. 2 )

[0051] After acquiring the teaching position data TPD1, the operator operates the input device 58 to change the posture of the hand 30 while gripping the workpiece W by operating the mechanical unit 42 of the robot 12. For example, the operator operates the input device 58 to input input data for rotating the hand 30 by a predetermined angle θ (for example, 10°) around the x-axis, y-axis, or z-axis of the tool coordinate system C2, which is set in the robot coordinate system C1 at this point.

[0052] Alternatively, the operator may input input data for rotating the hand 30 by a predetermined angle θ around the x-axis, y-axis, or z-axis of the workpiece coordinate system C4, which is currently set in the robot coordinate system C1. For example, if the y-axis of the workpiece coordinate system C4 coincides with the axis B of the workpiece W, the operator may input input data for rotating the hand 30 around the y-axis of the workpiece coordinate system C4 (i.e., the axis B).

[0053] The processor 50 sends a command to the servo motor 40 of the robot 12 via the control device 16 in response to input data from the operator, and operates the mechanism unit 42 to change the posture of the hand 30 gripping the workpiece W. In this manner, in this embodiment, the processor 50 functions as a robot control unit 90 (FIG. 2) that operates the robot 12 to change the posture of the hand 30.

[0054] When the posture of the hand 30 is changed, the operator operates the input device 58 to cause the visual sensor 14 to capture an image of the workpiece W held by the hand 30, and the processor 50 functions as an image data acquisition unit 82 to acquire image data ID2 of the workpiece W. Then, the processor 50 functions as a workpiece position acquisition unit 84, and by applying the workpiece model WM to the image data ID2 using the method described above, the coordinate Q of the workpiece coordinate system C4 in the sensor coordinate system C3 is obtained. SW_2 (X SW_2 ,Y SW_2 ,Z SW_2 ,W SW_2 ,P SW_2 ,R SW_2 ) (first coordinate).

[0055] Next, the processor 50 functions as the workpiece position acquisition unit 84, and calculates the acquired coordinate Q SW_2 By converting this into the robot coordinate system C1, the coordinate Q of the workpiece coordinate system C4 in the robot coordinate system C1 is obtained as the workpiece position data WPD2 at the time of capturing the image data ID2. RW_2 (X RW_2 ,Y RW_2 ,Z RW_2 ,WRW_2 ,P RW_2 ,R RW_2 ) (second coordinate).

[0056] The processor 50 also functions as a hand position acquisition unit 86, and acquires the coordinates Q of the tool coordinate system C2 in the robot coordinate system C1 as hand position data HPD2 at the time of capturing the image data ID2 using the method described above. RT_2 (X RT_2 ,Y RT_2 ,Z RT_2 ,W RT_2 ,P RT_2 ,R RT_2 ) (third coordinate).

[0057] Then, the processor 50 functions as a teaching position acquisition unit 88, and acquires, by the above-described method, the coordinate Q of the workpiece coordinate system C4 in the tool coordinate system C2 as teaching position data TPD2 at the time of capturing the image data ID2. TW_2 (X TW_2 ,Y TW_2 ,Z TW_2 ,W TW_2 ,P TW_2 ,R TW_2 ) or the coordinate Q of the tool coordinate system C2 in the work coordinate system C4 WT_2 (X WT_2 ,Y WT_2 ,Z WT_2 ,W WT_2 ,P WT_2 ,R WT_2 ) to get the

[0058] In this way, the operator operates the input device 58 to repeatedly change the posture of the hand 30, and the visual sensor 14 captures an image of the workpiece W held by the hand 30 each time the posture of the hand 30 is changed. The processor 50 functions as an image data acquisition unit 82 and acquires a plurality of image data IDs captured by the visual sensor 14. n Obtain (n=1,2,3,···).

[0059] Then, the processor 50 functions as the work position acquisition unit 84 and acquires each image data ID by the above-described method. nWork position data WPD when the image was captured n : Coordinate Q RW_n (X RW_n ,Y RW_n ,Z RW_n ,W RW_n ,P RW_n ,R RW_n ) and functions as the hand position acquisition unit 86, and acquires each image data ID n HPD: Hand position data when the image was captured n : Coordinate Q RT_n (X RT_n ,Y RT_n ,Z RT_n ,W RT_n ,P RT_n ,R RT_n ) respectively.

[0060] The processor 50 then functions as a teaching position acquisition unit 88 and acquires the workpiece position data WPD by the above-described method. n (coordinate Q RW_n ) and each hand position data HPD n (coordinate Q RT_n ) and each image data ID n TPD is the teaching position data when the image was captured. n : Coordinate Q TW_n (X TW_n ,Y TW_n ,Z TW_n ,W TW_n ,P TW_n ,R TW_n ), or coordinate Q WT_n (X WT_n ,Y WT_n ,Z WT_n ,W WT_n ,P WT_n ,R WT_n ) respectively. In this way, the processor 50 acquires a plurality of teaching position data TPD corresponding to various postures of the hand 30 and the workpiece W. n can be obtained.

[0061] Next, the processor 50 functions as a teaching position acquisition unit 88 and acquires the plurality of teaching position data TPD nBased on this, new teaching position data TPD0 is calculated which is used for the operation of making the robot 12 actually grasp the workpiece W with the hand 30. n As the coordinate Q of the tool coordinate system C2, TW_n (X TW_n ,Y TW_n ,Z TW_n ,W TW_n ,P TW_n ,R TW_n ) is acquired, a method for obtaining new teaching position data TPD0 will be described.

[0062] First, the processor 50 calculates a plurality of coordinates Q TW_n Specifically, the processor 50 performs a process PR1 to exclude coordinates outside a predetermined allowable range for each of the coordinates Q TW_n Among them, the coordinates that represent the position (X TW_n ,Y TW_n ,Z TW_n ), a process PR1 is performed to exclude coordinates outside the allowable range.

[0063] As an example, the processor 50 may calculate a distance Δ n Δ n =(X TW_n 2 +Y TW_n 2 +Z TW_n 2 ) 1 / 2 Then, the processor 50 calculates the calculated distance Δ n is within a predetermined tolerance range [Δ th1 ,Δ th2 ] and determine whether it is within the distance Δ n is the tolerance [Δ th1 ,Δ th2 ] (i.e., Δ th1 ≦Δ n ≦Δ th2 ) if the obtained coordinate Q TW_n is registered in the memory 52 as a valid coordinate group GRP, while the distance Δ n is the tolerance [Δ th1 ,Δ th2 ] outside (i.e., Δn <Δ th1 , or Δ th2 <Δ n ) if the obtained coordinate Q TW_n is excluded from the valid coordinate group GRP (or deleted from the memory 52).

[0064] As another example, processor 50 may calculate coordinate Q TW_n (X TW_n ,Y TW_n ,Z TW_n ) average coordinate Q TW_AV (X TW_AV ,Y TW_AV ,Z TW_AV Specifically, the processor 50 calculates X TW_AV =1 / n Σ(X TW_n ), Y TW_AV =1 / n Σ(Y TW_n ), and Z TW_AV =1 / n Σ(Z TW_n ) from the formula, the average coordinate Q TW_AV (X TW_AV ,Y TW_AV ,Z TW_AV ) is found.

[0065] The processor 50 also calculates the coordinate Q TW_n (X TW_n ,Y TW_n ,Z TW_n ) standard deviation σ X , σ Y and σ Z For example, the processor 50 calculates σ X =(1 / n·Σ{X TW_n -X TW_AV}) 1 / 2 , σ Y =(1 / n·Σ{Y TW_n -Y TW_AV}) 1 / 2 , σ Z =(1 / n·Σ{Z TW_n -Z TW_AV}) 1 / 2 It is calculated from the following formula.

[0066] Then, the processor 50 calculates the coordinate Q TW_n Coordinate X in TW_n , YTW_n , and Z TW_n For each of the above, the tolerance range is calculated by using the calculated mean and standard deviation σ and a predetermined coefficient α (for example, α is a positive integer) to obtain [X TW_AV -ασ X ,X TW_AV +ασ X ] (i.e., X TW_AV -ασ X ≦X TW_n ≦X TW_AV +ασ X ), [Y TW_AV -ασ Y ,Y TW_AV +ασ Y ] (i.e., Y TW_AV -ασ Y ≦Y TW_n ≦Y TW_AV +ασ Y ), and [Z TW_AV -ασ Z ,Z TW_AV +ασ Z ] (i.e., Z TW_AV -ασ Z ≦Z TW_n ≦Z TW_AV +ασ Z ) is defined as follows.

[0067] The processor 50 calculates the coordinate X TW_n is within the tolerance range [X TW_AV -σ X ,X TW_AV +σ X ] whether it is within the Y coordinate TW_n is within the tolerance range [Y TW_AV -σ Y ,Y TW_AV +σ Y ] and coordinate Z TW_n is within the tolerance range [Z TW_AV -σ Z ,Z TW_AV +σ Z ] and determine whether it is within the range.

[0068] Then, the processor 50 calculates the coordinate X TW_n , Y TW_n , and Z TW_n If all of the above are within the tolerance range, the obtained coordinate QTW_n is registered as a valid coordinate group GRP, while coordinate X TW_n , Y TW_n , and Z TW_n If at least one of the coordinates is outside the tolerance range, the obtained coordinate Q TW_n is excluded from the valid coordinate group GRP.

[0069] The processor 50 also calculates the coordinate Q TW_n Among them, the coordinates that represent the posture (W TW_n ,P TW_n ,R TW_n ) that are outside the allowable range. Specifically, the processor 50 first performs a process PR1 to exclude coordinates that are outside the allowable range. TW_n ,P TW_n ,R TW_n ) into the known 3x3 matrix M3 _n It is expressed as:

[0070] This matrix M3 _n In the above, the vector VT1 is expressed by the three parameters in the first column. _n is a unit vector that indicates the rotation component around the x-axis of the tool coordinate system C2, and is the vector VT2 expressed by the three parameters in the second column. _n is a unit vector that indicates the rotation component around the y-axis of the tool coordinate system C2, and is the vector VT3 expressed by the three parameters in the third column. _n is a unit vector indicating the rotation component around the z-axis of the tool coordinate system C2.

[0071] For example, the processor 50 may calculate the first coordinate Q TW_n (W TW_n ,P TW_n ,R TW_n ) matrix M3 _n Vector VT1 _n and the second coordinate Q TW_n+1 (W TW_n+1 ,P TW_n+1 ,R TW_n+1 ) matrix M3 _n+1 Vector VT1 _n+1 The dot product IP1 n This inner product IP1 nis the vector VT1 _n and vector VT1 _n+1 and the angle φ1 (specifically, cosφ1) between them, that is, the amount of change in the rotation component around the x-axis of the tool coordinate system C2.

[0072] The processor 50 also calculates the matrix M3 _n Vector VT3 _n and matrix M3 _n+1 Vector VT3 _n+1 The dot product IP3 n This inner product IP3 n is the vector VT3 _n and Vector VT3 _n+1 and the angle φ3 (specifically, cosφ3) between them, that is, the amount of change in the rotation component around the z-axis of the tool coordinate system C2.

[0073] Then, the processor 50 calculates the calculated inner product IP1 n is a predetermined threshold IP1 th More than (IP1 n ≧IP1 th ) and determine whether the inner product IP3 n However, the predetermined threshold IP3 th More than (IP3 n ≧IP3 th The processor 50 determines whether IP1 n ≧IP1 th , and IP3 n ≧IP3 th If so, the first coordinate Q is obtained. TW_n and the second coordinate Q TW_n+1 Both of these are registered in the memory 52 as a valid coordinate group GRP.

[0074] Meanwhile, processor 50 is n <IP1 th , or IP3 n <IP3 th If so, the first coordinate Q is obtained. TW_n and the second coordinate Q TW_n+1 Either one of the first coordinate Q and the second coordinate Q is excluded from the valid coordinate group GRP (or deleted from the memory 52).TW_n and the second coordinate Q TW_n+1 The operator may predetermine which of the above should be excluded.

[0075] The processor 50 calculates the first coordinate Q TW_n (W TW_n ,P TW_n ,R TW_n ) matrix M3 _n Vector VT1 _n and the first coordinate Q TW_n Each coordinate Q other than TW_i (W TW_i ,P TW_i ,R TW_i ) matrix M3 _i Vector VT1 _i ("i" is a positive integer other than "n") and the dot product IP1 i Similarly, the processor 50 may calculate the first coordinate Q TW_n matrix M3 _n Vector VT3 _n and the first coordinate Q TW_n Each coordinate Q other than TW_i matrix M3 _i Vector VT3 _i The dot product IP3 i may be calculated respectively.

[0076] Then, the processor 50 calculates the calculated inner product IP1 i Each of these has a threshold IP1 th More than (IP1 i ≧IP1 th ) and determine whether the inner product IP3 i Each of these has a threshold IP3 th More than (IP3 i ≧IP3 th The processor 50 may determine whether the calculated inner product IP1 i At least one (or all) of the IP1 i ≧IP1 th and the calculated inner product IP3 i At least one (or all) of the IP3 i ≧IP3 th If the first coordinate Q is satisfied,TW_n may be registered in the memory 52 as a valid coordinate group GRP.

[0077] Meanwhile, the processor 50 calculates the calculated inner product IP1 i All (or at least one) of the IP1 i <IP1 th Or the calculated inner product IP3 i All (or at least one) of the i <IP3 th If so, the first coordinate Q is obtained. TW_n The processor 50 may perform this process PR1 on the acquired coordinate Q TW_n may be repeated for all of the

[0078] Alternatively, the processor 50 may use vector VT1 _1 , VT1 _2 , VT1 _3 , VT1 _n The composite vector VT1 R =Σ(VT1 _n ) and calculate the resultant vector VT1 R and each vector VT1 _n The dot product IP1 R_n Then, the processor 50 calculates the calculated inner product IP1 R_n is a predetermined threshold IP1 Rth More than (IP1 R_n ≧IP1 Rth The processor 50 determines whether IP1 R_n ≧IP1 Rth If Q is TW_n is registered in memory 52 as an effective coordinate group GRP, while IP1 R_n <IP1 Rth If Q is TW_n the effective coordinate group GRP Exclude from (or delete it from memory 52).

[0079] The processor 50 uses the vector VT1 _n Similarly, the vector VT2_n or VT3 _n Regarding the composite vector VT2 R =Σ(VT2 _n ) or VT3 R =Σ(VT3 _n ) and calculate the resultant vector VT2 R or VT3 R and each vector VT2 _n or VT3 _n The inner product IP2 R_n or IP3 R_n Find the threshold IP2 Rth or IP3 Rth By comparing with the coordinate Q, the coordinate Q to be excluded from the valid coordinate group GRP TW_n can also be determined.

[0080] In this way, the processor 50 calculates a plurality of coordinates Q TW_n By this process PR1, the coordinates Q obtained by the false detection are excluded. TW_n It should be noted that the threshold values Δ th1 , Δ th2 , IP1 th , IP3 th , IP1 Rth , IP2 Rth or IP3 Rth (or the coefficient α) is predetermined by an operator.

[0081] After the exclusion process PR1, the processor 50 excludes the coordinate Q registered in the valid coordinate group GRP. TW_m (m is the coordinate Q registered in the valid coordinate group GRP TW_n Specifically, the processor 50 performs a process PR2 to average the coordinates Q registered in the valid coordinate group GRP. TW_m Among them, the coordinates that represent the position (X TW_m ,Y TW_m ,Z TW_m ) average coordinate (X TW_0 ,Y TW_0 ,Z TW_0 ) is found.

[0082] Specifically, the processor 50 TW_0 =1 / k Σ(X TW_m ), Y TW_0 =1 / k Σ(Y TW_m ), and Z TW_0 =1 / k Σ(Z TW_m ) from the formula, the average coordinate (X TW_0 ,Y TW_0 ,Z TW_0 ) is calculated. In this formula, "k" is the coordinate Q registered in the valid coordinate group GRP. TW_m Indicates the number of.

[0083] The processor 50 also calculates the coordinate Q registered in the valid coordinate group GRP. TW_m Among them, the coordinates that represent the posture (W TW_m ,P TW_m ,R TW_m Specifically, the processor 50 performs a process PR2 to average the coordinates (W TW_m ,P TW_m ,R TW_m ) for the above vector VT1 _m The composite vector VT1 R =Σ(VT1 _m ) and vector VT3 _m The composite vector VT3 R =Σ(VT3 _m ) is found.

[0084] The processor 50 then calculates the resultant vector VT1 R unit vector VT1 R ' and the composite vector VT3 R unit vector VT3 R ' and calculate the cross product OP1. This cross product OP1 is the unit vector VT1 R ' and unit vector VT3 R Then, the processor 50 normalizes the vector represented by the cross product OP1 to obtain the unit vector VT2 R ' is required.

[0085] The processor 50 then calculates the unit vector VT2 R' and unit vector VT3 R ' and the cross product OP2 is calculated, and the vector represented by the cross product OP2 is normalized to obtain the unit vector VT1 R Thus, the processor 50 calculates the unit vector VT1 R ", VT2 R ' and VT3 R ' to get.

[0086] The processor 50 then calculates these unit vectors VT1 R ", VT2 R ' and VT3 R The attitude represented by '(W TW_0 ,P TW_0 ,R TW_0 The coordinates of this posture indicate the direction of each axis of the workpiece coordinate system C4 in the tool coordinate system C2, and the x-axis direction of the workpiece coordinate system C4 is determined by the above-mentioned unit vector VT1. R " and the y-axis direction is the direction of the unit vector VT2 R ', and the z-axis direction is the unit vector VT3 R ' direction.

[0087] Alternatively, the processor 50 may calculate the coordinates of the pose (W TW_0 ,P TW_0 ,R TW_0 ) to find the unit vector VT1 R ' with vector VT2 _m The composite vector VT2 R =Σ(VT2 _m ) unit vector VT2 R ', and the unit vector VT1 R ' and the unit vector VT2 R ' and the cross product OP3 may be found.

[0088] The processor 50 then normalizes the vector represented by the cross product OP3 to obtain the unit vector VT3 R ', and the unit vector VT3 R ' and unit vector VT1 R' and the cross product OP4 is calculated, and the vector represented by the cross product OP4 is normalized to obtain the unit vector VT2 R The processor 50 may calculate the unit vector VT1 R ', VT2 R " and VT3 R ', the coordinates of the posture (W TW_0 ,P TW_0 ,R TW_0 ) can be obtained.

[0089] By the above-mentioned method, the processor 50 calculates the coordinate Q registered in the valid coordinate group GRP. TW_m As a result, the processor 50 calculates the coordinate Q as the teaching position data TPD0. TW_0 (X TW_0 ,Y TW_0 ,Z TW_0 ,W TW_0 ,P TW_0 ,R TW_0 ) can be obtained.

[0090] This coordinate Q TW_0 is the origin position (X TW_0 ,Y TW_0 ,Z TW_0 ), and the direction of each axis (W TW_0 ,P TW_0 ,R TW_0 In this way, the processor 50 functions as a teaching position acquisition unit 88 and acquires the plurality of coordinates Q TW_n From (n=1,2,3,4...), one teaching position data TPD0 (coordinate Q TW_0 ) is found.

[0091] The processor 50 uses the teaching position data TPD n As a result, the coordinate Q of the tool coordinate system C2 in the work coordinate system C4 WT_n (X WT_n ,Y WT_n ,Z WT_n ,W WT_n ,P WT_n ,R WT_n) is acquired, the coordinate Q of the tool coordinate system C2 in the work coordinate system C4 is used as new teaching position data TPD0 by the above method. WT_0 (X WT_0 ,Y WT_0 ,Z WT_0 ,W WT_0 ,P WT_0 ,R WT_0 ) can be obtained.

[0092] Next, the processor 50 uses the obtained teaching position data TPD0 to calculate the teaching position data TPD0 (i.e., the coordinate Q TW_0 or Q WT_0 ) is defined as an instruction code. Therefore, the processor 50 functions as an operating program generator 92 (FIG. 2) that generates the operating program OP.

[0093] The processor 70 of the control device 16 operates the robot 12 in accordance with the operation program OP on the actual work line, and executes an operation of grasping and picking up the workpieces W randomly piled in the container A with the hand 30. Specifically, the processor 70 operates the visual sensor 14 to capture an image of the workpieces W in the container A, and stores the captured image data ID. W (Second image data) is acquired from the visual sensor 14.

[0094] Next, the processor 70, like the work position acquisition unit 84 described above, W Based on the image data ID W Workpiece position data WPD showing the position and orientation of the workpiece W in the robot coordinate system C1 W Specifically, the processor 70 acquires the workpiece model WM and the image data ID W The workpiece model WM is placed so as to coincide with the workpiece W shown in the image, and a workpiece coordinate system C4 is set for the placed workpiece model WM.

[0095] Then, the processor 70 calculates the coordinate Q of the sensor coordinate system C3 of the set work coordinate system C4.SW_W (X SW_W ,Y SW_W ,Z SW_W ,W SW_W ,P SW_W ,R SW_W ) and obtain the coordinate Q SW_W is converted into the robot coordinate system C1, the workpiece position data WPD W As a result, the coordinate Q of the workpiece coordinate system C4 in the robot coordinate system C1 is RW_W (X RW_W ,Y RW_W ,Z RW_W ,W RW_W ,P RW_W ,R RW_W In this way, the processor 70 obtains the workpiece position data WPD , which indicates the position and orientation of the workpiece W in the robot coordinate system C1. W (coordinate Q RW_W ) to get the

[0096] Next, the processor 70 calculates the obtained workpiece position data WPD W and the teaching position data TPD0 defined in the operation program OP, the position and posture of the hand 30 in the robot coordinate system C1 when gripping the workpiece W imaged by the visual sensor 14 is determined.

[0097] Specifically, the processor 70 calculates the workpiece position data WPD W The coordinate Q obtained as RW_W and the teaching position data TPD0 (specifically, coordinate Q TW_0 or Q WT_0 ) and coordinate Q RW_W The coordinate Q of the robot coordinate system C1 has a positional relationship indicated by the teaching position data TPD0 with respect to the workpiece coordinate system C4 expressed as RT_0 (X RT_0 ,Y RT_0 ,Z RT_0 ,W RT_0 ,P RT_0 ,R RT_0 The processor 70 calculates the coordinates Q RT_0By setting the above, the position and posture of the hand 30 when gripping the workpiece W are determined.

[0098] Then, the processor 70 calculates the coordinates Q of the robot coordinate system C1 by calculating the coordinates Q of the hand 30 with the claws 34 and 36 kept closed. RT_0 The hand 30 is moved by operating the mechanism 42 so as to place the hand 30 in a position and orientation defined by the tool coordinate system C2 set at

[0047] . As a result, the claws 34 and 36 are inserted into the through-hole H1 of the workpiece W.

[0099] Then, the processor 70 operates the claw drive unit 38 to open the claws 34 and 36, thereby gripping the large ring portion W2 of the workpiece W with the claws 34 and 36. As a result, as shown in Fig. 5, the hand 30 can grip the workpiece W at the gripping position taught by the taught position data TPD0.

[0100] The processor 70 then operates the mechanism unit 42 to retract the hand 30 holding the workpiece W from the container A, thereby picking up the workpiece W. The processor 70 then repeatedly performs the above-described series of operations for each of the workpieces W randomly stacked in the container A, thereby performing the task of picking up the workpieces W randomly stacked in the container A with the hand 30.

[0101] As described above, in this embodiment, the processor 50 of the teaching device 18 functions as an image data acquisition unit 82, a work position acquisition unit 84, a hand position acquisition unit 86, a teaching position acquisition unit 88, a robot control unit 90, and an operation program generation unit 92, and teaches the position and posture in which the robot 12 grasps the workpiece W with the hand 30 in the robot coordinate system C1.

[0102] Therefore, the image data acquisition unit 82, workpiece position acquisition unit 84, hand position acquisition unit 86, teaching position acquisition unit 88, robot control unit 90, and operation program generation unit 92 constitute a device 100 (FIG. 2) that teaches the robot 12 the position and posture at which the hand 30 grasps the workpiece W. That is, in this embodiment, the device 100 is implemented in the teaching device 18, and the processor 50 of the teaching device 18 executes the functions of the device 100.

[0103] In this device 100, the image data acquisition unit 82 acquires image data ID of the workpiece W captured by the visual sensor 14 arranged at a known position in the control coordinate system C (robot coordinate system C1) when the robot 12 is gripping the workpiece W with the hand 30. n The work position acquisition unit 84 acquires the image data ID n Based on the image data ID n Work position data WPD indicating the position and posture of the workpiece W in the control coordinate system C (robot coordinate system C1) at the time of capturing the image n Get.

[0104] In addition, the hand position acquisition unit 86 acquires the image data ID n Hand position data HPD indicating the position and posture of the hand 30 in the control coordinate system C (robot coordinate system C1) at the time of capturing the image n The teaching position acquisition unit 88 acquires the workpiece position data WPD n and hand position data HPD n Based on the image data ID n Teaching position data TPD indicating the positional relationship between the hand 30 and the workpiece W in the control coordinate system C (tool coordinate system C2, workpiece coordinate system C4) at the time of imaging. n Get.

[0105] In this way, the image data ID captured when the workpiece W is actually gripped by the hand 30 at the gripping position that the operator wants to teach is displayed. n Based on the teaching position data TPD n By acquiring the above information, the operator can teach the robot 12 the gripping position that he or she wishes to teach with high accuracy.

[0106] In the apparatus 100, the robot control unit 90 operates the robot 12 so as to repeatedly change the posture of the hand 30 gripping the workpiece W, and the image data acquisition unit 82 acquires a plurality of image data IDs captured by the visual sensor 14 each time the robot control unit 90 changes the posture of the hand 30. n In addition, the work position acquisition unit 84 acquires each image data ID n Based on the workpiece position data WPD n The hand position acquisition unit 86 acquires each image data ID n HPD: Hand position data at the time of imaging n are obtained respectively.

[0107] Then, the teaching position acquisition unit 88 receives each workpiece position data WPD n and each hand position data HPD n Based on the image data ID n TPD: Teaching position data at the time of imaging n In this way, the image data ID of the workpiece W in various postures is obtained. n Multiple teaching position data TPD based on n By collecting these data, the gripping position of the workpiece W can be taught to the robot 12 with higher accuracy.

[0108] In the apparatus 100, the teaching position acquisition unit 88 acquires a plurality of teaching position data TPD n Based on this, new teaching position data TPD0 is calculated which is used for the operation of the robot 12 to grip the workpiece W with the hand 30. In this way, the image data ID of the workpiece W in various postures is calculated. n Multiple teaching position data TPD corresponding to n By obtaining the teaching position data TPD0 from the teaching position data TPD0, the position and posture of the hand 30 when gripping the workpiece W in various postures can be determined with higher accuracy by the teaching position data TPD0.

[0109] In addition, in the device 100, the teaching position data TPD nis the coordinate Q of the control coordinate system C (tool coordinate system C2, work coordinate system C4) TW_n , Q WT_n The teaching position acquisition unit 88 receives a plurality of teaching position data TPD n Coordinate Q of TW_n , Q WT_n Among them, coordinates outside the predetermined tolerance range are excluded, and coordinate Q TW_m , Q WT_m By calculating the average of these, new teaching position data TPD0 is calculated.

[0110] According to this configuration, the coordinate Q obtained due to erroneous detection or the like is TW_n , Q WT_n can be excluded and the coordinate Q TW_m , Q WT_m By averaging these data, more accurate teaching position data TPD0 can be obtained. This makes it possible to determine with even greater precision the position and posture of the hand 30 when gripping workpieces W in various postures. Furthermore, in the device 100, the operation program generation unit 92 generates an operation program OP in which the teaching position data TPD0 is defined. With this configuration, it is possible to automatically generate an operation program OP in which the teaching position data TPD0 obtained as described above is defined.

[0111] In the device 100, the workpiece position acquisition unit 84 acquires the workpiece model WM as an image data ID n The data indicating the position and orientation of the workpiece model WM in the control coordinate system C (robot coordinate system C2) when the workpiece model WM is matched with the workpiece W (3D point cloud image) shown in the image is called the workpiece position data WPD n According to this configuration, the image data ID captured by the visual sensor 14 is acquired as n From work position data WPD n can be detected with high accuracy.

[0112] In addition, in the device 100, the control coordinate system C has a robot coordinate system C1, a work coordinate system C4, a tool coordinate system C2 whose positional relationship with the robot coordinate system C1 is known, and a sensor coordinate system C3 whose positional relationship with the robot coordinate system C1 is known, and the visual sensor 14 is positioned at a known position in the robot coordinate system C1.

[0113] Then, the work position acquisition unit 84 acquires the image data ID n The first coordinate Q in the sensor coordinate system C3 of the workpiece coordinate system C4, which indicates the position and posture of the workpiece W captured in the image. SW_n and obtain the first coordinate Q SW_n is converted into the robot coordinate system C1, the second coordinate Q of the workpiece coordinate system C4 in the robot coordinate system C1 is RW_n Work position data WPD n Obtain as.

[0114] The hand position acquisition unit 86 also acquires the third coordinate Q in the robot coordinate system C1 of the tool coordinate system C2, which indicates the position and posture of the hand 30. RT_n Hand position data HPD n The teaching position acquisition unit 88 acquires the second coordinate Q RW_n and the third coordinate Q RT_n Based on the teaching position data TPD n coordinate Q of the workpiece coordinate system C4 in the tool coordinate system C2 TW_n , or the coordinate Q of the tool coordinate system C2 in the work coordinate system C4 WT_n According to this configuration, the teaching position data TPD is acquired based on the robot coordinate system C1, the tool coordinate system C2, the sensor coordinate system C3, and the workpiece coordinate system C4 used as the control coordinate system C. n coordinate Q in the control coordinate system C TW_n or Q WT_n can be obtained as.

[0115] In the above embodiment, the operator manually operates the teaching device 18 to change the posture of the hand 30 gripping the workpiece W. However, the present invention is not limited to this. The processor 50 may also be configured to detect changes in the posture of the hand 30, image data ID, etc. nAcquisition of workpiece position data WPD n Obtaining hand position data HPD n Acquisition of teaching position data TPD n The series of operations of obtaining the above information may be performed automatically.

[0116] Such an embodiment will be described below with reference to Fig. 8. The flow shown in Fig. 8 starts when the processor 50 of the teaching device 18 receives a teaching start command CM. For example, the operator operates the input device 58 of the teaching device 18 to cause the hand 30 to grasp the workpiece W as shown in Fig. 5, and then inputs the teaching start command CM to the teaching device 18.

[0117] In step S1, the processor 50 acquires the teaching position data TPD n The number "n" that specifies the number of image data IDs is set to "1". n , work position data WPD n , hand position data HPD n , and teaching position data TPD n corresponds to "n".

[0118] In step S2, the processor 50 operates the mechanical unit 42 of the robot 12 to move the workpiece W held by the hand 30 so that it is within the field of view of the visual sensor 14. As one example, the processor 50 may move the hand 30 along a predetermined movement path. This movement path may be determined by an operator teaching the robot 12 in advance. As another example, the processor 50 may acquire the coordinates of the tool coordinate system C2 and the sensor coordinate system C3 in the robot coordinate system C1, and move the hand 30 based on the coordinates.

[0119] In step S3, the processor 50 functions as the image data acquisition unit 82 and acquires the image data ID n Specifically, the processor 50 sends an image capturing command to the visual sensor 14, operates the visual sensor 14, and acquires image data ID of the workpiece W held by the hand 30. nThe processor 50 then functions as an image data acquisition unit 82 to acquire the image data ID from the visual sensor 14. n Get.

[0120] In step S4, the processor 50 functions as the work position acquisition unit 84 and acquires the image data ID acquired in the most recent step S3 by the method described above. n Based on the workpiece position data WPD n (For example, coordinate Q RW_n In step S5, the processor 50 functions as the hand position acquisition unit 86 and acquires the image data ID acquired in the most recent step S3 by the method described above. n HPD: Hand position data at the time of imaging n (For example, coordinate Q RT_n ) to get the

[0121] In step S6, the processor 50 functions as the teaching position acquisition unit 88, and acquires the workpiece position data WPD acquired in the most recent step S4 by the above-described method. n and the hand position data HPD obtained in the most recent step S5 n Based on the teaching position data TPD n (For example, coordinate Q TW_n or Q WT_n ) to get the

[0122] In step S7, the processor 50 determines whether the number "n" is equal to a predetermined number n MAX reached (n=n MAX ) and determine whether this number n MAX can be predetermined by an operator. The processor 50 determines whether n=n MAX If so, the answer is YES and the process proceeds to step S10. <n MAX If so, the result is determined to be NO, and the process proceeds to step S8. In step S8, the processor 50 increments the number "n" by "1" (n=n+1).

[0123] In step S9, the processor 50 functions as the robot control unit 90 and operates the mechanism unit 42 of the robot 12 to change the posture of the hand 30 gripping the workpiece W. Specifically, the processor 50 rotates the hand 30 by a predetermined angle θ around the x-axis, y-axis, or z-axis of the tool coordinate system C2, which is set in the robot coordinate system C1 at this point in time.

[0124] Alternatively, the processor 50 may rotate the hand 30 by an angle θ around the x-axis, y-axis (i.e., axis B of the workpiece W), or z-axis of the workpiece coordinate system C4 set in the robot coordinate system C1 at this time. The angle θ and direction by which the hand 30 is rotated when performing step S9 may be predetermined by an operator, or may be automatically (e.g., randomly) determined each time step S9 is performed by the processor 50, taking into consideration the positional relationship between the tool coordinate system C2 (or workpiece coordinate system C4) and the sensor coordinate system C3 set at this time, so that the entire workpiece W is oriented within the field of view of the visual sensor 14.

[0125] After step S9, the processor 50 returns to step S3 and repeatedly executes the loop of steps S3 to S9 until it determines YES in step S7. In this way, the processor 50 generates a plurality of taught position data TPD n (n=1,2,3,4···) can be obtained automatically.

[0126] If the determination in step S7 is YES, in step S10, the processor 50 functions as the teaching position acquisition unit 88 and acquires the plurality of teaching position data TPD acquired by the above-described method. n Based on this, new teaching position data TPD0 (for example, coordinate Q TW_0 or Q WT_0 ) is found.

[0127] In step S11, the processor 50 functions as the operation program generating unit 92, and generates the teaching position data TPD0 (i.e., the coordinate Q TW_0 or Q WT_0) is defined. As described above, according to this embodiment, the processor 50 can automatically generate the operation program OP by automatically executing the series of operations in steps S1 to S11. This configuration can speed up and simplify the process of teaching the robot 12 the position at which to grip the workpiece W.

[0128] In step S9 described above, the processor 50 may function as the robot control unit 90 to change the position of the hand 30 instead of changing the attitude of the hand 30. Specifically, in step S9, the processor 50 may operate the mechanism unit 42 of the robot 12 to change the position of the hand 30 by translating the hand 30 by a predetermined distance δ in the direction of the x-axis, y-axis, or z-axis of the tool coordinate system C2 (or the workpiece coordinate system C4) set in the robot coordinate system C1 at this point in time. Here, "translation" may be defined as an operation of moving the hand 30 without changing the attitude of the hand 30 (i.e., the direction of each axis of the tool coordinate system C2).

[0129] Alternatively, the processor 50 may alternate between changing the posture of the hand 30 and changing the position of the hand 30 each time step S9 is executed, or may change the posture of the hand 30 and change the position of the hand 30 in step S9.

[0130] In the above embodiment, the case has been described where the functions of the device 100 are implemented in the teaching device 18. However, this is not limiting, and the functions of the device 100 can also be implemented in the control device 16 (or the visual sensor 14). In this case, the processor 70 of the control device 16 (or the processor of the visual sensor 14) functions as the device 100 (i.e., the image data acquisition unit 82, the workpiece position acquisition unit 84, the hand position acquisition unit 86, the teaching position acquisition unit 88, the robot control unit 90, and the operation program generation unit 92).

[0131] In addition, some of the image data acquisition unit 82, work position acquisition unit 84, hand position acquisition unit 86, teaching position acquisition unit 88, robot control unit 90, and operation program generation unit 92 may be implemented in one of the control device 16, teaching device 18, and visual sensor 14, while other parts may be implemented in the other of the control device 16, teaching device 18, and visual sensor 14.

[0132] For example, the image data acquisition unit 82 may be implemented in the visual sensor 14, the workpiece position acquisition unit 84, the hand position acquisition unit 86, and the robot control unit 90 may be implemented in the control device 16, and the teaching position acquisition unit 88 and the operation program generation unit 92 may be implemented in the teaching device 18. In this case, the processor of the visual sensor 14, the processor 70 of the control device 16, and the processor 50 of the teaching device 18 constitute the device 100.

[0133] In the above embodiment, the processor 50 functions as the hand position acquisition unit 86 and acquires the image data ID n Based on the sensor coordinate system C 3 and obtains data indicating the position and posture of the hand 30 at the time of the hand position data HPD n Specifically, as shown in FIG. 6, the visual sensor 14 may acquire image data ID while keeping the workpiece W and the hand 30 holding the workpiece W within its field of view. n The image is captured.

[0134] The processor 50 functions as an image data acquisition unit 82 to acquire the image data ID n is acquired from the visual sensor 14, the hand model 30M, which is a model of the hand 30, is acquired together with the workpiece model WM. This hand model 30M is, for example, a three-dimensional CAD model, and is stored in advance in the memory 52.

[0135] The processor 50 then uses predetermined pattern matching parameters to match the image data ID n The three-dimensional point cloud image of the hand 30 shown in the image is analyzed, and the hand model 30M is generated based on the image data ID.n The image data ID is then stored in the memory. n 3 is set for the hand model 30M placed at the tool coordinate system C2 in the positional relationship shown in FIG. 3. Then, the processor 50 calculates the coordinate Q of the set tool coordinate system C2 in the sensor coordinate system C3. ST_n and obtain the coordinate Q ST_n is converted to the robot coordinate system C1, the hand position data HPD n (coordinate Q RT_n ) to get the

[0136] Such a function can be performed by the visual sensor 14. For example, the visual sensor 14 (specifically, the processor) may receive captured image data ID n By applying the workpiece model WM and hand model 30M to the sensor coordinate system C3, the coordinate Q of the tool coordinate system C2 is ST_n and the coordinate Q of the workpiece coordinate system C4 in the sensor coordinate system C3. SW_n and may be provided to the teaching device 18.

[0137] Alternatively, the visual sensor 14 may function as a workpiece position acquisition unit 84 and acquire the coordinate Q of the workpiece coordinate system C4 in the sensor coordinate system C3. SW_n , work position data WPD n and also functions as a hand position acquisition unit 86 to acquire the coordinate Q of the tool coordinate system C2 in the sensor coordinate system C3. ST_n , hand position data HPD n It may be obtained as.

[0138] The visual sensor 14 functions as a teaching position acquisition unit 88 and acquires workpiece position data WPD n (coordinate Q SW_n ) and hand position data HPD n (coordinate Q ST_n ) based on the teaching position data TPD n (For example, coordinate Q TW_n or Q WT_n) may be acquired. In other words, in this case, the processor of the visual sensor 14 functions as the device 100.

[0139] In the above embodiment, the processor 50 calculates a plurality of coordinates Q TW_n or Q WT_n After performing the process PR1 to exclude each of the coordinates, the coordinates Q registered in the valid coordinate group GRP are TW_m or Q WT_m However, the present invention is not limited to this. The processor 50 may perform the process PR2 of averaging a plurality of coordinates Q TW_n or Q WT_n Alternatively, only one of the exclusion process PR1 and the averaging process PR2 may be performed.

[0140] For example, the processor 50 may select the plurality of acquired coordinates Q without performing the exclusion process PR1. TW_n or Q WT_n Alternatively, the processor 50 may perform a process PR2 to average the coordinates Q TW_n or Q WT_n Only the process PR1 for excluding coordinates Q is performed, and as a result of the process PR1, the coordinates Q registered in the valid coordinate group GRP are TW_m or Q WT_m One teaching position data TPD0 may be automatically selected from the above in accordance with a predetermined condition.

[0141] In the above embodiment, the processor 50 acquires the plurality of teaching position data TPD n However, the present invention is not limited to this. For example, the processor 50 may obtain new teaching position data TPD based on the plurality of teaching position data TPD n Alternatively, image data showing the above in list form may be generated and displayed on the display device 56.

[0142] Then, the operator operates the input device 58 to input the plurality of teaching position data TPD displayed on the display device 56.n Input data for selecting the desired teaching position data TPD0 from among these is provided to the processor 50. In response to the input data, the processor 50 creates an operation program OP in which the teaching position data TPD0 selected by the operator is specified.

[0143] In the above embodiment, the processor 50 functions as the robot control unit 90 to repeatedly change the posture of the hand 30, thereby generating a plurality of teaching position data TPD n In the above description, the case where the teaching position data TPD1 is acquired has been described. However, this is not limiting, and the processor 50 may acquire only the teaching position data TPD1 described above without changing the posture of the hand 30. In this case, the processor 50 may create an operation program OP in which the teaching position data TPD1 is defined. In other words, in this case, the robot control unit 90 can be omitted from the device 100.

[0144] In the above embodiment, the processor 50 functions as the operation program generator 92 to generate the operation program OP. However, the present invention is not limited to this. The operator may generate the operation program OP by using the teaching position data TPD acquired by the processor 50. n In this case, the operation program generating unit 92 can be omitted from the device 100.

[0145] In the above embodiment, the processor 50 also converts the workpiece model WM into image data ID n By applying this, the workpiece position data WPD n However, the present invention is not limited to this. The processor 50 may acquire image data ID without using the work model WM. n By analyzing the image of the workpiece W captured in the n You can also obtain the following.

[0146] In the above embodiment, the teaching position data TPD is calculated based on the robot coordinate system C1, the tool coordinate system C2, the sensor coordinate system C3, and the workpiece coordinate system C4 as the control coordinate system C. n However, the present invention is not limited to this. For example, the teaching position data TPD may be acquired based on the world coordinate system C5. n The world coordinate system C5 is a control coordinate system C that defines the three-dimensional space of the work cell, and is set fixed to the work cell.

[0147] The visual sensor 14 may be fixed to the mechanical part 42 (for example, the upper arm 26 or the lower arm 24) of the robot 12, instead of to the holding frame 44. In this case, the visual sensor 14 is attached to the mechanical part 42 so as to be able to capture an image of the workpiece W held by the hand 30. In the above-described embodiment, the workpiece position data WPD n , hand position data HPD n , and teaching position data TPD n is the coordinate Q of the control coordinate system C, but the present invention is not limited to this and may be expressed as any other data.

[0148] Also, instead of gripping the large ring portion W2, the hand 30 may grip the small ring portion W3 by pressing the claws 34 and 36 against the inner wall surface of the through hole H2. 18 In this case, the position and posture for gripping the small ring part W3 with the hand 30 are taught in the same manner as in the above-described embodiment.

[0149] 4 (connecting rod), the workpiece W may have any shape, and the hand 30 may be of any type. For example, the hand 30 may have an adsorption part (a vacuum device, a suction cup, a magnet, etc.) instead of the openable and closable claws 34 and 36, and may adsorb and grip the workpiece W with the adsorption part.

[0150] Furthermore, the visual sensor 14 may be a two-dimensional camera. In this case, the robot system 10 may further include a distance sensor fixed to the visual sensor 14 and capable of measuring the distance d between the visual sensor 14 and a subject (workpiece W). Furthermore, the teaching device 18 may be directly connected to the robot 12 (servo motor 40) or the visual sensor 14. While the present disclosure has been described above through the embodiments, the above-described embodiments do not limit the invention according to the claims. [Explanation of symbols]

[0151] 10 Robot Systems 12. Robot 14 Visual Sensor 16 Control device 18 Teaching device 30 hands 70,100 processors 82 Image data acquisition unit 84 Work position acquisition unit 86 Hand position acquisition unit 88 Teaching position acquisition unit 90 Robot control unit 92 Operation program generation unit

Claims

1. A device for teaching a position and posture in a control coordinate system for controlling a robot at which the robot grasps a workpiece with its hand, comprising: an image data acquisition unit that acquires image data of the workpiece captured by a visual sensor arranged at a known position in the control coordinate system when the robot is gripping the workpiece with the hand; a work position acquisition unit that acquires, based on the image data, work position data that indicates the position and orientation of the work in the control coordinate system when the visual sensor captures the image data; a hand position acquisition unit that acquires hand position data indicating the position and orientation of the hand in the control coordinate system when the visual sensor captures the image data; and a teaching position acquisition unit that acquires teaching position data indicating the positional relationship between the hand and the workpiece in the control coordinate system when the visual sensor captured the image data, based on the workpiece position data and the hand position data.

2. a robot control unit that operates the robot so as to repeatedly change the posture of the hand that is gripping the workpiece, the image data acquisition unit acquires the plurality of image data captured by the visual sensor each time the robot control unit changes the posture of the hand, the workpiece position acquisition unit acquires, based on each of the image data acquired by the image data acquisition unit, the workpiece position data at the time when each of the image data was captured; the hand position acquisition unit acquires the hand position data when each of the image data is captured, The device described in claim 1, wherein the teaching position acquisition unit acquires the teaching position data at the time each of the image data was captured based on each of the work position data acquired by the work position acquisition unit and each of the hand position data acquired by the hand position acquisition unit.

3. The apparatus according to claim 2 , wherein the teaching position acquisition unit determines new teaching position data to be used for an operation of causing the robot to grasp the workpiece with the hand, based on the plurality of pieces of teaching position data acquired.

4. the teaching position data is expressed as coordinates in the control coordinate system, The device according to claim 3, wherein the teaching position acquisition unit obtains the new teaching position data by excluding coordinates of the plurality of teaching position data that are outside a predetermined tolerance range, or by calculating an average of the coordinates of the plurality of teaching position data.

5. 5. The device according to claim 1, further comprising an operation program generating unit that generates an operation program in which the teaching position data is defined.

6. The device according to any one of claims 1 to 5, wherein the work position acquisition unit acquires, as the work position data, data indicating the position and orientation of the work model in the control coordinate system when a work model that models the work is matched with the work depicted in the image data.

7. The control coordinate system is a robot coordinate system set in the robot; a workpiece coordinate system set on the workpiece; a tool coordinate system that is set in the hand and whose positional relationship with the robot coordinate system is known; a sensor coordinate system that is set in the visual sensor and has a known positional relationship with the robot coordinate system; the visual sensor is positioned at the known position in the robot coordinate system; The workpiece position acquisition unit Acquire first coordinates in the sensor coordinate system of the workpiece coordinate system that indicate the position and orientation of the workpiece captured in the image data; By converting the first coordinates into the robot coordinate system, second coordinates of the workpiece coordinate system in the robot coordinate system are acquired as the workpiece position data; the hand position acquisition unit acquires, as the hand position data, third coordinates in the robot coordinate system of the tool coordinate system that indicate the position and orientation of the hand; The device described in any one of claims 1 to 6, wherein the teaching position acquisition unit acquires the teaching position data as coordinates of the workpiece coordinate system in the tool coordinate system, or coordinates of the tool coordinate system in the workpiece coordinate system, based on the second coordinates and the third coordinates.

8. a robot having a hand capable of grasping a workpiece; a visual sensor that captures an image of the workpiece; A robot system comprising: the device according to any one of claims 1 to 7.

9. a control device that controls the robot so that the hand grips the workpiece based on second image data of the workpiece captured by the visual sensor; The control device Data indicating the position and orientation of the workpiece in the control coordinate system shown in the second image data is acquired as second workpiece position data; The robot system according to claim 8 , wherein the position and orientation of the hand in the control coordinate system when gripping the workpiece imaged by the visual sensor are determined based on the second workpiece position data and the taught position data.

10. A method for teaching a position and posture in a control coordinate system for controlling a robot at which the robot grasps a workpiece with its hand, comprising: The processor: When the robot is gripping the workpiece with the hand, a visual sensor arranged at a known position in the control coordinate system acquires image data of the workpiece; based on the image data, acquiring workpiece position data indicating the position and orientation of the workpiece in the control coordinate system when the visual sensor captured the image data; acquiring hand position data indicating the position and orientation of the hand in the control coordinate system when the visual sensor captured the image data; A method for acquiring, based on the workpiece position data and the hand position data, teaching position data indicating the positional relationship between the hand and the workpiece in the control coordinate system when the visual sensor captured the image data.

Images