DEVICE AND METHOD FOR DETERMINING A TRANSPORT DEVICE COORDINATE SYSTEM IN A ROBOT COORDINATE SYSTEM

Abstract

The present invention provides a technology that facilitates the easier establishment of a transport device coordinate system within a robot coordinate system (i.e., calibration between a robot coordinate system and a transport device coordinate system). A device 60 comprises a first index 18; a camera 14, which acquires first image data in which the first index 18 is captured, and second image data in which the first index 18, transported by a transport device 16 after capture, is captured;a position data acquisition unit 52, which acquires first position data, which specifies a three-dimensional position in a first-index coordinate system, expressed by the first index 18 acquired in the first image data, with respect to the camera, and second position data, which specifies a three-dimensional position in the first-index coordinate system, expressed by the first index acquired in the second image data, with respect to the camera; a transport direction acquisition unit 54, which derives the transport direction; and a coordinate system setting unit 56, which, based on the transport direction, sets the transport device coordinate system in the robot coordinate system.

Description

Apparatus and method for setting a transport device coordinate system to a robot coordinate system

[0001] The present disclosure relates to an apparatus and method for setting a transport device coordinate system to a robot coordinate system.

[0002] There is known a device that sets a transport device coordinate system in a robot coordinate system by touching up a robot's hand to the transport device (see, for example, Patent Document 1). There is also known a device that performs visual touch-up by capturing images of multiple marks provided on a peripheral device of the robot with a camera, thereby calibrating the robot coordinate system with respect to the peripheral device (see, for example, Patent Document 2).

[0003] JP 2015-174171 A JP 2005-149299 A

[0004] There is a demand for a technique that allows the transfer device coordinate system to be set in the robot coordinate system more easily (that is, allows the robot coordinate system and the transfer device coordinate system to be calibrated).

[0005] In one aspect of the present disclosure, an apparatus for setting a transport device coordinate system that specifies the transport direction of a transport device that transports a workpiece in a robot coordinate system that is set on a robot that performs work on a workpiece includes a first index that represents a first index coordinate system, the first index being installed on the transport device so as to be transported by the transport device, a camera that acquires first image data that captures the first index, and second image data that captures the first index that has been transported by the transport device after the first image data has been captured.

[0006] The device also includes a position data acquisition unit that acquires first position data indicating the three-dimensional position of the first index coordinate system represented by the first index in the first image data relative to the camera, and second position data indicating the three-dimensional position of the first index coordinate system represented by the first index in the second image data relative to the camera, a transport direction acquisition unit that determines the transport direction based on the first position data and the second position data, and a coordinate system setting unit that sets the transport device coordinate system in the robot coordinate system based on the transport direction determined by the transport direction acquisition unit.

[0007] In another aspect of the present disclosure, a method for setting a transport device coordinate system that specifies the transport direction of a transport device that transports a workpiece in a robot coordinate system that is set on a robot that performs work on a workpiece includes installing a first index that represents a first index coordinate system on the transport device so that the first index is transported by the transport device, acquiring first image data by capturing an image of the first index with a camera, and acquiring second image data by capturing an image of the first index that has been transported by the transport device after capturing the first image data.

[0008] This method also acquires first position data indicating the three-dimensional position of a first index coordinate system represented by a first index in the first image data relative to the camera, and second position data indicating the three-dimensional position of the first index coordinate system represented by a first index in the second image data relative to the camera, determines a conveying direction based on the first position data and the second position data, and sets a conveying device coordinate system in the robot coordinate system based on the conveying direction determined by the conveying direction acquisition unit.

[0009] 11 is a schematic perspective view of a robot system according to one embodiment. It is a block diagram of the robot system shown in FIG. 1. It is a flowchart showing an example of a method for setting a transport device coordinate system in a robot coordinate system in the robot system shown in FIG. 1. It shows an example of image data captured in step S1 in FIG. 3. It shows a state in which a first index has been transported in step S3 in FIG. 3. It shows an example of image data captured in step S4 in FIG. 3. It shows a transport device coordinate system set in the robot coordinate system in the robot system shown in FIG. 1. It is a schematic perspective view of a robot system according to another embodiment. It is a block diagram of the robot system shown in FIG. 8. It shows the rear side of the terminal device shown in FIG. 8. It is a flowchart showing an example of a method for setting a transport device coordinate system in a robot coordinate system in the robot coordinate system in the robot system shown in FIG. 8. It shows an example of image data captured in step S11 in FIG. 11. It shows a state in which a first index has been transported in step S13 in FIG. 11. It shows an example of image data captured in step S14 in FIG. It shows a transport device coordinate system set in the robot coordinate system in the robot system shown in FIG. 8. It is a schematic perspective view of a robot system according to yet another embodiment. It is a block diagram of the robot system shown in FIG. 16. It is a flowchart showing an example of a method for setting a transport device coordinate system in a robot coordinate system in the robot coordinate system in the robot system shown in FIG. 18. An example of image data captured in step S22 in Fig. 18 is shown. FIG. 18 shows a state in which a first index has been transported in step S24 in Fig. 18. FIG. 18 shows an example of image data captured in step S25 in Fig. 18. FIG. 18 shows an example of image data captured in step S27 in Fig. 18. FIG. 16 shows a transport device coordinate system set in the robot coordinate system in the robot system shown in Fig. 16. FIG. 17 is a flowchart showing another example of a method for setting the transport device coordinate system in the robot coordinate system in the robot system shown in Fig. 1.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In various embodiments described below, like elements will be designated by like reference numerals, and duplicated descriptions will be omitted. First, a robot system 10 according to one embodiment will be described with reference to FIGS. 1 and 2. The robot system 10 includes a robot 12, a camera 14, a transport device 16, a first index 18, and a control device 20.

[0011] In this embodiment, the robot 12 is a vertical articulated robot that performs a predetermined task (e.g., workpiece handling, welding, cutting, laser processing, etc.) on a workpiece (not shown). Specifically, the robot 12 has a robot base 22, a rotating body 24, a lower arm 26, an upper arm 28, and a wrist 30.

[0012] The robot base 22 is fixed to the floor of a work cell or on an automated guided vehicle (AGV). The rotating body 24 is attached to the robot base 22 so as to be rotatable about a vertical axis. The lower arm 26 has a base end attached to the rotating body 24 so as to be rotatable about a horizontal axis, and the upper arm 28 has a base end attached to the tip of the lower arm 26 so as to be rotatable.

[0013] The wrist 30 has a wrist base 30a provided at the tip of the upper arm 28 so as to be rotatable about two axes that are perpendicular to each other, and a wrist flange 30b provided on the wrist base 30a so as to be rotatable about the wrist axis A1. An end effector (robot hand, welding torch, cutting tool, laser processing head, etc.) that performs an operation on a workpiece is detachably attached to the wrist flange 30b.

[0014] A plurality of servo motors 32 (FIG. 2) are provided on each of the robot base 22, the rotating body 24, the lower arm 26, the upper arm 28, and the wrist 30. These servo motors 32 rotate each movable component of the robot 12 (i.e., the rotating body 24, the lower arm 26, the upper arm 28, the wrist 30, and the wrist flange 30b) in response to commands from the control device 20, thereby moving an end effector (not shown) attached to the wrist flange 30b to a desired position.

[0015] In this embodiment, the camera 14 is detachably attached to the wrist flange 30b instead of the end effector. The camera 14 is a two-dimensional camera having an image sensor (CMOS, CCD, etc.) and an optical lens (collimator lens, focus lens, etc.) that guides the subject image to the image sensor, and captures an image of the subject in response to commands from the control device 20. The camera 14 captures the subject image along the optical axis A2 and forms the subject image on the image sensor via the optical lens, thereby obtaining image data ID of the subject image. In this embodiment, the camera 14 is moved to any position by the robot 12.

[0016] The transport device 16 is disposed around the robot 12 and transports the workpiece in the transport direction D1. For example, the transport device 16 is a belt conveyor or the like, and has a transport surface 34 on which the workpiece is placed and a drive mechanism 36 that moves the transport surface 34 in the transport direction D1. The transport surface 34 may be defined by the outer surface of a timing belt.

[0017] The drive mechanism 36 may also include a servo motor (not shown) that drives the timing belt (i.e., the conveying surface 34) in response to commands from the control device 20. The conveying device 16 is positioned so that at least a portion of the conveying surface 34 is within the operating range of the robot 12 (i.e., the range within which the end effector can be positioned).

[0018] The control device 20 controls the robot 12, the camera 14, and the transport device 16. Specifically, as shown in Fig. 2, the control device 20 is a computer having a processor 40, a memory 42, and an I / O interface 44. The processor 40 has a CPU or a GPU, etc., and is communicatively connected to the memory 42 and the I / O interface 44 via a bus 46, and performs calculations to realize a function of setting a transport device coordinate system, which will be described later, while communicating with these components.

[0019] The memory 42 includes RAM, ROM, or the like, and temporarily or permanently stores various data. The memory 42 may be configured from a computer-readable storage medium such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium. The I / O interface 44 includes, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices via wired or wireless communication under instructions from the processor 40. In this embodiment, the servo motors 32, the camera 14, and the transport device 16 (specifically, the drive mechanism 36) of the robot 12 are communicatively connected to the I / O interface 44.

[0020] The control device 20 is also provided with a display device 48 and an input device 50. The display device 48 and the input device 50 are communicatively connected to the I / O interface 44. The display device 48 has a liquid crystal display, an organic EL display, or the like, and visibly displays various data under instructions from the processor 40.

[0021] The input device 50 has push buttons, switches, a keyboard, a mouse, a touch panel, or the like, and receives data input from an operator. The display device 48 and the input device 50 may be integrated into the housing of the control device 20, or may be attached externally to the housing of the control device 20 as a separate computer (such as a PC).

[0022] 1, a robot coordinate system C1 is set for the robot 12. The robot coordinate system C1 is a coordinate system for automatically controlling the operation of each movable element of the robot 12. In this embodiment, the robot coordinate system C1 is fixed to the robot base 22 so that its origin is located at the center of the robot base 22 and its z axis is parallel to (specifically, coincides with) the rotation axis of the rotating body 24.

[0023] On the other hand, a camera coordinate system C2 is set for the camera 14. In this embodiment, the camera coordinate system C2 is set for the camera 14 so that its origin is located at the center of the image sensor of the camera 14 and its z axis is parallel to (specifically, coincides with) the optical axis A2. The camera 14 may be attached to the wrist flange 30b so that its optical axis A2 is parallel to (or coincides with) the wrist axis A1.

[0024] The camera coordinate system C2 defines the coordinates of each pixel of the image data ID (or the image sensor) captured by the camera 14. In this embodiment, the position of the camera 14 in the robot coordinate system C1 (in other words, the position of the wrist flange 30b) is known. Therefore, the positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 of the camera 14 placed at an arbitrary position PSn in the robot coordinate system C1 is １_２_ｎ is known.

[0025] Therefore, the coordinates Q (X, Y, Z, W, P, R) of the camera coordinate system C2 in the robot coordinate system C1 are known, and the coordinates of the robot coordinate system C1 and the coordinates of the camera coordinate system C2 are transformed by a known transformation matrix M １_２_ｎ The control device 20 can convert the positional relationship R １_２_ｎ As data indicating １_２_ｎ The parameters of the above, the coordinates of the camera coordinate system C2 in the robot coordinate system C1, etc. are stored in the memory 42.

[0026] When the robot 12 and the transport device 16 cooperate to perform work on a workpiece, it is necessary to set a transport device coordinate system in the robot coordinate system C1. The transport device coordinate system is set relative to the transport device 16 and defines the position of the transport surface 34 and the transport direction D1. The first index 18 is used to set the transport device coordinate system in the robot coordinate system C1 (in other words, to make the positional relationship between the robot coordinate system C1 and the transport device coordinate system known).

[0027] Specifically, the first indicator 18 is made of a flat material such as paper or board, and has a pattern 18a visibly displayed on its surface. The pattern 18a includes, for example, a figure consisting of a plurality of lines or curves, or a dot pattern. The first indicator 18 is configured to represent the first indicator coordinate system C3 using the pattern 18a, and is placed on the conveying surface 34 of the conveying device 16 so as to be conveyed by the conveying device 16. Note that the first indicator 18 may be fixed to the conveying surface 34 using a fastener (such as a bolt) or a jig so as to be immovable relative to the conveying surface 34. Alternatively, the pattern 18a of the first indicator 18 may be directly formed on the conveying surface 34 by printing, engraving, or the like.

[0028] Next, a method for setting the transport device coordinate system in the robot coordinate system C1 in the robot system 10 will be described with reference to Figure 3. Before starting the flow of Figure 3, the operator places the first index 18 on the transport surface 34 of the transport device 16 as shown in Figure 1. At this time, the operator may place the first index 18 within the operating range of the robot 12. After placing the first index 18, the operator operates the input device 50 to issue a coordinate system setting command to the processor 40. When the processor 40 receives the coordinate system setting command, it starts the flow of Figure 3.

[0029] In step S1, the processor 40 causes the camera 14 to capture an image of the first index 18. Specifically, the processor 40 operates the robot 12 to move the camera 14 to an imaging position PS1 where the first index 18 can be placed within the field of view of the camera 14. Then, the processor 40 operates the camera 14 to capture an image of the first index 18.

[0030] An example of image data ID1 captured by the camera 14 in step S1 is shown in Figure 4. Each pixel of the image data ID1 is expressed as a coordinate in the camera coordinate system C2, and the image data ID1 includes the first index 18. The camera 14 acquires the image data ID1 (first image data) capturing the image of the first index 18 in this manner, and supplies the image data ID1 to the processor 40.

[0031] In step S2, the processor 40 acquires position data PD1 (first position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 captured in the image data ID1 relative to the camera 14. Here, the pattern 18a of the first index 18 captured in the image data ID1 is configured to represent the three-dimensional position of the first index coordinate system C3 in the camera coordinate system C2 set in the camera 14 at the time the image data ID1 was captured.

[0032] The processor 40 executes an image analysis program PG1 (or an image analysis application) for reading the first index coordinate system C3 from the pattern 18a, analyzes the pattern 18a of the first index 18 shown in the image data ID1, and identifies the first index coordinate system C3 represented by the first index 18 within the image data ID1. The processor 40 then acquires coordinates Q1 (X1, Y1, Z1, W1, P1, R1) in the camera coordinate system C2 of the identified first index coordinate system C3.

[0033] Of this coordinate Q1, the coordinate (X1, Y1, Z1) indicates the origin position of the first index coordinate system C3 in the camera coordinate system C2 of Figure 4, and the coordinate (W1, P1, R1) indicates the attitude (i.e., the direction of each axis) of the first index coordinate system C3 relative to the camera coordinate system C2 of Figure 4 (so-called yaw, pitch, and roll).

[0034] In this way, the coordinate Q1 indicates the three-dimensional position of the first index coordinate system C3 in the camera coordinate system C2 in FIG. 4, in other words, the three-dimensional position of the first index coordinate system C3 relative to the camera 14 when capturing the image data ID1. In other words, in this document, "position" may refer to position and orientation. In step S2, the processor 40 acquires the coordinate Q1 as the position data PD1. Therefore, the processor 40 functions as a position data acquisition unit 52 (FIG. 2) that acquires the position data PD1 (specifically, the coordinate Q1).

[0035] In step S3, the processor 40 transports the first index 18 by the transport device 16. Specifically, the processor 40 operates the drive mechanism 36 of the transport device 16 to transport the transport surface 34 a predetermined distance δ in the transport direction D1. As a result, as shown in FIG. 5, the first index 18 is transported the distance δ in the transport direction D1 from its position when the image data ID1 was captured. Note that, for ease of understanding, FIG. 5 also shows the position of the first index 18 when the image data ID1 was captured by a dotted line B, and also shows a first index coordinate system C3 represented by the first index 18.

[0036] In step S4, the processor 40 causes the camera 14 to capture an image of the first index 18. In this embodiment, in step S4, the processor 40 causes the camera 14 to capture an image of the first index 18 with the camera 14 positioned at the same imaging position PS1 as in step S1 described above. That is, the imaging position PS1 is defined in the robot coordinate system C1 so that the first index 18 before and after the transfer in step S3 can be within the field of view of the camera 14.

[0037] An example of the image data ID2 captured by the camera 14 in step S4 is shown in Figure 6. This image data ID2 captures the first index 18 after transport in step S3. The camera 14 acquires image data ID2 (second image data) capturing the image of the first index 18 after transport, and supplies the image data ID2 to the processor 40.

[0038] In step S5, the processor 40 functions as the position data acquisition unit 52 and acquires position data PD2 (second position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 in the image data ID2 relative to the camera 14. Specifically, similar to step S2 described above, the processor 40 executes the image analysis program PG1 to acquire coordinates Q2 (X2, Y2, Z2, W2, P2, R2) in the camera coordinate system C2 of the first index coordinate system C3 represented by the first index 18 in the image data ID2. This coordinate Q2 indicates the three-dimensional position of the first index coordinate system C3 relative to the camera 14 at the time the image data ID2 was captured. Thus, in step S5, the processor 40 acquires the coordinate Q2 as the position data PD2.

[0039] In step S6, the processor 40 calculates the conveying direction D1 based on the position data PD1 and PD2. Specifically, the processor 40 calculates the conveying direction D1 based on the known positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 arranged at the image capturing position PS1. １_２_１ (Specifically, the transformation matrix M １_２_１ ), coordinates Q1 (X1, Y1, Z1, W1, P1, R1) in the camera coordinate system C2 acquired as position data PD1 in step S2 are converted into coordinates Q3 (X3, Y3, Z3, W3, P3, R3) in the robot coordinate system C1. This coordinate Q3 indicates the three-dimensional position (specifically, the three-dimensional position and orientation) in the robot coordinate system C1 of the first index coordinate system C3 represented by the first index C1 when the image data ID1 was captured (i.e., when step S1 was executed).

[0040] Similarly, the processor 40 calculates the positional relationship R １_２_１ Using this, the coordinate Q2 (X2, Y2, Z2, W2, P2, R2) in the camera coordinate system C2 acquired as the position data PD2 in step S5 above is converted into a coordinate Q4 (X4, Y4, Z4, W4, P4, R4) in the robot coordinate system C1. This coordinate Q4 indicates the three-dimensional position in the robot coordinate system C1 of the first index coordinate system C3 represented by the first index C1 when the image data ID2 was captured (i.e., when step S4 was executed).

[0041] Then, the processor 40 calculates a vector VT from the coordinates (X3, Y3, Z3) of the coordinates Q3 to the coordinates (X4, Y4, Z4) of the coordinates Q4. ３_４ is calculated. This vector VT ３_４ coincides with the axis A3 connecting the coordinates (X3, Y3, Z3) and the coordinates (X4, Y4, Z4), and indicates the position of the vector in the conveying direction D1 in the robot coordinate system C1.

[0042] This vector VT ３_４ (or axis A3) indicates the trajectory in the robot coordinate system C1 of the origin of the first index coordinate system C3 that has been moved in the conveying direction D1 in step S3, and can be considered to be located on the conveying surface 34 (or at a position a known distance above the conveying surface 34). ３_４ (or axis A3) indicates the conveying direction D1 in the robot coordinate system C1, and also serves as data indicating the position of the conveying surface 34 in the robot coordinate system C1.

[0043] In this way, the processor 40 determines the vector VT as the conveying direction D1 in the robot coordinate system C1 based on the position data PD1 and PD2. ３_４ (or axis A3). Therefore, in this embodiment, the processor 40 functions as a conveying direction acquisition unit 54 (FIG. 2) that determines the conveying direction D1 based on the position data PD1 and PD2.

[0044] In step S7, the processor 40 sets the transport device coordinate system C4 (FIG. 7) in the robot coordinate system C1 based on the transport direction D1 obtained in step S6. Specifically, the processor 40 sets the transport device coordinate system C4 (FIG. 7) in the robot coordinate system C1 based on the transport direction D1 obtained in step S6 (specifically, the vector VT ３_４ ) is defined as the positive y-axis direction of the transport device coordinate system C4.

[0045] The processor 40 also defines the z-axis direction of the transport device coordinate system C4 as a direction that is parallel to the y-z plane of the robot coordinate system C1 and perpendicular to the y-axis of the transport device coordinate system C4 defined as above, and defines the x-axis direction of the transport device coordinate system C4 as a direction that is perpendicular to the y-axis and z-axis of the transport device coordinate system C4.

[0046] At this time, the processor 40 may randomly determine the positive z-axis direction and the positive x-axis direction of the transport device coordinate system C4. Alternatively, the operator may operate the input device 50 to input setting information for the positive z-axis direction or the positive x-axis direction in advance, and the processor 40 may determine the positive z-axis direction or the positive x-axis direction of the transport device coordinate system C4 in accordance with the setting information.

[0047] This setting information may include information that defines the positive z-axis direction of the transport device coordinate system C4 as a direction closer to the positive z-axis direction of the robot coordinate system C1 (i.e., vertically upward) (i.e., a direction in which the inner product of the two becomes larger). Alternatively, the setting information may include information that defines the positive x-axis direction of the transport device coordinate system C4 as a direction opposite to that of the robot coordinate system C1.

[0048] The processor 40 also determines the origin of the transport device coordinate system C4 on the robot coordinate system C1. As one example, the processor 40 determines the above-mentioned coordinate Q3 (X1, Y1, Z1) or coordinate Q4 (X4, Y4, Z4) as the origin of the transport device coordinate system C4. As another example, when the processor 40 acquires the axis A3 in the above-mentioned step S6, the processor 40 may determine the origin of the transport device coordinate system C4 as a predetermined position on the axis A3 (for example, the upstream end or downstream end of the transport surface 34, or the center, upstream end, or downstream end of the operating range of the robot 12).

[0049] The coordinate Q3 indicates the position and orientation (directions of each axis) of the first index coordinate system C3 in the robot coordinate system C1 before the transfer in step S3. In step S7, the processor 40 calculates the coordinates (W3, P3, R3) indicating the orientation among the coordinates Q3 by calculating the coordinates W3, P3, R3 from the coordinates Q3, so that the positive y-axis direction of the first index coordinate system C3 before the transfer is the transfer direction D1 (vector VT ３_４ ), the processor 40 may determine the coordinates (W3c, P3c, R3c) of the new attitude as the directions of the axes of the transport device coordinate system C4.

[0050] Similarly, the processor 40 may obtain new coordinates (W4c, P4c, R4c) of the orientation by correcting the coordinates (W4, P4, R4) indicating the orientation among the coordinates Q4 indicating the position and orientation of the first index coordinate system C3 in the robot coordinate system C1 after the transfer in step S3. Then, the processor 40 may determine the new coordinates (W4c, P4c, R4c) of the orientation as the directions of the axes of the transfer device coordinate system C4.

[0051] In this way, as shown in FIG. 7, the transport device coordinate system C4 is set in the robot coordinate system C1. The transport device coordinate system C4 is fixed with respect to the robot coordinate system C1, and its positive y-axis direction indicates with high precision the transport direction D1 in the robot coordinate system C1. The x-y plane (or origin) of the transport device coordinate system C4 indicates the position of the transport surface 34 in the robot coordinate system C1. In this way, in this embodiment, the processor 40 functions as a coordinate system setting unit 56 (FIG. 2) that sets the transport device coordinate system C4 in the robot coordinate system C1 based on the transport direction D1.

[0052] As described above, the processor 40 functions as the position data acquisition unit 52, the transport direction acquisition unit 54, and the coordinate system setting unit 56, and sets the transport device coordinate system C4 in the robot coordinate system C1 based on the image data ID1 and ID2 captured by the camera 14 of the first index 18. Therefore, the camera 14, the first index 18, the position data acquisition unit 52, the transport direction acquisition unit 54, and the coordinate system setting unit 56 constitute a device 60 ( FIG. 2 ) that sets the transport device coordinate system C4 in the robot coordinate system C1.

[0053] In this device 60, a first index 18 representing a first index coordinate system C3 is placed on the transport device 16 (specifically, on the transport surface 34) so ​​as to be transported by the transport device 16. The camera also acquires first image data ID1 (FIG. 4) capturing an image of the first index 18, and second image data ID2 (FIG. 6) capturing an image of the first index 18 transported by the transport device 16 in step S3 after capturing the first image data ID1 (steps S1 and S4).

[0054] The position data acquisition unit 52 also acquires first position data PD1 (specifically, coordinate Q1) indicating the three-dimensional position of the first index coordinate system C4 represented by the first index 18 in the first image data ID1 relative to the camera 14 (specifically, camera coordinate system C2) (step S2). Next, the position data acquisition unit 52 acquires second position data PD2 (specifically, coordinate Q2) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 in the second image data ID2 relative to the camera 14 (camera coordinate system C2) (step S5).

[0055] Then, the conveying direction acquisition unit 54 determines the conveying direction D1 (specifically, the vector VT) in the robot coordinate system C1 based on the first position data PD1 and the second position data PD2. ３_４ , or axis A3) is obtained (step S6). The coordinate system setting unit 56 sets the transfer device coordinate system C4 in the robot coordinate system C1 based on the transfer direction D1 obtained by the transfer direction acquisition unit 54 (step S7).

[0056] According to this configuration, the transport device coordinate system C4 can be set in the robot coordinate system C1 by capturing an image of the first target 18 transported by the transport device 16 with the camera 14, without touching up the hand (wrist flange 30b or end effector) of the robot 12 to the transport device 16 or performing visual touch-up by capturing an image of multiple marks provided on the transport device 16 with the camera 14. This makes it easier to set the transport device coordinate system C4 relative to the robot coordinate system C1 (i.e., calibration).

[0057] Furthermore, in the device 60, the first index 18 includes a pattern 18a that represents the three-dimensional position of the first index coordinate system C3 in the camera coordinate system C2 that is set in the camera 14 that captured the image data ID. According to this configuration, the pattern 18a of the first index 18 that appears in the image data ID can more effectively represent the first index coordinate system C3 in the camera coordinate system C2.

[0058] In this embodiment, in step S4, the first index 18 is imaged with the camera 14 positioned at the same imaging position PS1 as in step S1. However, this is not limiting, and in step S4, the processor 40 may move the camera 14 from imaging position PS1 to imaging position PS2 and acquire image data ID2′ of the first index 18 captured at imaging position PS2.

[0059] In this case, the processor 40 functions as the position data acquisition unit 52 in step S5, and acquires coordinates Q2' (X2', Y2', Z2', W2', P2', R2') in the camera coordinate system C2 of the first index coordinate system C3 identified in the image data ID2'. Then, in step S7, the processor 40 calculates the known positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 of the camera 14 placed at the imaging position PS2. １_２_２ (Specifically, the transformation matrix M １_２_２ ) to transform the coordinate Q2' into coordinate Q4 (X4, Y4, Z4, W4, P4, R4) in the robot coordinate system C1.

[0060] Next, a robot system 70 according to another embodiment will be described with reference to Figures 8 and 9. The robot system 70 differs from the robot system 10 described above in that it further includes a terminal device 72 and a second indicator 74. The terminal device 72 is, for example, a smartphone, a tablet terminal device, or a portable computer such as a notebook PC, and can be carried by the operator.

[0061] Specifically, as shown in Fig. 9, the terminal device 72 has a processor 80, a memory 82, an I / O interface 84, a display device 86, an input device 88, and a camera 14. The display device 86 and the input device 88 are integrally provided on the front surface of the terminal device 72 as shown in Fig. 8, while the camera 14 is integrally provided on the back surface of the terminal device 72 as shown in Fig. 10. A camera coordinate system C2 is set for the camera 14, as in the above-described embodiment.

[0062] The processor 80 is communicatively connected to the memory 82, the I / O interface 84, the display device 86, the input device 88, and the camera 14 via a bus 90, and performs arithmetic processing to realize the function of setting a transport device coordinate system, which will be described later, while communicating with these components. The I / O interface 84 is communicatively connected to the I / O interface 44 of the control device 20 via a wired or wireless connection.

[0063] The configurations of the processor 80, memory 82, I / O interface 84, display device 86, and input device 88 are similar to those of the processor 40, memory 42, I / O interface 44, display device 48, and input device 50 described above, respectively, and therefore redundant explanations will be omitted.

[0064] The second index 74 is used to set the transport device coordinate system C4 in the robot coordinate system C1 together with the first index 18. Specifically, as shown in Fig. 8, the second index 74 is made of a flat paper or plate material, similar to the above-mentioned first index 18, and has a pattern 74a visibly displayed on its surface.

[0065] The pattern 74a includes, for example, a figure consisting of a plurality of lines or curves, or a dot pattern. The second indicator 74 is fixed to the robot 12 (for example, the robot base 22) and is configured to represent a second indicator coordinate system C5 by the pattern 74a. Here, the second indicator 74 is configured to represent a positional relationship R between the second indicator coordinate system C5 represented by the pattern 74a and the robot coordinate system C1. １_５ is positioned relative to the robot 12 so that .times. ...

[0066] Therefore, the coordinates of the robot coordinate system C1 and the coordinates of the second index coordinate system C5 are transformed by a known transformation matrix M １_５ In this embodiment, the control device 20 can convert the positional relationship R １_５ As data indicating １_５and the coordinates of the second index coordinate system C5 in the robot coordinate system C1, and are stored in the memory 42. In Fig. 8, the robot coordinate system C1 is shown by a dotted line for ease of understanding.

[0067] 11, a method for setting the transport device coordinate system C4 in the robot coordinate system C1 in the robot system 70 will be described. As in the above-described embodiment, the operator places the first index 18 on the transport surface 34 of the transport device 16 before starting the flow of FIG.

[0068] In step S11, the processor 80 of the terminal device 72 captures images of the first index 18 and the second index 74 using the camera 14. Specifically, the operator places the terminal device 72 at an imaging position PS3 where the first index 18 and the second index 74 can be placed within the field of view of the camera 14 of the terminal device 72. At this time, the processor 80 may display image data ID continuously captured by the camera 14 on the display device 86 of the terminal device 72 in real time (so-called live view display).

[0069] The operator then operates the input device 88 of the terminal device 72 placed at the imaging position PS3 to issue an imaging command to the processor 80. In response to the imaging command, the processor 80 operates the camera 14 to capture images of the first index and the second index 74. An example of image data ID3 captured by the camera 14 in step S11 is shown in FIG.

[0070] The camera 14 acquires image data ID3 (first image data) capturing the first index 18 and the second index in this manner, and supplies the image data ID3 to the processor 80. When capturing the image data ID3, the operator may hold the terminal device 72 in his / her hand, or may fix the terminal device 72 to a fixed position using a fixture (such as a tripod or a stand).

[0071] In step S12, the processor 80 functions as the position data acquisition unit 52 ( FIG. 9 ) and acquires position data PD3 (first position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 captured in the image data ID3 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 described above to analyze the pattern 18a of the first index 18 captured in the image data ID3 and identify the first index coordinate system C3 represented by the first index 18 within the image data ID3. The image analysis program PG1 may be stored in advance in the memory 82.

[0072] Then, the processor 80 acquires a coordinate Q5 (X5, Y5, Z5, W5, P5, R5) in the camera coordinate system C2 of the identified first index coordinate system C3. This coordinate Q5 indicates the three-dimensional position of the first index coordinate system C3 relative to the camera 14 when the image data ID3 was captured. In this step S12, the processor 80 acquires the coordinate Q5 as position data PD3.

[0073] The processor 80 also functions as the position data acquisition unit 52, and acquires position data PD4 (third position data) indicating the three-dimensional position of the second index coordinate system C5 represented by the second index 74 captured in the image data ID3 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 to analyze the pattern 74a of the second index 74 captured in the image data ID3, and identifies the second index coordinate system C5 represented by the second index 74 within the image data ID3.

[0074] Then, the processor 80 acquires a coordinate Q6 (X6, Y6, Z6, W6, P6, R6) in the camera coordinate system C2 of the identified second index coordinate system C5. This coordinate Q6 indicates the three-dimensional position of the second index coordinate system C5 relative to the camera 14 at the time of capturing the image data ID3. In this step S12, the processor 80 acquires the coordinate Q6 as position data PD4. In this way, in this step S12, the processor 80 acquires the position data PD3 of the first index coordinate system C3 as well as the position data PD4 of the second index coordinate system C5.

[0075] When step S12 is completed, the processor 80 transmits a completion signal SG1 to the control device 20. The processor 40 of the control device 20 executes step S13 in response to the completion signal SG1. Specifically, in step S13, the processor 40 operates the conveying device 16 to convey the first index 18 by the distance δ in the conveying direction D1, similar to step S3 described above.

[0076] When the processor 40 receives the completion signal SG1, it may generate a notification signal NS1 informing the operator that the first index 18 should be transported by the transport device 16, and output the notification signal NS1 to the display device 48 (or a speaker provided in the control device 20). Then, when the operator recognizes the notification signal NS1, he or she may execute step S13 and manually operate the transport device 16 to transport the first index 18. The state after completion of step S13 is shown in FIG. 13. For ease of understanding, FIG. 13 also shows the position of the first index 18 at the time of capturing the image data ID3 by a dotted line B, and also shows a first index coordinate system C3 represented by the first index 18.

[0077] After completing step S13, the processor 40 transmits a completion signal SG2 to the terminal device 72. Upon receiving the completion signal SG2, the processor 80 of the terminal device 72 executes step S14. At this time, the processor 80 may generate a notification signal NS2 notifying the user that the first index 18 and the second index 74 should be imaged again by the camera 14, and output the notification signal NS2 to the display device 86 (or a speaker provided in the terminal device 72).

[0078] In step S14, the processor 80 of the terminal device 72 captures images of the first index 18 and the second index 74 using the camera 14. Specifically, when the operator recognizes the notification signal NS2 output to the display device 86 (or the speaker of the terminal device 72), the operator places the terminal device 72 at an imaging position PS4 where the first index 18 and the second index 74 can be within the field of view of the camera 14. This imaging position PS4 may be different from or the same as the imaging position PS3 of step S11 described above. The processor 80 may also execute the live view display described above at this time.

[0079] The operator then operates the input device 88 of the terminal device 72 to operate the camera 14, which captures images of the first index 18 and the second index 74. An example of image data ID4 captured by the camera 14 in step S14 is shown in FIG. 14. This image data ID4 captures the second index 74 along with the first index 18 after transport in step S13. The camera 14 acquires image data ID4 (second image data) capturing the images of the transported first index 18 and second index 74, and supplies the image data to the processor 80.

[0080] In addition, if the terminal device 72 is fixed at a fixed point at the same imaging position PS3 = PS4 using a fixing device in steps S11 and S14, the processor 80 may automatically execute step S14 when it receives the above-mentioned completion signal SG2, and automatically capture image data ID4 using the camera 14.

[0081] In step S15, the processor 80 functions as the position data acquisition unit 52 (Figure 9) and, similar to step S12 described above, acquires position data PD5 (second position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 in the image data ID4 relative to the camera 14.

[0082] Specifically, the processor 80 executes the image analysis program PG1 to acquire coordinates Q7 (X7, Y7, Z7, W7, P7, R7) in the camera coordinate system C2 of the first index coordinate system C3 represented by the first index 18 captured in the image data ID4. The coordinates Q7 indicate the three-dimensional position of the first index coordinate system C3 relative to the camera 14 at the time the image data ID4 was captured. In this way, the processor 80 acquires the coordinates Q7 as position data PD5.

[0083] The processor 80 also functions as the position data acquisition unit 52 and acquires position data PD6 (fourth position data) indicating the three-dimensional position of the second index coordinate system C5 represented by the second index 74 shown in the image data ID4 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 to acquire coordinates Q8 (X8, Y8, Z8, W8, P8, R8) in the camera coordinate system C2 of the second index coordinate system C5 represented by the second index 74 shown in the image data ID4.

[0084] This coordinate Q8 indicates the three-dimensional position of the second index coordinate system C5 relative to the camera 14 when the image data ID4 was captured. The processor 80 acquires the coordinate Q8 as position data PD6. Thus, in step S15, the processor 80 acquires the position data PD5 of the first index coordinate system C3 and the position data PD6 of the second index coordinate system C5.

[0085] In step S16, the processor 80 functions as the conveying direction acquisition unit 54 (FIG. 9) and obtains the conveying direction D1 based on the position data PD3, PD4, PD5, and PD6. Specifically, the processor 80 obtains the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 represented in the camera coordinate system C2 of the image data ID3 (FIG. 12) based on the position data PD3 of the first index coordinate system C3 (i.e., the coordinate Q5 in the camera coordinate system C2) and the position data PD4 of the second index coordinate system C5 (i.e., the coordinate Q6 in the camera coordinate system C2) obtained in the above-mentioned step S12. ３_５_１ (First positional relationship) is calculated.

[0086] Here, the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 in the three-dimensional space defined by the camera coordinate system C2 of the image data ID3 is ３_５_１ is known from the position data PD3 and PD4. The processor 80 calculates the positional relationship R ３_５_１ As the data, coordinates Q9 (X9, Y9, Z9, W9, P9, R9) of the first index coordinate system C3 in the second index coordinate system C5 shown in FIG. 12 are acquired.

[0087] Similarly, the processor 80 calculates the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 represented in the camera coordinate system C2 of the image data ID4 (FIG. 14) based on the position data PD5 of the first index coordinate system C3 (i.e., the coordinate Q7 in the camera coordinate system C2) and the position data PD6 of the second index coordinate system C5 (i.e., the coordinate Q8 in the camera coordinate system C2) acquired in the above-mentioned step S15. ３_５_２ Specifically, the processor 80 calculates the positional relationship R (second positional relationship) from the position data PD5 and PD6. ３_５_２ As the data, the coordinates Q10 (X10, Y10, Z10, W10, P10, R10) of the first index coordinate system C3 in the second index coordinate system C5 shown in FIG. 14 are acquired.

[0088] Then, the processor 80 calculates a vector VT from the coordinates (X9, Y9, Z9) of the coordinates Q9 to the coordinates (X10, Y10, Z10) of the coordinates Q10. ９_１０ This vector V ９_１０ coincides with the axis A3 connecting the coordinates (X9, Y9, Z9) and the coordinates (X10, Y10, Z10), and indicates the position of the vector of the conveying direction D1 (in other words, the axis A3) in the second index coordinate system C5.

[0089] This vector VT ９_１０ (or axis A3) is expressed as a coordinate (or function) of the second index coordinate system C5, and indicates the locus in the second index coordinate system C5 of the origin of the first index coordinate system C3 that has been moved in the conveying direction D1 in step S13, and can be considered to be located on the conveying surface 34. In this way, the processor 80 calculates the vector VT in the second index coordinate system C5 as the conveying direction D1. ９_１０ (or axis A3).

[0090] After completing step S16, the processor 80 calculates the data of the conveying direction D1 acquired in step S15 (specifically, the vector VT in the second index coordinate system C5). ９_１０ or coordinates or function data of the axis A3) to the control device 20. When the processor 40 of the control device 20 receives the data of the conveying direction D1, it executes step S17.

[0091] In step S17, the processor 40 functions as the coordinate system setting unit 56 (FIG. 9) and sets the transport device coordinate system C4 (FIG. 15) in the robot coordinate system C1 based on the transport direction D1 calculated in step S16. Specifically, the processor 40 calculates the known positional relationship R between the second index coordinate system C5 and the robot coordinate system C1. １_５ (Specifically, the transformation matrix M １_５ ) to calculate the conveying direction D1 (for example, vector VT) calculated as the coordinates (or function) of the second index coordinate system C5 in step S16. ９_１０ ) is transformed into the robot coordinate system C1.

[0092] Then, in the same manner as in step S7 described above, the processor 40 calculates the conveying direction D1 (vector VT ９_１０ ) as the positive y-axis direction of the transport device coordinate system C4. Thereafter, the processor 40 determines the positive x-axis direction and the positive z-axis direction of the transport device coordinate system C4 in the same manner as in step S7 described above.

[0093] The processor 40 also determines the origin of the transport device coordinate system C4 in the robot coordinate system C1. As one example, the processor 40 converts the above-mentioned coordinate Q9 (X9, Y9, Z9) into coordinate C9' (X9', Y9', Z9') in the robot coordinate system C1 and determines the coordinate C9' as the origin of the transport device coordinate system C4. As another example, the processor 40 may convert the above-mentioned coordinate Q10 (X10, Y10, Z10) into coordinate C10' (X10', Y10', Z10') in the robot coordinate system C1 and determine the coordinate C10' as the origin of the transport device coordinate system C4.

[0094] As yet another example, the processor 40 may convert the axis A3 obtained as the coordinate (or function) of the second index coordinate system C5 in step S16 into the robot coordinate system C1, and determine the origin of the transport device coordinate system C4 as a predetermined position on the axis A3 in the robot coordinate system C1. In this way, the transport device coordinate system C4 is set in the robot coordinate system C1, as shown in FIG.

[0095] As described above, in this embodiment, the processor 80 of the terminal device 72 functions as the position data acquisition unit 52 and the conveying direction acquisition unit 54, while the processor 40 of the control device 20 functions as the coordinate system setting unit 56. The processors 40 and 80 cooperate with each other to set the conveying device coordinate system C4 in the robot coordinate system C1 based on the image data ID3 and ID4 captured by the camera 14 of the first index 18 and the second index 74. Therefore, the camera 14, the first index 18, the position data acquisition unit 52, the conveying direction acquisition unit 54, the coordinate system setting unit 56, and the second index 74 constitute an apparatus 100 ( FIG. 9 ) that sets the conveying device coordinate system C4 in the robot coordinate system C1.

[0096] In this device 100, a second index 74 representing a second index coordinate system C5 is placed at a known position in the robot coordinate system C1, and the camera 14 acquires first image data ID3 capturing an image of the first index 18 and the second index 74, and second image data ID4 capturing an image of the first index 18 and the second index 74 transported in step S13 (steps S11, S14).

[0097] The position data acquisition unit 52 (processor 80) also acquires third position data PD4 (specifically, coordinate Q6) indicating the three-dimensional position of the second index coordinate system C5 represented by the second index 74 in the first image data ID3 relative to the camera 14 (camera coordinate system C2) (step S12). Next, the position data acquisition unit 52 acquires fourth position data PD6 (specifically, coordinate Q8) indicating the three-dimensional position of the second index coordinate system C5 represented by the second index 74 in the second image data ID4 relative to the camera 14.

[0098] Then, the conveying direction acquisition unit (processor 80) determines the conveying direction D1 in the robot coordinate system C1 based on the first position data PD3 (coordinate Q5), the second position data PD5 (coordinate Q7), the third position data PD4 (coordinate Q6), and the fourth position data PD6 (coordinate Q8) (step S16).

[0099] According to this configuration, the transport device coordinate system C4 can be set in the robot coordinate system C1 simply by capturing images of the first index 18 and the second index 74 with the camera 14. Therefore, the transport device coordinate system C4 can be easily set relative to the robot coordinate system C1 (i.e., calibration).

[0100] In the device 100, the conveying direction acquisition unit 54 calculates a first positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 in the first image data ID3 (FIG. 12) based on the first position data PD3 and the third position data PD4. ３_５_１ (Specifically, coordinate Q9) is obtained (step S16).

[0101] Furthermore, the conveying direction acquisition unit 54 calculates the second positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 in the second image data ID4 (FIG. 14) based on the second position data PD5 and the fourth position data PD6. ３_５_２ (Specifically, the coordinate Q10) is obtained (step S16).

[0102] Then, the conveying direction acquisition unit 54 calculates the first positional relationship R ３_５_１ and the second positional relationship R ３_５_２ (Step S16) According to this configuration, the conveying direction D1 can be quickly and accurately determined using an existing algorithm (specifically, the image analysis program PG1 or the like).

[0103] Furthermore, in the device 100, the second index 74 includes a pattern 74a that represents the three-dimensional position of the second index coordinate system C5 in the camera coordinate system C2 that is set in the camera 14 that captured the image data ID3 and ID4. According to this configuration, the pattern 74a of the second index 74 that appears in the image data ID3 and ID4 can more effectively represent the second index coordinate system C5 in the camera coordinate system C2.

[0104] 16 and 17, a robot system 110 according to still another embodiment will be described. The robot system 110 differs from the robot system 70 described above in that it further includes a sensor 112. The sensor 112 detects a displacement Δ of the camera 14.

[0105] More specifically, the sensor 112 has at least one of a gyro sensor and an acceleration sensor, and is provided on the terminal device 72 so as to be fixed relative to the camera 14. In this embodiment, the sensor 112 is communicatively connected to the I / O interface 84 of the terminal device 72, detects a displacement Δ of the position of the camera 14 (in other words, the terminal device 72) (more specifically, the amount and direction of displacement of the position and attitude), and supplies the detected displacement to the processor 80.

[0106] 18 , a method for setting the transport device coordinate system C4 in the robot coordinate system C1 in the robot system 110 will be described. As in the above-described embodiment, the operator places the first index 18 on the transport surface 34 of the transport device 16 before starting the flow of FIG.

[0107] In step S21, the processor 80 of the terminal device 72 starts detecting the displacement Δ of the camera 14 using the sensor 112. Specifically, the processor 80 operates the sensor 112 to continuously (e.g., periodically) detect the displacement Δ of the position of the camera 14 (or the terminal device 72). The processor 80 sequentially acquires detection data of the displacement Δ from the sensor 112 and stores it in the memory 42.

[0108] In step S22, the processor 80 captures an image of the first index 18 using the camera 14. Specifically, the operator places the terminal device 72 at an imaging position PS5 where the first index 18 can be placed within the field of view of the camera 14 of the terminal device 72. At this time, the processor 80 may execute the live view display described above.

[0109] The operator then operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the first index 18. An example of image data ID5 captured by the camera 14 in step S22 is shown in Figure 19. The camera 14 acquires the image data ID5 (first image data) of the first index 18 in this manner and supplies it to the processor 80.

[0110] In step S23, the processor 80 functions as the position data acquisition unit 52 (FIG. 17) and acquires position data PD7 (first position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 in the image data ID5 relative to the camera 14. Specifically, as in the above-described embodiment, the processor 80 executes the image analysis program PG1 to acquire, as the position data PD7, coordinates Q11 (X11, Y11, Z11, W11, P11, R11) of the first index coordinate system C3 in the camera coordinate system C2 of FIG. 19. When step S23 is completed, the processor 80 transmits a completion signal SG1 to the control device 20.

[0111] The processor 40 of the control device 20 executes step S24 in response to the completion signal SG1. Specifically, in step S24, the processor 40 operates the conveying device 16 to convey the first index 18 a distance δ in the conveying direction D1, similar to step S13 described above. The state after completion of step S24 is shown in FIG. 20. Note that, for ease of understanding, FIG. 20 also shows the position of the first index 18 at the time of capturing the image data ID5 by a dotted line B, and also shows a first index coordinate system C3 represented by the first index 18.

[0112] After completing step S24, the processor 40 transmits a completion signal SG2 to the terminal device 72. Upon receiving the completion signal SG2, the processor 80 of the terminal device 72 executes step S25. At this time, the processor 80 may output a notification signal NS3 to notify the user that an image of the first index 18 should be captured again by the camera 14, as in the above-described embodiment.

[0113] In step S25, the processor 80 of the terminal device 72 captures an image of the first index 18 using the camera 14. Specifically, when the operator recognizes the notification signal NS3 output to the display device 86 (or the speaker of the terminal device 72), the operator places the terminal device 72 at an imaging position PS6 where the first index 18 can be placed within the field of view of the camera 14. This imaging position PS6 may be different from or the same as the imaging position PS5 of step S22 described above. The processor 80 may also execute the live view display described above at this time.

[0114] The operator then operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the first index 18. An example of image data ID6 captured by the camera 14 in step S25 is shown in Figure 21. The camera 14 acquires image data ID6 (second image data) of the transported first index 18 and supplies it to the processor 80.

[0115] In step S26, the processor 80 functions as the position data acquisition unit 52 (FIG. 17) and acquires position data PD8 (second position data) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 captured in the image data ID6 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 in the same manner as in step S23 described above, thereby acquiring, as the position data PD8, coordinates Q12 (X12, Y12, Z12, W12, P12, R12) of the first index coordinate system C3 in the camera coordinate system C2 of FIG.

[0116] In step S27, the processor 80 causes the camera 14 to capture an image of the second index 74. Specifically, the operator places the terminal device 72 at an imaging position PS7 where the second index 74 can be placed within the field of view of the camera 14 of the terminal device 72. This imaging position PS7 is different from the above-mentioned imaging positions PS5 and PS6. At this time, the processor 80 may execute the above-mentioned live view display.

[0117] The operator then operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the second index 74. An example of image data ID7 captured by the camera 14 in step S27 is shown in FIG. 22 . The camera 14 acquires image data ID7 (third image data) of the image of the second index 74 and supplies it to the processor 80. In this manner, in this embodiment, the operator separately captures images of the first index 18 and the second index 74 in steps S22 and S25, and in step S27.

[0118] In step S28, the processor 80 functions as the position data acquisition unit 52 (FIG. 17) and acquires position data PD9 (third position data) indicating the three-dimensional position of the second index coordinate system C5 represented by the second index 74 captured in the image data ID7 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1, as in the above-described embodiment, to acquire, as the position data PD9, coordinates Q13 (X13, Y13, Z13, W13, P13, R13) of the second index coordinate system C5 in the camera coordinate system C2 of FIG.

[0119] In step S29, the processor 80 calculates the conveying direction D1 based on the position data PD7, PD8, and PD9 and the displacement Δ detected by the sensor 112. After the start of step S21, the sensor 112 continues to detect the displacement Δ of the camera 14 (terminal device 72) while capturing the image data ID5 in step S22, the image data ID6 in step S25, and the image data ID7 in step S27.

[0120] Therefore, the processor 80 calculates, from the detection data of the sensor 112, a displacement Δ ５_６ and the displacement Δ from the imaging position PS6 to the imaging position PS7 where the image data ID7 was captured in step S27. ６_７ It is possible to find:

[0121] The processor 80 calculates the displacement Δ ５_６ and Δ ６_７ From the position data PD7 (coordinate Q11) acquired in step S23 and the position data PD9 (coordinate Q13) acquired in step S27, the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 before the transfer in step S24 is calculated. ３_５_３ Specifically, the processor 80 calculates the positional relationship R ３_５_３ As the data, coordinates Q14 (X14, Y14, Z14, W14, P14, R14) in the second index coordinate system C5 of the first index coordinate system C3 before transport are acquired.

[0122] The processor 80 also calculates the displacement Δ ６_７ From the position data PD8 (coordinate Q12) acquired in step S26 and the position data PD9 (coordinate Q13) acquired in step S27, the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 after the transfer in step S24 is calculated. ３_５_４ Specifically, the processor 80 calculates the positional relationship R ３_５_４ As the data, the coordinates Q15 (X15, Y15, Z15, W15, P15, R15) in the second index coordinate system C5 of the first index coordinate system C3 after transport are acquired.

[0123] Then, the processor 80 calculates a vector VT from the coordinates (X14, Y14, Z14) of the coordinates Q14 to the coordinates (X15, Y15, Z15) of the coordinates Q15. １４_１５ This vector V １４_１５ coincides with the axis A3 connecting the coordinates (X14, Y14, Z14) and the coordinates (X15, Y15, Z15), and indicates the position of the vector of the conveying direction D1 (in other words, the axis A3) in the second index coordinate system C5.

[0124] This vector VT １４_１５ (or axis A3) is expressed as a coordinate (or function) of the second index coordinate system C5, and indicates the locus in the second index coordinate system C5 of the origin of the first index coordinate system C3 that has been moved in the conveying direction D1 in step S24, and can be considered to be located on the conveying surface 34. In this way, the processor 80 calculates the vector VT in the second index coordinate system C5 as the conveying direction D1. １４_１５ (or axis A3).

[0125] After the completion of step S29, the processor 80, similarly to the above-described embodiment, calculates the data of the conveying direction D1 acquired in step S29 (specifically, the vector VT in the second index coordinate system C5). １４_１５ or coordinates or function data of the axis A3) to the control device 20. When the processor 40 of the control device 20 receives the data of the conveying direction D1, it executes step S30.

[0126] In step S30, the processor 40 functions as the coordinate system setting unit 56 (FIG. 17) and sets the transport device coordinate system C4 (FIG. 23) in the robot coordinate system C1 based on the transport direction D1 obtained in step S29. Specifically, the processor 40 sets the known positional relationship R between the second index coordinate system C5 and the robot coordinate system C1, similar to step S17 described above. １_５ The conveying direction D1 (for example, vector VT) calculated as the coordinates (or function) of the second index coordinate system C5 in step S29 is calculated using the vector VT １４_１５ ) is transformed into the robot coordinate system C1.

[0127] Then, similarly to the above-described embodiment, the processor 40 calculates the conveying direction D1 (vector VT １４_１５ ) is defined as the y-axis positive direction of the transport device coordinate system C4, and also as the x-axis positive direction and z-axis positive direction of the transport device coordinate system C4.

[0128] The processor 40 also determines the origin of the transport device coordinate system C4 in the robot coordinate system C1. As one example, the processor 40 converts the above-mentioned coordinate Q14 (X14, Y14, Z14) into coordinate C14' (X14', Y14', Z14') in the robot coordinate system C1 and determines the coordinate C14' as the origin of the transport device coordinate system C4. As another example, the processor 40 may convert the above-mentioned coordinate Q15 (X15, Y15, Z15) into coordinate C15' (X15', Y15', Z15') in the robot coordinate system C1 and determine the coordinate C15' as the origin of the transport device coordinate system C4.

[0129] As yet another example, the processor 40 may convert the axis A3 obtained as the coordinate (or function) of the second index coordinate system C5 in step S29 into the robot coordinate system C1, and set the origin of the transport device coordinate system C4 as a predetermined position on the axis A3 in the robot coordinate system C1. In this way, the transport device coordinate system C4 is set in the robot coordinate system C1, as shown in FIG.

[0130] As described above, in this embodiment, the processors 40 and 80 cooperate with each other to function as the position data acquisition unit 52, the conveying direction acquisition unit 54, and the coordinate system setting unit 56, and set the conveying device coordinate system C4 in the robot coordinate system C1 based on the image data ID5, ID6, and ID7 of the first index 18 and the second index 74 captured by the camera 14 and the displacement Δ detected by the sensor 112. Therefore, the camera 14, the first index 18, the position data acquisition unit 52, the conveying direction acquisition unit 54, the coordinate system setting unit 56, the second index 74, and the sensor 112 constitute a device 120 ( FIG. 17 ) that sets the conveying device coordinate system C4 in the robot coordinate system C1.

[0131] In this device 120, the camera 14 acquires third image data ID3 (step S27) that captures the second index 74, along with first image data ID5 (step S22) and second image data ID6 (step S25) that capture the first index 18. The position data acquisition unit 52 (processor 80) also acquires third position data PD9 (step S28) that indicates the three-dimensional position of the second index coordinate system C5 represented by the second index 74 captured in the third image data ID3 relative to the camera 14 (camera coordinate system C2).

[0132] The sensor 112 also detects the displacement Δ of the camera 14 while capturing the first image data ID5, the second image data ID6, and the third image data ID7. ５_６ and Δ ６_７ Then, the conveying direction acquisition unit 54 detects the first position data PD7, the second position data PD8, the third position data PD9, and the displacement Δ ５_６ and Δ ６_７ Based on this, the conveying direction D1 is determined (step S29).

[0133] Here, due to space constraints in the work cell, it may not be possible to simultaneously capture images of the first index and the second index 74 using the camera 14, as shown in Figures 12 and 14. According to this embodiment, as shown in Figures 19, 21, and 22, the transport direction D1 can be determined even if images of the first index and the second index 74 are captured separately. Therefore, even if there is limited space in the work cell, the transport device coordinate system C4 can be effectively set in the robot coordinate system C1.

[0134] 3, 11, or 18, the processor 40 or 80 has been described as capturing an image of the first index 18 with the camera 14 before and after the conveyance device 16 conveys the first index 18 only once, and determining the conveyance direction D1. However, the present invention is not limited to this, and the processor 40 or 80 may convey the first index 18 multiple times with the conveyance device 16, capture an image of the first index 18 with the camera 14 before and after the conveyance, and determine the conveyance direction D1 multiple times.

[0135] Such an embodiment will be described below with reference to Fig. 24. The flow in Fig. 24 shows the flow of another method for setting the transport device coordinate system C4 in the robot coordinate system C1 in the above-described robot system 10. In step S31, the processor 40 sets the number "n" indicating the number of times an image of the first index 18 is to be captured in step S32, which will be described later, to "1" (n=1).

[0136] In step S32, the processor 40 images the first index 18 with the camera 14. Specifically, the processor 40 operates the robot 12 to position the camera 14 at an imaging position PSn where the first index 18 can be placed within the field of view of the camera 14. The processor 40 then operates the camera 14 to image the first index 18, thereby acquiring image data IDn as shown in, for example, FIG. 4 or 6. If the number "n" is set to n=1 at the start of step S32, the processor 40 images image data ID1 with the camera 14 positioned at imaging position PS1.

[0137] In step S33, the processor 40 functions as the position data acquisition unit 52 (Figure 2), similar to step S2 or S5 described above, and acquires position data PDn (specifically, coordinate Qn in the camera coordinate system C2) indicating the three-dimensional position of the first index coordinate system C3 represented by the first index 18 in the image data IDn relative to the camera 14.

[0138] If the number "n" is set to n=1 at the start of step S33, the processor 40 acquires position data PD1 (specifically, coordinate Q1 in the camera coordinate system C2) indicating the three-dimensional position of the first index coordinate system C3 represented in the image data ID1 relative to the camera 14. In step S34, the processor 40 increments the number "n" by 1 (n=n+1).

[0139] In step S35, the processor 40 determines whether the number "n" is n>2. If the processor 40 determines YES, it proceeds to step S36, whereas if the processor 40 determines NO, it proceeds to step S38. If the processor 40 proceeds to step S35 after executing step S32 for the first time (i.e., with n set to 1), the number "n" at this point has been incremented to n=2 in step S34. Therefore, in this case (i.e., when the image of the first index 18 has been captured only once), the processor 40 determines NO in step S35 and proceeds to step S38.

[0140] In step S36, the processor 40 functions as the conveying direction acquisition unit 54 (FIG. 2) in the same manner as in step S6 described above, and acquires the conveying direction D1 based on the position data PDn (specifically, the coordinate Qn) acquired in the most recently executed step S33 and the position data PDn-1 (specifically, the coordinate Qn-1) acquired in the step S33 executed before the acquisition of the position data PDn. _ｎ Ask for.

[0141] 24, the processor 40 repeatedly executes a loop of steps S32 to S38 until it determines YES in step S37, which will be described later. Therefore, the processor 40 repeatedly conveys the first index 18 by the conveying device 16 in step S38, which will be described later, and acquires position data PDn-1 (first position data) in step S33, which is executed before the conveyance, from image data IDn-1 (first image data) captured in step S32 before the conveyance. Meanwhile, the processor 40 acquires position data PDn (second position data) in step S33, which is executed after the conveyance, from image data IDn (second image data) captured in step S32 after the conveyance.

[0142] In step S36, the processor 40 calculates the position data PDn (second position data) acquired in the most recent step S33 and the position data PDn-1 (first position data) acquired in step S33 executed before the acquisition of the position data PDn in the conveying direction D1. _ｎ (For example, the above-mentioned vector V ３_４ Vector V _ｎ ) is required.

[0143] In step S37, the processor 40 determines whether the number "n" is equal to n ＭＡＸ It is determined whether this number n ＭＡＸ is the maximum number of times (for example, n ＭＡＸ = 10) and is predetermined by the operator. If the processor 40 determines that the answer is YES, the process proceeds to step S39, whereas if the processor 40 determines that the answer is NO, the process proceeds to step S38.

[0144] In step S38, similarly to step S3 described above, the processor 40 conveys the first index 18 by the conveying device 16. Note that while the processor 40 conveys the first index 18 in the conveying direction D1 in the nth step S38, the processor 40 may convey the first index 18 by the conveying device 16 in the direction D2 opposite to the conveying direction D1 in the (n+1)th step S38 executed thereafter.

[0145] In step S36, which is executed after the conveyance in the opposite direction D2, the processor 40 calculates a vector VT in the opposite direction D2 from the position data PDn-1 and PDn. _ｎ In this case, the processor 40 calculates the vector V _ｎ ' and the vector VT _ｎ in the conveying direction D1 _ｎ In this way, the processor 40 repeatedly executes the loop of steps S32 to S38 until it determines YES in step S37, and each time it executes step S36, it calculates the conveying direction D1 _ｎ Ask for.

[0146] In step S39, the processor 40 functions as the conveying direction acquisition unit 54 and acquires the final conveying direction D1 _Ｆ Specifically, the processor 40 calculates the conveying direction D1 _ｎ A plurality of vectors V _１ , V.T. _２ , ...VT _ｎ The composite vector VT _Ｆ (or average vector) is found, and the vector VT _Ｆ The final conveying direction D1 _Ｆ Obtain as.

[0147] In step S40, the processor 40 functions as the coordinate system setting unit 56 (FIG. 2) and, similarly to step S6 described above, calculates the conveying direction D1 obtained in step S39. _Ｆ Specifically, the processor 40 sets the transport device coordinate system C4 in the robot coordinate system C1 based on the vector V _Ｆ is defined as the positive y-axis direction of the transport device coordinate system C4.

[0148] In this manner, in this embodiment, the processor 40 calculates the conveying direction D1 each time the conveying device 16 conveys the first index 18 multiple times. _ｎ and the plurality of conveying directions D1 _ｎ Based on this, the final conveying direction D1 _ＦAccording to this configuration, the conveying direction D1 of the conveying device 16 can be obtained with higher accuracy.

[0149] 24 (i.e., the conveying device 16 conveys the first index 18 multiple times in the conveying direction D1 _ｎ are calculated, and a plurality of conveying directions D1 _ｎ Based on the final conveying direction D1 _Ｆ It should be understood that the process of obtaining the conveying directions D1 and D2 can be applied to the flow of Fig. 11 or Fig. 18. For example, in the flow of Fig. 11, the processors 40 and 80 cooperate with each other to repeatedly execute the loop of steps S11 to S16, and obtain the conveying directions D1 and D2 obtained each time step S16 is executed. _ｎ Based on this, the final conveying direction D1 _Ｆ You may ask for:

[0150] 18, the processor 80 of the terminal device 72 executes steps S27 and S28 after step S21. Thereafter, the processors 40 and 80 cooperate with each other to repeatedly execute a loop of steps S22 to S26 and S29, and each time step S29 is executed, the processors 40 and 80 calculate the conveying direction D1 obtained. _ｎ Based on this, the final conveying direction D1 _Ｆ You may ask for:

[0151] In the above embodiment, the second indicator 74 includes a pattern 74a that represents the second indicator coordinate system C5 in the camera coordinate system C2. However, this is not limiting, and the second indicator 74 may be configured with a shape 74b of the robot 12. As an example, the shape 74b may be configured with a plurality of surfaces, edges, concave portions, and convex portions formed on a component of the robot 12 (e.g., the robot base 22). As another example, the shape 74b may be configured with the shape of the entire robot 12 (i.e., the robot base 22, the rotating body 24, the lower arm 26, the upper arm 28, the wrist 30, and the wrist flange 30b) that is stationary in a predetermined position.

[0152] In this embodiment, when the camera 14 captures a shape 74b as the second index 74 in the above-mentioned steps S11, S14 or S27, the shape 74b captured in the captured image data ID represents the three-dimensional position (i.e., coordinate Q) of the second index coordinate system C5 in the camera coordinate system C2 of the camera 14.

[0153] Then, in the above-mentioned steps S12, S15 or 28, the processor 40 or 80 functions as a position data acquisition unit 52, executes an image analysis program PG2 (image analysis application) to read the second index coordinate system C4 from the shape 74b, and identifies the second index coordinate system C4 represented by the shape 74b within the image data ID.

[0154] Then, the processor 40 or 80 acquires the coordinate Q in the camera coordinate system C2 of the identified second index coordinate system C4 as the position data PD. Note that the first index 18 is not limited to the pattern 18a, and may include, for example, a shape 18b (a surface, an edge, a recess, a protrusion) formed on the conveying surface 34.

[0155] In the above-described robot systems 70 ( FIG. 9 ) and 110 ( FIG. 17 ), the processor 80 of the terminal device 72 functions as the position data acquisition unit 52 and the conveying direction acquisition unit 54. However, in the robot system 70 or 110, the processor 40 of the control device 20 may function as the position data acquisition unit 52, the conveying direction acquisition unit 54, and the coordinate system setting unit 56. In this case, the processor 80 of the terminal device 72 supplies the image data ID3, ID4, ID5, ID6, and ID7 captured by the camera 14 to the control device 20.

[0156] Alternatively, the terminal device 72 may be omitted, and the camera 14 may be, for example, a digital camera. The operator may then transmit the image data ID3, ID4, ID5, ID6, and ID7 captured by the camera 14 to the I / O interface 44 of the control device 20 via a wired or wireless connection.

[0157] In step S6, the processor 40 calculates the conveying direction D1 by a vector VT from the coordinate Q3 (X3, Y3, Z3) to the coordinate Q4 (X4, Y4, Z4) which indicates the locus of the origin of the first index coordinate system C3. ３_４ However, the present invention is not limited to this, and the processor 40 can also calculate the conveying direction D1 based on the amount of change between the coordinates (W3, P3, Q3) that indicate the attitude among the coordinates Q3 and the coordinates (W4, P4, R4) that indicate the attitude among the coordinates Q4.

[0158] Similarly, in the above-mentioned steps S16, S29 or S36, the processor 80 may determine the conveying direction D1 based on the amount of change in the coordinates Q9 (W9, P9, R9) and coordinate Q10 (W10, P10, R10) indicating the posture, the amount of change in the coordinates Q14 (W14, P14, R14) and coordinate Q15 (W15, P15, R15) indicating the posture, or the amount of change in the coordinates Qn-1 and coordinate Qn indicating the posture.

[0159] 3, 11, 18, or 24 in accordance with a computer program PG3 pre-stored in the memory 42 or 82. The functions of the position data acquisition unit 52, the conveying direction acquisition unit 54, and the coordinate system setting unit 56 executed by the processor 40 or 80 may be functional modules realized by the computer program PG3.

[0160] Furthermore, the control device 20 may have a first control device 20A that controls the robot 12 and a second control device 20B that controls the transport device 16. Furthermore, the robot 12 is not limited to a vertical articulated robot, but may be any type of robot, such as a horizontal articulated robot or a parallel link robot. Furthermore, the transport device 16 is not limited to a belt conveyor, but may be any device capable of transporting workpieces. While the present disclosure has been described above through the embodiments, the above-described embodiments do not limit the invention according to the claims.

[0161] 10, 70, 110 Robot system 12 Robot 14 Camera 16 Conveying device 18 First index 20 Control device 40, 80 Processor 52 Position data acquisition unit 54 Conveying direction acquisition unit 56 Coordinate system setting unit 60, 100, 120 Device 74 Second index 112 Sensor

Claims

1. An apparatus for setting a transport device coordinate system that defines the transport direction of a transport device that transports a workpiece in a robot coordinate system that is set on a robot that performs work on the workpiece, the apparatus comprising: a first index that represents a first index coordinate system, the first index being installed on the transport device so that the workpiece can be transported by the transport device; a camera that acquires first image data that captures an image of the first index and second image data that captures the first index transported by the transport device after the first image data is captured; a position data acquisition unit that acquires first position data that indicates a three-dimensional position with respect to the camera of the first index coordinate system represented by the first index that appears in the first image data, and second position data that indicates the three-dimensional position with respect to the camera of the first index coordinate system represented by the first index that appears in the second image data; a transport direction acquisition unit that determines the transport direction based on the first position data and the second position data; and a coordinate system setting unit that sets the transport device coordinate system in the robot coordinate system based on the transport direction determined by the transport direction acquisition unit.

2. The device described in claim 1, further comprising a second index installed at a known position in the robot coordinate system and representing a second index coordinate system, wherein the camera acquires the first image data capturing the first index and the second index and the second image data capturing the transported first index and the second index, wherein the position data acquisition unit further acquires third position data indicating a three-dimensional position relative to the camera of the second index coordinate system represented by the second index captured in the first image data, and fourth position data indicating a three-dimensional position relative to the camera of the second index coordinate system represented by the second index captured in the second image data, and wherein the transport direction acquisition unit determines the transport direction further based on the third position data and the fourth position data.

3. The device described in claim 2, wherein the transport direction acquisition unit: determines a first positional relationship between the first index coordinate system and the second index coordinate system in the first image data based on the first position data and the third position data; determines a second positional relationship between the first index coordinate system and the second index coordinate system in the second image data based on the second position data and the fourth position data; and determines the transport direction based on the first positional relationship and the second positional relationship.

4. The device described in claim 1, comprising: a second index installed at a known position in the robot coordinate system and representing a second index coordinate system; and a sensor for detecting displacement of the camera, wherein the camera further acquires third image data capturing an image of the second index; the position data acquisition unit further acquires third position data indicating a three-dimensional position relative to the camera of the second index coordinate system represented by the second index captured in the third image data; the sensor detects the displacement of the camera while capturing the first image data, the second image data, and the third image data; and the conveying direction acquisition unit determines the conveying direction further based on the third position data and the displacement detected by the sensor.

5. A device described in any one of claims 2 to 4, wherein the second index includes a pattern or the shape of the robot that represents the three-dimensional position of the second index coordinate system in a camera coordinate system set in the camera that captured the image data.

6. A device according to any one of claims 1 to 5, wherein the first index includes a pattern representing the three-dimensional position of the first index coordinate system in a camera coordinate system set in the camera that captured the image data.

7. A method for setting a transport device coordinate system that defines the transport direction of a transport device that transports a workpiece in a robot coordinate system that is set on a robot that performs work on the workpiece, comprising: installing a first index that represents a first index coordinate system on the transport device so that the workpiece is transported by the transport device; acquiring first image data by capturing an image of the first index with a camera; acquiring second image data by capturing an image of the first index that has been transported by the transport device after capturing the first image data; acquiring first position data that indicates a three-dimensional position relative to the camera of the first index coordinate system represented by the first index that appears in the first image data; and second position data that indicates the three-dimensional position relative to the camera of the first index coordinate system represented by the first index that appears in the second image data; determining the transport direction based on the first position data and the second position data; and setting the transport device coordinate system in the robot coordinate system based on the determined transport direction.

Images