Apparatus and method for setting transport device coordinate system to robot coordinate system

Abstract

It is desirable to more simply set a transport device coordinate system to a robot coordinate system (i.e., a calibration between the robot coordinate system and the transport device coordinate system). The device (60) has: a first indicator (18); a camera (14) that acquires first image data obtained by imaging the first index (18), and second image data obtained by imaging the first index (18) transported by the transport device (16) after the imaging; a position data acquisition unit (52) that acquires first position data indicating a three-dimensional position with respect to the camera in a first index coordinate system indicated by a first index (18) appearing in the first image data, and second position data indicating a three-dimensional position with respect to the camera in a second index coordinate system indicated by a second index (18) appearing in the second image data; the second position data represents the three-dimensional position of a first index coordinate system represented by a first index shown in the second image data relative to the camera; a conveyance direction acquisition unit (54) that obtains a conveyance direction; and a coordinate system setting unit (56) that sets the transport device coordinate system to the robot coordinate system in accordance with the transport direction.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and method for setting a conveyor coordinate system to a robot coordinate system. Background Art

[0002] There is a known device that sets the coordinate system of the conveying device to the robot coordinate system by bringing the fingertips of the robot into contact with the conveying device (for example, Patent Document 1). There is also a known device that performs visual correction by photographing a plurality of marks provided on the peripheral device of the robot with a camera, thereby calibrating the robot coordinate system for the peripheral device (for example, Patent Document 2).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2015-174171

[0006] Patent Document 2: Japanese Patent Application Publication No. 2005-149299 Summary of the invention

[0007] Problems to be solved by the invention

[0008] A technology is desired that can more easily set the carrier coordinate system to the robot coordinate system (ie, calibrate the robot coordinate system and the carrier coordinate system).

[0009] Means for solving problems

[0010] In one embodiment of the present invention, a device is provided that sets a conveying device coordinate system to a robot coordinate system, wherein the robot coordinate system is a coordinate system set in a robot that operates on a workpiece, and the conveying device coordinate system specifies a conveying direction of the conveying device that conveys the workpiece, wherein the device comprises: a first indicator that represents a first indicator coordinate system and is disposed on the conveying device so as to be conveyed by the conveying device; and a camera that acquires first image data and second image data, wherein the first image data is data obtained by photographing the first indicator, and the second image data is data obtained by photographing the first indicator after being conveyed by the conveying device after the first image data is photographed.

[0011] In addition, the device also has: a position data acquisition unit, which acquires first position data and second position data, wherein the first position data represents the three-dimensional position of a first indicator coordinate system represented by a first indicator appearing in the first image data relative to the camera, and the second position data represents the three-dimensional position of a first indicator coordinate system represented by the first indicator appearing in the second image data relative to the camera; a transport direction acquisition unit, which calculates the transport direction based on the first position data and the second position data; and a coordinate system setting unit, which sets the transport device coordinate system to the robot coordinate system based on the transport direction calculated by the transport direction acquisition unit.

[0012] In other embodiments of the present disclosure, a method is provided for setting a conveying device coordinate system to a robot coordinate system, wherein the robot coordinate system is a coordinate system set in a robot that operates on a workpiece, and the conveying device coordinate system specifies a conveying direction of the conveying device that conveys the workpiece, wherein in the method: a first indicator representing a first indicator coordinate system is set on the conveying device so that the conveying device conveys the workpiece; the first indicator is photographed by a camera to obtain first image data, and after photographing the first image data, the first indicator after being transported by the conveying device is photographed by the camera to obtain second image data.

[0013] In addition, in this method, first position data and second position data are obtained, wherein the first position data represents the three-dimensional position of a first indicator coordinate system represented by a first indicator appearing in the first image data relative to the camera, and the second position data represents the three-dimensional position of a first indicator coordinate system represented by the first indicator appearing in the second image data relative to the camera; based on the first position data and the second position data, a conveying direction is determined; and based on the determined conveying direction, the conveying device coordinate system is set to the robot coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a schematic perspective view of a robot system according to an embodiment.

[0015] Figure 2 yes Figure 1 Block diagram of the robot system shown.

[0016] Figure 3 It means in Figure 1 A flowchart of an example of a method for setting a transport device coordinate system to a robot coordinate system in the robot system shown.

[0017] Figure 4 Indicated in Figure 3 An example of image data captured in step S1 in .

[0018] Figure 5 Indicated in Figure 3In step S3 in the above, the state of the first indicator is transferred.

[0019] Figure 6 Indicated in Figure 3 An example of image data captured in step S4 in .

[0020] Figure 7 Indicated in Figure 1 In the robot system shown, the carrier coordinate system is set to the robot coordinate system.

[0021] Figure 8 This is a schematic perspective view of a robot system according to another embodiment.

[0022] Fig. 9 yes Figure 8 Block diagram of the robot system shown.

[0023] Fig.10 express Figure 8 The back side of the terminal device is shown.

[0024] Fig.11 It means in Figure 8 A flowchart of an example of a method of setting a transport device coordinate system to a robot coordinate system in the robot system shown.

[0025] Fig.12 Indicated in Fig.11 An example of image data captured in step S11 in .

[0026] Fig.13 Indicated in Fig.11 In step S13 in the above, the state of the first indicator is conveyed.

[0027] Fig.14 Indicated in Fig.11 An example of image data captured in step S14 in .

[0028] Fig.15 Indicated in Figure 8 In the robot system shown, the carrier coordinate system is set to the robot coordinate system.

[0029] Fig.16 It is a schematic perspective view of a robot system according to still another embodiment.

[0030] Fig.17 yes Fig.16 Block diagram of the robot system shown.

[0031] Fig.18 It means in Fig.16 A flowchart of an example of a method for setting a transport device coordinate system to a robot coordinate system in the robot system shown.

[0032] Fig.19 Indicated in Fig.18 An example of image data captured in step S22 in .

[0033] Fig. 20 Indicated in Fig.18 In step S24 in the above process, the state of the first indicator is transferred.

[0034] Fig.21 Indicated in Fig.18 An example of image data captured in step S25 in .

[0035] Fig. 22 Indicated in Fig.18 An example of image data captured in step S27 in .

[0036] Fig.23 Indicated in Fig.16 In the robot system shown, the carrier coordinate system is set to the robot coordinate system.

[0037] Fig.24 It means in Figure 1 A flowchart of another example of a method of setting the transport device coordinate system to the robot coordinate system in the robot system shown. DETAILED DESCRIPTION

[0038] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in the various embodiments described below, the same elements are marked with the same reference numerals, and repeated descriptions are omitted. Figure 1 as well as Figure 2 , a robot system 10 according to an embodiment will be described. The robot system 10 includes a robot 12, a camera 14, a conveying device 16, a first indicator 18, and a control device 20.

[0039] In this embodiment, the robot 12 is a vertical multi-joint robot that performs a predetermined operation (e.g., workpiece processing, welding, cutting, laser processing, etc.) on a workpiece (not shown). Specifically, the robot 12 includes a robot base 22, a rotary body 24, a lower arm 26, an upper arm 28, and a wrist 30.

[0040] The robot base 22 is fixed to the floor of the work unit or the automated guided vehicle (AGV). The rotary body 24 is provided on the robot base 22 so as to be rotatable about a vertical axis. The lower arm 26 is provided on the rotary body 24 so that its base end can rotate about a horizontal axis, and the base end of the upper arm 28 is provided on the front end of the lower arm 26 so as to be rotatable.

[0041] The wrist part 30 includes: a wrist base 30a which is provided at the front end of the upper arm part 28 so as to be rotatable around two axes orthogonal to each other; and a wrist flange 30b which is provided at the wrist base 30a so as to be rotatable around the wrist axis A1. An end effector (a robot manipulator, a welding torch, a cutting tool, a laser processing head, etc.) for operating a workpiece is detachably mounted on the wrist flange 30b.

[0042] A plurality of servo motors 32 ( Figure 2 ). These servo motors 32 rotate the movable components of the robot 12 (i.e., the rotating body 24, the lower arm 26, the upper arm 28, the wrist 30, and the wrist flange 30b) according to the command from the control device 20, thereby moving the end effector (not shown) mounted on the wrist flange 30b to an arbitrary position.

[0043] In this embodiment, the camera 14 is installed on the wrist flange 30b in a detachable manner instead of the end effector. The camera 14 is a two-dimensional camera having an imaging sensor (CMOS, CCD, etc.) and an optical lens (collimating lens, focusing lens, etc.) for guiding the subject image to the imaging sensor, and images the subject according to the command from the control device 20. The camera 14 takes in the subject image along the optical axis A2, and forms the subject image on the imaging sensor through the optical lens, thereby obtaining the image data ID of the subject image. In this embodiment, the camera 14 is moved to an arbitrary position by the robot 12.

[0044] The conveying device 16 is arranged around the robot 12 and conveys the workpiece in the conveying direction D1. For example, the conveying device 16 is a belt conveyor, etc., and has a conveying surface 34 on which the workpiece is placed and a driving mechanism 36 that moves the conveying surface 34 in the conveying direction D1. The conveying surface 34 can be defined by the outer surface of a timing belt.

[0045] In addition, the drive mechanism 36 may include a servo motor (not shown) that drives the timing belt (i.e., the conveying surface 34) according to a command from the control device 20. The conveying device 16 is configured so that at least a portion of its conveying surface 34 is located within the operating range of the robot 12 (i.e., within the range in which the end effector can be positioned).

[0046] The control device 20 controls the robot 12, the camera 14, and the conveying device 16. Specifically, Figure 2As shown, the control device 20 is a computer having a processor 40, a memory 42, and an I / O interface 44. The processor 40 has a CPU or a GPU, etc., and is connected to the memory 42 and the I / O interface 44 via a bus 46 so as to be communicable, communicates with these components, and performs calculation processing for realizing a function of setting a conveyor coordinate system described later.

[0047] The memory 42 has a RAM or a ROM, etc., and stores various data temporarily or permanently. The memory 42 can be composed of a computer-readable storage medium such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium. The I / O interface 44 has, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices by wire or wirelessly under instructions from the processor 40. In this embodiment, each servo motor 32, camera 14, and conveying device 16 (specifically, drive mechanism 36) of the robot 12 can be connected to the I / O interface 44 in a communicative manner.

[0048] In addition, the control device 20 is provided with a display device 48 and an input device 50. The display device 48 and the input device 50 are connected to the I / O interface 44 so as to be communicable. The display device 48 has a liquid crystal display or an organic EL display, etc., and can display various data visually under the instruction from the processor 40.

[0049] The input device 50 includes a push button, a switch, a keyboard, a mouse, or a touch panel, and receives data input from an operator. In addition, the display device 48 and the input device 50 may be integrally assembled in the housing of the control device 20, or may be externally placed in the housing as a computer (PC, etc.) separate from the housing of the control device 20.

[0050] like Figure 1 As shown, a robot coordinate system C1 is set for the robot 12. The robot coordinate system C1 is a coordinate system for automatically controlling the actions of each movable element of the robot 12. In the present embodiment, the robot coordinate system C1 is fixed to the robot base 22 in such a manner that its origin is arranged at the center of the robot base 22 and its z-axis is parallel to (specifically, coincident with) the rotation axis of the rotating body 24.

[0051] On the other hand, the camera coordinate system C2 is set for the camera 14. In the present embodiment, the camera coordinate system C2 is set for the camera 14 so that its origin is arranged at the center of the imaging sensor of the camera 14 and its z-axis is parallel to (specifically, coincident with) the optical axis A2. In addition, the camera 14 can be mounted on the wrist flange 30b so that its optical axis A2 is parallel to (or coincident with) the wrist axis A1.

[0052] The camera coordinate system C2 defines the coordinates of each pixel of the image data ID (or the imaging sensor) captured by the camera 14. Here, in the present embodiment, the position of the camera 14 in the robot coordinate system C1 (in other words, the position of the wrist flange 30b) is known. Therefore, the positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 of the camera 14 disposed at an arbitrary position PSn of the robot coordinate system C1 is 1_2_n is known.

[0053] Therefore, the coordinates Q (X, Y, Z, W, P, R) of the camera coordinate system C2 in the robot coordinate system C1 are known, and the coordinates of the robot coordinate system C1 and the camera coordinate system C2 can be transformed via the known transformation matrix M 1_2_n The control device 20 holds the transformation matrix M 1_2_n The parameters of the robot coordinate system C1 and the coordinates of the camera coordinate system C2 are used to represent the positional relationship R 1_2_n The data is stored in the memory 42.

[0054] Here, when the robot 12 cooperates with the conveying device 16 to perform work on the workpiece, the conveying device coordinate system needs to be set to the robot coordinate system C1. The conveying device coordinate system is set for the conveying device 16, and the conveying device coordinate system specifies the position of the conveying surface 34 and the conveying direction D1. The first index 18 is used to set the conveying device coordinate system to the robot coordinate system C1 (in other words, the positional relationship between the robot coordinate system C1 and the conveying device coordinate system is set to be known).

[0055] Specifically, the first indicator 18 is composed of a flat paper material or a plate material, etc., and a pattern 18a is displayed on its surface in a visually recognizable manner. The pattern 18a, for example, includes a figure or a dot pattern composed of a plurality of lines or curves. The first indicator 18 is configured to represent the first indicator coordinate system C3 by its pattern 18a, and is arranged on the conveying surface 34 of the conveying device 16 so as to be conveyed by the conveying device 16. In addition, the first indicator 18 can also be fixed to the conveying surface 34 using fasteners (bolts, etc.) or clamps in a manner that it cannot move relative to the conveying surface 34. Alternatively, the pattern 18a of the first indicator 18 can also be directly formed on the conveying surface 34 by printing or engraving.

[0056] Next, refer to Figure 3 , a method for setting the transport device coordinate system to the robot coordinate system C1 in the robot system 10 is described. Figure 3 Before the process, the operator will first indicate 18 as Figure 1The first indicator 18 is set on the conveying surface 34 of the conveying device 16 as shown. At this time, the operator can also configure the first indicator 18 within the operating range of the robot 12. After setting the first indicator 18, the operator operates the input device 50 to provide the processor 40 with a coordinate system setting instruction. When the processor 40 receives the coordinate system setting instruction, it starts Figure 3 The process shown.

[0057] In step S1, the processor 40 uses the camera 14 to capture the first indicator 18. Specifically, the processor 40 operates the robot 12 to move the camera 14 to the capturing position PS1 where the first indicator 18 can be within the field of view of the camera 14. Then, the processor 40 operates the camera 14 to capture the first indicator 18.

[0058] Figure 4 An example of the image data ID1 captured by the camera 14 in step S1 is shown. Each pixel of the image data ID1 is represented as a coordinate of the camera coordinate system C2, and the first indicator 18 appears in the image data ID1. The camera 14 acquires the image data ID1 (first image data) obtained by capturing the first indicator 18 in this way, and supplies it to the processor 40.

[0059] In step S2, the processor 40 obtains position data PD1 (first position data) indicating the three-dimensional position of the first indicator coordinate system C3 represented by the first indicator 18 shown in the image data ID1 relative to the camera 14. Here, the pattern 18a of the first indicator 18 shown in the image data ID1 is configured as a three-dimensional position indicating the first indicator coordinate system C3 in the camera coordinate system C2 set for the camera 14 when the image data ID1 is captured.

[0060] The processor 40 executes the image analysis program PG1 (or image analysis application) for reading the first indicator coordinate system C3 from the pattern 18a, analyzes the pattern 18a of the first indicator 18 appearing in the image data ID1, and determines the first indicator coordinate system C3 represented by the first indicator 18 in the image data ID1. Then, the processor 40 obtains the coordinates Q1 (X1, Y1, Z1, W1, P1, R1) of the determined first indicator coordinate system C3 in the camera coordinate system C2.

[0061] The coordinates (X1, Y1, Z1) in the coordinate Q1 represent Figure 4 The origin position of the first index coordinate system C3 in the camera coordinate system C2, the coordinates (W1, P1, R1) represent the first index coordinate system C3 relative to Figure 4 The posture (i.e., the direction of each axis) of the camera coordinate system C2 (so-called yaw, pitch, roll).

[0062] Thus, coordinate Q1 represents Figure 4 The three-dimensional position of the first indicator coordinate system C3 in the camera coordinate system C2, in other words, the three-dimensional position of the first indicator coordinate system C3 relative to the camera 14 when the image data ID1 was captured. That is, in the present application, "position" sometimes means position and posture. In this step S2, the processor 40 obtains the coordinate Q1 as the position data PD1. Therefore, the processor 40 serves as the position data acquisition unit 52 ( Figure 2 ) to perform its function.

[0063] In step S3, the processor 40 transports the first index 18 by the transport device 16. Specifically, the processor 40 operates the drive mechanism 36 of the transport device 16 to transport the transport surface 34 by a predetermined distance δ in the transport direction D1. Figure 5 As shown in FIG. 1 , the first indicator 18 is transported by a distance δ from the position when the image data ID1 was captured in the transport direction D1. Figure 5 In FIG. 1 , for easy understanding, the position of the first indicator 18 when the image data ID1 is captured is indicated by a dotted line B, and a first indicator coordinate system C3 indicated by the first indicator 18 is also described.

[0064] In step S4, the processor 40 photographs the first indicator 18 through the camera 14. Here, in the present embodiment, in step S4, the processor 40 photographs the first indicator 18 through the camera 14 while placing the camera 14 at the same photographing position PS1 as in step S1. That is, the photographing position PS1 is determined in the robot coordinate system C1 so that the first indicator 18 before and after the conveyance in step S3 can be contained within the field of view of the camera 14.

[0065] Figure 6 An example of the image data ID2 photographed by the camera 14 in step S4 is shown. The first indicator 18 after the transport in step S3 appears in the image data ID2. The camera 14 acquires the image data ID2 (second image data) obtained by photographing the first indicator 18 after the transport, and supplies it to the processor 40.

[0066] In step S5, the processor 40 functions as the position data acquisition unit 52, and acquires the position data PD2 (second position data), which indicates the three-dimensional position of the first indicator coordinate system C3 indicated by the first indicator 18 shown in the image data ID2 relative to the camera 14. Specifically, the processor 40 executes the image analysis program PG1 as in the above-mentioned step S2, and acquires the coordinates Q2 (X2, Y2, Z2, W2, P2, R2) of the first indicator coordinate system C3 indicated by the first indicator 18 shown in the image data ID2 in the camera coordinate system C2. The coordinates Q2 indicate the three-dimensional position of the first indicator coordinate system C3 relative to the camera 14 when the image data ID2 was captured. In this way, the processor 40 acquires the coordinates Q2 as the position data PD2 in step S5.

[0067] In step S6, the processor 40 obtains the conveying direction D1 based on the position data PD1 and PD2. Specifically, the processor 40 uses the known positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 disposed at the imaging position PS1. 1_2_1 (Specifically, the transformation matrix M 1_2_1 ), the coordinates Q1 (X1, Y1, Z1, W1, P1, R1) of the camera coordinate system C2 obtained as the position data PD1 in the above step S2 are transformed into the coordinates Q3 (X3, Y3, Z3, W3, P3, R3) of the robot coordinate system C1. The coordinates Q3 represent the three-dimensional position (specifically, the three-dimensional position and posture) of the first indicator coordinate system C3 represented by the first indicator C1 in the robot coordinate system C1 when the image data ID1 is captured (i.e., when step S1 is executed).

[0068] Likewise, processor 40 uses positional relationship R 1_2_1 , the coordinates Q2 (X2, Y2, Z2, W2, P2, R2) of the camera coordinate system C2 obtained as the position data PD2 in the above step S5 are transformed into the coordinates Q4 (X4, Y4, Z4, W4, P4, R4) of the robot coordinate system C1. The coordinates Q4 represent the three-dimensional position of the first index coordinate system C3 represented by the first index C1 in the robot coordinate system C1 when the image data ID2 is captured (i.e., when step S4 is executed).

[0069] Then, the processor 40 calculates the vector VT from the coordinates (X3, Y3, Z3) in the coordinate Q3 to the coordinates (X4, Y4, Z4) in the coordinate Q4. 3_4 The vector VT 3_4 The vector position of the conveying direction D1 in the robot coordinate system C1 is consistent with the axis line A3 connecting the coordinates (X3, Y3, Z3) and the coordinates (X4, Y4, Z4).

[0070] The vector VT 3_4 (or axis A3) represents the trajectory of the origin of the first index coordinate system C3 in the robot coordinate system C1 after the first index coordinate system C3 is moved in the conveying direction D1 by step S3, and can be regarded as being arranged on the conveying surface 34 (or, at a position away from the conveying surface 34 by a known distance upward). That is, the vector VT 3_4 (or the axis line A3 ) indicates the conveying direction D1 in the robot coordinate system C1 , and serves as position data indicating the conveying surface 34 in the robot coordinate system C1 .

[0071] In this way, the processor 40 obtains the vector VT according to the position data PD1 and PD2. 3_4 (or axis A3) as the conveying direction D1 in the robot coordinate system C1. Therefore, in the present embodiment, the processor 40 serves as a conveying direction acquisition unit 54 ( Figure 2 ) to perform its function.

[0072] In step S7, the processor 40 sets the conveying device coordinate system C4 to the robot coordinate system C1 ( Figure 7 Specifically, the processor 40 converts the transport direction D1 (specifically, the vector VT) obtained in step S6 into 3_4 ) is determined as the positive direction of the y-axis of the conveying device coordinate system C4.

[0073] In addition, the processor 40 determines the z-axis direction of the conveyor coordinate system C4 as a direction parallel to the yz plane of the robot coordinate system C1 and orthogonal to the y-axis of the conveyor coordinate system C4 determined as described above. In addition, the processor 40 determines the x-axis direction of the conveyor coordinate system C4 as a direction orthogonal to the y-axis and z-axis of the conveyor coordinate system C4.

[0074] At this time, the processor 40 may randomly determine the positive z-axis direction and the positive x-axis direction of the conveying device coordinate system C4. Alternatively, the operator may operate the input device 50 to input setting information of the positive z-axis direction or the positive x-axis direction in advance, and the processor 40 may determine the positive z-axis direction or the positive x-axis direction of the conveying device coordinate system C4 according to the setting information.

[0075] The setting information may include information that determines the positive z-axis direction of the transport device coordinate system C4 to be a direction close to the positive z-axis direction of the robot coordinate system C1 (i.e., vertically upward) (i.e., a direction in which the inner product of the two becomes larger). Alternatively, the setting information may include information that determines the positive x-axis direction of the transport device coordinate system C4 to be a direction opposite to the robot coordinate system C1.

[0076] In addition, the processor 40 determines the origin of the conveying device coordinate system C4 in the robot coordinate system C1. As an example, the processor 40 determines the coordinates Q3 (X1, Y1, Z1) or the coordinates Q4 (X4, Y4, Z4) as the origin of the conveying device coordinate system C4. As another example, the processor 40 may determine the origin of the conveying device coordinate system C4 as a predetermined position on the axis A3 (for example, the upstream end or downstream end of the conveying surface 34, or the center, upstream end or downstream end of the range of motion of the robot 12) when the axis A3 is obtained in the above-mentioned step S6.

[0077] In addition, the coordinates Q3 above represent the position and posture (directions of each axis) of the first index coordinate system C3 in the robot coordinate system C1 before the transport in step S3. In step S7, the processor 40 can correct the coordinates (W3, P3, R3) representing the posture in the coordinates Q3 so that the positive direction of the y axis of the first index coordinate system C3 before the transport is aligned with the transport direction D1 (vector VT 3_4 ), thereby obtaining the coordinates (W3c, P3c, R3c) of the new posture. Furthermore, the processor 40 may determine the coordinates (W3c, P3c, R3c) of the new posture as the directions of the axes of the conveying device coordinate system C4.

[0078] Similarly, the processor 40 can correct the coordinates (W4, P4, R4) representing the position of the first index coordinate system C3 in the robot coordinate system C1 after the transport in step S3, and the coordinates Q4 representing the posture, thereby obtaining the coordinates (W4c, P4c, R4c) of the new posture. In addition, the processor 40 can determine the coordinates (W4c, P4c, R4c) of the new posture as the directions of the axes of the transport device coordinate system C4.

[0079] In this way, Figure 7 As shown, the conveying device coordinate system C4 is set to the robot coordinate system C1. The conveying device coordinate system C4 is fixed to the robot coordinate system C1, and its y-axis positive direction represents the conveying direction D1 in the robot coordinate system C1 with high accuracy. In addition, the xy plane (or origin) of the conveying device coordinate system C4 represents the position of the conveying surface 34 in the robot coordinate system C1. In this way, in the present embodiment, the processor 40 serves as a coordinate system setting unit 56 ( Figure 2 ) to perform its function.

[0080] As described above, the processor 40 functions as the position data acquisition unit 52, the transport direction acquisition unit 54, and the coordinate system setting unit 56, and sets the transport device coordinate system C4 to the robot coordinate system C1 based on the image data ID1 and ID2 obtained by the camera 14 photographing the first indicator 18. Therefore, the camera 14, the first indicator 18, the position data acquisition unit 52, the transport direction acquisition unit 54, and the coordinate system setting unit 56 constitute a device 60 ( Figure 2 ).

[0081] In the device 60, the first indicator 18 indicating the first indicator coordinate system C3 is set on the conveying device 16 (specifically, the conveying surface 34) so ​​as to be conveyed by the conveying device 16. In addition, the camera acquires the first image data ID1 ( Figure 4 ), and the second image data ID2 ( Figure 6 )(Steps S1, S4).

[0082] In addition, the position data acquisition unit 52 acquires the first position data PD1 (specifically, coordinates Q1) indicating the three-dimensional position of the first indicator coordinate system C4 indicated by the first indicator 18 appearing in the first image data ID1 relative to the camera 14 (specifically, the camera coordinate system C2) (step S2). Next, the position data acquisition unit 52 acquires the second position data PD2 (specifically, coordinates Q2) indicating the three-dimensional position of the first indicator coordinate system C3 indicated by the first indicator 18 appearing in the second image data ID2 relative to the camera 14 (camera coordinate system C2) (step S5).

[0083] Then, the transport direction acquisition unit 54 obtains the transport direction D1 (specifically, the vector VT 3_4 Or the axis line A3) (step S6). The coordinate system setting unit 56 sets the transport device coordinate system C4 to the robot coordinate system C1 based on the transport direction D1 obtained by the transport direction acquisition unit 54 (step S7).

[0084] According to this configuration, the first index 18 conveyed by the conveying device 16 is photographed by the camera 14, without bringing the fingertips (wrist flange 30b or end effector) of the robot 12 into contact with the conveying device 16 or performing visual correction by photographing a plurality of marks provided on the conveying device 16 by the camera 14, thereby enabling the conveying device coordinate system C4 to be set to the robot coordinate system C1. As a result, the conveying device coordinate system C4 can be set (i.e., calibrated) with respect to the robot coordinate system C1 more easily.

[0085] In the device 60, the first indicator 18 includes a pattern 18a indicating a three-dimensional position of the first indicator coordinate system C3 in the camera coordinate system C2 set for the camera 14 that captured the image data ID. According to this configuration, the first indicator coordinate system C3 can be more effectively indicated in the camera coordinate system C2 by the pattern 18a of the first indicator 18 that appears in the image data ID.

[0086] In addition, in the present embodiment, the case where the first indicator 18 is photographed in step S4 with the camera 14 arranged at the same photographing position PS1 as in step S1 is described. However, the present invention is not limited thereto, and in step S4, the processor 40 may move the camera 14 from the photographing position PS1 to the photographing position PS2, and obtain the image data ID2' obtained by photographing the first indicator 18 at the photographing position PS2.

[0087] In this case, the processor 40 functions as the position data acquisition unit 52 in step S5, and acquires the coordinates Q2' (X2', Y2', Z2', W2', P2', R2') of the first indicator coordinate system C3 specified in the image data ID2' in the camera coordinate system C2. Then, in step S7, the processor 40 uses the known positional relationship R between the robot coordinate system C1 and the camera coordinate system C2 of the camera 14 disposed at the shooting position PS2. 1_2_2 (Specifically, the transformation matrix M 1_2_2 ), transform the coordinate Q2' into the coordinate Q4 (X4, Y4, Z4, W4, P4, R4) of the robot coordinate system C1.

[0088] Next, refer to Figure 8 as well as Fig. 9 , a robot system 70 of another embodiment is described. The robot system 70 is different from the above-mentioned robot system 10 in that it further includes a terminal device 72 and a second indicator 74. The terminal device 72 is, for example, a portable computer such as a smartphone, a tablet terminal device, or a notebook PC, and can be carried by the operator.

[0089] Specifically, if Fig. 9As shown, the terminal device 72 has: a processor 80, a memory 82, an I / O interface 84, a display device 86, an input device 88, and a camera 14. Figure 8 As shown, the display device 86 and the input device 88 are integrally arranged on the surface of the terminal device 72. Fig.10 As shown, the camera 14 is integrally provided on the back surface of the terminal device 72. A camera coordinate system C2 is set for the camera 14 in the same manner as in the above-mentioned embodiment.

[0090] The processor 80 is connected to the memory 82, the I / O interface 84, the display device 86, the input device 88, and the camera 14 via the bus 90 so as to be communicable, communicates with these components, and performs arithmetic processing for realizing a function of setting a conveyor coordinate system described later. The I / O interface 84 is connected to the I / O interface 44 of the control device 20 so as to be communicable by wire or wirelessly.

[0091] In addition, the structures of the processor 80, memory 82, I / O interface 84, display device 86 and input device 88 are respectively the same as the above-mentioned processor 40, memory 42, I / O interface 44, display device 48 and input device 50, so repeated descriptions are omitted.

[0092] The second index 74 is used together with the first index 18 to set the conveyor coordinate system C4 to the robot coordinate system C1. Figure 8 As shown, the second indicator 74 is made of a flat paper material or plate material, similar to the above-mentioned first indicator 18, and a pattern 74a is displayed on the surface thereof in a visually recognizable manner.

[0093] The pattern 74a includes, for example, a graphic or dot pattern composed of a plurality of lines or curves. The second indicator 74 is configured to be fixed relative to the robot 12 (e.g., the robot base 22), and the second indicator coordinate system C5 is represented by its pattern 74a. Here, the second indicator 74 is positioned relative to the robot 12 so that the positional relationship R between the second indicator coordinate system C5 represented by the pattern 74a and the robot coordinate system C1 is 1_5 Become known.

[0094] Therefore, the coordinates of the robot coordinate system C1 and the coordinates of the second index coordinate system C5 can be transformed via the known transformation matrix M 1_5 In this embodiment, the control device 20 holds the transformation matrix M 1_5 The parameters of and the coordinates of the second index coordinate system C5 in the robot coordinate system C1 are used as the representation of the positional relationship R 1_5 The data is stored in the memory 42. Figure 8In FIG. 1 , for easy understanding, the robot coordinate system C1 is shown by a dotted line.

[0095] Next, refer to Fig.11 , a method for setting the conveyor coordinate system C4 to the robot coordinate system C1 in the robot system 70 will be described. As in the above-mentioned embodiment, the operator starts Fig.11 Before the process, the first indicator 18 is set on the conveying surface 34 of the conveying device 16.

[0096] In step S11, the processor 80 of the terminal device 72 photographs the first indicator 18 and the second indicator 74 through the camera 14. Specifically, the operator places the terminal device 72 at a photographing position PS3 that can accommodate the first indicator and the second indicator 74 within the field of view of the camera 14 of the terminal device 72. At this time, the processor 80 can display the image data ID continuously photographed by the camera 14 on the display device 86 of the terminal device 72 in real time (so-called live view display).

[0097] Then, the operator operates the input device 88 of the terminal device 72 disposed at the imaging position PS3 to provide an imaging command to the processor 80. In response to the imaging command, the processor 80 operates the camera 14 to image the first indicator 74 and the second indicator 74. Fig.12 An example of image data ID3 captured by the camera 14 in step S11 is shown.

[0098] The camera 14 obtains the image data ID3 (first image data) obtained by photographing the first index 18 and the second index, and supplies it to the processor 80. When photographing the image data ID3, the operator may hold the terminal device 72 by hand, or may fix the terminal device 72 at a fixed point using a fixture (tripod, bracket, etc.).

[0099] In step S12, the processor 80 functions as the position data acquisition unit 52 ( Fig. 9 ) functions to obtain position data PD3 (first position data), which indicates the three-dimensional position of the first indicator coordinate system C3 represented by the first indicator 18 appearing in the image data ID3 relative to the camera 14. Specifically, the processor 80 executes the above-mentioned image analysis program PG1 to analyze the pattern 18a of the first indicator 18 appearing in the image data ID3, and determines the first indicator coordinate system C3 represented by the first indicator 18 in the image data ID3. The image analysis program PG1 can be stored in the memory 82 in advance.

[0100] Then, the processor 80 obtains the coordinates Q5 (X5, Y5, Z5, W5, P5, R5) of the determined first indicator coordinate system C3 in the camera coordinate system C2. The coordinates Q5 represent the three-dimensional position of the first indicator coordinate system C3 relative to the camera 14 when the image data ID3 was captured. In this step S12, the processor 80 obtains the coordinates Q5 as the position data PD3.

[0101] In addition, the processor 80 functions as the position data acquisition unit 52, and acquires the position data PD4 (third position data), which indicates the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 appearing in the image data ID3 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1, analyzes the pattern 74a of the second indicator 74 appearing in the image data ID3, and determines the second indicator coordinate system C5 indicated by the second indicator 74 in the image data ID3.

[0102] Then, the processor 80 obtains the coordinates Q6 (X6, Y6, Z6, W6, P6, R6) of the determined second indicator coordinate system C5 in the camera coordinate system C2. The coordinates Q6 represent the three-dimensional position of the second indicator coordinate system C5 relative to the camera 14 when the image data ID3 was captured. In this step S12, the processor 80 obtains the coordinates Q6 as the position data PD4. Thus, in this step S12, the processor 80 obtains the position data PD4 of the second indicator coordinate system C5 together with the position data PD3 of the first indicator coordinate system C3.

[0103] When step S12 is completed, processor 80 sends completion signal SG1 to control device 20. Processor 40 of control device 20 executes step S13 according to completion signal SG1. Specifically, in step S13, processor 40 operates conveying device 16 to convey first index 18 in conveying direction D1 by distance δ, similar to step S3 described above.

[0104] Furthermore, when the processor 40 receives the completion signal SG1, it may generate a notification signal NS1 for notifying that the first indicator 18 should be transported by the transport device 16, and output the notification signal to the display device 48 (or a speaker provided in the control device 20). When the operator recognizes the notification signal NS1, the operator may execute step S13 to manually operate the transport device 16 to transport the first indicator 18. Fig.13 Indicates the state of completing step S13. Fig.13 In FIG. 1 , for easy understanding, the position of the first indicator 18 when the image data ID3 is captured is indicated by a dotted line B, and a first indicator coordinate system C3 indicated by the first indicator 18 is also described.

[0105] After step S13 is completed, the processor 40 sends a completion signal SG2 to the terminal device 72. When receiving the completion signal SG2, the processor 80 of the terminal device 72 executes step S14. At this time, the processor 80 may also generate a notification signal NS2 for notifying that the first indicator 18 and the second indicator 74 should be photographed again by the camera 14, and output it to the display device 86 (or a speaker provided in the terminal device 72).

[0106] In step S14, the processor 80 of the terminal device 72 photographs the first indicator 18 and the second indicator 74 through the camera 14. Specifically, if the operator recognizes the notification signal NS2 output to the display device 86 (or the speaker of the terminal device 72), the terminal device 72 is configured to a photographing position PS4 that can accommodate the first indicator and the second indicator 74 in the field of view of the camera 14. The photographing position PS4 may be different from the photographing position PS3 of the above-mentioned step S11, or may be the same. In addition, at this time, the processor 80 may also perform the above-mentioned live view display.

[0107] Then, the operator operates the input device 88 of the terminal device 72 to operate the camera 14 to capture images of the first indicator 74 and the second indicator 74 . Fig.14 An example of the image data ID4 captured by the camera 14 in step S14 is shown. In the image data ID4, the second indicator 74 appears together with the first indicator 18 after being transported in step S13. The camera 14 acquires the image data ID4 (second image data) obtained by capturing the first indicator 18 and the second indicator 74 being transported, and supplies it to the processor 80.

[0108] In addition, when the terminal device 72 is fixed at the same shooting position PS3=PS4 by the fixing means in steps S11 and S14, the processor 80 can automatically execute step S14 upon receiving the above-mentioned completion signal SG2, and automatically shoot the image data ID4 by the camera 14.

[0109] In step S15, the processor 80 functions as the position data acquisition unit 52 ( Fig. 9 ) functions, and similarly to the above-mentioned step S12, obtains position data PD5 (second position data), wherein the position data PD5 represents the three-dimensional position of the first indicator coordinate system C3 represented by the first indicator 18 appearing in the image data ID4 relative to the camera 14.

[0110] Specifically, the processor 80 executes the image analysis program PG1 to obtain the coordinates Q7 (X7, Y7, Z7, W7, P7, R7) of the first indicator coordinate system C3 represented by the first indicator 18 shown in the image data ID4 in the camera coordinate system C2. The coordinates Q7 represent the three-dimensional position of the first indicator coordinate system C3 relative to the camera 14 when the image data ID4 was captured. In this way, the processor 80 obtains the coordinates Q7 as the position data PD5.

[0111] In addition, the processor 80 functions as the position data acquisition unit 52, and acquires the position data PD6 (fourth position data) indicating the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 appearing in the image data ID4 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1, and acquires the coordinates Q8 (X8, Y8, Z8, W8, P8, R8) of the second indicator coordinate system C5 indicated by the second indicator 74 appearing in the image data ID4 in the camera coordinate system C2.

[0112] The coordinates Q8 represent the three-dimensional position of the second indicator coordinate system C5 relative to the camera 14 when the image data ID4 was captured. The processor 80 acquires the coordinates Q8 as the position data PD6. Thus, in step S15, the processor 80 acquires the position data PD6 of the second indicator coordinate system C5 together with the position data PD5 of the first indicator coordinate system C3.

[0113] In step S16, the processor 80 functions as the transport direction acquisition unit 54 ( Fig. 9 ) functions to obtain the conveying direction D1 based on the position data PD3, PD4, PD5, and PD6. Specifically, the processor 80 obtains the image data ID3 ( Fig.12 ) and the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 represented by the camera coordinate system C2 3_5_1 (First position relationship).

[0114] Here, the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 in the three-dimensional space defined by the camera coordinate system C2 of the image data ID3 is 3_5_1 The processor 80 obtains the position data PD3 and PD4 based on the position data PD3 and PD4. Fig.12 The coordinates Q9 (X9, Y9, Z9, W9, P9, R9) of the first index coordinate system C3 in the second index coordinate system C5 shown in the figure are used as the positional relationship R3_5_1 data.

[0115] Similarly, the processor 80 obtains the image data ID4 ( Fig.14 ) and the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 represented by the camera coordinate system C2 3_5_2 (Second Positional Relationship) Specifically, the processor 80 obtains the second positional relationship based on the position data PD5 and PD6. Fig.14 The coordinates Q10 (X10, Y10, Z10, W10, P10, R10) of the first index coordinate system C3 in the second index coordinate system C5 shown in FIG. 5 are used as the positional relationship R 3_5_2 data.

[0116] Then, the processor 80 calculates the vector VT from the coordinates (X9, Y9, Z9) in the coordinate Q9 to the coordinates (X10, Y10, Z10) in the coordinate Q10. 9_10 The vector VT 9_10 The axis line A3 connecting the coordinates (X9, Y9, Z9) and the coordinates (X10, Y10, Z10) corresponds to the position of the vector (in other words, the axis line A3) in the conveyance direction D1 in the second index coordinate system C5.

[0117] The vector VT 9_10 (or axis A3) is represented as a coordinate (or function) of the second index coordinate system C5, which represents the trajectory of the origin of the first index coordinate system C3 moved in the conveying direction D1 in step S13 in the second index coordinate system C5, and can be regarded as being arranged on the conveying surface 34. In this way, the processor 80 obtains the vector VT in the second index coordinate system C5 9_10 (or axis A3) is used as the conveying direction D1.

[0118] After step S16 is completed, the processor 80 converts the data of the conveying direction D1 obtained in step S15 (specifically, the vector VT in the second index coordinate system C5) into 9_10 The data of the coordinates or functions of the axis A3 or the axis A4 are supplied to the control device 20. When the processor 40 of the control device 20 receives the data of the conveying direction D1, it executes step S17.

[0119] In step S17, the processor 40 functions as the coordinate system setting unit 56 ( Fig. 9 ) functions to set the conveying device coordinate system C4 to the robot coordinate system C1 ( Fig.15Specifically, the processor 40 uses the known positional relationship R between the second index coordinate system C5 and the robot coordinate system C1. 1_5 (Specifically, the transformation matrix M 1_5 ), the transport direction D1 (for example, vector VT 9_10 ), transformed to the robot coordinate system C1.

[0120] Then, similar to the above step S7, the processor 40 transforms the conveying direction D1 (vector VT 9_10 ) is determined as the positive y-axis direction of the conveying device coordinate system C4. Thereafter, the processor 40 determines the positive x-axis direction and the positive z-axis direction of the conveying device coordinate system C4, respectively, as in the above-mentioned step S7.

[0121] In addition, the processor 40 determines the origin of the conveying device coordinate system C4 in the robot coordinate system C1. As an example, the processor 40 transforms the above-mentioned coordinate Q9 (X9, Y9, Z9) into the coordinate C9' (X9', Y9', Z9') of the robot coordinate system C1, and determines the coordinate C9' as the origin of the conveying device coordinate system C4. As another example, the processor 40 may also transform the above-mentioned coordinate Q10 (X10, Y10, Z10) into the coordinate C10' (X10', Y10', Z10') of the robot coordinate system C1, and determine the coordinate C10' as the origin of the conveying device coordinate system C4.

[0122] As another example, the processor 40 may transform the axis A3 obtained as the coordinate (or function) of the second index coordinate system C5 in the above step S16 into the robot coordinate system C1, and determine the origin of the conveyor coordinate system C4 as a predetermined position on the axis A3 in the robot coordinate system C1. Fig.15 As shown, the transfer device coordinate system C4 is set to the robot coordinate system C1.

[0123] As described above, in this embodiment, the processor 80 of the terminal device 72 functions as the position data acquisition unit 52 and the transport direction acquisition unit 54, and on the other hand, the processor 40 of the control device 20 functions as the coordinate system setting unit 56. In addition, the processors 40 and 80 cooperate with each other to set the transport device coordinate system C4 to the robot coordinate system C1 based on the image data ID3 and ID4 obtained by the camera 14 shooting the first indicator 18 and the second indicator 74. Therefore, the camera 14, the first indicator 18, the position data acquisition unit 52, the transport direction acquisition unit 54, the coordinate system setting unit 56, and the second indicator 74 constitute the device 100 ( Fig. 9 ).

[0124] In the device 100, the second indicator 74 representing the second indicator coordinate system C5 is set at a known position of the robot coordinate system C1, and the camera 14 obtains the first image data ID3 obtained by photographing the first indicator 18 and the second indicator 74, and the second image data ID4 obtained by photographing the first indicator 18 and the second indicator 74 transported in step S13 (steps S11, S14).

[0125] In addition, the position data acquisition unit 52 (processor 80) acquires third position data PD4 (specifically, coordinates Q6) indicating the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 appearing in the first image data ID3 relative to the camera 14 (camera coordinate system C2) (step S12). Next, the position data acquisition unit 52 acquires fourth position data PD6 (specifically, coordinates Q8) indicating the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 appearing in the second image data ID4 relative to the camera 14.

[0126] Then, the transport direction acquisition unit (processor 80) calculates the transport direction D1 in the robot coordinate system C1 based on the first position data PD3 (coordinate Q5), the second position data PD5 (coordinate Q7), the third position data PD4 (coordinate Q6) and the fourth position data PD6 (coordinate Q8) (step S16).

[0127] According to this configuration, the transport device coordinate system C4 can be set to the robot coordinate system C1 simply by photographing the first index 18 and the second index 74 with the camera 14. Therefore, the transport device coordinate system C4 can be easily set (ie, calibrated) to the robot coordinate system C1.

[0128] In the device 100, the conveying direction acquisition unit 54 obtains the first image data ID3 ( Fig.12 ) in the first index coordinate system C3 and the second index coordinate system C5. 3_5_1 (Specifically, coordinate Q9) (step S16).

[0129] In addition, the conveying direction acquisition unit 54 obtains the second image data ID4 ( Fig.14 ) and the second positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 3_5_2 (Specifically, coordinate Q10) (step S16).

[0130] Then, the conveying direction acquisition unit 54 determines the first position relationship R3_5_1 And the second position relationship R 3_5_2 Determine the conveyance direction D1 (step S16 ). According to this configuration, the conveyance direction D1 can be determined quickly and accurately using an existing algorithm (specifically, the image analysis program PG1 or the like).

[0131] In the device 100, the second indicator 74 includes a pattern 74a, and the pattern 74a indicates the three-dimensional position of the second indicator coordinate system C5 in the camera coordinate system C2 set for the camera 14 that captured the image data ID3 and ID4. According to this configuration, the second indicator coordinate system C5 can be more effectively indicated in the camera coordinate system C2 by the pattern 74a of the second indicator 74 that appears in the image data ID3 and ID4.

[0132] Next, refer to Fig.16 as well as Fig.17 Next, a robot system 110 according to another embodiment will be described. The robot system 110 is different from the robot system 70 described above in that it further includes a sensor 112 . The sensor 112 detects a displacement Δ of the camera 14 .

[0133] More specifically, the sensor 112 includes at least one of a gyro sensor and an acceleration sensor, and is provided on the terminal device 72 in a fixed manner relative to the camera 14. In the present embodiment, the sensor 112 is communicatively connected to the I / O interface 84 of the terminal device 72, detects the displacement Δ of the position of the camera 14 (in other words, the terminal device 72) (more specifically, the displacement amount and displacement direction of the position and posture), and supplies it to the processor 80.

[0134] Next, refer to Fig.18 , a method for setting the conveyor coordinate system C4 to the robot coordinate system C1 in the robot system 110 is described. Fig.18 Before the process, the first indicator 18 is set on the conveying surface 34 of the conveying device 16.

[0135] In step S21, the processor 80 of the terminal device 72 starts detecting the displacement Δ of the camera 14 by the sensor 112. Specifically, the processor 80 activates the sensor 112 to continuously (e.g., periodically) detect the displacement Δ of the position of the camera 14 (or the terminal device 72). The processor 80 sequentially obtains the detection data of the displacement Δ from the sensor 112 and stores it in the memory 42.

[0136] In step S22, processor 80 uses camera 14 to photograph first indicator 18. Specifically, the operator places terminal device 72 at photographing position PS5 where first indicator 18 can be within the field of view of camera 14 of terminal device 72. At this time, processor 80 may perform the above-mentioned live view display.

[0137] Then, the operator operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the first indicator 18 . Fig.19 An example of the image data ID5 captured by the camera 14 in step S22 is shown. The camera 14 acquires the image data ID5 (first image data) captured by capturing the first indicator 18 in this manner, and supplies the image data ID5 to the processor 80 .

[0138] In step S23, the processor 80 functions as the position data acquisition unit 52 ( Fig.17 ) functions to obtain position data PD7 (first position data), which indicates the three-dimensional position of the first indicator coordinate system C3 represented by the first indicator 18 shown in the image data ID5 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 as in the above embodiment to obtain Fig.19 The coordinates Q11 (X11, Y11, Z11, W11, P11, R11) of the first index coordinate system C3 in the camera coordinate system C2 are used as the position data PD7. When the step S23 is completed, the processor 80 sends a completion signal SG1 to the control device 20.

[0139] The processor 40 of the control device 20 executes step S24 according to the completion signal SG1. Specifically, in step S24, the processor 40 operates the conveying device 16 to convey the first index 18 in the conveying direction D1 by a distance δ, similar to the above-mentioned step S13. Fig. 20 Indicates the state of completing step S24. Fig. 20 In FIG. 1 , for easy understanding, the position of the first indicator 18 when the image data ID5 is captured is indicated by a dotted line B, and a first indicator coordinate system C3 indicated by the first indicator 18 is also described.

[0140] After step S24 is completed, the processor 40 sends a completion signal SG2 to the terminal device 72. When receiving the completion signal SG2, the processor 80 of the terminal device 72 executes step S25. At this time, the processor 80 may output a notification signal NS3 indicating that the first indicator 18 should be photographed again by the camera 14, as in the above embodiment.

[0141] In step S25, the processor 80 of the terminal device 72 uses the camera 14 to photograph the first indicator 18. Specifically, when the operator recognizes the notification signal NS3 output to the display device 86 (or the speaker of the terminal device 72), the terminal device 72 is configured to be in a photographing position PS6 that can accommodate the first indicator 18 within the field of view of the camera 14. The photographing position PS6 may be different from the photographing position PS5 of the above-mentioned step S22, or may be the same. In addition, at this time, the processor 80 may perform the above-mentioned live view display.

[0142] Then, the operator operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the first indicator 18 . Fig.21 An example of the image data ID6 captured by the camera 14 in step S25 is shown. The camera 14 acquires the image data ID6 (second image data) captured by capturing the first indicator 18 being conveyed, and supplies the image data ID6 to the processor 80 .

[0143] In step S26, the processor 80 functions as the position data acquisition unit 52 ( Fig.17 ) functions to obtain position data PD8 (second position data), which indicates the three-dimensional position of the first indicator coordinate system C3 indicated by the first indicator 18 shown in the image data ID6 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 as in the above step S23 to obtain Fig.21 The coordinates Q12 (X12, Y12, Z12, W12, P12, R12) of the first index coordinate system C3 in the camera coordinate system C2 are used as the position data PD8.

[0144] In step S27, the processor 80 uses the camera 14 to photograph the second indicator 74. Specifically, the operator places the terminal device 72 at a photographing position PS7 where the second indicator 74 can be contained within the field of view of the camera 14 of the terminal device 72. The photographing position PS7 is different from the above-mentioned photographing positions PS5 and PS6. At this time, the processor 80 can perform the above-mentioned live view display.

[0145] Then, the operator operates the input device 88 of the terminal device 72 to operate the camera 14 and capture an image of the second indicator 74 . Fig. 22 An example of the image data ID7 captured by the camera 14 in step S27 is shown. The camera 14 obtains the image data ID7 (third image data) obtained by capturing the second indicator 74, and supplies it to the processor 80. Thus, in this embodiment, the operator captures the first indicator 18 and the second indicator 74 in steps S22 and S25 and step S27, respectively.

[0146] In step S28, the processor 80 functions as the position data acquisition unit 52 ( Fig.17 ) functions to obtain position data PD9 (third position data) indicating the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 shown in the image data ID7 relative to the camera 14. Specifically, the processor 80 executes the image analysis program PG1 as in the above-mentioned embodiment to obtain Fig. 22 The coordinates Q13 (X13, Y13, Z13, W13, P13, R13) of the second index coordinate system C5 in the camera coordinate system C2 are used as the position data PD9.

[0147] In step S29, the processor 80 obtains the conveying direction D1 based on the position data PD7, PD8, and PD9 and the displacement Δ detected by the sensor 112. Here, after the start of step S21, the sensor 112 continues to detect the displacement Δ of the camera 14 (terminal device 72) while capturing the image data ID5 of step S22, the image data ID6 of step S25, and the image data ID7 of step S27.

[0148] Therefore, the processor 80 can obtain the displacement Δ from the photographing position PS5 at which the image data ID5 is photographed in step S22 to the photographing position PS6 at which the image data ID6 is photographed in step S25 based on the detection data of the sensor 112. 5_6 , and the displacement Δ from the photographing position PS6 to the photographing position PS7 where the image data ID7 is photographed in step S27 6_7 .

[0149] The processor 80 can be configured to calculate the displacement Δ 5_6 and Δ 6_7 , the position data PD7 (coordinate Q11) obtained in step S23, the position data PD9 (coordinate Q13) obtained in step S27, and the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 before transport in step S24 3_5_3 Specifically, the processor 80 obtains the coordinates Q14 (X14, Y14, Z14, W14, P14, R14) of the first index coordinate system C3 before transport in the second index coordinate system C5 as the positional relationship R 3_5_3 data.

[0150] In addition, the processor 80 can be configured to calculate the displacement Δ 6_7 , the position data PD8 (coordinate Q12) obtained in step S26 and the position data PD9 (coordinate Q13) obtained in step S27, and the positional relationship R between the first index coordinate system C3 and the second index coordinate system C5 after the transport in step S24 is obtained. 3_5_4Specifically, the processor 80 obtains the coordinates Q15 (X15, Y15, Z15, W15, P15, R15) of the first index coordinate system C3 after transport in the second index coordinate system C5 as the positional relationship R 3_5_4 data.

[0151] Then, the processor 80 calculates the vector VT from the coordinates (X14, Y14, Z14) in the coordinate Q14 to the coordinates (X15, Y15, Z15) in the coordinate Q15. 14_15 The vector VT 14_15 The axis line A3 connecting the coordinates (X14, Y14, Z14) and the coordinates (X15, Y15, Z15) coincides with the position of the vector (in other words, the axis line A3) in the conveyance direction D1 in the second index coordinate system C5.

[0152] The vector VT 14_15 (or axis A3) is represented as a coordinate (or function) of the second index coordinate system C5, which represents the trajectory of the origin of the first index coordinate system C3 moved in the conveying direction D1 through step S24 in the second index coordinate system C5, and can be regarded as being arranged on the conveying surface 34. In this way, the processor 80 obtains the vector VT in the second index coordinate system C5 14_15 (or axis A3) is used as the conveying direction D1.

[0153] After step S29 is completed, the processor 80 converts the data of the conveying direction D1 obtained in step S29 (specifically, the vector VT in the second index coordinate system C5) into 14_15 The data of the coordinates or functions of the axis A3 are supplied to the control device 20. When the processor 40 of the control device 20 receives the data of the conveying direction D1, it executes step S30.

[0154] In step S30, the processor 40 functions as the coordinate system setting unit 56 ( Fig.17 ) functions to set the conveying device coordinate system C4 to the robot coordinate system C1 ( Fig.23 Specifically, as in step S17 above, the processor 40 uses the known positional relationship R between the second index coordinate system C5 and the robot coordinate system C1. 1_5 The transport direction D1 (for example, vector VT 14_15 ) is transformed into the robot coordinate system C1.

[0155] Then, similarly to the above embodiment, the processor 40 transforms the conveying direction D1 (vector VT 14_15) is determined as the positive y-axis direction of the conveying device coordinate system C4, and the positive x-axis direction and the positive z-axis direction of the conveying device coordinate system C4 are determined respectively.

[0156] In addition, the processor 40 determines the origin of the conveying device coordinate system C4 in the robot coordinate system C1. As an example, the processor 40 transforms the above-mentioned coordinate Q14 (X14, Y14, Z14) into the coordinate C14' (X14', Y14', Z14') of the robot coordinate system C1, and determines the coordinate C14' as the origin of the conveying device coordinate system C4. As another example, the processor 40 may also transform the above-mentioned coordinate Q15 (X15, Y15, Z15) into the coordinate C15' (X15', Y15', Z15') of the robot coordinate system C1, and determine the coordinate C15' as the origin of the conveying device coordinate system C4.

[0157] As another example, the processor 40 may transform the axis A3 obtained as the coordinate (or function) of the second index coordinate system C5 in the above step S29 into the robot coordinate system C1, and determine the origin of the conveyor coordinate system C4 as a predetermined position on the axis A3 in the robot coordinate system C1. Fig.23 As shown, the transfer device coordinate system C4 is set to the robot coordinate system C1.

[0158] As described above, in the present embodiment, the processors 40 and 80 cooperate with each other to function as the position data acquisition unit 52, the transport direction acquisition unit 54, and the coordinate system setting unit 56, and set the transport device coordinate system C4 to the robot coordinate system C1 based on the image data ID5, ID6, and ID7 obtained by the camera 14 photographing the first index 18 and the second index 74, and the displacement Δ detected by the sensor 112. Therefore, the camera 14, the first index 18, the position data acquisition unit 52, the transport direction acquisition unit 54, the coordinate system setting unit 56, the second index 74, and the sensor 112 constitute the device 120 ( Fig.17 ).

[0159] In the device 120, the camera 14 acquires the first image data ID5 (step S22) and the second image data ID6 (step S25) obtained by photographing the first indicator 18, and the third image data ID3 (step S27) obtained by photographing the second indicator 74. In addition, the position data acquisition unit 52 (processor 80) acquires the third position data PD9 indicating the three-dimensional position of the second indicator coordinate system C5 indicated by the second indicator 74 shown in the third image data ID3 relative to the camera 14 (camera coordinate system C2) (step S28).

[0160] In addition, the sensor 112 detects the displacement Δ of the camera 14 during the period of capturing the first image data ID5, the second image data ID6, and the third image data ID7. 5_6 and Δ 6_7 Then, the conveying direction acquisition unit 54 obtains the first position data PD7, the second position data PD8, the third position data PD9, and the displacement Δ 5_6 and Δ 6_7 , find the conveying direction D1 (step S29).

[0161] Here, due to the space limitation of the work unit, e.g. Fig.12 and Fig.14 As shown, sometimes it is impossible to simultaneously capture the first indicator and the second indicator 74 by the camera 14. According to this embodiment, Fig.19 , Fig.21 as well as Fig. 22 As shown, even if the first index and the second index 74 are photographed separately, the conveying direction D1 can be obtained. Therefore, even when there is not enough space in the work unit, the conveying device coordinate system C4 can be effectively set in the robot coordinate system C1.

[0162] In addition, in the above Figure 3 , Fig.11 or Fig.18 In the illustrated flow, the processor 40 or 80 describes a case where the first indicator 18 is photographed by the camera 14 and the transport direction D1 is obtained before and after the first indicator 18 is transported only once by the transport device 16. However, the present invention is not limited thereto, and the processor 40 or 80 may transport the first indicator 18 multiple times by the transport device 16, photograph the first indicator 18 by the camera 14 before and after the transport, and obtain the transport direction D1 multiple times.

[0163] Below, refer to Fig.24 , this method is explained. Fig.24 The process of FIG. 1 shows another method of setting the conveyor coordinate system C4 to the robot coordinate system C1 in the robot system 10. In step S31, the processor 40 sets the number "n" indicating the number of times the first index 18 is photographed in step S32 described later to "1" (n=1).

[0164] In step S32, the processor 40 uses the camera 14 to photograph the first indicator 18. Specifically, the processor 40 operates the robot 12 and arranges the camera 14 at a photographing position PSn that can accommodate the first indicator 18 within the field of view of the camera 14. Then, the processor 40 operates the camera 14 to photograph the first indicator 18, thereby obtaining, for example, Figure 4 or Figure 6Assume that, when the start time point number "n" of step S32 is set to n=1, the processor 40 captures the image data ID1 by the camera 14 disposed at the capturing position PS1.

[0165] In step S33, similar to the above-mentioned step S2 or S5, the processor 40 acts as the position data acquisition unit 52 ( Figure 2 ) functions to obtain position data PDn (specifically, coordinate Qn of the camera coordinate system C2), wherein the position data PDn represents the three-dimensional position of the first indicator coordinate system C3 represented by the first indicator 18 appearing in the image data IDn relative to the camera 14.

[0166] Assuming that the start time point number "n" of step S33 is set to n=1, the processor 40 obtains the position data PD1 (specifically, the coordinate Q1 of the camera coordinate system C2) indicating the three-dimensional position of the first index coordinate system C3 indicated by the image data ID1 relative to the camera 14. In step S34, the processor 40 increases the number "n" by "1" (n=n+1).

[0167] In step S35, the processor 40 determines whether the number "n" is n>2. The processor 40 proceeds to step S36 if the determination is yes, and proceeds to step S38 if the determination is no. It is assumed that when the step S35 is entered after the first execution of step S32 (i.e., in the state where n=1 is set), the number "n" at this time point is increased to n=2 in step S34. Therefore, in this case (i.e., when the first indicator 18 is only photographed once), the processor 40 determines no in step S35 and proceeds to step S38.

[0168] In step S36, similar to the above-mentioned step S6, the processor 40 acts as the conveying direction acquisition unit 54 ( Figure 2 ) functions to obtain the conveying direction D1 based on the position data PDn (specifically, the coordinates Qn) obtained in the most recently executed step S33 and the position data PDn-1 (specifically, the coordinates Qn-1) obtained in the step S33 executed before the position data PDn is obtained. _n .

[0169] Here, in Fig.24In the flow shown, the processor 40 repeatedly executes the loop of steps S32 to S38 until it is determined to be yes in step S37 described later. Therefore, in step S38 described later, the processor 40 repeatedly transports the first index 18 by the transport device 16, and obtains the position data PDn-1 (first position data) in step S33 executed before the transport based on the image data IDn-1 (first image data) captured in step S32 before the transport. On the other hand, the processor 40 obtains the position data PDn (second position data) in step S33 executed after the transport based on the image data IDn (second image data) captured in step S32 after the transport.

[0170] In step S36, the processor 40 obtains the conveying direction D1 based on the position data PDn (second position data) obtained in the most recent step S33 and the position data PDn-1 (first position data) obtained in step S33 executed before obtaining the position data PDn, in the same manner as in step S6. _n (For example, with the above vector VT 3_4 Same vector VT _n ).

[0171] In step S37, the processor 40 determines whether the number "n" is n=n. MAX The number n MAX represents the maximum number of times (for example, n) of photographing the first indicator 18 in step S32. MAX =10), which is predetermined by the operator. The processor 40 proceeds to step S39 when the determination is yes, and proceeds to step S38 when the determination is no.

[0172] In step S38, similar to the above-mentioned step S3, the processor 40 transports the first indicator 18 through the transport device 16. Alternatively, the processor 40 may transport the first indicator 18 in the transport direction D1 in the nth step S38, and on the other hand, transport the first indicator 18 in the direction D2 opposite to the transport direction D1 through the transport device 16 in the n+1th step S38 executed thereafter.

[0173] In step S36 executed after the transport is carried out in the opposite direction D2, the processor 40 calculates the vector VT of the opposite direction D2 based on the position data PDn-1 and PDn. _n '. In this case, the processor 40 can find the vector VT _n 'The vector VT in the opposite direction _n As transport direction D1 _nIn this way, the processor 40 repeatedly executes the loop of steps S32 to S38 until a YES is determined in step S37. Each time step S36 is executed, the conveying direction D1 is obtained. _n .

[0174] In step S39, the processor 40 functions as the transport direction acquisition unit 54 to obtain the final transport direction D1. _F Specifically, each time step S36 is executed, the processor 40 obtains the transport direction D1 _n The multiple vectors VT obtained _1 VT _2 ,…VT _n The resultant vector VT _F (or average vector), obtain the vector VT _F As the final transport direction D1 _F .

[0175] In step S40, the processor 40 functions as the coordinate system setting unit 56 ( Figure 2 ) performs the function, similar to the above step S6, according to the conveying direction D1 obtained in step S39 _F , the conveyor coordinate system C4 is set to the robot coordinate system C1. Specifically, the processor 40 sets the vector VT obtained in step S39 _F This is determined as the positive y-axis direction of the transport device coordinate system C4.

[0176] Thus, in this embodiment, the processor 40 obtains the conveying direction D1 each time the first indicator 18 is conveyed multiple times by the conveying device 16. _n , according to the multiple transport directions D1 obtained _n Get the final transport direction D1 _F According to this configuration, the conveying direction D1 of the conveying device 16 can be obtained with higher accuracy.

[0177] also, Fig.24 The concept of the process (that is, each time the first indicator 18 is transported multiple times by the transport device 16, the transport direction D1 is obtained. _n , according to multiple transport directions D1 _n Get the final transport direction D1 _F ) can be applied to Fig.11 or Fig.18 For example, in Fig.11 In the case of the process, the processors 40 and 80 can cooperate with each other to repeatedly execute the loop of steps S11 to S16, and the plurality of conveying directions D1 obtained each time step S16 is executed. _n , find the final transport direction D1 _F .

[0178] In addition, Fig.18 In the case of the process of FIG. 4 , the processor 80 of the terminal device 72 executes steps S27 and S28 after step S21. Thereafter, the processors 40 and 80 may cooperate with each other to repeatedly execute the loop of steps S22 to S26 and S29, and the plurality of conveying directions D1 obtained each time step S29 is executed are used to determine the conveying direction D1. _n , find the final transport direction D1 _F .

[0179] In addition, in the above-mentioned embodiment, the case where the second indicator 74 includes the pattern 74a representing the second indicator coordinate system C5 in the camera coordinate system C2 is described. However, the second indicator 74 is not limited thereto, and the second indicator 74 may also be constituted by the shape 74b of the robot 12. As an example, the shape 74b may be constituted by a plurality of faces, edges, concave portions, and convex portions formed in a component of the robot 12 (e.g., the robot base 22). As another example, the shape 74b may also be constituted by the shape of the entire robot 12 (i.e., the robot base 22, the rotating body 24, the lower arm 26, the upper arm 28, the wrist 30, and the wrist flange 30b) stationary at a predetermined position.

[0180] In this embodiment, when the camera 14 captures the shape 74b serving as the second indicator 74 in the above-mentioned steps S11, S14 or S27, the shape 74b appearing in the captured image data ID represents the three-dimensional position (i.e., coordinate Q) of the second indicator coordinate system C5 in the camera coordinate system C2 of the camera 14.

[0181] Then, the processor 40 or 80 functions as the position data acquisition unit 52 in the above-mentioned steps S12, S15 or 28, executes the image analysis program PG2 (image analysis application) for reading the second indicator coordinate system C4 from the shape 74b, and determines the second indicator coordinate system C4 represented by the shape 74b in the image data ID.

[0182] Then, the processor 40 or 80 obtains the coordinates Q of the determined second indicator coordinate system C4 in the camera coordinate system C2 as position data PD. In addition, the first indicator 18 is not limited to the pattern 18a, and may include, for example, a shape 18b (surface, edge, concave portion, convex portion) formed on the conveying surface 34.

[0183] In addition, in the above-mentioned robot system 70 ( Fig. 9 ) and 110( Fig.17), the case where the processor 80 of the terminal device 72 functions as the position data acquisition unit 52 and the conveying direction acquisition unit 54 is described. However, in the robot system 70 or 110, the processor 40 of the control device 20 may also function as the position data acquisition unit 52, the conveying direction acquisition unit 54, and the coordinate system setting unit 56. In this case, the processor 80 of the terminal device 72 supplies the image data ID3, ID4, ID5, ID6, and ID7 captured by the camera 14 to the control device 20.

[0184] Alternatively, the terminal device 72 may be omitted, and the camera 14 may be formed of a digital camera, for example. The operator may send the image data ID3, ID4, ID5, ID6, and ID7 captured by the camera 14 to the I / O interface 44 of the control device 20 by wire or wirelessly.

[0185] In step S6, the processor 40 uses the conveying direction D1 as a vector VT from the coordinates Q3 (X3, Y3, Z3) to the coordinates Q4 (X4, Y4, Z4) representing the origin trajectory of the first index coordinate system C3. 3_4 However, the processor 40 may also obtain the conveying direction D1 based on the change amount between the coordinates (W3, P3, Q3) representing the posture in the coordinates Q3 and the coordinates (W4, P4, R4) representing the posture in the coordinates Q4.

[0186] Similarly, in the above-mentioned steps S16, S29 or S36, the processor 80 can calculate the conveying direction D1 based on the changes in the coordinates Q9 (W9, P9, R9) and Q10 (W10, P10, R10) representing the posture, the changes in the coordinates Q14 (W14, P14, R14) and Q15 (W15, P15, R15) representing the posture, or the changes in the coordinates Qn-1 and Qn representing the posture.

[0187] In addition, the processor 40 or 80 may also execute the computer program PG3 pre-stored in the memory 42 or 82. Figure 3 , Fig.11 , Fig.18 or Fig.24 In addition, the functions of the position data acquisition unit 52, the conveying direction acquisition unit 54, and the coordinate system setting unit 56 executed by the processor 40 or 80 may be functional modules implemented by the computer program PG3.

[0188] In addition, the control device 20 may also include a first control device 20A for controlling the robot 12 and a second control device 20B for controlling the conveying device 16. In addition, the robot 12 is not limited to a vertical multi-joint robot, but may be any type of robot such as a horizontal multi-joint robot or a parallel linkage robot. In addition, the conveying device 16 is not limited to a belt conveyor, but may be any device capable of conveying a workpiece. The present disclosure has been described above through the embodiments, but the above embodiments do not limit the invention involved in the claims.

[0189] Explanation of symbols

[0190] 10, 70, 110 robot systems

[0191] 12 Robot

[0192] 14 Camera

[0193] 16 conveying device

[0194] 18 First Indicator

[0195] 20 Control device

[0196] 40, 80 processors

[0197] 52 Position data acquisition unit

[0198] 54 Transport direction acquisition part

[0199] 56 Coordinate system setting unit

[0200] 60, 100, 120 units

[0201] 74 Second indicator

[0202] 112 sensors.

Claims

1. A device for setting a conveyor coordinate system to a robot coordinate system, wherein: The robot coordinate system is a coordinate system set in a robot that performs operations on a workpiece, and the transport device coordinate system specifies a transport direction of a transport device that transports the workpiece, and is characterized in that: The device has: a first index representing a first index coordinate system and provided on the conveying device so as to be conveyed by the conveying device; a camera that acquires first image data and second image data, wherein the first image data is data obtained by photographing the first indicator, and the second image data is data obtained by photographing the first indicator after being transported by the transport device after the first image data is photographed; a position data acquisition unit configured to acquire first position data and second position data, wherein the first position data indicates a three-dimensional position of the first indicator coordinate system represented by the first indicator appearing in the first image data relative to the camera, and the second position data indicates a three-dimensional position of the first indicator coordinate system represented by the first indicator appearing in the second image data relative to the camera; a conveying direction acquiring unit configured to obtain the conveying direction based on the first position data and the second position data; and A coordinate system setting unit sets the transport device coordinate system to the robot coordinate system based on the transport direction obtained by the transport direction acquisition unit.

2. The device according to claim 1, characterized in that The device further comprises: a second index, which represents a second index coordinate system and is arranged at a known position of the robot coordinate system, The camera acquires: the first image data obtained by photographing the first index and the second index, and the second image data obtained by photographing the first index and the second index after the transport. The position data acquisition unit further acquires third position data and fourth position data, wherein the third position data indicates a three-dimensional position of the second indicator coordinate system represented by the second indicator appearing in the first image data relative to the camera, and the fourth position data indicates a three-dimensional position of the second indicator coordinate system represented by the second indicator appearing in the second image data relative to the camera. The transport direction acquisition unit further obtains the transport direction based on the third position data and the fourth position data.

3. The device according to claim 2, characterized in that The conveying direction acquisition unit obtains a first positional relationship between the first indicator coordinate system and the second indicator coordinate system in the first image data based on the first position data and the third position data. The conveying direction acquisition unit obtains a second positional relationship between the first indicator coordinate system and the second indicator coordinate system in the second image data based on the second position data and the fourth position data. The transport direction acquisition unit obtains the transport direction based on the first positional relationship and the second positional relationship.

4. The device according to claim 1, characterized in that The device has: a second index representing a second index coordinate system and disposed at a known position in the robot coordinate system; and a sensor that detects displacement of the camera, The camera further acquires third image data obtained by photographing the second indicator. The position data acquisition unit further acquires third position data indicating a three-dimensional position of the second indicator coordinate system indicated by the second indicator appearing in the third image data relative to the camera, the sensor detects the displacement of the camera during capturing the first image data, the second image data, and the third image data, The transport direction acquisition unit further obtains the transport direction based on the third position data and the displacement detected by the sensor.

5. The device according to any one of claims 2 to 4, characterized in that: The second indicator includes a pattern indicating a three-dimensional position of the second indicator coordinate system in a camera coordinate system set for the camera that captures the image data, or a shape of the robot.

6. The device according to any one of claims 1 to 5, characterized in that: The first indicator includes a pattern indicating a three-dimensional position of the first indicator coordinate system in a camera coordinate system set for the camera that captures the image data.

7. A method for setting a handling device coordinate system to a robot coordinate system, wherein: The robot coordinate system is a coordinate system set in a robot that operates on a workpiece, and the conveying device coordinate system specifies a conveying direction of a conveying device that conveys the workpiece. It is characterized in that, in the method: placing a first index representing a first index coordinate system on the transport device so as to be transported by the transport device; The first indicator is photographed by a camera to obtain first image data, and after the first image data is photographed, the first indicator transported by the transport device is photographed by the camera to obtain second image data; Obtaining first position data and second position data, wherein the first position data indicates a three-dimensional position of the first indicator coordinate system represented by the first indicator appearing in the first image data relative to the camera, and the second position data indicates a three-dimensional position of the first indicator coordinate system represented by the first indicator appearing in the second image data relative to the camera; Calculating the conveying direction based on the first position data and the second position data; and The transport device coordinate system is set to the robot coordinate system based on the obtained transport direction.

Images