Device, robot system, method, and computer program for obtaining the position of a vision sensor in a control coordinate system of a robot

Abstract

Conventionally, in order to measure the position of a visual sensor in a control coordinate system, it is necessary to change the relative posture of the visual sensor with respect to a marker, but there is a possibility that the marker will deviate from the visual sensor's field of view after the posture change. A processor of a device (18, 16) for obtaining the position of a visual sensor (14) in a control coordinate system (C2) causes a robot (12) to operate so as to change the posture of the visual sensor (14) or the marker by a first posture change amount, and obtains the position of the visual sensor (14) in the control coordinate system (C2) as a trial measurement position based on image data of the marker captured by the visual sensor (14) before and after the posture change. The robot (12) is operated so as to change the posture by a second posture change amount greater than the first posture change amount, and obtains the position of the visual sensor (14) in the control coordinate system (C2) as a main measurement position based on image data of the marker captured by the visual sensor (14) before and after the posture change.

Description

Technical Field

[0001] The present invention relates to an apparatus, a robot system, a method, and a computer program for obtaining the position of a vision sensor in a control coordinate system of a robot. Background Art

[0002] Conventionally, there is known a device that measures the position and posture of a vision sensor in a control coordinate system of a robot based on image data obtained by capturing a marker with the vision sensor (for example, Patent Documents 1 and 2).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2005-201824

[0006] Patent Document 2: Japanese Patent Application Laid-Open No. 2005-300230 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] Conventionally, to measure the position of a vision sensor in a control coordinate system, the relative posture of the vision sensor to the marker needs to be changed (for example, by rotating the vision sensor or the marker around a predetermined axis). In this case, the marker may be out of the field of view of the vision sensor.

[0009] Means for solving problems

[0010] In one embodiment of the present disclosure, a device for obtaining the position of a visual sensor in a control coordinate system for controlling a robot that causes relative movement of a visual sensor and a marker comprises a processor that performs the following processing: causing the robot to move so that the posture of the visual sensor or the marker changes by a first posture change amount; obtaining the position of the visual sensor in the control coordinate system as a trial measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the first posture change amount; causing the robot to move so that the posture changes by a second posture change amount that is larger than the first posture change amount in a posture change direction determined based on the trial measurement position; and obtaining the position of the visual sensor in the control coordinate system as a main measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the second posture change amount.

[0011] In one embodiment of the present disclosure, in a method for obtaining the position of a visual sensor in a control coordinate system of a robot for controlling relative movement of a visual sensor and a marker, a processor performs the following processing: causing the robot to move so that the posture of the visual sensor or the marker changes by a first posture change amount; obtaining the position of the visual sensor in the control coordinate system as a trial measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the first posture change amount; causing the robot to move so that the posture changes by a second posture change amount that is larger than the first posture change amount in a posture change direction determined based on the trial measurement position; and obtaining the position of the visual sensor in the control coordinate system as a main measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the second posture change amount.

[0012] Effects of the Invention

[0013] According to the present disclosure, the visual sensor's trial measurement position in the control coordinate system is estimated by changing the visual sensor's posture by a relatively small amount. Subsequently, the visual sensor's main measurement position in the control coordinate system is determined by changing the visual sensor's posture by a larger amount. This configuration prevents the marker from deviating from the visual sensor's field of view after the posture change, and allows the main measurement position, which accurately represents the visual sensor's position in the control coordinate system, to be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram of a robot system according to one embodiment.

[0015] Figure 2 yes Figure 1 Block diagram of the robotic system shown.

[0016] Figure 3 This is an example of a logo.

[0017] Figure 4 This is a flowchart showing an example of a method for obtaining the position of the vision sensor in the control coordinate system.

[0018] Figure 5 Yes Figure 4 Flowchart of an example of step S1 in .

[0019] Figure 6 This shows an example of image data obtained by imaging a marker using a visual sensor.

[0020] Figure 7 Yes Figure 4 Flowchart of an example of step S2 in .

[0021] Figure 8 Yes Figure 4 Flowchart of an example of step S3 in .

[0022] Figure 9 It is a diagram of a robot system according to another embodiment.

[0023] Figure 10 Indicates that it is set Figure 9 The identity of the robot shown. DETAILED DESCRIPTION

[0024] Hereinafter, the embodiments of the present disclosure will be described in detail based on the accompanying drawings. In addition, in the various embodiments described below, the same elements are marked with the same reference numerals and repeated descriptions are omitted. Figure 1 as well as Figure 2 Next, a robot system 10 according to an embodiment will be described. The robot system 10 includes a robot 12 , a visual sensor 14 , a control device 16 , and a teaching device 18 .

[0025] In this embodiment, the robot 12 is a vertically articulated robot having a robot base 20, a rotating body 22, a robot arm 24, and a wrist 26. The robot base 20 is fixed to the floor of the work cell. The rotating body 22 is mounted on the robot base 20 so as to be rotatable about a vertical axis. The robot arm 24 has a lower arm 28 mounted on the rotating body 22 so as to be rotatable about a horizontal axis, and an upper arm 30 mounted rotatably at the front end of the lower arm 28.

[0026] The wrist portion 26 includes a wrist base 32 rotatably connected to the distal end of the upper arm portion 30, and a wrist flange 34 rotatably mounted on the wrist base 32 about the axis A. The wrist flange 34 is a cylindrical member centered about the axis A and has a mounting surface 34a at its distal end. The wrist portion 26 rotates the wrist flange 34 about the axis A.

[0027] An end effector (not shown) for performing work on a workpiece is detachably mounted on the mounting surface 34a. The end effector is a robot arm, welding gun, laser processing head, or paint applicator, and performs a predetermined operation (such as workpiece handling, welding, laser processing, or coating) on ​​the workpiece.

[0028] Each component of the robot 12 (ie, the robot base 20, the rotating body 22, the robot arm 24, and the wrist 26) has a built-in servo motor 36 ( Figure 2 The servo motor 36 drives each movable element of the robot 12 (ie, the rotating body 22 , the robot arm 24 , and the wrist 26 ) according to a command from the control device 16 .

[0029] The robot 12 has a robot coordinate system C1 ( Figure 1The robot coordinate system C1 is a control coordinate system used to control the motion of each movable element of the robot 12 and is fixed in three-dimensional space. In this embodiment, the robot coordinate system C1 is set for the robot 12 so that its origin is located at the center of the robot base 20 and its z-axis is aligned with the rotation axis of the rotating body 22.

[0030] On the other hand, Figure 1 As shown, a mechanical interface (hereinafter referred to as "MIF") coordinate system C2 is established at the fingertips of the robot 12 (specifically, the wrist flange 34). The MIF coordinate system C2 is a control coordinate system used to control the position and posture of the wrist flange 34 (or end effector) in the robot coordinate system C1. In this embodiment, the MIF coordinate system C2 is established at the fingertips of the robot 12 such that its origin is located at the center of the mounting surface 34a of the wrist flange 34 and its z-axis is aligned with the axis A.

[0031] When moving the wrist flange 34 (end effector), the processor 40 sets the MIF coordinate system C2 in the robot coordinate system C1 and controls the servo motors 36 of the robot 12 to position the wrist flange 34 (end effector) at the position and posture indicated by the set MIF coordinate system C2. In this way, the processor 40 can position the wrist flange 34 (end effector) at any position and posture in the robot coordinate system C1.

[0032] The visual sensor 14 is, for example, a camera or a three-dimensional visual sensor, and includes an imaging sensor (such as a CCD or CMOS) that receives and photoelectrically converts the image of a subject, and an optical lens (such as a condenser lens or a focus lens) that collects the subject image and focuses the imaging sensor. The visual sensor 14 captures an image of the subject and transmits the captured image data to the control device 16. In this embodiment, the visual sensor 14 is fixed to a predetermined position relative to the wrist flange 34.

[0033] A sensor coordinate system C3 is set in the vision sensor 14. The sensor coordinate system C3 is a coordinate system that defines the coordinates of each pixel of the image data captured by the vision sensor 14. The sensor coordinate system C3 is set so that its origin is located at the center of the light-receiving surface (or optical lens) of the imaging sensor of the vision sensor 14, its x-axis and y-axis are arranged parallel to the horizontal and vertical directions of the imaging sensor, and its z-axis is aligned with the line of sight (or optical axis) O of the vision sensor 14.

[0034] The control device 16 controls the operation of the robot 12 and the vision sensor 14. Specifically, the control device 16 is a computer having a processor 40, a memory 42, and an I / O interface 44. The processor 40 includes a CPU or a GPU, and is communicatively connected to the memory 42 and the I / O interface 44 via a bus 46. While communicating with the memory 42 and the I / O interface 44, the processor 40 sends commands to the robot 12 and the vision sensor 14, thereby controlling the operation of the robot 12 and the vision sensor 14.

[0035] The memory 42 includes RAM or ROM, etc., and temporarily or permanently stores various data. The I / O interface 44 includes, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices wirelessly or wired under instructions from the processor 40. The servo motor 36 and the visual sensor 14 are connected to the I / O interface 44 in a manner that enables wireless or wired communication.

[0036] The teaching device 18 is, for example, a handheld device (such as a teaching pendant or a tablet terminal device) for teaching the robot 12 the actions required to perform a predetermined task. Specifically, the teaching device 18 is a computer having a processor 50, a memory 52, an I / O interface 54, an input device 56, and a display device 58. The processor 50 includes a CPU or a GPU, and is communicatively connected to the memory 52, the input device 56, the display device 58, and the I / O interface 54 via a bus 60.

[0037] The memory 52 includes RAM or ROM, etc., and temporarily or permanently stores various data. The I / O interface 54 includes, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices wirelessly or wired under instructions from the processor 50. The I / O interface 54 is connected to the I / O interface 44 of the control device 16 via a wired or wireless connection, enabling communication between the control device 16 and the teaching device 18.

[0038] The input device 56 includes buttons, switches, a keyboard, a touch panel, etc., receives input operations from the operator, and transmits the input information to the processor 50. The display device 58 includes an LCD or an organic EL display, etc., and displays various information in response to instructions from the processor 50. By operating the input device 56, the operator can jog the robot 12 and teach the robot 12 a movement.

[0039] In this embodiment, the positional relationship between the MIF coordinate system C2 and the sensor coordinate system C3 is not calibrated and is unknown. However, when the robot 12 is caused to perform work on a workpiece based on image data captured by the vision sensor 14, the position (i.e., the origin position of the sensor coordinate system C3) and the posture (i.e., the axis directions of the sensor coordinate system C3) of the vision sensor 14 in the control coordinate system (i.e., the robot coordinate system C1 and the MIF coordinate system C2) used to control the robot 12 must be known.

[0040] In this embodiment, the teaching device 18 obtains data on the position and posture of the vision sensor 14 in the control coordinate system (robot coordinate system C1 , MIF coordinate system C2 ) based on image data of the identification ID captured by the vision sensor 14 . Figure 3 This shows an example of an ID. In this embodiment, the ID is provided on the upper surface of the structure B and is composed of a circular line C and two mutually orthogonal straight lines D and E. The ID is provided on the structure B in a visually recognizable form, such as a pattern using paint or an engraving (convex and concave) formed on the upper surface of the structure B.

[0041] Next, refer to Figure 4 , a method of obtaining data on the position and posture of the vision sensor 14 in the control coordinate system (robot coordinate system C1, MIF coordinate system C2) will be described. Figure 4 The process shown in FIG1 is started when the processor 50 of the teaching device 18 receives an action start instruction from an operator, a host controller or a computer program CP. In addition, the processor 50 can also execute the action in accordance with the computer program CP. Figure 4 The computer program CP may also be stored in the memory 52 in advance.

[0042] In step S1, the processor 50 executes a posture acquisition process. Figure 5 In step S11 , the processor 50 operates the robot 12 and positions the visual sensor 14 at the initial position PS0 and the initial posture OR0 relative to the marker ID.

[0043] The initial position PS0 and initial posture OR0 are predetermined so that when the vision sensor 14 is positioned at the initial position PS0 and initial posture OR0, the identifier ID enters the field of view of the vision sensor 14. The data of the initial position PS0 and initial posture OR0 (i.e., the coordinates of the origin of the MIF coordinate system C2 in the robot coordinate system C1 and the directions of the axes) are predetermined by the computer program CP and stored in the memory 52.

[0044] In step S12, the processor 50 activates the visual sensor 14 to capture the marker ID and obtain the relative position of the marker ID with respect to the visual sensor 14. Specifically, the processor 50 activates the visual sensor 14 at the initial position PS0 and initial posture OR0 to obtain image data JD0 of the marker ID.

[0045] The processor 50 obtains the image data JD0 from the visual sensor 14 via the control device 16 and stores it in the memory 52. ​​Alternatively, the processor 50 may obtain the image data JD0 directly from the visual sensor 14 without going through the control device 16. In this case, the I / O interface 54 may be connected to the visual sensor 14 in a communicative manner via a wired or wireless connection.

[0046] Next, the processor 50 obtains data indicating the relative position of the marker ID relative to the visual sensor 14 when the image data JD0 is captured. n and posture OR n Image data JD captured by the visual sensor 14 n , find the JD of the image data captured n The relative position data of the identification ID relative to the visual sensor 14 at that time. The method is described below.

[0047] Figure 6 Indicates configuration at any position PS n and posture OR n Image data JD captured by the visual sensor 14 n For example. Figure 6 As shown, in this embodiment, the origin of the sensor coordinate system C3 is arranged at the image data JD n However, the origin of the sensor coordinate system C3 may also be configured at the center of the image data JD n Any known position (pixel) of .

[0048] The processor 50 processes the image data JD n Analyze and determine the image data JD n The processor 50 then obtains the coordinates (x n ,y n ) as the representation of image data n The data of the position of the identification ID.

[0049] In addition, the processor 50 processes the image data JD n Analyze and determine the image data JDn Then, the processor 50 obtains the area of ​​the circle C in the sensor coordinate system C3 (or the number of pixels included in the image area of ​​the circle C) as the image data JD n The size of the logo ID shown in IS n Data (unit: [pixel]).

[0050] The processor 50 also obtains the size RS (unit [mm]) of the marker ID in real space, the focal length FD of the optical lens of the vision sensor 14, and the size SS (unit [mm / pixel]) of the imaging sensor of the vision sensor 14. These size RS, focal length FD, and size SS are pre-stored in the memory 52.

[0051] Then, the processor 50 uses the obtained coordinates (x n ,y n ), size IS n , size RS, focal length FD and size SS, obtain vector (X n , Y n , Z n ). Here, X n Can be based on X n =x n ×IS n ×SS / RS is obtained by equation (1). n Can be based on Y n =y n ×IS n ×SS / RS is obtained by equation (2). n Can be based on Z n =IS n ×SS×FD / RS is obtained from the formula (3).

[0052] The vector (X n , Y n , Z n ) is from shooting to image data JD n The vector from the visual sensor 14 (i.e., the origin of the sensor coordinate system C3) to the identification ID (specifically, the intersection F) is data representing the relative position of the identification ID relative to the visual sensor 14 (or, the coordinates of the sensor coordinate system C3).

[0053] In this way, the processor 50 is based on the image data JD n The position of the ID in (x n ,y n ), image data JD n The size of the logo ID shown in IS n, the size RS of the identifier ID in the actual space, the focal length FD, and the size SS of the imaging sensor to obtain the captured image data JD n The relative position data (X) of the identification ID relative to the visual sensor 14 at the time n , Y n , Z n In step S12, the processor 50 obtains the relative position data (X0, Y0, Z0) of the identification ID with respect to the visual sensor 14 when the image data JD0 is captured.

[0054] In step S13, the processor 50 activates the robot 12 to translate the visual sensor 14. Here, the robot 12 "translates" the fingertip, meaning that the robot 12 moves the fingertip without changing its posture. In this embodiment, the processor 50, with the visual sensor 14 configured in the initial posture OR0, causes the robot 12 to translate the visual sensor 14 from the initial position PS0 to the x-axis direction of the MIF coordinate system C2 at that point in time (i.e., the initial position PS0 and the initial posture OR0) by a predetermined distance δx (e.g., δx = 5 mm). As a result, the visual sensor 14 is configured in the position PS1 and the posture OR0 relative to the identifier ID.

[0055] In step S14, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD1 of the marker ID using the vision sensor 14 in position PS1 and orientation OR0, and obtains the coordinates (x1, y1) and size IS1 of the intersection point F of the marker ID reflected in the image data JD1.

[0056] The processor 50 then uses the acquired coordinates (x1, y1) and size IS1, and equations (1) to (3) above, to obtain the relative position data (X1, Y1, Z1) of the identifier ID relative to the vision sensor 14 at the time the image data JD1 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0057] In step S15, the processor 50 activates the robot 12 to translate the vision sensor 14. Specifically, with the vision sensor 14 positioned in its initial posture OR0, the processor 50 translates the vision sensor 14 from its initial position PS0 in the y-axis direction of the MIF coordinate system C2 by a predetermined distance δy (e.g., δy = 5 mm) through the robot 12. As a result, the vision sensor 14 is positioned at position PS2 and posture OR0 relative to the identifier ID.

[0058] In step S16, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD2 of the marker ID using the vision sensor 14 in position PS2 and orientation OR0, and obtains the coordinates (x2, y2) and size IS2 of the intersection point F of the marker ID reflected in the image data JD2.

[0059] The processor 50 then uses the acquired coordinates (x2, y2) and size IS2, and the aforementioned equations (1) to (3), to obtain the relative position data (X2, Y2, Z2) of the identifier ID relative to the vision sensor 14 when the image data JD2 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0060] In step S17, the processor 50 activates the robot 12 to translate the vision sensor 14. Specifically, with the vision sensor 14 positioned in its initial posture OR0, the processor 50 translates the vision sensor 14 from its initial position PS0 to the z-axis of the MIF coordinate system C2 by a predetermined distance δz (e.g., δz = 5 mm) through the robot 12. As a result, the vision sensor 14 is positioned at position PS3 and posture OR0 relative to the identifier ID.

[0061] In step S18, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD3 of the marker ID using the vision sensor 14 in position PS3 and orientation OR0, and obtains the coordinates (x3, y3) and size IS3 of the intersection point F of the marker ID reflected in the image data JD3.

[0062] The processor 50 then uses the acquired coordinates (x3, y3) and size IS3, and the aforementioned equations (1) to (3), to obtain the relative position data (X3, Y3, Z3) of the identifier ID relative to the vision sensor 14 when the image data JD3 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0063] In step S19, the processor 50 obtains data indicating the posture of the visual sensor 14 in the control coordinate system. Specifically, the processor 50 uses the relative position data (X) obtained in steps S12, S14, S16, and S18 to obtain the relative position data. n , Y n , Zn )(n=0, 1, 2, 3), obtain the matrix M1 shown in the following formula.

[0064] [Formula 1]

[0065]

[0066] This matrix M1 is a rotation matrix representing the posture (W, P, R) of the vision sensor 14 (or sensor coordinate system C3) in the MIF coordinate system C2. This rotation matrix can be represented by three parameters: roll, pitch, and yaw. Here, the W coordinate in the posture (W, P, R) corresponds to the yaw value, the P coordinate corresponds to the pitch value, and the R coordinate corresponds to the roll value. These posture coordinates W, P, and R can be calculated using matrix M1.

[0067] In this way, the processor 50 obtains the posture data (W, P, R) of the vision sensor 14 in the MIF coordinate system C2 and stores it in the memory 52. ​​This posture data (W, P, R) specifies the direction of each axis of the sensor coordinate system C3 in the MIF coordinate system C2 (i.e., the line of sight O). In addition, the coordinates of the MIF coordinate system C2 and the robot coordinate system C1 can be transformed into each other using a known transformation matrix. Therefore, the posture data (W, P, R) in the MIF coordinate system C2 can be transformed into the coordinates (W', P', R') of the robot coordinate system C1.

[0068] Here, the initial position PS0 and initial posture OR0, as well as the aforementioned distances δx, δy, and δz, are determined so that the identifier ID, at all positions and postures where the vision sensor 14 is positioned in steps S11, S13, S15, and S17, enters the field of view of the vision sensor 14. For example, the operator determines the initial position PS0 and initial posture OR0 so that the line of sight O of the vision sensor 14 passes inside the circle C of the identifier ID.

[0069] The positional relationship between the line of sight O of the vision sensor 14 and the marker ID at the initial position PS0 and initial posture OR0 can be estimated based on, for example, the design values ​​of the image data (CAD data, etc.) of the vision sensor 14, robot 12, and structure B. This allows the marker ID shown in the image data JD0 to be positioned near the origin of the sensor coordinate system C3. Furthermore, the distances δx, δy, and δz may have different values.

[0070] Refer again Figure 4 In step S2, the processor 50 performs a test measurement process. Figure 7Step S2 will be described. In step S21, the processor 50 changes the posture of the visual sensor 14 by rotating the visual sensor 14. Specifically, the processor 50 first sets the reference coordinate system C4 in the MIF coordinate system C2 at that point in time (initial position PS0 and initial posture OR0).

[0071] In this embodiment, the processor 50 sets the reference coordinate system C4 in the MIF coordinate system C2 so that its origin is located at the origin of the MIF coordinate system C2 and its posture (the directions of each axis) is consistent with the posture (W, P, R) acquired in step S19. Therefore, the directions of the x-axis, y-axis, and z-axis of the reference coordinate system C4 are parallel to the x-axis, y-axis, and z-axis of the sensor coordinate system C3, respectively.

[0072] Next, the processor 50 operates the robot 12 to rotate the vision sensor 14 (i.e., the wrist flange) from its initial position PS0 and initial posture OR0 by a posture change amount θ1 (a first posture change amount) about the z-axis of the reference coordinate system C4 (i.e., an axis parallel to the direction of the line of sight O), thereby relocating the robot 12 to position PS4 and posture OR1. This posture change amount θ1 is predetermined by the operator as an angle (e.g., θ1 = 5°) and stored in the memory 52. ​​In this manner, the processor 50 changes the posture of the vision sensor 14 from its initial posture OR0 to posture OR1.

[0073] In step S22, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD4 of the marker ID using the vision sensor 14 in position PS4 and orientation OR1, and obtains the coordinates (x4, y4) and size IS4 of the intersection F of the marker ID reflected in the image data JD4.

[0074] The processor 50 then uses the acquired coordinates (x4, y4) and size IS4, and the aforementioned equations (1) to (3), to obtain the relative position data (X4, Y4, Z4) of the identifier ID relative to the vision sensor 14 when the image data JD4 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0075] In step S23, the processor 50 rotates the vision sensor 14 to change its posture. Specifically, the processor 50 causes the robot 12 to operate, rotating the vision sensor 14 from its initial position PS0 and initial posture OR0 by a posture change amount θ2 (a first posture change amount) about the x-axis or y-axis of the reference coordinate system C4 (i.e., an axis perpendicular to the direction of the line of sight O), thereby repositioning the vision sensor 14 to position PS5 and posture OR2. This posture change amount θ2 is predetermined by the operator as an angle (e.g., θ2 = 5°) and stored in the memory 52. ​​In this manner, the processor 50 changes the posture of the vision sensor 14 from its initial posture OR0 to posture OR2.

[0076] In step S24, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD5 of the marker ID using the vision sensor 14 in position PS5 and orientation OR2, and obtains the coordinates (x5, y5) and size IS5 of the intersection point F of the marker ID reflected in the image data JD5.

[0077] The processor 50 then uses the acquired coordinates (x5, y5) and size IS5, and equations (1) to (3) above, to obtain the relative position data (X5, Y5, Z5) of the identifier ID relative to the vision sensor 14 when the image data JD5 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0078] In step S25, the processor 50 obtains the trial measurement position of the visual sensor 14. Here, if the vector from the origin of the reference coordinate system C4 (in this embodiment, the origin of the MIF coordinate system C2) in the MIF coordinate system C2 to the origin of the sensor coordinate system C3, whose position is unknown, is represented by (ΔX1, ΔY1, ΔZ1), then the following equations (4) and (5) are satisfied.

[0079] [Formula 2]

[0080]

[0081] [Formula 3]

[0082] cosθ2·Y0-sinθ2·(Z0+ΔZ1)=Y5...Equation (5)

[0083] The processor 50 can estimate the vector (ΔX1, ΔY1, ΔZ1) from the origin of the reference coordinate system C4 in the MIF coordinate system C2 to the origin of the unknown sensor coordinate system C3 by solving the above equations (4) and (5). This vector (ΔX1, ΔY1, ΔZ1) is data representing the estimated position of the visual sensor 14 (the origin of the sensor coordinate system C3) in the MIF coordinate system C2. In step S25, the processor 50 obtains the test measurement position as the coordinate (x T ,y T , z T ). In this embodiment, x T =ΔX1,y T =ΔY1,z T =ΔZ1.

[0084] By rotating the visual sensor 14 in the direction of the z-axis of the reference coordinate system C4 in step S21, the test measurement position (x T ,y T , z T ) in (x T ,y T )=(ΔX1,ΔY1). Try to measure the position (x T ,y T )=(ΔX1, ΔY1) represents the estimated position of the line of sight O in the MIF coordinate system C2 (in other words, the estimated position of the origin of the sensor coordinate system C3 within a plane orthogonal to the line of sight O).

[0085] On the other hand, by rotating the visual sensor 14 in the direction of the x-axis or y-axis around the reference coordinate system C4 in step S23, the test measurement position (x T ,y T , z T ) T (=ΔZ1). Try to measure the position z T (=ΔZ1) represents the estimated position of the visual sensor 14 (or the origin of the sensor coordinate system C3) in the direction along the line of sight O in the MIF coordinate system C2.

[0086] As described above, the processor 50 obtains the trial measurement position (x 0 , Y 0 , Z 0 ) based on the posture change amounts θ1 and θ2, the relative position data (X0, Y0, Z0) when the image data JD0 is captured before the posture change (i.e., the initial posture OR0), and the relative position data (X4, Y4, Z4) and (X5, Y5, Z5) when the image data JD4 and JD5 are captured after the posture change (i.e., the postures OR1 and OR2). T ,y T , zT The processor 50 updates the coordinates of the origin of the unknown sensor coordinate system C3 in the MIF coordinate system C2 to the obtained trial measurement position (x T ,y T , z T ) and stored in memory 52.

[0087] Refer again Figure 4 , the processor 50 performs the main measurement process in step S3. Figure 8 Step S3 will be described. In step S31 , the processor 50 changes the posture of the visual sensor 14 by rotating the visual sensor 14 .

[0088] Specifically, the processor 50 first determines the direction DR1 (posture change direction) in which the visual sensor 14 is moved in order to change its posture in step S31 as the direction around the z-axis of the sensor coordinate system C3 whose origin position is updated in step S25. The origin position of the sensor coordinate system C3 in the MIF coordinate system C2 at this time point is the test measurement position (x T ,y T , z T ), so the z-axis of the sensor coordinate system C3 is configured at the test measurement position (x T ,y T , z T ) is parallel to the direction of the line of sight O. In this way, the processor 50 is based on the measured position (x T ,y T , z T ) to determine the posture change direction DR1.

[0089] Next, the processor 50 operates the robot 12 to rotate the vision sensor 14 from the initial position PS0 and initial posture OR0 in the posture change direction DR1 (the direction around the z-axis of the sensor coordinate system C3) by a posture change amount θ3 (a second posture change amount), thereby disposing the robot 12 to the position PS6 and posture OR3. This posture change amount θ3 is predetermined by the operator (e.g., θ3 = 180°) as an angle greater than the posture change amount θ1 (θ3 > θ1) and is stored in the memory 52.

[0090] In step S32, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD6 of the marker ID using the vision sensor 14 in position PS6 and orientation OR3, and obtains the coordinates (x6, y6) and size IS6 of the intersection point F of the marker ID reflected in the image data JD6.

[0091] The processor 50 then uses the acquired coordinates (x6, y6) and size IS6 and the above equations (1) to (3) to obtain the relative position data (X6, Y6, Z6) of the identifier ID relative to the vision sensor 14 when the image data JD6 was captured. The processor 50 then uses the robot 12 to return the vision sensor 14 to its initial position PS0 and initial posture OR0.

[0092] In step S33, the processor 50 rotates the visual sensor 14 to change the posture of the visual sensor 14. Specifically, the processor 50 first uses the trial measurement position (x T ,y T , z T ) and the relative position data (X0, Y0, Z0) obtained in the above step S12 to determine the posture reference position RP.

[0093] More specifically, the processor 50 determines the posture reference position RP as the tentatively measured position (x) from the origin of the sensor coordinate system C3 in the MIF coordinate system C2 set in the above-mentioned step S11 (ie, the initial position PS0 and the initial posture OR0). T ,y T , z T ) leaves the position of the vector (X0, Y0, Z0) (i.e., the coordinates (x T +X0,y T +Y0,z T +Z0) position).

[0094] When the posture reference position RP is determined in this manner, the posture reference position RP in the MIF coordinate system C2 of the initial position PS0 and the initial posture OR0 is relative to the trial measurement position (x T ,y T , z T ) is the same as the relative position (X0, Y0, Z0) of the identification ID relative to the visual sensor 14 when the image data JD0 is captured in step S12. T ,y T , z T ) is used as a reference to determine the posture reference position RP, thereby enabling the posture reference position RP to be configured near the intersection G of the identifier ID.

[0095] Next, the processor 50 sets the reference coordinate system C5 in the MIF coordinate system C2 at that point in time (i.e., initial position PS0 and initial posture OR0). Specifically, the processor 50 sets the reference coordinate system C5 in the MIF coordinate system C2 so that its origin is located at the posture reference position RP and its posture (the directions of each axis) matches the posture (W, P, R) acquired in step S19. Therefore, the directions of the x-axis, y-axis, and z-axis of the reference coordinate system C5 are parallel to the x-axis, y-axis, and z-axis of the sensor coordinate system C3, respectively.

[0096] Next, the processor 50 determines the direction DR2 (posture change direction) in which the visual sensor 14 is moved in order to change its posture in step S33 as a direction around the x-axis or y-axis of the reference coordinate system C5. The x-axis or y-axis of the reference coordinate system C5 is an axis orthogonal to the direction of the line of sight O and is arranged at the posture reference position RP. As described above, the processor 50 determines the direction DR2 (posture change direction) in which the visual sensor 14 is moved in order to change its posture in step S33 as a direction around the x-axis or y-axis of the reference coordinate system C5. T ,y T , z T ) to determine the posture reference position RP, and determine the posture change direction DR2 based on the reference coordinate system C5 set at the reference position RP.

[0097] Next, the processor 50 operates the robot 12 to rotate the vision sensor 14 from the initial position PS0 and initial posture OR0 in the posture change direction DR2 (a direction around the x-axis or y-axis of the reference coordinate system C5) by a posture change amount θ4 (a second posture change amount), thereby disposing the robot 12 to position PS7 and posture OR4. This posture change amount θ4 is predetermined by the operator (e.g., θ4 = 30°) as an angle greater than the posture change amount θ2 (θ4 > θ2) described above and is stored in the memory 52.

[0098] In step S34, similar to step S12 described above, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position of the marker ID with respect to the vision sensor 14. Specifically, the processor 50 obtains image data JD7 of the marker ID using the vision sensor 14 in position PS7 and orientation OR4, and obtains the coordinates (x7, y7) and size IS7 of the intersection F of the marker ID reflected in the image data JD7.

[0099] Then, the processor 50 uses the obtained coordinates (x7, y7) and size IS7 and the above-mentioned equations (1) to (3) to obtain the relative position data (X7, Y7, Z7) of the identification ID relative to the visual sensor 14 when the image data JD7 is captured.

[0100] In step S35, the processor 50 obtains the main measurement position of the vision sensor 14 based on the relative position data (X0, Y0, Z0), (X6, Y6, Z6) and (X7, Y7, Z7). Here, if the trial measurement position (x T ,y T , z T ) to the exact origin position of the sensor coordinate system C3 and the vector in the plane perpendicular to the z-axis of the sensor coordinate system C3 (i.e., the line of sight O) is set to (ΔX2, ΔY2), then the following formula (6) holds.

[0101] [Formula 4]

[0102]

[0103] In addition, if the posture reference position RP (x T +X0,y T +Y0,z T +Z0) (i.e., the origin position of the reference coordinate system C5 set in step S34) to the exact origin position of the sensor coordinate system C3, and the vector in the direction of the z-axis (i.e., the line of sight O) of the sensor coordinate system C3 is set to ΔZ2, then the following formula (7) holds.

[0104] [Formula 5]

[0105] cosθ4·Y0-sinθ4·(Z0+ΔZ2)=Y7...Equation (7)

[0106] By solving equations (6) and (7), the processor 50 can determine the vector (ΔX2, ΔY2) and the vector ΔZ2 in the MIF coordinate system C2. The vector (ΔX2, ΔY2) represents the exact position of the line of sight O in the MIF coordinate system C2 (in other words, the position of the origin of the sensor coordinate system C3 within a plane perpendicular to the line of sight O). Furthermore, the vector ΔZ2 represents the exact position of the visual sensor 14 (or the origin of the sensor coordinate system C3) in the MIF coordinate system C2 along the line of sight O.

[0107] Based on ΔX2, ΔY2, and ΔZ2, the position (x R 、y R 、z R) as the main measurement position. Thus, in step S35, the processor 50 obtains the main measurement position (x 0, Y 0, Z 0) based on the posture change amounts θ3 and θ4, the relative position data (X0, Y0, Z0) when the image data JD0 is captured before the posture change (i.e., the initial posture OR0), and the relative position data (X6, Y6, Z6) and (X7, Y7, Z7) when the image data JD6 and JD7 are captured after the posture change (i.e., the postures OR3 and OR4). R ,y R , z R ).

[0108] The processor 50 changes the coordinates of the origin of the sensor coordinate system C3 in the MIF coordinate system from the trial measurement position (x T ,y T , z T ) is updated to the main measurement position (x R ,y R , z R ) and stored in the memory 52. ​​The main measurement position (x R ,y R , z R ) highly accurately represents the position of the visual sensor 14 in the MIF coordinate system (specifically, the origin coordinates of the sensor coordinate system C3), and represents the positional relationship between the MIF coordinate system C2 and the sensor coordinate system C3.

[0109] In this way, the sensor coordinate system C3 can be calibrated with respect to the control coordinate system (robot coordinate system C1, MIF coordinate system C2), and the control device 16 can recognize the position and posture of the vision sensor 14 in the control coordinate system. Therefore, the control device 16 can obtain the position of the workpiece in the robot coordinate system C1 based on the image data of the workpiece (not shown) captured by the vision sensor 14, and accurately operate the workpiece using the end effector attached to the fingertip of the robot 12.

[0110] As described above, in the present embodiment, during the trial measurement process of step S2, the processor 50 changes the posture of the visual sensor 14 by the first posture change amounts θ1 and θ2, and estimates the trial measurement position (x) of the visual sensor 14 in the control coordinate system (MIF coordinate system C2). T ,y T , z T ), in the main measurement process of step S3, the posture of the visual sensor 14 is changed by a larger posture change amount θ2, θ4, thereby obtaining the main measurement position (x R ,y R , z R ).

[0111] Assuming that the position of the vision sensor 14 in the control coordinate system is determined during the first measurement without performing the trial measurement or the main measurement, the posture of the vision sensor 14 must be significantly changed by the posture changes θ2 and θ4 during the first measurement. This is because without significantly changing the posture of the vision sensor 14, the measurement accuracy of the position of the vision sensor 14 in the control coordinate system decreases. However, if the posture of the vision sensor 14 is significantly changed during the first measurement, there is a possibility that the marker ID will be out of the field of view of the vision sensor 14 after the posture change, and the marker ID may not be captured.

[0112] Therefore, in this embodiment, the process of measuring the position of the visual sensor 14 in the control coordinate system is divided into a trial measurement process and a main measurement process. In steps S21 and S23 of the trial measurement process, the posture of the visual sensor 14 is changed by relatively small first posture change amounts θ1 and θ2. This can prevent the identification ID from deviating from the field of view of the visual sensor 14 after the posture change, and can estimate the trial measurement position (x T ,y T , z T ).

[0113] Then, in the main measurement process of step S3, the posture of the visual sensor 14 is adjusted based on the trial measurement position (x T ,y T , z T ) The second posture change amount θ3, θ4 is a larger change in the posture change direction DR1, DR2 determined by the method. According to this structure, it is possible to prevent the identification ID from deviating from the field of view of the visual sensor 14 after the posture change, and it is possible to obtain the accurate position (x) of the visual sensor 14 in the control coordinate system (MIF coordinate system C2). R ,y R , z R ).

[0114] In addition, in this embodiment, the processor 50 performs the test measurement based on the position (x T ,y T , z T ) determines the posture reference position RP, and determines the direction around the x-axis or y-axis of the reference coordinate system C5 configured at the posture reference position RP as the posture change direction DR2. With this configuration, it is possible to more effectively prevent the marker ID from deviating from the field of view of the visual sensor 14 during the execution of step S33.

[0115] Then, the processor 50 adjusts the posture reference position RP relative to the test measurement position (x T ,y T , z TThe posture reference position RP is determined so that the relative position of the marker ID (X0, Y0, Z0) matches the relative position (X0, Y0, Z0) of the marker ID relative to the vision sensor 14 when the image data JD0 is captured. With this configuration, the posture reference position RP can be arranged near the intersection G of the marker ID, thereby more effectively preventing the marker ID from deviating from the field of view of the vision sensor 14 when executing step S33.

[0116] In addition, in this embodiment, the processor 50 obtains the relative position data (X n , Y n , Z n ), and based on the relative position data (X n , Y n , Z n ) to obtain the test measurement position (x T ,y T , z T ) and the main measurement position (x R ,y R , z R ). According to this structure, it is not necessary to make the image data JD captured by the visual sensor 14 n By aligning the position of the marker ID (intersection F) in the sensor coordinate system C3 with a predetermined position (e.g., the center), the position of the vision sensor 14 in the control coordinate system (trial measurement position, main measurement position) can be obtained. This allows for faster operation.

[0117] Furthermore, in step S21, the processor 50 may also set a reference coordinate system C4 for the robot coordinate system C1 so that its origin is located at the origin of the robot coordinate system C1. In this case, the processor 50 can also modify the above equations (4) to (7) based on the origin position of the reference coordinate system C4 to determine the trial measurement position and the main measurement position.

[0118] In the above embodiment, the robot coordinate system C1 and the interface coordinate system C2 are exemplified as control coordinate systems. However, other coordinate systems such as the world coordinate system C6, the workpiece coordinate system C7, and the user coordinate system C8 can be set as control coordinate systems. The world coordinate system C6 is a coordinate system that defines the three-dimensional space of the work unit where the robot 12 performs work and is fixed relative to the robot coordinate system C1. The workpiece coordinate system C7 is a coordinate system that defines the position and posture of the workpiece that the robot 12 performs work on in the robot coordinate system C1 (or the world coordinate system C7).

[0119] The user coordinate system C8 is a coordinate system arbitrarily set by the operator to control the robot 12. For example, the operator can set the user coordinate system C8 to a known position and posture of the MIF coordinate system C2. That is, the origin of the user coordinate system C8 in this case is configured at the known coordinates (x C ,y C , z C ).

[0120] As an example, the user coordinate system C8 is set for the MIF coordinate system C2 so that its origin is closer to the center of the light receiving surface (or optical lens) of the imaging sensor of the visual sensor 14 than the origin of the MIF coordinate system C2, that is, the position where the origin of the sensor coordinate system C3 should be configured.

[0121] Here, the position of the center of the light-receiving surface (or optical lens) of the imaging sensor of the vision sensor 14 relative to the center of the mounting surface 34a on which the origin of the MIF coordinate system C2 is located can be estimated based on information such as the specifications of the vision sensor 14 and the mounting position of the vision sensor 14 relative to the robot 12 (wrist flange 34). Alternatively, the operator can obtain the design value of the position of the center of the light-receiving surface of the imaging sensor of the vision sensor 14 relative to the center of the mounting surface 34a from, for example, graphic data (CAD data, etc.) of the vision sensor 14 and the robot 12.

[0122] Referring to such estimated values ​​or design values, the operator sets the coordinates (x C ,y C , z C ), so that the origin of the user coordinate system C8 is arranged at the center of the light receiving surface (or optical lens) of the imaging sensor of the visual sensor 14. In this case, in the above-mentioned step S21, the processor 50 may also set the reference coordinate system C4 in the MIF coordinate system C2 so that its origin is arranged at the origin (x C ,y C , z C ), its posture (direction of each axis) is consistent with the posture (W, P, R) obtained in step S19.

[0123] Furthermore, the processor 50 may also rotate the visual sensor 14 around the z-axis of the reference coordinate system C4 by the movement of the robot 12. In addition, the processor 50 may also rotate the visual sensor 14 around the x-axis or y-axis of the reference coordinate system C4 in step S23. According to this structure, the origin of the reference coordinate system C4 can be arranged at an accurate position (x-axis) close to the origin of the sensor coordinate system C3. R ,y R , z R) position, thus effectively preventing the identification ID from deviating from the field of view of the visual sensor 14 in steps S21 and S23.

[0124] In the above embodiment, the robot 12 is described as moving the visual sensor 14. However, the robot 12 may also move the identification ID relative to the visual sensor 14. Figure 9 . Figure 9 The robot system 10 ′ shown differs from the robot system 10 described above in the configuration of the visual sensor 14 and the identification ID.

[0125] Specifically, in the robot system 10', the visual sensor 14 is fixedly mounted on the upper surface of the structure B. Figure 10 As shown, the identification ID is provided on the mounting surface 34a of the wrist flange 34 of the robot 12. In the robot system 10', the processor 50 of the teaching device 18 can also execute Figure 4 、 Figure 5 、 Figure 7 as well as Figure 8 The position of the visual sensor 14 in the control coordinate system is obtained by the process shown.

[0126] Next, the operation of the robot system 10' will be described. Figure 5 In step S11, the processor 50 causes the robot 12 to operate, positioning the identifier ID (i.e., the wrist flange 34) at the initial position PS0 and initial posture OR0 relative to the vision sensor 14. At this point, the identifier ID enters the field of view of the vision sensor 14. In step S12, the processor 50 captures the identifier ID with the vision sensor 14 to obtain image data JD0 and acquires relative position data (X0, Y0, Z0) of the identifier ID relative to the vision sensor 14.

[0127] In step S13, the processor 50 translates the marker ID from its initial position PS0 and initial posture OR0 in the x-axis direction of the robot coordinate system C1 by a predetermined distance δx. In step S14, the processor 50 captures the marker ID using the vision sensor 14 to obtain image data JD1 and acquires relative position data (X1, Y1, Z1) of the marker ID relative to the vision sensor 14.

[0128] In step S15, the processor 50 translates the marker ID from its initial position PS0 and initial posture OR0 in the y-axis direction of the robot coordinate system C1 by a predetermined distance δy. In step S16, the processor 50 captures the marker ID using the vision sensor 14 to obtain image data JD2 and acquires relative position data (X2, Y2, Z2) of the marker ID relative to the vision sensor 14.

[0129] In step S17, the processor 50 moves the marker ID from the initial position PS0 and initial posture OR0 in the z-axis direction of the robot coordinate system C1 by a predetermined distance δz. In step S18, the processor 50 obtains image data JD3 by photographing the marker ID with the vision sensor 14, and obtains relative position data (X3, Y3, Z3) of the marker ID relative to the vision sensor 14. In step S19, the processor 50 uses the relative position data (X3, Y3, Z3) to obtain the relative position data (X3, Y3, Z3) of the marker ID. n , Y n , Z n )(n=0, 1, 2, 3) to obtain the matrix M1, and obtain the posture data (W, P, R) of the visual sensor 14 from the matrix M1.

[0130] Reference Figure 7 In step S21, the processor 50 rotates the identifier ID to change its posture. Specifically, the processor 50 first sets the reference coordinate system C4 in the MIF coordinate system C2 at that point in time (initial position PS0 and initial posture OR0), with its origin positioned at the origin of the MIF coordinate system C2. Its posture (the directions of each axis) matches the posture (W, P, R) acquired in step S19. The processor 50 then operates the robot 12 to rotate the identifier ID from its initial position PS0 and initial posture OR0 about the z-axis of the reference coordinate system C4 (i.e., the axis parallel to the direction of the line of sight O) by a posture change amount θ1.

[0131] In step S22, the processor 50 operates the vision sensor 14 to capture the marker ID and obtains the relative position data (X4, Y4, Z4) of the marker ID relative to the vision sensor 14. In step S23, the processor 50 operates the robot 12 to rotate the marker ID from its initial position PS0 and initial posture OR0 by a posture change amount θ2 about the x-axis or y-axis of the reference coordinate system C4 (i.e., an axis orthogonal to the direction of the line of sight O).

[0132] In step S24, the processor 50 activates the visual sensor 14 to capture the marker ID and obtains the relative position data (X5, Y5, Z5) of the marker ID relative to the visual sensor 14. In step S25, the processor 50 obtains the test position of the visual sensor 14. Specifically, the processor 50 uses the relative position data (X0, Y0, Z0), (X4, Y4, Z4), and (X5, Y5, Z5) and the above equations (4) and (5) to calculate the vector (ΔX1, ΔY1, ΔZ1) from the origin of the reference coordinate system C4 in the MIF coordinate system C2 to the origin of the unknown sensor coordinate system C3.

[0133] Then, the processor 50 obtains the position of the visual sensor 14 (the origin of the sensor coordinate system C3) from the vector (ΔX1, ΔY1, ΔZ1) as the coordinate (x T ,y T , z T ), obtain the coordinates (x T ,y T , z T ) is transformed into the coordinates of the robot coordinate system C1 (x T ',y T ', z T ') is the trial measurement position of the visual sensor 14 in the robot coordinate system C1. The trial measurement position (x T '、y T '、z T ') represents the estimated position of the vision sensor 14 in the robot coordinate system C1.

[0134] Reference Figure 8 In step S31, the processor 50 changes the posture of the marker ID by rotating the marker ID. Specifically, the processor 50 determines the direction DR1 (posture change direction) in which the marker ID is moved in step S31 to change its posture as the direction around the z-axis of the sensor coordinate system C3 whose origin position was updated in step S25.

[0135] The origin position of the sensor coordinate system C3 in the robot coordinate system C1 at this time point is the test measurement position (x T ',y T ', z T '), so the z-axis of the sensor coordinate system C3 is configured at the test measurement position (x T ',y T ', z T ') is parallel to the direction of the line of sight O. In this way, the processor 50 is based on the measured position (x T '、y T '、z T Next, the processor 50 operates the robot 12 to rotate the identifier ID from the initial position PS0 and initial posture OR0 in the posture change direction DR1 (the direction around the z-axis of the sensor coordinate system C3) by a posture change amount θ3 (a second posture change amount).

[0136] In step S32, the processor 50 operates the visual sensor 14 to capture the marker ID and obtains relative position data (X6, Y6, Z6) of the marker ID relative to the visual sensor 14. In step S33, the processor 50 rotates the marker ID to change its posture.

[0137] Specifically, the processor 50 first determines the direction DR2 (posture change direction) in which the marker ID is moved in order to change its posture in step S33 as the direction around the x-axis or y-axis of the sensor coordinate system C3 whose origin position is updated in step S25. The origin position of the sensor coordinate system C3 in the robot coordinate system C1 at this time point is the test measurement position (x T ',y T ', z T '), so the x-axis or y-axis of the sensor coordinate system C3 is configured at the test measurement position (x T ,y T , z T ) is an axis perpendicular to the line of sight O.

[0138] In this way, the processor 50 is based on the trial measurement position (x T '、y T '、z T Next, the processor 50 operates the robot 12 to rotate the identifier ID from the initial position PS0 and initial posture OR0 in the posture change direction DR2 (a direction around the x-axis or y-axis of the sensor coordinate system C3) by a posture change amount θ4 (a second posture change amount).

[0139] In step S34 , the processor 50 operates the visual sensor 14 to capture the marker ID and obtains relative position data ( X7 , Y7 , Z7 ) of the marker ID relative to the visual sensor 14 .

[0140] Specifically, the processor 50 uses the relative position data (X0, Y0, Z0), (X6, Y6, Z6) and (X7, Y7, Z7) and the above equations (6) and (7) to calculate the test position (x0, Y0, Z0) in the robot coordinate system C1 obtained in step S25. T '、y T '、z T ') to calculate the accurate vector (ΔX2, ΔY2, ΔZ2) to the origin of the sensor coordinate system C3. Then, the processor 50 obtains the position of the vision sensor 14 (the origin of the sensor coordinate system C3) in the robot coordinate system C1 based on the vector (ΔX2, ΔY2, ΔZ2) as the main measurement position (x R ',y R ', z R ').

[0141] Thus, in the robot system 10', the processor 50 obtains the test measurement position (x T ',yT ', z T ') and the main measurement position (x R ',y R ', z R According to this embodiment, similarly to the above-mentioned embodiment, it is possible to prevent the marker ID from deviating from the visual field of the visual sensor 14 in steps S21, S23, S31, and S33.

[0142] In addition, Figure 8 In the process shown, the processor 50 may also use the relative position data (X0, Y0, Z0) and (X6, Y6, Z6) and the above formula (6) after step S32 to calculate the vector (ΔX2, ΔY2), and obtain the main measurement position (x2) of the line of sight O in the MIF coordinate system C2 based on the vector (ΔX2, ΔY2). R ,y R Then, the processor 50 calculates the main measurement position (x R ,y R ), the test measurement position (x T ,y T , z T ) is updated to the trial measurement position (x R ,y R , z T ).

[0143] Then, in Figure 8 In step S33, the processor 50 uses the updated trial measurement position (x R ,y R , z T ) and the relative position data (X0, Y0, Z0) obtained in step S12 to determine the posture reference position RP. Specifically, the processor 50 determines the posture reference position RP as the position from the updated trial measurement position (x R ,y R , z T ) leaves the position of the vector (X0, Y0, Z0) (i.e., the coordinates (x R +X0,y R +Y0,z T +Z0) position).

[0144] According to this structure, the updated trial measurement position (x R ,y R , z T ) in the coordinates (x R ,y R) represents the exact position of the line of sight O in the MIF coordinate system, so the posture reference position RP can be more accurately set at the intersection F of the marker ID. Therefore, it is possible to more effectively prevent the marker ID from deviating from the visual field of the visual sensor 14 in step S33.

[0145] In addition, in the above embodiment, the case where steps S21, S23, S31 and S33 are executed starting from the initial position PS0 and the initial posture OR0 is described, but the present invention is not limited thereto. Alternatively, at the start time of step S3 or S4, the visual sensor 14 may be arranged at a second initial position PS0 different from the initial position PS0 and the initial posture OR0. 0_2 and the second initial posture OR 0_2 To capture the image of the ID, the relative position data (X) is obtained based on the image data. 0_2 、Y 0_2 、Z 0_2 In this case, the processor 50 performs the following operations based on the relative position data (X) in step S25 or S35. 0_2 , Y 0_2 , Z 0_2 ) obtain the trial measurement position or the main measurement position.

[0146] Furthermore, in the above-mentioned embodiment, the processor 50 is configured to process the image based on the relative position (X n , Y n , Z n ) to obtain the position of the visual sensor 14 in the control coordinate system. However, the concept of the present invention can also be applied to a method of obtaining the position of the visual sensor 14 in the control coordinate system by, for example, the methods described in Patent Documents 1 and 2.

[0147] Next, another method of obtaining the position of the visual sensor 14 is described. First, the processor 50 moves the visual sensor 14 or the identification ID through the robot 12, and uses the visual sensor 14 to capture the identification ID, and executes the captured image data JD n The processor 50 then obtains the coordinates CD1 (initial position) of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at the time when the alignment process PP is completed.

[0148] Next, after translating the vision sensor 14 or the identification ID from its initial position, the processor 50 uses the vision sensor 14 to capture the identification ID again, executing the aforementioned alignment process PP to obtain the coordinates CD2 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this point. The processor 50 obtains the direction (i.e., posture) of the line of sight O of the vision sensor 14 in the robot coordinate system C1 from the coordinates CD1 and CD2.

[0149] Next, as a test measurement process, the processor 50 rotates the vision sensor 14 or the marker ID from its initial position by a posture change amount θ1 about an axis parallel to the direction of the acquired line of sight O. The vision sensor 14 then images the marker ID and executes the aforementioned alignment process PP. The processor 50 then obtains the coordinates CD3 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this point. Based on the coordinates CD1 and CD3, the processor 50 determines the position TP1 of the line of sight O in the robot coordinate system C1.

[0150] Next, as a trial measurement process, the processor 50 rotates the visual sensor 14 or the identification ID from the initial position to the direction of the axis arranged at position TP1 and perpendicular to the line of sight O by a posture change amount θ2, and then uses the visual sensor 14 to photograph the identification ID, executes the above-mentioned alignment process PP, and obtains the coordinate CD4 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time.

[0151] Then, the processor 50 calculates the position TP2 of the vision sensor 14 (the origin of the sensor coordinate system C3) in the robot coordinate system C1 in the direction along the line of sight O based on the coordinates CD1 and CD4. Based on these positions TP1 and TP2, the test measurement position (x T '、y T '、z T ').

[0152] Next, as a main measurement process, the processor 50 determines the posture change direction to be around the position (x T ',y T ', z T After rotating the vision sensor 14 or the marker ID from its initial position in the direction of the posture change by the posture change amount θ3 (>θ1) along an axis parallel to the direction of the line of sight O, the vision sensor 14 is used to capture an image of the marker ID, and the aforementioned alignment process PP is executed. The processor 50 then obtains the coordinates CD5 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time, and calculates the position TP3 of the line of sight O in the robot coordinate system C1 based on the coordinates CD1 and CD5.

[0153] Next, as a main measurement process, the processor 50 determines the posture change direction to be around the position (x T ',y T ', z T After rotating the visual sensor 14 or the identification ID from its initial position in the direction of the posture change by the posture change amount θ4 (>θ2) about an axis perpendicular to the line of sight O, the aforementioned alignment process PP is executed. The processor 50 then obtains the coordinates CD6 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time.

[0154] Then, the processor 50 calculates the position TP4 of the vision sensor 14 (the origin of the sensor coordinate system C3) in the robot coordinate system C1 in the direction along the line of sight O based on the coordinates CD1 and CD6. Based on these positions TP3 and TP4, the main measurement position (x R '、y R '、z R ').

[0155] In this method, the processor 50 also obtains the position of the vision sensor 14 in the control coordinate system based on the image data of the identification ID captured by the vision sensor 14 before the posture change (image data captured during the alignment process PP for determining the initial position) and the image data of the identification ID captured by the vision sensor 14 after the posture change (image data captured during the alignment process PP for determining coordinates CD3, CD4, and CD5). This method also allows the processor 50 to obtain the position of the vision sensor 14 in the control coordinate system (trial measurement position, main measurement position).

[0156] In the above embodiment, the teaching device 18 acquires the data of the position and posture of the visual sensor 14 in the control coordinate system. However, the control device 16 may also acquire the data of the position and posture of the visual sensor 14 in the control coordinate system. In this case, the processor 40 of the control device 16 executes the computer program CP. Figure 4 The process shown.

[0157] Alternatively, a device other than the teaching device 18 and the control device 16 may obtain data on the position and posture of the visual sensor 14 in the control coordinate system. In this case, the other device includes a processor that executes the computer program CP. Figure 4 The process shown.

[0158] Furthermore, the identification ID is not limited to an artificial pattern as in the above-described embodiment. For example, any visually identifiable visual feature such as a hole, edge, concave-convex portion, or tip formed in the retaining structure B or the wrist flange 34 may be used as an identification. Furthermore, the robot 12 is not limited to a vertical multi-joint robot. It may be any type of robot such as a horizontal multi-joint robot or a parallel robot that enables relative movement of the visual sensor 14 and the identification ID. The present disclosure has been described above through the embodiments, but the above-described embodiments do not limit the invention to the scope of the requested patent protection.

[0159] Explanation of symbols

[0160] 10.10' Robotic System

[0161] 12 robots

[0162] 14 Vision Sensors

[0163] 16 control devices

[0164] 18 teaching devices.

Claims

1. A device for obtaining the position of a vision sensor in a control coordinate system for controlling a robot that moves the vision sensor and a marker relative to each other, characterized in that: The device comprises a processor, The processor performs the following processing: causing the robot to move so that the posture of the visual sensor or the marker changes by a first posture change amount; obtaining a position of the visual sensor in the control coordinate system as a trial measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the first posture change amount; causing the robot to move so that the posture changes in a posture change direction determined based on the trial measurement position by a second posture change amount that is greater than the first posture change amount; as well as The position of the vision sensor in the control coordinate system is acquired as a main measurement position based on the image data of the marker captured by the vision sensor before and after the posture is changed by the second posture change amount.

2. The device according to claim 1, characterized in that The processor performs the following processing: Pre-acquire the direction of the line of sight of the visual sensor in the control coordinate system; In order to change the posture by the first posture change amount, the robot is operated so that the visual sensor or the marker is rotated in a direction around an axis parallel to the direction of the line of sight; determining a direction around the parallel axis disposed at the trial measurement position as the posture change direction; In order to change the posture by the second posture change amount, the robot is moved so that the visual sensor or the marker is rotated in the posture change direction; as well as The position of the line of sight in the control coordinate system is acquired as the trial measurement position and the main measurement position.

3. The device according to claim 1, characterized in that The processor performs the following processing: Pre-acquire the direction of the line of sight of the visual sensor in the control coordinate system; In order to change the posture by the first posture change amount, the robot is operated so that the visual sensor or the marker is rotated in a direction around an axis orthogonal to the direction of the line of sight; determining a direction around the orthogonal axis arranged at a posture reference position determined based on the trial measurement position as the posture change direction; In order to change the posture by the second posture change amount, the robot is moved so that the visual sensor or the marker is rotated in the posture change direction; as well as The position in the direction of the line of sight of the vision sensor in the control coordinate system is acquired as the trial measurement position and the main measurement position.

4. The device according to claim 3, characterized in that The processor performs the following processing: obtaining, based on the image data captured by the visual sensor before the posture is changed by the second posture change amount, a relative position of the marker with respect to the visual sensor when the image data is captured; as well as The posture reference position is determined based on the trial measurement position so that the acquired relative position is the same as the relative position of the posture reference position with respect to the trial measurement position.

5. The device according to any one of claims 1 to 4, characterized in that The visual sensor has: an imaging sensor that receives an image of a subject; and an optical lens for enabling the imaging sensor to focus on the subject image; The processor performs the following processing: Obtaining a relative position of the marker with respect to the visual sensor when the image data was captured based on the position of the marker in the image data, the size of the marker reflected in the image data, the size of the marker in real space, the focal length of the optical lens, and the size of the imaging sensor; acquiring the trial measurement position based on the first posture change amount, the relative position when the image data is captured before the posture is changed by the first posture change amount, and the relative position when the image data is captured after the posture is changed by the first posture change amount; as well as The main measurement position is acquired based on the second posture change amount, the relative position when the image data is captured before the posture is changed by the second posture change amount, and the relative position when the image data is captured after the posture is changed by the second posture change amount.

6. The device according to any one of claims 1 to 5, characterized in that The device is a teaching device or a control device of the robot.

7. A robot system, characterized in that: have: Vision sensors; a robot that moves the vision sensor and the marker relative to each other; and The device according to any one of claims 1 to 6.

8. A method for obtaining the position of a vision sensor in a control coordinate system for controlling a robot that moves the vision sensor and a marker relative to each other, characterized in that: The processor performs the following processing: causing the robot to move so that the posture of the visual sensor or the marker changes by a first posture change amount; obtaining a position of the visual sensor in the control coordinate system as a trial measurement position based on image data of the marker captured by the visual sensor before and after the posture is changed by the first posture change amount; causing the robot to move so that the posture changes in a posture change direction determined based on the trial measurement position by a second posture change amount that is greater than the first posture change amount; as well as The position of the vision sensor in the control coordinate system is acquired as a main measurement position based on the image data of the marker captured by the vision sensor before and after the posture is changed by the second posture change amount.

9. A computer program, characterized in that The processor is caused to execute the method of claim 8.

Images