Camera position shift measuring device and method

Abstract

A position shift measuring device of a camera measures a position shift of a hand camera attached to a robot arm using an external camera, and includes a gauge member provided above the external camera and having a first measurement hole penetrating therethrough. The position shift measuring device acquires a position of the first measurement hole by capturing the gauge member using the external camera, and acquires the position of the first measurement hole by capturing the gauge member using the hand camera after moving the hand camera to the acquired position. Further, the position shift measuring device acquires a position shift of the camera by capturing a fingertip of the robot using the external camera after moving the fingertip to the acquired position of the first measurement hole.

Description

TECHNICAL FIELD

[0001] The present specification discloses a position offset measurement device and a position offset measurement method for a camera. BACKGROUND

[0002] Conventionally, a robot system is proposed which is provided with a robot having a robot arm and a robot camera, and an external camera which is provided separately from the robot, and the robot system calculates correction data of the external camera by capturing a correction pattern using the robot camera whose correction has been completed in advance and capturing the correction pattern using the external camera (for example, refer to Patent Literature 1). The system captures the correction pattern using the robot camera and the external camera respectively to acquire pattern images respectively, and calculates the correction data of the external camera based on the acquired pattern images and known correction data for the robot camera.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2019-69493 SUMMARY OF THE INVENTION

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] However, in the technology described in Patent Literature 1 described above, although the case of correcting the external camera is described, there is no mention of the case of correcting the robot camera.

[0008] The main object of the present disclosure is to appropriately measure the offset of a camera mounted on an arm of a robot.

[0009] TECHNICAL SOLUTION FOR SOLVING THE PROBLEMS

[0010] In order to achieve the main object described above, the present disclosure adopts the following technical solution.

[0011] A camera position shift measuring device of the present disclosure measures a position shift of a hand-held camera in a robot system that has a robot having a mechanical arm and the hand-held camera attached to the mechanical arm, and an external camera disposed outside the robot, wherein the camera position shift measuring device has a gauge member disposed above the external camera and having a first measurement hole that penetrates vertically, a first position acquisition section that photographs the gauge member using the external camera and acquires a position of the first measurement hole from a photographed image of the first measurement hole, a second position acquisition section that moves the hand-held camera to the position acquired by the first position acquisition section and photographs the gauge member using the hand-held camera, and acquires a position of the first measurement hole from a photographed image of the first measurement hole, and a position shift acquisition section that moves a fingertip of the robot to the position of the first measurement hole acquired by the second position acquisition section and photographs the fingertip of the robot using the external camera, and acquires a position shift of the camera from a photographed image.

[0012] A camera position shift measuring device of the present disclosure measures a position shift of a hand-held camera in a robot system that has a robot having a mechanical arm and the hand-held camera attached to the mechanical arm, and an external camera disposed outside the robot, wherein the camera position shift measuring device has a gauge member disposed above the external camera and having a first measurement hole that penetrates vertically, a first position acquisition section that photographs the gauge member using the external camera and acquires a position of the first measurement hole from a photographed image of the first measurement hole, a second position acquisition section that moves the hand-held camera to the position acquired by the first position acquisition section and photographs the gauge member using the hand-held camera, and acquires a position of the first measurement hole from a photographed image of the first measurement hole, and a position shift acquisition section that moves a fingertip of the robot to the position of the first measurement hole acquired by the second position acquisition section and photographs the fingertip of the robot using the external camera, and acquires a position shift of the camera from a photographed image. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is an appearance perspective view of a robot system.

[0014] Figure 2 is a block diagram showing an electrical connection relationship between a robot and a control device.

[0015] Figure 3 is an explanatory view explaining a world coordinate system.

[0016] Figure 4 is an explanatory view explaining a base coordinate system, a camera coordinate system, and a mechanical interface coordinate system.

[0017] Figure 5 is an explanatory view showing a hand-held camera position measurement process.

[0018] Figure 6 is an appearance view of the marked end effector.

[0019] Figure 7 is a flowchart showing a workpiece camera position measurement process for a hand camera position measurement.

[0020] Figure 8 is an explanatory view showing a case where the workpiece camera position is measured.

[0021] Figure 9 is an explanatory view showing an offset amount of the workpiece camera position.

[0022] Figure 10 is an explanatory view showing a case where the hole gauge is mounted.

[0023] Figure 11 is a flowchart showing an example of a hole position measurement process for a hand camera position measurement.

[0024] Figure 12 is an explanatory view showing a case where a three-point hole of a hole gauge is imaged by a workpiece camera.

[0025] Figure 13 is a flowchart showing an example of a hand camera angle measurement process.

[0026] Figure 14 is an explanatory view showing a case where a hole gauge is imaged by a hand camera.

[0027] Figure 15 is an explanatory view showing phases of a three-point hole measured by a workpiece camera and a hand camera, respectively.

[0028] Figure 16 is a flowchart showing an example of a hand camera position measurement process.

[0029] Figure 17 is an explanatory view showing a case where a marked end effector is imaged by a workpiece camera.

[0030] Figure 18 is an explanatory view showing a distance from a hand camera center to a center hole.

[0031] Figure 19 is an explanatory view showing a relationship between a marked pin position (average value), a hole center position for a hand camera position measurement, and a position offset amount. DETAILED DESCRIPTION

[0032] Next, a mode for implementing the present disclosure will be described with reference to the drawings.

[0033] Figure 1 is an appearance perspective view of a robot. Figure 2is a block diagram showing an electrical connection relationship of the robot and the control device.

[0034] The robot system 1 is a work robot system that picks up a workpiece W (component) supplied from a workpiece supply section P and mounts it to a substrate S. As shown in Figure 1 2 The robot system 1 is provided with a work table 2, a robot 10, a hand camera 60 mounted to a robot arm 20 of the robot 10, a workpiece camera 70 as an external camera, and a control device 90 that controls the robot 10. The work table 2 is provided with a main pole 3a extending in the vertical direction and a sub pole 3b extending in the vertical direction in a manner separated from the main pole 3a.

[0035] In the present embodiment, the robot 10 is a SCARA robot provided with a base 11 and a robot arm 20.

[0036] The base 11 is fixed to the work table 2 and supports the base end side of the robot arm 20. The robot arm 20 is provided with a first arm 21, a first arm drive section 30, a second arm 22, a second arm drive section 40, a shaft 23, and a shaft drive section 50. The base end section of the first arm 21 is linked to the base 11 via a first joint shaft J1, and the first arm 21 is configured to be able to rotate (horizontal rotation) in the horizontal plane with respect to the base 11 by rotation of the first joint shaft J1. The base end section of the second arm 22 is linked to the front end section of the first arm 21 via a second joint shaft J2, and the second arm 22 is configured to be able to rotate (horizontal rotation) in the horizontal plane with respect to the first arm 21 by rotation of the second joint shaft J2. The shaft 23 is linked to the front end section of the second arm 22 via a third joint shaft J3 and is configured to be able to rotate around the axis of the third joint shaft J3 with respect to the second arm 22 and to be able to ascend and descend in the axial direction of the third joint shaft J3. In the robot system 1 of the present embodiment, a workpiece holding section 24 that picks up and holds a workpiece W is provided as an end effector at the front end of the shaft 23. In addition, as the workpiece holding section 24, for example, a suction nozzle that sucks a workpiece W by negative pressure, a mechanical chuck that holds a workpiece W by a pair of jaws, an electromagnetic chuck that holds a workpiece W by an electromagnet, or the like can be cited.

[0037] The first arm drive section 30 is provided with a motor 32 and an encoder 34. The rotation axis of the motor 32 is linked to the first joint shaft J1 via a reducer not shown. The first arm drive section 30 transmits torque to the first joint shaft J1 via the reducer by driving the motor 32, thereby causing the first arm 21 to rotate with the first joint shaft J1 as a fulcrum. The encoder 34 is installed to the rotation axis of the motor 32 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 32.

[0038] ​The second arm drive unit 40, like the first arm drive unit 30, includes a motor 42 and an encoder 44. The rotation shaft of the motor 42 is connected to the second joint shaft J2 via a reducer (not shown). The second arm drive unit 40 drives the motor 42, and the torque transmitted to the second joint shaft J2 via the reducer causes the second arm 22 to rotate about the second joint shaft J2 as a fulcrum. The encoder 44 is mounted on the rotation shaft of the motor 42 and is configured as a rotary encoder to detect the rotational displacement of the motor 42.

[0039] The shaft drive unit 50 includes motors 52a and 52b and encoders 54a and 54b. The rotating shaft of motor 52a is connected to shaft 23 via a belt (not shown). The shaft drive unit 50 drives motor 52a to rotate shaft 23. The rotating shaft of motor 52b is connected to shaft 23 via a ball screw mechanism (not shown). The shaft drive unit 50 drives motor 52b and uses the ball screw mechanism to convert the rotational motion of motor 52b into linear motion, thereby causing shaft 23 to move up and down. Encoder 54a is configured as a rotary encoder to detect the rotational displacement of shaft 23. Encoder 54b is configured as a linear encoder to detect the vertical position of shaft 23.

[0040] like Figure 2 As shown, the control device 90 includes a CPU 91, a ROM 92 for storing processing programs, a RAM 93 as working memory, a storage device 94 such as an HDD or SSD, and an input / output interface (not shown). Position signals from encoders 34, 44, 54a, and 54b, and image signals from the handheld camera 60 are input to the control device 90 via the input / output interface. Drive signals for motors 32, 42, 52a, and 52b are output from the control device 90 via the input / output interface.

[0041] A handheld camera 60 is mounted on the front end of the second arm 22. The handheld camera 60 captures an image of the workpiece W at the workpiece supply unit P from above and outputs the image to the control device 90. The control device 90 processes the captured image to identify the position of the workpiece W. Furthermore, since the handheld camera 60 is mounted on the second arm 22, in this embodiment, the robotic arm 20 cannot move the handheld camera 60 in the vertical direction.

[0042] A workpiece camera 70 is positioned between the workpiece supply section P and the base plate S on the worktable 2. The workpiece camera 70 is housed in a housing 71 with a rectangular opening 71a at the top. The workpiece camera 70 captures images of the workpiece W held by the workpiece holding section 24 of the robot 10 from below and outputs these images to the control device 90. The control device 90 processes the captured images to determine whether the workpiece W is properly held in the workpiece holding section 24.

[0043] Next, the operation of the robot system 1 configured as described will be explained. The operation of the robot system 1 includes a workpiece position recognition operation, a picking operation, a picking confirmation operation, and an installation operation. In the workpiece position recognition operation, the position of the workpiece W is identified by taking a picture of the workpiece W using a handheld camera 60. In the picking operation, the fingertip (workpiece holding part 24) of the robot system 1 is moved to the identified position of the workpiece W to pick it up. In the picking confirmation operation, the workpiece W picked up by the fingertip is moved above the workpiece camera 70 and the workpiece camera 70 takes a picture of the workpiece W to confirm whether the picking status of the workpiece W is good. In the installation operation, the picked-up workpiece W is installed on the substrate S.

[0044] Here, the coordinate system of robot system 1 will be explained. In this embodiment, as... Figure 3 , Figure 4 As shown, the coordinate system of robot system 1 has a world coordinate system ∑ w , base coordinate system ∑ b Camera coordinate system ∑ c and mechanical interface coordinate system ∑ m In the world coordinate system ∑ w In the base coordinate system, the origin is set at the front end of the main rod 3a, the X-axis is set in the direction passing through the front ends of the main rod 3a and the secondary rod 3b, the Z-axis is set in the vertical direction, and the Y-axis is set in the direction orthogonal to the X-axis and Z-axis. b In this context, the origin is set at the bottom surface of the base 11 of robot 10, and the X-axis, Y-axis, and Z-axis are set to their respective directions relative to the world coordinate system ∑. w The corresponding axes are consistent. In the mechanical interface coordinate system ∑ m In this context, the origin is set at the fingertip of the robotic arm 20, and the X, Y, and Z axes are set to the directions relative to the world coordinate system ∑. w The corresponding axes are consistent. In the camera coordinate system ∑ c In this system, the origin is set at the bottom surface of the second arm 22 and the center of the third joint axis J3. The X-axis, Y-axis, and Z-axis are set to their directions relative to the world coordinate system ∑. w The corresponding axes are consistent. World coordinate system ∑ w , base coordinate system ∑ b Camera coordinate system ∑ c and mechanical interface coordinate system ∑ m They can be converted to each other using a transformation matrix.

[0045] In the workpiece position recognition operation, the CPU 91 of the control device 90 first sets the base 11 (for taking pictures of the workpiece W supplied from the workpiece supply unit P) to be photographed. Figure 4 The base coordinate system shown is ∑ b The target position X observed by the handheld camera at 60 degrees. btag Ybtag Z btag Next, the CPU91 measures the target position X of the handheld camera 60. btag Y btag Z btag Solve the inverse kinematics to calculate the position of the handheld camera at 60° relative to the target position X. btag Y btag Z btag The angle θ of the first joint axis J1 is consistent. J1 The angle θ of the second joint axis J2 J2 .

[0046] Next, CPU91 considers error factors such as the torsion and deflection of the first arm 21 and the deflection of the second arm 22, based on the angle θ of the first joint axis J1. J1 The angle θ of the second joint axis J2 J2 Solve the forward kinematics to calculate the estimated position X of the handheld camera at 60°. best Y best Z best That is, assuming that the angles θ of the first and second joint axes J1 and J2 are calculated through inverse kinematics... J1 θ J2 The angle command value is used to control the position of the handheld camera 60 relative to the target position X. btag Y btag Z btag Under consistent conditions, CPU91 calculates the error caused by the torsion and deflection of the first arm 21 and the deflection of the second arm 22, thus affecting the distance from the target position X. btag Y btag Z btag The deviation position is the estimated position X best Y best Z best .

[0047] CPU91 calculates the estimated position X best Y best Z best Then, by obtaining the calculated estimated position X best Y best Z best With target position X btag Y btag Z btag The position offset Δ is calculated from the difference between them. X b、Δ Y b、Δ Z b. Next, CPU91 sets the target position X... btag Y btag Z btag Offset position offset ΔX b、Δ Y b、Δ Z Based on the quantity b, solve the inverse kinematics, and thereby calculate the kinematics used to move the handheld camera 60 to the target position X. btag Y btag Z btag The angle command value θ of the first joint axis J1 J1 * Angle command value θ for the second joint axis J2 J2 Then, CPU91 controls the corresponding motors 31 and 32 through feedback control, so that the angle of the first joint axis J1 detected by encoder 34 matches the calculated angle command value θ. J1 * Consistent, and ensuring that the angle of the second joint axis J2 detected by encoder 44 matches the calculated angle command value θ. J2 *Consistent.

[0048] Next, the CPU 91 uses the handheld camera 60 to photograph the workpiece W supplied from the workpiece supply unit P, and performs image processing on the photographed image to identify the position of the workpiece W. Then, the CPU 91 transitions to a picking action, in which the target position X of the fingertip (workpiece holding unit 24) used to pick up the workpiece W is determined. btag Y btag Set the position of the identified workpiece W, and move the fingertip to the target position X. btag Y btag Z btag To pick up the target location X. btag Y btag The setting is based on the identified position of workpiece W. Target position Z btag The settings are based on the pre-input height information of the workpiece W.

[0049] During the pickup action, the CPU91 first uses inverse kinematics calculations to move the fingertip to the set target position (X). btag Y btag Z btag The angle θ of the first joint axis J1) J1 The angle θ of the second joint axis J2 J2 Angle θ of axis 23 J4 The lifting position Z of shaft 23 s Next, CPU91 considers error factors such as the torsion and deflection of the first arm 21 and the deflection of the second arm 22, based on the calculated θ. J1 θ J2 θ J4 Z s The estimated position X of the fingertip is calculated using forward kinematics. best Y best Zbest Next, CPU91 obtains the estimated position X of the fingertip. best Y best Z best With target position X btag Y btag Z btag The position offset Δ is calculated from the difference between them. X b、Δ Y b、Δ Z b. Next, CPU91 sets the target position X... btag Y btag Z btag Offset position offset Δ X b、Δ Y b、Δ Z Based on the quantity b, solve the inverse kinematics, and then calculate the kinematics used to move the fingertip to the target position X. btag Y btag Z btag The angle command value θ of the first joint axis J1 J1 * The angle command value θ of the second joint axis J2 J2 * Angle command value θ for axis 23 J4 * The lifting position command value Z of shaft 23 s Then, CPU91 controls the corresponding motors 31, 32, 33a, and 33b through feedback control based on each instruction value.

[0050] Thus, robot system 1 uses a handheld camera 60 mounted on robotic arm 20 to photograph workpiece W to identify its position, and then moves the fingertip of robotic arm 20 to the identified position to pick up workpiece W. Therefore, if the positional relationship between robot system 1 and handheld camera 60 is not properly determined, the position of workpiece W cannot be accurately identified using the handheld camera 60. That is, the fingertip of robot system 1 cannot be used to properly pick up workpiece W. Therefore, in this embodiment, the positional offset of handheld camera 60 relative to a designed position is measured in advance, and the measured positional offset is reflected in the control of the workpiece position identification action, thereby enabling accurate identification of the position of workpiece W. Furthermore, it is assumed that robot system 1 has been calibrated.

[0051] Figure 5 This is an explanatory diagram illustrating the process of determining the position of a handheld camera. In step S10, the marked end effector EE (marking pin) is installed on the front end of shaft 23. Figure 6 This is an external view of the marked end effector EE. The marked end effector EE is a pin-shaped component extending vertically when mounted on shaft 23. A mark M is provided at the front end of the pin of the marked end effector EE.

[0052] In step S20, the robot system 1 is caused to execute a hand camera position measurement workpiece camera position measurement process of measuring the position (hand camera position measurement workpiece camera position X w , Y wwc , θ wwc ) of the workpiece camera 70 in the world coordinate system ∑ wwc Figure 7 is a flowchart showing an example of the hand camera position measurement workpiece camera position measurement process executed by the CPU 91 of the control device 90. In the hand camera position measurement workpiece camera position measurement process, the CPU 91 first accepts input of the control point Zm (intrinsic value) of the marked end effector EE from the operator (step S100). The control point Zm is the distance between the front end of the shaft 23 and the front end (mark M) of the marked end effector EE. The coordinates of the front end of the marked end effector EE are set at a position that is Zm away downward in the Z-axis direction from the origin of the mechanical interface coordinate system ∑ m . Next, the CPU 91 stands by until the workpiece camera position measurement button is pressed by the operator (step S110). When it is determined that the workpiece camera position measurement button is pressed, the CPU 91 causes the marked end effector EE to move upward of the workpiece camera 70 (step S120), and measures the position (workpiece camera position X wwc , Y wwc , θ wwc ) of the workpiece camera 70 by capturing the mark M provided on the front end face of the marked end effector EE with the workpiece camera 70 (step S130). Figure 8 is an explanatory diagram showing the case where the hand camera position measurement workpiece camera position is measured. As shown in the drawing, the CPU 91 moves the front end of the marked end effector EE in the X-axis direction and the Y-axis direction, respectively, to pass through the designed center position of the workpiece camera 70, while capturing the mark M with the workpiece camera 70. The CPU 91 acquires the movement trajectories of the mark M in the X-axis direction and the Y-axis direction as appearing in the obtained captured images. Then, as shown in Figure 9 , the CPU 91 calculates the difference between the image center of the captured image and the intersection point of each of the acquired movement trajectories, respectively, in the X-axis direction and the Y-axis direction of the captured image, thereby finding the position offset amounts Δx, Δy of the workpiece camera 70. Then, as shown in Figure 9 , the CPU 91 calculates the rotational offset amount Δθ of the workpiece camera 70 from the difference in the rotational direction between the X-axis direction of the captured image and the X-axis direction of the movement trajectory. Then, the CPU 91 finds the position after the designed center position of the workpiece camera 70 is offset by the position offset amounts Δx, Δy as the workpiece camera position X wwc , Y wwc ​and the angle of the workpiece camera 70 after being offset in the rotational direction by the rotational offset amount Δθ is obtained as the workpiece camera position θ wwc . After thus determining the workpiece camera positions X wwc , Y wwc , θ wwc , the CPU 91 registers the determined workpiece camera positions X wwc , Y wwc , θ wwc to the storage device 94 (step S140) and ends the process.

[0053] In step S30, the hole gauge 80 for hand camera position determination is installed in the opening portion 71a provided in the upper portion of the housing box 71. Figure 10 is an explanatory view showing the state of installation of the hole gauge. As shown in the drawing, the hole gauge 80 is fixed by the fixing member 82 in a state of being fitted in the opening portion 71a of the housing box 71. The hole gauge 80 is formed with a center hole 81a located directly above the workpiece camera 70 and passing through vertically and three-point holes 81b located around the center hole 81a at intervals in the circumferential direction and passing through vertically. Note that the number of holes of the three-point holes 81b is not limited to three, and can be two or more than four.

[0054] In step S40, the robot system 1 is caused to execute the hole position determination process for hand camera position determination to determine the position (center hole positions X w , Y wwv , Z wwv ) of the center hole 81a of the hole gauge 80 in the world coordinate system Σ wwv by the workpiece camera 70. Figure 11 is a flowchart showing an example of the hole position determination process for hand camera position determination executed by the CPU 91 of the control device 90. In the hole position determination process for hand camera position determination, the CPU 91 first stands by until the hand camera position determination button is pressed by the operator (step S200). When it is determined that the hand camera position determination button is pressed, the CPU 91 captures the hole gauge 80 by the workpiece camera 70 and determines the distances a m of the three-point holes 81b of the hole gauge 80 from the obtained captured image (step S210). Figure 12 is an explanatory view showing the state of capturing the three-point holes of the hole gauge by the workpiece camera. The determination of the distances a m of the three-point holes 81b is performed by determining the distances between the holes of the three-point holes 81b appearing in the captured image. Then, the CPU 91 calculates the center hole position Z m for hand camera position determination from the determined distances a wwv(Step S220), and calculate the center hole position Z wwv Register to storage device 94 (step S230). In formula (1), a0 represents the design distance of the three-point hole 81b, WD represents the working distance of the workpiece camera 70, and Z represents the shooting height of the workpiece camera 70.

[0055] [Formula 1]

[0056]

[0057] Next, CPU91 uses the captured image obtained in step S210 to align the hole gauge 80 in the world coordinate system ∑ w The position of the center hole X wwv Y wwv The measurement is performed (step S240). Then, the CPU91 transmits the measured center hole position X. wwv Y wwv Register to storage device 94 (step S250) and end this process.

[0058] In step S50, the robot system 1 is made to adjust the angle (position θ) of the handheld camera 60. ci The handheld camera angle measurement process was used for the measurement. Figure 13 This is a flowchart illustrating an example of handheld camera angle measurement processing performed by the CPU 91 of the control device 90. In the handheld camera angle measurement processing, the CPU 91 first uses the workpiece camera 70 to photograph the hole gauge 80 used for handheld camera position measurement to determine the position of the three-point hole 81b of the hole gauge 80 in the world coordinate system ∑. w Phase θ in wwv (Step S300), and the measured three-point aperture phase θ wwv Registered to storage device 94 (step S310). Three-point aperture phase θ wwv The measurement is performed by determining the angle of the triangle formed by connecting the three holes 81b that appear in the obtained captured image, relative to the X-axis. Then, as... Figure 14 As shown, CPU 91 moves the control point of handheld camera 60 to the workpiece camera position X for handheld camera position measurement registered in step S20. wwc Y wwc and the center hole position Z for handheld camera position determination registered in step S40 wwv (Step S320). Next, CPU91 uses handheld camera 60 to take a picture of the hole gauge 80, and based on the obtained picture, in the same manner as in step S300, determines the three-point hole 81b of the hole gauge 80 in the world coordinate system ∑. w Phase θ in whv(Step S330). Then, the CPU 91 calculates the handheld camera position θ wwv based on the three-point hole phase θ whv measured using the work camera 70 in steps S300, S310 and the three-point hole phase θ ci measured using the handheld camera 60 in step S330 (step S340). The calculated handheld camera position θ ci is registered to the storage device 94 (step S350), and the present processing is ended. Figure 15 is an explanatory view showing the three-point hole phases θ ci measured by the work camera and the handheld camera, respectively. As shown in the figure, the handheld camera position θ wwv is calculated by calculating the difference θe (= θ whv - θ wwv ) between the three-point hole phase θ whv measured from below by the work camera 70 and the three-point hole phase θ ci measured from above by the handheld camera 60, and offsetting the designed handheld camera position (intrinsic value) in the rotation direction by the amount of the difference θe.

[0059] In step S60, the robot system 1 is caused to execute a handheld camera position measurement processing of measuring the position (handheld camera position X ci , Y wwc ) of the handheld camera 60. Figure 16 is a flowchart showing an example of the handheld camera position measurement processing executed by the control device. In the handheld camera position measurement processing, the CPU 91 first stands by until the handheld camera position measurement button is pressed by the operator (step S400). The CPU 91, upon determining that the handheld camera position measurement button is pressed, causes the control point of the handheld camera 60 to move to the handheld camera position measurement work camera position X wwc , Y wwv registered in step S20 and the handheld camera position measurement center hole position Z whv registered in step S40 (step S410). Next, the CPU 91 captures the hole gauge 80 with the handheld camera 60, and measures the center hole position X whv , Y whv from the obtained captured image (step S420). Next, as shown in Figure 18 , the CPU 91 calculates the distance L between the image center of the captured image and the measured center hole position X whv , Y whv (Step S430). When taking the image center as the origin, the center hole position X whvWhen b is set, the distance L can be calculated by the following equation (2). Then, the CPU 91 determines whether the calculated distance L is below a predetermined allowable value Lref (step S440). The CPU 91 returns to step S410 to repeat the processing until the distance L becomes below the allowable value Lref when the distance L is below the allowable value Lref, and proceeds to step S450 when the distance L becomes below the allowable value Lref. Alternatively, the processing can proceed to step S450 regardless of whether the distance L is below the allowable value Lref.

[0060] [Equation 2]

[0061]

[0062] When the CPU 91 determines that the distance L is below the allowable value Lref, the CPU 91 moves the control point of the marker-equipped end effector EE (marker pin) to the hand camera position measurement workpiece camera position X wwc , Y wwc registered in step S20 and the hand camera position measurement hole position Z wwv registered in step S40 (step S450). Next, the CPU 91 rotates the marker-equipped end effector EE to angles of 0°, 90°, 180°, and 270°, respectively, and measures the marker pin positions (X0, Y0), (X90, Y90), (X180, Y180), and (X270, Y270) in the world coordinate system ∑ w using the workpiece camera 70 at each angle (step S460). Next, the CPU 91 calculates the average values X mc , Y mc of the marker pin positions (X0, Y0), (X90, Y90), (X180, Y180), and (X270, Y270) by the following equation (3). Then, the CPU 91 calculates the difference between the center hole position X wwc , Y wwc measured by the workpiece camera 70 in step S40 and the marker pin position (average value) X mc , Y mc by the following equation (4), and thereby calculates the positional offset X wwv , Y wwv of the end effector center from the hand camera position measurement hole center position X me , Y me (step S190). Figure 19 The marker pin position (average value) X mc , Y mc , the hand camera position measurement hole center position X wwv , Y wwv , and the positional offset Xme , Y me between the world coordinate system ∑ me and the camera coordinate system ∑ me .

[0063] [Equation 3]

[0064]

[0065] (X me ,Y me )=(X mc -X wwc ,Y mc -Y wwc )…(4)

[0066] Then, the CPU 91 converts the coordinate system of the position offset X me , Y me from the world coordinate system ∑ w to the camera coordinate system ∑ c , and calculates new hand camera positions X mo , Y mo that are obtained by offsetting the converted position offset X rme , Y rme of the design hand camera positions X ci , Y ci (intrinsic values) (step S470). In order to convert the coordinate system of the position offset X me , Y me from the world coordinate system ∑ w to the camera coordinate system ∑ c , it is only necessary to rotate the position offset X me , Y me by the amount of the phase difference θrm between the world coordinate system ∑ w and the camera coordinate system ∑ c . When the phase of the base origin of the robot 10 in the world coordinate system ∑ w is set to θ rw , and the respective rotation angles of the first joint axis J1 and the second joint axis J2 at the time of photographing the center hole 81a of the hole gauge 81 with the hand camera 60 are set to θ J1 , θ J2 , the phase difference θrm is calculated by the following equation (5). Therefore, the converted position offset X rme , Y rme can be calculated using the rotation matrices of the following equations (6) and (7). The new hand camera positions X mm , Y mm can be calculated by the following equation (8) using the hand camera positions X mo , Y mo as intrinsic values, which become the hand camera positions X ci , Y ciThe CPU91 calculates the new handheld camera position X in this way. ci Y ci Then, the calculated handheld camera position X ci Y ci Register to storage device 94 (step S480).

[0067] [Formula 4]

[0068] θ rm =θ rw +θ J1 +θ J2 ...(5)

[0069] X me =X me cos(-θ rm )-Y me sin(-θ rm (6)

[0070] Y me =X me sin(-θ rm )+Yy me cos(-θ rm )…(7)

[0071] (X mm y mm )=(X mo -X rme ,Y mo -Y rme )…(8)

[0072] Then, CPU 91 calculates the change in position of the handheld camera E using the following formula (9) (step S490), and determines whether the change in position of the handheld camera E is below a predetermined allowable value Eref (step S500). If CPU 91 determines that the change in position of the handheld camera E is not below the allowable value Eref, it returns to step S400; if it determines that the change in position of the handheld camera E is below the allowable value Eref, it ends the process. Thus, the accurate handheld camera position X is stored in the storage device 94. ci Y ci θ ci Therefore, CPU 91 uses the handheld camera position X stored in storage device 94. ci Y ci θ ci The position of the handheld camera 60 can be controlled to accurately determine the position of the workpiece W (object) captured by the handheld camera 60, thus enabling more accurate operation on the workpiece W.

[0073] [Formula 5]

[0074]

[0075] In step S70, the marked end effector EE and the hole gauge 80 for hand camera position measurement are detached. Thus, the hand camera position measurement process is completed.

[0076] Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure recited in the claims will be described. That is, in the present embodiment, the robot 10 corresponds to the robot, the hand camera 60 corresponds to the hand camera, the work camera 70 corresponds to the external camera, the hole gauge 80 corresponds to the gauge member, the center hole 81a corresponds to the first measurement hole, the CPU 91 of the control device 90 that executes the processing of steps S240, S250 of the hole position measurement processing for hand camera position measurement corresponds to the first position acquisition section, the CPU 91 of the control device 90 that executes the processing of steps S410, S420 of the hand camera position measurement processing corresponds to the second position acquisition section, and the CPU 91 of the control device 90 that executes the processing of steps S450 to S480 of the hand camera position measurement processing corresponds to the position offset measurement section. In addition, the CPU 91 of the control device 90 that executes the work camera position measurement processing for hand camera position measurement corresponds to the external camera position acquisition section, the CPU 91 of the control device 90 that executes the processing of steps S300, S310 of the hand camera angle measurement processing corresponds to the first phase acquisition section, the CPU 91 of the control device 90 that executes the processing of steps S320, S330 of the hand camera angle measurement processing corresponds to the second phase acquisition section, and the CPU 91 of the control device 90 that executes the processing of step S340 of the hand camera angle measurement processing corresponds to the phase offset measurement section. In addition, the CPU 91 of the control device 90 that executes the processing of steps S210, S220 of the hole position measurement processing for hand camera position measurement corresponds to the height acquisition section.

[0077] In addition, the present disclosure is not limited to the above-described embodiment, and can be implemented in various ways as long as it belongs to the technical scope of the present disclosure, which is self-evident.

[0078] For example, in the above-described embodiment, the robot 10 is configured as a horizontal multi-joint robot (SCARA robot), but is not limited thereto, and can be configured by any other structure such as a vertical multi-joint robot.

[0079] As explained above, the camera position shift measuring device of the present disclosure measures a position shift of a hand-held camera in a robot system that has a robot having a mechanical arm and the hand-held camera mounted to the mechanical arm, and an external camera provided outside the robot, wherein the camera position shift measuring device has a gauge member provided above the external camera and having a first measurement hole passing through vertically, a first position acquiring section that photographs the gauge member with the external camera and acquires a position of the first measurement hole from a photographed image of the first measurement hole, a second position acquiring section that moves the hand-held camera to the position acquired by the first position acquiring section and photographs the gauge member with the hand-held camera, and acquires a position of the first measurement hole from a photographed image of the first measurement hole, and a position shift measuring section that moves a fingertip of the robot to the position of the first measurement hole acquired by the second position acquiring section and photographs the fingertip of the robot with the external camera, and measures a position shift of the camera from a photographed image.

[0080] The camera position shift measuring device of the present disclosure includes a gauge member provided above an external camera and having a first measurement hole passing through vertically. Also, the position shift measuring device photographs the gauge member with the external camera to acquire a position of the first measurement hole, and moves a hand-held camera to the acquired position to photograph the gauge member with the hand-held camera to acquire a position of the first measurement hole. Also, the position shift measuring device moves a fingertip of a robot to the acquired position of the first measurement hole to photograph the fingertip of the robot with the external camera, and measures a position shift of the camera. Thus, a position shift of a camera mounted to an arm of a robot can be properly measured.

[0081] In such a camera position shift measuring device of the present disclosure, the camera position shift measuring device can also include: a gauge member disposed above the external camera and having a plurality of second measurement holes that are vertically through unlike the first measurement hole; an external camera position acquisition section that moves the fingertip of the robot above the external camera and captures the fingertip using the external camera, and acquires the position of the external camera based on the captured image of the fingertip; a first phase acquisition section that captures the gauge member using the external camera, and acquires the phase of the second measurement hole based on the captured image of the second measurement hole; a second phase acquisition section that moves the hand camera to the position of the external camera acquired by the external camera position acquisition section and captures the gauge member using the hand camera, and acquires the phase of the second measurement hole based on the captured image of the second measurement hole; and a phase shift measurement section that measures the phase shift of the hand camera based on the phases of the second measurement hole acquired by the first and second phase acquisition sections. In this way, using the gauge member and the external camera, the phase shift can be appropriately measured in addition to the position shift of the hand camera. Also, the second measurement holes can be formed at intervals in the circumferential direction around the first measurement hole of the gauge member.

[0082] In addition, in the camera position shift measuring device of the present disclosure, the camera position shift measuring device can also include: a gauge member disposed above the external camera and having a plurality of second measurement holes that are vertically through unlike the first measurement hole; and a height acquisition section that captures the gauge member using the external camera, and acquires the height of the first measurement hole based on the captured image of the second measurement hole, the second position acquisition section moving the hand camera to the position acquired by the first position acquisition section and the height acquired by the height acquisition section. In this way, even if the set height of the gauge member contains an error, the position shift of the hand camera can be favorably measured.

[0083] In addition, in the camera position shift measuring device of the present disclosure, the camera position shift measuring device can also include: a gauge member disposed above the external camera and having a plurality of second measurement holes that are vertically through unlike the first measurement hole; and a height acquisition section that captures the gauge member using the external camera, and acquires the height of the first measurement hole based on the captured image of the second measurement hole, the second position acquisition section moving the hand camera to the position acquired by the first position acquisition section and the height acquired by the height acquisition section. In this way, even if the set height of the gauge member contains an error, the position shift of the hand camera can be favorably measured.

[0084] Further, the present disclosure is a manner of a position shift measuring device of a camera, but can also be a manner of a position shift measuring method of a camera.

[0085] Industrial applicability

[0086] The present disclosure can be utilized in manufacturing industries of robot systems and the like.

[0087] Explanation of reference numerals

[0088] 1 robot system 2 work table 3a main rod 3b sub rod 10 robot 11 base 20 robot arm 21 first arm 22 second arm 23 shaft 24 workpiece holding portion 30 first arm driving portion 31 motor 32 motor 33a motor 33b motor 34 encoder 40 second arm driving portion 42 motor 44 encoder 50 shaft driving portion 52a motor 52b motor 54a encoder 54b encoder 60 hand-held camera 70 workpiece camera 71 housing case 71a opening portion 80 hole gauge 81a center hole 81b three-point hole 82 fixing member 90 control device 91 CPU 92 ROM 93 RAM 94 storage device EE labeled end effector J1 first joint shaft J2 second joint shaft J3 third joint shaft M mark P workpiece supply portion S substrate

Claims

1. A camera position shift measuring device that measures a position shift of a hand-held camera in a robot system that has a robot having a mechanical arm and the hand-held camera attached to the mechanical arm, and an external camera provided outside the robot, wherein the camera position shift measuring device comprises: a gauge member provided above the external camera and having a first measurement hole that penetrates vertically; a first position acquisition section that captures the gauge member with the external camera and acquires a position of the first measurement hole from a captured image of the first measurement hole; a second position acquisition section that moves the hand-held camera to the position acquired by the first position acquisition section, captures the gauge member with the hand-held camera, and acquires a position of the first measurement hole from a captured image of the first measurement hole; and a position shift acquisition section that moves a fingertip of the robot to the position of the first measurement hole acquired by the second position acquisition section, captures the fingertip of the robot with the external camera, and acquires a position shift of the hand-held camera from a captured image. The camera position shift measuring device comprises: the gauge member having a plurality of second measurement holes that penetrate vertically differently from the first measurement hole; an external camera position acquisition section that moves the fingertip of the robot to above the external camera, captures the fingertip with the external camera, and acquires a position of the external camera from a captured image of the fingertip; a first phase acquisition section that captures the gauge member with the external camera and acquires a phase of the second measurement hole from a captured image of the second measurement hole; a second phase acquisition section that moves the hand-held camera to the position of the external camera acquired by the external camera position acquisition section, captures the gauge member with the hand-held camera, and acquires a phase of the second measurement hole from a captured image of the second measurement hole; and a phase shift acquisition section that acquires a phase shift of the hand-held camera based on the phases of the second measurement hole acquired by the first phase acquisition section and the second phase acquisition section. The camera position shift measuring device comprises: the gauge member having a plurality of second measurement holes that penetrate vertically differently from the first measurement hole; and a height acquisition section that captures the gauge member with the external camera and acquires a height of the first measurement hole from a captured image of the second measurement hole, the second position acquisition section moving the hand-held camera to the position acquired by the first position acquisition section and the height acquired by the height acquisition section. The camera position shift measuring device comprises: the gauge member having a plurality of second measurement holes that penetrate vertically differently from the first measurement hole; and a height acquisition section that captures the gauge member with the external camera and acquires a height of the first measurement hole from a captured image of the second measurement hole, the second position acquisition section moving the hand-held camera to the position acquired by the first position acquisition section and the height acquired by the height acquisition section. ​ ​ ​ 2. The camera position shift measuring apparatus according to claim 1, wherein ​ ​ ​ ​ ​ ​ ​ 3. The camera position shift measuring apparatus according to claim 1 or 2, wherein ​ ​ ​ ​ 4. The camera position shift measuring apparatus according to claim 1 or 2, wherein ​ ​ The height obtaining section obtains the height of the first measurement hole based on a captured image of the second measurement hole obtained by the external camera capturing the gauge member, The position offset measurement section moves the hand camera to the position obtained by the second position obtaining section and the height obtained by the height obtaining section.

5. A camera position offset measurement method for measuring a position offset of a hand camera in a robot system including a robot having a robot arm and the hand camera attached to the robot arm, and an external camera disposed outside the robot, wherein The camera position offset measurement method includes: a gauge member having a first measurement hole extending vertically is disposed above the external camera; and the hand camera is moved to a position of the first measurement hole obtained based on a captured image of the first measurement hole obtained by the external camera capturing the gauge member, and the robot finger is moved to the position of the first measurement hole obtained based on the captured image of the first measurement hole obtained by the external camera capturing the robot finger, and a position offset of the hand camera is measured based on a captured image of the robot finger obtained by the external camera.

6. The method of claim 5, wherein The camera position offset measurement method includes: the gauge member having a plurality of second measurement holes extending vertically differently from the first measurement hole is disposed above the external camera; and the robot finger is moved to above the external camera, and a position of the external camera is obtained based on a captured image of the finger obtained by the external camera capturing the finger, the gauge member is captured by the external camera, and phases of the second measurement holes are obtained based on captured images of the second measurement holes obtained by the external camera capturing the gauge member, the hand camera is moved to the position of the external camera obtained, and the phases of the second measurement holes are obtained based on captured images of the second measurement holes obtained by the hand camera capturing the gauge member, and a phase offset of the hand camera is measured based on the phases of the second measurement holes obtained by the external camera and the hand camera, respectively.

Images