Method for measuring camera position deviation

Abstract

A camera position deviation measurement method is applied to a robot comprising a robot arm, a hand, and a camera. The hand is attached to the front end of the robot arm via a rotation axis, and the camera is attached to the robot arm with its optical axis parallel to the rotation axis. The method comprises: step (a) attaching a tool having a marker to the hand; step (b) rotating the hand about the rotation axis and imaging the marker with the camera at multiple rotational positions; and step (c) calculating the camera position deviation based on the difference between the position of the marker captured in each image captured at the multiple rotational positions and the ideal position of the marker.

Description

Technical Field

[0001] This specification discloses a method for measuring positional deviation of a camera. Background Art

[0002] In the past, in a robot equipped with an arm, a robot that calibrates a camera that is provided independently of the arm has been proposed (for example, refer to Patent Document 1). The processing step of the camera calibration first sets the three rotation axes X, Y, and Z of the jaw coordinate system so that the arm is moved in such a manner that the calibration pattern held at the end of the hand rotates around each rotation axis. Next, a camera is used to capture pattern images of the calibration pattern at multiple rotation positions rotated around each rotation axis. Furthermore, these pattern images are used to estimate the coordinate transformation matrix between the jaw coordinate system and the camera coordinate system. According to this processing step, external parameters that can calculate the coordinate transformation between the jaw coordinate system and the camera coordinate system can be obtained, so that the position detection of the object using the camera can be performed.

[0003] Prior art literature

[0004] Patent Document 1: Japanese Patent Application Publication No. 2019-14031 Summary of the Invention

[0005] Problems to be solved by the invention

[0006] Patent Document 1 described above describes calibration of a camera provided independently of an arm, but does not mention calibration (positional deviation measurement) of a camera attached to the arm.

[0007] A main object of the present disclosure is to measure positional deviation of a camera mounted on an arm using a simple method.

[0008] Means for solving problems

[0009] The camera position deviation measurement method disclosed herein employs the following means to achieve the above-mentioned main purpose.

[0010] The present invention provides a method for measuring a position deviation of a camera in a robot having a robot arm, a hand, and a camera, wherein the hand is mounted on a front end portion of the robot arm via a rotation axis, and the camera is mounted on the robot arm such that an optical axis is parallel to the rotation axis.

[0011] The above-mentioned method for measuring the position deviation of the camera has the following steps:

[0012] Step (a), installing a tool with a mark on the hand;

[0013] Step (b), rotating the hand around the rotation axis and photographing the mark at a plurality of rotation positions using the camera; and

[0014] Step (c) is to calculate the position deviation of the camera based on the difference between the position of the marker captured in each of the captured images captured at the plurality of rotational positions and the ideal position of the marker.

[0015] The disclosed method for measuring the positional deviation of a camera measures the positional deviation of a camera in a robot having a robot arm, a hand, and a camera. The hand is mounted on the front end of the robot arm via a rotation axis, and the camera is mounted on the robot arm so that the optical axis is parallel to the rotation axis. In this method, a tool with a marker is attached to the hand, the hand is rotated about the rotation axis, and the marker is photographed at multiple rotational positions using the camera. The positional deviation of the camera is then calculated based on the difference between the position of the marker captured in each image captured at the multiple rotational positions and the ideal position of the marker. This makes it possible to measure the positional deviation of a camera mounted on an arm through a simple process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a three-dimensional diagram of the working robot.

[0017] Figure 2 It is a side view of the robot body.

[0018] Figure 3 This is a block diagram showing the electrical connection relationship between the robot body and the control device.

[0019] Figure 4 This is an explanatory diagram showing an example of a camera position deviation measurement process.

[0020] Figure 5 It is an explanatory diagram showing a state of installation of a marking tool.

[0021] Figure 6 This is an explanatory diagram showing the designed camera field of view and the actual camera field of view when a positional deviation occurs in the camera.

[0022] Figure 7 It indicates the X based on the actual camera field of view. vr Y vr Position deviation in the coordinate system (X c , Y c ) and rotation deviation θ c 's illustration.

[0023] Figure 8 This is to explain the X based on the camera field of view in the design. vi Y vi Position deviation in the coordinate system (Xci , Y ci ) illustration. DETAILED DESCRIPTION

[0024] Next, refer to the attached Figure 1 The method for implementing the present disclosure will be described.

[0025] Figure 1 This is a three-dimensional diagram of the working robot. Figure 2 It is a side view of the robot body. Figure 3 This is a block diagram showing the electrical connection relationship between the robot body and the control device.

[0026] The working robot 1 is configured as a horizontal multi-joint robot that performs a predetermined operation on a workpiece W (e.g., a conveying operation of picking up and conveying the workpiece W, an assembling operation of picking up the workpiece W and assembling it into an object, etc.). The working robot 1 includes a robot body 10 (see Figures 1 to 3 ) and the control device 70 (refer to Figure 3 ).like Figure 1 、 Figure 2 As shown, the robot body 10 includes a base 11 and a multi-joint arm 20 .

[0027] The base 11 is fixed to the workbench 2 and supports the base end side of the multi-joint arm 20. The multi-joint arm 20 includes a first arm 21, a first arm drive unit 30, a second arm 22, a second arm drive unit 40, a shaft 23, a shaft drive unit 50, and a camera 60. The first arm 21 is configured such that its base end is connected to the base 11 via a first joint axis J1, and can rotate (horizontally rotate) relative to the base 11 in a horizontal plane by rotating the first joint axis J1. The second arm 22 is configured such that its base end is connected to the front end of the first arm 21 via a second joint axis J2, and can rotate (horizontally rotate) relative to the first arm 21 in a horizontal plane by rotating the second joint axis J2. The shaft 23 is configured such that it is connected to the front end of the second arm 22 via a third joint axis J3, and can rotate relative to the second arm 22 around the axis of the third joint axis J3 and can be raised and lowered along the axial direction of the third joint axis J3. A tool holding portion 24 for holding various tools used to operate the workpiece W is provided at the front end of the shaft 23.

[0028] like Figure 3 As shown, the first arm drive unit 30 includes a motor 32 and an encoder 34. The rotation shaft of the motor 32 is connected to the first joint axis J1 via a speed reducer (not shown). The first arm drive unit 30 uses the torque transmitted to the first joint axis J1 via the speed reducer by driving the motor 32 to horizontally rotate the first arm 21 with the first joint axis J1 as a fulcrum. The encoder 34 is a rotary encoder mounted on the rotation shaft of the motor 32 to detect the rotational displacement of the motor 32.

[0029] The second arm drive unit 40 includes a motor 42 and an encoder 44, similar to the first arm drive unit 30. The rotation shaft of the motor 42 is connected to the second joint axis J2 via a reducer (not shown). The second arm drive unit 40 uses the torque transmitted to the second joint axis J2 via the reducer by driving the motor 42 to horizontally rotate the second arm 22 with the second joint axis J2 as a fulcrum. The encoder 44 is configured as a rotary encoder that is mounted on the rotation shaft of the motor 42 and detects the rotational displacement of the motor 42.

[0030] like Figure 3 As shown, the shaft drive unit 50 includes motors 52a, 52b and encoders 54a, 54b. The rotating shaft of the motor 52a is connected to the shaft 23 via a belt (not shown), causing the shaft 23 to rotate around the shaft. The rotating shaft of the motor 52b is connected to a ball screw nut (not shown) that passes through the shaft 23 via a belt. The shaft 23 is raised and lowered by rotating the ball screw nut. The encoder 54a is configured as a rotary encoder that detects the rotational displacement of the shaft 23. The encoder 54b is configured as a linear encoder that detects the raised and lowered position of the shaft 23.

[0031] The camera 60 is mounted on the side surface of the front end of the second arm 22 so that its optical axis is parallel to the axis of the shaft 23. The camera 60 captures the workpiece W as a work target and outputs the captured image to the control device 70. The control device 70 recognizes the position of the workpiece W by processing the captured image.

[0032] like Figure 3 As shown, the control device 70 includes a CPU 71, a ROM 72, a RAM 73, a nonvolatile memory (storage device 74), and an input / output interface (not shown). Position signals from the encoders 34, 44, 54a, and 54b and image signals from the camera 60 are input to the control device 70 via the input / output interface. Drive signals for the motors 32, 42, 52a, and 52b are output from the control device 70 via the input / output interface.

[0033] Next, the operation of the working robot 1 thus configured will be described. The CPU 71 of the control device 70 first controls the first arm drive unit 30, the second arm drive unit 40, and the axis drive unit 50 so that the camera 60 moves above the workpiece W and the end-hand arm moves to a predetermined target position. The CPU 71 then uses the camera 60 to capture an image of the workpiece W. The CPU 71 then performs image processing on the captured image to measure the position of the workpiece W in the camera coordinate system. Image processing is performed by measuring the coordinate values ​​of the workpiece W captured in the captured image and applying positional and rotational offset corrections to the measured coordinate values ​​using correction values. The correction values ​​are used to correct for deviations in the positional relationship between the camera 60 and the end-hand arm caused by machining errors or assembly errors in the camera 60. These correction values ​​are pre-determined in the camera position deviation measurement process described below and stored in the storage device 74. The CPU 71 then transforms the position of the workpiece W into the robot coordinate system and sets the target position of the end-hand arm for picking up the workpiece W based on the transformed workpiece W position. The CPU 71 controls the first arm driving unit 30, the second arm driving unit 40, and the axis driving unit 50 to move the hand tip to the set target position. The workpiece W is picked up by the hand tip (workpiece holding unit 24) as the hand tip moves to the target position.

[0034] Next, the camera position deviation measuring process will be described. Figure 4 100 is an explanatory diagram showing an example of a camera position deviation measuring process. The camera position deviation measuring process is executed through steps S100 to S150.

[0035] In step S100, Figure 5 As shown, a tool holding portion 24 (hand end) provided at the front end of the shaft 23 holds a marked tool 100. The marked tool 100 has a mark M at a position away from the mounting portion mounted on the tool holding portion 24 in a direction perpendicular to the shaft 23. In step S110, the shaft driving portion 50 is controlled so as to rotate the shaft 23 by a predetermined angle (for example, 10 degrees each time), and each time the shaft 23 rotates by the predetermined angle, the mark M is photographed by the camera 60. Thus, photographed images of the mark M are obtained at a plurality of rotation positions n (n = 1, 2, 3 ...). In step S120, the photographed images are obtained in a coordinate system (X vr Y vr The mark position (X vn , Y vn )(n=1,2,3…). Mark the position (X vn , Y vn ) is measured by processing the captured image to obtain the center coordinates of the marker M captured in the image.

[0036] In step S130, based on the vr Y vr The marker position (X vn , Y vn )(n=1,2,3…)with vr Y vr The ideal mark position (X vi , Y vi )(n=1,2,3 . . . ) to derive the position deviation (X) of the camera 60 c , Y c ) and rotation deviation θ c Position deviation (X c , Y c ) represent X vr Position deviation in the axis direction and Y vr The position deviation in the axial direction. In addition, the rotation deviation θ c represents the amount of deviation in the rotational direction of the actual camera field of view including the position deviation of the camera 60 relative to the designed camera field of view not including the position deviation of the camera 60. In this embodiment, step S130 derives X defined by the following equations (1) to (3) by using a well-known minimization algorithm such as a genetic algorithm or the Newton-Raphson method. vr The difference f in the axial direction xk With Y vr The difference f in the axial direction yk The minimum sum of squares F of the position deviation (X c , Y c ) and rotation deviation θ c In step S140, the rotation deviation θ is used. c , the position deviation (X c , Y c ) from the coordinate system based on the actual camera field of view (X vr Y vr coordinate system) is transformed into a coordinate system based on the camera field of view in the design (X vi Y vi coordinate system). Therefore, in X vi Y vi In the coordinate system, derive the X coordinate of camera 60 vi Y vi Position deviation in the axis direction (X ci , Y ci In step S150, the position deviation (X ci , Y ci ) The position offset value and the rotation deviation θ of the same amount and opposite direction cThe rotational offset values ​​of the same magnitude and opposite direction are registered as correction values ​​in the storage device 74. Thus, when picking up the workpiece W, by performing offset corrections (positional offset and rotational offset) on the position of the workpiece W measured by imaging the workpiece W with the camera 60, the position of the workpiece W can be accurately recognized regardless of positional deviations of the camera 60. As a result, the accuracy of the workpiece (picking) operation can be further improved.

[0037] [Number 1]

[0038] f xk =x in -{(x rn +x c )cosθ c -(y rn +y c )sinθ c}…(1)

[0039] f yk =y in -{(x rn +x c )sinθ c +(y rn +y c )cosθ c}…(2)

[0040]

[0041] x ci =x c cosθ c -y c sinθ c …(4)

[0042] y ci =x c sinθ c +y c cosθ c …(5)

[0043] Here, in step S130, at the measured mark position (X vn , Y vn ) and the ideal marker position (X vi , Y vi ), theoretically, the following equations (6) and (7) hold. Therefore, the position deviation (X) of the camera 60 can be derived using equations (6) and (7). c , Y c ) and rotation deviation θ c However, in reality, the relationship between the two contains various errors, so it is possible to obtain the difference f xk、f yk The minimum sum of squares F of the position deviation (X c , Y c ) and rotation deviation θ c to further improve the correction accuracy.

[0044] [Number 2]

[0045] x in =(x rn +x c )cosθ c -(y rn +y c )sinθ c …(6)

[0046] y in =(x rn +x c )sinθ c +(y rn +y c )cosθ c …(7)

[0047] Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure described in the claims will be described. That is, in this embodiment, the multi-joint arm 20 corresponds to the robot arm, the tool holding unit 24 corresponds to the hand, and the camera 60 corresponds to the camera.

[0048] In addition, it goes without saying that the present disclosure is not limited to the above-mentioned embodiment at all, and can be implemented in various forms as long as it falls within the technical scope of the present disclosure.

[0049] For example, in the above-mentioned embodiment, the camera position deviation measurement method disclosed herein is applied to a horizontal multi-jointed robot. However, this method is not limited to this and can be applied to any robot structure, such as a vertical multi-jointed robot, as long as the robot structure includes a robot arm, a hand attached to the front end of the robot arm via a rotation axis, and a camera attached to the robot arm so that the optical axis is parallel to the rotation axis.

[0050] As described above, the camera position deviation measurement method disclosed in the present invention measures the position deviation of the camera in a robot having a robot arm, a hand and a camera, wherein the hand is mounted on the front end portion of the robot arm via a rotation axis, and the camera is mounted on the robot arm in such a manner that the optical axis is parallel to the rotation axis. The camera position deviation measurement method comprises the following steps: step (a), mounting a tool with a mark on the hand; step (b), rotating the hand around the rotation axis and photographing the mark at multiple rotation positions using the camera; and step (c), calculating the camera position deviation based on the difference between the position of the mark captured in each captured image captured at the multiple rotation positions and the ideal position of the mark.

[0051] The disclosed method for measuring the positional deviation of a camera measures the positional deviation of a camera in a robot having a robot arm, a hand, and a camera. The hand is mounted on the front end of the robot arm via a rotation axis, and the camera is mounted on the robot arm so that the optical axis is parallel to the rotation axis. In this method, a tool with a marker is attached to the hand, the hand is rotated about the rotation axis, and the marker is photographed at multiple rotational positions using the camera. The positional deviation of the camera is then calculated based on the difference between the position of the marker captured in each image captured at the multiple rotational positions and the ideal position of the marker. This makes it possible to measure the positional deviation of a camera mounted on an arm through a simple process.

[0052] In the camera position deviation measurement method disclosed in the present invention, it is also possible to set: in the above step (c), the X vr Y vr In the coordinate system, the position of the marker measured by photographing at each rotation position n (n=1, 2, ...) in the above step (b) is defined as (X vn , Y vn ), let the ideal mark position at each rotation position n (n = 1, 2 ...) be (X in , Y in ), X vr Axis direction and Y vr The position deviations of the cameras in the axis direction are set as (X c , Y c ) and the rotation deviation of the actual camera field of view relative to the designed camera field of view in the rotation direction is set as θ c When the above-mentioned measured marker position (X vn , Y vn ) and the ideal marking position (X in , Y in ) to obtain the above position deviation (X c , Yc ) and the above rotation deviation θ c , based on the above rotation deviation θ c The above position deviation (X c , Y c )'s coordinate system is from the above X vr Y vr The coordinate system is transformed into the X coordinate system based on the camera field of view in the above design. vi Y vi In this way, the position deviation of the camera can be obtained by a simple process based on the position of the marker measured by the camera at each rotation position n.

[0053] In this case, it is also possible to determine, in the above step (c), using a predetermined minimization algorithm to find the position deviation (X) that minimizes the sum of squares F defined by equations (1) to (3). c , Y c ) and the above rotation deviation θ c The above position deviation (X c , Y c )'s coordinate system is from the above X vr Y vr The coordinate system is transformed into the above X vi Y vi coordinate system, and thus find the above X vi Y vi Position deviation in the coordinate system (X ci , Y ci ). This can further improve the measurement accuracy of the position deviation and rotation deviation of the camera.

[0054] Furthermore, in the camera position deviation measurement method disclosed herein, an offset value corresponding to the camera position deviation and rotation deviation calculated in step (c) may be registered as a correction value for correcting the position of an object measured by imaging with the camera. This allows the camera to accurately identify the position of the object.

[0055] Industrial applicability

[0056] The present disclosure can be utilized in the robot manufacturing industry and the like.

[0057] Description of Reference Numerals

[0058] 1 Working robot, 2 Working platform, 10 Robot body, 11 Base, 20 Multi-joint arm, 21 First arm, 22 Second arm, 23 Axis, 30 First arm driving unit, 32 Motor, 34 Encoder, 40 Second arm driving unit, 42 Motor, 44 Encoder, 50 Axis driving unit, 52a, 52b Motors, 54a, 54b Encoders, 60 Camera, 70 Control device, 71 CPU, 72 ROM, 73 RAM, 74 Storage device, J1 First joint axis, J2 Second joint axis, J3 Third joint axis.

Claims

1. A method for measuring positional deviation of a camera, comprising measuring positional deviation of a camera in a robot having a robot arm, a hand, and a camera, wherein the hand is attached to a front end portion of the robot arm via a rotation axis, and the camera is attached to the robot arm such that an optical axis is parallel to the rotation axis. The method for measuring the position deviation of a camera comprises the following steps: Step (a), installing a tool with a mark on the hand; Step (b), rotating the hand around the rotation axis and photographing the mark on the tool using the camera at multiple rotation positions; and Step (c) of calculating a position deviation of the camera based on a difference between a position of the marker captured in each of the images captured at the plurality of rotational positions and an ideal position of the marker at the rotational position. In the step (c), the X vr Y vr In the coordinate system, the position of the marker measured by photographing at each rotation position n (n=1, 2...k) in step (b) is defined as (X vn , Y vn ), the ideal mark position at each rotation position n (n = 1, 2 ... k) is set to (X in , Y in ), X vr Axis direction and Y vr The position deviation of each camera in the axis direction is set as (X c , Y c ) and the rotation deviation of the actual camera field of view relative to the designed camera field of view in the rotation direction is set to θ c , using the marker position (X vn , Y vn ) and the ideal marking position (X in , Y in ) to obtain the position deviation (X c , Y c ) and the rotation deviation θ c , based on the rotation deviation θ c The position deviation (X c , Y c )'s coordinate system from the X vr Y vr The coordinate system is transformed into X based on the camera field of view on the design vi Y vi coordinate system, thereby calculating the position deviation of the camera, Furthermore, in the step (c), a predetermined minimization algorithm is used to find the position deviation (X c , Y c ) and the rotation deviation θ c The position deviation (X c , Y c )'s coordinate system from the X vr Y vr The coordinate system is transformed into the X vi Y vi coordinate system, so as to find the X vi Y vi Position deviation in the coordinate system (X ci , Y ci ), f xn =x in -{(x vn +x c )cosθ c -(y vn +y c )sinθ c }...(1) f yn =y in -{(x vn +x c )sinθ c +(y vn +y c )cosθ c }...(2) x ci =x c cosθ c -y c sinθ c ...(4) y ci =x c sinθ c +y c cosθ c ...(5)。 2. The method for measuring position deviation of a camera according to claim 1, wherein: The camera position deviation measurement method further includes the step (d) of registering an offset value corresponding to the camera position deviation and rotation deviation obtained in the step (c) as a correction value for correcting the position of an object measured by imaging with the camera.

Images