Camera position deviation correction method and robot device

Abstract

In a method of correcting a positional deviation of a camera according to the present invention, positional deviation of a camera with respect to an arm of a robot device is corrected. In this method, the arm and the camera are controlled so that the camera captures an image of a single mark member, the position of the mark member in a base coordinate system is determined on the basis of the captured image, and the position is stored in advance as a reference position. When correction of positional deviation is requested, control is performed so that the camera captures an image of the mark member at a first position which is above the mark member and which is an image capturing position, the position of the mark member in the base coordinate system is determined on the basis of the captured image, and the positional relation of the camera in the horizontal direction is corrected so that the determined position matches the reference position. Furthermore, control is performed so that the camera captures an image of the mark member at a second position which is offset horizontally in a given direction from the first position and which is an image capturing position, the position of the mark member in the base coordinate system is determined on the basis of the captured image, and the positional relation of the camera in the rotation direction is corrected so that the determined position matches the reference position.

Description

[Technical Field]

[0001] This specification discloses a method for correcting camera position deviation and a robot device. [Background technology]

[0002] A robot system has been proposed that includes a robot having a robot arm and a robot camera, and an external camera installed separately from the robot, and that captures an image of a calibration pattern with a pre-calibrated robot camera and also captures an image of the calibration pattern with the external camera to determine calibration data for the external camera (see, for example, Patent Document 1). This system captures an image of a calibration pattern with the robot camera and the external camera, respectively, to obtain pattern images, and determines calibration data for the external camera based on the obtained pattern images and known calibration data for the robot camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-69493 Summary of the Invention [Problem to be solved by the invention]

[0004] However, although the technology described in the above-mentioned Patent Document 1 describes the calibration of an external camera, it does not mention anything about the calibration of a robot camera.

[0005] A primary object of the present disclosure is to appropriately correct the positional deviation of a camera attached to the arm of a robot using a simple method. [Means for solving the problem]

[0006] The present disclosure has adopted the following means to achieve the above-mentioned main object.

[0007] The camera positional deviation correction method disclosed herein is a camera positional deviation correction method for a robot device having an arm and a camera attached to the arm, which corrects a positional deviation of the camera relative to the arm, and includes controlling the arm and the camera so that an image of one mark member is captured by the camera, determining the position of the mark member in a base coordinate system based on the captured image obtained, and pre-storing the determined position of the mark member as a reference position, and when correction of the positional deviation is requested, controlling the arm and the camera so that the camera captures the mark member using a first position above the mark member as an imaging position, and pre-storing the position of the mark member in the base coordinate system based on the captured image obtained. a first correction process for determining the position of the mark member in the base coordinate system and correcting the horizontal positional relationship of the camera with respect to the arm so that the determined position of the mark member coincides with the reference position; after performing the first correction process, the arm and the camera are controlled so that the camera captures an image of the mark member at a second position horizontally offset in a predetermined direction from the first position, and a second correction process for determining the position of the mark member in the base coordinate system based on the obtained captured image and correcting the positional relationship of the camera with respect to the arm in the rotational direction so that the determined position of the mark member coincides with the reference position;

[0008] The camera positional deviation correction method disclosed herein controls an arm and a camera to capture an image of a single mark member with the camera, determines the position of the mark member in a base coordinate system based on the captured image, and stores the determined position of the mark member in advance as a reference position. When a request for positional deviation correction is received, the method executes a first correction process and a second correction process to correct the positional deviation of the camera relative to the arm. The first correction process controls the arm and the camera to capture an image of the mark member with the camera using a first position above the mark member as the imaging position. The first correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the horizontal positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. The second correction process controls the arm and the camera to capture an image of the mark member with the camera using a second position horizontally offset from the first position in a predetermined direction. The second correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the rotational positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. As a result, if the position of the mark member is measured using an arm or camera with guaranteed accuracy before the robot device is shipped and stored in advance as a reference position, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after the robot device is shipped, the positional relationship between the arm and camera can be corrected on site without the need for a special measuring jig.

[0009] a control device that, when a request for correction of a positional deviation of the camera relative to the arm is made, controls the arm and the camera so that the camera captures an image of the mark member using a first position above the mark member as an imaging position, determines the position of the mark member in the base coordinate system based on the captured image, and performs a first correction process to correct a horizontal positional relationship of the camera with respect to the arm so that the determined position of the mark member coincides with the reference position; and, after the first correction process has been made, controls the arm and the camera so that the camera captures an image of the mark member using a second position horizontally offset from the first position in a predetermined direction as an imaging position, and performs a second correction process to determine the position of the mark member in the base coordinate system based on the captured image, and corrects a positional relationship of the camera with respect to the arm in a rotational direction so that the determined position of the mark member coincides with the reference position.

[0010] The robot device disclosed herein includes an arm, a camera, one marking member, a storage device, and a control device. The position of the marking member in a base coordinate system is pre-stored in the storage device as a reference position. When a request is made to correct the positional deviation of the camera relative to the arm, the control device executes first and second correction processes similar to the camera positional deviation correction method disclosed herein above. As a result, similar to the camera positional deviation correction method disclosed herein, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after shipping, the positional relationship between the arm and camera can be corrected on-site without the need for a special measuring jig. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an external perspective view of a robot system. [Figure 2] FIG. 2 is a block diagram showing the electrical connection relationship between the robot device and a control device. [Figure 3] FIG. 2 is an explanatory diagram illustrating a base coordinate system, a camera coordinate system, and a mechanical interface coordinate system. [Figure 4] 10 is a flowchart illustrating an example of a reference position registration process. [Figure 5] 10 is a flowchart illustrating an example of a hand camera position correction process. [Figure 6] FIG. 10 is an explanatory diagram showing the field of view of each hand camera when capturing images of a mark at two locations. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, embodiments of the present disclosure will be described with reference to the drawings.

[0013] Fig. 1 is a perspective view of the appearance of the robot system, and Fig. 2 is a block diagram showing the electrical connection relationship between the robot device and a control device.

[0014] The robot system 1 is a working robot system that, for example, picks up workpieces W (components) supplied by a workpiece supply unit P and mounts them on a target object (substrate S). As shown in FIGS. 1 and 2, the robot system 1 includes a workbench 2, a robot device 10 having a robot arm 20, a hand camera 60 attached to the robot arm 20, a workpiece camera 70 as an external camera, and a control device 90 that controls the robot device 10. A pole-shaped marking member 3 extending vertically is installed on the workbench 2.

[0015] In this embodiment, the robot device 10 is a SCARA robot, and includes a base 11 and a robot arm 20.

[0016] The base 11 is fixed to the workbench 2 and supports the base end side of the robot arm 20. The robot arm 20 includes a first arm 21, a first arm driver 30, a second arm 22, a second arm driver 40, a shaft 23, and a shaft driver 50. The first arm 21 has a base end connected to the base 11 via a first joint axis J1 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the base 11 by rotation of the first joint axis J1. The second arm 22 has a base end connected to a tip end of the first arm 21 via a second joint axis J2 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the first arm 21 by rotation of the second joint axis J2. The shaft 23 is connected to a tip end of the second arm 22 via a third joint axis J3 and is configured to be rotatable around the axis of the third joint axis J3 relative to the second arm 22 and to be movable up and down along the axial direction of the third joint axis J3. In the robot system 1 of this embodiment, a workpiece holding unit 24 that serves as an end effector and picks up and holds the workpiece W is provided at the tip of the shaft 23. Examples of the workpiece holding unit 24 include a suction nozzle that suctions the workpiece W by negative pressure, a mechanical chuck that grips the workpiece W with a pair of claws, and an electromagnetic chuck that suctions the workpiece W by an electromagnet.

[0017] The first arm driver 30 includes a motor 32 and an encoder 34. The rotation shaft of the motor 32 is connected to the first joint shaft J1 via a reducer (not shown). By driving the motor 32, the first arm driver 30 transmits torque to the first joint shaft J1 via the reducer, causing the first arm 21 to rotate around the first joint shaft J1 as a fulcrum. The encoder 34 is attached to the rotation shaft of the motor 32 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 32.

[0018] Similar to the first arm driver 30, the second arm driver 40 includes a motor 42 and an encoder 44. The rotation shaft of the motor 42 is connected to the second joint shaft J2 via a reducer (not shown). By driving the motor 42, the second arm driver 40 transmits torque to the second joint shaft J2 via the reducer, causing the second arm 22 to rotate around the second joint shaft J2 as a fulcrum. The encoder 44 is attached to the rotation shaft of the motor 42 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 42.

[0019] The shaft driving unit 50 includes motors 52a and 52b and encoders 54a and 54b. The rotational axis of the motor 52a is connected to the shaft 23 via a belt (not shown). The shaft driving unit 50 rotates the shaft 23 around its axis by driving the motor 52a. The rotational axis of the motor 52b is connected to the shaft 23 via a ball screw mechanism (not shown). The shaft driving unit 50 drives the motor 52b, and the ball screw mechanism converts the rotational motion of the motor 52b into linear motion, thereby raising and lowering the shaft 23. The encoder 54a is configured as a rotary encoder that detects the amount of rotational displacement of the shaft 23. The encoder 54b is configured as a linear encoder that detects the raised and lowered position of the shaft 23.

[0020] 2, the control device 90 includes a CPU 91, a ROM 92 that stores a processing program, a RAM 93 as a work memory, a storage device 94 such as an HDD or SSD, and an input / output interface (not shown). Position signals from encoders 34, 44, 54a, and 54b, image signals from hand camera 60, and image signals from work camera 70 are input to the control device 90 via the input / output interface. Drive signals for motors 32, 42, 52a, and 52b, drive signals for hand camera 60, and drive signals for work camera 70 are output from the control device 90 via the input / output interface.

[0021] The hand camera 60 is attached to the tip of the second arm 22. The hand camera 60 captures an image of the workpiece W in the workpiece supply section P from above and outputs the captured image to the control device 90. The control device 90 processes the captured image to recognize the position of the workpiece W.

[0022] The work camera 70 is installed between the work supply unit P and the substrate S on the work table 2. The work camera 70 captures an image of the work W held by the work holding unit 24 of the robot device 10 from below, and outputs the captured image to the control device 90. The control device 90 processes the captured image to determine whether the work W is being held properly by the work holding unit 24.

[0023] Next, the operation of the robot system 1 configured in this manner will be described. The operation of the robot system 1 includes a work position recognition operation, a collection operation (picking operation), a collection confirmation operation, and a mounting operation (placement operation). In the work position recognition operation, the hand camera 60 captures an image of the work W to recognize the position of the work W. In the collection operation, the hand (work holding unit 24) of the robot system 1 is moved to the recognized position of the work W to pick up the work W. In the collection confirmation operation, the work W picked up by the hand is moved above the work camera 70, and an image of the work W is captured by the work camera 70 to confirm whether the work W has been picked up properly. In the mounting operation, the picked work W is mounted on the board S.

[0024] Here, a description will be given of the coordinate system of the robot system 1. In this embodiment, as shown in FIG. 2, a base coordinate system Σ b and the camera coordinate system Σ c and the mechanical interface coordinate system Σ m The base coordinate system Σ b The origin of the mechanical interface coordinate system Σ is set on the bottom surface of the base 11 of the robot device 10, the X axis is set in the direction of the home position of the robot arm 20, the Z axis is set in the vertical direction, and the Y axis is set in a direction perpendicular to the X and Z axes. mThe origin is set at the tip of the robot arm 20, and the X-axis, Y-axis, and Z-axis are respectively in the base coordinate system Σ b The camera coordinate system Σ is set so that its axis and direction are the same. c is set such that the origin is the bottom surface of the second arm 22 and the center of the third joint axis J3, and the X-axis, Y-axis, and Z-axis are respectively in the base coordinate system Σ b The axes and directions of the base coordinate system Σ are set to coincide with each other. b and the camera coordinate system Σ c and the mechanical interface coordinate system Σ m are converted into each other using a transformation matrix.

[0025] In the workpiece position recognition operation, the CPU 91 of the control device 90 first calculates the coordinate system of the base 11 (base coordinate system Σ b ) the target position X of the hand camera 60 (hand camera center) btag ,Y btag ,Z btag Next, the CPU 91 sets the target position X btag ,Y btag ,Z btag By solving the inverse kinematics for btag ,Y btag ,Z btag The angle θ of the first joint axis J1 to match J1 and the angle θ of the second joint axis J2 J2 Here, the camera coordinate system Σ c The position of the hand camera 60 at ci ,Y ci ,θ ci is registered in advance in the storage device 94. c Since the origin of the hand camera position X is set at the center of the shaft 23 connected to the tip of the robot arm 20, the CPU 91 ci ,Y ci ,θ ci Based on the base coordinate system Σ b The position of the hand camera 60 at the hand position X can be converted to the position of the end effector.ci ,Y ci ,θ ci Based on the base coordinate system Σ b The CPU 91 can convert the hand position X ci ,Y ci ,θ ci Based on the base coordinate system Σ b Target position X of hand camera 60 at btag ,Y btag is converted into the target position of the hand, and the angle θ of the first joint axis J1 is calculated by solving the inverse kinematics based on the converted target position of the hand. J1 and the angle θ of the second joint axis J2 J2 It is possible to calculate:

[0026] Next, the CPU 91 calculates the angle θ of the first joint axis J1 taking into consideration error factors such as the twist and deflection of the first arm 21 and the deflection of the second arm 22. J1 and the angle θ of the second joint axis J2 J2 The estimated position of the hand is calculated by solving the forward kinematics based on the hand position and the hand camera position X ci ,Y ci ,θ ci The estimated position X of the hand camera 60 based on best ,Y best ,Z best That is, the CPU 91 calculates the target position X btag ,Y btag ,Z btag The angle θ of the first and second joint axes J1 and J2 calculated by inverse kinematics so as to match J1 ,θ J2 is controlled as an angle command value, the target position X btag ,Y btag ,Z btag Estimated position X, which is a position that is shifted from best ,Y best ,Z best Calculate.

[0027] CPU91 estimates position Xbest ,Y best ,Z best , the calculated estimated position X best ,Y best ,Z best and target position X btag ,Y btag ,Z btag By taking the difference from b ,ΔY b ,ΔZ b Next, the CPU 91 calculates the target position X btag ,Y btag ,Z btag The positional deviation amount ΔX b ,ΔY b ,ΔZ b After offsetting the hand camera 60 by the amount of btag ,Y btag ,Z btag Angle command value θ of the first joint axis J1 to move to J1 * and the angle command value θ of the second joint axis J2 J2 Then, the CPU 91 calculates the angle command value θ J1 * and the angle of the second joint axis J2 detected by the encoder 44 is the calculated angle command value θ J2 The corresponding motors 32 and 42 are controlled by feedback control so as to match the values ​​of the respective signals.

[0028] Next, the CPU 91 uses the hand camera 60 to take an image of the workpiece W supplied from the workpiece supply unit P, and performs image processing on the taken image to calculate the coordinate system Σ b The position of the workpiece W in the image is recognized. The image processing is performed by calculating the pixel position of the mark member 3 in the captured image and the hand camera position X ci ,Y ci ,θ ci and based on the camera coordinate system Σ c The position of the mark member 3 at the angle θ J1 and the angle θ of the second joint axis J2 J2 and based on the camera coordinate system Σ c From the base coordinate system Σb Then, the CPU 91 converts the target position X of the hand (workpiece holding unit 24) for picking up the workpiece W into btag ,Y btag is set to the position of the recognized workpiece W, and the end effector is moved to the target position X btag ,Y btag ,Z btag The target position X is moved to the target position X. btag ,Y btag The setting of target position Z is based on the recognized position of the workpiece W. btag The setting is made based on the height information of the workpiece W input in advance.

[0029] In the collection operation, the CPU 91 first sets the target position (X btag ,Y btag ,Z btag ) the angle θ of the first joint axis J1 J1 and the angle θ of the second joint axis J2 J2 and the angle of shaft 23 (shaft angle) θ J4 and the lift position Z of shaft 23 s Then, the CPU 91 calculates the calculated θ 2 by taking into consideration error factors such as the twist and deflection of the first arm 21 and the deflection of the second arm 22. J1 ,θ J2 ,θ J4 ,Z s The estimated position of the hand based on best ,Y best ,Z best Next, the CPU 91 calculates the estimated position X of the hand. best ,Y best ,Z best and target position X btag ,Y btag ,Z btag The difference between this and the positional deviation ΔX b ,ΔY b ,ΔZ b Next, the CPU 91 calculates the target position X btag ,Y btag ,Z btag The positional deviation amount ΔX b ,ΔY b ,ΔZb After offsetting by the amount of btag ,Y btag ,Z btag Angle command value θ of the first joint axis J1 to move to J1 * and the angle command value θ of the second joint axis J2 J2 * and the angle command value θ of shaft 23 J4 * and the vertical position command value Z of shaft 23 s Then, the CPU 91 controls the corresponding motors 32, 42, 52a, and 52b by feedback control based on each command value.

[0030] In this way, the robot system 1 captures an image of the workpiece W with the hand camera 60 attached to the robot arm 20, recognizes the position of the workpiece W, and moves the tip of the robot arm 20 to the recognized position to pick up the workpiece W. For this reason, the positional relationship between the tip of the robot arm 20 and the hand camera 60 (hand camera position X ci ,Y ci ,θ ci ) is not properly grasped, the hand camera 60 cannot accurately recognize the position of the workpiece W, and a positional deviation occurs when the workpiece W is picked up. The robot system 1 of this embodiment is shipped after measuring the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 using a dedicated measuring jig. However, if the robot arm 20 collides with a peripheral device or the like at the shipping destination, an error will occur in the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 due to deformation or displacement of the components that make up the robot arm 20. Because dedicated measuring jigs are expensive, if an error occurs in the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 at the shipping destination, it is not practical to remeasure the positional relationship using a dedicated measuring jig on-site.

[0031] The robot system 1 of this embodiment is provided with a marking member 3 on the workbench 2, and before shipping, a dedicated measuring tool is used to measure the positional relationship between the tip of the robot arm 20 and the hand camera 60 (hand camera position X ci ,Y ci ,θ ci) is measured, the mark member 3 is imaged by the hand camera 60, and the position of the mark member 3 (mark position X b ,Y b ) and measure the measured mark position X b ,Y b Reference position X r ,Y r When an error occurs in the positional relationship between the tip of the robot arm 20 and the hand camera 60 at the shipping destination, the robot system 1 captures an image of the mark member 3 with the hand camera 60 and stores the mark position X b ,Y b and the reference position X that was measured and stored before the error occurred. r ,Y r The hand camera position X is set to the same result as ci ,Y ci ,θ ci This makes it possible to re-correct the positional relationship between the tip of the robot arm 20 and the hand camera 60 without using an expensive measuring jig. Below, we will first explain the details of the reference position registration process that registers the reference position, and then explain the details of the hand camera position correction process that re-corrects the position of the hand camera 60.

[0032] 4 is a flowchart showing an example of the reference position registration process. The reference position registration process is executed before the robot system 1 is shipped. In the reference position registration process, the CPU 91 of the control device 90 first determines whether or not the reference position measurement button has been pressed (step S100). If the CPU 91 determines that the reference position measurement button has not been pressed, it simply ends the reference position registration process. On the other hand, if the CPU 91 determines that the reference position measurement button has been pressed, it moves the hand camera 60 to a target position (image capture position) set above the mark member 3, captures an image of the mark member 3 with the hand camera 60, performs image processing on the captured image, and registers the base coordinate system Σ b The position of the mark part 3 (mark position X b ,Y b ) is calculated (step S110). As described above, the image processing is performed by calculating the pixel position of the mark member 3 in the captured image and the hand camera position Xci ,Y ci ,θ ci and based on the camera coordinate system Σ c The position of the mark member 3 in the camera coordinate system Σ c From the base coordinate system Σ b The CPU 91 converts the mark position X b ,Y b When measuring, the measured mark position X b ,Y b Reference position X r ,Y r The reference position is registered in the storage device 94 as the reference position (step S120), and the reference position registration process is completed.

[0033] FIG. 5 is a flowchart showing an example of the hand camera position correction process. As described above, the hand camera position correction process is executed when the robot arm 20 collides with a peripheral device or the like, causing an error in the positional relationship between the hand of the robot arm 20 and the hand camera 60. If a collision with a peripheral device or the like causes deformation or the like of a component constituting the robot arm 20, a positional deviation may occur in the hand of the robot arm 20. In this case, prior to the hand camera position correction process, the positional deviation of the hand of the robot arm 20 due to the deformation or the like is first corrected. The positional deviation of the hand can be determined, for example, by attaching a marking member to the hand of the robot arm 20, capturing an image of the marking member with the work camera 70, and measuring the position of the marking member. For example, the positional deviation of the hand can be determined by setting a target position for the hand above the work camera 70, moving the hand to the target position, capturing an image of the marking member attached to the hand with the work camera 70, measuring the position of the marking member, and calculating the difference between the measured position of the marking member and the target position.

[0034] In the hand camera position correction process, the CPU 91 first determines whether the hand camera position correction button has been pressed (step S200). If the CPU 91 determines that the hand camera position correction button has not been pressed, it ends the hand camera position correction process. On the other hand, if the CPU 91 determines that the hand camera position correction button has been pressed, it moves the control point of the hand camera 60 above the mark member 3, for example, directly above it (step S210). Next, the CPU 91 captures an image of the mark member 3 with the hand camera 60, and performs image processing on the captured image to calculate the base coordinate system Σ b Mark position X at b ,Y b (Step S220). As described above, the image processing is performed by determining the pixel position of the mark member 3 in the captured image and the hand camera position X ci ,Y ci ,θ ci and based on the camera coordinate system Σ c The position of the mark member 3 in the camera coordinate system Σ c From the base coordinate system Σ b This is done by converting

[0035] CPU91 is mark position X b ,Y b When measuring, the measured mark position X b ,Y b and the reference position X registered in advance in the storage device 94 in the above-mentioned reference position registration process. r ,Y r By taking the difference with the base coordinate system Σ b X in b Axial and Y directions b Axial misalignment of the mark position X m ,Y m (step S230). Then, the CPU 91 calculates the calculated positional deviation amount X m ,Y m Based on hand camera position X ci ,Y ci and the camera position change amount E are calculated (step S240). ci ,Y ciThe calculation is performed using the following equations (1) to (4). Equations (1) and (2) are based on the base coordinate system Σ b Position deviation amount X m ,Y m in the camera coordinate system Σ c Position deviation amount X cie ,Y cie This is a conversion formula for converting θ J1 is the angle of the first joint axis J1, and θ J2 is the angle of the second joint axis J2. Furthermore, equations (3) and (4) are used to calculate the hand camera position X cio ,Y cio The positional deviation amount X cie ,Y cie The camera position change amount E is calculated using the following equation (5). r ,Y r From mark position X b ,Y b The CPU 91 calculates the straight-line distance between the hand camera position X ci ,Y ci and the camera position change amount E are calculated, the calculated hand camera position X ci ,Y ci is newly registered in the storage device 94 (step S250). This makes it possible to re-correct the positional relationship in the horizontal direction between the tip of the robot arm 20 and the hand camera 60. Then, the CPU 91 determines whether the calculated camera position change amount E is equal to or less than the allowable value Eref (step S260).

[0036]

number

[0037] If the CPU 91 determines that the camera position change amount E is greater than the allowable value Eref, the process returns to step S210 and the newly registered hand camera position X ci ,Y ci and hand camera position θ ciThe position of the mark member 3 is measured by the hand camera 60 using the hand camera position X ci ,Y ci The process of steps S210 to S250 for calculating the hand camera position θ and the camera position change amount E is repeated. ci Due to the difference in resolution and the error between the true value and the hand camera position X, it is difficult to obtain the correct value with just one measurement. ci ,Y ci This is because it is not possible to accurately determine

[0038] When the CPU 91 determines that the camera position change amount E is equal to or less than the allowable value Eref, the control point (field of view) of the hand camera 60 is then calculated from the position directly above the mark member 3 in the base coordinate system Σ b Y b Then, the CPU 91 captures an image of the mark member 3 with the hand camera 60, performs image processing on the captured image, and calculates the coordinates of the mark member 3 in the base coordinate system Σ b Mark position X at b ,Y b Next, the CPU 91 measures the mark position X b ,Y b and the reference position X r ,Y r By taking the difference with the base coordinate system Σ b The amount of deviation of the mark position at X m ,Y m (step S290). Then, the CPU 91 calculates the calculated positional deviation amount X m ,Y m Based on the hand camera position θ ci (Step S300). ci The calculation is performed using the following equations (6) and (7): m ,Y m The arctangent angle (positional deviation θ cie ) is a conversion formula for converting the hand camera position θ cio The positional deviation amount θ cie This is a correction (offset) for the amount of

[0039]

number

[0040] The CPU 91 thus calculates the hand camera position θ ci , the calculated hand camera position θ ci is newly registered in the storage device 94 (step S310), and the hand camera position correction process is completed.

[0041] In this way, the CPU 91 captures an image of one mark member 3 with the hand camera 60 and determines the position X of the mark member 3. b ,Y b and recognize the recognized position X b ,Y b is the reference position X that has been registered in advance. r ,Y r Hand camera position X to match ci ,Y ci Then, the CPU 91 takes an image of one mark member 3 from another position using the hand camera 60 and calculates the position X of the mark member 3. b ,Y b and recognize the recognized position X b ,Y b is the reference position X that has been registered in advance. r ,Y r The hand camera position θ is set to match ci This allows the positional relationship between the tip of the robot arm 20 and the hand camera 60 to be re-corrected without using an expensive measuring jig.

[0042] Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure described in the claims will be described. That is, in this embodiment, the robot device 10 corresponds to the robot, the robot arm 20 corresponds to the arm, the hand camera 60 corresponds to the camera, and the mark member 3 corresponds to the mark member. Also, the CPU 91 of the control device 90 that executes the hand camera position correction process corresponds to the control device.

[0043] It goes without saying that the present disclosure is not limited to the above-described embodiments, and can be embodied in various forms as long as they fall within the technical scope of the present disclosure.

[0044] For example, in the above-described embodiment, the robot device 10 is configured as a horizontal articulated robot (SCARA robot), but this is not limited to this, and the robot device may be any other type of robot device, such as a vertical articulated robot, as long as it has a camera attached to its arm.

[0045] As described above, the camera positional deviation correction method disclosed herein controls an arm and a camera to capture an image of a single mark member with the camera, determines the position of the mark member in a base coordinate system based on the captured image, and stores the determined position of the mark member in advance as a reference position. When a request for positional deviation correction is received, the positional deviation of the camera relative to the arm is corrected by executing a first correction process and a second correction process. The first correction process controls the arm and the camera to capture an image of the mark member with the camera using a first position above the mark member as the imaging position. The first correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the horizontal positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. The second correction process controls the arm and the camera to capture an image of the mark member with the camera using a second position horizontally offset from the first position in a predetermined direction. The second correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the rotational positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. As a result, if the position of the mark member is measured using an arm or camera with guaranteed accuracy before the robot device is shipped and stored in advance as a reference position, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after the robot device is shipped, the positional relationship between the arm and camera can be corrected on site without the need for a special measuring jig.

[0046] In the camera positional deviation correction method according to the present disclosure, the first correction process may involve calculating distances between the determined position of the mark member and the reference position in orthogonal first and second axis directions of the base coordinate system, and then converting the determined distances into coordinates in the camera coordinate system to correct the horizontal positional relationship between the arm and the camera. This allows the horizontal positional relationship between the arm and the camera to be corrected by simple calculation.

[0047] In the camera positional deviation correction method according to the present disclosure, the first correction process may be repeated until the amount of positional deviation between the position of the mark member in the base coordinate system and the reference position becomes equal to or less than a predetermined amount, thereby enabling accurate correction of the positional relationship between the arm and the camera.

[0048] Furthermore, in the camera positional deviation correction method disclosed herein, the second correction process may involve calculating distances between the determined position of the mark member and the reference position in the first axis direction and the second axis direction that are orthogonal to each other in the base coordinate system, and converting the calculated distances into arctangent angles to correct the positional relationship between the arm and the camera in the rotational direction. In this way, the positional relationship between the arm and the camera in the rotational direction can be corrected by simple calculation.

[0049] The present disclosure is not limited to a method for correcting positional deviation of a camera, but may also be applied to a robot device. [Industrial Applicability]

[0050] The present disclosure is applicable to the robot device manufacturing industry and the like. [Explanation of symbols]

[0051] 1 robot system, 2 work table, 3 marking member, 10 robot device, 11 base, 20 robot arm, 21 first arm, 22 second arm, 23 shaft, 24 workpiece holder, 30 first arm drive unit, 32 motor, 34 encoder, 40 second arm drive unit, 42 motor, 44 encoder, 50 shaft drive unit, 52a motor, 52b motor, 54a encoder, 54b encoder, 60 hand camera, 70 workpiece camera, 90 control device, 91 CPU, 92 ROM, 93 RAM, 94 storage device, J1 first joint axis, J2 second joint axis, J3 third joint axis, P workpiece supply unit, S substrate, W workpiece.

Claims

1. 1. A method for correcting a positional deviation of a camera with respect to an arm in a robot device having an arm and a camera attached to the arm, comprising: controlling the arm and the camera so as to capture an image of one marking member with the camera, determining the position of a mark affixed to the marking member in a base coordinate system based on the captured image, and pre-storing the determined position of the mark of the marking member in the base coordinate system as a reference position; When correction of the positional deviation is requested, the arm and the camera are controlled so that the camera captures an image of the mark member using a first position above the mark member as an imaging position, and a first correction process is performed to determine the position of the mark affixed to the mark member in a base coordinate system based on the captured image, and to correct the horizontal positional relationship of the camera with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position; after performing the first correction process, the arm and the camera are controlled so that the camera captures an image of the mark member using a second position moved horizontally a predetermined distance from the first position in a predetermined direction so that the mark can be captured, and a second correction process is performed to determine the position of the mark affixed to the mark member in the base coordinate system based on the captured image, and to correct the positional relationship of the camera in the rotational direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position. Camera position correction method.

2. 2. The camera position deviation correction method according to claim 1, the first correction process is to calculate distances in a first axis direction and a second axis direction orthogonal to each other in the base coordinate system between the calculated position of the mark of the mark member in the base coordinate system and the stored reference position, and to correct the positional relationship between the arm and the camera in the horizontal direction by coordinate conversion of the calculated distances into a camera coordinate system; Camera position correction method.

3. 3. The camera position deviation correction method according to claim 1, further comprising: repeating the first correction process until a positional deviation between the position of the mark of the mark member in the base coordinate system and the stored reference position becomes equal to or less than a predetermined amount; Camera position correction method.

4. 4. The camera position deviation correction method according to claim 1, further comprising: the second correction process is to calculate distances in a first axis direction and a second axis direction orthogonal to each other in the base coordinate system between the calculated position of the mark of the mark member in the base coordinate system and the stored reference position, and convert the calculated distances into arctangent angles to correct the positional relationship between the arm and the camera in the rotation direction. Camera position correction method.

5. Arm and a camera attached to the arm; one marking member; a storage device that stores in advance the position of the mark on the mark member in a base coordinate system as a reference position; a control device that, when a correction of positional deviation of the camera with respect to the arm is requested, controls the arm and the camera so that the camera captures an image of the mark member using a first position above the mark member as an imaging position, determines the position of the mark affixed to the mark member in a base coordinate system based on the captured image, and performs a first correction process to correct a positional relationship of the camera in a horizontal direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position, and after performing the first correction process, controls the arm and the camera so that the camera captures an image of the mark member using a second position moved horizontally a predetermined distance from the first position so that the mark can be captured, and performs a second correction process to determine the position of the mark affixed to the mark member in the base coordinate system based on the captured image, and corrects a positional relationship of the camera in a rotational direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position; A robotic device comprising:

Images