A photographing apparatus for calculating a three-dimensional position based on an image photographed by a vision sensor

Abstract

A photographing apparatus includes a focal position detection section that detects a focal position at the time of focusing of a vision sensor. The photographing apparatus includes a parameter setting section that sets a parameter used to calculate a three-dimensional position corresponding to a specific position in an image photographed by the vision sensor. The photographing apparatus includes a feature position calculation section that calculates a three-dimensional position of a feature portion using the parameter set by the parameter setting section. The parameter setting section sets the parameter in accordance with the focal position.

Description

Technical Field

[0001] This invention relates to an imaging device for calculating three-dimensional position based on images captured by a vision sensor. Background Technology

[0002] Previously, it was known to have apparatuses for detecting the three-dimensional position of an object by processing images obtained from images of the object captured by a vision sensor. For example, apparatuses for calculating the three-dimensional position of specific parts of an object by capturing two-dimensional images from two directions are known (e.g., Japanese Patent Application Publication No. 2016-706475). Alternatively, it is known to use a vision sensor called a stereo camera to simultaneously capture images using two two-dimensional cameras and calculate the three-dimensional position of feature points based on the parallax in the two images.

[0003] Such a device for calculating three-dimensional position can be installed on a robot used to move a workpiece to perform a predetermined task. In a robot system for handling workpieces, a camera is used to photograph the workpiece placed at a predetermined position. The three-dimensional position of the workpiece is detected based on the image captured by the camera. The robot's position and posture are changed so that it can hold the workpiece according to its position. Through such control, the accurate position of the workpiece can be detected, and the task can be performed reliably.

[0004] In calculating the three-dimensional position of an object based on a two-dimensional image captured by a vision sensor, a computational model is used to transform the position in the image into a three-dimensional position. The computational model includes predetermined parameters such as coefficients and constants. By using the computational model, the three-dimensional position can be calculated based on the position in an image captured by a camera.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2016-70674 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] The parameters in the computational model used to calculate the three-dimensional position of an object depend on factors such as the camera setup, lens characteristics, and individual lens variations. These parameters can be predetermined through calculation or experimentation. For example, after positioning the camera at a predetermined location, images of the object can be actually captured, and the parameters can be calculated in advance.

[0010] In conventional camera bodies and lenses, the camera is fixed in a predetermined position. The position of the camera lens is fixed, and parameters are pre-calculated. However, sometimes there are individual differences in the objects being photographed. Or, the position of the object may shift from the desired position when the camera takes the picture. As a result, the image of the workpiece may be blurry.

[0011] Therefore, controlling the camera's focus is considered. For example, in a camera with autofocus, focusing is considered based on the position of the workpiece or the individual differences of the workpiece. However, when focusing is performed, the position of the focusing lens changes, and therefore, the parameters in the calculation model also change. Since the parameters are determined corresponding to a focal position, there is a problem that the focus cannot be aligned to a position other than the predetermined focal position. Furthermore, it is necessary to use the parameters regardless of the focal position.

[0012] Furthermore, to fix the focal position, it's considered to align the camera's relative position to the workpiece with a predetermined position. However, sometimes the position of at least one of the workpiece or the camera changes. For example, depending on how the workpiece is handled, the position and orientation of the workpiece placed on the worktable may change. In this case, the robot can change its position and orientation based on the workpiece's position and orientation. However, there are situations where the robot interferes with obstacles such as fences surrounding the robot system. Or, sometimes the robot's travel has limits. Therefore, it can be difficult to align the camera's relative position to the workpiece with a predetermined position.

[0013] Methods for solving problems

[0014] One aspect of the imaging apparatus disclosed herein includes: a vision sensor for photographing an object; and a focus position detection unit for detecting the focus position when the vision sensor is focusing. The imaging apparatus includes a parameter setting unit that sets parameters for calculating a three-dimensional position corresponding to a specific position in an image captured by the vision sensor. The imaging apparatus also includes a storage unit that stores setting information for setting the parameters corresponding to the focus position. The imaging apparatus further includes: a feature detection unit that detects predetermined feature portions in an image of the object; and a feature position calculation unit that calculates the three-dimensional position of the feature portions using the parameters set by the parameter setting unit. The parameter setting unit sets the parameters based on the focus position and the setting information.

[0015] Invention Effects

[0016] According to the present disclosure, an imaging device is available that can detect the three-dimensional position of a feature portion with high precision when the focal position changes. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the first robot system in the implementation method.

[0018] Figure 2 This is a block diagram of the first robot system in the implementation method.

[0019] Figure 3 This is a plan view of the workpiece according to the implementation method.

[0020] Figure 4 It is a rough diagram illustrating the focus point of the camera and the field of view of the camera.

[0021] Figure 5 This is an example of an image when the focus position is changed.

[0022] Figure 6 This is a schematic diagram of the image sensor, lens, and feature parts of the workpiece when the lens position is moved for focusing.

[0023] Figure 7 This is a flowchart illustrating the control of the first robot system.

[0024] Figure 8 This is a first schematic diagram illustrating the other controls of the first robot system.

[0025] Figure 9 This is a second schematic diagram illustrating other controls of the first robot system.

[0026] Figure 10 This is a schematic diagram of the second robot system in the implementation method.

[0027] Figure 11 This is a schematic diagram of the conveying device in the implementation method.

[0028] Figure 12 This is a block diagram of the conveying device in the implementation method. Detailed Implementation

[0029] Reference Figures 1 to 12 The imaging device in this embodiment will be described. The imaging device of this embodiment functions as a three-dimensional position acquisition device that calculates the three-dimensional position of a specific location in an image based on an image captured by a vision sensor.

[0030] Figure 1 This is a schematic diagram of a first robot system equipped with the shooting device in this embodiment. Figure 2 This is a block diagram of the first robot system in this embodiment. The robot system in this embodiment detects the position of the workpiece as the object and moves the workpiece.

[0031] Reference Figure 1 and Figure 2The first robot system 3 includes a hand 5 that serves as a working tool for holding the workpiece 38 and a robot 1 that moves the hand 5. The robot system 3 also includes a control device 2 for controlling the robot system 3. In addition, the robot system 3 includes a platform 95 for placing the workpiece 38.

[0032] In this embodiment, the hand 5 is a work tool for holding or releasing the workpiece 38. As a work tool mounted on the robot 1, it is not limited to this method; any work tool corresponding to the work performed by the robot system 3 can be used. For example, as an end effector, a welding work tool can be used.

[0033] The robot 1 in this embodiment is a multi-joint robot comprising multiple joints 18. Robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is supported on a rotating base 13. The rotating base 13 is supported on a base 14. Robot 1 includes a wrist 15 connected to the end of the upper arm 11. The wrist 15 includes a flange 16 for fixing a hand 5. The components of robot 1 are configured to rotate about a predetermined drive axis. However, robot 1 is not limited to this configuration; any robot capable of moving working tools can be used.

[0034] The robot 1 of this embodiment includes a robot drive unit 21 having a drive motor for driving components such as an upper arm 11. The hand 5 includes a hand drive unit 22 for driving the hand 5. In this embodiment, the hand drive unit 22 drives the hand 5 by air pressure. The hand drive unit 22 includes an air pump for supplying compressed air to a cylinder and a solenoid valve.

[0035] The control device 2 includes a control device body 40 and a teaching operation panel 26 for the operator to operate the control device body 40. The control device body 40 includes an arithmetic processing unit (computer) with a CPU (Central Processing Unit) as a processor. The arithmetic processing unit has RAM (Random Access Memory) and ROM (Read Only Memory) connected to the CPU via a bus. The robot 1 is driven based on the action instructions from the control device 2. The robot 1 automatically transports the workpiece 38 based on the action program 61. The robot drive device 21 and the hand drive device 22 are controlled by the control device 2.

[0036] The control device main body 40 includes a storage unit 42 for storing arbitrary information related to the robot system 3. The storage unit 42 can be constructed from a non-transient storage medium capable of storing information. For example, the storage unit 42 can be constructed from a storage medium such as volatile memory, non-volatile memory, magnetic storage medium, or optical storage medium.

[0037] The control device 2 inputs a pre-generated motion program 61 for performing the actions of the robot 1. Alternatively, the operator can use the teach pendant 26 to drive the robot 1, thereby setting the teach point of the robot 1. The control device 2 can then generate the motion program 61 based on the teach point.

[0038] The motion program 61 is stored in the storage unit 42. The motion control unit 43 sends motion commands to the robot drive unit 44 for driving the robot 1 based on the motion program 61. The robot drive unit 44 includes circuitry for driving a drive motor and supplies power to the robot drive device 21 based on the motion commands. Additionally, the motion control unit 43 sends motion commands to the hand drive unit 45 for driving the hand drive device 22. The hand drive unit 45 includes circuitry for driving an air pump, etc., and supplies power to the air pump, etc., based on the motion commands.

[0039] The motion control unit 43 is equivalent to a processor that is driven according to the motion program 61. The processor is configured to read information stored in the storage unit 42. The processor reads the motion program 61 and implements the control determined by the motion program 61, thereby performing the function of the motion control unit 43.

[0040] Robot 1 includes state detectors for detecting the position and orientation of robot 1. In this embodiment, the state detectors include position detectors 23 of the drive motors mounted on each drive shaft of the robot drive unit 21. The position detectors 23 can be, for example, encoders that detect the rotational position of the output shafts of the drive motors. The position and orientation of robot 1 are detected based on the output of the position detectors 23.

[0041] In robot system 3, a reference coordinate system 71 is set that remains stationary when the position and orientation of robot 1 change. Figure 1 In the example shown, the origin of a reference coordinate system 71 is positioned on the base 14 of robot 1. The reference coordinate system 71 is also called the world coordinate system. In the reference coordinate system 71, the position of the origin is fixed, and the orientation of the coordinate axes is also fixed. The reference coordinate system 71 has mutually orthogonal X-axis, Y-axis, and Z-axis as coordinate axes. Additionally, the W-axis is set as the coordinate axis around the X-axis. The P-axis is set as the coordinate axis around the Y-axis. The R-axis is set as the coordinate axis around the Z-axis.

[0042] The teach pendant 26 is connected to the control unit main body 40 via a communication device. The teach pendant 26 includes an input section 27 for inputting information related to the robot 1 and the hand 5. The input section 27 consists of input components such as a keyboard and a dial. The teach pendant 26 also includes a display section 28 for displaying information related to the robot 1 and the hand 5. The display section 28 can be any display panel, such as a liquid crystal display panel or an organic EL (Electroluminescence) display panel. Furthermore, if the teach pendant has a touch panel display, the display panel functions as both an input section and a display section.

[0043] A tool coordinate system is established in robot system 3, which has an origin set at any position on the working tool. The position and posture of the tool coordinate system change along with the working tool. In this embodiment, the origin of the tool coordinate system is set at the tool tip point of the hand 5. The position of robot 1 corresponds to the position of the tool tip point (the position of the origin of the tool coordinate system). In addition, the posture of robot 1 corresponds to the posture of the tool coordinate system relative to the reference coordinate system 71.

[0044] The robot system 3 in this embodiment includes an imaging device for detecting the position of the workpiece 38. Before the hand 5 grasps the workpiece 38, the imaging device detects the position of the workpiece 38 on the platform 95. The imaging device includes a camera 6, which serves as a vision sensor for capturing images of the workpiece 38. The camera 6 in this embodiment is a two-dimensional camera for capturing two-dimensional images. The camera 6 is supported on the robot 1. The camera 6 is fixed to the hand 5 via a support member.

[0045] Camera 6 is capable of capturing images within the field of view 6a. Camera 6 has a focus adjustment mechanism 24 for adjusting the focus. The focus adjustment mechanism 24 in this embodiment has an autofocus function. That is, camera 6 has an autofocus function. Camera 6 is configured to automatically focus on workpiece 38 to capture images of workpiece 38 when robot 1 changes position and posture. As a focus adjustment mechanism, a mechanism that performs focusing through arbitrary control, such as contrast detection or phase difference, can be used.

[0046] Alternatively, a camera equipped with a liquid lens can be used as a vision sensor. In this case, a mechanism for changing the shape of the liquid lens can be used as the focus adjustment mechanism. For example, a mechanism for changing the voltage applied to the liquid lens or a mechanism for moving a liquid lens holding member to change the water pressure applied to the liquid lens can be used.

[0047] In robot system 3, a camera coordinate system 72 is set as the sensor coordinate system for camera 6. The position and orientation of the camera coordinate system 72 change along with camera 6. The origin of the camera coordinate system 72 is set at a predetermined position of camera 6, such as the center of the lens or the optical center. The camera coordinate system 72 has mutually orthogonal X-axis, Y-axis, and Z-axis. In this embodiment, the camera coordinate system 72 is set such that the Z-axis is parallel to the optical axis of the lens of camera 6.

[0048] The imaging apparatus of this embodiment includes a moving device that moves one of the workpiece 38 (the object) and the camera 6 to change the relative position of one with respect to the other. In the first robot system 3, the robot 1 functions as the moving device. If the position and posture of the robot 1 change, the position and posture of the camera 6 also change.

[0049] The imaging device includes an image processing unit that processes images captured by a vision sensor. In the robot system 3 of this embodiment, the control unit main body 40 functions as an image processing unit. The control unit main body 40 includes an image processing unit 51 that processes images captured by a camera 6. The image processing unit 51 includes an imaging control unit 58 that sends instructions to the camera 6 to capture images.

[0050] The image processing unit 51 includes a focus position detection unit 52 that detects the focus position when the camera 6 is focusing. The image processing unit 51 includes a parameter setting unit 53 that sets parameters for calculating a three-dimensional position corresponding to a specific position in the image captured by the camera 6. The image processing unit 51 includes a feature detection unit 54 that detects predetermined feature portions in the image of the workpiece 38. The image processing unit 51 includes a feature position calculation unit 55 that calculates the three-dimensional position of the feature portions using the parameters set by the parameter setting unit 53. The image processing unit 51 includes a distance calculation unit 56 that calculates the distance from the camera 6 to the workpiece 38. The image processing unit 51 includes a motion command generation unit 59 that generates motion commands for the robot 1 and the hand 5 based on the results of image processing.

[0051] The image processing unit 51 is equivalent to a processor driven according to the action program 61. In particular, each unit of the focus position detection unit 52, parameter setting unit 53, feature detection unit 54, feature position calculation unit 55, distance calculation unit 56, shooting control unit 58, and motion command generation unit 59 is equivalent to a processor driven according to the action program 61. The processor reads the action program 61 and implements the control determined by the action program 61, thereby performing its function as a unit.

[0052] Reference Figure 1In the first robot system 3 of this embodiment, the workpiece 38 is arranged on the surface of the platform 95 by a predetermined method. For example, an operator or other robot system arranges the workpiece 38 on the surface of the platform 95. Then, the robot 1 changes its position and posture, and uses its hand 5 to grasp the workpiece 38 arranged on the upper surface of the platform 95. The robot system 3 transports the workpiece 38 to a predetermined position by changing the position and posture of the robot 1.

[0053] When workpiece 38 is positioned on the surface of the support 95, there is a possibility of positional offset of workpiece 38 within the support 95. Figure 1 In the example shown, position P38a is determined during the teaching operation when the teaching robot 1 is in position and orientation. Position P38a is the desired position for configuring workpiece 38, and is the reference position for configuring workpiece 38.

[0054] However, when actually placing the workpiece 38 on the upper surface of the platform 95, sometimes the workpiece 38 is placed at a position P38b offset from the reference position P38a. Alternatively, there may be dimensional errors in the workpiece 38. In the robot system 3, the workpiece 38 is photographed using the camera 6. Then, the image processing unit 51 calculates the three-dimensional position of the workpiece 38 based on the image. For example, the image processing unit 51 detects the three-dimensional position of a feature portion of the workpiece 38. Then, the image processing unit 51 calculates the position of the workpiece 38 based on the three-dimensional position of the feature portion. This position of the workpiece 38 can be calculated in the reference coordinate system 71. The image processing unit 51 controls the position and posture of the robot 1 in a manner corresponding to the position of the workpiece 38. Then, the hand 5 grasps the workpiece 38 and moves it to the desired, predetermined position.

[0055] Figure 3 This is a plan view showing the workpiece 38 in this embodiment. (Refer to...) Figure 1 and Figure 3 The workpiece 38 has a plate-shaped portion 38a and a plate-shaped portion 38b formed on the upper side of the plate-shaped portion 38a. Each plate-shaped portion 38a, 38b has a cuboid shape. The plate-shaped portion 38b has an edge portion 38c on the outer periphery of its upper surface. The edge portion 38c is the portion corresponding to the corner formed on the plate-shaped portion 38b. In this embodiment, the edge portion 38c, which has a quadrilateral shape when viewed from above, becomes a characteristic part of the workpiece 38.

[0056] Reference Figure 1In this embodiment, the camera 6 is positioned above the workpiece 38 in the vertical direction. The distance from the surface of the workpiece 38 with the feature portion to the camera 6 is predetermined. In this example, the position and posture of the robot 1 are controlled so that the position of the upper surface of the plate-shaped portion 38b is a predetermined value of the Z-axis of the camera coordinate system 72. Furthermore, the posture of the camera 6 is adjusted so that the optical axis of the camera 6 is approximately perpendicular to the surface of the plate-shaped portion 38b of the workpiece 38 with the feature portion.

[0057] Camera 6 focuses to capture an image of workpiece 38. Feature detection unit 54 of image processing unit 51 detects edge portion 38c as a feature portion of workpiece 38 by performing pattern matching. Reference image for detecting the position of edge portion 38c is pre-generated and stored in storage unit 42. Feature detection unit 54 uses the reference image to detect edge portion 38c as a feature portion in the image captured by camera 6.

[0058] The feature position calculation unit 55 calculates the position of the workpiece in three-dimensional space based on the position of the feature portion in the image captured by the camera. The position of the workpiece can be calculated as the position of any set point defined on the workpiece. The position of the workpiece 38 can be obtained in the reference coordinate system 71.

[0059] The motion command generation unit 59 calculates the position and posture of the robot 1 based on the position of the workpiece 38 calculated by the feature position calculation unit 55. Then, the position and posture of the robot 1 used to hold the workpiece 38 are sent to the motion control unit 43. Based on the motion commands received from the motion command generation unit 59, the motion control unit 43 drives the robot 1 and the hand 5 to hold the workpiece 38.

[0060] In the workpiece holding control of this embodiment, the feature detection unit 54 detects feature portions, and the feature position calculation unit 55 calculates the three-dimensional position of the workpiece with high precision based on the position of the feature portions. Therefore, the robot system 3 can hold the workpiece 38 more reliably. Even when the position of the workpiece 38 in the platform 95 (the position of the workpiece 38 in the reference coordinate system 71) differs from the reference position or when there is a dimensional error in the workpiece, the robot system 3 can still reliably hold the workpiece 38.

[0061] Reference Figure 1Due to positional deviations of the workpiece 38 or individual differences in the workpiece, the focal position of the camera may shift. Here, the focal position can be any variable corresponding to the position of the lens driven by the focus adjustment mechanism 24. For example, the focus adjustment mechanism 24 may sometimes include a drive motor that drives the lens to focus. In this case, the rotational position of the output shaft of the drive motor during focusing can be used as the focal position. Alternatively, a focus ring for focusing the lens may be provided. The position of the focus ring can be determined as the focal position. Or, in the case where the camera has multiple lenses, the position of a predetermined lens can be determined as the focal position. Furthermore, in the case of a camera equipped with a liquid lens, the magnitude of the voltage applied to the liquid lens can be used as the focal position. Alternatively, the focal position can be determined by the rotational position of the output shaft of the motor included in the drive mechanism of the holding member that changes the pressure applied to the liquid lens.

[0062] Figure 4 This is a schematic diagram illustrating the field of view as the focus position changes. The field of view of camera 6 corresponds to the angle of view or shooting range. Figure 4 The image shows focus positions A and B. When focusing at position A, the camera 6's field of view is A. When focusing at position B, the camera 6's field of view is B. Thus, the size of the field of view changes when the focus position changes. With the position of the feature portion of the workpiece remaining the same, the position of the feature portion in the image changes when the focus position changes.

[0063] Figure 5 Examples of images captured by a camera when the focus position changes are shown. Image 66 is an image when the focus is on one position, for example, with... Figure 4 Position A corresponds to this. Image 67 is the image when the focus is on other positions, such as when it is in focus with... Figure 4 The image corresponding to position B. Image coordinate system 73 is set in images 66 and 67.

[0064] Images 66 and 67 each contain images 68a and 68b, which are feature parts of a hole. By changing the focus position, as shown by arrow 101, the position of image 68a of the hole in image 66 becomes the position of image 68b of the hole in image 67. When the focus position changes, the position of the feature point in the image changes. In the mathematical model, the change in focus position is synonymous with the change in focal length.

[0065] Figure 6 A schematic diagram illustrating the positions of feature points of the object being photographed is shown. A lens 37 is positioned between the surface of the workpiece and the image sensor that generates the image for the camera. Focal lengths f1 and f2 correspond to the distances from the image sensor to the center of lens 37. Figure 6 The diagram shows focusing positions A and B. Position A corresponds to a lens 37 positioned at focal length f1. Position B corresponds to a lens 37 positioned at focal length f2. The distances z1 and z2 from the lens 37 to the workpiece change according to the focal lengths f1 and f2.

[0066] At position A, the image sensor detects a feature portion 69 disposed on the surface of the workpiece at a distance X1 from the optical axis at a distance u1 from the optical axis. At position B, the image sensor detects the feature portion 69 disposed at the same distance X2 as X1 at a distance u2. For example, with respect to position B, the relationship X2 / z2 = u2 / f2 holds. Since the focal length f2 is greater than the focal length f1, the distance u2 in the image sensor is greater than the distance u1. Thus, even if the positions on the surface of the workpiece are the same, the position of the feature portion in the image from the camera varies depending on the focal point position.

[0067] The parameter setting unit 53 calculates parameters in a calculation model based on the focal position, which calculates the position of the feature portion of the workpiece from the image captured by the camera. The feature position calculation unit 55 uses the calculation model to calculate the three-dimensional position based on a specific position in the image. Using parameters set corresponding to the focal position, the three-dimensional position of the feature portion on the workpiece surface is calculated.

[0068] Next, a computational model for calculating three-dimensional position in space based on images captured by a camera will be explained. The position in the camera image corresponding to any position in space is generally represented by the following equation (1) based on the pinhole camera model.

[0069] [Formula 1]

[0070]

[0071] [u, v]: Coordinates in the camera coordinate system

[0072] f x f y The product of focal length and effective pixel size

[0073] C x C y Image Center

[0074] [X, Y, Z]: Coordinate values ​​in the reference coordinate system

[0075] The coordinates (X, Y, Z) of the three-dimensional position are represented, for example, by the reference coordinate system 71. The coordinates (u, v) of the position on the image are represented, for example, by the image coordinate system 73. The matrix of extrinsic parameters is a transformation matrix used to transform the three-dimensional position in space into coordinates in the camera coordinate system 72. Additionally, the matrix of intrinsic parameters is a matrix used to transform the coordinates in the camera coordinate system 72 into coordinates in the image coordinate system 73. Here, the Z-axis value of the three-dimensional position, or the z-axis coordinate value in the camera coordinate system 72, is predetermined in relation to the distance from the camera to the workpiece.

[0076] Equation (1) above is an ideal example where there is no lens distortion, etc. In reality, we need to consider the changes in parameters caused by lens distortion, etc. First, the operation of the matrix part of the three-dimensional position in space and the external parameters in equation (1) can be represented by the following equation (2).

[0077] [Formula 2]

[0078]

[0079] t: [t1, t2, t3]

[0080] R: 3×3 rotation matrix

[0081] Equation (2) transforms the coordinate values ​​(X, Y, Z) represented by the reference coordinate system 71 into coordinate values ​​(x, y, z) represented by the camera coordinate system 72. Next, to account for lens distortion in the camera, variables x' and y' are defined as shown in equations (3) and (4). Furthermore, variables x' and y', taking distortion into account, are calculated as shown in equations (5) and (6). Here, the relationship between variables x', y', and r is shown in equation (7).

[0082] [Formula 3]

[0083] x′=x / z…(3)

[0084] y′=y / z…(4)

[0085]

[0086]

[0087] Where, r 2 =x′ 2 +y′ 2 …(7)

[0088] In equations (5) and (6), coefficients k1 to k6 are coefficients related to the distortion of the lens in the radial direction, and coefficients p1 and p2 are coefficients related to the distortion of the lens in the circumferential direction. Using the variables x” and y” that take into account the distortion of the lens, the coordinate values ​​(u, v) on the image in image coordinate system 73 can be calculated as shown in equations (8) and (9) below. Equations (8) and (9) are the parts corresponding to the matrix operations of the internal parameters in equation (1) above.

[0089] [Formula 4]

[0090] u = f x *x"+c x …(8)

[0091] υ=f y *y"+c y …(9)

[0092] The above description explained a method for calculating the position in an image based on a three-dimensional position in space. However, in this embodiment, based on the aforementioned relationship, the three-dimensional position (X, Y, Z) in space is calculated using the coordinate values ​​(u, v) of the position in the image and the distance z from the camera 6 to the workpiece 38 in the camera coordinate system 72. The distance z from the camera 6 to the workpiece 38 can be predetermined and stored in the storage unit 42. The feature position calculation unit 55 calculates the three-dimensional position (X, Y, Z) in space based on the calculation model and the coordinate values ​​(u, v) of a specific position in the image.

[0093] Here, referring to equations (2) to (9), in the calculation model for calculating the three-dimensional position based on the position in the image, the product of the focal length and the effective size of the image, f, is required. x f y Image center c x c y And the coefficients related to distortion, k1 to k6, p1, and p2. These parameters vary depending on the focal point position when the camera is focusing.

[0094] In this embodiment, setting information 63 for setting parameters corresponding to the focus position is predetermined. Setting information 63 is stored in the storage unit 42. The parameter setting unit 53 uses the focus position and setting information 63 to set these parameters. Table 1 shows the values ​​of the parameters corresponding to the focus position pp, which are the setting information 63.

[0095] [Table 1]

[0096] 1 2 2 4 3 6 4 8

[0097] Here, as an example of a parameter, the product of focal length and effective pixel size, f, is shown. xThe setting information 63 here predetermines the parameter values ​​for multiple discrete focus positions pp. The parameter setting unit 53 sets the parameters in the calculation model based on the parameter values ​​determined for each focus position. For example, when the focus position pp is 1.4 when taking an image using the camera 6, the parameter setting unit 53 can interpolate the product f as a parameter. x The value is set to 2.8. In setting the parameters, a table containing discrete parameter values ​​can be used to set the parameters in any way. For example, the middle value of the two parameters corresponding to the two focal positions pp can be used, or the value of the parameter corresponding to the focal position pp of either of the two closest focal positions can be used.

[0098] By using setting information containing parameter values ​​for discrete focal positions, the parameter setting unit can set parameters corresponding to any focal position. The parameter setting unit can set parameters through simple calculations. Alternatively, even when it is difficult to set the function described below, parameters corresponding to the focal position can be set.

[0099] As setting information, the function used to calculate parameters based on the focal position pp can be determined. The parameters can be calculated using a formula containing the focal position pp. For example, as shown in equation (10), the function f(pp) for calculating the product f of the focal length and the effective size of the pixel for the focal position pp can be determined in advance. Alternatively, as shown in equation (11), the function k(pp) for calculating the distortion coefficient k for the focal position pp can be determined.

[0100] [Formula 5]

[0101] f = f(pp)...(10)

[0102] k=k(pp)…(11)

[0103] As such a function, for example, a higher-order formula can be used, where the focal position pp is the variable. The parameter setting unit 53 can use a function to set various parameters, such as those related to distortion. The feature position calculation unit 55 can calculate the three-dimensional position of the feature portion based on the parameters set by the parameter setting unit 53.

[0104] Figure 7 A flowchart illustrating the control process in this embodiment. (Refer to...) Figure 1 , Figure 2 as well as Figure 7 The operator pre-determines the parameter settings for the calculation model. Then, the operator causes the storage unit 42 to store the setting information 63.

[0105] In step 80, the motion control unit 43 moves the camera 6 to a shooting position for photographing the workpiece 38. In this embodiment, the camera 6 is positioned directly above the reference position P38a of the workpiece 38. Furthermore, the posture of the camera 6 is adjusted so that the Z-axis of the camera coordinate system 72 is parallel to the vertical direction. Here, the distance from the surface of the plate-shaped portion 38b of the workpiece 38, where the feature portion is formed, to the camera 6 is predetermined.

[0106] Next, in step 81, the focus adjustment mechanism 24 of the camera 6 focuses the camera 6. The focus adjustment mechanism 24 of this embodiment has an autofocus function, therefore focusing automatically. In step 82, the shooting control unit 58 captures an image using the camera 6. The image is captured while in focus.

[0107] In step 83, the focus position detection unit 52 detects the focus position when the image is captured. The focus position detection unit 52 detects, for example, a predetermined variable corresponding to the position of the lens. In step 84, the parameter setting unit 53 sets the parameters of the calculation model used to calculate the three-dimensional position of the feature portion based on the focus position and setting information.

[0108] Next, in step 85, the feature detection unit 54 detects feature portions in the image by performing pattern matching. In this embodiment, the edge portion 38c in the image is detected by performing pattern matching using a reference image of the edge portion 38c of the plate-shaped portion 38b. The feature position calculation unit 55 detects the position of the feature portion in the image.

[0109] Next, in this embodiment, when the position of a feature portion of the workpiece cannot be detected, control is implemented to change the position of the camera 6 relative to the workpiece 38 to capture an image. For example, sometimes the illumination light reflects off the feature portion, causing it to appear white and become unclear. In such cases, by moving the position of the camera, it is sometimes possible to capture a clear image of the feature portion.

[0110] In step 86, the image processing unit 51 determines whether the position of the feature portion has been detected. If the feature position calculation unit 55 cannot detect the position of the feature portion, control is transferred to step 87.

[0111] In step 87, the motion command generation unit 59 generates a command to change the position of the camera 6. For example, the motion command generation unit 59 generates a command to move the camera 6 in a predetermined direction by a predetermined amount of parallel movement. (See reference...) Figure 1For example, the motion command generation unit 59 generates a command to move the camera 6 in the X-axis direction of the camera coordinate system 72. The motion command generation unit 59 sends the motion command of the robot 1 to the motion control unit 43. The motion control unit 43 changes the position and posture of the robot 1. Then, control returns to step 81. The image processing unit 51 repeats the control from steps 81 to 86.

[0112] In step 86, if the feature position calculation unit 55 calculates the position of the feature portion, control is transferred to step 88. Furthermore, if the feature portion cannot be detected even after multiple changes to the robot's position and posture, control can be stopped.

[0113] In step 88, the feature position calculation unit 55 calculates the three-dimensional position of the feature portion based on its position in the image. The coordinate values ​​of the reference coordinate system 71 are calculated based on the coordinate values ​​of the image coordinate system 73 in the image. The feature position calculation unit 55 calculates the position of the workpiece based on the three-dimensional position of the feature portion. The position of the workpiece can be calculated, for example, using the reference coordinate system 71.

[0114] In step 89, the motion command generation unit 59 calculates the position and posture of the robot 1 based on the position of the workpiece. Then, in step 90, the motion command generation unit 59 sends motion commands to drive the robot 1 to the motion control unit 43. The motion control unit 43 drives the robot 1 and the hand 5 based on the motion commands.

[0115] Thus, in this embodiment, the imaging device sets parameters for a calculation model that calculates the three-dimensional position corresponding to a specific location in the image captured by the vision sensor, based on the focal point position. Then, the three-dimensional position of the specific location is calculated based on the parameters corresponding to the focal point position.

[0116] By implementing this control, the three-dimensional position of the feature portion can be detected with high precision even when the focal position changes. In particular, it eliminates the need to pre-fix the focal position, allowing images to be captured at any focal position and calculating the three-dimensional position with minimal error. In other words, it eliminates the need to pre-determine the relative focus position of the camera relative to the workpiece, allowing the camera to be positioned at any location and captured after focusing. In this embodiment, even when the focus position (the relative position of the camera relative to the workpiece) increases, no new parameters for the calculation model are required. Compared to existing technologies, the imaging device of this embodiment can expand the range of robot operation or increase the number of robot operation modes.

[0117] In particular, when the vision sensor's focus adjustment mechanism has an autofocus function, the focus position is set at any position within a predetermined range. In this case, the imaging device can set parameters corresponding to the focus position and detect the accurate position of the workpiece. Alternatively, the vision sensor may not have an autofocus function. In this case, the operator can manually focus. For example, the operator can focus by operating the input unit 27 while observing the image displayed on the display unit 28 of the teach pendant 26.

[0118] Furthermore, the robot system in this embodiment includes a robot that serves as a moving device for moving at least one of the workpiece and the vision sensor. Thus, when the robot changes the relative position of the camera to the workpiece, the focal point position changes. In this case, the imaging device can also set parameters corresponding to the focal point position, enabling accurate detection of the workpiece's position.

[0119] Reference Figure 2 In this embodiment, the display unit 28 of the teach pendant 26 displays the values ​​of the parameters set by the parameter setting unit 53. The operator can observe the parameters displayed on the display unit 28 of the teach pendant 26 to confirm the parameter values. In particular, if the three-dimensional position calculated by the feature position calculation unit 55 is incorrect, the operator can confirm the values ​​of the parameters set according to each focal point position.

[0120] However, in this embodiment, the distance calculation unit 56 of the image processing unit 51 can calculate the distance from the camera 6 to the workpiece 38 based on the focal position detected by the focal position detection unit 52. The focal position depends on the distance between the camera 6 and the workpiece 38. Therefore, when the focal position is determined, the distance between the camera 6 and the workpiece 38 can be estimated.

[0121] When focusing on the surface of the plate-shaped portion 38b, the distance calculation unit 56 estimates the distance from the origin of the camera coordinate system 72 to the surface of the plate-shaped portion 38b of the workpiece 38. For example, the operator can pre-generate a function that uses the focus position pp as a variable to calculate the coordinate value of the z-axis of the camera coordinate system 72. The coordinate value of the z-axis of the camera coordinate system 72 corresponds to the distance from the camera 6 to the workpiece 38. The distance calculation unit 56 can use the focus position pp and the function to calculate the coordinate value of the z-axis of the camera coordinate system 72. Alternatively, the operator can determine the distance from the camera 6 to the workpiece 38 for each of a plurality of discrete focus positions. The distance calculation unit 56 can calculate the distance from the camera 6 to the workpiece 38 based on the actually detected focus position pp by calculations such as interpolation.

[0122] Thus, the distance calculation unit 56 of this embodiment can calculate the distance from the camera 6 to the object. Normally, the distance from the camera to the object needs to be predetermined. However, by including the distance calculation unit 56 in the image processing unit 51, the distance from the camera 6 to the workpiece 38 can be calculated. For example, when manually focusing while observing the image, the distance from the camera to the workpiece can be calculated. Therefore, even without setting the distance from the camera to the object, the image processing unit 51 can calculate the three-dimensional position of the feature portion of the workpiece.

[0123] Figure 8 This diagram illustrates a first step in the control of the first robot system in this embodiment. In this other control, the image processing unit 51 detects the three-dimensional position of the workpiece 38 based on an image captured by the camera 6 positioned at a first shooting position. Based on the position of the workpiece 38, the image processing unit 51 calculates a second shooting position that is closer to the workpiece 38 than the first shooting position. As indicated by arrow 102, the image processing unit 51 moves the position and posture of the robot 1 to the second shooting position. The second shooting position is a position where the distance from the object to the vision sensor is smaller than the first shooting position. Furthermore, the second shooting position is approximately the center of the image where the workpiece 38 is positioned.

[0124] Next, the image processing unit 51 calculates the three-dimensional position of the workpiece 38 based on the image captured at the second shooting position. Then, based on the position of the workpiece 38, it implements control to drive the robot 1 to hold the workpiece 38.

[0125] When the workpiece 38 is positioned on the platform 95, its position may sometimes deviate significantly. Therefore, the position and posture of the robot 1 can be predetermined by positioning the camera 6 at a first shooting position away from the workpiece 38. At the first shooting position, autofocus control is implemented using an autofocus function. At the first shooting position, the workpiece 38 is captured as a small image in the image taken by the camera 6. However, the position of the workpiece 38 can be detected by the feature detection unit 54 and the feature position calculation unit 55.

[0126] Next, the motion command generation unit 59 calculates a second shooting position for the camera 6 in order to capture the workpiece 38 at a position closer to the first shooting position. The second shooting position is determined by positioning the workpiece approximately in the center of the image. Furthermore, the second shooting position is set directly above the workpiece 38, indicating the position where the camera 6 is close to the workpiece 38.

[0127] The relationship between the current position of workpiece 38, the amount of movement of camera 6, and the position of workpiece 38 in camera coordinate system 72 is represented by, for example, the relationship of the following equation (12).

[0128] [Formula 6]

[0129]

[0130] [t x , t y , t z ]: Camera movement

[0131] [X, Y, Z]: Current position of the workpiece

[0132] [x, y, z]: Position of the workpiece in the camera coordinate system

[0133] R: Rotation matrix

[0134] Here, in order to position the workpiece 38 approximately in the center of the image, it is sufficient to position the workpiece on the optical axis of the camera. That is, in the camera coordinate system 72, x = y = 0. Substituting x = y = 0 into equation (12) and transforming the equation, we can obtain the following equation (13). Then, we can calculate the amount of camera movement (t) in the reference coordinate system 71 according to equation (13). x t y t z ).

[0135] [Formula 7]

[0136]

[0137] Next, regarding the distance from workpiece 38 to camera 6, the z-axis coordinate value of the surface of workpiece 38 in the camera coordinate system 72 at the first shooting position is set as z'. The ratio of the workpiece size to the image size at the first shooting position is set as a. For example, the ratio of the workpiece length to the image length in one direction of the image can be used as the ratio of the workpiece size to the image size. The ratio of the workpiece size to the image size can be detected by the feature position calculation unit 55.

[0138] Next, the z-axis coordinate value of the surface of workpiece 38 in camera coordinate system 72 at the second shooting position is set as z". The desired scale of the workpiece relative to the image size is set as k. This scale can be predetermined by the operator. Here, when camera 6 moves from the first shooting position to the second shooting position, the focal position of camera 6 changes. However, in the calculation here, it is assumed that even if the shooting position changes from the first shooting position to the second shooting position, the focal length corresponding to the focal position is fixed. The coordinate value z” at the second shooting position can be represented by z” = (k / a)z’. It is possible to move camera 6 closer to workpiece 38 so that the distance from camera 6 to workpiece 38 becomes the coordinate value z”.

[0139] The motion command generation unit 59 changes the position and posture of the robot 1 based on the movement amount of the x-axis, y-axis and z-axis directions of the camera coordinate system 72, as shown by arrow 102, so as to position the camera 6 in the second shooting position.

[0140] Figure 9 This is a schematic diagram illustrating the second process of other controls of the first robot system. Figure 9 This is a schematic diagram of the robot system with the camera positioned in the second shooting position. The second shooting position of camera 6 is closer to workpiece 38 than the first shooting position. In the second shooting position, automatic focus control is implemented using the autofocus function of camera 6. In the image captured at the first shooting position, the position of workpiece 38 is sometimes difficult to detect accurately because its proportion is small. In the image captured at the second shooting position, workpiece 38 occupies a larger proportion. Therefore, the position of workpiece 38 can be calculated more accurately.

[0141] In this way, the motion command generation unit 59 can calculate the second shooting position based on the three-dimensional position of the workpiece 38 in the image captured at the first shooting position. Then, the feature position calculation unit 55 calculates the three-dimensional position of the feature portion based on the image captured at the second shooting position. Even if the focus position of the camera is different between the first and second shooting positions, the three-dimensional position of the feature portion can be detected at each shooting position. In particular, the robot system performs operations based on the image captured at the second shooting position, thereby enabling the work tool to be moved to an accurate position to perform high-precision operations.

[0142] In the first robot system 3, the position of the workpiece 38 is detected in order to hold the workpiece 38 disposed on the platform 95, but this method is not limited to. The image processing unit 51 can detect the workpiece 38 based on the captured image, thereby performing an inspection of the workpiece 38. For example, the image processing unit 51 can measure the size of the workpiece based on the captured image. Moreover, the image processing unit 51 can perform a size inspection of the workpiece based on a predetermined size determination value. In this case, it is also possible not to install a working tool on the robot 1, but only to install a vision sensor on the robot 1.

[0143] As for workpiece inspection, it is not limited to checking the dimensions of the workpiece; any inspection can be performed. For example, it can be used to check whether a predetermined component is disposed on the surface of the workpiece. Alternatively, it can be used to check whether there is damage on the surface of the workpiece. In either case, the position of the feature portion can be accurately detected based on the focal point, thus enabling high-precision inspection.

[0144] In the first robot system 3, the workpiece is stationary while the camera moves via a moving device, but this is not the only option. Alternatively, the camera position can be fixed while the workpiece is moved via the moving device. Or, the moving device can be configured to allow both the camera and the workpiece to move.

[0145] Figure 10 This is a schematic diagram of the second robot system of this embodiment. In the second robot system 4, the camera 6 is fixed to the platform 96. The workpiece 38 is supported by the robot 1. The second robot system 4 transports the workpiece 38, which is placed on the platform 97, to the platform 98, as shown by arrow 103. Through changes in the position and posture of the robot 1, the workpiece 38 is transported from position P38s to position P38e. The imaging device of the second robot system 4 detects the displacement of the position within the hand 5 when the hand 5 holds the workpiece 38.

[0146] The control device 2 controls the position and posture of the robot 1 to position the workpiece 38 at a predetermined shooting position for detecting the three-dimensional position of the workpiece 38. The image processing unit 51 detects the three-dimensional position of characteristic portions of the workpiece 38 based on the image captured by the camera 6. For example, the edge of the bottom surface of the workpiece 38 can be detected as a characteristic portion. The image processing unit 51 detects the position of the workpiece 38. The predetermined position and posture of the robot 1, which serve as a reference position for the workpiece 38, are stored in the storage unit 42. The image processing unit 51 can calculate the offset of the gripping of the workpiece 38 in the hand 5 based on the reference position of the workpiece 38.

[0147] The motion command generation unit 59 calculates the position and posture of the robot 1 based on the offset of the workpiece 38 within the hand 5, so as to place the workpiece 38 in the desired position P38e on the platform 98. Then, the motion control unit 43 drives the robot 1 to place the workpiece 38 in position P38e.

[0148] In the second robot system 4, when the workpiece 38 is positioned at a predetermined shooting position, the camera 6 focuses. The parameter setting unit 53 calculates the parameters of the calculation model based on the focus position. The feature position calculation unit 55 calculates the three-dimensional position of the feature portion based on the calculated parameters. Then, based on the position of the feature portion, the position of the workpiece 38 is detected.

[0149] In the second robot system 4, there may be instances where the photographed position of workpiece 38 deviates from the desired position. Alternatively, the dimensions of workpiece 38 may vary. In such cases, the camera 6 can focus and calculate the accurate position of workpiece 38. As a result, the robot system 4 can move workpiece 38 to the desired position.

[0150] In the second robot system, the image processing unit 51 can also capture an image of the workpiece from a first shooting position away from the camera and detect the approximate position of the workpiece, and then calculate a second shooting position closer to the camera than the first shooting position. Then, the gripping offset of the workpiece 38 can be calculated based on the image captured at the second shooting position. Furthermore, in the second robot system, inspections such as workpiece dimensional checks can also be performed.

[0151] The other structures, functions, and effects of the second robot system are the same as those of the first robot system, so they will not be described again here.

[0152] Figure 11 A schematic diagram showing the transport system of this embodiment. Figure 12 This is a block diagram illustrating the transport system in this embodiment. (Refer to...) Figure 11 as well as Figure 12 The transport system 9 is equipped with a camera to inspect the workpiece 38. The transport system 9 also includes a conveyor belt 7 as a moving device for the workpiece 38. The transport system 9 has a structure in which the conveyor belt 7 is configured to replace the robot 1 of the second robot system 4.

[0153] Workpiece 38 is moved in the direction indicated by arrow 104 by being driven by conveyor belt 7. That is, the position of workpiece 38 changes by driving conveyor belt 7. Camera 6, which serves as a vision sensor, is supported on support member 99.

[0154] The conveying system 9 includes a control device 8 for controlling the conveyor belt 7 and the camera 6. The control device 8 comprises a processing unit including a CPU, etc. The control device 8 includes a conveyor belt drive unit 46. The conveyor belt 7 includes a conveyor belt drive unit 30 having a drive motor for driving the conveyor belt. A position detector 31 for detecting the rotational position of each drive motor is provided. The control device 8 includes an image processing unit 51 for processing images captured by the camera 6.

[0155] The control device 8 includes an operation panel 32. The operation panel 32, like the teach pendant 26, has an input section 27 and a display section 28. The display section 28 can display parameters set by the parameter setting section 53, etc. Other aspects of the control device 8 are similar in structure to... Figure 2 The control device 2 of the robot system shown is the same.

[0156] In the conveying system 9, the camera 6 is fixed at a predetermined position where it can take pictures of the workpiece 38. The conveying system 9 allows for the detection of the position of the workpiece 38 or the inspection of the workpiece 38 based on the images captured by the camera 6. Sometimes the workpiece 38, transported by the conveyor belt 7, stops at different shooting positions. Alternatively, sometimes the size of the workpiece 38 varies individually.

[0157] In the handling system 9, focusing is also performed when photographing workpiece 38. Then, based on the focus position, parameters of a calculation model are set to calculate the three-dimensional position corresponding to a specific position in the image. Based on the parameters, the position of the feature portion of workpiece 38 is calculated. Furthermore, workpiece 38 can be positioned based on the position of its feature portion, or workpiece 38 can be inspected.

[0158] In this way, the position of camera 6 can also be fixed. Furthermore, the moving device can be any device capable of moving the object or the camera. The structure, function, and effects of the other transport systems are the same as those of the first and second robot systems described above, and therefore will not be repeated here.

[0159] In each of the above-described controls, the order of steps can be appropriately changed without altering the function or effect. The above-described implementation methods can be appropriately combined.

[0160] In the above figures, identical or equal parts are labeled with the same symbols. Furthermore, the above embodiments are illustrative and do not limit the invention. Additionally, the embodiments include modifications to the embodiments shown in the scope of the claimed patent.

[0161] Symbol Explanation

[0162] 1 robot;

[0163] 2. Control device;

[0164] 3, 4 Robot Systems;

[0165] 6 cameras;

[0166] 7 conveyor belts;

[0167] 8. Control devices;

[0168] 9. Handling system;

[0169] 24. Focus on adjusting institutions;

[0170] 28 Display Units;

[0171] 37 lenses;

[0172] 38 workpieces;

[0173] 38c edge;

[0174] 40. Main body of the control device;

[0175] 42 storage units;

[0176] 43. Motion control unit;

[0177] 51 Image Processing Department;

[0178] 52. Focus position detection unit;

[0179] 53. Parameter setting section;

[0180] 54 Feature Detection Unit;

[0181] 55 Feature position calculation unit;

[0182] 56 Distance Calculation Unit;

[0183] 59. Action Instruction Generation Unit;

[0184] 61. Action Procedure;

[0185] 63. Setting Information;

[0186] Images 66 and 67;

[0187] Images of holes 68a and 68b;

[0188] 69. Feature section.

Claims

1. A shooting device, characterized in that, have: A visual sensor, which captures images of the object; A focus position detection unit detects the focus position of the vision sensor when it is focusing. The parameter setting unit sets parameters for calculating the three-dimensional position corresponding to a specific position in the image captured by the vision sensor; The storage unit stores setting information for setting parameters corresponding to the focus position; A feature detection unit detects predetermined feature portions in an image of the object; and The feature position calculation unit uses parameters set by the parameter setting unit to calculate the three-dimensional position of the feature portion. The vision sensor is configured to capture two-dimensional images. The feature location calculation unit uses a calculation model to calculate the three-dimensional location of the feature portion based on its specific location in the two-dimensional image. The distance from the vision sensor to the workpiece is predetermined in the calculation model, including parameters of at least one of the vision sensor's internal and external parameters. The parameter setting unit sets at least one parameter in the calculation model based on the focus position and the setting information.

2. The shooting device according to claim 1, characterized in that, The vision sensor has an autofocus function.

3. The shooting device according to claim 1 or 2, characterized in that, The setting information includes parameters predetermined for multiple discrete focal positions. The parameter setting unit sets parameters for calculating the three-dimensional position based on parameters stored in the storage unit for each of the multiple focal positions.

4. The shooting device according to claim 1 or 2, characterized in that, The imaging device includes a movement device for moving at least one of the object and the visual sensor.

5. The shooting device according to claim 4, characterized in that, The shooting device includes: a motion command generation unit that generates motion commands for the moving device to change the shooting position of at least one of the object and the visual sensor. The motion command generation unit calculates the second shooting position based on the three-dimensional position of the object in the image captured at the first shooting position. The second shooting position is a position where the distance from the object to the visual sensor is less than the first shooting position. The feature position calculation unit calculates the three-dimensional position of the feature portion based on the image captured at the second shooting position.

6. The shooting device according to claim 1 or 2, characterized in that, The shooting device includes a display unit that displays the values ​​of parameters set by the parameter setting unit.

7. The shooting device according to claim 1 or 2, characterized in that, The imaging device includes a distance calculation unit that calculates the distance from the visual sensor to the object. The distance calculation unit calculates the distance from the visual sensor to the object based on the focal position detected by the focal position detection unit.

Images