Image processing device and robot system

Abstract

In one embodiment, an image processing device comprises: a detection data setting unit that sets detection data for capturing an image by a camera and detecting a workpiece; an imaging control unit that controls the camera; and a feature detection unit that detects a feature portion of the workpiece by image processing. The detection data setting unit sets an imaging parameter on the basis of the temperature measured by the camera and an imaging reference parameter. The imaging control unit controls the camera such that the camera captures an image of the workpiece on the basis of the imaging parameter. The feature detection unit detects the feature portion of the workpiece on the basis of the image of the workpiece and processing data.

Description

Image processing device and robot system

[0001] The present disclosure relates to an image processing device and a robot system.

[0002] Image processing devices that detect the position of an object by processing an image of the object captured by a camera have been known. For example, a device is known that detects the position of a workpiece by capturing an image of the workpiece placed at a predetermined distance from the camera using a two-dimensional camera. Another device is known that detects the three-dimensional position of a workpiece using a camera that measures the three-dimensional position, such as a stereo camera equipped with two two-dimensional cameras.

[0003] JP 2013-93013 A

[0004] The image captured by the camera may be affected depending on the conditions at the time the image is captured by the camera. For example, the temperature of the camera may rise as the camera captures images continuously, or the temperature of the camera may rise or fall depending on the temperature of the surrounding environment. Alternatively, the temperature of the camera may rise significantly depending on the time elapsed since the power was turned on. As a result, changes in the camera's temperature may cause distortion of the camera's light-receiving element and lens. This may result in distortion of the image. In a device that detects the position of a workpiece, changes in the camera's temperature may make it impossible to detect the workpiece from the image, or the workpiece may be detected as being misaligned.

[0005] Alternatively, the amount of light emitted by the lighting device placed when capturing an image of a workpiece may change as the temperature changes. That is, even if the lighting device is controlled in a certain way, the amount of light emitted may change depending on the temperature of the lighting device. As a result, the workpiece may be too bright or too dark, making it impossible to detect. Or, the workpiece may be detected in a misaligned position.

[0006] A first image processing device of the present disclosure includes a temperature acquisition unit that acquires the measured temperature of the camera, and a detection data setting unit that sets detection data for capturing an image with the camera and detecting a workpiece. The image processing device includes a memory unit that stores detection reference data, which is reference data for setting the detection data, and an imaging control unit that controls the camera that captures the image. The image processing device includes a feature detection unit that detects characteristic portions of the workpiece through image processing. The detection data includes imaging parameters for capturing the image and processing data for performing the image processing. The detection data setting unit sets the imaging parameters based on the measured temperature of the camera and the imaging reference parameters. The imaging control unit controls the camera to capture an image of the workpiece based on the imaging parameters. The feature detection unit detects the characteristic portions of the workpiece based on the image of the workpiece and the processing data, and detects the position of the workpiece based on the characteristic portions.

[0007] A second image processing device of the present disclosure includes a temperature acquisition unit that acquires the measured temperature of the lighting device, and a detection data setting unit that sets detection data for capturing an image with a camera and detecting a workpiece. The image processing device includes a memory unit that stores detection reference data, which is reference data for setting the detection data, and an imaging control unit that controls the camera that captures the image. The image processing device includes a feature detection unit that detects characteristic portions of the workpiece through image processing. The detection data includes imaging parameters for capturing the image and processing data for performing the image processing. The detection data setting unit sets the imaging parameters based on the measured temperature of the lighting device and the imaging reference parameters. The imaging control unit controls the camera to capture an image of the workpiece based on the imaging parameters. The feature detection unit detects the characteristic portions of the workpiece based on the image of the workpiece and the processing data, and calculates the position of the workpiece based on the characteristic portions.

[0008] The robot system of the present disclosure includes the image processing device described above, a camera that captures an image of a workpiece, and a robot that moves the camera or the workpiece.

[0009] 1 is a schematic diagram of a robot system according to a first embodiment. FIG. 2 is a block diagram of a robot system according to the first embodiment. FIG. 3 is an explanatory diagram of detection reference data. FIG. 4 is a plan view of a workpiece according to an embodiment. FIG. 5 is an image of a workpiece according to an embodiment. FIG. 6 is a control flowchart of a robot system performing work according to an embodiment. FIG. 7 is a control flowchart of generating detection reference data according to the first embodiment. FIG. 8 is a control flowchart of automatically creating processing reference data according to the first embodiment. FIG. 9 is a perspective view of a camera including a lighting device. FIG. 10 is a block diagram of a robot system according to a second embodiment. FIG. 11 is a control flowchart of generating a light intensity reference parameter for a lighting device according to the second embodiment.

[0010] 1 to 9, an image processing device and a robot system including the image processing device according to a first embodiment will be described. The image processing device of this embodiment detects the position of a workpiece based on an image captured by a camera.

[0011] Fig. 1 is a schematic diagram of a robot system equipped with an image processing device according to this embodiment. Fig. 2 is a block diagram of the robot system according to this embodiment. Referring to Figs. 1 and 2, a robot system 3 according to this embodiment detects the position of a workpiece 38 as an object. The robot system 3 grasps the workpiece 38 and transports it to a predetermined position.

[0012] The robot system 3 includes a robot device. The robot device includes a hand 5 as a work tool for gripping a workpiece 38, a robot 1 for moving the hand 5, and a control device 2 for controlling the hand 5 and the robot 1. The robot system 3 also includes a stand 34 on which the workpiece 38 is placed.

[0013] The hand 5 in this embodiment is a work tool that grips and releases the workpiece 38. The work tool attached to the robot 1 is not limited to this form, and any work tool can be used depending on the work to be performed by the robot system 3. For example, a spray gun for painting the workpiece and a welding torch for welding can be used as the work tool.

[0014] The robot 1 of this embodiment is an articulated robot including multiple joints 18. The robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is supported by a swivel base 13. The swivel base 13 is supported by a base 14. The robot 1 includes a wrist 15 connected to the end of the upper arm 11. The wrist 15 includes a flange 16 to which a hand 5 is fixed. The components of the robot 1 are configured to rotate around a predetermined drive axis. The robot 1 is not limited to this configuration, and any robot that can change the position and posture of a work tool can be used.

[0015] The robot 1 of this embodiment includes a robot driving device 21 having a driving motor that drives components such as the upper arm 11. The hand 5 includes a hand driving device 22 that drives the hand 5. The hand driving device 22 of this embodiment drives the hand 5 by air pressure. The hand driving device 22 includes an air pump and a solenoid valve for supplying compressed air to a cylinder.

[0016] The control device 2 includes a control device main body 40 and a teaching pendant 26 for an operator to operate the control device main body 40. The control device main body 40 includes an arithmetic processing device (computer) having a CPU (Central Processing Unit) as a processor. The arithmetic processing device has RAM (Random Access Memory), ROM (Read Only Memory), etc. connected to the CPU via a bus. The robot 1 is driven based on operation commands from the control device 2.

[0017] The control device main body 40 includes a storage unit 42 that stores any information related to the robot system 3. The storage unit 42 can be configured with a non-transitory storage medium capable of storing information. For example, the storage unit 42 can be configured with a storage medium such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium. The processor is configured to be able to read the information stored in the storage unit 42.

[0018] An operation program 66 created in advance for operating the robot 1 is input to the control device 2. Alternatively, an operator can set teaching points for the robot 1 by operating the teaching pendant 26 to drive the robot 1. The control device 2 can then generate the operation program 66 based on the teaching points. The operation program 66 is stored in the storage unit 42.

[0019] The operation control unit 43 sends operation commands to the robot driving unit 44 to drive the robot 1 based on the operation program 66. The robot driving unit 44 includes an electrical circuit that drives the drive motor, and supplies electricity to the robot driving device 21 based on the operation commands. The operation control unit 43 also sends operation commands to the hand driving unit 45 to drive the hand driving device 22. The hand driving unit 45 includes an electrical circuit that drives an air pump or the like, and supplies electricity to the air pump or the like based on the operation commands.

[0020] The operation control unit 43 corresponds to a processor that operates in accordance with the operation program 66. The processor reads the operation program 66 and performs the control defined in the operation program 66, thereby functioning as the operation control unit 43.

[0021] The robot 1 includes a state detector for detecting the position and posture of the robot 1. In this embodiment, the state detector includes a position detector 23 attached to the drive motor of each drive shaft of the robot drive device 21. The position detector 23 can be configured, for example, by an encoder that detects the rotational position of the output shaft of the drive motor. The position and posture of the robot 1 are detected based on the output of each position detector 23.

[0022] In the robot system 3, a reference coordinate system 71 is set that remains stationary even when the position and posture of the robot 1 change. In the example shown in Fig. 1, the origin of the reference coordinate system 71 is located on the base 14 of the robot 1. The reference coordinate system 71 is also called a world coordinate system. In the reference coordinate system 71, the position of the origin is fixed, and further, the orientation of the coordinate axes is fixed.

[0023] A tool coordinate system 72 is set in the robot system 3, with its origin set at an arbitrary position on the work tool. The position and orientation of the tool coordinate system 72 change along with the work tool. In this embodiment, the origin of the tool coordinate system 72 is set at the tool tip point of the hand 5. The position of the robot 1 corresponds to the position of the tool tip point in the reference coordinate system 71 (the position of the origin of the tool coordinate system 72). The orientation of the robot 1 corresponds to the orientation of the tool coordinate system 72 with respect to the reference coordinate system 71.

[0024] The teaching pendant 26 is connected to the control device main body 40 via a communication device. The teaching pendant 26 includes an input unit 27 for inputting information about the robot 1 and the hand 5. The input unit 27 is composed of input members such as a keyboard and a dial. The teaching pendant 26 includes a display unit 28 for displaying information about the robot 1 and the hand 5. The display unit 28 can be composed of a display panel capable of displaying information, such as a liquid crystal display panel or an organic EL (Electro Luminescence) display panel. When the teaching pendant is equipped with a touch panel type display panel, the display panel functions as both the input unit and the display unit.

[0025] The robot system 3 in this embodiment includes an imaging device that detects the position of the workpiece 38. The imaging device includes a camera 6 that captures an image of the workpiece 38. The camera 6 in this embodiment is a two-dimensional camera that captures a two-dimensional image. The camera 6 in this embodiment is supported by the robot 1. The camera 6 is fixed to the hand 5 via a support member. The position and orientation of the camera 6 change together with the hand 5. In other words, when the position and orientation of the robot 1 change, the position and orientation of the camera 6 also change.

[0026] The camera 6 can capture an image in a field of view 6a. The camera 6 of this embodiment is provided with a temperature sensor 6c for detecting the temperature of the camera 6. The temperature sensor 6c can be attached to any part of the camera. For example, the temperature sensor 6c can be arranged to measure the temperature of the outer surface of the housing of the camera 6, the temperature of a substrate inside the housing of the camera 6, or the temperature of the air around the camera 6.

[0027] The camera 6 of this embodiment includes a temperature adjustment device 6b for adjusting the temperature of the camera 6. The temperature adjustment device 6b of this embodiment includes a blower fan for cooling the camera 6 and a heater for raising the temperature of the camera. The temperature adjustment device 6b is not limited to this configuration, and any device capable of adjusting the temperature of the camera can be used. For example, the temperature adjustment device may include a semiconductor cooling device such as a Peltier element or a hot air generator that supplies hot air. Alternatively, the temperature adjustment device may be configured to adjust the temperature of the camera by turning on and off the power or communication of the camera.

[0028] The imaging device of this embodiment includes an illumination device 7 that illuminates a workpiece 38. The illumination device of this embodiment includes an LED (Light Emitting Diode). The illumination device 7 is supported on a stand 35 so as to illuminate the workpiece 38. The illumination device 7 of this embodiment is configured to be able to adjust the amount of light. For example, the illumination device is configured to be able to control the current supplied to the LED. In this case, the light amount parameter that controls the amount of light is the current value supplied to the LED. Alternatively, the illumination device 7 may be configured to be able to adjust the amount of light by PWM (Pulse Width Modulation) control or the like.

[0029] In the robot system 3, a camera coordinate system 73 is set relative to the camera 6. The position and orientation of the camera coordinate system 73 change along with the camera 6. The origin of the camera coordinate system 73 is set at a predetermined position of the camera 6, such as the lens center or optical center of the camera 6. The camera coordinate system 73 has an X-axis, a Y-axis, and a Z-axis that are orthogonal to one another. In the present embodiment, the camera coordinate system 73 is set so that the Z-axis extends in a direction parallel to the optical axis of the lens of the camera 6. In this embodiment, since the camera 6 is fixed to the hand 5, the relative position and orientation of the camera coordinate system 73 with respect to the tool coordinate system 72 are constant. The relative position and orientation of the camera coordinate system 73 with respect to the tool coordinate system 72 are measured in advance.

[0030] The robot system 3 includes an image processing device that processes images captured by the camera 6. In this embodiment, the control device main body 40 functions as the image processing device. The control device main body 40 includes an image processing unit 51 that processes images captured by the camera 6. The image processing unit 51 includes an imaging control unit 56 that sends a command to the camera 6 to capture an image. The imaging control unit 56 sends a command to turn the lighting device 7 on or off. Furthermore, the imaging control unit 56 sends a command to change a light intensity parameter that controls the light intensity of the lighting device 7.

[0031] The image processing unit 51 includes a temperature acquisition unit 52 that acquires the temperature measured by the camera 6 from the temperature sensor 6c. The image processing unit 51 includes a detection data setting unit 53 that sets detection data for capturing an image with the camera 6 and detecting the workpiece 38 based on the temperature measured by the camera 6. The image processing unit 51 includes a feature detection unit 54 that detects predetermined feature portions of the workpiece 38 in the image of the workpiece 38 by image processing. The feature detection unit 54 calculates a specific position as a three-dimensional position of the feature portion of the workpiece 38 using the detection data set by the detection data setting unit 53. The image processing unit 51 includes an operation command generation unit 55 that generates operation commands for the robot 1 and the hand 5 based on the results of image processing.

[0032] The image processing unit 51 corresponds to a processor that operates in accordance with the operation program 66. In particular, each of the units, namely, the temperature acquisition unit 52, the detection data setting unit 53, the feature detection unit 54, the operation command generation unit 55, and the imaging control unit 56, corresponds to a processor that operates in accordance with the operation program 66. The processor reads the operation program 66 and performs the control defined in the operation program 66, thereby functioning as each unit.

[0033] 1, in the robot system 3 of this embodiment, a workpiece 38 is placed on the surface of the pedestal 34 by a predetermined method. For example, a worker or another robot system places the workpiece 38 at a predetermined position on the surface of the pedestal 34. The control device 2 changes the position and posture of the robot 1, and the hand 5 grasps the workpiece 38 placed on the top surface of the pedestal 34. The robot 1 then transports the workpiece 38 to the predetermined position.

[0034] When the workpiece 38 is placed on the surface of the base 34, the position of the workpiece 38 on the base 34 may shift. Furthermore, the workpiece 38 has dimensional errors. In the robot system 3 of this embodiment, the three-dimensional position of the workpiece 38 is detected based on an image captured by the camera 6. In the robot system 3 of this embodiment, when the workpiece 38 is to be grasped, the camera 6 captures an image of the workpiece 38. The image processing unit 51 calculates the three-dimensional position of the workpiece 38 based on the image of the workpiece 38. The image processing unit 51 generates operation commands for the position and posture of the robot 1 so as to correspond to the position of the workpiece 38. Then, the operation control unit 43 changes the position and posture of the robot 1 to grasp the workpiece 38 with the hand 5.

[0035] 3 is a block diagram of detection reference data in this embodiment. Referring to FIGS. 2 and 3, storage unit 42 stores detection reference data 67, which is reference data for setting detection data. Detection reference data 67 includes imaging reference parameters 67a, which are reference data for setting imaging parameters used when capturing an image with a camera. Detection reference data 67 also includes processing reference data 67b, which is reference data for setting processing data used when processing an image in image processing unit 51.

[0036] Fig. 4 shows a plan view of a workpiece according to this embodiment. Referring to Figs. 1 and 4, workpiece 38 according to this embodiment has plate-shaped portion 38a and plate-shaped portion 38b formed above plate-shaped portion 38a. Each of plate-shaped portions 38a, 38b has a rectangular parallelepiped shape. Plate-shaped portion 38b has edge portion 38c on the outer periphery of its upper surface. In this embodiment, edge portion 38c is a characteristic portion of workpiece 38.

[0037] 1 , in this embodiment, when the workpiece 38 is gripped, the camera 6 is positioned vertically above the workpiece 38. At this time, the distance from the surface of the plate-like portion 38b having the characteristic portion of the workpiece 38 to the camera 6 is determined in advance. In this example, the position and attitude of the robot 1 are controlled so that the position of the upper surface of the plate-like portion 38b is at a predetermined Z-axis value in the camera coordinate system 73. In addition, the attitude of the camera 6 is adjusted so that the optical axis of the lens of the camera 6 is approximately perpendicular to the surface of the plate-like portion 38b of the workpiece 38.

[0038] 5 shows an example of an image of a workpiece captured by a camera. Image 68 includes an image 69 of the workpiece 38. An image coordinate system 74 is also set for image 68 captured by the camera. The feature detection unit 54 of the image processing unit 51 performs pattern matching to detect the edge 38c as a characteristic part of the workpiece 38. In pattern matching, a template image for detecting the position of the edge 38c is created in advance and stored in the storage unit 42. The template image is, for example, an image of the edge 38c actually captured by the camera 6. Alternatively, the template image may be generated from three-dimensional design data of the workpiece, etc.

[0039] The feature detection unit 54 uses the template image to detect the edge 38c, which is a characteristic part, in the image 68 captured by the camera 6. The feature detection unit 54 calculates the specific position of the characteristic part of the workpiece 38 in the image captured by the camera 6. For example, the feature detection unit 54 detects the position of the center of gravity of the rectangle of the edge 38c in the image 69 using coordinate values ​​in the image coordinate system 74.

[0040] The feature detection unit 54 detects the three-dimensional specific position of the feature using a calculation model based on the specific position of the feature in the image 68. The specific position of such a feature of the workpiece 38 can be detected in a camera coordinate system 73. Then, the feature detection unit 54 converts the coordinate values ​​of the camera coordinate system 73 into coordinate values ​​of a tool coordinate system 72. Furthermore, the feature detection unit 54 can convert the coordinate values ​​of the tool coordinate system 72 into coordinate values ​​of a reference coordinate system 71 based on the position and posture of the robot 1.

[0041] The feature detection unit 54 can calculate the three-dimensional position of the workpiece 38 based on the three-dimensional specific position of the feature portion of the workpiece 38. As the position of the workpiece 38, the position of any set point set for the workpiece 38 can be calculated. For example, the three-dimensional specific position of the feature portion of the workpiece 38 can be set as the position of the workpiece 38. The feature detection unit 54 can also calculate the position of the workpiece 38 in the reference coordinate system 71.

[0042] The operation command generation unit 55 calculates the position and orientation of the robot 1 based on the three-dimensional position of the workpiece 38 calculated by the feature detection unit 54. Then, the operation command generation unit 55 sends the position and orientation of the robot 1 for gripping the workpiece 38 to the operation control unit 43. Based on the operation command received from the operation command generation unit 55, the operation control unit 43 drives the robot 1 and the hand 5 to grip the workpiece 38.

[0043] In the control of workpiece transport in this embodiment, the feature detection unit 54 detects feature portions of the workpiece 38 and calculates the three-dimensional position of the workpiece 38 based on the specific positions of the feature portions. This allows the robot system 3 to grip the workpiece 38 with high precision. In particular, even if the position of the workpiece 38 on the platform 34 is deviated from the reference position or if there is a dimensional error in the workpiece 38, the robot system 3 can grip the workpiece 38 with high precision.

[0044] However, the temperature of the camera may change depending on the temperature of the environment around the camera and the operating state of the camera. If the temperature of the camera changes, distortion may occur in the image captured by the camera. Alternatively, if the camera's exposure time, aperture, gain, and light intensity of the lighting device are not appropriate, the image may be too bright or too dark. In these cases, the image processing unit 51 may not be able to accurately detect the characteristic parts of the workpiece.

[0045] In this embodiment, the detection data used when performing work is changed based on the temperature of the camera 6. The detection data in this embodiment includes imaging parameters for capturing an image with the camera 6 and processing data for the image processing unit 51 to process the image. The imaging parameters are parameters used by the camera 6 when capturing an image. The imaging parameters include, for example, at least one parameter from the exposure time of the camera 6, the aperture of the camera 6, the gain of the camera 6, and the light amount parameter of the lighting device 7. The gain in this embodiment includes gains for converting light-related variables into electrical signals, such as a gain when converting the intensity of light received by the camera into an electrical signal and a gain when converting the amount of light incident on the light-receiving element into an electrical signal.

[0046] The processing data is data used when performing image processing after capturing an image with the camera 6. In this embodiment, the processing data includes at least one of a template image of a characteristic part of a workpiece for pattern matching and a specific position of the characteristic part in the template image. The position of the workpiece can be calculated based on the specific position of the characteristic part.

[0047] 1 to 3, the detection data setting unit 53 sets imaging parameters based on the temperature measured by the camera 6 and imaging reference parameters 67a. The detection data setting unit 53 sets processing data based on the temperature measured by the camera 6 and processing reference data 67b.

[0048] In this example, one piece of detection reference data 67 is defined for one temperature measured by the camera 6. That is, one imaging reference parameter 67a and one piece of processing reference data 67b are defined for one temperature measured by the camera 6. The storage unit 42 stores multiple pieces of detection reference data 67 corresponding to multiple temperatures measured by the camera 6. For example, the temperatures measured by the camera 6 are defined at 5°C intervals within a range of 30°C to 60°C. Detection reference data 67 corresponding to each measured temperature is defined. That is, the imaging reference parameters 67a define the camera exposure time, camera aperture, camera gain, and light intensity parameters of the lighting device for every 5°C of the temperature measured by the camera. The processing reference data 67b defines a template image of a characteristic portion of a workpiece and a specific position of the characteristic portion in the template image for every 5°C of the temperature measured by the camera.

[0049] The detection data setting unit 53 of this embodiment sets imaging parameters based on a plurality of imaging reference parameters 67a corresponding to a plurality of measurement temperatures, and sets processing data based on a plurality of processing reference data 67b corresponding to a plurality of measurement temperatures. For example, the detection data setting unit 53 selects two measurement temperatures that are closest to the temperatures actually measured by the camera. By interpolating or extrapolating the imaging reference parameters corresponding to the respective measurement temperatures using an interpolation formula, the imaging parameters corresponding to the temperatures measured by the camera can be calculated.

[0050] Furthermore, the detection data setting unit 53 can generate a template image having an intermediate shape between the contours of two template images included in the processing reference data corresponding to two temperatures measured by the camera by generating an intermediate contour between the contours of the two template images. Furthermore, the detection data setting unit 53 can set the specific position of the characteristic part in the newly created template image using any calculation method. For example, the detection data setting unit 53 can select the center of gravity of the shape of the characteristic part in the newly created template image as the specific position. Alternatively, the detection data setting unit 53 may select one corner of the characteristic part as the specific position.

[0051] In this way, the detection data setting unit 53 can set the imaging parameters based on a plurality of imaging reference parameters corresponding to a plurality of temperatures measured by the camera. This control allows imaging to be performed using imaging parameters corresponding to the actual temperatures measured by the camera, thereby preventing the image from being too bright or too dark. In other words, it is possible to prevent the image from becoming unclear.

[0052] Furthermore, the detection data setting unit 53 can set processing data based on multiple processing reference data corresponding to multiple temperatures measured by the camera. By performing image processing using processing data corresponding to the actual temperatures measured by the camera, it is possible to detect characteristic parts of the workpiece while reflecting image distortion, etc. This makes it possible to prevent the characteristic parts of the workpiece from being unable to be detected due to image distortion. Furthermore, the detection accuracy of the workpiece position is improved. Note that in this embodiment, processing reference data for the temperature actually measured by the camera is generated based on processing reference data for each of multiple temperatures, but this is not limited to this form. A single predetermined processing reference data may be adopted regardless of the temperature measured by the camera.

[0053] However, the detection reference data is not limited to this format. One detection reference data may be defined for multiple temperatures measured by the camera. In other words, common detection reference data may be included for multiple temperatures measured by the camera.

[0054] Alternatively, the detection data setting unit 53 may set the imaging reference parameters of the measurement temperature closest to the temperature measured by the camera as the imaging parameters of the current camera.The detection data setting unit 53 may set the processing reference data of the measurement temperature closest to the temperature measured by the camera as the processing data of the current image.

[0055] 6 shows a control flowchart for capturing an image of a workpiece and transporting the workpiece using a robot system. Referring to FIGS. 1, 2, and 6, in step 81, robot 1 places camera 6 in a predetermined position and orientation based on operation program 66. In this embodiment, camera 6 is placed directly above workpiece 38. In step 82, temperature acquisition unit 52 of image processing unit 51 acquires the temperature measured by camera 6 based on the output of temperature sensor 6c of camera 6.

[0056] In step 83, the detection data setting unit 53 sets detection data based on the temperature measured by the camera 6. The detection data setting unit 53 can set appropriate imaging parameters and processing data corresponding to the temperature measured by the camera 6.

[0057] Next, in step 84, the imaging control unit 56 captures an image with the camera 6 using the imaging parameters set by the detection data setting unit 53. That is, an image can be captured using appropriately selected exposure time, aperture, gain, and light intensity parameters of the lighting device.

[0058] In step 85, the feature detection unit 54 detects the feature portion of the workpiece using the processing data set by the detection data setting unit 53. In this embodiment, the edge 38c of the workpiece 38 is detected as a contour. The feature detection unit 54 detects the specific position of the feature portion based on the contour of the feature portion. For example, the feature detection unit 54 calculates the position of the center of gravity of the contour. The specific position of the feature portion can be detected in the image coordinate system 74 and then calculated in the camera coordinate system 73 and the tool coordinate system 72. Then, the feature detection unit 54 can calculate the specific position of the feature portion in the reference coordinate system 71 based on the position and posture of the robot 1.

[0059] In step 86, the feature detection unit 54 calculates the position of the workpiece based on the specified position of the feature portion. The position of the workpiece can be calculated in the reference coordinate system 71. Here, the specified position of the feature portion and the position of the workpiece may be the same.

[0060] Next, in step 87, the motion command generation unit 55 calculates the position and orientation of the robot for gripping the workpiece 38 based on the position of the workpiece in the reference coordinate system 71. In this case, the motion command generation unit 55 may calculate a correction amount for the position and orientation of the robot that is predetermined in the operation program. The motion command generation unit 55 generates a motion command for the position and orientation of the robot.

[0061] In step 88, the operation control unit 43 drives the robot 1 based on the operation command received from the operation command generation unit 55. The operation control unit 43 drives the hand 5 so that the hand 5 grips the workpiece 38. Thereafter, the operation control unit 43 drives the robot 1 based on the operation program 66, thereby transporting the workpiece 38 to a desired target position.

[0062] In this embodiment, imaging parameters are set corresponding to the temperature measured by the camera, and processing data is also set corresponding to the temperature measured by the camera. As a result, images with excellent workpiece detection accuracy can be captured even if the camera temperature changes. Furthermore, the specific position of the characteristic part can be calculated with high accuracy even if the camera temperature changes. As a result, the detection accuracy of the workpiece position is improved, and the workpiece can be gripped with high accuracy.

[0063] To suppress temperature changes in the camera and lighting device, it is conceivable to install a heat dissipation device or cooling device to maintain a constant temperature while the robot system is working. However, installing such a device increases the power consumption of the robot system while it is working. Another method is to lengthen the camera's image capture interval, thereby cooling the camera and lighting device with ambient air and suppressing temperature changes. However, this increases the waiting time for cooling. As a result, the cycle time increases, and there are problems with reduced work efficiency.

[0064] In contrast, the image processing device of this embodiment uses detection data corresponding to the temperature measured by the camera, so the position of the workpiece can be detected with high accuracy when the robot system performs work without driving a heat dissipation device, etc. Also, there is no need to lengthen the interval between images to cool the camera, which can prevent a decrease in work efficiency.

[0065] Next, control for creating detection reference data in this embodiment will be described. Referring to FIG. 2, image processing unit 51 in this embodiment includes a reference data adjustment unit 57 that adjusts the detection reference data. Reference data adjustment unit 57 includes a temperature adjustment unit 58 that controls temperature adjustment device 6b of camera 6. That is, temperature adjustment unit 58 adjusts the temperature of camera 6. Reference data adjustment unit 57 includes a reference data generation unit 60 that generates detection reference data. Reference data adjustment unit 57 includes a similarity determination unit 59 that determines the degree of similarity between a template image used in pattern matching and an image of a characteristic portion captured by camera 6.

[0066] Each of the units, the reference data adjustment unit 57, the temperature adjustment unit 58, the reference data generation unit 60, and the similarity determination unit 59, corresponds to a processor that operates in accordance with the operation program 66. The processor reads the operation program and performs the control defined in the operation program, thereby functioning as each unit.

[0067] FIG. 7 shows a flowchart of a first generation control for generating detection reference data according to this embodiment. Referring to FIGS. 1, 2, and 7, in step 91, the operator sets the camera's measured temperatures TC1 to TCn. For example, the measured temperatures TC1 to TCn may be temperatures within the temperature range reached by the camera 6 during the actual image capture period. Next, the operator operates the teaching console 26 to place the camera 6 at a predetermined position. Here, the operator adjusts the position of the robot so that the camera 6 is positioned vertically above the workpiece 38. The operator also adjusts the robot's posture so that the optical axis of the camera 6 is perpendicular to the surface of the characteristic portion of the workpiece 38.

[0068] In step 93, variable i is set to 1. In step 94, temperature adjustment unit 58 controls temperature adjustment device 6b to adjust the temperature of camera 6 to measured temperature TCi. At this time, temperature acquisition unit 52 can acquire the temperature of camera 6 from temperature sensor 6c. Temperature adjustment unit 58 can control temperature adjustment device 6b based on the measured temperature of camera 6 acquired by temperature acquisition unit 52.

[0069] In step 95, when the camera temperature reaches the measurement temperature TCi, detection reference data Si is generated. Here, the operator repeatedly captures images of the workpiece and checks the images while changing the detection data. The operator can set imaging parameters that enable accurate detection of the workpiece. For example, the operator sets the camera's exposure time, aperture, gain, and light intensity parameters of the lighting device 7 so that the contours of the image of the workpiece's characteristic parts are clearly defined.

[0070] Furthermore, the operator can generate processing reference data based on the captured workpiece image. The operator can generate a template image by specifying the outline of a characteristic part for the workpiece image displayed on the display unit 28 of the teaching pendant 26. The operator can also set a specific position of the characteristic part for the template image of the characteristic part. For example, the center of gravity of the planar shape of the characteristic part can be set to a specific position. In this way, one detection reference data Si can be created for the measured temperature TCi of one camera.

[0071] In step 96, the operator operates the input section 27 of the teaching pendant 26 to store the temperature TCi measured by the camera and the detection reference data Si as a pair in the storage section 42.

[0072] Next, in step 97, it is determined whether the variable i has reached the upper limit value n. If the variable i has not reached the upper limit value n, control proceeds to step 98. In step 98, 1 is added to the variable i. Then, the control from step 94 to step 97 is repeated. Detection reference data Si for each measured temperature TCi is generated. If the variable i has reached the upper limit value n in step 97, this control ends.

[0073] In this way, while gradually changing the temperature TCi measured by the camera, it is possible to generate detection reference data Si corresponding to each measured temperature TCi. Here, n pieces of detection reference data can be created for n predetermined measured temperatures.

[0074] FIG. 8 shows a flowchart of the second generation control for generating detection reference data in this embodiment. In the second generation control, processing reference data is generated for each measured temperature. In the second generation control, a template image is automatically generated. Also, specific positions of characteristic portions of the template image are automatically set.

[0075] Steps 101 to 106 are similar to steps 91 to 96 of the first generation control (see FIG. 7). In step 101, the operator sets the camera measurement temperatures TC1 to TCn. In step 102, the operator places the camera 6 in a predetermined position. In step 103, the variable i is set to 1. In step 104, the temperature adjustment unit 58 adjusts the temperature of the camera 6 to the measurement temperature TC1.

[0076] In step 105, the operator creates detection reference data S1 using control similar to that in step 95 of the first generation control of this embodiment (see FIG. 7 ). The operator generates imaging reference parameters I1 by repeatedly capturing images of the workpiece and adjusting the imaging parameters. Furthermore, the operator generates processing reference data P1 based on the images captured using the imaging reference parameters I1.

[0077] In step 106, the storage unit 42 stores the measured temperature TC1 and the detection reference data S1 as a set. At this time, the detection reference data S1 includes the imaging reference parameter I1 and the processing reference data P1.

[0078] Next, in steps 111 to 118, the reference data adjustment unit 57 automatically generates processing reference data Pi. In step 111, the reference data generation unit 60 adds 1 to the variable i. In step 112, the temperature adjustment unit 58 adjusts the temperature of the camera 6 to the measurement temperature TCi. In step 113, the imaging control unit 56 images the workpiece 38 using the imaging reference parameter I(i-1) of the current detection reference data S(i-1).

[0079] In step 114, the feature detection unit 54 performs pattern matching using a template image included in the current processing reference data P(i-1). The feature detection unit 54 calculates the similarity between the image of the feature portion captured by the camera 6 and the template image included in the processing reference data P(i-1). The feature detection unit 54 can calculate a score indicating the similarity using, for example, the SSD (Sum of Squared Difference) method or the SAD (Sum of Absolute Difference) method. The feature detection unit 54 detects the position of the template image relative to the image captured by the camera so as to show the best similarity. The feature detection unit 54 detects the feature portion based on the position of the template image.

[0080] Furthermore, the similarity determination unit 59 determines whether the score indicating the similarity is outside the determination range. In this example, if the similarity is low, it is outside the determination range. If the score indicating the similarity is not outside the determination range, control returns to step 111. In other words, if the similarity is high, control returns to step 111. Then, 1 is added to the variable i, and steps 112 to 114 are repeated. Images are captured while the temperature TCi measured by the camera is changed slightly, and the similarity determination is repeated.

[0081] In step 114, if the score indicating the similarity falls outside the determination range, control proceeds to step 115. That is, if the similarity is low, control proceeds to step 115.

[0082] In step 115, the reference data generating unit 60 detects the contour of the characteristic portion of the workpiece in the image newly captured in step 113. That is, the contour of the characteristic portion of the workpiece captured at the measurement temperature TCi is detected. The contour of the characteristic portion can be detected using the current template image included in the processing reference data P(i-1). Then, the reference data generating unit 60 detects a portion with high contrast in a range close to the contour line based on the current template image. The reference data generating unit 60 can detect a portion with contrast greater than a predetermined judgment value as the contour of the characteristic portion of the workpiece.

[0083] In step 116, the reference data generating unit 60 sets the image of the characteristic portion of the workpiece generated from the newly captured image as a new template image. Next, the reference data generating unit 60 sets the specific position of the characteristic portion in the newly generated template image. The reference data generating unit 60 can set the specific position of the characteristic portion using any calculation method. For example, the reference data generating unit 60 can set the center of gravity of the outline shape of the characteristic portion as the specific position. Alternatively, the specific position may be a corner of the characteristic portion. In this way, the reference data generating unit 60 can generate processing reference data Pi corresponding to the measured temperature TCi so as to include the newly generated template image and the specific position of the template image.

[0084] In step 117, the storage unit 42 stores a pair of the measured temperature TCi and the detection reference data Si including the processing reference data Pi. In step 118, the reference data generation unit 60 determines whether the variable i has reached the upper limit value n. If the variable i has not reached the upper limit value n in step 118, the control returns to step 111. 1 is added to the variable i. Then, the control from step 112 to step 117 is repeated. If the variable i has reached the upper limit value n in step 118, this control ends.

[0085] In the second generation control for generating the detection reference data of this embodiment, steps 101 to 106 can be performed by an operator operating the teaching pendant 26. In contrast, steps 111 to 118 can be performed automatically by the reference data adjustment unit 57 to generate the processing reference data.

[0086] In particular, the reference data generating unit 60 detects the outline of a characteristic part of the workpiece in the image captured by the camera. Based on the outline of the characteristic part, the reference data generating unit 60 can set a template image of the characteristic part and a specific position in the template image according to the temperature measured by the camera. This control makes it possible to automatically create multiple detection reference data corresponding to multiple measured temperatures.

[0087] In the second generation control for generating detection reference data, the similarity determination unit 59 determines the similarity between the template image and the image of the characteristic portion captured by the camera. If the similarity falls outside a predetermined determination range, the reference data generation unit 60 generates detection reference data corresponding to the temperature measured by the camera based on the image of the characteristic portion captured by the camera. Here, if the similarity between the current template image and the newly captured image of the characteristic portion is high, accurate pattern matching can be performed even if the temperature changes. The control for determining the similarity in this embodiment prevents the generation of more detection reference data than necessary.

[0088] 9 shows a perspective view of another camera according to this embodiment. In this camera 8 according to this embodiment, a camera body 8a and a lighting device 8c are integrated together. The lighting device 8c here includes an LED. The lighting device 8c is arranged around the lens 8b of the camera 8. This type of camera 8 can be fixed to the wrist of the robot 1 or to a work tool. By employing a camera in which the camera body 8a and the lighting device 8c are integrated together, there is no need to provide a separate lighting device, and lighting and imaging can be performed by a single camera.

[0089] Second Embodiment An image processing device and a robot system including the image processing device according to a second embodiment will be described with reference to FIGS. 10 and 11. FIG.

[0090] A block diagram of the robot system according to this embodiment is shown in Fig. 10. In the robot system 4 according to this embodiment, imaging parameters are set based on the measured temperature TLi of the lighting device 7.

[0091] The illumination device 7 of this embodiment is configured so that the light intensity can be adjusted by the imaging control unit 56 changing the light intensity parameter. The illumination device 7 is provided with a temperature adjustment device 7b for adjusting the temperature of the illumination device 7. As with the temperature adjustment device 6b of the first embodiment, any device capable of adjusting the measured temperature of the illumination device 7 can be used as the temperature adjustment device 7b. For example, a cooling fan for cooling the illumination device 7, a heater for increasing the temperature of the illumination device 7, or the like can be used.

[0092] In this embodiment, the image processing unit 51 sets detection data based on the measured temperature of the lighting device 7 when the robot system 4 actually performs work. The lighting device 7 is provided with a temperature sensor 7c for detecting the temperature of the lighting device 7. The temperature sensor 7c can be arranged to detect the temperature of the housing of the lighting device 7 or the temperature of the air around the lighting device 7. The temperature acquisition unit 52 acquires the measured temperature of the lighting device 7 based on the output of the temperature sensor 7c of the lighting device 7.

[0093] 6, when robot system 4 of the present embodiment performs a task, camera 6 is placed at a predetermined position and posture in step 81. In step 82, temperature acquisition unit 52 acquires the temperature of lighting device 7 based on the output of temperature sensor 7c.

[0094] In step 83, the detection data setting unit 53 sets the current detection data. A plurality of detection reference data corresponding to a plurality of measured temperatures of the lighting device 7 are created in advance and stored in the storage unit 42. The detection data setting unit 53 sets the detection data based on the measured temperatures of the lighting device 7 and the detection reference data.

[0095] In particular, the detection data setting unit 53 sets the imaging parameters based on the measured temperature of the lighting device 7 and the imaging reference parameters. The imaging reference parameters include light intensity reference parameters that are references for light intensity parameters that control the light intensity of the lighting device 7. The detection data setting unit 53 sets the light intensity parameters based on the measured temperature of the lighting device 7 and the light intensity reference parameters.

[0096] Next, in step 84, the imaging control unit 56 images the workpiece based on the imaging parameters set by the detection data setting unit 53. In particular, the imaging control unit 56 drives the irradiation device using the light intensity parameters set by the detection data setting unit 53. The subsequent control from step 85 to step 88 is the same as the control in the first embodiment.

[0097] Here, when the temperature of the lighting device 7 changes, the light intensity of the lighting device 7 may change even if the light intensity parameter that controls the light intensity of the lighting device 7 is constant. In particular, the light intensity of a lighting device that includes an LED as a light source changes depending on the temperature of the lighting device. If the lighting device 7 is too bright, the outline of the feature may appear white and may not be detected. On the other hand, if the lighting device 7 is too dark, the area around the outline of the feature may appear black and the outline may not be detected.

[0098] In this embodiment, the illumination device can be controlled with an appropriate light intensity parameter corresponding to the temperature of the illumination device to capture an image of the workpiece. The illumination device can be set to an appropriate light intensity. As a result, the workpiece can be captured with appropriate brightness. The contour of the workpiece can be captured clearly, improving the accuracy of workpiece detection.

[0099] The detection parameter setting unit can set any of the imaging parameters according to the temperature of the lighting device. The imaging reference parameters include reference parameters such as the exposure time, aperture, and gain of the camera for each of multiple measurement temperatures of the lighting device. For example, the detection parameter setting unit may change at least one of the camera exposure time, aperture, and gain for converting light-related variables into electrical signals based on the temperature of the lighting device.

[0100] Other controls for driving the robot system and for detecting the position of the workpiece by image processing are the same as those in the first embodiment.

[0101] Next, control for generating a light intensity reference parameter among the imaging reference parameters will be described. Referring to Fig. 10 , the image processing unit 51 of the robot system 4 includes a reference data adjustment unit 61. The reference data adjustment unit 61 includes a temperature adjustment unit 58 that controls the temperature adjustment device 7b of the lighting device 7. The reference data adjustment unit 61 includes a reference data generation unit 60 that generates the imaging reference parameters. The reference data generation unit 60 of this embodiment generates the light intensity reference parameters for the lighting device 7. Furthermore, the reference data generation unit 60 sets the light intensity parameters when controlling the generation of the imaging reference parameters.

[0102] Each of the reference data adjustment unit 61, the temperature adjustment unit 58, and the reference data generation unit 60 corresponds to a processor that operates according to an operation program 66. The processor reads the operation program and performs the control defined in the operation program, thereby functioning as each unit.

[0103] FIG. 11 shows a flowchart of the control for generating the light intensity reference parameters in this embodiment. In step 121, the operator sets the imaging reference parameters and processing reference data included in the detection reference data. For example, the first generation control for generating the detection reference data in the first embodiment can be implemented (see FIG. 7). Here, detection reference data corresponding to a temperature measured by one camera is set. One measured temperature can be set in advance for the camera's measured temperature. For example, the average temperature measured by the camera during the period when the camera is capturing images can be used.

[0104] Alternatively, the temperature of the camera may be maintained constant using a temperature control device for the camera so that the temperature measured by the camera is maintained at a predetermined temperature. In this case, the detection reference data corresponding to the temperature measured by the camera can be used.

[0105] Next, in step 122, the operator sets a measurement temperature TLi of the lighting device 7. For example, a plurality of measurement temperatures TL1 to TLn are set at intervals of 5° C., between 30° C. and 60° C. In step 123, the operation control unit 43 places the camera 6 at a predetermined position.

[0106] In steps 124 to 129, the reference data adjustment unit 61 can automatically generate, among the imaging reference parameters, a light intensity reference parameter Hi for the lighting device 7. In step 124, 1 is set to the variable i.

[0107] In step 125, the temperature adjustment unit 58 adjusts the lighting device 7 to the measurement temperature TLi. In step 126, the reference data generation unit 60 sets one light intensity parameter. For example, the reference data generation unit 60 sets a current value to be supplied to the lighting device 7 as the light intensity parameter for controlling the light intensity. The imaging control unit 56 controls the lighting device 7 using the one light intensity parameter, and then images the workpiece 38 with the camera 6.

[0108] Next, the reference data generating unit 60 detects the contour of the characteristic portion of the workpiece 38 using the template image of the processing reference data generated in step 121. The reference data generating unit 60 acquires a contrast value of the contour of the workpiece 38. As the contrast value, for example, the difference in pixel brightness between bright and dark portions or the ratio of pixel brightness can be used. Then, one light quantity parameter and the contrast value are stored in the storage unit.

[0109] The imaging control unit 56 and the reference data generation unit 60 repeatedly change the light intensity parameters of the lighting device 7 and capture images using the camera. The plurality of light intensity parameters of the lighting device 7 can be determined in advance. The reference data generation unit 60 can set the light intensity parameter with the largest contrast value among the plurality of light intensity parameters as the light intensity reference parameter Hi.

[0110] For example, when the contrast value is within a predetermined range, the reference data generating unit 60 can set the light intensity parameter at that time as the light intensity reference parameter Hi. Alternatively, the reference data generating unit 60 can adopt the light intensity parameter with the largest contrast value among the plurality of light intensity parameters as the light intensity reference parameter Hi.

[0111] In step 127, the measured temperature TLi of the illumination device 7 and the imaging reference parameters Ii including the light intensity reference parameter Hi of the illumination device 7 are stored in the storage unit .

[0112] In step 128, it is determined whether the variable i has reached the upper limit value n. If the variable i has reached the upper limit value n, this control is terminated. If the variable i has not reached the upper limit value n, control proceeds to step 129. In step 129, 1 is added to the variable i, and control returns to step 125. In steps 125 to 127, the light intensity reference parameter Hi at the measured temperature TLi of each lighting device 7 is set.

[0113] In this manner, in this embodiment, after the operator first sets one piece of detection data, the light intensity reference parameter among the imaging reference parameters can be automatically generated. The light intensity reference parameter Hi when the lighting device 7 is at the measurement temperature TLi can be automatically generated. Note that the operator may set the light intensity reference parameter by changing the light intensity parameter of the lighting device and evaluating the captured image.

[0114] In this embodiment, the light intensity reference parameter is generated as one of the imaging reference parameters, but the present invention is not limited to this. For example, it is possible to maintain the light intensity parameter of the lighting device constant while changing any imaging parameters such as the exposure time, aperture, and gain of the camera. In other words, it is also possible to change at least one of the imaging parameters to detect the contrast value and generate the imaging reference parameter so as to obtain an appropriate contrast value.

[0115] The other configurations, operations, and effects are the same as those of the first embodiment, and therefore will not be described repeatedly here.

[0116] In the above embodiment, the robot changes the position and orientation of the camera, but this is not limiting. The robot may also change the position and orientation of the workpiece. In this case, the position and orientation of the camera may be fixed. Furthermore, the moving device that changes the position and orientation of the camera is not limited to a robot, and any device that changes the position and orientation of the camera may be used. For example, the robot system may include a conveyor that transports the workpiece, a camera fixed to a base, and a robot that removes the workpiece from the conveyor.

[0117] The camera in the above embodiment is a two-dimensional camera that captures two-dimensional images, but is not limited to this. Any camera, such as a three-dimensional camera, can be applied to the above embodiment. For example, a stereo camera equipped with multiple two-dimensional cameras and a distance camera that measures the time of flight of light can be used.

[0118] According to at least one of the embodiments described above, it is possible to provide an image processing device and a robot system that can detect the position of a workpiece with high accuracy.

[0119] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0120] The following supplementary notes are disclosed regarding the above-described embodiment and modifications.

[0121] (Supplementary Note 1) An image processing device comprising: a temperature acquisition unit that acquires the measured temperature of the camera; a detection data setting unit that captures an image with the camera and sets detection data for detecting a workpiece; a memory unit that stores detection reference data that is reference data for setting the detection data; an imaging control unit that controls the camera that captures the image; and a feature detection unit that detects characteristic parts of the workpiece by image processing, wherein the detection data includes imaging parameters for capturing the image and processing data for performing the image processing, the detection data setting unit sets the imaging parameters based on the measured temperature of the camera and the imaging reference parameters, the imaging control unit controls the camera to capture an image of the workpiece based on the imaging parameters, and the feature detection unit detects the characteristic parts of the workpiece based on the image of the workpiece and the processing data, and detects the position of the workpiece based on the characteristic parts.

[0122] (Supplementary Note 2) The image processing device according to Supplementary Note 1, wherein the imaging parameters include at least one parameter selected from the group consisting of an exposure time of a camera, an aperture of a camera, a gain for converting a light-related variable into an electrical signal, and a light intensity parameter for controlling the light intensity of a lighting device.

[0123] (Supplementary Note 3) The image processing device according to Supplementary Note 1 or 2, wherein the detection data setting unit sets the processing data based on the temperature measured by the camera and the processing reference data.

[0124] (Supplementary Note 4) The image processing device according to any one of Supplementary Notes 1 to 3, wherein the processing data includes at least one of a template image of a characteristic part of a workpiece for performing pattern matching and a specific position of the characteristic part in the template image.

[0125] (Appendix 5) An image processing device as described in Appendix 3, wherein the memory unit stores detection standard data corresponding to a plurality of measurement temperatures of the camera, and the detection data setting unit sets imaging parameters based on a plurality of imaging standard parameters corresponding to the plurality of measurement temperatures, and sets processing data based on a plurality of processing standard data corresponding to the plurality of measurement temperatures.

[0126] (Supplementary Note 6) An image processing device according to Supplementary Note 3, comprising a reference data adjustment unit that adjusts detection reference data, wherein the reference data adjustment unit includes a temperature adjustment unit that adjusts the temperature of the camera and a reference data generation unit that generates the detection reference data, wherein the temperature adjustment unit adjusts the temperature of the camera to a predetermined measurement temperature, wherein the reference data generation unit sets a template image of the characteristic part and a specific position of the characteristic part in the template image based on the outline of the characteristic part of the workpiece in the image, and wherein the memory unit stores the measured temperature of the camera and processing reference data including the template image of the characteristic part and the specific position of the characteristic part.

[0127] (Supplementary Note 7) An image processing device as described in Supplementary Note 6, wherein the reference data adjustment unit includes a similarity determination unit that determines the similarity between the template image and an image of a characteristic part captured by the camera, and when the similarity deviates from a predetermined determination range, the reference data adjustment unit generates processing reference data corresponding to the temperature measured by the camera.

[0128] (Supplementary Note 8) A robot system comprising: the image processing device according to Supplementary Note 1; a camera that captures an image of a workpiece; and a robot that moves the camera or the workpiece.

[0129] (Supplementary Note 9) An image processing device comprising: a temperature acquisition unit that acquires the measured temperature of the lighting device; a detection data setting unit that captures an image with a camera and sets detection data for detecting a workpiece; a memory unit that stores detection reference data that is reference data for setting the detection data; an imaging control unit that controls the camera that captures the image; and a feature detection unit that detects characteristic parts of the workpiece by image processing, wherein the detection data includes imaging parameters for capturing the image and processing data for performing the image processing, the detection data setting unit sets the imaging parameters based on the measured temperature of the lighting device and the imaging reference parameters, the imaging control unit controls the camera to capture an image of the workpiece based on the imaging parameters, and the feature detection unit detects the characteristic parts of the workpiece based on the image of the workpiece and the processing data, and calculates the position of the workpiece based on the characteristic parts.

[0130] (Supplementary Note 10) An image processing device according to Supplementary Note 9, comprising a reference data adjustment unit that adjusts detection reference data, wherein the reference data adjustment unit includes a temperature adjustment unit that adjusts the temperature of the lighting device and a reference data generation unit that generates detection reference data, wherein the temperature adjustment unit adjusts the temperature of the lighting device to a predetermined measurement temperature, wherein the imaging control unit changes the light intensity of the lighting device by changing a light intensity parameter, wherein the reference data generation unit sets the light intensity parameter to a light intensity parameter that brings the contrast of the outline of a characteristic part of the workpiece in the image within a predetermined judgment range, and wherein the memory unit stores the measured temperature of the lighting device and the imaging reference parameters including the light intensity reference parameter.

[0131] REFERENCE SIGNS LIST 1 Robot 2 Control device 3, 4 Robot system 6 Camera 6b Temperature adjustment device 6c Temperature sensor 7 Lighting device 7b Temperature adjustment device 7c Temperature sensor 8 Camera 8a Camera body 8c Lighting device 26 Teaching operation panel 27 Input unit 28 Display unit 38 Workpiece 38c Edge portion 40 Control device body 42 Memory unit 43 Operation control unit 51 Image processing unit 52 Temperature acquisition unit 53 Detection data setting unit 54 Feature detection unit 55 Operation command generation unit 56 Imaging control unit 57, 61 Reference data adjustment unit 58 Temperature adjustment unit 59 Similarity determination unit 60 Reference data generation unit 67 Detection reference data 67a Imaging reference parameters 67b Processing reference data 68, 69 Image

Claims

1. A temperature acquisition unit that acquires the measured temperature of a camera, a detection data setting unit that sets detection data for detecting a workpiece by imaging an image with the camera, a storage unit that stores detection reference data that is reference data for setting the detection data, an imaging control unit that controls the camera that images the image, and a feature detection unit that detects a feature portion of the workpiece by image processing. The detection data includes imaging parameters for imaging the image and processing data for performing image processing. The detection data setting unit sets the imaging parameters based on the measured temperature of the camera and imaging reference parameters. The imaging control unit controls the camera to image the workpiece based on the imaging parameters. The feature detection unit detects a feature portion of the workpiece based on the image of the workpiece and the processing data, and detects the position of the workpiece based on the feature portion. An image processing apparatus.

2. The imaging parameters according to claim 1, including at least one parameter among the exposure time of the camera, the aperture of the camera, the gain for converting a variable related to light into an electrical signal, and the light quantity parameter for controlling the light quantity of the illumination device.

3. The image processing apparatus according to claim 1 or 2, wherein the detection data setting unit sets the processing data based on the measured temperature of the camera and processing reference data.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the processing data includes at least one of a template image of a feature portion of the workpiece for performing pattern matching and a specific position of the feature portion in the template image.

5. The storage unit stores detection reference data corresponding to a plurality of measured temperatures of the camera. The detection data setting unit sets the imaging parameters based on a plurality of imaging reference parameters corresponding to the plurality of measured temperatures, and sets the processing data based on a plurality of processing reference data corresponding to the plurality of measured temperatures. The image processing apparatus according to claim 3.

6. The apparatus further comprises a reference data adjustment unit for adjusting detection reference data, the reference data adjustment unit including a temperature adjustment unit for adjusting the temperature of the camera and a reference data generation unit for generating detection reference data, the temperature adjustment unit adjusting the temperature of the camera to a predetermined measurement temperature, the reference data generation unit setting a template image of the characteristic part and a specific position of the characteristic part in the template image based on the contour of the characteristic part of the workpiece in the image, and the storage unit storing the measurement temperature of the camera and the processing reference data including the template image of the characteristic part and the specific position of the characteristic part. The image processing apparatus according to claim 3.

7. The reference data adjustment unit includes a similarity determination unit for determining the similarity between the template image and the image of the characteristic part captured by the camera. When the similarity deviates from a predetermined determination range, the reference data adjustment unit generates processing reference data corresponding to the measurement temperature of the camera. The image processing apparatus according to claim 6.

8. A robot system comprising the image processing apparatus according to claim 1, a camera for imaging a workpiece, and a robot for moving the camera or the workpiece.

9. A temperature acquisition unit for acquiring the measurement temperature of the lighting device, a detection data setting unit for setting detection data for detecting a workpiece by imaging an image with a camera, a storage unit for storing detection reference data which is reference data for setting the detection data, an imaging control unit for controlling the camera for imaging an image, and a feature detection unit for detecting a characteristic part of the workpiece by image processing. The detection data includes imaging parameters for imaging an image and processing data for performing image processing. The detection data setting unit sets the imaging parameters based on the measurement temperature of the lighting device and the imaging reference parameters. The imaging control unit controls the camera to image the workpiece based on the imaging parameters. The feature detection unit detects a characteristic part of the workpiece based on the image of the workpiece and the processing data, and calculates the position of the workpiece based on the characteristic part. An image processing apparatus.

10. The image processing apparatus according to claim 9, further comprising a reference data adjustment unit that adjusts detection reference data, wherein the reference data adjustment unit includes a temperature adjustment unit that adjusts the temperature of the lighting device and a reference data generation unit that generates detection reference data, the temperature adjustment unit adjusts the temperature of the lighting device to a predetermined measurement temperature, the imaging control unit changes the light amount of the lighting device by changing a light amount parameter, the reference data generation unit sets, as a light amount reference parameter, a light amount parameter within a predetermined determination range in which the contrast of the contour of a characteristic portion of the workpiece in the image is located, and the storage unit stores the measurement temperature of the lighting device and imaging reference parameters including the light amount reference parameter.

Images