Image processing device and robot control device

Abstract

Provided are an image processing device and a robot control device capable of determining an appropriate range of exposure times and number of shots for photographing a subject. The image processing device, which processes a captured image obtained by photographing the subject, comprises: a first exposure time determination unit that determines a minimum exposure time for photographing the subject; a second exposure time determination unit that determines a maximum exposure time for photographing the subject; a photographing condition determination unit that determines the exposure time and number of shots for photographing the subject based on an exposure time range that includes the determined minimum and maximum exposure times; and a composite image generation unit that generates a composite image by synthesizing a plurality of captured images obtained by photographing the subject using the determined exposure times and number of shots.

Description

Technical Field

[0001] The present invention relates to an image processing device and a robot control device. Background Art

[0002] Traditionally, it has been necessary to accurately identify the position of a workpiece and the deviation of the workpiece held by a robot in order to accurately handle or process the workpiece using the robot. Therefore, in recent years, vision sensors have been used to visually identify the position of the workpiece and its deviation (see, for example, Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2013-246149 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] When capturing an image of a subject (such as a workpiece) using a visual sensor, it's sometimes impossible to properly represent the brightness range within a single image. For example, when brightness is adjusted to match bright areas within the field of view, dark areas become completely black and cannot be seen. Conversely, when brightness is adjusted to match dark areas within the field of view, bright areas become completely white and cannot be seen.

[0008] To address this problem, a technique called HDR (High Dynamic Range) synthesis is known, which synthesizes multiple captured images to generate an image with a wide dynamic range that cannot be obtained with a single image.

[0009] Since capturing multiple images takes time, it is desirable to capture images as few times as possible. Therefore, a technique for determining an appropriate range of exposure time and number of capture times for capturing an object is desired.

[0010] Solutions for solving problems

[0011] The image processing device involved in the present disclosure processes a captured image obtained by capturing a subject, and the image processing device includes: a first exposure time determination unit, which determines a minimum exposure time for capturing the subject; a second exposure time determination unit, which determines a maximum exposure time for capturing the subject; a shooting condition determination unit, which determines the exposure time for capturing the subject and the number of times the subject is captured based on an exposure time range including the determined minimum exposure time and the determined maximum exposure time; and a composite image generation unit, which generates a composite image by synthesizing a plurality of captured images obtained by capturing the subject using the determined exposure time and the determined number of times.

[0012] The robot control device involved in the present disclosure has an image processing device that processes a captured image obtained by capturing a subject, and the robot control device includes: a first exposure time determination unit, which determines a minimum exposure time for capturing the subject; a second exposure time determination unit, which determines a maximum exposure time for capturing the subject; a shooting condition determination unit, which determines the exposure time for capturing the subject and the number of times the subject is captured based on an exposure time range including the determined minimum exposure time and the determined maximum exposure time; and a synthetic image generation unit, which synthesizes a plurality of captured images obtained by capturing the subject using the determined exposure time and the determined number of times to generate a synthetic image.

[0013] The image processing device involved in the present disclosure processes a captured image obtained by capturing a subject, and the image processing device includes: a first exposure time determination unit, which determines the minimum value of the optical parameter used to capture the subject; a second exposure time determination unit, which determines the maximum value of the optical parameter used to capture the subject; a shooting condition determination unit, which determines the optical parameter used to capture the subject and the number of times to capture the subject based on an optical parameter range including the determined minimum value of the optical parameter and the determined maximum value of the optical parameter; and a synthetic image generation unit, which synthesizes multiple captured images obtained by capturing the subject using the determined optical parameters and the determined number of times to capture, to generate a synthetic image.

[0014] Effects of the Invention

[0015] According to the present invention, it is possible to determine an appropriate exposure time range and number of imaging times for imaging a subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a diagram showing the structure of a robot system.

[0017] Figure 2 It is a diagram showing the structure of a robot control device.

[0018] Figure 3 This is a diagram showing the brightness that can be obtained in an HDR composite image.

[0019] Figure 4 This is a diagram showing a specific example of the ratio of making all white and the ratio of making all black in the luminance histogram.

[0020] Figure 5 This is a flowchart showing the flow of processing by the image processing apparatus.

[0021] Figure 6 This is a diagram schematically showing an example of an image processing system in which a plurality of visual sensors are connected, according to one embodiment of the present invention.

[0022] Figure 7 This is a diagram schematically showing an example of an image processing system in which a plurality of image processing apparatuses are connected, according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] An example of an embodiment of the present invention will be described below.

[0024] Figure 1 is a diagram showing the structure of the robot system 100. Figure 1 As shown, the robot system 100 includes a robot controller 1 , a robot 2 , an arm 3 , and a vision sensor 4 .

[0025] A hand or tool is attached to the front end of the robot 2's arm 3. The robot 2 performs operations such as gripping and processing a workpiece W under the control of the robot controller 1. Furthermore, a vision sensor 4 is attached to the front end of the robot 2's arm 3. The vision sensor 4 does not need to be attached to the robot 2; for example, it can be fixed at a predetermined location.

[0026] The vision sensor 4 captures an image of the workpiece W under the control of the robot controller 1. A two-dimensional camera having an imaging element composed of a CCD (Charge Coupled Device) image sensor and an optical system including a lens may be used as the vision sensor 4. Furthermore, it is desirable to use an imaging element that can increase the speed of imaging by specifying a binning level for the captured images.

[0027] The robot control device 1 executes a robot program for the robot 2 to control the operation of the robot 2. The robot control device 1 uses images captured by the vision sensor 4 to correct the operation of the robot 2 so that the robot 2 performs a predetermined operation on the workpiece W.

[0028] Figure 2 This figure shows the structure of a robot control device 1. The robot control device 1 includes an image processing device 10. While the robot control device 1 has a general structure for controlling the robot 2, this is omitted for simplicity of explanation. The image processing device 10 processes images captured by the vision sensor 4. The image processing device 10 includes a control unit 11 and a storage unit 12.

[0029] The control unit 11 is a processor such as a CPU (Central Processing Unit), and realizes various functions by executing programs stored in the storage unit 12 .

[0030] The control unit 11 includes a first exposure time determination unit 111 , a second exposure time determination unit 112 , a third exposure time determination unit 113 , a shooting condition determination unit 114 , and a composite image generation unit 115 .

[0031] The storage unit 12 is a storage device such as a ROM (Read Only Memory) or RAM (Random Access Memory) that stores the OS (Operating System) and application programs, as well as a hard disk drive or SSD (Solid State Drive) that stores various other information. The storage unit 12 stores various information, such as robot programs.

[0032] The first exposure time determination unit 111 determines the exposure time for photographing the subject (for example, Figure 1 The minimum exposure time of the workpiece W) shown.

[0033] Specifically, the first exposure time determination unit 111 calculates the brightness of an image obtained by capturing a subject at the minimum exposure time. If the value based on the calculated brightness is less than a first threshold value H1, the minimum exposure time is changed. The first exposure time determination unit 111 then repeats capturing the subject, calculating the brightness, and changing the minimum exposure time until the value based on the brightness exceeds the first threshold value H1, thereby determining the minimum exposure time.

[0034] More specifically, the first exposure time determination unit 111 presets a minimum exposure time for capturing an object, and calculates a first histogram of brightness of an image captured by capturing the object at the minimum exposure time.

[0035] Next, if the maximum brightness value in the first histogram is less than the first threshold value H1, the first exposure time determination unit 111 adjusts the minimum exposure time so that the maximum brightness value approaches the first threshold value H1. For example, the first exposure time determination unit 111 adjusts the minimum exposure time value by multiplying the minimum exposure time value by a predetermined value. The first threshold value H1 is a value indicating that the maximum brightness value in the first histogram is sufficiently large.

[0036] Then, the first exposure time determination unit 111 repeats photographing the subject, calculating the first histogram, and changing the minimum exposure time until the maximum brightness value in the first histogram becomes equal to or greater than the first threshold value H1, thereby determining the minimum exposure time.

[0037] The second exposure time determination unit 112 determines the exposure time for photographing the subject (for example, Figure 1 The maximum value of the exposure time of the workpiece W) shown.

[0038] Specifically, the second exposure time determination unit 112 calculates the brightness of the captured image obtained by capturing the subject at the maximum exposure time, and changes the maximum exposure time when the value based on the calculated brightness is greater than the second threshold value H2. The second exposure time determination unit 112 repeats the capturing of the subject, the calculation of the brightness, and the change of the maximum exposure time until the value based on the brightness becomes below the second threshold value, thereby determining the maximum exposure time.

[0039] More specifically, the second exposure time determination unit 112 presets a maximum exposure time for capturing an object, and calculates a second histogram of brightness of an image captured by capturing the object at the maximum exposure time.

[0040] Next, if the minimum brightness value in the second histogram is greater than the second threshold value H2, the second exposure time determination unit 112 adjusts the maximum exposure time so that the minimum brightness value approaches the second threshold value H2. For example, the second exposure time determination unit 112 adjusts the maximum exposure time by multiplying the maximum exposure time by a predetermined value. The second threshold value H2 indicates that the minimum brightness value in the second histogram is sufficiently small.

[0041] Then, the second exposure time determination unit 112 repeats photographing the subject, calculating the second histogram, and changing the maximum exposure time until the minimum brightness value in the second histogram becomes equal to or smaller than the second threshold value H2, thereby determining the maximum exposure time.

[0042] When the first exposure time determination unit 111 sets the minimum exposure time, it uses the minimum exposure time of the previously captured images. When the second exposure time determination unit sets the maximum exposure time, it uses the maximum exposure time of the previously captured images.

[0043] Specifically, the first exposure time determination unit 111 pre-stores the minimum exposure time value of the image captured during the previous shooting. Then, when pre-setting the minimum exposure time value, the first exposure time determination unit 111 uses the minimum exposure time value of the image captured during the previous shooting.

[0044] Similarly, the second exposure time determination unit 112 pre-stores the maximum exposure time of the image captured during the previous shooting. When setting the maximum exposure time, the second exposure time determination unit 112 uses the maximum exposure time of the image captured during the previous shooting. This allows the image processing device 10 to accelerate the measurement of the exposure time range.

[0045] Furthermore, when the first exposure time determination unit 111 pre-sets the minimum exposure time, it may use the minimum exposure time for the captured image specified externally (e.g., a teaching operation panel operated by an operator). Furthermore, when the second exposure time determination unit 112 pre-sets the maximum exposure time, it may use the maximum exposure time for the captured image specified externally (e.g., a teaching operation panel operated by an operator).

[0046] The third exposure time determination unit 113 calculates a reference histogram of brightness of a captured image obtained by capturing the subject at a reference exposure time between the minimum exposure time and the maximum exposure time, and stores the calculated reference histogram in the storage unit 12 .

[0047] Then, the third exposure time determination unit 113 calculates a third histogram of the brightness of the captured image obtained by capturing the subject at the reference exposure time, and compares the third histogram with the reference histogram. Figure 1 The exposure time coefficient is calculated in a consistent manner.

[0048] The imaging condition determination unit 114 determines the exposure time for imaging the subject and the number of imaging times for imaging the subject based on the exposure time range including the determined minimum exposure time and maximum exposure time.

[0049] For example, the shooting condition determination unit 114 divides the exposure time range including the determined minimum exposure time and the maximum exposure time into appropriate prescribed intervals, sets the divided part as the exposure time, and sets the number of divisions as the number of shots, thereby determining the exposure time and the number of shots.

[0050] For example, when the imaging condition determination unit 114 divides the exposure time range into five, that is, determines the number of imaging times to be five, the predetermined interval includes interval A1, interval A2, interval A3, and interval A4.

[0051] Furthermore, the imaging condition determination unit 114 divides the exposure time range so that the length of interval A2 is twice the length of interval A1, the length of interval A3 is four times the length of interval A1, and the length of interval A4 is eight times the length of interval A1. In other words, the lengths of intervals A1, A2, A3, and A4 are proportional.

[0052] Furthermore, the imaging condition determination unit 114 calculates the exposure time range based on the minimum exposure time value, the maximum exposure time value, and the exposure time coefficient.

[0053] Specifically, the imaging condition determination unit 114 multiplies the minimum and maximum exposure times by the exposure time coefficient calculated by the third exposure time determination unit 113 to determine the minimum and maximum exposure times that take into account the reference exposure time. The imaging condition determination unit 114 then calculates an exposure time range that includes the determined minimum and maximum exposure times. This allows the image processing device 10 to calculate an exposure time range that takes into account the reference histogram and the reference exposure time.

[0054] Furthermore, the captured image used to determine the exposure time is a reduced image. Thus, the image processing device 10 can speed up the process of determining the exposure time by using the reduced image, compared to the case of using a captured image of normal size.

[0055] The composite image generation unit 115 generates a composite image by combining a plurality of captured images of the subject using the determined exposure time and number of images. In this way, the image processing device 10 generates a composite image by performing HDR (High Dynamic Range) synthesis.

[0056] Furthermore, instead of using the maximum and minimum luminance values ​​described above, the first exposure time determination unit 111 may determine the minimum value based on, for example, the first percentile value from the bright side. Furthermore, instead of using the histogram described above, the second exposure time determination unit 112 may determine the maximum value based on, for example, the first percentile value from the dark side.

[0057] Figure 3 is a diagram showing the brightness that can be obtained in an HDR composite image. Figure 3 As shown in FIG. 1 , the range of brightness that can be obtained in the HDR composite image becomes wider than the range of brightness that can be obtained in a single captured image. Therefore, the image processing device 10 can obtain a captured image with high resolution.

[0058] In addition, the composite image generation unit 115 is able to specify at least one of a ratio of making a plurality of captured images completely white and a ratio of making a plurality of captured images completely black, and the composite image generation unit 115 performs tone mapping (tonemapping) of the composite image in a state where pixels of the ratio of making a plurality of captured images completely white are set to white and pixels of the ratio of making a plurality of images completely black are set to black.

[0059] Figure 4 : is a diagram showing a specific example of the ratio of making it completely white and the ratio of making it completely black in the brightness histogram. Figure 4 As shown, in the luminance histogram of the composite image, the composite image generating unit 115 sets the pixels in the 10% region with the smallest luminance to black, and sets the pixels in the 10% region with the largest luminance to white.

[0060] The composite image generator 115 can also store images before generating the composite image. For example, the composite image generator 115 can store all of the multiple captured images, or store images before tone mapping. This allows the image processing device 10 to use the stored images to adjust parameters related to image synthesis if object detection or inspection based on the composite image fails. Furthermore, the image processing device 10 can automatically try other parameter adjustment methods to avoid system downtime.

[0061] Figure 5 1 is a flowchart showing the flow of processing by the image processing apparatus 10 .

[0062] In step S1 , the first exposure time determination unit 111 presets a minimum exposure time for capturing an object, and the second exposure time determination unit 112 presets a maximum exposure time for capturing an object.

[0063] In step S2 , the visual sensor 4 captures an image of the subject using a preset minimum exposure time.

[0064] In step S3 , the first exposure time determination unit 111 calculates a first histogram of brightness of a captured image obtained by capturing the subject at the minimum exposure time.

[0065] In step S4, the first exposure time determination unit 111 determines whether the maximum brightness value Lmax in the first histogram calculated in step S3 is greater than or equal to the first threshold value H1. If Lmax is greater than or equal to the first threshold value H1 ("Yes"), the process proceeds to step S6. If Lmax is less than the first threshold value H1 ("No"), the process proceeds to step S5.

[0066] In step S5 , the first exposure time determination unit 111 changes the minimum value of the exposure time so that the maximum brightness value approaches the first threshold value H1 .

[0067] In step S6 , the first exposure time determination unit 111 determines the minimum value of the exposure time by repeating the processes from step S2 to step S5 .

[0068] In step S7 , the visual sensor 4 captures an image of the subject with a preset maximum exposure time.

[0069] In step S8 , the second exposure time determination unit 112 calculates a second histogram of brightness of a captured image obtained by capturing the subject at the maximum exposure time.

[0070] In step S9, the second exposure time determination unit 112 determines whether the minimum brightness value Lmin in the second histogram calculated in step S8 is less than or equal to the second threshold value H2. If Lmin is less than or equal to the second threshold value H2 ("Yes"), the process proceeds to step S11. If Lmin exceeds the second threshold value H2 ("No"), the process proceeds to step S10.

[0071] In step S10 , the second exposure time determination unit 112 changes the maximum value of the exposure time so that the minimum brightness value approaches the second threshold value H2 .

[0072] In step S11 , the second exposure time determination unit 112 determines the maximum value of the exposure time by repeating the processes from step S7 to step S10 .

[0073] In step S12 , the imaging condition determination unit 114 determines the exposure time for imaging the subject and the number of imaging times based on the exposure time range including the minimum exposure time determined in step S6 and the maximum exposure time determined in step S11 .

[0074] In step S13 , the composite image generating unit 115 generates a composite image by combining a plurality of captured images obtained by capturing the subject using the exposure time and the number of capture times determined in step S12 .

[0075] As described above, according to this embodiment, the image processing device 10 includes: a first exposure time determination unit 111, which determines the minimum value of the exposure time for photographing a subject; a second exposure time determination unit 112, which determines the maximum value of the exposure time for photographing a subject; a shooting condition determination unit 114, which determines the exposure time for photographing a subject and the number of shots for photographing the subject based on an exposure time range including the determined minimum value of the exposure time and the determined maximum value of the exposure time; and a composite image generation unit, which synthesizes a plurality of captured images obtained by photographing the subject using the determined exposure time and number of shots to generate a composite image.

[0076] Thus, the image processing device 10 can determine an appropriate exposure time range and number of shots for capturing an image of a subject and obtain a composite image without including a photometric sensor or the like.

[0077] Furthermore, the first exposure time determination unit 111 calculates the brightness of an image obtained by capturing a subject at the minimum exposure time. If the value based on the calculated brightness is less than the first threshold value H1, the minimum exposure time value is changed. The first exposure time determination unit 111 then repeats capturing the subject, calculating the brightness, and changing the minimum exposure time value until the value based on the brightness exceeds the first threshold value H1, thereby determining the minimum exposure time value. This allows the image processing device 10 to appropriately determine the minimum exposure time value.

[0078] Furthermore, the second exposure time determination unit 112 calculates the brightness of an image captured by capturing the subject at the maximum exposure time. If the value based on the calculated brightness exceeds the second threshold value H2, the second exposure time determination unit 112 changes the maximum exposure time. The second exposure time determination unit 112 then repeats capturing the subject, calculating the brightness, and changing the maximum exposure time until the value based on the brightness falls below the second threshold value, thereby determining the maximum exposure time. This allows the image processing device 10 to appropriately determine the maximum exposure time.

[0079] Furthermore, the captured image used to determine the exposure time is a reduced image. Thus, the image processing device 10 can speed up the process of determining the exposure time by using the reduced image, compared to the case of using a captured image of normal size.

[0080] Furthermore, when the first exposure time determination unit 111 pre-sets a minimum exposure time value, it uses the minimum exposure time value of previously captured images. When the second exposure time determination unit 112 pre-sets a maximum exposure time value, it uses the maximum exposure time value of previously captured images. This allows the image processing device 10 to accelerate the measurement of exposure time ranges.

[0081] Furthermore, when the first exposure time determination unit 111 pre-sets a minimum exposure time value, it may also use an externally specified minimum exposure time value for the captured image. When the second exposure time determination unit 112 pre-sets a maximum exposure time value, it may also use an externally specified maximum exposure time value for the captured image. Thus, the image processing device 10 can accelerate the measurement of exposure time ranges.

[0082] In addition, the third exposure time determination unit 113 calculates a reference histogram of the brightness of the captured image obtained by capturing the subject at a reference exposure time between the minimum exposure time and the maximum exposure time, and stores the reference histogram in the storage unit 12. Next, the third exposure time determination unit 113 calculates a third histogram of the brightness of the captured image obtained by capturing the subject at the reference exposure time, and compares the third histogram with the reference histogram. Figure 1 The exposure time coefficient is calculated in a consistent manner.

[0083] Furthermore, the imaging condition determination unit 114 calculates the exposure time range based on the minimum exposure time, the maximum exposure time, and the exposure time coefficient. This allows the image processing device 10 to calculate the exposure time range in consideration of the reference histogram and the reference exposure time.

[0084] Furthermore, the composite image generation unit 115 can specify at least one of a ratio of completely white and a ratio of completely black in the plurality of captured images. The composite image generation unit 115 performs tone mapping on the composite image, setting pixels in the ratio of completely white to white and pixels in the ratio of completely black to black. This allows the image processing device 10 to appropriately obtain a composite image with high resolution.

[0085] Furthermore, the composite image generation unit 115 can generate composite images using different synthesis methods by recording information about the original image before generating the composite image. This allows the image processing device 10 to adjust parameters related to image synthesis using the stored image if object detection or inspection using the composite image fails.

[0086] Figure 6 Schematically shows an example of an image processing system 201 connected to a plurality of visual sensors 4 according to an embodiment of the present invention. Figure 6 In the embodiment, N visual sensors 4 are connected to a unit controller 200 via a network bus 210. The unit controller 200 has the same function as the above-mentioned image processing device 10, and acquires captured images acquired from the N visual sensors 4, respectively.

[0087] In such Figure 6 In the image processing system 201 shown, the unit controller 200 may also include a machine learning engine (not shown). The machine learning engine acquires a set of learning data stored in the unit controller 200 and performs supervised learning. In this example, the learning process can also be processed sequentially online.

[0088] Figure 7 FIG. 1 is a diagram schematically showing an example of an image processing system 301 in which a plurality of image processing apparatuses 10 are connected according to an embodiment of the present invention. Figure 7 In the image processing system 301 , m image processing devices 10 are connected to the unit controller 200 via the network bus 210. One or more visual sensors 4 are connected to each of the image processing devices 10. The image processing system 301 as a whole includes a total of n visual sensors 4.

[0089] In such Figure 7 In the image processing system 301 shown, the unit controller 200 may also include, for example, a machine learning engine (not shown). The unit controller 200 may also store a collection of learning data sent from multiple image processing devices 10 as a learning dataset and perform machine learning to construct a learning model. The learning model can be utilized in each image processing device 10.

[0090] Furthermore, in the above-described embodiment, the image processing device 10 performs control related to exposure time. However, it is also possible to perform control related to optical parameters other than exposure time. For example, instead of performing control related to exposure time, the image processing device 10 may perform control related to optical parameters such as the gain of the imaging element or the aperture of the lens.

[0091] In this case, the image processing device 10 includes: a first exposure time determination unit 111 that determines the minimum value of the optical parameter for capturing the subject; a second exposure time determination unit 112 that determines the maximum value of the optical parameter for capturing the subject; a capturing condition determination unit 114 that determines the optical parameter for capturing the subject and the number of capture times for capturing the subject based on an optical parameter range that includes the determined minimum and maximum values ​​of the optical parameter; and a composite image generation unit that generates a composite image by synthesizing multiple captured images obtained by capturing the subject using the determined optical parameters and the determined number of capture times. Thus, the image processing device 10 can determine an appropriate range of optical parameters and number of capture times for capturing the subject and obtain a composite image without including a light metering sensor or the like.

[0092] The embodiments of the present invention have been described above. The robot control device 1 described above can be implemented using hardware, software, or a combination thereof. Furthermore, the control method performed by the robot control device 1 described above can also be implemented using hardware, software, or a combination thereof. Here, implementation using software means that the control method is implemented by a computer reading and executing a program.

[0093] The program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media (tangible storage media). Examples of non-transitory computer-readable media include magnetic recording media (such as hard disk drives), optical magnetic recording media (such as optical magnetic disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, semiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)).

[0094] The above-described embodiments are preferred embodiments of the present invention. However, the scope of the present invention is not limited to the above-described embodiments, and the present invention can be implemented in various modified forms without departing from the spirit of the present invention.

[0095] Description of Reference Numerals

[0096] 1: Robot control device; 2: Robot; 3: Arm; 4: Visual sensor; 10: Image processing device; 11: Control unit; 12: Storage unit; 100: Robot system; 111: First exposure time determination unit; 112: Second exposure time determination unit; 113: Third exposure time determination unit; 114: Shooting condition determination unit; 115: Composite image generation unit.

Claims

1. A robotic system comprising: a robot, which performs operations on a workpiece; a robot control device that controls the robot; a visual sensor that captures the workpiece as a subject; and an image processing device for processing the image captured by the visual sensor, The robot control device uses the captured image to correct the movement of the robot so that the robot performs a predetermined operation on the position of the workpiece. The image processing device comprises: a first exposure time determination unit configured to determine a minimum exposure time for photographing the subject; a second exposure time determination unit configured to determine a maximum value of the exposure time for photographing the subject; a photographing condition determination unit that determines the exposure time for photographing the subject and a number of photographing times for photographing the subject based on an exposure time range including the determined minimum exposure time and maximum exposure time; and a composite image generating unit for generating a composite image by synthesizing a plurality of captured images obtained by capturing the subject using the determined exposure time and the determined number of times of capturing, The robot system further includes a third exposure time determination unit that calculates a reference histogram of brightness of the captured image obtained by capturing the subject at a reference exposure time between the minimum exposure time and the maximum exposure time, and stores the reference histogram in a storage unit. The third exposure time determination unit calculates a third histogram of brightness of the captured image obtained by capturing the subject at the reference exposure time, and calculates an exposure time coefficient so that the third histogram matches the reference histogram.

2. The robot system according to claim 1, wherein: The first exposure time determination unit pre-sets a minimum value of the exposure time for photographing the subject. The first exposure time determination unit calculates brightness of the captured image obtained by capturing the subject at the minimum exposure time, and changes the minimum exposure time when a value based on the calculated brightness is smaller than a first threshold value. The first exposure time determination unit determines the minimum exposure time by repeating the imaging of the subject, the calculation of the brightness, and the change of the minimum exposure time until the value based on the brightness becomes equal to or greater than the first threshold.

3. The robot system according to claim 1, wherein: The second exposure time determination unit pre-sets a maximum value of the exposure time for photographing the subject. The second exposure time determination unit calculates brightness of the captured image obtained by capturing the subject at the maximum exposure time, and changes the maximum exposure time when a value based on the calculated brightness is greater than a second threshold value. The second exposure time determination unit determines the maximum exposure time by repeating the imaging of the subject, the calculation of the brightness, and the change of the maximum exposure time until the value based on the brightness becomes equal to or less than the second threshold.

4. The robot system according to claim 1, wherein: The captured image used to determine the exposure time is a reduced image.

5. The robot system according to claim 1, wherein: When presetting the minimum value of the exposure time, the first exposure time determination unit uses the minimum value of the exposure time of the captured image captured in advance. When presetting the maximum value of the exposure time, the second exposure time determination unit uses the maximum value of the exposure time of the captured image captured in advance.

6. The robot system according to claim 1, wherein: When the minimum value of the exposure time is preset, the first exposure time determination unit uses the minimum value of the exposure time of the captured image specified externally. When the maximum value of the exposure time is set in advance, the second exposure time determination unit uses the maximum value of the exposure time of the captured image specified externally.

7. The robot system according to claim 1, wherein: The imaging condition determination unit calculates the exposure time range based on the minimum exposure time, the maximum exposure time, and the exposure time coefficient.

8. The robot system according to claim 1, wherein: The composite image generating unit is capable of specifying at least one of a ratio of making the plurality of captured images completely white and a ratio of making the plurality of captured images completely black. The synthetic image generation unit performs tone mapping on the synthetic image in a state where pixels having a ratio of being made entirely white are set to white and pixels having a ratio of being made entirely black are set to black.

9. The robot system according to claim 1, wherein: The composite image generating unit can generate composite images using different synthesis methods by recording information of an original image before generating the composite image.

Images