Image processing device

Abstract

The image processing parameters for detecting a workpiece are quantified to adjust the image processing parameters. An image processing device for automatically adjusting detection parameters used in image processing for detecting an imaged object comprises: a detection parameter generating unit that generates a plurality of detection parameter combinations; an imaging condition setting unit that sets a plurality of imaging conditions for each combination of detection parameters generated by the detection parameter generating unit; a detection feasibility determination unit that determines whether the imaged object can be detected for each combination of detection parameters and imaging conditions; an imaging range calculation unit that calculates the range of imaging conditions that the detection feasibility determination unit determines will result in detection of the imaged object; and a parameter determination unit that determines the combination of detection parameters that maximizes the range of imaging conditions calculated by the imaging range calculation unit.

Description

Technical Field

[0001] The present invention relates to an image processing device. Background Art

[0002] Conventionally, when using a visual sensor such as a camera to detect an imaging target in a production line equipped with robots, appropriate lighting is generally provided to create an environment in which an image can be captured in which the imaging target is easily visible.

[0003] However, ideal lighting conditions may not always be achieved due to factors such as the lighting performance and overhead lighting used. For example, uniform brightness may not be achieved across the entire imaging range, with the ends of the imaging range being darker than the center. When there is uneven lighting between the center and ends of the imaging range, artifacts may be found at the ends of the imaging range but not in the center, or vice versa.

[0004] In conventional methods, imaging objects are placed at the ends and the center of the imaging range, and the instructor of the image processing device manually adjusts parameters to enable the imaging objects to be found at both the ends and the center while visually checking the image processing results.

[0005] In this regard, the following technology is known: by manually inputting multiple sets of parameters consisting of a combination of photographic parameters (photographic conditions) for photographing a camera object to obtain an image and image processing parameters for detecting the camera object from the image, multiple reduced images representing the execution results of a processing sequence including photography and image processing set for the multiple sets of parameters are displayed at a glance on a display unit as multiple result images (for example, refer to patent document 1).

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2019-204185 Summary of the Invention

[0009] Problems to be solved by the invention

[0010] However, since the quality of image processing parameters for detecting an imaging object from an image cannot be numerically expressed, there is a problem that the adjustment of image processing parameters may vary among individuals depending on the skill level of the instructor.

[0011] Solutions for solving problems

[0012] One embodiment of the present disclosure is an image processing device that automatically adjusts detection parameters used in image processing for detecting an imaging object, the image processing device comprising: a detection parameter generating unit that generates a plurality of combinations of the detection parameters; an imaging condition setting unit that sets a plurality of imaging conditions for each combination of the detection parameters generated by the detection parameter generating unit; a detection feasibility determination unit that determines whether the imaging object can be detected for each combination of the detection parameters and the imaging conditions; an imaging range calculation unit that calculates the range of imaging conditions that the detection feasibility determination unit determines to be the range of imaging conditions for detecting the imaging object; and a parameter determination unit that determines the combination of detection parameters for which the range of the imaging conditions calculated by the imaging range calculation unit is the largest.

[0013] Effects of the Invention

[0014] According to one embodiment, by quantifying the quality of the detection parameters, the detection parameters can be adjusted regardless of the instructor's skill level. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a functional block diagram of an image processing device according to one embodiment.

[0016] Figure 2 This is a flowchart showing the operation of the image processing device according to one embodiment.

[0017] Figure 3 This is a table showing a method for determining optimal parameters of an image processing device according to one embodiment.

[0018] Figure 4 This is a table showing a method for determining optimal parameters of an image processing device according to one embodiment.

[0019] Figure 5 This is a table showing a method for determining optimal parameters of an image processing device according to one embodiment.

[0020] Figure 6 This is an example of an operation screen of an image processing device according to an embodiment. DETAILED DESCRIPTION

[0021] Below, by referring to Figures 1 to 6 Embodiments of the present invention will be described.

[0022] [1. Structure of Embodiment]

[0023] Figure 1 1 is a functional block diagram of the image processing apparatus 1 according to the present embodiment. The image processing apparatus 1 includes a control unit 10 , a storage unit 20 , and an operation unit 30 .

[0024] The control unit 10 includes a CPU, a ROM, a RAM, a CMOS memory, and the like. These components are configured to be able to communicate with each other via a bus, and are well known to those skilled in the art.

[0025] The CPU is a processor that controls the image processing apparatus 1 as a whole. The CPU reads out the system program and application program stored in the ROM via the bus, and controls the image processing apparatus 1 as a whole according to the system program and application program. Figure 1 As shown, the control unit 10 is configured to realize the functions of a detection parameter generating unit 11 , an imaging condition setting unit 12 , a detection possibility determining unit 13 , an imaging range calculating unit 14 , and a detection parameter determining unit 15 .

[0026] The detection parameter generating unit 11 generates a plurality of detection parameter combinations for each of a plurality of detection parameters.

[0027] The “detection parameters” vary depending on the image processing algorithm, and include, for example, a score threshold, a contrast threshold, a deformation tolerance value, and the like.

[0028] The "Score Threshold" is the threshold that determines whether a detection result is OK or NG. Specifically, the accuracy of the detection result is expressed as a score (in %), with 100 being the maximum score. If the score is above the "Score Threshold," the detection is OK; if it is below the "Score Threshold," the detection is NG.

[0029] The "Contrast Threshold" indicates the degree of contrast (difference in light and dark) in a captured image that is recognized as a feature. Setting a low contrast threshold value allows for the detection of objects that are not clearly visible, but this increases image processing time. To erroneously detect low-contrast objects such as simple dirt within the object, increase the contrast threshold.

[0030] The "Distortion Allowable Value" is the threshold (in pix) for the shape deviation (distortion) allowed between the trained model and the pattern matching the object captured in the image. Specifying a larger value for the Distortion Allowable Value allows detection even with large shape deviations, but also increases the likelihood of false detection.

[0031] The imaging condition setting unit 12 sets a plurality of imaging conditions for each combination of the detection parameter sets generated by the detection parameter generating unit 11 .

[0032] The “imaging conditions” vary depending on the imaging device, and include, for example, the exposure time of the camera of the imaging device, the light intensity of the illumination used for imaging, and the reduction ratio of the captured image.

[0033] The imaging condition setting unit 12 prepares a plurality of groups of combinations, such as score thresholds, contrast thresholds, and deformation allowable values, as combinations of detection parameters (detection parameter sets), and sets a plurality of imaging conditions for each set, such as the exposure time of the camera, the amount of light used in the imaging, the reduction ratio of the captured image, etc.

[0034] Furthermore, the prepared plurality of detection parameter sets may be all combinations of values ​​that can be set for each parameter. In this case, the value of each parameter is determined by, for example, equally dividing the range that can be set for the parameter.

[0035] Alternatively, all combinations of values ​​that can be set within a predetermined range centered around a value manually adjusted to a certain extent by the user of the image processing apparatus 1 may be used.

[0036] The detection possibility determination unit 13 determines whether or not the imaging target object can be detected for each combination of a detection parameter set and an imaging condition.

[0037] Specifically, first, the detection possibility determination unit 13 determines whether detection of the imaging target object is possible for imaging under the first imaging condition using the first detection parameter set.

[0038] Next, the detection possibility determination unit 13 determines whether detection of the imaging target object is possible for imaging under the second imaging condition using the first detection parameter set.

[0039] Next, similarly, the detection possibility determination unit 13 determines whether or not detection of the imaging target object is possible for imaging under all imaging conditions using the first detection parameter set.

[0040] After determining whether the imaging object can be detected for all imaging conditions using the first detection parameter set, the detection determination unit 13 determines whether the imaging object can be detected for all imaging conditions using the second detection parameter set.

[0041] Next, similarly, the detection possibility determination unit 13 determines whether or not detection of the imaging target object is possible for imaging under all imaging conditions based on all detection parameter sets.

[0042] Here, the detection feasibility determination unit 13 may determine whether detection is possible based on whether the detection result of the imaging target object matches a predetermined correct answer condition.

[0043] Alternatively, the detection feasibility determination unit 13 may determine whether detection is possible based on whether the detection position where the known imaging target object is actually detected matches the known installation position of the imaging target object.

[0044] When determining whether detection is possible based on whether the detection conditions are met, a step for determining the correct answer conditions can be performed before the step of determining whether detection is possible (step S5 in the embodiment described below). Examples of correct answer conditions include "detection accuracy," "detection position," "detection quantity," and "detection time."

[0045] "Detection accuracy" is the deviation in the detection position when the same imaging object is repeatedly detected without changing parameters. When the image reduction ratio is increased, the "detection time" is shortened, but the detection accuracy also decreases.

[0046] By pre-specifying the position and range of the detected object based on the correct answer condition based on the "detection position," it is possible to determine that a detection from an unexpected position is a false detection. False detections are more likely to occur if the score threshold is too low or the distortion tolerance is too large.

[0047] Based on the correct answer condition based on the "Number of Detections," if the actual number of detections differs from the pre-set number of detections, the detection can be determined to have failed. If the "Number of Detections" is 0, it can be determined that the parameters are too strict, resulting in a detection failure. Conversely, if the "Number of Detections" is greater than the expected number, it can be determined that the parameters are too loose, resulting in a detection failure.

[0048] "Detection time" is the time required for detection. Even if detection is successful, if the "Detection time" is too long, it will not meet the cycle time and become an unacceptable parameter. "Detection time" is even longer when the image reduction ratio is small or the score threshold is low.

[0049] The imaging range calculation unit 14 calculates the range of the imaging condition under which the detection possibility determination unit 13 determines that the imaging object is detected.

[0050] Specifically, after the detection feasibility determination unit 13 determines whether the imaging object can be detected under all imaging conditions using the first detection parameter set, the imaging range calculation unit 14 calculates the range of imaging conditions in which the imaging object can be successfully detected.

[0051] Next, after the detection feasibility determination unit 13 determines whether the imaging object can be detected under all imaging conditions using the second detection parameter set, the imaging range calculation unit 14 calculates the range of imaging conditions in which the imaging object can be successfully detected.

[0052] Next, similarly, the imaging range calculation unit 14 calculates the range of imaging conditions for successful detection of the imaging target object for all detection parameter sets.

[0053] The detection parameter determination unit 15 determines an image processing parameter set based on the range of the imaging condition calculated by the imaging range calculation unit 14 .

[0054] For example, the detection parameter determination unit 15 may determine the detection parameter set having the largest range of imaging conditions calculated by the imaging range calculation unit 14 as the image processing parameter set to be actually used.

[0055] The storage unit 20 stores, for example, the imaging conditions set by the imaging condition setting unit 12 , the detection results of the detection determination unit 13 , the imaging range calculated by the imaging range calculation unit 14 , and the detection parameter set determined by the detection parameter determination unit 15 .

[0056] The operation unit 30 performs the setting of the detection parameters and the imaging conditions, and the adjustment of the detection parameters. Figure 6 In this case, the monitor is stacked with a touch panel for receiving operations from the user of the image processing device 1. Alternatively, the operation unit 30 can be configured to display the operation screen. Figure 6 This is achieved by using input devices such as keyboard and mouse to input the operation screen.

[0057] As another embodiment, an image may be divided into one or more regions, and a detection parameter set may be determined for each region.

[0058] [Operation of 2nd embodiment]

[0059] Figure 2 This is a flowchart showing the operation of the image processing device 1 according to this embodiment. Figure 2 Next, the operation of the image processing device 1 will be described.

[0060] In step S1, the detection parameter generation unit 11 sets a parameter set (detection parameter set) for detecting an imaging object. More specifically, the detection parameter set is preliminarily set by combining, for example, a score threshold, a contrast threshold, and a distortion tolerance value.

[0061] In step S2, the control unit 10 sets the value of the score threshold value, the contrast threshold value, and the deformation tolerance value that are initially set to 0. Figure 2In the illustrated example, as described later, the contrast threshold and the distortion allowable value are fixed at their initial settings, and the score threshold is incremented by 1 from 0 to 100 while attempting to detect an imaged object. However, the present embodiment is not limited to this. For example, the score threshold and the distortion allowable value may be fixed at their initial settings while attempting to detect an imaged object while varying the contrast threshold, or the score threshold and the contrast threshold may be fixed at their initial settings while attempting to detect an imaged object while varying the distortion allowable value.

[0062] In step S3, the imaging condition setting unit 12 sets the imaging conditions and captures the image of the imaging object by the imaging device. More specifically, for example, after the imaging condition setting unit 12 sets a certain exposure time as the imaging condition, the image of the imaging object is captured by the visual sensor.

[0063] In step S4 , the control unit 10 detects the imaging target object from the image captured in step S3 using the detection parameters.

[0064] If the detection determination unit 13 determines that the detection of the imaging object is OK in step S5 (S5: Yes), the process proceeds to step S6. If the detection determination unit 13 determines that the detection of the imaging object is NG (S5: No), the process proceeds to step S7.

[0065] Furthermore, a step of determining the correct answer condition may be performed before step S5 of determining whether or not the detection is possible.

[0066] In step S6 , the control unit 10 adds 1 to the score threshold.

[0067] In step S7, if detection has been performed under all imaging conditions (S7: YES), the process moves to step S8. If detection has not been performed under all imaging conditions (S7: NO), the process moves to step S3, the imaging conditions are changed, and an image is captured.

[0068] In step S8 , the imaging range calculation unit 14 calculates the range of imaging conditions determined to detect the imaging object, and the detection parameter determination unit 15 selects and updates the optimal detection parameter set based on the calculated imaging condition range.

[0069] Figures 3 to 5 This is an example of selecting the best detection parameter set.

[0070] Figure 3This table shows the ranges of imaging conditions for each detection parameter set when the illumination light intensity and image reduction values ​​are fixed and the exposure time is varied from 10 ms to 400 ms. The imaging condition range for parameter set A is 20 ms. The imaging condition range for parameter set B is 20 to 30 ms. The imaging condition range for parameter set C is 10 to 40 ms. The imaging condition range for parameter set D is 10 to 400 ms. Of parameter sets A to D, parameter set D has the widest range of imaging conditions, so the detection parameter determination unit 15 sets parameter set D as the optimal detection parameter set.

[0071] Figure 4 This table shows the ranges of imaging conditions for each detection parameter set when the exposure time and image reduction values ​​are fixed and the illumination light intensity is varied from 1 to 16. The illumination light intensity range for parameter set A is 2 to 3. The illumination light intensity range for parameter set B is 2 to 4. The illumination light intensity range for parameter set C is 1 to 4. The imaging conditions range for parameter set D is 1 to 16. Of parameter sets A to D, parameter set D has the widest range of imaging conditions, so the detection parameter determination unit 15 sets parameter set D as the optimal detection parameter set.

[0072] Figure 5 This table shows the ranges of imaging conditions for each detection parameter set when the exposure time and illumination light intensity are fixed and the image reduction ratio is varied from 1 to 1 / 8. The reduction ratio for parameter set A ranges from 1 to 1 / 2. The reduction ratio for parameter set B ranges from 1 to 1 / 3. The reduction ratio for parameter set C ranges from 1 to 1 / 4. The reduction ratio for parameter set D ranges from 1 to 1 / 8. Of parameter sets A through D, parameter set D has the widest range of imaging conditions, so the detection parameter determination unit 15 selects parameter set D as the optimal detection parameter set.

[0073] also, Figures 3-5 Although one imaging condition is changed to determine the optimal detection parameter set, the present invention is not limited thereto, and multiple imaging conditions may be changed to determine the optimal detection parameter set.

[0074] In step S9, if all detection parameter sets have been evaluated (S9: Yes), all processing is terminated. If not all detection parameter sets have been evaluated (S9: No), the process returns to step S1 and the detection parameter set for detecting the imaging object is changed.

[0075] [3 Examples]

[0076] exist Figure 6 An example of the operation screen of the image processing device 1 is shown in FIG.

[0077] Figure 6The illustrated operation screen includes setting items for detection parameters, setting items for imaging conditions, and buttons for determining detection parameters.

[0078] You can set the value range and adjustment range for each detection parameter. You can also set the operation screen for adding or deleting detection parameters to be adjusted.

[0079] You can set the value range and adjustment range for each imaging condition. You can also set an operation screen for adding or deleting imaging conditions.

[0080] After setting the automatic adjustment range and adjustment width of the detection parameters and imaging conditions, the detection parameters are adjusted by pressing the button for determining the detection parameters.

[0081] Instead of pressing buttons on the operation screen, you can call the API for adjusting detection parameters from the program to make adjustments.

[0082] [4 Effects of the Implementation Method]

[0083] The image processing device involved in this embodiment (for example, the above-mentioned "image processing device 1") automatically adjusts the detection parameters used in image processing for detecting an imaging object, and the image processing device includes: a detection parameter generating unit (for example, the above-mentioned "detection parameter generating unit 11"), which generates multiple detection parameter sets for each combination of detection parameters; an imaging condition setting unit (for example, the above-mentioned "imaging condition setting unit 12"), which sets multiple imaging conditions for each combination of the detection parameter sets generated by the detection parameter generating unit; a detection feasibility determination unit (for example, the above-mentioned "detection feasibility determination unit 13"), which determines whether the imaging object can be detected for each combination of the detection parameter set and the imaging condition; an imaging range calculation unit (for example, the above-mentioned "imaging range calculation unit 14"), which calculates the range of imaging conditions for which the detection feasibility determination unit determines that the imaging object is detected; and a detection parameter determination unit (for example, the above-mentioned "detection parameter determination unit 15"), which determines the detection parameter set based on the range of imaging conditions calculated by the imaging range calculation unit.

[0084] By quantifying the quality of detection parameters, automatic adjustment of detection parameters is possible. Quantifying evaluation indicators eliminates individual differences between instructors, enabling stable adjustment. Furthermore, automatic adjustment of detection parameters allows for efficient readjustment even when imaging conditions, such as indoor lighting, change.

[0085] In addition, the parameter determination unit may determine the combination of detection parameters that maximizes the range of imaging conditions as the combination of detection parameters.

[0086] This makes it possible to cope with a wider range of imaging conditions.

[0087] Furthermore, the detection feasibility determination unit may determine whether the detection is possible based on whether the detection result of the imaging target object matches a predetermined correct answer condition.

[0088] This can meet the need to manually set correct answer conditions in advance.

[0089] Furthermore, the detection feasibility determination unit may determine whether the detection is possible based on whether the detection position where the known imaging target object is actually detected matches the known installation position of the imaging target object.

[0090] Therefore, as long as the known installation position of the imaging target object is known, it can be determined whether detection is possible.

[0091] Furthermore, the image processing apparatus may further include an operation unit configured to set detection parameters and imaging conditions, and to adjust detection parameters.

[0092] This allows the user of the image processing device to manually adjust the detection parameters.

[0093] The embodiments of the present invention have been described above, but the present invention is not limited to the aforementioned embodiments. In addition, the effects described in the embodiments are merely examples of the best effects produced by the present invention, and the effects of the present invention are not limited to the effects described in the embodiments.

[0094] The image processing method performed by the image processing device 1 is implemented by software. When implemented by software, the program constituting the software is installed on the computer (image processing device 1). Furthermore, these programs can be distributed to users by being recorded on removable media or by being downloaded to the user's computer via a network. Furthermore, these programs can be provided to the user's computer (image processing device 1) as a web service via a network, rather than being downloaded.

[0095] Description of Reference Numerals

[0096] 1: Image processing device; 10: Control unit; 11: Detection parameter generation unit; 12: Camera condition setting unit; 13: Detection feasibility judgment unit; 14: Camera range calculation unit; 15: Detection parameter determination unit (parameter determination unit); 20: Storage unit; 30: Operation unit.

Claims

1. An image processing device that automatically adjusts detection parameters used in image processing for detecting an imaged object, the image processing device comprising: a detection parameter generating unit, which generates a plurality of combinations of the detection parameters; an imaging condition setting unit configured to set a plurality of imaging conditions for each combination of the detection parameters generated by the detection parameter generating unit; a detection possibility determination unit for determining whether detection of the imaging target object is possible for each combination of the detection parameter and the imaging condition; an imaging range calculation unit that calculates a range of imaging conditions determined by the detection possibility determination unit to have detected the imaging object; as well as A parameter determination unit determines a combination of the detection parameters based on the range of the imaging condition calculated by the imaging range calculation unit.

2. The image processing device according to claim 1, wherein The parameter determination unit determines the combination of the detection parameters that maximizes the range of the imaging conditions as the combination of detection parameters.

3. The image processing device according to claim 1 or 2, characterized in that The detection feasibility determination unit determines whether the detection is possible based on whether a detection result of the imaging target object matches a predetermined correct answer condition.

4. The image processing device according to claim 1 or 2, characterized in that The detection feasibility determination unit determines whether the detection is possible based on whether a detection position where the known imaging target object is actually detected matches a known installation position of the imaging target object.

5. The image processing device according to claim 1 or 2, characterized in that An operation unit is further provided for setting the detection parameters and imaging conditions and adjusting the detection parameters.

6. The image processing device according to claim 1 or 2, characterized in that The image is divided into one or more regions, and the combination of the detection parameters is determined for each region.

Images