Imaging control device

Abstract

One form of an imaging control device capable of improving recognition accuracy while ensuring processing speed comprises: an imaging processing unit that uses a camera to acquire an imaged image at an exposure time and gain that were set; an image processing unit that performs image processing on the imaged image and acquires information; a storage unit that stores set values for the exposure time and the gain, a processing time target value for the image processing in the image processing unit, and image processing results; and a setting adjustment unit that adjusts the set values for the exposure time and the gain such that the processing time in the image processing unit becomes at most the target value.

Description

[Technical Field]

[0001] The present invention relates to an imaging control device. [Background technology]

[0002] For example, in workpiece handling systems and the like, systems that capture an image of an object and recognize the position of the object in the captured image through image processing are widely used. It is known that the accuracy of object recognition can be improved by adjusting the exposure time during image capture or amplifying the luminance signal of the captured image. For example, Patent Document 1 proposes applying gain (amplifying the luminance signal) when sufficient brightness is still not obtained even after controlling the shutter speed, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-113070 Summary of the Invention [Problem to be solved by the invention]

[0004] Increasing the exposure time takes time to capture an image, which increases the cycle time for recognition processing and can reduce the system's processing power. On the other hand, increasing the gain too much increases noise, which can actually reduce recognition accuracy. For this reason, a technology that can improve recognition accuracy while maintaining processing speed is desired. [Means for solving the problem]

[0005] An imaging control device according to one aspect of the present disclosure includes an imaging processing unit that uses a camera to acquire an image captured at a set exposure time and gain, an image processing unit that performs image processing on the captured image to acquire information, a memory unit that stores set values ​​for the exposure time and the gain, as well as a target value for the processing time of the image processing in the image processing unit and the result of the image processing, and a setting adjustment unit that adjusts the set values ​​for the exposure time and the gain so that the processing time in the image processing unit is equal to or less than the target value. [Effects of the Invention]

[0006] According to the present disclosure, it is possible to provide an imaging control device that can improve recognition accuracy while ensuring processing speed. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of an object recognition system including an imaging control device according to a first embodiment of the present disclosure. [Figure 2] 4 is a flowchart showing an outline of the procedure for adjusting setting values ​​in the setting adjustment unit of FIG. 1. [Figure 3] 3 is a flowchart showing a detailed procedure of the correction candidate derivation step in FIG. 2. [Figure 4] 4 is a flowchart showing a detailed procedure of the step of deriving correction candidates for alternatives in FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of an object recognition system 1 including an imaging control device 10 according to a first embodiment of the present disclosure.

[0009] The object recognition system 1 includes an imaging control device 10, a camera 20 that is controlled by the imaging control device 10 and captures an image of an area where an object may exist, and a display / input device 30 that provides a user interface for the imaging control device 10. The object recognition system 1 recognizes the position of an object and inputs the position and orientation of the object to a robot R, for example, so that the robot R can pick up the object.

[0010] The imaging control device 10 includes an imaging processing unit 11, an image processing unit 12, a storage unit 13, and a setting adjustment unit 14. The imaging control device 10 can be realized by causing a computer device having, for example, a processor, a memory, an input / output interface, etc. to execute an appropriate control program. The imaging processing unit 11, the image processing unit 12, the storage unit 13, and the setting adjustment unit 14 are classifications of the functions of the imaging control device 10, and do not necessarily have to be clearly distinguishable in terms of physical configuration and program configuration.

[0011] The camera 20 is provided with an imaging element that generates a luminance signal for each pixel, and is configured so that imaging operations, including adjustment of exposure time, etc., can be controlled by an external signal.

[0012] The display / input device 30 may be a device in which a display device and an input device are integrated, such as a touch panel, or a combination of a display device such as a display panel and an input device such as a keyboard or mouse.

[0013] The imaging processing unit 11 uses the camera 20 to acquire an image captured at a set exposure time (time for exposing the imaging element) and gain (amplification factor of the luminance signal of the imaging element). The set values ​​of the exposure time and gain are stored in the storage unit 13. When adjusting the set values, the imaging processing unit 11 acquires an image captured by the camera 20 in accordance with a request from the setting adjustment unit 14. The function of amplifying the luminance signal with the set gain may be implemented in the camera 20 or in the imaging processing unit 11.

[0014] The image processing unit 12 processes the captured image to acquire information. The image processing unit 12 in this embodiment is assumed to identify an object having a predetermined shape present in the captured image by pattern matching processing, and acquire information specifying the position and orientation of the object.

[0015] The storage unit 13 stores information related to processing in the imaging control device 10, including exposure time and gain setting values, a target value for the processing time of image processing in the image processing unit, feature amounts of a pattern of an object to be image processed, setting values ​​such as conditions for pattern matching processing, and results of image processing. The exposure time and gain setting values ​​can be rewritten by the setting adjustment unit 14. Furthermore, the initial values ​​of the exposure time and gain setting values ​​can be set in a program or can be input by the user via the display / input device 30.

[0016] As part of the calibration process, the setting adjustment unit 14 adjusts the exposure time and gain settings so that the processing time in the image processing unit 12 is equal to or less than a target value. The target value for the processing time can be input by the user via the display / input device 30, for example.

[0017] The setting adjustment unit 14 may complete the process of correcting the exposure time and gain setting values ​​entirely automatically, or may display image processing information about the captured image before and after adjusting the exposure time and gain setting values ​​on the display / input device 30 so that the user can judge whether the adjusted exposure time and gain setting values ​​are acceptable.

[0018] The setting adjustment unit 14 may be configured to use values ​​such as a luminance histogram of the captured image, the amount of noise in the captured image, and a matching score of pattern matching derived during image processing of the captured image as indicators for determining whether the exposure time and gain setting values ​​are appropriate.

[0019] Next, a detailed description will be given of the procedure for adjusting the setting values ​​by the setting adjustment unit 14. Fig. 2 is a flowchart showing an outline of the procedure for adjusting the setting values ​​by the setting adjustment unit.

[0020] The setting value adjustment process by the setting adjustment unit 14 may include an initial imaging step (S01), an initial image processing step (S02), an initial processing time confirmation step (S03), a correction candidate derivation step (S04), a correction candidate application result display step (S05), a user selection confirmation step (S06), a setting value correction step (S07), and a processing time target value reset acceptance step (S08).

[0021] In the initial imaging step of S01, the setting adjustment unit 14 causes the imaging processing unit 11 to acquire an image by applying initial values ​​for the exposure time and gain settings. The initial values ​​for the settings may be the setting values ​​used immediately before or may be values ​​set separately as initial values ​​for adjusting the setting values, and the procedure for adjusting the setting values ​​may further include a step of accepting initial values ​​from the user.

[0022] In the initial image processing step of S02, the setting adjustment unit 14 causes the image processing unit 12 to process the captured image acquired in the initial image capturing step, and acquires the initial processing time, which is the time required for the image processing.

[0023] In the initial processing time confirmation step of S03, the setting adjustment unit 14 checks the magnitude relationship between the processing time acquired in the initial image processing step and the target value of the processing time stored in the memory unit 13. If the processing time acquired in the initial image processing step is equal to or less than the target value, the exposure time and gain setting values ​​are determined to be appropriate, and the process ends. On the other hand, if the processing time acquired in the initial image processing step exceeds the target value, the process proceeds to S04.

[0024] In the setting value candidate derivation step S04, the setting adjustment unit 14 derives correction candidates for the setting values ​​of the exposure time and the gain in a procedure described below. In the correction candidate derivation step, a captured image is acquired using the setting values ​​that are the correction candidates, and the image is processed.

[0025] In the correction candidate application result display step of S05, the setting adjustment unit 14 displays on the display / input device 30 the correction candidates for the setting values ​​derived in the correction candidate derivation step, as well as pre-set items from among the captured image and image processing results acquired using the correction candidates. Items to be displayed are selected that are useful for the user to determine the appropriateness of the correction candidate for the setting values. Furthermore, the correction candidate application result display step may accept input from the user as to whether or not to adopt the correction candidate for the setting values. For this reason, a button or the like prompting the user to input information may be displayed on the screen displayed in the correction candidate application result display step.

[0026] In the user selection confirmation step of S06, the setting adjustment unit 14 confirms the user's input to select whether or not to approve the adoption of the correction candidate for the setting value. If the user approves the adoption of the correction candidate, the process proceeds to S7, and if the user does not approve the adoption of the correction candidate, the process proceeds to S8.

[0027] In the setting value correction step of S07, the setting adjustment unit 14 rewrites the setting values ​​of the exposure time and gain stored in the storage unit 13 with the correction candidate values.

[0028] In the target processing time reset receiving step of S08, the setting adjustment unit 14 receives a reset of the target processing time from the user. For this purpose, it is preferable that the display / input device 30 displays a message prompting the user to input the target processing time, such as a combination of the current target value and a text box for inputting a new target value. Once the target processing time has been reset, the process returns to S04, where the above-described process of deriving correction candidates and checking whether they are adopted is repeated.

[0029] Fig. 3 shows detailed steps of the correction candidate derivation process in the correction candidate derivation step S04 in Fig. 2. The correction candidate derivation process includes a gain change step (S11), an exposure time temporary setting step (S12), an adjustment imaging step (S13), a histogram confirmation step (S14), a correction candidate setting step (S15), an exposure time increase step (S16), an exposure time confirmation step (S17), and a correction candidate image processing step (S18).

[0030] In the gain change step S11, the setting adjustment unit 14 sets a candidate for gain correction based on the initial processing time measured in the initial image processing step of Fig. 2. As a specific example, if the difference between the initial processing time and the target value for processing time is T, the current exposure time setting is S1, and the current gain setting is G1, the candidate for gain correction G2 can be the minimum value that can be set and satisfies G2 ≥ (S1 / T) × G1.

[0031] In the provisional exposure time setting step of S12, the setting adjustment unit 14 initially sets a provisional exposure time value to be used in the confirmation process for deriving correction candidates. The provisional exposure time value is temporarily stored in a working memory area different from the correction candidates. The initial provisional exposure time value is preferably the minimum value that can be set in the camera 20.

[0032] In the adjustment imaging step of S13, the setting adjustment unit 14 acquires a captured image to which the provisional setting value of the exposure time and the correction candidate of the gain have been applied.

[0033] In the histogram confirmation step of S14, the setting adjustment unit 14 confirms the degree of match between the luminance histogram of the captured image acquired in the adjustment imaging step and the luminance histogram of the captured image acquired in the initial imaging step of S01 in Fig. 2. The degree of match between the histograms can be calculated based on one or more indices, such as the median, the average, a predetermined rank value, or the percentage of pixels within a predetermined range. In the histogram confirmation step, if the degree of match between the histogram of the captured image acquired with the provisional setting value of the current exposure time and the captured image acquired in the initial imaging step is at its highest, the process proceeds to S15; otherwise, the process proceeds to S16.

[0034] In the correction candidate setting step of S15, the setting adjustment unit 14 rewrites the correction candidate value for the exposure time to the provisional setting value of the current exposure time. Also, in the correction candidate setting step, the captured image acquired with the provisional setting value of the current exposure time, its histogram, and the degree of agreement with the captured image acquired in the initial capturing step are also stored as information on the correction candidate. Therefore, in the histogram checking step, the degree of agreement value of the correction candidate can be compared.

[0035] In the exposure time increasing step of S16, the setting adjustment unit 14 rewrites the provisional setting value of the exposure time to an increased value. The increase in the provisional setting value of the exposure time may be constant, or the provisional setting value may be increased at a constant rate.

[0036] In the exposure time confirmation in S17, the setting adjustment unit 14 checks whether the provisional setting value of the exposure time has been exceeded. The upper limit of the exposure time is set to a value at which the processing time can be equal to the target value, and can be set as (S1-T).

[0037] In the correction candidate image processing step of S18, the setting adjustment unit 14 captures an image and processes the image at the exposure time currently selected as the correction candidate in order to obtain information to be displayed in the correction candidate application result display step of S05 in Fig. 2. Furthermore, in this correction candidate image processing step, a process of deriving an optimal gain value based on the result of the image processing may be further performed to correct the correction candidate gain setting value to a gain optimized for the correction candidate exposure time.

[0038] As described above, the imaging control device 10 includes a setting adjustment unit 14 that adjusts the exposure time and gain settings so that the processing time in the image processing unit 12 is equal to or less than a target value, more specifically, that first changes the gain setting and then adjusts the exposure time setting so that the degree of match between the luminance histograms of the captured image before and after changing the gain setting is as high as possible. Therefore, the imaging control device 10 can maintain recognition accuracy while improving processing speed without changing the image processing settings in the image processing unit 12.

[0039] Furthermore, in the imaging control device 10, the setting adjustment unit 14 displays image processing information for captured images before and after adjustment of the exposure time and gain settings in a comparative manner, and allows the user to determine whether or not to accept correction candidates for the exposure time and gain settings after adjustment, thereby more reliably ensuring the accuracy of image processing. Note that even if the steps for such user confirmation (S05, S06, S08) are omitted, the accuracy of image processing can be sufficiently guaranteed.

[0040] Furthermore, Fig. 4 shows an alternative procedure for the correction candidate deriving process that can be adopted in place of the processing procedure of Fig. 3. The correction candidate deriving process of Fig. 4 includes a gain change step (S11), an exposure time temporary setting step (S12), an adjustment image capture step (S13), a noise amount confirmation step (S21), an adjustment image processing step (S22), a matching score confirmation step (S23), a correction candidate setting step (S24), and an exposure time confirmation step (S25). Note that steps similar to those of Fig. 3 are assigned the same reference numerals, and detailed explanations thereof will be omitted.

[0041] In the noise amount confirmation step of S21, the setting adjustment unit 14 checks the noise amount of the captured image acquired in the adjustment imaging step. The noise amount can be calculated from the change in each pixel value of the captured image before and after noise removal processing, for example, as the number of changed pixels, the integrated value of the change amount, or other values. The noise removal processing can be performed according to a well-known algorithm. If the noise amount is equal to or less than a predetermined threshold, the process proceeds to S22. If the noise amount exceeds the predetermined threshold, the process skips S22, S23, and S24 and proceeds to the next step, S18.

[0042] In the adjustment image processing step of S22, the setting adjustment unit 14 performs image processing on the captured image acquired in the adjustment imaging step using the image processing unit 12, and in addition to the result, obtains the value of the matching score calculated for the pattern matching process.

[0043] In the matching score confirmation step of S23, the setting adjustment unit 14 checks whether the matching score in the adjustment image processing step is the maximum. Specifically, it can check the magnitude relationship with the matching score in the current adjustment candidate. If the matching score in the current provisional setting value is the maximum, proceed to S24, but if the matching score in the current provisional setting value is the maximum, skip S24 and proceed to S25.

[0044] In the correction candidate setting step of S24, the setting adjustment unit 14 rewrites the correction candidate value of the exposure time to the provisional setting value of the current exposure time. In the correction candidate setting step, the setting adjustment unit 14 also stores the captured image acquired with the provisional setting value of the current exposure time, the amount of noise, etc. as correction candidate information.

[0045] In the exposure time confirmation step of S25, the setting adjustment unit 14 checks whether the processing time of the image processing in the adjustment image processing step exceeds the target value of the processing time.

[0046] 4, the imaging control device 10 first changes the gain setting, and then adjusts the exposure time setting so that the matching score in image processing is maximized within a range in which the amount of noise in the captured image does not exceed a predetermined threshold before and after changing the gain setting. This allows the imaging control device 10 to maintain recognition accuracy while improving processing speed.

[0047] Although the embodiments of the present disclosure have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the above-described embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the above-described embodiments.

[0048] The imaging control device according to the present disclosure can be adopted in any system that acquires information by image processing of an image captured using a camera, in addition to a system that recognizes an object by pattern matching. [Explanation of symbols]

[0049] 1 Object Recognition System 10. Imaging control device 11. Imaging processing section 12 Image processing section 13 Storage section 14 Setting adjustment section 20 Camera 30 Display and input devices R Robot

Claims

1. an imaging processing unit that uses a camera to acquire an image captured at a set exposure time and gain; an image processing unit that processes the captured image to acquire information; a storage unit that stores the set values ​​of the exposure time and the gain, a target value of the processing time of the image processing in the image processing unit, and a result of the image processing; a setting adjustment unit that adjusts the setting values ​​of the exposure time and the gain so that the processing time in the image processing unit is equal to or less than the target value; Equipped with The setting adjustment unit adjusts the setting value of the exposure time so that a degree of coincidence between luminance histograms of the captured image before and after changing the setting value of the gain is increased.

2. 2. The imaging control device according to claim 1, wherein the setting adjustment unit displays the image processing information for the captured image before and after adjustment of the exposure time and gain setting values ​​in a comparative manner, and allows a user to determine whether the adjusted exposure time and gain setting values ​​are acceptable.

Images