Camera control device

Abstract

A camera control device that ensures processing speed and improves recognition accuracy includes: a camera processing unit that uses a camera to acquire a camera image at a set exposure time and gain; an image processing unit that performs image processing on the camera image to obtain information; a memory unit that stores the set values ​​of the exposure time and gain, the target value of the image processing time of the image processing unit, and the result of the image processing; and a setting adjustment unit that adjusts the set values ​​of the exposure time and gain so that the processing time of the image processing unit is below the target value.

Description

Technical Field Field of the Invention

[0001] The present invention relates to an imaging control device. Prior Art Background of the Invention

[0002] For example, in a workpiece processing system or the like, a system that widely uses an object to be photographed and identifies the position of the object in the captured image through image processing is used. It is known that the recognition accuracy of the object can be improved by adjusting the exposure time during imaging or amplifying the luminance signal of the captured image. As an example, for instance, in Patent Document 1, it is proposed to apply gain (amplify the luminance signal) when sufficient brightness cannot be obtained even by controlling the shutter speed or the like. Prior Art Documents Patent Document

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2008-113070 Summary of the Invention Summary of the Invention

[0004] Brief Explanation of the Drawings

[0005] FIG. 1 is a block diagram showing the configuration of an object recognition system including an imaging control device according to the first embodiment of the present disclosure.

[0006] FIG. 2 is a flowchart showing a schematic procedure for adjusting the set value of the setting adjustment unit in FIG. 1.

[0007] FIG. 3 is a flowchart showing a detailed procedure for the correction candidate derivation step in FIG. 2.

[0008] FIG. 4 is a flowchart showing a detailed procedure for the correction candidate derivation step of an alternative in FIG. 3. Summary of the Invention Problems to be Solved by the Invention

[0009] Since imaging takes time if the exposure time is lengthened, the cycle time of the recognition process increases, and the processing ability of the system sometimes decreases. In addition, since noise becomes large if the gain is increased excessively, the recognition accuracy sometimes decreases instead. Therefore, a technique that can ensure the processing speed and improve the recognition accuracy is desired. Means for Solving the Problem

[0010] The imaging control device according to one aspect of the present disclosure includes: an imaging processing unit that uses a camera to acquire an imaging image at a set exposure time and gain; an image processing unit that performs image processing on the imaging image to acquire information; a memory unit that stores set values of the exposure time and the gain, a target value of the processing time of the image processing by the image processing unit, and the result of the image processing; and a setting adjustment unit that adjusts the set values of the exposure time and the gain so that the processing time of the image processing unit becomes equal to or less than the target value. Effect of the Invention

[0011] According to the present disclosure, an imaging control device that can ensure a processing speed and improve recognition accuracy can be provided. Brief Explanation of the Drawings FIG. 1 is a block diagram showing the configuration of an object recognition system including the imaging control device according to the first embodiment of the present disclosure. FIG. 2 is a flowchart showing a schematic procedure for adjusting the set value of the setting adjustment unit in FIG. 1. FIG. 3 is a flowchart showing a detailed procedure of the candidate correction derivation step in FIG. 2. FIG. 4 is a flowchart showing a detailed procedure of the candidate correction derivation step for an alternative in FIG. 3. Embodiment Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an object recognition system 1 including an imaging control device 10 according to the first embodiment of the present disclosure.

[0013] The object recognition system 1 includes: an imaging control device 10; a camera 20 that is controlled by the imaging control device 10 and captures an image of an area where an object may exist; and a display / input device 30 that provides a user interface for the imaging control device 10. The object recognition system 1 recognizes the position of the object and inputs the position and posture of the object to the robot R, for example, in order to pick up the object by the robot R.

[0014] The camera control device 10 includes a camera processing unit 11, an image processing unit 12, a memory unit 13, and a setting adjustment unit 14. The camera control device 10 can be implemented by having a computer device having, for example, a processor, memory, and an input / output interface execute an appropriate control program. The camera processing unit 11, image processing unit 12, memory unit 13, and setting adjustment unit 14 can also be used to categorize the functions of the camera control device 10, rather than being clearly distinguished by their physical and program configurations.

[0015] The camera 20 has an imaging element that generates a brightness signal for each pixel, and is configured to control imaging actions, including adjusting exposure time, by means of external signals.

[0016] As a display / input device 30, in addition to using a device that integrates a display device and an input device, such as a touch panel, a combination of a display device such as a display panel and an input device such as a keyboard and a mouse can also be used.

[0017] The image processing unit 11 uses the camera 20 to acquire an image at a preset exposure time (the exposure time of the image sensor) and gain (the amplification rate of the brightness signal of the image sensor). The settings for exposure time and gain are stored in the memory unit 13. When the settings are adjusted, the image processing unit 11 acquires the image captured by the camera 20 according to the requirements from the setting adjustment unit 14. The function of amplifying the brightness signal with the preset gain can be installed in either the camera 20 or the image processing unit 11.

[0018] The image processing unit 12 performs image processing on the captured image to obtain information. In this embodiment, the image processing unit 12 is designed to identify objects with pre-defined shapes existing in the captured image by pattern matching processing, so as to obtain information for specifying the position and orientation of the objects.

[0019] The memory unit 13 stores information related to the processing of the camera control device 10. This information includes the set values ​​of exposure time and gain, the target value of the image processing time of the image processing unit, the characteristic values ​​of the object being processed, the set values ​​of the pattern matching processing conditions, and the results of the image processing. The set values ​​of exposure time and gain can be rewritten by the setting adjustment unit 14. Furthermore, the initial values ​​of exposure time and gain can also be set in the program or input by the user through the display / input device 30.

[0020] As a correction process, the setting adjustment unit 14 adjusts the settings of exposure time and gain so that the processing time of the image processing unit 12 is below the target value. The target value of the processing time can be input by the user through, for example, a display / input device 30.

[0021] Although the setting adjustment unit 14 can also automatically complete the correction of the exposure time and gain settings, it can also display the image processing information related to the captured image before and after the adjustment of the exposure time and gain settings on the display / input device 30 for comparison, so that the user can judge whether the adjusted exposure time and gain settings are acceptable.

[0022] The setting adjustment unit 14 may also be configured to use values ​​such as the brightness histogram of the captured image, the noise level of the captured image, and the matching score of the pattern matching derived from the image processing of the captured image as indicators to determine whether the setting values ​​of exposure time and gain are appropriate.

[0023] Next, the detailed procedure for adjusting the setting value using the setting adjustment unit 14 will be explained. Figure 2 is a flowchart showing a general procedure for adjusting the setting value using the setting adjustment unit.

[0024] The setting adjustment unit 14 may perform the following steps for adjusting the setting value: initial imaging step (S01), initial image processing step (S02), initial processing time confirmation step (S03), correction candidate export step (S04), correction candidate application result display step (S05), user selection confirmation step (S06), setting value correction step (S07), and processing time target value resetting acceptance step (S08).

[0025] In the initial image capture step S01, the setting adjustment unit 14 instructs the image processing unit 11 to use the initial values ​​of the exposure time and gain settings to acquire the image. The initial value of the setting can be a setting that has just been used, or it can be a value that has been set separately to be used as the initial value for adjusting the setting, or a step of receiving the initial value from the user can be further set in the setting adjustment processing procedure.

[0026] In the initial image processing step S02, the setting adjustment unit 14 instructs the image processing unit 12 to perform image processing on the image captured in the initial imaging step, and obtains the time required for image processing, i.e., the initial processing time.

[0027] In the initial processing time confirmation step S03, the setting adjustment unit 14 confirms the relationship between the processing time obtained in the initial image processing step and the target value of the processing time stored in the memory unit 13. If the processing time obtained in the initial image processing step is less than or equal to the target value, the exposure time and gain settings are deemed appropriate, and the processing ends. Alternatively, if the processing time obtained in the initial image processing step exceeds the target value, the process proceeds to S04.

[0028] In the setting value candidate export step S04, the setting adjustment unit 14 exports correction candidates for the setting values ​​of exposure time and gain in the steps described later. In the correction candidate export step, the image is acquired and processed using the setting values ​​that are used as correction candidates.

[0029] In the correction candidate application result display step S05, the setting adjustment unit 14 displays preset items on the display / input device 30 from the correction candidates for setting values ​​exported in the correction candidate export step, as well as the acquired camera images and image processing results. The items to be displayed are those useful for the user to determine the appropriateness of the correction candidates for setting values. Furthermore, in the correction candidate application result display step, the user can input whether to adopt the correction candidates for setting values. Therefore, buttons urging the user to input can also be displayed on the screen displayed in the correction candidate application result display step.

[0030] In the user selection confirmation step S06, the setting adjustment unit 14 confirms the user's input, which is the input to select whether to accept the adoption of the setting value correction candidate. When the user accepts the adoption of the correction candidate, proceed to S7; when the user does not accept the adoption of the correction candidate, proceed to S8.

[0031] In the setting correction step of S07, the setting adjustment unit 14 rewrites the setting values ​​of exposure time and gain stored in the memory unit 13 into correction candidate values.

[0032] In the processing time target value reset acceptance step of S08, the setting adjustment unit 14 accepts the reset of the processing time target value performed by the user. Therefore, the display / input device 30 should display a prompt to input the processing time target value, such as a combination of a text box for inputting the current target value and the new target value. If the reset of the processing time target value is completed, return to S04 and repeat the above-mentioned export of correction candidates and confirmation of whether to adopt them.

[0033] Figure 3 illustrates the detailed procedure of the corrected candidate export process in step S04 of Figure 2. The corrected candidate export process includes a gain change step (S11), a temporary exposure time setting step (S12), an adjustment camera step (S13), a histogram confirmation step (S14), a corrected candidate setting step (S15), an exposure time increase step (S16), an exposure time confirmation step (S17), and a corrected candidate image processing step (S18).

[0034] In the gain change step S11, the setting adjustment unit 14 sets a gain correction candidate based on the initial processing time measured in the initial image processing step of FIG2. As a specific example, if the difference between the initial processing time and the target value of the processing time is set as T, the current exposure time setting value is set as S1, and the current gain setting value is set as G1, the gain correction candidate G2 can be the minimum settable value that satisfies G2 ≥ (S1 / T) × G1.

[0035] In the temporary exposure time setting step S12, the setting adjustment unit 14 performs an initial setting of the temporary exposure time setting value used in the confirmation process of exporting correction candidates. The temporary setting value is stored once in a different operation memory area than the correction candidates. The initial value of the temporary exposure time setting value should preferably be the minimum value that can be set in the camera 20.

[0036] In the adjustment camera step of S13, the setting adjustment unit 14 obtains a camera image with a temporary setting value for exposure time and a correction candidate for gain.

[0037] In the histogram confirmation step S14, the setting adjustment unit 14 confirms the consistency between the histogram of the brightness of the image obtained by the adjusted imaging step and the histogram of the brightness of the image obtained by the initial imaging step in S01 of FIG2. The consistency of the histogram can be calculated based on one or more indicators such as the median, average, predetermined ranking value, pixel ratio within a predetermined range, etc. In the histogram confirmation step, when the consistency between the histogram of the image obtained with the current temporary setting value of the exposure time and the image obtained by the initial imaging step is the greatest, proceed to S15; when it is not the greatest, proceed to S16.

[0038] In the correction candidate setting step S15, the setting adjustment unit 14 rewrites the correction candidate value of the exposure time to the current temporary setting value of the exposure time. Furthermore, in the correction candidate setting step, the image obtained with the current temporary exposure time setting, its histogram, and its consistency with the image obtained with the initial imaging step are also stored as correction candidate information. Therefore, in the histogram confirmation step, it is only necessary to compare it with the consistency value of the correction candidate.

[0039] In the exposure time increase step S16, the setting adjustment unit 14 rewrites the temporary setting value of the exposure time to the increased value. The amount of increase in the temporary setting value of the exposure time can be constant, or the temporary setting value can be increased at a certain rate.

[0040] In the exposure time confirmation in S17, the setting adjustment unit 14 checks whether the temporary setting value of the exposure time has been exceeded. The upper limit of the exposure time is a value that may be equal to the target value when the processing time is used, and can be set as (S1-T).

[0041] In the corrected candidate image processing step S18, the setting adjustment unit 14 obtains the information displayed in the corrected candidate application result display step S05 of FIG2, and uses it as the exposure time of the current corrected candidate to perform image acquisition and image processing. Furthermore, in this corrected candidate image processing step, the optimal value of gain can also be derived based on the image processing results, and the corrected candidate with the gain setting value is corrected to a gain that has been optimized for the corrected candidate with the exposure time.

[0042] As described above, the camera control device 10 includes a setting adjustment unit 14. This setting adjustment unit 14 adjusts the settings for exposure time and gain to bring the processing time of the image processing unit 12 below a target value. More specifically, it first changes the gain setting, then adjusts the exposure time setting to maximize the consistency of the brightness histogram of the captured image before and after the gain setting change. Therefore, the camera control device 10 can increase processing speed and maintain recognition accuracy without changing the image processing settings of the image processing unit 12.

[0043] Furthermore, since the setting adjustment unit 14 can compare and display image processing information related to the captured images before and after adjusting the exposure time and gain settings, allowing the user to judge the feasibility of the adjusted exposure time and gain setting candidates, the camera control device 10 can more reliably ensure the accuracy of image processing. Moreover, even if such user-confirmed steps (S05, S06, S08) are omitted, the accuracy of image processing can still be fully guaranteed.

[0044] Furthermore, Figure 4 shows an alternative procedure for the modified candidate derivation process, replacing the processing procedure in Figure 3. The modified candidate derivation process in Figure 4 includes a gain change step (S11), a temporary exposure time setting step (S12), an adjustment imaging step (S13), a noise level confirmation step (S21), an adjustment image processing step (S22), a matching score confirmation step (S23), a modified candidate setting step (S24), and an exposure time confirmation step (S25). Furthermore, the same symbols are used for the steps identical to those in Figure 3, and detailed explanations are omitted.

[0045] In the noise level confirmation step of S21, the setting adjustment unit 14 confirms and adjusts the noise level of the captured image obtained by the imaging step. The noise level can be calculated from the changes in pixel values ​​before and after noise cancellation processing of the captured image, such as the number of changed pixels or the cumulative value of the change. The noise cancellation processing can be performed according to a well-known algorithm. When the noise level is below a predetermined threshold, proceed to S22; when the noise level exceeds the predetermined threshold, skip S22, S23, and S24 and proceed to the next step, S18.

[0046] In the adjustment image processing step of S22, the adjustment unit 14 performs image processing on the camera image obtained by the adjustment camera step through the image processing unit 12. In addition to obtaining the result, it also obtains the matching score value calculated for the pattern matching process.

[0047] In the matching score confirmation step of S23, the setting adjustment unit 14 confirms whether the matching score of the image processing step for adjustment is the maximum. Specifically, it can confirm the relationship between the matching score and the current adjustment candidate. When the matching score of the current temporary setting value is the maximum, proceed to S24; when the matching score of the current temporary setting value is not the maximum, skip S24 and proceed to S25.

[0048] In the correction candidate setting step S24, the setting adjustment unit 14 rewrites the correction candidate value of the exposure time to the current temporary setting value of the exposure time. Furthermore, in the correction candidate setting step, the image captured with the current temporary setting value of the exposure time, its noise level, etc., are also memorized as correction candidate information.

[0049] In the exposure time confirmation step of S25, the setting adjustment unit 14 confirms whether the processing time of the image processing step for adjustment exceeds the target value of the processing time.

[0050] As described above, the camera control device 10 employing the corrected candidate export processing shown in Figure 4 first changes the gain setting value. Then, before and after the gain setting value change, while ensuring the noise level in the captured image does not exceed a predetermined threshold, it adjusts the exposure time setting value to maximize the matching score in image processing. Therefore, the camera control device 10 can improve processing speed while maintaining recognition accuracy.

[0051] The above describes the embodiments disclosed herein, but the present invention is not limited to the foregoing embodiments. Furthermore, the effects described in the foregoing embodiments only illustrate the best effects produced by the present invention, but the effects of the present invention are not limited to the effects described in the foregoing embodiments.

[0052] The camera control device disclosed herein can be used not only in systems that identify objects by pattern matching, but also in any system that obtains information by image processing of video images obtained by a camera.

[0053] 1: Object Recognition System

[0054] 10: Camera control device

[0055] 11: Camera Processing Department

[0056] 12: Image Processing Department

[0057] 13: Memory Department

[0058] 14: Setting Adjustment Section

[0059] 20: Camera

[0060] 30: Display / Input Device

[0061] R: Robot

[0062] S01~S08, S11~S18, S21~S25: Steps

Claims

1. A camera control device comprising: a camera processing unit that uses a camera to acquire a camera image at a preset exposure time and gain; an image processing unit that performs image processing on the camera image to acquire information; a memory unit that stores the preset values ​​of the exposure time and gain, a target value of the processing time of the image processing unit, and the result of the image processing; and a setting adjustment unit that adjusts the preset values ​​of the exposure time and gain so that the processing time of the image processing unit is below the target value, wherein the setting adjustment unit adjusts the preset value of the exposure time so that the consistency of the brightness histogram of the camera image before and after the change of the preset value of the gain is maximized.

2. The camera control device of claim 1, wherein the aforementioned setting adjustment unit can comparatively display the aforementioned image processing information related to the aforementioned camera images before and after the adjustment of the aforementioned exposure time and the aforementioned gain setting values, so that the user can judge whether the adjusted aforementioned exposure time and the aforementioned gain setting values ​​are acceptable.

Images