Imaging device, control method for imaging device, and program

Abstract

To make it possible to reduce an occurrence of a motion blur in a motion region and reduce an occurrence of noise.SOLUTION: An imaging apparatus is configured to divide a captured image into a plurality of regions and carry out imaging under a different exposure condition for each of the regions. The imaging apparatus includes: acquisition means (103) configured to acquire an exposure value for each of the regions in the captured image; detection means (104) configured to detect a motion region on the basis of the captured image; and determination means (105) configured to determine an exposure time and a gain on the basis of the exposure value for each of the regions and a result of detection of the motion region. The determination means (105) sets a limitation on an adjustment range of at least any of the exposure time and the gain in accordance with a detection result of the motion region.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device and its control technology.

Background Art

[0002] Generally, the dynamic range of CCD or CMOS imaging sensors used in imaging devices such as digital cameras and digital video cameras is narrow compared to the dynamic range of nature. Therefore, when imaging an imaging scene with a wide dynamic range (referred to as a high dynamic range and hereinafter abbreviated as "HDR") in a normal way, so-called black crush or white blooming occurs. Thus, as in Patent Document 1, there is a technique for obtaining an HDR scene using a sensor capable of controlling exposure conditions such as the accumulation time (exposure time) and the gain at the time of reading (hereinafter abbreviated as gain) for each region based on a preliminary shot. Further, in Patent Document 2, a technique is disclosed in which when determining the optimal exposure condition for each region based on a preliminary shot, at the boundary portion of the exposure conditions, shooting is performed by changing to a darker exposure condition from the surrounding exposure conditions. According to the technique of Patent Document 2, when a positional shift occurs at the boundary portion due to the time difference between the preliminary shot and the main shot, white blooming occurring at the boundary portion can be reduced. [[ID=​​​​​​​​​​​​​​​​​​​​​​​​​However, if the exposure conditions for a certain area are changed to be darker than the surrounding exposure conditions, and the gain for the area imaged under those exposure conditions is increased, the noise in the image will increase. When noise increases in this way, the temporal correlation decreases, which means that the compression ratio decreases, for example, when compressing image data. And when the compression ratio decreases, the traffic during data transfer increases, and the storage capacity required for data storage also increases. Furthermore, for example, if the subject moves in a dark area where the exposure time tends to be long, motion blur due to the long exposure time is likely to occur. When motion blur occurs in this way, the visibility of the captured subject image decreases, and if image recognition processing is performed, for example, the recognition rate of the subject will decrease.

[0005] Therefore, the present invention aims to suppress the occurrence of motion blur in areas with movement and to reduce the generation of noise. [Means for solving the problem]

[0006] The present invention relates to an imaging device that divides a captured image into multiple regions and captures images under different exposure conditions for each region, comprising: an acquisition means for acquiring exposure values ​​for each region of the captured image; a detection means for detecting moving regions based on the captured image; and a determination means for determining exposure time and gain based on the exposure values ​​for each region and the detection results of the moving regions, wherein the determination means limits the adjustment range of at least one of the exposure time and gain according to the detection results of the moving regions. [Effects of the Invention]

[0007] According to the present invention, it is possible to suppress the occurrence of motion blur in areas with movement, and to reduce the generation of noise. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of the imaging device according to the first embodiment. [Figure 2] This is a flowchart of the overall processing of the imaging device. [Figure 3] This figure shows an example of the correspondence between exposure value and exposure conditions. [Figure 4] This is a flowchart of the process for calculating exposure values ​​for each region. [Figure 5] This is a flowchart for the dynamic area determination process. [Figure 6] This is a diagram used to explain the dynamic area determination process. [Figure 7] This is a flowchart of the exposure condition determination process in the first embodiment. [Figure 8] This is a diagram used to explain the exposure condition correction process. [Figure 9] This is a flowchart for exposure condition correction processing. [Figure 10] This is an explanatory diagram of a correction process that reduces exposure time under the same exposure conditions. [Figure 11] This is an explanatory diagram of a correction process that reduces gain under the same exposure conditions. [Figure 12] This is an explanatory diagram of the correction process when exposure time and gain are limited. [Figure 13] This is a flowchart of the process when there are no exposure conditions that satisfy the constraints. [Figure 14] This figure shows an example of the configuration of the imaging device according to the second embodiment. [Figure 15] This figure shows an example of an exposure condition table. [Figure 16] This is a flowchart of the exposure condition determination process in the second embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the configurations shown in these embodiments are merely examples, and the present invention is not limited to the illustrated configurations. Furthermore, identical components or processes are denoted by the same reference numerals, and their descriptions are omitted as appropriate. First, an overview of the control process of the imaging device in this embodiment will be described. In this embodiment, it is assumed that the imaging sensor included in the imaging device is a sensor capable of setting different exposure times and different gains for each region (including pixel units). Hereinafter, the exposure time and gain for each region are referred to as region-specific exposure conditions. The imaging device of this embodiment performs imaging with an imaging sensor for which region-specific exposure conditions are set to obtain a captured image, and obtains a moving object region (hereinafter referred to as a moving region) and a luminance distribution based on the captured image. Then, the imaging device of this embodiment determines region-specific exposure conditions for the main shooting based on the detection results of those moving regions and the luminance distribution. Note that by setting exposure conditions for each region, taking images with a difference in exposure amount for each region, and synthesizing them, it becomes possible to obtain an image with an expanded dynamic range.

[0010] <First Embodiment> FIG. 1 is a block diagram showing a configuration example of an imaging device 101 according to the first embodiment that can perform imaging under region-specific exposure conditions as described above. In the imaging device 101, the imaging processing unit 102 includes an optical system including a shutter, a lens, an aperture, an optical low-pass filter, etc., and an imaging element capable of capturing a color image. The imaging processing unit 102 can change the exposure time and gain for each region, that is, can set region-specific exposure conditions, and can obtain an image captured under the exposure conditions set for each region. Then, the imaging processing unit 102 outputs digital image data (hereinafter simply referred to as a captured image) obtained by digitally converting an image signal of a subject or the like obtained under region-specific exposure conditions.

[0011] The exposure value calculation unit 103 performs exposure value acquisition processing. The exposure value calculation unit 103 calculates an exposure value for each region (hereinafter referred to as a region-specific exposure value) based on the captured image input from the imaging processing unit 102. Details of the region-specific exposure value calculation processing by the exposure value calculation unit 103 will be described later. The motion area determination unit 104 performs motion area detection processing. The motion area determination unit 104 accumulates past captured images and determines the presence or absence of motion for each area from a plurality of captured images with different shooting times. Details of the motion area detection processing (hereinafter referred to as motion area determination processing) by the motion area determination unit 104 will be described later.

[0012] The condition determination unit 105 performs exposure condition determination processing for each area. The condition determination unit 105 determines exposure conditions for each area based on the exposure value for each area obtained from the exposure value calculation unit 103 and the detection result (information regarding the presence or absence of motion) of the motion area for each area obtained from the motion area determination unit 104. Then, the condition determination unit 105 updates the settings of the imaging processing unit 102 according to the determined exposure conditions for each area. Details of the exposure condition determination processing for each area by the condition determination unit 105 will be described later.

[0013] The exposure correction unit 106 performs exposure amount correction processing. The exposure correction unit 106 corrects the difference in exposure amount for each area of the captured image based on the exposure value for each area calculated by the exposure value calculation unit 103. The captured image in which the difference in exposure amount for each area is corrected by this exposure correction unit 106 is sent to the development processing unit 107 as the captured image (main image) obtained by this shooting. Details of the exposure amount correction processing for each area by the exposure correction unit 106 will be described later.

[0014] The development processing unit 107 performs development processing such as white balance processing, deinterlacing processing, noise reduction processing, sharpness processing, and gamma correction processing on the captured image after the difference in exposure amount for each area is corrected by the exposure correction unit 106. The image output unit 108 outputs the image data after the development processing by the development processing unit 107 to a storage medium such as a printer, a display, or a memory card via a cable or wireless communication.

[0015] <Overall Processing of Imaging Device> Figure 2 is a flowchart showing the overall processing flow from the determination of shooting conditions to the output of the captured image by the imaging device 101 of the present embodiment. In step S201, the condition determination unit 105 sets shooting conditions such as the aperture value and the exposure value which will serve as the reference for the entire image, and sets these shooting conditions to the imaging processing unit 102.

[0016] In step S202, the exposure value calculation unit 103 calculates the exposure value for each region (region-specific exposure value), and the condition determination unit 105 determines the region-specific exposure conditions. Details of the region-specific exposure value calculation process and region-specific exposure condition determination process in step S202 will be described later.

[0017] Next, in step S203, the imaging processing unit 102 performs imaging processing and acquires an image based on the shooting conditions calculated in step S201 and the region-specific exposure conditions set in step S202. Next, in step S204, the exposure compensation unit 106 performs exposure amount correction processing for each region on the captured image acquired in step S203 using different exposure conditions for each region. Here, the region-specific exposure amount correction processing in the exposure compensation unit 106 can be expressed, for example, by the following equation (1).

[0018] C'=ai·C (C=R,G,B) ai=2.0^(EVi-EVbase) Equation (1)

[0019] In equation (1), C represents one of the three primary colors R (red), G (green), or B (blue), ai is the exposure correction value in region i, EVi is the exposure value in region i, and EVbase represents the reference exposure value. For EVbase, for example, the average value of the exposure values ​​in each region is used. The exposure value is an index that indicates the brightness of the shooting conditions corresponding to the exposure time, gain, and aperture value as shown in Figure 3, and is expressed as a power of 2.

[0020] Next, in step S205, the development processing unit 107 performs development processing on the image after the region-specific exposure correction processing has been performed in step S204. Subsequently, in step S206, the image output unit 108 outputs the image after the development process.

[0021] <Exposure value calculation process by region> Figure 4 is a detailed flowchart of the region-specific exposure value calculation process performed during the region-specific exposure condition determination in step S202 of Figure 2. In step S401, the exposure value calculation unit 103 acquires a previously captured image. The previously captured image is, for example, an image taken one frame before the current shot.

[0022] Next, in step S402, the exposure value calculation unit 103 initializes the position of the processing area related to the region-specific exposure value calculation process. The processing area can be of any size as long as it is large enough to control the exposure conditions. For example, the processing area may consist of one pixel or multiple pixels. In this embodiment, the processing area is a region obtained by dividing the image into 128 pixel × 128 pixel units.

[0023] Next, in step S403, the exposure value calculation unit 103 converts the input captured image into a luminance image in which each pixel has a luminance value. Any method can be used for converting to a luminance image, as long as it can convert the pixel values ​​in the processing area into luminance values. For example, if the captured image has an RGB Bayer structure, it can be converted to a common luminance value Y for 2x2 pixels using, for example, the following equation (2).

[0024] Y = 0.2126 × R + 0.7152 × G + 0.0722 × B Equation (2)

[0025] Next, in step S404, the exposure value calculation unit 103 calculates exposure compensation values ​​(EV compensation values) for each region based on the luminance values ​​of the luminance image calculated in step S403. Here, the exposure compensation value (EV compensation value) is a value that represents how much the exposure value should be changed from the reference exposure value for each region to achieve the optimal conditions. For example, if the maximum luminance value of a region is Ymax and the maximum luminance value that the image can take is max, the EV compensation value is shown in the following equation (3).

[0026] if(Ymax==max) EV correction value = -1 else EV correction value = log2(max / Ymax) Equation (3)

[0027] Next, in step S405, the exposure value calculation unit 103 calculates the exposure value after correction using the exposure compensation value (EV compensation value) calculated in step S404. The corrected exposure value (EV) can be calculated as shown in the following equation (4).

[0028] EV = EVbase + EV correction value if(EV<1) EV=1 else if (EV > EVmax) EV = EVmax Equation (4)

[0029] Note that in equation (4), EVbase is a reference exposure value as shown in Figure 3, and EVmax is the exposure value for the darkest possible shooting condition among the set aperture values. Next, in step S406, the exposure value calculation unit 103 determines whether processing has been performed for all regions. If the exposure value calculation unit 103 determines that processing has been performed for all regions, it terminates the region-specific exposure value calculation process shown in Figure 4; otherwise, it proceeds to the process in step S407. When the process moves to step S407, the exposure value calculation unit 103 updates the position of the processing area, then returns to step S403, and repeats the processing from step S403 to step S406.

[0030] <Motion area determination process> Figure 5 is a flowchart detailing the dynamic region determination process in the dynamic region determination unit 104 of Figure 1. Note that steps S401, S402, S406, and S407 in Figure 5 are the same as the processes with the same reference numerals in Figure 4, except that they are performed in the dynamic region determination unit 104, so their explanations are omitted. The dynamic region determination unit 104 uses two captured images (captured image (t) and captured image (t+1)) taken at different times to determine the dynamic region. That is, in the case of Figure 5, in step S401, captured image (t) is acquired as a past image for captured image (t+1). Then, after the processing region position initialization process in step S402, the dynamic region determination unit 104 moves on to the process in step S503.

[0031] In step S503, the motion region determination unit 104 converts the captured image into a luminance image. The conversion process to a luminance image is the same as the luminance conversion process performed in step S403 of the region-specific exposure value calculation process in Figure 4, so its explanation is omitted. Next, in step S504, the motion region determination unit 104 determines the motion region using the brightness image converted in step S503.

[0032] Figure 6 is a diagram used to explain the overview of the motion region determination process in the motion region determination unit 104. As shown in Figure 6, the motion region determination unit 104 receives a luminance image 601 obtained by luminance conversion of the captured image (t) and a luminance image 602 obtained by luminance conversion of the captured image (t+1). Note that each grid in luminance image 601 and luminance image 602 represents a processing region. The motion region determination unit 104 uses these luminance images 601 and luminance image 602 to perform a motion region determination process 603 based on the luminance difference for each region and detects the motion region 604. In the example of luminance images 601 and luminance image 602 shown in Figure 6, there is a luminance difference between the person region 610 and the person region 611, so the region with these luminance differences is detected as the motion region 604. Note that any method can be used to determine whether or not there is movement in each region. For example, as shown in equation (5), the motion Mi can be calculated by summing the absolute difference in brightness in region i, performing gain processing, and then threshold processing.

[0033] Mi=g×(Σ|Yt,i(j)-Yt+1,i(j)|-Th) Equation (5)

[0034] In equation (5), Yt,i is the luminance value in region i within the luminance image of the captured image (t), Yt+1,i is the luminance value in region i within the luminance image of the captured image (t+1), j is the pixel position, g is the gain, and Th is the threshold. The gain g and threshold Th are parameters for adjusting the calculated region error. By adjusting the gain g, it is possible to control the dynamic region intensity. Furthermore, by controlling the threshold Th, which is the absolute difference used to determine a dynamic region, it is possible to control how much luminance difference is considered a dynamic region.

[0035] <Exposure condition determination process> Next, the exposure condition determination process by the condition determination unit 105 will be explained in detail. The condition determination unit 105 determines the exposure conditions based on the dynamic region detection results for each region acquired by the dynamic region determination unit 104 (hereinafter referred to as the dynamic region map) and the exposure values ​​for each region calculated by the exposure value calculation unit 103 (hereinafter referred to as the exposure value map).

[0036] Figure 7 is a detailed flowchart of the exposure condition determination process performed by the condition determination unit 105. In step S701, the condition determination unit 105 selects the processing area position. Next, in step S702, the condition determination unit 105 refers to the exposure value map and obtains the exposure value of the processing area. Next, in step S703, the condition determination unit 105 acquires standard exposure conditions corresponding to the exposure value of the processing area.

[0037] Next, in step S704, the condition determination unit 105 compares the exposure value of the processing area with an appropriate reference value (threshold) to determine whether the processing area is subject to processing, that is, whether it is a dark area with an exposure value below the threshold. If the condition determination unit 105 determines that the area is subject to processing (determined that the exposure value is below the threshold), it proceeds to step S705. On the other hand, if it determines that the area is not subject to processing (determined that the exposure value is not below the threshold), it proceeds to step S708. If the process proceeds to step S708, no exposure condition correction processing is performed for the processing area.

[0038] When the process moves to step S705, the condition determination unit 105 refers to the dynamic region map and obtains the dynamic region detection processing result for the processing region. Next, in step S706, the condition determination unit 105 determines whether the processing area belongs to the dynamic area. If it belongs to the dynamic area, it moves the process to step S707; otherwise, it moves the process to step S708.

[0039] Moving to step S707, the condition determination unit 105 performs exposure condition correction processing. The details of this exposure condition correction processing will be described later. On the other hand, when the process moves to step S708, the condition determination unit 105 determines whether all areas have been processed. If the condition determination unit 105 determines that all areas have been processed, it terminates the processing shown in the flowchart of Figure 7; otherwise, it returns to step S701.

[0040] <Exposure condition correction processing> The exposure condition correction process performed in step S707 of Figure 7 will be described below. In the exposure condition correction process of step S707, the condition determination unit 105 corrects the exposure conditions based on the exposure condition limitations set externally. In this embodiment, the exposure condition limitations are set to the exposure time and the gain adjustment range. In this embodiment, an upper limit is set for the exposure time adjustment range as a limitation on the exposure time, and an upper limit is set for the gain adjustment range as a limitation on the gain. By setting an upper limit for the exposure time adjustment range in this way, it is possible to suppress the occurrence of motion blur in the moving region and improve visibility. Furthermore, by setting an upper limit for the gain adjustment range, it is possible to suppress the generation of noise and improve the image compression ratio.

[0041] Here, we will explain the relationship between exposure value and exposure conditions. Figure 8 is a diagram showing exposure time on the horizontal axis and gain value on the vertical axis. The range 801, indicated by an arrow in Figure 8, is the range in which the exposure time can be changed (referred to as the variable exposure time range 801), and corresponds to the adjustment range of the exposure time. Similarly, the range 802, also indicated by an arrow, is the range in which the gain can be changed (referred to as the variable gain range 802), and corresponds to the adjustment range of the gain. Region 804 represents the region in which both exposure time and gain can be changed. Line 803 is a line that shows the relationship between exposure time and gain value corresponding to the same exposure value. In other words, if the intersection point where the exposure time and gain value intersect is on the diagram (on line 803), the exposure value will always be the same regardless of the combination of exposure time and gain value (exposure condition). On the other hand, line 806 is a line that defines the combination of exposure time and gain value (exposure condition) corresponding to the exposure value. That is, in the region 804 in which both exposure time and gain can be changed, the exposure condition 805 at the intersection of line 806 and line 803 represents the exposure condition for that exposure value. In the example shown in Figure 8, the exposure time corresponding to exposure condition 805 is value 807, and the gain is value 808.

[0042] Figure 9 is a detailed flowchart of the exposure condition correction process in step S707 of Figure 7. In step S901, the condition determination unit 105 checks whether an upper limit value for the exposure time adjustment range (hereinafter referred to as the exposure time upper limit) exists. If an exposure time upper limit is set, the condition determination unit 105 proceeds to step S902; otherwise, it proceeds to step S904.

[0043] When the process moves to step S902, the condition determination unit 105 refers to the diagram shown in Figure 8 and compares the exposure time value obtained from the exposure value of the processing area with the upper limit of the exposure time. If the exposure time value exceeds the upper limit of the exposure time, the condition determination unit 105 moves the process to step S903; on the other hand, if it is less than or equal to the upper limit of the exposure time, it moves the process to step S904.

[0044] When the process moves to step S903, the condition determination unit 105 performs exposure time correction processing, changing the exposure time to below the upper limit without changing the exposure value. In other words, the condition determination unit 105 performs exposure time correction processing to reduce the exposure time under the same exposure conditions. To put it another way, the condition determination unit 105 performs exposure time correction processing to reduce the exposure time so that the position of the intersection point where the exposure time and the gain value intersect does not deviate from the diagram (on line 803). Details of this process will be described later.

[0045] Next, in step S904, the condition determination unit 105 checks whether an upper limit value for the gain adjustment range (hereinafter referred to as the gain upper limit value) exists. If a gain upper limit value is set, the condition determination unit 105 moves to step S905; otherwise, it terminates the process shown in the flowchart of Figure 9.

[0046] When the process moves to step S905, the condition determination unit 105 refers to the diagram shown in Figure 8 and compares the gain value obtained from the exposure value of the processing area with the upper limit of the gain. If the gain value exceeds the upper limit of the gain, the condition determination unit 105 moves the process to step S906; otherwise, it terminates the process shown in the flowchart of Figure 9.

[0047] When the process moves to step S906, the condition determination unit 105 performs gain correction processing, changing the gain to below the upper limit without changing the exposure value. In other words, the condition determination unit 105 performs gain correction processing to reduce the gain under the same exposure conditions. To put it another way, the condition determination unit 105 performs gain correction processing to reduce the gain value so that the position of the intersection point where the exposure time and the gain value intersect does not deviate from the diagram (on line 803). Details of this process will be described later.

[0048] <Exposure time correction processing> The exposure time correction process in step S903 of Figure 9 will be explained using the diagram shown in Figure 10. In Figure 10, the horizontal axis represents exposure time and the vertical axis represents gain. Figure 10 shows the same variable range 801 for exposure time, variable range 802 for gain, line 803, line 806, region 804, and exposure condition 805 as in Figure 8. Figure 10 also shows the upper limit of exposure time 1007. The condition determination unit 105 refers to the diagram in Figure 10 and obtains the exposure time corresponding to exposure condition 805, which is the intersection of line 803 for the same exposure value and line 806 that defines the exposure condition, and compares it with the upper limit of exposure time 1007. In the example in Figure 10, since the exposure time corresponding to exposure condition 805 exceeds the upper limit of exposure time 1007, the condition determination unit 105 corrects the exposure condition to the correction point 1008, which is the intersection of line 803 for the same exposure value and the line for the upper limit of exposure time 1007. The exposure time and gain values ​​at correction point 1008 represent the exposure conditions for that exposure value.

[0049] <Gain correction processing> The gain correction process in step S906 of Figure 9 will be explained using the diagram shown in Figure 11. In Figure 11, the horizontal axis represents exposure time and the vertical axis represents gain. Figure 11 shows the same variable range 801 for exposure time, variable range 802 for gain, line 803, line 806, region 804, and exposure condition 805 as in Figure 8. Figure 11 also shows the upper limit of gain 1107. The condition determination unit 105 refers to the diagram in Figure 11 and obtains the gain value corresponding to exposure condition 805, which is the intersection of line 803 for the same exposure value and line 806 that defines the exposure condition, and compares it with the upper limit of gain 1107. In the example in Figure 11, since the gain value corresponding to exposure condition 805 exceeds the upper limit of gain 1107, the condition determination unit 105 corrects the exposure condition to the correction point 1108, which is the intersection of line 803 for the same exposure value and the line for the upper limit of gain 1107.

[0050] Figure 10 shows the case where an upper limit of exposure time is set, and Figure 11 shows the case where an upper limit of gain is set. However, the case where both an upper limit of exposure time and an upper limit of gain are set will be explained using the diagram in Figure 12. Figure 12 explains the process when both an upper limit of exposure time and an upper limit of gain are set, and it is not possible to change the exposure conditions while keeping the same exposure value. In Figure 12, the horizontal axis represents exposure time, and the vertical axis represents gain. Figure 12 shows the same variable range of exposure time 801, variable range of gain 802, line 803, line 806, region 804, and exposure conditions 805 as in Figure 8. Figure 12 also shows the upper limit of exposure time 1007 and the upper limit of gain 1107.

[0051] The condition determination unit 105, referring to the diagram in Figure 12, obtains the exposure time corresponding to exposure condition 805, which is the intersection of the line 803 representing the same exposure value and the line 806 defining the exposure conditions, as explained in Figure 10, and compares it with the upper limit of exposure time 1007. In the example in Figure 12, since the exposure time corresponding to exposure condition 805 exceeds the upper limit of exposure time 1007, the condition determination unit 105 corrects the exposure conditions to the corrected exposure condition candidate 1209, which is the intersection of the line 803 representing the same exposure value and the line representing the upper limit of exposure time 1007.

[0052] Next, the condition determination unit 105 compares the candidate corrected exposure condition 1209 with the gain upper limit value 1107. In the example in Figure 12, correction is necessary because the candidate corrected exposure condition 1209 exceeds the gain upper limit value 1107. However, there are no exposure conditions on the line 803 with the same exposure value that do not exceed either the exposure time upper limit value 1007 or the gain upper limit value 1107. In this case, it is possible to satisfy the constraints of exposure time and gain by lowering the exposure value (updating the exposure value for each region calculated by the exposure value calculation unit 103). In this embodiment, the exposure condition 1211, which is the intersection of the line 1210 representing an exposure value one step lower than the line 803 with the same exposure value and the gain upper limit value 1107, is used as the final corrected exposure condition.

[0053] Figure 13 is a detailed flowchart of the process in steps S903 and S906 of Figure 9 when no exposure conditions satisfy the exposure time and gain constraints, as explained in the example of Figure 12. In step S1301, the condition determination unit 105 determines whether there is an exposure condition in the processing area that satisfies the constraints. That is, the condition determination unit 105 determines whether the candidate corrected exposure condition 1209 in Figure 12 satisfies the constraints of the upper limit of exposure time 1007 and the upper limit of gain 1107. If the constraints are met, the condition determination unit 105 moves the process to step S1302, where it changes the exposure condition according to the candidate corrected exposure condition 1209 and terminates the process. On the other hand, if the constraints are not met, the condition determination unit 105 moves the process to step S1303, where it lowers the exposure value by one stop, that is, changes the diagram used for the same exposure value from line 803 to line 1210, and then returns to the determination process in step S1301.

[0054] Furthermore, regarding the correction process when there are no exposure conditions that do not exceed the upper limit of exposure time and the upper limit of gain, the present invention is not limited to the process of lowering the exposure value as described above. For example, if there are no exposure conditions that do not exceed the upper limit of exposure time and the upper limit of gain, one of the correction processes may be omitted. Alternatively, the aperture value of the imaging processing unit 102 may be varied to omit one or both of the correction processes (to achieve the same exposure conditions).

[0055] As described above, according to this embodiment, by setting an upper limit on the exposure time, motion blur in the dynamic range can be suppressed and the visibility of the captured image can be improved. Furthermore, according to this embodiment, by setting an upper limit on the gain value, the effects of noise can be suppressed and the compression ratio of the captured image can be improved. Moreover, according to this embodiment, by adjusting the upper limits on the exposure time and gain, it is possible to achieve improved visibility and compression ratio tailored to the application. For example, in cases where images with different exposure levels in different areas are captured and combined to expand the dynamic range of the captured image, this embodiment determines the areas with movement and changes the exposure conditions for each area with movement according to the application. As a result, this embodiment makes it possible to suppress motion blur in moving subjects in dark areas and reduce noise in the captured image, in accordance with purposes such as prioritizing visibility or compression ratio.

[0056] <Second Embodiment> In the first embodiment, a method for correcting exposure conditions based on a diagram was described. In the second embodiment, a method for correcting exposure conditions by switching between tables that derive exposure conditions from exposure values, without using a diagram, according to the application, will be described.

[0057] Figure 14 shows an example of the configuration of the imaging device in the second embodiment. In the imaging device 101 of the second embodiment, the processing of the imaging processing unit 102, exposure value calculation unit 103, dynamic region determination unit 104, exposure correction unit 106, development processing unit 107, and image output unit 108 is the same as in the first embodiment (Figure 1), so their descriptions are omitted. The overall processing flow in the imaging device 101 of the second embodiment is the same as in Figure 2 of the first embodiment, so its description is omitted. In addition, the region-specific exposure value calculation process and the dynamic region determination process are the same as in the first embodiment, so their descriptions are omitted.

[0058] In the second embodiment, the condition determination unit 105 updates the settings of the imaging processing unit 102 by selecting a table that defines region-specific exposure conditions based on the exposure values ​​for each region obtained from the exposure value calculation unit 103 and the information regarding the presence or absence of movement for each region obtained from the movement region determination unit 104. In this embodiment, it is assumed that exposure condition table 1401, exposure condition table 1402, and exposure condition table 1403 are provided as tables that define region-specific exposure conditions. It is assumed that exposure condition table 1401 defines exposure condition A, which shows the relationship between exposure value and exposure condition, similarly, exposure condition table 1402 defines exposure condition B, and exposure condition table 1403 defines exposure condition C. The condition determination unit 105 selects one of exposure conditions A to C by selecting one of exposure condition tables 1401 to 1403.

[0059] The following describes the process for calculating exposure values ​​by region in the second embodiment. Figure 15 shows an example of an exposure condition table that lists combinations of exposure time and gain corresponding to exposure values. Figure 15 shows three types of exposure condition settings: exposure condition A (exposure condition table 1401), exposure condition B (exposure condition table 1402), and exposure condition C (exposure condition table 1403). In this embodiment, exposure condition A is a visibility-prioritizing setting for the moving region (the region where moving objects are present), and is an exposure condition in which the exposure time is shortened when the exposure value is small (dark). Exposure condition B is a compression-prioritizing setting for the moving region, and is an exposure condition in which the exposure time is shortened when the exposure value is small (dark), with the gain value limited to a maximum of 16 as an example. Exposure condition C is a setting for the stationary region. In this embodiment, the user pre-selects either the first exposure condition mode, which is the visibility-prioritizing mode, or the second exposure condition mode, which is the compression-prioritizing mode, as the exposure condition for the moving region. For example, if the visibility-prioritizing mode is selected, exposure condition A (exposure condition table 1401) is selected in the moving region. On the other hand, if the compression priority mode is selected, exposure condition B (exposure condition table 1402) is selected in the dynamic range.

[0060] Figure 16 is a flowchart showing the processing flow of the condition determination unit 105 in the second embodiment. Note that steps S701 to S704 and step S708 are the same as the processes with the same reference numerals in Figure 7 of the first embodiment, so their explanations are omitted. In the second embodiment, the condition determination unit 105 proceeds to step S705 if the exposure value is less than or equal to the threshold value as a result of the determination in step S704, and proceeds to step S1611 if the exposure value is not less than or equal to the threshold value. The processes in steps S705 and S706 are the same as the processes with the same reference numerals in Figure 7 of the first embodiment, so their explanation is omitted.

[0061] In the second embodiment, the condition determination unit 105 proceeds to the process of step S1607 if the result of the determination in step S706 belongs to the dynamic region, and proceeds to the process of step S1611 if it does not belong to the dynamic region. When the process moves to step S1607, the condition determination unit 105 checks the exposure condition mode of the dynamic region, which is set in advance. If the exposure condition mode is the visibility priority mode, the condition determination unit 105 moves the process to step S1609, and if the exposure condition mode is the compression priority mode, it moves the process to step S1610.

[0062] When the process moves to step S1609, the condition determination unit 105 selects exposure condition A (exposure condition table 1401), finds the exposure condition corresponding to the exposure value of the region by looking up the table, and then moves the process to step S708. When the process moves to step S1610, the condition determination unit 105 selects exposure condition B (exposure condition table 1402), finds the exposure condition corresponding to the exposure value of the region by referring to the table, and then moves the process to step S708. When the process moves to step S1611, the condition determination unit 105 selects exposure condition C (exposure condition table 1403), finds the exposure condition corresponding to the exposure value of the region by referring to the table, and then moves the process to step S708.

[0063] As explained above, according to the second embodiment, by applying exposure conditions with a shortened exposure time in visibility priority mode to the motion region, motion blur in the motion region can be suppressed and the visibility of the captured image can be improved. Furthermore, according to the second embodiment, by applying exposure conditions with a shortened exposure time without increasing the gain value too much in compression priority mode to the motion region, the effects of noise can be suppressed and the compression ratio of the captured image can be improved.

[0064] <Other Embodiments> In the above-described embodiment, dynamic area determination was performed on a processing area basis, and exposure condition determination processing was carried out, but this is not limited to this. For example, the processing may be applied only to important areas such as the center of the screen based on external settings or the results of image recognition processing. Alternatively, the exposure time, the upper limit of the gain adjustment range, and the exposure condition table may be changed for each area based on external settings or the results of image recognition processing, and the processing may be applied accordingly. Furthermore, in the above-described embodiment, motion region detection was performed by evaluating the brightness difference for each region from two captured images taken at different times, but this is not limited to this. For example, motion region detection may be performed using three or more captured images, or motion may be searched by taking the Sum of Absolute Difference (SAD) of pixels between captured images. Alternatively, motion region detection may also be performed by obtaining the optical flow of the captured images. Furthermore, while the first embodiment sets upper limits on the adjustment range of exposure time and gain as restrictions in the exposure condition correction process, it is not limited to this. Depending on the purpose, restrictions may be set on only one of the exposure time or gain value, or a lower limit may be added as a restriction.

[0065] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features. [Explanation of Symbols]

[0066] 101: Imaging device, 102: Imaging processing unit, 103: Exposure value calculation unit, 104: Motion area determination unit, 105: Condition determination unit, 106: Exposure correction unit, 107: Development processing unit, 108: Image output unit

Claims

1. An imaging device that divides a captured image into a plurality of regions and can capture images under different exposure conditions for each region, an acquisition means for acquiring an exposure value for each of the regions of the captured image; a detection means for detecting a moving area based on the captured image; a determining means for determining an exposure time and a gain based on the exposure value for each of the regions and the detection result of the moving region, The imaging device is characterized in that the determining means sets a limit on an adjustment range of at least one of the exposure time and the gain according to the detection result of the moving area.

2. 2. The imaging device according to claim 1, wherein the limit is an upper limit of an adjustment range of the exposure time.

3. 3. The imaging device according to claim 2, wherein the determining unit corrects the exposure time to be equal to or less than the upper limit value when the exposure time corresponding to the exposure value acquired by the acquiring unit exceeds the upper limit value of the exposure time.

4. 4. The imaging device according to claim 1, wherein the limit is an upper limit of an adjustment range of the gain.

5. 5. The imaging device according to claim 4, wherein, when the gain corresponding to the exposure value acquired by the acquisition means exceeds the upper limit value of the gain, the determination means corrects the gain to be equal to or less than the upper limit value.

6. 6. The imaging device according to claim 3, wherein the determining means performs the correction so as not to deviate from a diagram that represents the relationship between a combination of exposure time and gain corresponding to the same exposure value.

7. The imaging device according to claim 6, characterized in that, when correction to the upper limit value or less results in deviation from a diagram representing the relationship between combinations of exposure time and gain corresponding to the same exposure value, the determination means performs the correction on a diagram representing the relationship between combinations of exposure time and gain corresponding to the same exposure value that is one step lower than the same exposure value.

8. The determining means a first exposure condition mode and a second exposure condition mode; 2. The image pickup apparatus according to claim 1, wherein the restriction is imposed by varying the adjustment range of at least one of exposure time and gain between the first exposure condition mode and the second exposure condition mode.

9. the determining means includes an exposure condition table that defines a combination of an exposure time and a gain corresponding to an exposure value, which corresponds to each of the first exposure condition mode and the second exposure condition mode; 9. The imaging apparatus according to claim 8, wherein the exposure condition table is switched depending on the result of detection of the moving area.

10. 10. The imaging device according to claim 1, wherein the detecting means detects the presence or absence of motion for each region based on luminance information for each region.

11. A control method for an imaging device that divides a captured image into a plurality of regions and can capture images under different exposure conditions for each region, comprising: an exposure value acquisition step of acquiring an exposure value for each of the regions; a detection step of detecting a moving area based on the captured image; a determining step of determining an exposure time and a gain based on the exposure value for each of the regions and the detection result of the moving region, The method for controlling an imaging apparatus, wherein the determining step sets a limit on an adjustment range of at least one of the exposure time and the gain depending on the detection result of the moving area.

12. A program for causing a computer included in an imaging apparatus to function as each of the means included in the imaging apparatus according to any one of claims 1 to 10.

Images