Imaging device

Abstract

To provide an imaging device that suppresses deterioration in image quality due to difference (OB level difference) between an output of an OB region and that of an effective region.SOLUTION: When imaging sensitivity of an imaging element is changed in a third frame period between a first frame period and a second frame period, an imaging device corrects a first signal output from a first pixel using a second reference value obtained from a second signal output from a second pixel in a fourth frame period in each frame period from a fourth frame period after the third frame period to a fifth frame period when a second period longer than a frame period has elapsed. In a case where imaging sensitivity of the imaging element is not changed, in each frame period from a sixth frame period after the second frame period to a seventh frame period when the third period longer than the frame period has elapsed, a first signal output from the first pixel is corrected using a third reference value obtained from the second signal output from the second pixel in the sixth frame period.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] BACKGROUND ART Conventionally, imaging devices that correct noise components and the like based on outputs from pixels in the OB region have been known (for example, Patent Document 1). However, due to the difference (OB step) that occurs between the output of the OB region and the output of the effective region, the brightness of the image is not stable for each frame, resulting in a problem of degradation of image quality. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-200109 Summary of the Invention

[0004] According to a first aspect, an imaging device includes an imaging element having: first pixels arranged in a first region, the first pixels outputting a first signal in each frame period of a plurality of consecutive frame periods; and second pixels arranged in a second region outside the first region, the second pixels outputting a second signal; and a control unit configured to correct the first signal output from the first pixel using a first reference value obtained from the second signal output from the second pixel in each frame period from the first frame period to a second frame period when a first period longer than the first frame period has elapsed among the plurality of frame periods. When the imaging sensitivity of the imaging element is changed in a third frame period between the first frame period and the second frame period among the plurality of frame periods, the control unit Among the frame periods, during each frame period from a fourth frame period after the third frame period to a fifth frame period when a second period longer than the frame period has elapsed, the first signal output from the first pixel is corrected using a second reference value obtained from the second signal output from the second pixel during the fourth frame period, and if the imaging sensitivity of the imaging element is not changed between the first frame period and the second frame period, during each frame period from a sixth frame period after the second frame period to a seventh frame period when a third period longer than the frame period has elapsed, the first signal output from the first pixel is corrected using a third reference value obtained from the second signal output from the second pixel during the sixth frame period. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a block diagram schematically illustrating an example of a configuration of a main part of an imaging device according to an embodiment. [Figure 2] FIG. 2 is a diagram schematically illustrating the configuration of an imaging element. [Figure 3] FIG. 2 is a diagram schematically illustrating output values ​​of a signal from an imaging element. [Figure 4] 10 is a timing chart when capturing a moving image. [Figure 5] 10 is a flowchart illustrating an operation when capturing a moving image or a live view image. DETAILED DESCRIPTION OF THE INVENTION

[0006] An imaging device according to an embodiment will be described with reference to the drawings. 1 is a block diagram showing a schematic configuration of a main part of a camera 1, which is an example of an imaging device according to an embodiment. The camera 1 includes a photographing optical system (imaging optical system) 2, an image sensor 3, an imaging control unit 4, a memory 5, a display unit 6, and an operation unit 7. The photographing optical system 2 has a plurality of lenses, including a focus adjustment lens (focus lens), and an aperture stop, and forms a subject image on the image sensor 3. The photographing optical system 2 may be detachable from the camera 1.

[0007] The imaging element 3 is an imaging element such as a CMOS image sensor. The imaging element 3 receives a light beam that has passed through the photographing optical system 2, and captures an image of a subject formed by the photographing optical system 2. The imaging element 3 has a plurality of pixels, each having a photoelectric conversion unit, arranged two-dimensionally (in the row and column directions). The photoelectric conversion unit is composed of a photodiode (PD). The imaging element 3 photoelectrically converts the received light to generate a signal, and outputs the generated signal to the imaging control unit 4. Details of the imaging element 3 will be explained later.

[0008] The imaging element 3 is configured so that its imaging sensitivity (exposure sensitivity) can be changed, for example, in predetermined steps. The imaging sensitivity is a quantity corresponding to the detection sensitivity of the electric charge accumulated in the imaging element 3 or the amplification gain of an amplifier circuit (not shown). An operation to change the imaging sensitivity of the imaging element 3 (gain change operation) is performed by the user using the operation unit 7, which will be described later. Furthermore, the image sensor 3 operates in different drive modes when capturing still images, capturing moving images, and capturing live view images in which a through image (live view image) of a subject is displayed on the display unit 6. In this case, the image sensor 3 generates signals from different numbers of pixel rows during still image capture, moving image capture, and live view image capture. For example, during still image capture, the image sensor 3 generates signals from all pixel rows, while during moving image capture, the image sensor 3 generates signals for each predetermined pixel row. During live view image capture, the image sensor 3 generates images for each predetermined pixel row, using fewer pixel rows than during moving image capture.

[0009] The memory 5 is a storage medium such as a memory card. Image data, control programs, etc. are recorded in the memory 5. Writing of data to the memory 5 and reading of data from the memory 5 are controlled by the imaging control unit 4. The display unit 6 displays an image based on the image data, information related to shooting such as the shutter speed and aperture value, and a menu screen, etc. The operation unit 7 includes various setting switches such as a release button, a power switch, a switch for switching between various modes, a sensitivity setting switch for a gain change operation to change the imaging sensitivity described above, and a switch for switching the drive mode of the image sensor 3, and outputs signals based on the respective operations to the imaging control unit 4.

[0010] The imaging control unit 4 is configured with a processor such as a CPU, FPGA, or ASIC, and memories such as a ROM or RAM, and controls each unit of the camera 1 based on a control program. The imaging control unit 4 supplies a signal for controlling the imaging element 3 to the imaging element 3, thereby controlling the operation of the imaging element 3. When taking a still image, a moving image, or a live view image, the imaging control unit 4 drives the imaging element 3 in a drive mode, captures a subject image, and generates a signal.

[0011] The imaging control unit 4 performs various types of image processing on the signals generated by the imaging element 3 to generate image data (still image data, moving image data, live view image data). The imaging control unit 4 includes a setting unit 41, a correction unit 42, and a control unit 43. The setting unit 41 sets a reference value (optical black clamp value) based on signals generated by pixels in a light-shielded region of the imaging element 3 (described later). When continuously acquiring images, the setting unit 41 resets the reference value when predetermined shooting conditions are met. The correction unit 42 corrects the black level of signals generated by pixels in an effective region of the imaging element 3 (described later) based on the reference value set by the setting unit 41. When continuously acquiring images, the control unit 43 changes a predetermined period or the number of frames, which is a cycle for setting the reference value, based on the imaging sensitivity set in the imaging element 3. The setting unit 41, correction unit 42, and control unit 43 will be described in detail below. At least one of the setting unit 41, the correction unit 42, and the control unit 43 may be included in the imaging element 3.

[0012] Next, the imaging element 3 of this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram schematically illustrating the configuration of the imaging element 3, with Fig. 2(a) showing a plan view of the imaging element 3 and Figs. 2(b) and 2(c) showing schematic cross-sectional structures of pixels provided in the imaging element 3. As shown in Fig. 2(a), the imaging element 3 is provided with an effective area R1 formed in its center and an OB area (optical black area) R2 arranged around the effective area R1. A plurality of pixels are two-dimensionally arranged in each of the effective area R1 and the OB area R2.

[0013] 2(b) shows a schematic cross-sectional structure of pixels 30a arranged in the effective area R1. Each pixel 30a has a microlens 301 that passes light from the imaging optical system 2, and a photoelectric conversion unit 302 that photoelectrically converts the light that has passed through the microlens 301 to generate an electric charge. The signal generated by the pixel 30a included in the effective area R1 of the image sensor 3 is an image signal corresponding to the subject image used to form the image pattern.

[0014] FIG. 2(c) shows a schematic cross-sectional structure of a pixel 30b arranged in the OB region R2. The pixel 30b includes a microlens 301 and a photoelectric conversion unit 302 similar to those of the pixel 30a, and also includes a light-shielding unit 303 on the incident side of the microlens 301. The light-shielding unit 303 is made of, for example, an aluminum film. The light-shielding unit 303 may be provided for each pixel 30b, or for multiple pixels 30b. The light-shielding unit 303 may also be located below the microlens 301 (between the microlens 301 and the photoelectric conversion unit 302). As described above, since the pixel 30b includes the light-shielding unit 303, the OB region R2 is a region (light-shielding region) where light beams from a subject are not incident. The signal generated by the pixel 30b included in the OB region R2 is used to remove components resulting from dark current from the signal output from the pixel 30a included in the effective region R1. That is, the signals generated by the pixels 30b included in the OB region R2 are signals that are not used to form the image pattern and are unrelated to the subject image. 2(c) is not limited to the case where the pixels 30b are arranged in the OB region R2. For example, as shown in the cross-sectional view of FIG. 2(d), pixels 30c each having a microlens 301 and a light-shielding portion 303 but no photoelectric conversion portion may be arranged in the OB region R2. In this case, too, the light-shielding portion 303 may be provided for each of the pixels 30c, or the light-shielding portion 303 may be arranged below the microlens 301.

[0015] The setting unit 41 of the imaging control unit 4 sets a reference value based on signals generated by the pixels 30b included in the OB region R2 during shooting (still image shooting, video image shooting, live view image shooting). In this case, the setting unit 41 sets the reference value to the average value of the signals generated by the pixels 30b in the OB region R2. 3 is a diagram schematically showing the output value of the signal generated by the image sensor 3. In FIG. 3, the vertical axis represents the output value of the signal generated by the image sensor 3, the horizontal axis represents the horizontal (row) position of the pixels of the image sensor 3, and the output value of the signal generated by the image sensor 3, i.e., brightness, is represented by the solid line indicated by the symbol L1. In this case, the output offset component D corresponds to the output value (i.e., the reference value) of the signal generated by pixel 30b included in the OB region R2.

[0016] The correction unit 42 subtracts the reference value set by the setting unit 41 (i.e., the average value of the output values ​​of the signals generated by the pixels 30b included in the OB region R2) from the output value of the signals generated by the pixels 30a included in the effective region R1 of the image sensor 3. As a result, the correction unit 42 outputs a signal included in the signal range P in FIG. 3, i.e., an image signal from which noise due to dark current has been removed (OB clamping process). Note that the output offset component D, i.e., the influence of dark current, changes depending on factors such as temperature.

[0017] When predetermined shooting conditions are met when continuously acquiring images (i.e., when shooting moving images or live view images), the setting unit 41 resets the reference value set as described above. In other words, when predetermined shooting conditions are not met, the setting unit 41 does not reset the already set reference value, and the correction unit 42 corrects the image of the next frame using the already set reference value (i.e., the same value as the reference value used for the image of the immediately preceding frame).

[0018] The above-mentioned photographing condition is at least one of the following: a predetermined period of time has elapsed or a predetermined number of frames have been photographed; the imaging sensitivity has been changed; and the drive mode of the imaging element 3 has been changed. When a predetermined period T has elapsed since the setting unit 41 set the reference value as described above, the setting unit 41 sets a new reference value again from the signal values of the pixels 30b included in the OB region R2. In this case, the imaging control unit 4 calculates an elapsed time tp (= tr × i) by multiplying the number of frames i acquired at the frame rate tr, and determines that the predetermined period T has elapsed when the elapsed time tp exceeds the predetermined period T (T < tp = tr × i). Note that the imaging control unit 4 may have a timer (not shown), and this timer may measure the predetermined time T. Further, the predetermined period T may be a fixed value of about several seconds, for example, or a value that can be changed by the user. Alternatively, when an image of the number of frames corresponding to the predetermined time T (that is, the number of frames of T / tr or more) is captured, the setting unit 41 sets a new reference value. In this case, the number of frames may be a fixed value or a value that can be set by the user.

[0019] When the imaging sensitivity is changed, the setting unit 41 sets a new reference value again from the signals of the pixels 30b included in the OB region R2. Note that the control unit 43 changes the above-described predetermined period T (or the number of frames corresponding to the predetermined period T) based on the changed imaging sensitivity. In this case, when the imaging sensitivity is set to a high sensitivity, the control unit 43 sets the predetermined period T to be longer. When the drive mode of the imaging device 3 is changed, the setting unit 41 sets a new reference value again from the signals of the pixels 30b included in the OB region R2.

[0020] Next, the timing for setting the reference value during moving image shooting or live view image shooting will be described. FIG. 4 is a timing chart during moving image shooting. In FIG. 4, the horizontal axis represents time, and indicates that moving image shooting has started at time t0. When video capture begins, the imaging control unit 4 controls the image sensor 3 to generate a signal in order to generate an image for the first frame. The setting unit 41 sets a reference value based on the signal from the pixel 30b included in the OB region R2. The set reference value is stored in the memory 5. The correction unit 42 reads out the reference value stored in the memory 5 and corrects the signal from the pixel 30a included in the effective region R1 using the read reference value. The imaging control unit 4 performs various image processes on the signal corrected using the reference value to generate an image for the first frame. The generated image is stored in the memory 5 and displayed on the display unit 6.

[0021] Thereafter, images of the second and subsequent frames are generated in the same manner until the predetermined period T has elapsed. That is, for the second and subsequent frames, the correction unit 42 also corrects the signals generated by the pixels 30a in the effective area R1 using the reference values ​​stored in the memory 5 (i.e., the reference values ​​used when generating the image of the first frame).

[0022] When a predetermined period T has elapsed (time t1=t0+T), the setting unit 41 resets the reference value. In Fig. 4, time t1 is the timing for capturing the nth frame image. In this case, when generating the nth frame image, the correction unit 42 corrects the signals generated by the pixels 30a in the effective area R1 using the reference value used when generating the (n-1)th frame image (i.e., the reference value stored in memory 5).

[0023] At time t2, the setting unit 41 sets a new reference value using signals generated by pixels 30b in the OB region R2 of the image sensor 3 when generating the (n+1)th frame image. The setting unit 41 overwrites and stores the newly set reference value on the reference value already stored in the memory 5. When generating the (n+1)th frame image, the correction unit 42 corrects the signals generated by pixels 30a in the effective region R1 using the newly stored reference value in the memory 5.

[0024] Thereafter, from time t2 until a predetermined period T has elapsed, the correction unit 42 performs correction using the reference value set when generating the (n+1)th frame image. 4, an example has been given in which the reference value is reset when the predetermined period T has elapsed, but the setting unit 41 also resets the reference value in a similar manner when the imaging sensitivity or the drive mode of the image sensor 3 is changed. That is, if the imaging sensitivity or the drive mode of the image sensor 3 is changed while the nth frame image is being captured, the setting unit 41 sets the reference value using a signal from the image in the OB region R2 of the image sensor 3 when generating the (n+1)th frame image. Furthermore, although FIG. 4 shows a timing chart for capturing moving images, in the case of capturing live view images, the setting unit 41 sets a reference value and the correction unit 42 performs correction in the same manner as in the case of capturing moving images.

[0025] The operation of camera 1 in this embodiment when capturing moving images or live view images will be described with reference to the flowchart shown in Fig. 5. Each process in the flowchart shown in Fig. 5 is performed by executing a program in imaging control unit 4. This program is stored in memory, and is started and executed by imaging control unit 4.

[0026] In step S1, the imaging control unit 4 sets the value of the predetermined period T, and the process proceeds to step S2. The user can set the value of this predetermined period T by operating the operation unit 7. In step S2, the imaging control unit 4 adds 1 to the value of the frame counter i, and the process proceeds to step S3. In step S3, it is determined whether the value of the frame counter i is 1. If the value of the frame counter i is 1, a positive decision is made in step S3, and the process proceeds to step S4. If the value of the frame counter i is greater than 1, a negative decision is made in step S3, and the process proceeds to step S7, which will be described later.

[0027] In step S4, the imaging control unit 4 causes the imaging element 3 to capture an image and generate a signal, and the process proceeds to step S5. In step S5, the setting unit 41 sets a reference value using the signal generated by the pixel 30b in the OB region R2, stores the reference value in the memory 5, and the process proceeds to step S6. In step S6, the correction unit 42 corrects the signal generated by the pixel 30a in the effective region R1 using the reference value set in step S5, and the process proceeds to step S9, which will be described later.

[0028] In step S7, to which the process proceeds after a negative decision is made in step S3, the imaging control unit 4 causes the image sensor 3 to capture an image of the next frame and generate a signal, and the process proceeds to step S8. In step S8, the correction unit 42 corrects the signal from the pixel 30a in the effective area R1 using the reference value set in step S5, and the process proceeds to step S9. In step S9, the imaging control unit 4 calculates the elapsed time tp, and the process proceeds to step S10. In this case, the imaging control unit 4 calculates the elapsed time tp (= tr × i) by multiplying the frame rate tr by the number of frames acquired (i.e., the value of the frame counter i).

[0029] In step S10, the imaging control unit 4 determines whether or not the imaging sensitivity has been changed. If the imaging sensitivity has been changed, an affirmative decision is made in step S10, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to 0. If the imaging sensitivity has not been changed, an affirmative decision is made in step S10, and the process proceeds to step S11. In step S11, the imaging control unit 4 determines whether or not the drive mode of the image sensor 3 has been changed (for example, whether it has been switched from live view image shooting to video image shooting). If the drive mode of the image sensor 3 has been changed, an affirmative decision is made in step S11, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to 0. If the drive mode of the image sensor 3 has not been changed, an affirmative decision is made in step S11, and the process proceeds to step S12.

[0030] In step S12, the imaging control unit 4 determines whether the elapsed time tp calculated in step S9 has exceeded a predetermined period T. If the elapsed time tp has exceeded the predetermined time T, an affirmative decision is made in step S12, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to 0. If the elapsed time tp is equal to or less than the predetermined time T, a negative decision is made in step S12, and the process proceeds to step S13.

[0031] In step S13, it is determined whether or not to end video image capture or live view image capture. If video image capture or live view image capture is to be ended, for example, if the user instructs to capture a still image or turn off the power to camera 1, an affirmative decision is made in step S13 and the process ends. If video image capture or live view mode image capture is not to be ended, a negative decision is made in step S13 and the process returns to step S2.

[0032] If the photographing conditions for resetting the reference value do not include a change in the imaging sensitivity, the process of step S10 may be skipped. Also, if the photographing conditions for resetting the reference value do not include a change in the drive mode of the imaging element 3, the process of step S11 may be skipped. Furthermore, if the control unit 43 changes the predetermined period T based on the changed imaging sensitivity, when step S10 is judged to be positive, the control unit 43 sets a new predetermined period T and the processing returns to step S2. Also, if the passage of the predetermined period T is determined based on the number of frames captured, the processing of step S9 can be skipped, and if the value of the frame counter i in step S12 is equal to or greater than T / tr, the processing returns to step S2.

[0033] The above-described embodiment provides the following advantageous effects. (1) The image sensor 3 has pixels 30a included in the effective area R1 and pixels 30b included in the OB area R2 (light-shielded area). The setting unit 41 sets a reference value based on signals generated from the pixels 30b in the OB area R2, and the correction unit 42 corrects signals generated from the pixels 30a in the effective area R1 based on the reference value. The setting unit 41 then resets the reference value when predetermined shooting conditions are met when continuously acquiring images. This allows OB clamping processing to be performed using the same reference value when generating images for multiple frames during moving image capture or live view image capture. If there is a difference in output value (OB step) between the signal generated by the pixel in the OB area and the signal generated by the pixel in the effective area, the output value of the signal generated by the pixel in the effective area after OB clamping processing will be higher or lower than the expected value. If an OB step occurs for each frame when capturing a moving image or live view image, if a reference value is set for each frame image and OB clamping processing is performed, the brightness of the entire image will differ for each frame of the generated moving image or live view image, resulting in flickering. In this embodiment, the same reference value is set for multiple frame images and OB clamping processing is performed, thereby suppressing the occurrence of the above-mentioned flicker and reducing degradation of image quality compared to when a reference value is set for each frame image.

[0034] (2) The setting unit 41 resets the reference value after a predetermined period T has elapsed or after a predetermined number of frames have been captured. This makes it possible to suppress flickering from occurring between frames compared to when a reference value is set for each frame and clamping is performed, and also makes it possible to correct black levels that have fluctuated due to factors such as the temperature of the image sensor 3 during operation.

[0035] (3) When the imaging sensitivity is changed, the setting unit 41 resets the reference value. This eliminates the need to secure an area in the memory 5 for storing a reference value for each imaging sensitivity. Also, since it is no longer necessary to adjust the fixed reference value for each individual imaging element 3, the takt time during production can be shortened.

[0036] (4) The control unit 43 changes the predetermined period T or the predetermined number of frames based on the imaging sensitivity. As a result, for example, when a high imaging sensitivity is set, noise increases, but by updating the reference value at the predetermined period T according to the noise generation status, such as by setting the predetermined period T to a long value, it is possible to prevent the reference value from being frequently changed and flickering from occurring in moving images or live view images.

[0037] (5) When the drive mode of the image sensor 3 is changed, the setting unit 41 resets the reference value. This eliminates the need to secure an area in the memory 5 for storing a reference value for each drive mode. Also, since it is no longer necessary to adjust the fixed reference value for each individual image sensor 3, the takt time during production can be shortened.

[0038] Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments that are conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0039] 1... camera 3... imaging element 4... imaging control units 30a, 30b... pixels 41... setting unit 42... correction unit 43... control unit 303... light blocking unit

Claims

[Claim 1] an imaging element including: a first pixel arranged in a first region, the first pixel outputting a first signal in each frame period of a plurality of consecutive frame periods; and a second pixel arranged in a second region outside the first region, the second pixel outputting a second signal; a control unit that corrects the first signal output from the first pixel by using a first reference value obtained from the second signal output from the second pixel during the first frame period, during each frame period from a first frame period to a second frame period when a first period longer than the frame period has elapsed, among the plurality of frame periods; Equipped with the control unit, when the imaging sensitivity of the imaging element is changed in a third frame period between the first frame period and the second frame period, among the plurality of frame periods, corrects the first signal output from the first pixel using a second reference value obtained from the second signal output from the second pixel in each frame period from a fourth frame period after the third frame period to a fifth frame period when a second period longer than the frame period has elapsed; and when the imaging sensitivity of the imaging element is not changed from the first frame period to the second frame period, corrects the first signal output from the first pixel using a third reference value obtained from the second signal output from the second pixel in each frame period from a sixth frame period after the second frame period to a seventh frame period when a third period longer than the frame period has elapsed. Imaging device.

Images