Imaging element and imaging device

Abstract

To provide an imaging element and an imaging device that improve an SN ratio.SOLUTION: There is provided an imaging element P which comprises a plurality of pixels each having a photoelectric conversion part (photodiode PD) for converting incident light into an electric charge and a holding part (pixel memory MEM) for holding the electric charge generated through the conversion of the photoelectric conversion part, and a pixel array consisting of the plurality of pixels is used to generate an image, and has a pixel line for image where a plurality of first pixels outputting a first signal based upon the electric charge held at the holding part are arranged, and a pixel line for detection where a plurality of second pixels outputting a second signal for removing noise included in the first signal and generated at the holding part of a first pixel are arranged.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an image sensor and an imaging device.

Background Art

[0002] An image sensor including a charge holding unit that holds charges generated in a photoelectric conversion unit is known (for example, Patent Document 1). Conventionally, improvement in the signal-to-noise ratio has been demanded.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] An image sensor according to an aspect of the present invention is an image sensor including a plurality of pixels each having a photoelectric conversion unit that converts incident light into charges and a holding unit that holds the charges converted by the photoelectric conversion unit, wherein the plurality of pixels include a plurality of first pixels that are used for generating an image and output a first signal based on the charges held in the holding unit, and a plurality of second pixels that output a second signal for removing noise generated in the holding unit of the first pixels included in the first signal.

[0005] An imaging device according to an aspect of the present invention includes the image sensor according to the above aspect and an image processing unit that removes the noise included in a first image generated using the first signal based on the second signal.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0007] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0008] FIG. 1 is a block diagram showing a schematic configuration of an imaging device according to an embodiment. The imaging device 1 includes, for example, a lens unit L, an imaging element 10, an image processing engine 20, and a release button R. The imaging device 1 is, for example, a digital camera such as a mirrorless camera. The imaging device 1 generates an image (moving image or still image) obtained by imaging the light of a subject (subject light) incident on the imaging element 10 via the lens unit L in response to an operation of the release button R by a user of the imaging device 1.

[0009] The release button R is an operation unit for a user of the imaging device 1 to instruct the shooting of a still image. The release button R is configured to give shooting instructions in two steps according to the degree of depression. In the following description, the operation of pressing the release button R to the first step is referred to as "half-press", and the operation of pressing it to the second step is referred to as "full-press". When the release button R is half-pressed, the imaging device 1 determines shooting conditions for shooting a still image, and when the release button R is fully pressed, shooting is performed under the determined shooting conditions to generate a still image.

[0010] The lens unit L forms a subject image on the image sensor 10 by condensing and transmitting subject light. The lens unit L includes, for example, a zoom mechanism for changing the focal length, a focus mechanism for adjusting the focus position (focus), a diaphragm mechanism for adjusting the amount of subject light to be transmitted, and the like. The operation of each mechanism included in the lens unit L is controlled by the image processing engine 20. The lens unit L may be configured to be easily detachable from the imaging device 1.

[0011] The image sensor 10 exposes the subject light incident through the lens unit L in accordance with the control from the image processing engine 20, and outputs a pixel signal representing the amount of the exposed subject light to the image processing engine 20. The image sensor 10 includes, for example, a pixel array 11, a scanning circuit 12, an analog-to-digital converter (ADC) 13, and an output unit 14. The image sensor 10 is an image sensor that realizes a so-called global electronic shutter function that realizes the simultaneity of photoelectric conversion by exposing all the pixels arranged in the pixel array 11 at the same timing.

[0012] The pixel array 11 includes a plurality of pixels each having a photoelectric conversion section that photoelectrically converts incident subject light into an electrical signal, arranged in a two-dimensional matrix. Outside the end of the effective pixel region where pixels for exposing subject light are arranged in the pixel array 11, there is provided a region where incident subject light is blocked, i.e., a so-called Optical Black (OB) region. The OB region may be an area equivalent to one row of pixels arranged in the effective pixel region, or may be an area equivalent to a plurality of rows. In the pixel array 11, the pixels arranged within the effective pixel region and the pixels arranged within the OB region have the same configuration. However, while the pixel signal of the pixel arranged within the effective pixel region represents the amount of light of the exposed subject light, the pixel signal of the pixel arranged within the OB region represents the magnitude of the weak dark current flowing in the blocked state. More specifically, the pixel signal of the pixel arranged within the OB region represents the magnitude of the weak dark current flowing during the period when the pixel arranged within the effective pixel region is exposing the subject light. The dark current is caused by, for example, the temperature rise of the imaging device 10, and for the imaging device 10, it is a factor of noise included in the pixel signal representing the amount of light of the exposed subject light, that is, a factor causing a deterioration in the image quality of the generated image (moving image or still image). The pixel signal of the pixel arranged within the OB region is used for processing to remove (reduce) the dark current noise in the image generated by the imaging device 1. Each pixel arranged in the pixel array 11 is driven by the scanning circuit 12 and outputs an electrical signal (analog electrical signal) that has photoelectrically converted the subject light or represents the magnitude of the dark current to the ADC 13.

[0013] The scanning circuit 12 generates drive signals for driving each pixel arranged in the pixel array 11 according to the control signals output by the image processing engine 20. The scanning circuit 12 generates drive signals for realizing the global electronic shutter function in the imaging device 10. The scanning circuit 12 outputs the generated drive signals to each pixel arranged in the pixel array 11. The scanning circuit 12 includes, for example, a vertical scanning circuit and a horizontal scanning circuit. The vertical scanning circuit generates vertical drive signals for driving each pixel arranged in the pixel array 11 in units of rows (pixel rows), and sequentially outputs the analog electrical signals obtained by photoelectrically converting the subject light to the corresponding vertical signal lines for each pixel row. The horizontal scanning circuit generates horizontal drive signals for driving each column (pixel column) of pixels arranged in the pixel array 11, and sequentially outputs the analog electrical signals of each pixel row output to the corresponding vertical signal lines according to the vertical drive signals, that is, sequentially for each pixel column, that is, sequentially for each pixel, to the ADC 13.

[0014] The ADC 13 converts the analog electrical signals sequentially output from each pixel arranged in the pixel array 11 into digital electrical signals. The ADC 13 sequentially outputs the converted digital electrical signals to the output unit 14.

[0015] The output unit 14 outputs the digital electrical signals output by the ADC 13 to the image processing engine 20 as pixel signals captured by the imaging device 10. The output unit 14 includes, for example, an amplifier circuit.

[0016] The configuration of the imaging device 10 is not limited to the configuration shown in FIG. 1. That is, the configuration of the imaging device 10 shown in FIG. 1 is merely an example, and some of the components may be omitted, or further other components may be added. For example, the imaging device 10 shown in FIG. 1 may further include a noise suppression processing circuit such as a correlated double sampling (CDS) circuit that suppresses noise included in an electrical signal. The CDS circuit may be arranged, for example, at a position between the pixel array 11 and the ADC 13. In this case, each analog electrical signal output by the pixel array 11 is output to the ADC 13 via the CDS circuit, and the ADC 13 converts the analog electrical signal that has been subjected to noise suppression processing by the CDS circuit into a digital electrical signal and outputs it to the output unit 14. The CDS circuit may be arranged, for example, at a position between the ADC 13 and the output unit 14. In this case, each digital electrical signal output by the ADC 13 is output to the output unit 14 via the CDS circuit, and the output unit 14 outputs the digital electrical signal that has been subjected to noise suppression processing by the CDS circuit as a pixel signal to the image processing engine 20.

[0017] <Configuration of the imaging device> Here, an example of the configuration of the pixel arranged in the pixel array 11 will be described. FIG. 2 is a circuit diagram showing an example of the schematic configuration of the pixel arranged in the pixel array 11 included in the imaging device 10 of the embodiment. FIG. 2 shows the configuration of one pixel P arranged in the pixel array 11.

[0018] The pixel P outputs an electrical signal that has photoelectrically converted the incident subject light or represents the magnitude of the dark current to the vertical signal line V. The pixel P includes, for example, a photodiode PD, a pixel memory MEM, a floating diffusion FD, a first transfer transistor T1, a PD reset transistor T2, a second transfer transistor T3, an FD reset transistor T4, an amplification transistor T5, and a selection transistor T6.

[0019] The photodiode PD is a photoelectric conversion element that converts incident light into electricity and generates and accumulates electric charges. The photodiode PD is an example of a "photoelectric conversion section".

[0020] The pixel memory MEM is a capacitor that temporarily holds and accumulates the electric charges accumulated by the photodiode PD. The pixel memory MEM is, for example, a capacitor formed on the silicon substrate of the imaging device 10. The pixel memory MEM is an example of a "holding section".

[0021] The floating diffusion FD is a capacitor that temporarily holds and accumulates the electric charges accumulated in the pixel memory MEM. The floating diffusion FD is a capacitor associated with the node connected to the gate terminal of the amplification transistor T5. The floating diffusion FD, together with the pixel memory MEM, is also an example of a "holding section".

[0022] The first transfer transistor T1 transfers the electric charges accumulated by the photodiode PD to the pixel memory MEM in response to the first transfer drive signal TG1 output by the scanning circuit 12. The electric charges transferred by the first transfer transistor T1 are accumulated in the pixel memory MEM.

[0023] The PD reset transistor T2 sets the electric charges accumulated by the photodiode PD to the level of the power supply potential VDD in response to the first reset drive signal RST1 output by the scanning circuit 12. That is, the PD reset transistor T2 discharges (resets) the electric charges accumulated by the photodiode PD to the power supply potential VDD side.

[0024] The second transfer transistor T3 transfers the electric charges accumulated in the pixel memory MEM to the floating diffusion FD in response to the second transfer drive signal TG2 output by the scanning circuit 12. The electric charges transferred by the second transfer transistor T3 are accumulated in the floating diffusion FD.

[0025] The FD reset transistor T4 sets the charge accumulated in the floating diffusion FD to the level of the power supply potential VDD in response to the second reset drive signal RST2 output by the scanning circuit 12. That is, the PD reset transistor T2 discharges (resets) the charge accumulated in the floating diffusion FD to the power supply potential VDD side.

[0026] The amplification transistor T5 outputs a voltage corresponding to the charge accumulated in the floating diffusion FD.

[0027] The selection transistor T6 outputs, in response to the selection drive signal SEL output by the scanning circuit 12, the voltage output by the amplification transistor T5 as an electrical signal representing the amount of subject light incident on the pixel P or the magnitude of the dark current to the vertical signal line V. The electrical signal output to the vertical signal line V is output to the ADC 13.

[0028] With such a configuration, in the image sensor 10 of the embodiment, in response to control from the image processing engine 20, pixel signals corresponding to the respective pixels P arranged in the pixel array 11 are output. Then, the image processing engine 20 performs processing (hereinafter referred to as "dark current noise reduction processing") for reducing the dark current noise of an image (moving image or still image) generated based on the pixel signals from the respective pixels P arranged in the effective pixel region based on the pixel signals representing the magnitudes of the dark currents from the respective pixels P arranged in the OB region.

[0029] The configuration of the pixel P arranged in the pixel array 11 is not limited to the configuration shown in FIG. 2. For example, a configuration in which the pixel P (which may be a plurality of pixels) arranged in another pixel row in the pixel array 11 shares the floating diffusion FD may be used.

[0030] Returning to FIG. 1, in response to an instruction to take a picture (pressing of the release button R) by the user of the imaging device 1, the image processing engine 20 generates a still image based on the pixel signals output by the imaging element 10. The image processing engine 20 causes the generated still image to be recorded on a storage medium such as, for example, a memory provided in the imaging device 1 or a memory card that can be easily attached to and detached from the imaging device 1. Until the still image is generated, the image processing engine 20 also generates a moving image for the user to confirm the subject to be photographed, that is, a so-called live view image. The live view image is an image with fewer pixels than the still image. At this time, when reading the pixel signals from the imaging element 10, the image processing engine 20 reduces the pixel rows and performs reduction of the pixel columns by image processing when generating the live view image. For example, when generating a live view image with 1 / 3 the number of pixels of the still image, the image processing engine 20 reduces the pixel signals of the pixel rows corresponding to three rows of the pixel array 11 to the pixel signals of one row of the live view image. More specifically, the image processing engine 20 groups the pixel rows corresponding to three rows of the pixel array 11 as a set, and among the set of pixel rows, does not use the pixel signals of 1 / 3 of the pixel rows for generating the live view image, and adds (pixel addition) the pixel signals of the remaining 2 / 3 of the pixel rows to reduce them to one row of the live view image. Thereafter, the image processing engine 20 reduces the pixel columns of the live view image, for example, by thinning out the image data of each generated pixel row. The image processing engine 20 causes the generated live view image to be displayed on a display device (not shown) provided in the imaging device 1, for example. The image processing engine 20 includes, for example, a control unit 21, an input unit 22, an image processing unit 23, and a memory 24.

[0031] The image processing engine 20, the control unit 21, the input unit 22, and the image processing unit 23 included in the image processing engine 20 each realize the following functions, for example, when a hardware processor executes a program (software). The hardware processor means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a large-scale integration circuit (LSI), an application-specific integrated circuit (ASIC), a programmable logic device (for example, a simple programmable logic device (SPLD) or a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), etc. (including hardware (circuitry)). Some or all of the functions of the image processing engine 20 and each component included in the image processing engine 20 may be realized by a dedicated LSI. The program may be stored in a semiconductor memory element such as a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc., or a storage device (a storage device having a non-transitory storage medium) such as a hard disk drive (HDD) provided in the imaging device 1 in advance, or may be directly incorporated into the circuit of the hardware processor. The hardware processor realizes each function by reading and executing the stored program.

[0032] The control unit 21 controls the shooting operation in the imaging device 1. When the control unit 21 receives an instruction to shoot a still image by the user pressing the release button R, it outputs a control signal for causing an exposure corresponding to the received instruction to shoot a still image to the imaging element 10 (more specifically, the scanning circuit 12). The control unit 21 controls the generation of a moving image (live view image) and a still image based on the pixel signals output by the imaging element 10.

[0033] The input unit 22 acquires the pixel signals output by the imaging element 10 in accordance with the control from the control unit 21. The input unit 22 outputs the acquired pixel signals to the image processing unit 23.

[0034] The image processing unit 23 performs various predetermined image processes on the pixel signals output by the input unit 22. The various image processes include, for example, auto focus (AF) control for controlling the focus mechanism provided in the lens unit L to focus on a subject, auto exposure (AE) control for determining an optimal combination of the shutter speed and aperture at the time of shooting, auto white balance (AWB) processing for optimizing the color tone of an image, defective pixel correction processing for correcting pixel signals output by defective pixels included in the pixels arranged in the pixel array 11, dark current noise reduction processing, compression processing for compressing the data of the generated image, and the like. The image processing unit 23 stores the generated image (including the image subjected to the compression process) in the memory 24. The image processing unit 23 may use the memory 24 as a temporary storage means for temporarily storing the image data and processing data generated in the process of image processing on the pixel signals output by the input unit 22.

[0035] In the following description, attention will be paid to and described regarding the dark current noise reduction process performed based on the pixel signals output by each pixel P arranged in the pixel array 11 provided in the imaging element 10. The image processing unit 23 includes, for example, a noise reduction processing unit 232.

[0036] The noise reduction processing unit 232 performs dark current noise reduction processing on the entire captured image based on the pixel signals output by the input unit 22. More specifically, the noise reduction processing unit 232 subtracts the value of the black level represented by the pixel signal of each pixel P arranged in the OB region from the value of the level of the incident subject light represented by the pixel signal of each pixel P arranged in the effective pixel region, thereby removing the component of dark current included in the pixel signal of the level of the amount of subject light. As a result, the image processing unit 23 can continue to perform predetermined image processing on the pixel signals with the dark current noise reduced by the noise reduction processing unit 232 to generate an image with reduced dark current noise. The noise reduction processing unit 232 (which may include the image processing unit 23) is also an example of an "image processing unit".

[0037] The memory 24 stores the data of the final image (moving image or still image) generated by the image processing unit 23, the image data and processing data of the image processing process in the image processing unit 23. The memory 24 is, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk drive HDD. The memory 24 may be a memory card.

[0038] Incidentally, the image sensor 10 is an image sensor that realizes a global electronic shutter function. Therefore, in the image sensor 10, all the pixels arranged in the pixel array 11 can start exposure at the same timing and end exposure at the same timing. However, even in the image sensor 10 that realizes the global electronic shutter function, the output of the pixel signals from the respective pixels is sequentially performed for each pixel row. For this reason, in the image sensor 10, the timing for outputting the pixel signals is different for each pixel row of the pixel array 11. That is, in the image sensor 10, the time during which the charge accumulated by the photodiode PD provided in each pixel P is held and accumulated in the pixel memory MEM provided in each pixel P is different between the first pixel row and the last pixel row that output an electrical signal to the vertical signal line V. More specifically, the time during which the pixel memory MEM arranged in the pixel row that finally outputs an electrical signal to the vertical signal line V holds and accumulates the charge is longer than the time during which the pixel memory MEM arranged in the pixel row that first outputs an electrical signal to the vertical signal line V holds and accumulates the charge. And the dark current flows even when the pixel memory MEM holds and accumulates the charge. Moreover, the charge based on the dark current increases (rises) in proportion to the time during which the pixel memory MEM holds and accumulates the charge, even if the temperature of the image sensor 10 is constant. And in the dark current noise reduction process based on the pixel signal of the pixel P arranged in the OB region, although the component of the dark current flowing during the period when the pixel P arranged in the effective pixel region is exposed to the subject light can be removed, the component of the dark current flowing during the period when the pixel memory MEM holds and accumulates the charge cannot be removed. Therefore, when generating the live view image, the image processing engine 20 uses each pixel P of the pixel rows not used for generating the live view image to obtain a pixel signal representing the magnitude of the dark current flowing during the period when the pixel memory MEM holds and accumulates the charge, and based on the obtained pixel signal, performs a dark current noise reduction process for removing the component of the dark current flowing during the period when the pixel memory MEM holds and accumulates the charge.

[0039] <An example of the operation sequence> Here, an example of the shooting operation of the live view image in the imaging device 1 (hereinafter simply referred to as the "shooting operation") will be described. FIG. 3 is a sequence diagram showing an outline of the driving sequence of the imaging element 10 in the shooting operation of the imaging device 1 of the embodiment. FIG. 3 shows an example of the driving sequence of the imaging element 10 when performing the dark current noise reduction process of the live view image when generating the live view image in the imaging device 1. In FIG. 3, the horizontal axis represents time, and the vertical axis represents the pixel rows of the pixel array 11 to be driven. FIG. 4 is a timing chart showing an example of the driving timing of the pixel P in the pixel array 11 provided in the imaging element 10 in the shooting operation of the imaging device 1 of the embodiment. FIG. 4 shows the driving timing of the pixel P (hereinafter referred to as "pixel Pi") for image use in any pixel row (hereinafter referred to as the "image pixel row") in which the pixel P used for generating the live view image is arranged in the pixel array 11, and the driving timing of the pixel P (hereinafter referred to as "pixel Pc") in any pixel row (hereinafter referred to as the "detection pixel row") in which the pixel P used for detecting the dark current instead of generating the live view image is arranged. In the following description, the driving sequence of the imaging element 10 shown in FIG. 3 will be described with appropriate reference to the timing chart shown in FIG. 4. Pixel Pi is an example of the "first pixel", and pixel Pc is an example of the "second pixel".

[0040] In FIG. 3, sequence 101 indicates a reset operation for resetting the pixel P arranged in the pixel array 11. Sequence 102 indicates a transfer operation for transferring the charge accumulated in the photodiode PD to the pixel memory MEM in the pixel P arranged in the pixel array 11. Sequence 103 indicates a read operation for sequentially outputting (reading) pixel signals from the pixel Pc in the detection pixel row for each pixel row. Sequence 104 indicates a read operation for sequentially outputting (reading) pixel signals from the pixel Pi in the image pixel row for each pixel row.

[0041] In the shooting operation of the imaging device 1, when the imaging device 1 is activated, the image processing engine 20 starts generating a live view image.

[0042] When starting to generate a live view image in the shooting operation of the imaging device 1, first, at time t1, the control unit 21 causes all the pixels arranged in the imaging element 10 to perform the reset operation of sequence 101 simultaneously. In sequence 101, the control unit 21 simultaneously resets the charges accumulated in the photodiode PD, pixel memory MEM, and floating diffusion FD provided in all the pixels P arranged in the pixel array 11. As a result, the scanning circuit 12 drives the pixels in the reset operation of sequence 101 according to the control from the control unit 21.

[0043] More specifically, as shown in FIG. 4, at time t11, the scanning circuit 12 sets the first reset drive signal RST1, the first transfer drive signal TG1, the second reset drive signal RST2, and the second transfer drive signal TG2 of all the pixels P to the "High" level. As a result, in all the pixels P, the PD reset transistor T2, the first transfer transistor T1, the FD reset transistor T4, and the second transfer transistor T3 are simultaneously turned on, and the charges accumulated in the photodiode PD, pixel memory MEM, and floating diffusion FD are reset.

[0044] Thereafter, as shown in FIG. 4, at time t12, the scanning circuit 12 sets the first reset drive signal RST1-I, the first transfer drive signal TG1-I, the second reset drive signal RST2-I, and the second transfer drive signal TG2-I of all the pixels Pi to the "Low" level. As a result, in all the pixels Pi, the PD reset transistor T2, the first transfer transistor T1, the FD reset transistor T4, and the second transfer transistor T3 are simultaneously turned off, and the reset of the photodiode PD, the pixel memory MEM, and the floating diffusion FD provided in all the pixels Pi is completed. Thereby, the pixels Pi of all the image pixels arranged in the pixel array 11 start to expose the incident subject light (generation and accumulation of charges obtained by photoelectric conversion). On the other hand, as shown in FIG. 4, for all the pixels Pc, the scanning circuit 12 sets only the first transfer drive signal TG1-C, the second reset drive signal RST2-C, and the second transfer drive signal TG2-C to the "Low" level at time t12. That is, the scanning circuit 12 maintains the first reset drive signal RST1-C at the "High" level. As a result, in all the pixels Pc, the first transfer transistor T1, the FD reset transistor T4, and the second transfer transistor T3 are simultaneously turned off, and the reset of the pixel memory MEM and the floating diffusion FD provided in all the pixels Pc is completed, but the PD reset transistor T2 remains on, and the reset of the photodiode PD continues. Thereby, the pixels Pc of all the detection pixel rows arranged in the pixel array 11 do not perform the exposure of the incident subject light (generation and accumulation of charges obtained by photoelectric conversion). However, in the pixel Pc, after the reset is completed, charges based on the dark current flowing during the exposure period EX are accumulated in the pixel memory MEM and the floating diffusion FD. This is the same for the pixel Pi. In FIGS. 3 and 4, the exposure period EX represents one exposure period.

[0045] Subsequently, at time t2 when a predetermined exposure period EX has elapsed, the control unit 21 causes the transfer operation of sequence 102 to be performed. In the transfer operation of sequence 102, the control unit 21 simultaneously resets the pixel memory MEM and the floating diffusion FD for all the pixels Pi of the image pixel rows, and then transfers the charge accumulated by the photoelectric conversion of the photodiode PD to the pixel memory MEM. On the other hand, in the transfer operation of sequence 102, the control unit 21 only causes the pixel memory MEM and the floating diffusion FD to be reset again simultaneously for all the pixels Pc of the detection pixel rows. As a result, the scanning circuit 12 drives the pixels in the transfer operation of sequence 102 in accordance with the control from the control unit 21.

[0046] More specifically, as shown in FIG. 4, at time t21, the scanning circuit 12 sets the second reset drive signal RST2 and the second transfer drive signal TG2 of all the pixels P to the "High" level. As a result, in all the pixels P, the FD reset transistor T4 and the second transfer transistor T3 are simultaneously turned on, and the charges accumulated in the pixel memory MEM and the floating diffusion FD (i.e., the charges based on the dark current flowing during the exposure period EX) are reset. Thereafter, as shown in FIG. 4, at time t22, the scanning circuit 12 sets the second reset drive signal RST2 and the second transfer drive signal TG2 of all the pixels P to the "Low" level. As a result, in all the pixels P, the FD reset transistor T4 and the second transfer transistor T3 are simultaneously turned off, and the reset of the pixel memory MEM and the floating diffusion FD provided in all the pixels P is completed. Further, as shown in FIG. 4, at time t22, the scanning circuit 12 sets the first reset drive signal RST1-C of all the pixels Pc to the "Low" level. As a result, in all the pixels Pc, the PD reset transistor T2 that has been maintaining the on state is turned off, and the reset of the continuously operating photodiode PD ends at time t22. When the reset of the photodiode PD, the pixel memory MEM, and the floating diffusion FD ends at time t22, in all the pixels P, the charges based on the dark current flowing after the reset ends are accumulated in the pixel memory MEM and the floating diffusion FD (including the photodiode PD).

[0047] Thereafter, as shown in FIG. 4, at time t23, the scanning circuit 12 sets the first transfer drive signal TG1-I of all the pixels Pi to the "High" level. As a result, in all the pixels Pi, the first transfer transistor T1 is simultaneously turned on, and the charges accumulated in the photodiode PD are transferred to and accumulated in the pixel memory MEM. Thereafter, the scanning circuit 12 ends the transfer of charges from the photodiode PD to the pixel memory MEM in all the pixels Pi by setting the first transfer drive signal TG1-I of all the pixels Pi to the "Low" level.

[0048] Subsequently, when it is time to output (read) the pixel signal, the control unit 21 first causes the read operation of the pixel signal of the pixel Pc in the sequence 103 to be performed. The control unit 21 may start the read operation of the sequence 103 at any timing as long as the transfer operation of the sequence 102 has ended. For example, the control unit 21 may start the read operation of the sequence 103 immediately after the transfer operation of the sequence 102 has ended. FIG. 3 shows a case where the read operation of the sequence 103 is started at time t3. In the read operation of the sequence 103, the control unit 21 sequentially outputs (reads) the pixel signals for each detection pixel row of the pixel array 11. As a result, the scanning circuit 12 drives the pixels in the read operation of the pixel signal of the pixel Pc in the sequence 103 according to the control from the control unit 21.

[0049] More specifically, as shown in FIG. 4, at time t31, the scanning circuit 12 sets the selection drive signal SEL-C of the detection pixel row for outputting the pixel signal to the "High" level. As a result, the selection transistor T6 of the detection pixel row for outputting the pixel signal is turned on, and the selection transistor T6 outputs the electrical signal of the voltage output by the amplification transistor T5 to the vertical signal line V.

[0050] Subsequently, at time t32, the scanning circuit 12 sets the second reset drive signal RST2-C to the "High" level. As a result, the FD reset transistor T4 is turned on, and the charge accumulated in the floating diffusion FD is (again) reset. From the amplification transistor T5, an electrical signal of a voltage corresponding to the reset level charge of the floating diffusion FD is output and output to the vertical signal line V via the selection transistor T6.

[0051] When the imaging device 10 includes a CDS circuit and the CDS circuit is disposed at a position between the pixel array 11 and the ADC 13, the CDS circuit may sample and hold (clamp) the electrical signal of the reset level of the floating diffusion FD output to each vertical signal line V.

[0052] Thereafter, the scanning circuit 12 sets the second reset drive signal RST2-C to the "Low" level to release the reset of each floating diffusion FD and terminate the output of the electrical signal of the reset level to the vertical signal line V.

[0053] Subsequently, at time t33, the scanning circuit 12 sets the second transfer drive signal TG2-C to the "High" level. As a result, the second transfer transistor T3 is turned on, and the charge stored in the pixel memory MEM is transferred to and stored in the floating diffusion FD. Here, the charge transferred to the floating diffusion FD is the charge based on the dark current stored in the pixel memory MEM after the reset in the transfer operation of sequence 102 is completed. Therefore, an electrical signal of a voltage at a signal level representing the magnitude of the dark current stored in the pixel memory MEM transferred to the floating diffusion FD is output from the amplification transistor T5 and output to the vertical signal line V via the selection transistor T6.

[0054] When the imaging device 10 includes a CDS circuit and the CDS circuit is disposed at a position between the pixel array 11 and the ADC 13, the CDS circuit may perform noise suppression processing by sampling the electrical signal of the signal level representing the magnitude of the dark current stored in the pixel memory MEM output to each vertical signal line V and taking the difference from the electrical signal of the reset level of the floating diffusion FD being held (clamped).

[0055] Thereafter, the scanning circuit 12 sets the second transfer drive signal TG2-C to the "Low" level to terminate the transfer of the charge of the pixel memory MEM to each floating diffusion FD and terminate the output of the electrical signal of the signal level.

[0056] Subsequently, the scanning circuit 12 ends the state in which the selection drive signal SEL-C is set to the "Low" level and the electrical signal of the voltage output by the amplification transistor T5 by the selection transistor T6 is output to the vertical signal line V. Then, the scanning circuit 12 sequentially outputs the electrical signals of the current pixel row output to each of the vertical signal lines V for each pixel (for each pixel column).

[0057] In this way, the control unit 21 causes the readout operation of the pixel signal of the pixel Pc in the sequence 103 to sequentially output the pixel signals from the detection pixel rows. Then, when the output (readout) of the pixel signals from all the detection pixel rows is completed, the control unit 21 ends the readout operation of the sequence 103. FIG. 3 shows the case where, at time t4, the output of the pixel signals from all the detection pixel rows is completed (the readout operation of the sequence 103 is ended). In FIG. 3, the period during which the pixel signals are sequentially output from the detection pixel rows is defined as the detection pixel signal readout period RdC. In FIG. 3, at time t4, based on the pixel signals of the pixel Pc (hereinafter referred to as "detection pixel signals") read from each detection pixel row during the detection pixel signal readout period RdC, a frame Fc of the dark current image for performing the dark current noise reduction process of the live view image is generated. The image data corresponding to each pixel Pc constituting the generated frame Fc represents the magnitude of the dark current accumulated in the pixel memory MEM provided in the pixel Pc arranged in each detection pixel row during the period from time t22 to time t33. Since the period from time t22 to time t33 is different for each detection pixel row, the magnitude of the dark current represented by each image data is different for each detection pixel row. Therefore, the noise reduction processing unit 232 can obtain the tendency of the change in the dark current in the entire region of the pixel array 11, that is, the surface tendency of the dark current, from the generated frame Fc. The noise reduction processing unit 232 performs a dark current noise reduction process to remove the dark current component included in the pixel signal for the image processing unit 23 to generate the frame F of the live view image in this exposure, using the obtained surface tendency of the dark current. The detection pixel signal is an example of the "second signal".

[0058] Next, the control unit 21 causes the pixel signal of the pixel Pi in the sequence 104 to be read out. The control unit 21 may start the readout operation of the sequence 104 at any timing as long as the readout operation of the pixel signal of the pixel Pc in the sequence 103 has ended. For example, the control unit 21 may start the readout operation of the sequence 104 immediately after the time t4 when the readout operation of the sequence 103 ends. FIG. 3 shows a case where the readout operation of the sequence 104 is started from the time t5. In the readout operation of the sequence 104, the control unit 21 sequentially outputs (reads out) pixel signals for each image pixel row of the pixel array 11. As a result, the scanning circuit 12 drives the pixels in the readout operation of the pixel signal of the pixel Pi in the sequence 104 in accordance with the control from the control unit 21.

[0059] More specifically, as shown in FIG. 4, at time t51, the scanning circuit 12 sets the selection drive signal SEL-I for the image pixel row that outputs the pixel signal to the “High” level. As a result, the selection transistor T6 for the image pixel row that outputs the pixel signal is turned on, and the selection transistor T6 outputs the electrical signal of the voltage output by the amplification transistor T5 to the vertical signal line V.

[0060] Subsequently, at time t52, the scanning circuit 12 sets the second reset drive signal RST2-I to the “High” level. As a result, the FD reset transistor T4 is turned on, and the charge accumulated in the floating diffusion FD is (again) reset. An electrical signal of a voltage corresponding to the reset level charge of the floating diffusion FD is output from the amplification transistor T5 and output to the vertical signal line V via the selection transistor T6.

[0061] When the imaging device 10 includes a CDS circuit and the CDS circuit is disposed at a position between the pixel array 11 and the ADC 13, the CDS circuit may sample and hold (clamp) the electrical signal of the reset level of the floating diffusion FD output to each vertical signal line V.

[0062] Thereafter, the scanning circuit 12 sets the second reset drive signal RST2-I to the "Low" level to release the reset of each floating diffusion FD and terminate the output of the reset-level electrical signal to the vertical signal line V.

[0063] Subsequently, at time t53, the scanning circuit 12 sets the second transfer drive signal TG2-I to the "High" level. As a result, the second transfer transistor T3 is turned on, and the charge stored in the pixel memory MEM, that is, the charge generated and stored by the photodiode PD provided in the pixel Pi during the exposure period EX and transferred at time t23, is transferred to and stored in the floating diffusion FD. Here, the charge transferred to the floating diffusion FD also includes the charge based on the dark current stored in the pixel memory MEM after the reset in the transfer operation of sequence 102 is completed. Therefore, an electrical signal with a signal level voltage, in which the charge stored by the photodiode PD due to exposure and the charge based on the dark current stored in the pixel memory MEM are combined, is output from the amplification transistor T5 and output to the vertical signal line V via the selection transistor T6.

[0064] When the imaging device 10 includes a CDS circuit and the CDS circuit is arranged at a position between the pixel array 11 and the ADC 13, the CDS circuit may perform noise suppression processing by sampling the electrical signal with a signal level in which the photodiode PD and the dark current combined and output to each vertical signal line V, holding (clamping) it, and taking the difference from the reset-level electrical signal of the floating diffusion FD.

[0065] Thereafter, the scanning circuit 12 sets the second transfer drive signal TG2-I to the "Low" level to terminate the transfer of the charge of the pixel memory MEM to each floating diffusion FD and terminate the output of the electrical signal with the signal level.

[0066] Subsequently, the scanning circuit 12 ends the state in which the selection drive signal SEL-I is set to the "Low" level and the selection transistor T6 outputs the voltage electrical signal output by the amplification transistor T5 to the vertical signal line V. Then, the scanning circuit 12 sequentially outputs the electrical signals of the current pixel row output to each of the vertical signal lines V for each pixel (for each pixel column).

[0067] In this way, the control unit 21 causes the read operation of the pixel signal of the pixel Pi in the sequence 104 to be performed, and sequentially outputs the pixel signals from the pixel rows for the image. Then, when the output (read) of the pixel signals from all the pixel rows for the image is completed, the control unit 21 ends the read operation of the sequence 104. FIG. 3 shows the case where the output of the pixel signals from all the pixel rows for the image is completed (the read operation of the sequence 104 is ended) at time t6. In FIG. 3, the period during which the pixel signals are sequentially output from the pixel rows for the image is defined as the image pixel signal read period RdI. In FIG. 3, at time t6, based on the pixel signals of the pixels Pi read from the respective pixel rows for the image in the image pixel signal read period RdI (hereinafter referred to as "image pixel signals"), a frame Fi of the original image before the dark current noise reduction process of the live view image is generated. By the way, when generating the live view image, as described above, one line of image data of the frame Fi is generated by pixel addition of the pixel signals of the pixel rows for the image used for generating the live view image. Therefore, the number of pixels of the generated frame Fi is the same as the number of pixels of the frame Fc. And the image data corresponding to each pixel Pi constituting the generated frame Fi includes the magnitude of the dark current accumulated in the pixel memory MEM provided in the pixel Pi arranged in each pixel row for the image during the period from time t22 to time t53. Since the period from time t22 to time t53 is different for each pixel row for the image, the magnitude of the dark current included in each image data is different for each pixel row for the image. However, the surface tendency of the dark current in the frame Fi is the same as the surface tendency of the dark current obtained from the frame Fc generated based on the detection pixel signals read in the detection pixel signal read period RdC. For this reason, the noise reduction processing unit 232 performs a dark current noise reduction process to remove the component of the dark current represented by the frame Fc from the generated frame Fi, and generates a frame F of the live view image after the dark current noise reduction process. The process of removing the component of the dark current represented by the frame Fc from the frame Fi is performed, for example, by performing an operation between the pixel signals (image data) at the same position in the respective images of the frame Fi and the frame Fc.In FIG. 3, at time t6, an operation of subtracting frame Fc from the generated frame Fi is performed to generate frame F in which dark current noise is reduced in the live view image in the current exposure. The pixel signal for the image is an example of the "first signal".

[0068] <An example of dark current noise reduction processing> Here, an example of the dark current noise reduction processing performed by the noise reduction processing unit 232 will be described. FIG. 5 is a diagram schematically showing an example of the relationship between the pixel signal read from the image sensor 10 of the embodiment and the generated image. FIG. 5 shows an example when generating a live view image with 1 / 3 the number of pixels of a still image. FIG. 5(a) shows an example when pixels P are arranged in 18 rows and 4 columns in the pixel array 11 provided in the image sensor 10. FIG. 5(b) shows an example of frame Fc of the dark current image generated based on the pixel signals of the respective pixels Pc of the detection pixel rows arranged in the pixel array 11 shown in FIG. 5(a). FIG. 5(c) shows an example of frame Fi of the original image before the dark current noise reduction processing of the live view image generated based on the pixel signals of the respective pixels Pi of the image pixel rows arranged in the pixel array 11 shown in FIG. 5(a).

[0069] In an example of the pixel array 11 shown in Fig. 5(a), the pixel rows from the first pixel row R1 to the sixth pixel row R6 are the OB regions, and the pixel rows from the seventh pixel row R7 to the eighteenth pixel row R18 are the effective pixel regions. At least, on-chip color filters that transmit visible light of red (R), green (G), or blue (B) are formed on the pixels P arranged in the effective pixel region. However, in Fig. 5(a), the color separation of the on-chip color filters formed on each pixel P is omitted. In an example of the pixel array 11 shown in Fig. 5(a), the second pixel row R2, the fifth pixel row R5, the eighth pixel row R8, the eleventh pixel row R11, the fourteenth pixel row R14, and the seventeenth pixel row R17 are the detection pixel rows. And, in an example of the pixel array 11 shown in Fig. 5(a), the first pixel row R1, the third pixel row R3, the fourth pixel row R4, the sixth pixel row R6, the seventh pixel row R7, the ninth pixel row R9, the tenth pixel row R10, the twelfth pixel row R12, the thirteenth pixel row R13, the fifteenth pixel row R15, the sixteenth pixel row R16, and the eighteenth pixel row R18 are the image pixel rows. That is, in an example of the pixel array 11 shown in Fig. 5(a), the detection pixel rows are provided at two-row intervals in the column direction. In Fig. 5(a), within the "(): parentheses" of each pixel row, "RC" indicating that it is a detection pixel row or "RI" indicating that it is an image pixel row is shown.

[0070] The pixel Pi in the OB region is an example of "a pixel where light is blocked" or "the first pixel where light is blocked", and the pixel Pc in the OB region is an example of "a pixel where light is blocked" or "the second pixel where light is blocked". The pixel Pi in the effective pixel region is an example of "a pixel where light is not blocked" and an example of "the first pixel where light is not blocked". The pixel Pc in the effective pixel region is an example of "a pixel where light is not blocked" and an example of "the second pixel where light is not blocked".

[0071] The noise reduction processing unit 232 generates a 6-row and 4-column frame Fc as shown in FIG. 5(b) based on the detection pixel signals of the pixel Pc read from each detection pixel row RC of the image sensor 10 by the read operation of the pixel signal of the pixel Pc in the sequence 103. At this time, as shown in FIG. 5(b), the noise reduction processing unit 232 does not perform column reduction when generating the frame Fc. The frame Fc is an example of the "second image".

[0072] Thereafter, the image processing unit 23 generates a frame Fi as shown in FIG. 5(c) based on the image pixel signals of the pixel Pi read from each image pixel row RI of the image sensor 10 by the read operation of the pixel signal of the pixel Pi in the sequence 104. At this time, the image processing unit 23 generates a 6-row and 4-column frame Fc whose number of pixels in the column direction and the row direction is the same as that of the frame Fc by pixel addition of the image pixel signals of the respective pixels Pi read from two image pixel rows RI before and after the detection pixel row RC. FIG. 5 schematically shows how the frame Fi is generated by pixel addition of the image pixel signals of the respective pixels Pi read from two image pixel rows RI. Also at this time, as shown in FIG. 5(c), the image processing unit 23 does not perform column reduction when generating the frame Fi. The frame Fc is an example of the "first image".

[0073] Thereafter, the noise reduction processing unit 232 performs a dark current noise reduction process to remove the component of the dark current represented by the frame Fc from the frame Fi generated by the image processing unit 23, and generates a frame F of the live view image after the dark current noise reduction process. FIG. 5 schematically shows how the frame F is generated by performing an operation of subtracting the frame Fc shown in FIG. 5(b) from the frame Fi shown in FIG. 5(c).

[0074] Next, the dark current noise reduction process will be described. FIG. 6 is a diagram schematically showing an example of the relationship between the drive sequence of the imaging element 10 in the imaging operation of the imaging device 1 of the embodiment and the dark current in the imaging element 10. FIG. 7 is a diagram schematically showing an example of the dark current noise reduction process in the imaging operation of the imaging device 1 of the embodiment. In FIG. 6, corresponding to the drive sequence of the imaging element 10 shown in FIG. 3, an example of the drive state and the dark current state of the pixels P in any one pixel column of the pixel array 11 shown in FIG. 5(a) is shown. In FIG. 6, for ease of explanation, the state of each pixel P is divided every unit time UT, and it is assumed that each pixel P is driven in each unit time UT. In FIG. 7, the signal level represented by the pixel signal read from the pixel array 11 shown in FIG. 6 is divided into the signal component SI for the image that is originally desired to be obtained for generating the live view image and the dark current component SC representing the magnitude of the dark current, and an example of specific numerical values is shown, indicating the change in numerical values in the calculation process of the dark current noise reduction process. In the following description, an example of the state of the pixel P shown in FIG. 6 and an example of the dark current noise reduction process shown in FIG. 7 will be referred to as appropriate for explanation.

[0075] First, with reference to FIG. 6, the relationship between the drive sequence of the imaging element 10 and the signal component SI and the dark current component SC included in the output (read) pixel signal will be described. In the following description, it is assumed that the component of the dark current increases (increases) by one level at a time for each unit time UT.

[0076] When the control unit 21 causes the reset operation of the sequence 101 to be performed at time t1, the scanning circuit 12 simultaneously resets the charges accumulated in the photodiode PD, the pixel memory MEM, and the floating diffusion FD included in the pixel P at the unit time UT = “A”. As a result, the pixels Pi in all the image pixels rows RI start to be exposed to the subject light from the unit time UT = “B”. On the other hand, the scanning circuit 12 continues the reset state for the pixels Pc in all the detection pixel rows RC.

[0077] Subsequently, when the control unit 21 causes the transfer operation of sequence 102 to be performed at time t2 when the exposure period EX has elapsed, the scanning circuit 12 resets the pixel memory MEM and the floating diffusion FD of the pixels Pi of all the image pixel rows RI at the unit time UT = “D”, and transfers the charge from the photodiode PD to the pixel memory MEM. On the other hand, the scanning circuit 12 only resets the pixel memory MEM and the floating diffusion FD for the pixels Pc of all the detection pixel rows RC. After this, in the pixel memory MEM, the dark current increases (rises) in proportion to the time during which the charge is held and accumulated.

[0078] Subsequently, when the control unit 21 causes the readout operation of the pixel signal of the pixel Pc in sequence 103 to be performed at time t3 when it is time to output (read out) the detection pixel signal, the scanning circuit 12 sequentially outputs the detection pixel signal. As a result, at the unit time UT = “F”, first, the detection pixel signal corresponding to the pixels Pc of the second detection pixel row R2 (RC) is output. The detection pixel signal output at the unit time UT = “F” is the signal level obtained by combining the signal component SI = “0” due to the reset state of the photodiode PD being continued during the exposure period EX and the dark current component SC = “1” during one unit time UT of the unit time UT = “E” during which the pixel memory MEM has accumulated charge (refer to the first row of the detection pixel row R2 (RC) of the frame Fc shown in FIG. 7).

[0079] Thereafter, detection pixel signals corresponding to the respective pixels Pc of the detection pixel row R5(RC) in the 5th row at unit time UT = "G", the detection pixel row R8(RC) in the 8th row at unit time UT = "H", the detection pixel row R11(RC) in the 11th row at unit time UT = "I", the detection pixel row R14(RC) in the 14th row at unit time UT = "J", and the detection pixel row R17(RC) in the 17th row at unit time UT = "K" are sequentially output. For example, at unit time UT = "H", although it is within the effective pixel region, due to the reset state of the photodiode PD being continued during the exposure period EX, the signal component SI = "0" of the pixel signal corresponding to the pixel Pc of the detection pixel row R8(RC) in the 8th row, and the dark current component SC = "3" during the three unit times UT from unit time UT = "E" to "G" when the pixel memory MEM was accumulating charge are combined, and a detection pixel signal with the signal level (refer to the detection pixel row R8(RC) in the 3rd row of the frame Fc shown in FIG. 7) is output. For example, at unit time UT = "K", although it is within the effective pixel region, due to the reset state of the photodiode PD being continued during the exposure period EX, the signal component SI = "0" of the pixel signal corresponding to the pixel Pc of the detection pixel row R17(RC) in the 17th row, and the dark current component SC = "6" during the six unit times UT from unit time UT = "E" to "J" when the pixel memory MEM was accumulating charge are combined, and a detection pixel signal with the signal level (refer to the detection pixel row R17(RC) in the 6th row of the frame Fc shown in FIG. 7) is output.

[0080] Subsequently, when the control unit 21 causes the reading operation of the pixel signal of pixel Pi in sequence 104 to be performed at time t5 when it is time to output (read) the pixel signal for the image, the scanning circuit 12 sequentially outputs the pixel signals for the image. As a result, in unit time UT = "P", first, a pixel signal for the image obtained by pixel addition of the pixel signal for the image corresponding to pixel Pi of the first row of the image pixel row R1 (RI) and the pixel signal for the image corresponding to pixel Pi of the third row of the image pixel row R3 (RI) is output. The detection pixel signal output in unit time UT = "P" is the signal component SI = "0" of the pixel signal obtained by pixel addition of the first row of the image pixel row R1 (RI) and the third row of the image pixel row R3 (RI), which are the OB regions, and the dark current component SC = "11" during the 11 unit times UT from unit time UT = "E" to "O" in which the pixel memory MEM has accumulated charge, combined signal level (refer to the first row of the image pixel row R1+R3 (RI) of frame Fic shown in FIG. 7) of the pixel signal.

[0081] Thereafter, similarly, in each unit time UT from unit time UT = "Q" to "U", pixel signals for the image obtained by pixel addition of the pixel signals for the image corresponding to pixel Pi arranged in two image pixel rows RI are sequentially output. For example, in unit time UT = "R", the signal component SI = "S1" of the pixel signal obtained by pixel addition of the seventh row of the image pixel row R7 (RI) and the ninth row of the image pixel row R9 (RI), which are the effective pixel regions, and the dark current component SC = "13" during the 13 unit times UT from unit time UT = "E" to "Q" in which the pixel memory MEM has accumulated charge, combined signal level (refer to the third row of the image pixel row R7+R9 (RI) of frame Fic shown in FIG. 7) of the pixel signal for the image is output. For example, in unit time UT = "U", the signal component SI = "S4" of the pixel signal obtained by pixel addition of the 16th row of the image pixel row R16 (RI) and the 18th row of the image pixel row R18 (RI), which are the effective pixel regions, and the dark current component SC = "16" during the 16 unit times UT from unit time UT = "E" to "T" in which the pixel memory MEM has accumulated charge, combined signal level (refer to the sixth row of the image pixel row R16+R18 (RI) of frame Fic shown in FIG. 7) of the pixel signal for the image is output.

[0082] Next, with reference to FIG. 7, the dark current noise reduction process in the noise reduction processing unit 232 will be described. In the dark current noise reduction process in the noise reduction processing unit 232, first, as the first-stage dark current noise reduction process P1, a component of the dark current corresponding to the time during which the pixel memory MEM represented by the frame Fc has accumulated charges is removed from the frame Fi. Then, in the dark current noise reduction process in the noise reduction processing unit 232, as the second-stage dark current noise reduction process P2, a component of the dark current represented by the pixel signal in the OB region is removed. In this way, the noise reduction processing unit 232 performs a dark current noise reduction process for removing (reducing) the overall dark current noise of the effective pixel region arranged in the pixel array 11. FIG. 7 schematically shows the state of the dark current noise reduction process at each stage in the noise reduction processing unit 232. The first-stage dark current noise reduction process P1 in the noise reduction processing unit 232 is an example of the "first process", and the second-stage dark current noise reduction process P2 is an example of the "second process".

[0083] In the first-stage dark current noise reduction process P1 in the noise reduction processing unit 232, for each piece of image data corresponding to each pixel Pi in each image pixel row RI of the frame Fi, an operation is performed to subtract the signal level of the detection pixel signal represented by the image data corresponding to the pixel Pc at the same position in the frame Fc from the signal level of the image pixel signal represented by the image data. In the following description, for the sake of simplicity, it is assumed that the operation is performed between the pixel rows of the image pixel row RI and the detection pixel row RC.

[0084] In the pixel row for image R1+R3(RI) in the first row of frame Fic, an operation is performed to subtract the detection pixel signal level of the detection pixel row R2(RC) in the second row, which is arranged between the pixel row for image R1(RI) in the first row and the pixel row for image R3(RI) in the third row of the pixel array 11, that is, the detection pixel signal level of the detection pixel row R2(RC) in the first row of frame Fc. More specifically, the signal level of the pixel signal for image in the first row of frame Fic, R1+R3(RI), is the signal component SI = "0" + the dark current component SC = "11", and the signal level of the detection pixel signal of the detection pixel row R2(RC) in the first row of frame Fc is the signal component SI = "0" + the dark current component SC = "1". Therefore, an operation of (0 + 11) - (0 + 1) is performed to obtain a signal level of "10".

[0085] In the pixel row for image R7+R9(RI) in the third row of frame Fic, an operation is performed to subtract the detection pixel signal level of the detection pixel row R8(RC) in the eighth row, which is arranged between the pixel row for image R7(RI) in the seventh row and the pixel row for image R9(RI) in the ninth row of the pixel array 11, that is, the detection pixel signal level of the detection pixel row R8(RC) in the third row of frame Fc. More specifically, the signal level of the pixel signal for image in the third row of frame Fic, R7+R9(RI), is the signal component SI = "S1" + the dark current component SC = "13", and the signal level of the detection pixel signal of the detection pixel row R8(RC) in the third row of frame Fc is the signal component SI = "0" + the dark current component SC = "3". Therefore, an operation of (S1 + 13) - (0 + 3) is performed to obtain a signal level of "S1 + 10".

[0086] In the pixel row R16+R18(RI) for the image in the 6th row of the frame Fic, an operation is performed to subtract the detection pixel signal of the detection pixel row R17(RC) in the 17th row, which is arranged between the pixel row R16(RI) for the image in the 16th row and the pixel row R18(RI) for the image in the 18th row of the pixel array 11, that is, the signal level of the detection pixel signal of the detection pixel row R17(RC) in the 6th row of the frame Fc. More specifically, the signal level of the pixel signal for the image in the 6th row of the frame Fic, pixel row R16+R18(RI), is the signal component SI = "S4" + the dark current component SC = "16", and the signal level of the detection pixel signal of the detection pixel row R17(RC) in the 6th row of the frame Fc is the signal component SI = "0" + the dark current component SC = "6". Therefore, an operation of (S4 + 16) - (0 + 6) is performed to obtain a signal level of "S4 + 10".

[0087] In this way, the noise reduction processing unit 232 performs the dark current noise reduction process P1 in the same manner on the image data corresponding to each pixel Pi of all the pixel rows RI for the image included in the frame Fi, and removes the component of the dark current corresponding to the time during which the pixel memory MEM accumulates charge. In FIG. 7, it is assumed that a frame Fr of the dark current noise reduction image of the live view image from which the component of the dark current has been removed by the dark current noise reduction process P1 is generated. As shown in FIG. 7, the frame Fr after the completion of the dark current noise reduction process P1 includes the signal level of the component of the dark current represented by the pixel signal of the pixel P in the OB region as a whole. The signal level obtained by removing the component of the dark current by the dark current noise reduction process P1, or the pixel signal of this signal level, is an example of the "third signal", and the frame Fr is an example of the "third image".

[0088] The noise reduction processing unit 232 performs an operation of subtracting the signal level of the dark current component represented by the image data of each pixel P in the OB region from the signal level of the image pixel signal represented by the image data corresponding to each pixel Pi in each image pixel row RI of the frame Fr by the second-stage dark current noise reduction processing P2. This operation is performed for each pixel column of the frame Fr. At this time, the noise reduction processing unit 232 may subtract the signal level of the dark current component represented by the image data of any one pixel P in the same pixel column from the image data corresponding to all the pixels Pi in the same pixel column, or may subtract the signal level obtained by averaging the signal levels of the dark current components represented by the image data of a plurality or all of the pixels P in the OB region of the same pixel column from the image data corresponding to all the pixels Pi in the same pixel column. FIG. 7 schematically shows a state in which the dark current noise reduction processing P2 is performed by subtracting the signal level obtained by averaging all the dark current components in the OB region of the same pixel column from the image data corresponding to all the pixels Pi in the same pixel column.

[0089] The dark current noise reduction processing P2 in the noise reduction processing unit 232 is the same as the dark current noise reduction processing using optical black in an imaging device equipped with a conventional imaging element. Therefore, a detailed description of the second-stage dark current noise reduction processing P2 in the noise reduction processing unit 232 is omitted.

[0090] In this way, the noise reduction processing unit 232 removes the component of the dark current corresponding to the time during which the pixel memory MEM in the frame Fi accumulated charge by the dark current noise reduction process P1, and removes the component of the dark current represented by the image data of the pixel P in the OB region by the dark current noise reduction process P2. As a result, as shown in FIG. 7, the noise reduction processing unit 232 generates a frame F of the live view image in which only the signal level of the signal component SI of the image that is originally desired to be obtained is represented in the image data corresponding to each pixel Pi of all the image pixel rows RI included in the frame Fi. The signal level obtained by removing the dark current represented by the image data of the pixel P in the OB region by the dark current noise reduction process P2, or the pixel signal of this signal level, is an example of the "fourth signal", and the frame F is an example of the final "image" or "fourth image".

[0091] In this way, in the imaging device 1 of the embodiment, when the image processing engine 20 generates a live view image, the control unit 21 also drives the detection pixel row RC that is not used for generating the live view image and is arranged in the pixel array 11 provided in the imaging element 10. As a result, in the imaging device 1 of the embodiment, the image processing engine 20 acquires a detection pixel signal representing the magnitude of the dark current flowing during the period in which the pixel memory MEM holds and accumulates charge from the pixel Pc of the detection pixel row RC. Then, in the imaging device 1 of the embodiment, the image processing unit 23 (more specifically, the noise reduction processing unit 232) performs a dark current noise reduction process for removing the component of the dark current flowing during the period in which the pixel memory MEM holds and accumulates charge in the pixel Pi of the image pixel row RI used for generating the live view image based on the acquired detection pixel signal. As a result, in the imaging device 1 of the embodiment, when the image processing engine 20 generates a live view image, the frame F of the live view image can be generated based on the pixel signal including only the signal level of the signal component SI of the image that is originally desired to be obtained. Thereby, in the imaging device 1 of the embodiment, a live view image with noise reduced due to dark current can be displayed, for example, on a display device (not shown) provided in the imaging device 1, that is, presented to the user of the imaging device 1.

[0092] As described above, according to the embodiment for carrying out the present invention, the magnitude of the dark current flowing during the period when the pixel memory included in the pixel holds and accumulates electric charge is detected by using the pixels arranged in the pixel rows not used for generating the live view image. Then, in the embodiment for carrying out the present invention, a dark current noise reduction process is performed to remove (reduce) the detected dark current component from the pixel signal corresponding to the pixel arranged in the pixel row used for generating the live view image. Thereby, in the embodiment for carrying out the present invention, it is possible to suppress a deterioration in the image quality of the generated live view image.

[0093] In the embodiment, the case where the detection pixel signal is acquired (read out) from the pixel Pc of the detection pixel row RC each time the live view image is generated to generate the frame Fc has been described. However, the timing for generating the frame Fc, that is, the timing for acquiring the detection pixel signal from the pixel Pc of the detection pixel row RC is not limited to the timing each time the live view image is generated, that is, the timing each time the frame Fi is generated based on the image pixel signal acquired (read out) from the pixel Pi of the image pixel row RI. For example, if the generated frame Fc is stored in the memory 24 and the noise reduction processing unit 232 is configured to use the frame Fc stored in the memory 24 for the dark current noise reduction process, the timing for acquiring the detection pixel signal from the pixel Pc of the detection pixel row RC may not be the timing each time the frame Fi is generated, such as at a predetermined interval or when the change in the temperature of the imaging device 10 (for example, a temperature increase) becomes equal to or greater than a predetermined value. In this case, if the noise reduction processing unit 232 updates the frame Fc stored in the memory 24 to the frame Fc generated based on the detection pixel signal acquired this time, the updated frame Fc can be used in the subsequent dark current noise reduction process.

[0094] In the embodiment, when generating a live view image with a pixel count that is one-third of that of a still image, three pixel rows of the pixel array 11 are grouped together. Among this group of pixel rows, one pixel row is used as the detection pixel row RC and is not used for generating the live view image, and the remaining two pixel rows are used as the image pixel rows RI. The case where an image pixel signal is pixel-added to form one pixel row of the live view image has been described. However, the reduction factor of the pixel columns when generating the live view image is not limited to one-third. For example, the live view image may have a pixel count that is one-fourth of that of the still image. In this case, for example, four pixel rows of the pixel array 11 are grouped together. Among this group of pixel rows, one pixel row is used as the detection pixel row RC and is not used for generating the live view image, and it is also conceivable to pixel-add the image pixel signals using the remaining three pixel rows as the image pixel rows RI. Also in this case, the noise reduction processing unit 232 can perform dark current noise reduction processing based on the detection pixel signal obtained (read out) from the pixel Pc of the detection pixel row RC that is close to (or adjacent in the column direction) the image pixel row RI, in the same manner as in the above-described embodiment. The dark current noise reduction processing in this case may be made equivalent to the dark current noise reduction processing described in the above-described embodiment. Therefore, a detailed description of the dark current noise reduction processing when the reduction factor of the pixel columns when generating the live view image is different is omitted.

[0095] In the embodiment, when the image processing engine 20 generates a live view image, a detection pixel signal representing a component of dark current flowing during the period when the pixel memory MEM included in the pixel P holds and accumulates electric charge is obtained from (read out from) the pixel Pc of the detection pixel row RC, and the dark current noise reduction process is described. However, the obtained detection pixel signal can also be used when generating an image other than the live view image. For example, when the image processing engine 20 generates a moving image, if there is a pixel row not used for generating the moving image, the dark current noise reduction process can be similarly performed using that pixel row as the detection pixel row RC. On the other hand, for example, even when there is no pixel row not used for generating an image, such as when generating a still image (which may include a moving image), a frame Fc generated based on the detection pixel signal obtained when generating the live view image is stored in the memory 24, and the component of the dark current represented by the stored frame Fc can be used for the dark current noise reduction process for the image pixel signal obtained from the pixel Pi of the image pixel row RI used for generating the image. In this case, for example, if the frame Fc is used for the dark current noise reduction process at a timing close to the timing of obtaining the image pixel signal used for generating the image, such as the timing of generating the live view image immediately before a shooting instruction (which may be a full press or a half press of the release button R) is given by the user of the imaging device 1, the component of the dark current can be more appropriately removed (reduced) from the image pixel signal. At this time, for example, it is also conceivable that there is a difference between the image pixel signal readout period RdI for reading the image pixel signal for the live view image and the image pixel signal readout period RdI for reading the image pixel signal for the image to be generated this time. That is, it is also conceivable that the surface tendency of the dark current obtained from the frame Fc generated based on the detection pixel signal read out when generating the live view image cannot be directly applied to the image to be generated this time. However, even in this case, for example, by converting (adjusting) the surface tendency of the dark current obtained from the frame Fc of the live view image using a predetermined coefficient or gain and then using it, the component of the dark current can be removed (reduced) from the image pixel signal with a surface tendency of the dark current suitable for the image to be generated this time.

[0096] In the embodiment, the case where the image processing engine 20 first acquires a detection pixel signal from the pixel Pc of the detection pixel row RC and then acquires an image pixel signal from the pixel Pi of the image pixel row RI has been described. In this case, for example, the image processing unit 23 included in the image processing engine 20 can perform so-called pipeline processing in which various image processes are performed as a series of image processes without temporarily storing a large amount of image data or processing data in a temporary storage means such as the memory 24 during the processing process of the image processing for generating an image. However, it is also conceivable that the image processing unit 23 is not configured to perform a series of image processes by pipeline processing, but is configured to use a temporary storage means such as the memory 24 during the processing process of each image process. In this case, the image processing engine 20 does not necessarily have to first acquire the detection pixel signal from the pixel Pc of the detection pixel row RC. For example, the image processing engine 20 may first acquire the image pixel signal from the pixel Pi of the image pixel row RI and then acquire the detection pixel signal from the pixel Pc of the detection pixel row RC, or may acquire the detection pixel signal from the pixel Pc of the detection pixel row RC when acquiring the image pixel signal from the pixel Pi of the image pixel row RI, that is, without separating the image pixel row RI and the detection pixel row RC, sequentially acquire pixel signals from each pixel P.

[0097] In the embodiment, as shown in FIG. 4, the case where the scanning circuit 12 is driven to maintain the reset state of the photodiode PD included in the pixel Pc of the detection pixel row RC during the exposure period EX has been described. However, the charges generated and accumulated by the photodiode PD included in the pixel Pc of the detection pixel row RC do not have to be transferred to the pixel memory MEM in the transfer operation of the sequence 102, like the charges generated and accumulated by the photodiode PD included in the pixel Pi of the image pixel row RI. For this reason, the scanning circuit 12 drives the pixel Pc of the detection pixel row RC in the same manner as the pixel Pi of the image pixel row RI in the reset operation of the sequence 101, and in the transfer operation of the sequence 102, when resetting the pixel memory MEM and the floating diffusion FD included in each pixel P again, or when transferring the charges generated and accumulated by the photodiode PD included in the pixel Pi of the image pixel row RI to the pixel memory MEM, the scanning circuit 12 may also be driven to reset the photodiode PD included in the pixel Pc of the detection pixel row RC. That is, even if the photodiode PD included in the pixel Pc of the detection pixel row RC is also exposed during the exposure period EX, the scanning circuit 12 may be driven to discard the charges accumulated by the photodiode PD.

[0098] As described above, the embodiments of the present invention have been described with reference to the drawings. However, the specific configuration is not limited to this embodiment, and various modifications within the scope not departing from the spirit of the present invention are also included.

Description of Reference Numerals

[0099] 1... Imaging device, 10... Image sensor, 11... Pixel array, P... Pixel, V... Vertical signal line, PD... Photodiode, MEM... Pixel memory, FD... Floating diffusion, T1... First transfer transistor, T2... PD reset transistor, T3... Second transfer transistor, T4... FD reset transistor, T5... Amplification transistor, T6... Selection transistor, 12... Scanning circuit, 13... ADC, 14... Output section, 20... Image processing engine, 21... Control section, 22... Input section, 23... Image processing section, 232... Noise reduction processing section, 24... Memory, L... Lens section, R... Release button

Claims

1. An imaging device including a plurality of pixels each having a photoelectric conversion unit that converts incident light into electric charges and a holding unit that holds the electric charges converted by the photoelectric conversion unit, wherein the plurality of pixels include a plurality of first pixels that are used for generating an image and output a first signal based on the electric charges held in the holding unit, and a plurality of second pixels that output a second signal for removing noise generated in the holding unit of the first pixels included in the first signal. The imaging device having the above.

2. The second pixels output the second signal based on the electric charges generated in the holding unit of the second pixels. The imaging device according to Claim 1.

3. The plurality of first pixels are provided adjacent to each other in the column direction. The imaging device according to Claim 1.

4. The plurality of second pixels are provided between the plurality of first pixels in the column direction. The imaging device according to Claim 3.

5. The plurality of pixels are arranged in a two-dimensional matrix, and the plurality of second pixels are provided in the column direction in the same number as the number of the first pixels in the column direction at a predetermined row interval. The imaging device according to Claim 1.

6. The first pixels and the second pixels include the pixels in which the light incident on the photoelectric conversion unit is blocked, and the first pixels and the second pixels in which the light is blocked are provided in a plurality in the row direction at the end of the two-dimensional matrix region in the same number as the number of the column direction. The imaging device according to Claim 5.

7. The second signal is output before the first signal. The imaging device according to Claim 1.

8. An imaging device including the imaging device according to any one of Claims 1 to 7, and an image processing unit that removes the noise included in a first image generated using the first signal based on the second signal. The imaging device having the above.

9. The image processing unit performs a first process of removing the noise by subtracting a second image generated using the second signal from the first image. The imaging device according to Claim 8.

10. The first process is an operation of subtracting the second signal from the first signal. The imaging device according to Claim 9.

11. The image processing unit performs the first process on the signals at the same positions in each of the first signals constituting the first image and each of the second signals constituting the second image, and generates a third image using the third signals obtained by performing the first process. The imaging device according to Claim 10.

12. The image processing unit Of each of the third signals constituting the third image, a second process is performed to remove the noise by subtracting the third signal corresponding to the pixel where the light is blocked from the third signal corresponding to the pixel where the light is not blocked. A fourth image is generated using the fourth signal obtained by the second process. The imaging device according to claim 11.

Images