Imaging apparatus, imaging method, and computer program

Abstract

To provide an imaging apparatus capable of reducing light while reducing the reduction of SNR in the imaging apparatus in which a photoelectric conversion part has a plurality of charge holding parts.SOLUTION: The imaging apparatus has a photoelectric conversion part 201, a plurality of signal holding parts for holding signals photoelectrically converted by the photoelectric conversion part 201, an addition part for adding the signals of the plurality of signal holding parts, and a division part 207 for dividing the signal added by the addition part by the number of the plurality of signal holding parts added by the addition part as a divisor.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, an imaging method, a computer program, and the like. [Background technology]

[0002] Conventionally, imaging elements such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor) sensors have an arrangement of multiple light-receiving elements (pixels), and each pixel converts light into electricity to detect the intensity of the light (brightness).

[0003] In addition, by placing a color filter that passes one of the wavelength bands of red (R), green (G), or blue (B) on each pixel, it is possible to obtain only the wavelength (color) of visible light. Using this mechanism, it is possible to record the visible light subject that we normally see as a video signal (electrical signal) in a storage device or display it on a display device.

[0004] In order to convert light into an electrical signal, a pixel is provided with a photoelectric conversion unit that converts light into an electric charge and a floating diffusion layer that converts the electric charge into an electrical signal. In addition to a configuration in which the electric charge converted by the photoelectric conversion unit is directly transferred to the floating diffusion layer, a configuration is also known in which a charge storage unit is provided between the photoelectric conversion unit and the floating diffusion layer, and an electric signal is generated by transferring the electric charge from the photoelectric conversion unit to the charge storage unit and then transferring the electric charge from the charge storage unit to the floating diffusion layer.

[0005] Patent Document 1 proposes a technology that has a charge drain section that drains the charge accumulated in the photoelectric conversion section, and performs the desired light reduction (exposure adjustment, accumulation time control) by repeatedly discharging the charge to the charge drain section and transferring it to the charge retention section. By using this technology, it is possible to generate a dimmed output image without using a physical element such as an ND filter. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-128380 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, when the technology of Patent Document 1 is used, the light is reduced by discharging the accumulated electric charge from the electric charge discharging section, and therefore the signal component is reduced, thereby reducing the SNR (Signal to Noise Ratio).In addition, when shooting a subject that emits light periodically, if the charge discharging cycle and the light emission cycle of the subject are synchronized, there is a problem that the desired light reduction cannot be achieved.

[0008] An object of the present invention is to provide an imaging device capable of reducing light while preventing a decrease in SNR. [Means for solving the problem]

[0009] In order to solve the above problems, an imaging device according to one aspect of the present invention comprises: A photoelectric conversion unit; a plurality of signal holding units for holding the signals photoelectrically converted by the photoelectric conversion units; an adder that adds up the signals of the plurality of signal holding units; a division unit that divides the signal added by the addition unit by the number of the signal holding units added by the addition unit as a divisor; The present invention is characterized by having the following. Effect of the Invention

[0010] According to the present invention, it is possible to realize an imaging device that can perform light reduction while preventing a decrease in SNR. [Brief description of the drawings]

[0011] [Figure 1]1 is a functional block diagram showing an example of the configuration of an imaging device according to a first embodiment. [Diagram 2] 2 is a functional block diagram showing an example of the configuration of an image sensor 102 and an image processing unit 103 according to the first embodiment. FIG. [Diagram 3] 2 is a diagram showing an example of the arrangement of a pixel of an image sensor 102 according to the first embodiment. [Figure 4] 4 is a timing chart showing an example of addition at a predetermined pixel of the image sensor according to the first embodiment. [Diagram 5] 5 is a continuation of the timing chart in FIG. 4. [Figure 6] 5 is a diagram showing an example of the relationship between SNR in a charge holding section, an adder section, and a divider section according to the first embodiment. FIG. [Figure 7] FIG. 11 is a functional block diagram showing an example of the configuration of an imaging element and an image processing unit in the case where the image processing unit has an adder unit according to the second embodiment as a signal adder unit. [Figure 8] 11 is a diagram showing an example of the configuration of a pixel of an image sensor according to a second embodiment. FIG. [Figure 9] 10 is a timing chart showing an example of a case where addition is performed by a signal addition unit of an image processing unit according to the second embodiment. [Figure 10] 10 is a continuation of the timing chart in FIG. 9. [Figure 11] 11 is a timing chart showing an example of control for a predetermined pixel of the image sensor 102 according to the third embodiment. [Figure 12] 12 is a continuation of the timing chart in FIG. 11 . [Figure 13] 13 is a timing chart showing an example of a case where addition is performed in a signal addition unit of an image processing unit by time-division transfer from a photoelectric conversion unit according to the fourth embodiment. [Figure 14] 14 is a continuation of the timing chart in FIG. 13. [Figure 15] 13 is a timing chart showing an example of addition at a predetermined pixel of an image sensor according to the fifth embodiment. [Figure 16] 16 is a continuation of the timing chart in FIG. 15. [Figure 17]13 is a timing chart showing an example of a case where addition is performed in a signal addition unit of an image processing unit according to the sixth embodiment. [Figure 18] 18 is a continuation of the timing chart in FIG. 17. [Figure 19] 13 is a timing chart showing an example of control for a predetermined pixel of the image sensor 102 according to the seventh embodiment. [Figure 20] 20 is a continuation of the timing chart in FIG. 19. [Figure 21] 20 is a timing chart showing an example of a case where addition is performed in a signal addition unit of an image processing unit according to the eighth embodiment. [Figure 22] 22 is a continuation of the timing chart in FIG. 21. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numbers, and duplicated descriptions are omitted or simplified.

[0013] <Embodiment 1> FIG. 1 is a functional block diagram showing an example of the configuration of an imaging device according to the first embodiment, and each block in FIG. 1 will be described in detail.

[0014] 1 are realized by causing a CPU or the like serving as a computer (not shown) included in the imaging device to execute a computer program stored in a memory (not shown) serving as a storage medium. However, some or all of these may be realized by hardware. As the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP), etc., may be used.

[0015] In addition, the functional blocks shown in Fig. 1 do not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths. The above explanation regarding Fig. 1 also applies to Figs. 2 and 7.

[0016] An optical lens 101 captures light from a subject and forms an image of the light on an image sensor 102. The image sensor 102 is a CCD image sensor, a CMOS image sensor, or the like, and receives incident light from the optical lens 101, converts it into an electrical signal, and outputs it.

[0017] The image processing unit 103 captures the image signal output from the image sensor 102, and performs processes such as removing fixed pattern noise from the image sensor 102 and clamping the black level. It also generates a compressed image for recording and generates an evaluation signal for controlling the image sensor. If AD conversion is not performed inside the image sensor 102, it also includes an analog front end that performs analog-to-digital conversion.

[0018] Moreover, the image processing unit 103 performs various image processing such as pixel addition, noise reduction, gamma correction, knee correction, digital gain correction, and scratch correction, which are typical image processing functions of an imaging device. In particular, one of the features of this embodiment is that it has a circuit for performing light reduction.

[0019] The signal recording unit 104 records the image signal received from the image processing unit 103 in a storage device or storage medium.

[0020] The control unit 105 calculates an optimal exposure amount based on the image signal supplied from the image processing unit 103. Furthermore, if the image sensor 102 is a type capable of outputting image plane phase difference information, the control unit 105 performs control for calculating a phase difference from the output of the image sensor.

[0021] The control unit 105 also has a built-in CPU as a computer, and functions as a control unit that controls the operation of each unit of the imaging device based on a computer program recorded in a memory (not shown).

[0022] FIG. 2 is a functional block diagram showing an example of the arrangement of the image sensor 102 and image processing unit 103 according to the first embodiment, and a specific pixel of the image sensor 102 will be described here.

[0023] In a predetermined pixel of the image sensor 102, the light of the subject is converted into electric charges and accumulated in the photoelectric conversion unit 201. The electric charges accumulated in the photoelectric conversion unit 201 are transferred to the first charge holding unit 204 and the second charge holding unit 205 by the first transfer unit 202 and the second transfer unit 203, respectively. Note that the first charge holding unit 204 and the second charge holding unit 205 function as a plurality of signal holding units for respectively holding signals photoelectrically converted by the photoelectric conversion unit during the period when the photoelectric conversion unit is performing photoelectric conversion.

[0024] At this time, the accumulation times in the photoelectric conversion unit 201 until transfer to the first transfer unit 202 and the second transfer unit 203 are controlled to be equal so that the amounts of charge held in the first charge holding unit 204 and the second charge holding unit 205 are approximately equal.

[0025] When the charge transfer from the photoelectric conversion unit 201 to the first transfer unit 202 and the second transfer unit 203 is completed, the charges accumulated in the first charge holding unit 204 and the second charge holding unit 205 are transferred to the charge addition unit 206, where the charges are added. Note that the charge addition unit 206 functions as an addition unit that performs analog addition of signals from the multiple signal holding units.

[0026] The charges added by the charge adding unit 206 are converted into an electric signal by a charge-voltage conversion circuit (not shown), and are converted into a digital signal by AD conversion in an ADC (Analog to Digital Converter) (not shown). The converted digital signal is then input to a division unit 207, where a predetermined division process is performed.

[0027] The division unit 207 performs division using the number of charge holding units as a divisor to obtain an arithmetic average. In this embodiment, since charges are transferred from two charge holding units to the charge addition unit 206, the division is performed using 2 as a divisor. The division unit 207 may be controlled by a computer to perform a division step of dividing the signal added by the addition unit by the number of the multiple signal holding units added by the addition unit as a divisor.

[0028] As described above, charges generated in the same accumulation time in photoelectric conversion unit 201 are distributed to first charge holding unit 204 and second charge holding unit 205, respectively, added in charge addition unit 206, and divided in division unit 207, thereby obtaining a dimming effect for the signal component by the amount of the divisor. In addition, since an arithmetic average is performed for the random noise component, the random noise component can be improved by the square root of the divisor for the signal of each charge holding unit.

[0029] 3 is a diagram showing an example of the configuration of a pixel of the image sensor 102 according to the first embodiment. The image sensor 102 has a PD (photodiode) as a photoelectric conversion unit 201, and converts light from a subject into an electric charge. The photoelectric conversion unit 201 (PD) is connected to a first charge holding unit 204 (MEM_A) via a first transfer unit 202 (TX_A1), and is also connected to a second charge holding unit 205 (MEM_B) via a second transfer unit 203 (TX_B1).

[0030] The first charge holding unit 204 is connected to the charge addition unit 206 (FD) via a third transfer unit 306 , and the second charge holding unit 205 is also connected to the charge addition unit 206 (FD) via a fourth transfer unit 307 .

[0031] The charge summing unit 206 (FD) is composed of, for example, a floating diffusion layer, and sums up the charges distributed to the first charge holding unit 204 and the second charge holding unit 205. That is, in the example shown in Fig. 3, the summing unit includes a floating diffusion layer for summing up the charges photoelectrically converted by the photoelectric conversion unit.

[0032] The charge adder 206 (FD) is connected to a reset unit 309 (RES) and a source follower unit 310 (SF). The reset unit 309 (RES) is connected to a power supply VDD, and by turning on the reset unit 309 (RES), the charge accumulated in the charge adder 206 (FD) is released to the power supply VDD, thereby performing a reset operation.

[0033] The source follower unit 310 (SF) is connected to a vertical output line 312 (VL) via a selector unit 311 (SEL), converts the charge accumulated in the charge adder unit 206 (FD) into a voltage, and outputs a signal corresponding to the converted voltage.

[0034] When a selector 311 (SEL) is selected in response to a pixel readout timing, the output from the source follower 310 (SF) is output to a vertical output line 312 (VL). The signal output to the vertical output line 312 (VL) is output to the outside of the image sensor 102 via a column amplifier, an ADC, a horizontal transfer circuit, etc. (not shown).

[0035] Next, Fig. 4 is a timing chart showing an example of addition at a predetermined pixel of the image sensor according to the first embodiment, and Fig. 5 is a timing chart continuing from Fig. 4. In Figs. 4 and 5, the Nth frame, which is a predetermined frame, will be particularly described.

[0036] At time t0 after the end of the (N-1)th frame, RES φ309 goes high and the reset unit 309 turns on, thereby starting resetting the charge addition unit 206.

[0037] At time t1, TX_A2 φ306 and TX_B2 φ307 go to Hi, and resetting of the first charge holding unit 204 and the second charge holding unit 205 begins.

[0038] At time t2, TX_A2 φ306 and TX_B2 φ307 go to Lo, and resetting of the first charge holding unit 204 and the second charge holding unit 205 ends.

[0039] At time t3, RES φ309 goes low, the reset unit 309 turns off, and the reset of the charge addition unit 206 ends.

[0040] At time t4, the SEL φ311 goes Hi, and the noise components of the charge adder 206 after the resetting is completed at time t3 are started to be read out to the vertical output line 312.

[0041] At time t5, SEL 311 becomes Lo, ending the readout of the noise components from the charge adder 206 after the reset is completed at time t3 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset at the start of the Nth frame in FIG.

[0042] At time t 6 , TX_A 1 φ 202 goes high, and the charge stored in the photoelectric conversion unit 201 starts to be transferred to the first charge holding unit 204 .

[0043] At time t7, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t7 are transferred to the first charge holding unit 204.

[0044] In FIG. 5, at time t8, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0045] At time t9, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t7 to t9 are transferred to the second charge holding unit 205.

[0046] At time t10, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the charge addition unit 206 begins.

[0047] At time t11, TX_A2 φ306 goes to Lo, and the readout of charge from the first charge holding unit 204 to the charge addition unit 206 ends.

[0048] At time t12, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the charge addition unit 206 begins.

[0049] At time t13, TX_B2 φ307 goes to Lo, and the readout of charges from the second charge holding unit 205 to the charge addition unit 206 ends.

[0050] At time t14, the SEL φ311 goes high, and the electric signal of the charge accumulated in the charge adder 206 starts to be read out to the vertical output line 312.

[0051] At time t15, the SEL φ311 goes to Lo, and the readout of the electric signals of the charges accumulated in the charge adder 206 to the vertical output line 312 ends.

[0052] Note that the signal read out during the period from time t14 to t15 contains a noise component, and this noise signal has a predetermined correlation with the noise component read out during the period from time t4 to t5. This operation completes the reading of the signal component (including the noise component) obtained by adding up the first charge holding unit 204 and the second charge holding unit 205 in the Nth frame in FIG.

[0053] As described above, the noise components are read out when the first charge holding unit 204, the second charge holding unit 205, and the charge addition unit 206 are reset, and the signal components (including noise components) of the charges accumulated in the first charge holding unit 204 and the second charge holding unit 205 are read out.

[0054] Thereby, by performing, for example, CDS (Correlated Double Sampling) in a column amplifier at the subsequent stage of the vertical output line 312, it is possible to reduce (subtract) noise components.

[0055] In addition, although the present embodiment describes a predetermined pixel, the same processing can be applied to a plurality of pixels arranged one-dimensionally and a plurality of pixels arranged two-dimensionally. For example, in a pixel array of m rows and n columns, each column of the mth row is commonly controlled at the same timing, and the (m+1)th row is read out by shifting each column by the same predetermined time so that the readout timing of the vertical output line 312 does not overlap. In other words, the present embodiment is not limited to the pixel array.

[0056] FIG. 6 is a diagram showing an example of the relationship between the SNR in the charge holding unit, the adder, and the divider according to the first embodiment.

[0057] In this embodiment, charges generated by the same accumulation time are transferred from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205. Therefore, when the light from the subject is approximately constant, the signal component and noise component accumulated in each charge holding unit are the same. Here, when the signal component is sufficiently large, the noise component is dominated by optical shot noise and has a size equal to the square root of the signal component.

[0058] Next, when the signals from the first charge holding unit 204 and the second charge holding unit 205 are added together in the charge adding unit 206, the signal components have the same added value, but the noise components are dominated by optical shot noise, so they become the square root of the added signal components.

[0059] When division is performed in division unit 207 using the sum of the signals from charge addition unit 206 as the divisor, the signal component becomes approximately the same as the value accumulated in first charge holding unit 204 or second charge holding unit 205. Therefore, a dimming effect of 1 / divisor can be obtained for the value generated in photoelectric conversion unit 201.

[0060] On the other hand, the noise component is a value obtained by dividing the noise component of the first charge holding unit 204 or the second charge holding unit 205 by the square root of the divisor. This makes it possible to prevent a decrease in SNR (signal to noise ratio) compared to charge accumulation time control in which light is reduced by discharging charge during the accumulation period.

[0061] As described above, in the first embodiment, in a configuration in which the photoelectric conversion unit has a plurality of charge storage units, the charges generated in the photoelectric conversion unit are transferred to each charge storage unit with the same accumulation time, and arithmetic averaging is performed by addition in the adder unit and division in the divider unit. This makes it possible to reduce the decrease in SNR while obtaining the light reduction effect according to the number of the distributed charge storage units.

[0062] <Embodiment 2> In the first embodiment, the addition of signals from the charge storage units is performed in a floating diffusion layer arranged in the image sensor. However, when the circuit scale of the image sensor is taken into consideration, the addition does not necessarily have to be performed in the floating diffusion layer. That is, in the second embodiment, an example in which signals from each charge storage unit are added in an image processing unit will be described. Note that the description of the contents common to the first embodiment will be omitted as appropriate.

[0063] 7 is a functional block diagram showing a configuration example of an image sensor and an image processing unit when the image processing unit has an adder unit according to embodiment 2 as a signal adder unit. In the image sensor 102, the photoelectric conversion unit 201, the first transfer unit 202, the second transfer unit 203, the first charge holding unit 204, and the second charge holding unit 205 are similar to the configuration shown in FIG.

[0064] The charges stored in the first charge holding unit 204 and the second charge holding unit 205 are converted into electric signals by circuits (not shown), and then converted into digital signals by an ADC (not shown). The converted digital signals are then input to a signal addition unit 701 provided in the image processing unit 103, where signal addition processing is performed.

[0065] 7, the adder adds the signals from the signal holding units after AD conversion. The signals are then input to the divider 207, where the same division process as shown in FIG.

[0066] Next, Fig. 8 is a diagram showing an example of the configuration of a pixel of an image sensor according to embodiment 2. In Fig. 8, a photoelectric conversion unit 201 (PD), a first transfer unit 202 (TX_A1), a first charge holding unit 204 (MEM_A), a second transfer unit 203 (TX_B1), and a second charge holding unit 205 (MEM_B) have the same configuration as those in Fig. 3. In addition, a third transfer unit 306 (TX_A2) and a fourth transfer unit 307 (TX_B2) also have the same configuration as those in Fig. 3.

[0067] However, the third transfer unit 306 (TX_A2) is connected to a floating diffusion layer 807 (FD_A), and the fourth transfer unit 307 (TX_B2) is connected to a floating diffusion layer 808 (FD_B). In addition, the floating diffusion layer 807 (FD_A) is connected to a reset unit 809 (RES_A) and a source follower unit 810 (SF_A).

[0068] The reset unit 809 (RES_A) is connected to a power supply VDD, and when the reset unit 809 (RES_A) is turned on, the charge accumulated in the floating diffusion layer 807 (FD_A) is discharged to the power supply VDD and reset. The source follower unit 810 (SF_A) is connected to the vertical output line 312 (VL) via a select unit 811 (SEL_A).

[0069] When electric charges are accumulated in the floating diffusion layer 807 (FD_A), an output corresponding to the converted voltage is output from the source follower section 810 (SF_A). The output from the source follower section 810 (SF_A) is output to the vertical output line 312 (VL) by turning on the select section 811 (SEL_A) in accordance with the pixel readout timing.

[0070] The signal output to the vertical output line 312 (VL) is output to the outside of the image sensor 102 via a column amplifier, an ADC, a horizontal transfer circuit, etc. (not shown). Note that the reset unit 812 (RES_B), the source follower unit 813 (SEL_B), and the select unit 814 (SEL_B) have the same configuration as the reset unit 809 (RES_A), the source follower unit 810 (SF_A), and the select unit 811 (SEL_A), respectively, and therefore description thereof will be omitted.

[0071] Next, Fig. 9 is a timing chart showing an example of a case where addition is performed in the signal addition unit of the image processing unit according to the second embodiment, and Fig. 10 is a continuation of the timing chart of Fig. 9. In Figs. 9 and 10, an Nth frame, which is a predetermined frame, will be described.

[0072] At time t0, RES_A φ809 goes high, turning on the reset unit 809, and resetting of the floating diffusion layer 807 begins.

[0073] At time t1, TX_A2 φ306 goes high, and resetting of the first charge holding unit 204 begins.

[0074] At time t2, TX_A2 φ306 goes low, and the reset of the first charge holding unit 204 ends.

[0075] At time t3, RES_A φ809 goes to Lo, and the reset unit 809 turns off, thereby completing the reset of the floating diffusion layer 807.

[0076] At time t4 in FIG. 10, SEL_A φ811 goes Hi, and the noise components of the floating diffusion layer 807 after the reset is completed at time t3 are read out to the vertical output line 312.

[0077] At time t5, SEL_A φ811 becomes Lo, completing the readout of the noise components in the floating diffusion layer 807 immediately after the reset at time t3 to the vertical output line 312. This operation completes the readout of the noise components in the floating diffusion layer 807 immediately after the reset of the Nth frame.

[0078] At time t6, TX_A1 φ202 goes high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 begins.

[0079] At time t7, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t7 are transferred to the first charge holding unit 204.

[0080] At time t8, RES_B φ812 goes high, turning on the reset unit 812, and resetting of the floating diffusion layer 808 begins.

[0081] At time t9, TX_B2 φ307 goes high, and resetting of the second charge holding unit 205 begins.

[0082] At time t10, TX_B2 φ307 goes to Lo, and the resetting of the second charge holding unit 205 ends.

[0083] At time t11, RES_B φ812 goes to Lo, and the reset unit 812 turns off, thereby completing the reset of the floating diffusion layer 808.

[0084] At time t12, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the floating diffusion layer 807 begins.

[0085] At time t13, TX_A2 φ306 goes to Lo, and the readout of charges from the first charge holding unit 204 to the floating diffusion layer 807 ends.

[0086] At time t14, SEL_A φ811 goes high, and the electric signal of the charge stored in the first charge holding unit 204 starts to be read out to the vertical output line 312.

[0087] At time t15, SEL_A φ811 goes to Lo, completing the readout of the electric signal of the charge accumulated in the first charge holding unit 204 to the vertical output line 312. This operation completes the readout of the signal component (including the noise component) of the first charge holding unit 204 in the Nth frame in FIG.

[0088] At time t16, SEL_B φ814 goes high, and the noise components of the floating diffusion layer 808, which was reset at time t11, start to be read out to the vertical output line 312.

[0089] At time t17, SEL_B φ814 goes Lo, completing the readout of the noise components of the floating diffusion layer 808 that was reset at time t11 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset in the floating diffusion layer 808 of the Nth frame.

[0090] At time t18, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0091] At time t19, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charge accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t7 to t19 is transferred to the second charge holding unit 205.

[0092] At time t20, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the floating diffusion layer 808 begins.

[0093] At time t21, TX_B2 φ307 goes to Lo, and the readout of charges from the second charge holding unit 205 to the floating diffusion layer 808 ends.

[0094] At time t22, SEL_B φ814 goes high, and reading out of the electrical signal of the charge stored in the second charge holding unit 205 to the vertical output line 312 begins.

[0095] At time t23, SEL_B φ814 goes to Lo, completing the readout of the electrical signal of the charge accumulated in the second charge holding unit 205 to the vertical output line 312. This operation completes the readout of the signal components (including noise components) from the second charge holding unit 205 in the Nth frame.

[0096] As described above, in the second embodiment, the reset noise components of the first charge retention unit 204 and the floating diffusion layer 807 and the signal components (including noise components) of the first charge retention unit 204 are successively read out. Then, the reset noise components of the second charge retention unit 205 and the floating diffusion layer 808 and the signal components (including noise components) of the second charge retention unit 205 are read out, whereby the noise components can be reduced (subtracted) by CDS in the column amplifier at the subsequent stage, as in the first embodiment.

[0097] Although only a predetermined pixel has been described in the second embodiment, the second embodiment can also be applied to a plurality of pixels arranged one-dimensionally or a plurality of pixels arranged two-dimensionally, as in the first embodiment. Then, by performing arithmetic averaging by adding signals read from the image sensor 102 in the signal adding unit 701 in the image processing unit 103 and dividing the signals in the dividing unit 207, it is possible to obtain the same light reduction effect and the effect of suppressing SNR reduction as described in FIG.

[0098] As described above, even when an adder unit is provided in the image processing unit as in embodiment 2, it is possible to suppress a decrease in SNR while obtaining a light reduction effect corresponding to the number of distributed charge holding units. Note that the image processing unit 103 may be provided in a semiconductor substrate having a multilayer structure together with the image sensor 102.

[0099] <Embodiment 3> In the first and second embodiments, the configuration in which signals stored in a plurality of charge storage units are added by the image sensor or the image processor has been described. However, in both embodiments, the configuration is such that charges are transferred to a predetermined charge storage unit, and then transferred to another charge storage unit.

[0100] In other words, the charge transfer to each charge storage unit in a given pixel is only once in a given frame. Therefore, when shooting a subject that periodically blinks, such as an LED (Light Emitting Diode), the photoelectric conversion unit may become saturated during the charge accumulation period for transferring the charge to the given charge storage unit.

[0101] In the third embodiment, therefore, a configuration and method for reducing the possibility of the photoelectric conversion unit becoming saturated in the configuration of the first embodiment for a subject that periodically blinks will be described.

[0102] Fig. 11 is a timing chart showing an example of control for a predetermined pixel of the image sensor 102 according to the third embodiment, and Fig. 12 is a continuation of the timing chart of Fig. 11. Figs. 11 and 12 show an example of a timing chart of control for a predetermined pixel of the image sensor 102 that reduces the possibility that only the photoelectric conversion unit and the predetermined charge holding unit become saturated when the configurations of Figs. 2 and 3 described in the first embodiment are used. An explanation will be given for the Nth frame, which is a predetermined frame.

[0103] At time t0, RES φ309 goes high and the reset unit 309 turns on, starting resetting the charge addition unit 206.

[0104] At time t1, TX_A2 φ306 and TX_B2 φ307 go to Hi, and resetting of the first charge holding unit 204 and the second charge holding unit 205 begins.

[0105] At time t2, TX_A2 φ306 and TX_B2 φ307 go to Lo, and resetting of the first charge holding unit 204 and the second charge holding unit 205 ends.

[0106] At time t3, RES φ309 goes to Lo, the reset unit 309 turns off, and the reset of the charge addition unit 206 ends. In addition, TX_A1 φ202 goes to Hi, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 starts.

[0107] At time t4, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. As a result, the charge accumulated in the photoelectric conversion unit 201 is transferred to the first charge holding unit 204 during the period (Tdiv) from time t-1 to t4.

[0108] At time t5, SEL φ311 goes high, and reading out to the vertical output line 312 of the noise components of the charge adder 206 that completed resetting at time t3 is started.

[0109] At time t6, SEL φ311 goes to Lo, completing the readout of the noise components of the charge adder 206 that completed resetting at time t3 to the vertical output line 312. This operation completes the readout of the noise components during resetting in the Nth frame.

[0110] At time t7, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0111] At time t8, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charge accumulated in the photoelectric conversion unit 201 is transferred to the second charge holding unit 205 during the period (Tdiv) from time t4 to t8.

[0112] At time t9, TX_A1 φ202 goes high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 begins.

[0113] At time t10, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. As a result, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t8 to t10 is newly transferred to the first charge holding unit 204.

[0114] At time t11 in FIG. 12, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0115] At time t12, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t10 to t12 is newly transferred to the second charge holding unit 205.

[0116] At time t13, TX_A1 φ202 goes high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 starts.

[0117] At time t14, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. At this time, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t12 to t14 is newly transferred to the first charge holding unit 204. In the Nth frame in FIG. 11 and FIG. 12, the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 is completed by the charge transfer described above.

[0118] At time t15, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the charge addition unit 206 begins.

[0119] At time t16, TX_A2 φ306 goes to Lo, and the readout of charge from the first charge holding unit 204 to the charge addition unit 206 ends.

[0120] At time t17, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0121] At time t18, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. At this time, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t14 to t18 is newly transferred to the second charge holding unit 205. In the Nth frame in FIG. 11 and FIG. 12, the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 is completed by the charge transfer described above.

[0122] At time t19, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the charge addition unit 206 begins.

[0123] At time t20, TX_B2 φ307 goes to Lo, and the readout of charge from the second charge holding unit 205 to the charge addition unit 206 ends.

[0124] At time t21, SEL φ311 goes high, and reading out of the electric signals of the charges accumulated in the charge adder 206 to the vertical output line 312 begins.

[0125] At time t22, SEL φ311 goes to Lo, completing the readout of the electrical signal of the charge accumulated in the charge summation unit 206 to the vertical output line 312. This operation completes the readout of the signal component obtained by adding up the charges in the first charge holding unit 204 and the second charge holding unit 205 in the Nth frame in FIGS.

[0126] As described above, in the third embodiment, the charge accumulated in the photoelectric conversion unit 201 is transferred alternately to the first charge holding unit 204 and the second charge holding unit 205 multiple times in a time-division manner. That is, in the third embodiment, the multiple signal holding units each receive and hold the signal photoelectrically converted in the photoelectric conversion unit multiple times from the photoelectric conversion unit. This makes it possible to reduce the possibility that the photoelectric conversion unit 201 will become saturated.

[0127] 11 and 12, each accumulation period Tdiv is equal to the length obtained by dividing Tnd shown in the first embodiment by the number of time-division transfers. Therefore, the charges accumulated in the photoelectric conversion unit 201 are transferred to the first charge holding unit 204 and the second charge holding unit 205 substantially for the period Tnd, respectively, and it is possible to obtain the same effect as in the first embodiment with respect to suppressing light attenuation and a decrease in the SNR.

[0128] <Embodiment 4> Next, Figure 13 is a timing chart showing an example of a case where addition is performed in the signal addition unit of the image processing unit by time-division transfer from the photoelectric conversion unit of embodiment 4, and Figure 14 is a continuation of the timing chart of Figure 13.

[0129] 13 and 14 show examples of timing charts of control for a predetermined pixel of the image sensor 102, which can reduce the possibility that only the photoelectric conversion unit and the predetermined charge holding unit are saturated when using the configurations shown in Figs. 7 and 8 described in embodiment 2. The Nth frame, which is a predetermined frame, will be described.

[0130] At time t0, RES_A φ809 goes high, turning on the reset unit 809, and resetting of the floating diffusion layer 807 begins.

[0131] At time t1, TX_A2 φ306 goes high, and resetting of the first charge holding unit 204 begins.

[0132] At time t2, TX_A2 φ306 goes low, and the reset of the first charge holding unit 204 ends.

[0133] At time t3, RES_A φ809 goes to Lo, turning off the reset unit 809 and ending the reset of the floating diffusion layer 807. In addition, TX_A1 φ202 goes to Hi, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 starts.

[0134] At time t4, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. As a result, the charge accumulated in the photoelectric conversion unit 201 is transferred to the first charge holding unit 204 during the period (Tdiv) from time t-1 to t4.

[0135] At time t5, RES_B φ812 goes high, turning on the reset unit 812, and resetting of the floating diffusion layer 808 begins.

[0136] At time t6, TX_B2 φ307 goes high, and resetting of the second charge holding unit 205 starts.

[0137] At time t7, TX_B2 φ307 goes to Lo, and the resetting of the second charge holding unit 205 ends.

[0138] At time t8, RES_B φ812 goes to Lo, and the reset unit 812 turns off, thereby completing the reset of the floating diffusion layer 808.

[0139] At time t9, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0140] At time t10, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charge accumulated in the photoelectric conversion unit 201 is transferred to the second charge holding unit 205 during the period (Tdiv) from time t4 to t10.

[0141] At time t11 in FIG. 14, TX_A1 φ202 goes high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 begins.

[0142] At time t12, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. As a result, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t10 to t12 is newly transferred to the first charge holding unit 204.

[0143] At time t13, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0144] At time t14, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charge accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t12 to t14 is newly transferred to the second charge holding unit 205.

[0145] At time t15, SEL_A φ811 goes high, and reading out to the vertical output line 312 of the noise components of the floating diffusion layer 807 that was reset at time t3 is started.

[0146] At time t16, SEL_A φ811 goes to Lo, completing the readout of the noise components of the floating diffusion layer 807 that was reset at time t3 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset in the floating diffusion layer 807 of the Nth frame in FIG.

[0147] At time t17, TX_A1 φ202 goes high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 starts.

[0148] At time t18, TX_A1 φ202 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204. As a result, the charges accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t14 to t18 are newly transferred to the first charge holding unit 204. Also, in the Nth frame in FIGS. 13 and 14, the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 is completed by the charge transfer.

[0149] At time t19, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the floating diffusion layer 807 begins.

[0150] At time t20, TX_A2 φ306 goes to Lo, and the readout of charges from the first charge holding unit 204 to the floating diffusion layer 807 ends.

[0151] At time t21, SEL_A φ811 goes high, and reading out of the electric signal of the charge stored in the first charge holding unit 204 to the vertical output line 312 begins.

[0152] At time t22, SEL_A φ811 goes to Lo, completing the readout of the electric signal of the charge accumulated in the first charge holding unit 204 to the vertical output line 312. This operation completes the readout of the signal component (including the noise component) from the first charge holding unit 204 in the Nth frame in FIGS. 13 and 14.

[0153] At time t23, SEL_B φ814 goes high, and the noise components of the floating diffusion layer 808, which was reset at time t3, start to be read out to the vertical output line 312.

[0154] At time t24, SEL_B φ814 goes Lo, completing the readout of the noise components of the floating diffusion layer 808 that was reset at time t10 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset in the floating diffusion layer 808 of the Nth frame in FIGS.

[0155] At time t25, TX_B1 φ203 goes high, and charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 starts.

[0156] At time t26, TX_B1 φ203 becomes Lo, ending the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205. As a result, the charges accumulated in the photoelectric conversion unit 201 during the period (Tdiv) from time t18 to t26 are newly transferred to the second charge holding unit 205. Also, in the Nth frame in Figures 13 and 14, the charge transfer from the photoelectric conversion unit 201 to the second charge holding unit 205 is completed by the above charge transfer.

[0157] At time t27, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the floating diffusion layer 808 begins.

[0158] At time t28, TX_B2 φ307 goes to Lo, and the readout of charges from the second charge holding unit 205 to the floating diffusion layer 808 ends.

[0159] At time t29, SEL_B φ814 goes high, and reading out of the electrical signal of the charge stored in the second charge holding unit 205 to the vertical output line 312 begins.

[0160] At time t30, SEL_B φ814 goes to Lo, completing the readout of the electrical signal of the charge accumulated in the second charge holding unit 205 to the vertical output line 312. This operation completes the readout of the signal component of the second charge holding unit 205 in the Nth frame in FIGS. 13 and 14.

[0161] As explained above in the same manner as in the explanations of Figures 11 and 12, by alternately transferring the charge accumulated in the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205 in a time-division manner, it is possible to reduce the possibility of the photoelectric conversion unit 201 becoming saturated.

[0162] 13 and 14, Tdiv is equal to the length obtained by dividing Tnd shown in the second embodiment by the number of time-division transfers. This is equivalent to the charge accumulated in the photoelectric conversion unit 201 being transferred to the first charge holding unit 204 and the second charge holding unit 205 for the period of Tnd, respectively, so that it is possible to obtain the same effect as in the second embodiment in terms of suppressing light attenuation and SNR reduction.

[0163] In the third and fourth embodiments, the number of time-division transfers is three for each charge storage unit, but the number of time-division transfers is not limited to three and may be determined arbitrarily depending on the control clock and control sequence of the image sensor, the moving speed of the subject, the blinking cycle of the light source, etc.

[0164] Also, similarly to the first and second embodiments, an example in which there are two charge holding portions has been described, but the number of charge holding portions is not limited to two, and the number of charge holding portions may be three or more.

[0165] In addition, when there are three or more charge holding sections, the same effect as that proposed in this specification can be obtained by transferring charges to two or more specified charge holding sections, so it is not necessarily necessary to transfer charges to all of the charge holding sections that the photoelectric conversion section has.

[0166] <Embodiment 5> Next, Fig. 15 is a timing chart showing an example of addition at a predetermined pixel of the image sensor according to the first embodiment, and Fig. 16 is a timing chart continuing from Fig. 15. In Fig. 15 and Fig. 16, the Nth frame, which is a predetermined frame, will be particularly described.

[0167] At time t0, RES φ309 goes high and the reset unit 309 turns on, starting resetting the charge addition unit 206.

[0168] At time t1, TX_A2 φ306 and TX_B2 φ307 go to Hi, and resetting of the first charge holding unit 204 and the second charge holding unit 205 begins.

[0169] At time t2, TX_A2 φ 306 and TX_B2 φ 307 go to Lo, and resetting of the first charge holding unit 204 and the second charge holding unit 205 ends.

[0170] At time t3, RES φ309 goes low, the reset unit 309 turns off, and the reset of the charge addition unit 206 ends.

[0171] At time t4, SEL φ311 goes high, and reading out to the vertical output line 312 of the noise components of the charge adder 206 that completed resetting at time t3 is started.

[0172] At time t5, the SEL φ311 goes to Lo, completing the readout of the noise components of the charge adder 206 that completed resetting at time t3 to the vertical output line 312. This operation completes the readout of the noise components during resetting in the Nth frame.

[0173] At time t 6 , TX_A 1 φ 202 and TX_B 1 φ 203 go to Hi, and the charge stored in the photoelectric conversion unit 201 starts to be transferred to the first charge holding unit 204 and the second charge holding unit 205 .

[0174] At time t7, TX_A1 φ202 and TX_B1 φ203 become Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t7 are transferred approximately evenly to the first charge holding unit 204 and the second charge holding unit 205.

[0175] At time t8, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the charge addition unit 206 begins.

[0176] At time t9, TX_A2 φ306 goes to Lo, and the readout of the charges from the first charge holding unit 204 to the charge addition unit 206 ends.

[0177] At time t10, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the charge addition unit 206 begins.

[0178] At time t11, TX_B2 φ307 goes to Lo, and the readout of charge from the second charge holding unit 205 to the charge addition unit 206 ends.

[0179] At time t12, the SEL φ311 goes high, and the electric signal of the charge accumulated in the charge adder 206 starts to be read out to the vertical output line 312.

[0180] At time t13, the SEL φ311 goes to Lo, and the readout of the electric signals of the charges accumulated in the charge adder 206 to the vertical output line 312 ends.

[0181] Note that the signal read out during the period from time t12 to t13 contains a noise component, and this noise signal has a predetermined correlation with the noise component read out during the period from time t4 to t5. This operation completes the reading of the signal component (including the noise component) obtained by adding up the first charge holding unit 204 and the second charge holding unit 205 in the Nth frame of FIG.

[0182] As described above, the noise components are read out when the first charge holding unit 204, the second charge holding unit 205, and the charge addition unit 206 are reset, and the signal components (including noise components) of the charges accumulated in the first charge holding unit 204 and the second charge holding unit 205 are read out.

[0183] This makes it possible to perform CDS (Correlated Double Sampling) in the column amplifier subsequent to the vertical output line 312, and to reduce (subtract) noise components that occur during resetting.

[0184] In addition, although the present embodiment describes a predetermined pixel, it is also applicable to a plurality of pixels arranged one-dimensionally and a plurality of pixels arranged two-dimensionally. For example, in a pixel array of m rows x n columns, each column of the mth row is commonly controlled at the same timing, and the (m+1)th row is read out by shifting each column by the same predetermined time so that the readout timing of the vertical output line 312 does not overlap. In other words, the present embodiment is not limited to the pixel array.

[0185] Moreover, the SNR relationships in the charge holding section, the adder section, and the divider section according to this embodiment are the same as those in FIG.

[0186] As described above, in the fifth embodiment, in a configuration in which the photoelectric conversion unit has a plurality of charge storage units, the charges generated in the photoelectric conversion unit are transferred to each charge storage unit at the same timing, and arithmetic averaging is performed by addition in the adder unit and division in the divider unit. This makes it possible to reduce the decrease in SNR while obtaining a dimming effect corresponding to the number of charge storage units transferred simultaneously.

[0187] <Embodiment 6> In the fifth embodiment, the addition of signals from the charge storage units is performed in a floating diffusion layer disposed in the image sensor. However, when the circuit scale of the image sensor is taken into consideration, the addition does not necessarily have to be performed in the floating diffusion layer. That is, in the sixth embodiment, an example in which signals from each charge storage unit are added in an image processing unit will be described. Note that the description of the contents common to the fifth embodiment will be omitted as appropriate.

[0188] Here, the configurations of the image sensor and image processing unit according to the sixth embodiment are the same as those in Fig. 7, and therefore the description thereof will be omitted. Also, the configuration of the pixel of the image sensor according to the sixth embodiment is the same as that in Fig. 8, and therefore the description thereof will be omitted.

[0189] Next, Fig. 17 is a timing chart showing an example of a case where addition is performed in a signal addition unit of an image processing unit according to the sixth embodiment, and Fig. 18 is a timing chart continued from Fig. 17. In Fig. 17 and Fig. 18, an explanation will be given for an Nth frame which is a predetermined frame.

[0190] At time t0, RES_A φ809 goes high, turning on the reset unit 809, and resetting of the floating diffusion layer 807 begins.

[0191] At time t1, TX_A2 φ306 goes high, and resetting of the first charge holding unit 204 begins.

[0192] At time t2, TX_A2 φ306 goes low, and the reset of the first charge holding unit 204 ends.

[0193] At time t3, RES_A φ809 goes to Lo, and the reset unit 809 turns off, thereby completing the reset of the floating diffusion layer 807.

[0194] At time t4, RES_B φ812 goes high, turning on the reset unit 812, and resetting of the floating diffusion layer 808 begins.

[0195] At time t5, TX_B2 φ307 goes high, and resetting of the second charge holding unit 205 starts.

[0196] At time t6, TX_B2 φ307 goes to Lo, and the resetting of the second charge holding unit 205 ends.

[0197] At time t7, RES_B φ812 goes to Lo, and the reset unit 812 turns off, thereby completing the reset of the floating diffusion layer 808.

[0198] At time t8, TX_A1 φ202 and TX_B1 φ203 go high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205 begins.

[0199] At time t9, TX_A1 φ202 and TX_B1 φ203 become Lo, ending the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t9 are transferred approximately evenly to the first charge holding unit 204 and the second charge holding unit 205.

[0200] At time t10, SEL_A φ811 goes high, and the noise components of the floating diffusion layer 807 after the reset is completed at time t3 are started to be read out to the vertical output line 312.

[0201] At time t11, SEL_A φ811 becomes Lo, completing the readout of the noise components in the floating diffusion layer 807 immediately after the reset at time t3 to the vertical output line 312. This operation completes the readout of the noise components in the floating diffusion layer 807 immediately after the reset of the Nth frame.

[0202] At time t12, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the floating diffusion layer 807 begins.

[0203] At time t13, TX_A2 φ306 goes to Lo, and the readout of charges from the first charge holding unit 204 to the floating diffusion layer 807 ends.

[0204] At time t14, SEL_A φ811 goes high, and reading out the electrical signal of the charge stored in the first charge holding unit 204 to the vertical output line 312 begins.

[0205] At time t15, SEL_A φ811 goes Lo, completing the readout of the electric signal of the charge accumulated in the first charge holding unit 204 to the vertical output line 312. This operation completes the readout of the signal component (including the noise component) of the first charge holding unit 204 in the Nth frame.

[0206] At time t16, SEL_B φ814 goes high, and the noise components of the floating diffusion layer 808, which was reset at time t7, start to be read out to the vertical output line 312.

[0207] At time t17, SEL_B φ814 goes Lo, completing the readout of the noise components of the floating diffusion layer 808 that was reset at time t7 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset in the floating diffusion layer 808 of the Nth frame.

[0208] At time t18, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the floating diffusion layer 808 begins.

[0209] At time t19, TX_B2 φ307 goes to Lo, and the readout of charges from the second charge holding unit 205 to the floating diffusion layer 808 ends.

[0210] At time t20, SEL_B φ814 goes high, and reading out the electrical signal of the charge stored in the second charge holding unit 205 to the vertical output line 312 begins.

[0211] At time t21, SEL_B φ814 goes Lo, completing the readout of the electrical signal of the charge accumulated in the second charge holding unit 205 to the vertical output line 312. This operation completes the readout of the signal components (including noise components) of the second charge holding unit 205 in the Nth frame.

[0212] As described above, in the sixth embodiment, the reset noise components of the first charge retention unit 204 and the floating diffusion layer 807 and the signal components (including noise components) of the first charge retention unit 204 are successively read out. Then, the reset noise components of the second charge retention unit 205 and the floating diffusion layer 808 and the signal components (including noise components) of the second charge retention unit 205 are read out, whereby the noise components can be reduced (subtracted) by CDS in the column amplifier at the subsequent stage, as in the first embodiment.

[0213] Although only a predetermined pixel has been described in the sixth embodiment, the sixth embodiment can also be applied to a plurality of pixels arranged one-dimensionally or a plurality of pixels arranged two-dimensionally, as in the fifth embodiment. Then, by performing arithmetic averaging by adding signals read from the image sensor 102 in the signal adding unit 701 in the image processing unit 103 and dividing the signals in the dividing unit 207, it is possible to obtain the same light reduction effect and the effect of suppressing SNR reduction as described in FIG.

[0214] As described above, even when an adder unit is provided in the image processing unit as in embodiment 6, it is possible to suppress a decrease in SNR while obtaining a light reduction effect corresponding to the number of allocated charge holding units. Note that the image processing unit 103 may be provided in a semiconductor substrate having a multilayer structure together with the image sensor 102.

[0215] <Embodiment 7> In the fifth and sixth embodiments, the configurations in which signals stored in a plurality of charge storage units are added by the image sensor or the image processor are described. However, in both embodiments, the configurations are such that photoelectric conversion is performed in the photoelectric conversion unit for a predetermined period of time, and then charge is transferred to the plurality of charge storage units at the same timing.

[0216] In other words, charge transfer to each charge storage unit in a given pixel is only performed at a given timing in a given frame. Therefore, when a subject illuminated with a flash of light using a strobe or the like is photographed, the photoelectric conversion unit may become saturated during the charge accumulation period for transferring charge to each charge storage unit.

[0217] In the seventh embodiment, therefore, a configuration and method for reducing the possibility of the photoelectric conversion unit becoming saturated in the configuration of the fifth embodiment when a subject is irradiated with a flash of light will be described.

[0218] Fig. 19 is a timing chart showing an example of control for a predetermined pixel of the image sensor 102 according to the seventh embodiment, and Fig. 20 is a continuation of the timing chart of Fig. 19. Figs. 19 and 20 show an example of a timing chart of control for a predetermined pixel of the image sensor 102, which reduces the possibility of saturating the photoelectric conversion unit when using the configurations of Figs. 2 and 3 described in the first embodiment. The Nth frame, which is a predetermined frame, will be described.

[0219] At time t0, RES φ309 goes high and the reset unit 309 turns on, starting resetting the charge addition unit 206.

[0220] At time t1, TX_A2 φ306 and TX_B2 φ307 go to Hi, and resetting of the first charge holding unit 204 and the second charge holding unit 205 begins.

[0221] At time t2, TX_A2 φ306 and TX_B2 φ307 go to Lo, and resetting of the first charge holding unit 204 and the second charge holding unit 205 ends.

[0222] At time t3, RES φ309 goes low, the reset unit 309 turns off, and the reset of the charge addition unit 206 ends.

[0223] At time t4, TX_A1 φ202 and TX_B1 φ203 go high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205 begins.

[0224] At time t5, SEL φ311 goes high, and reading out to the vertical output line 312 of the noise components of the charge adder 206 that completed resetting at time t3 is started.

[0225] At time t6, SEL φ311 goes Lo, completing the readout of the noise components of the charge adder 206 that completed resetting at time t3 to the vertical output line 312. This operation completes the readout of the noise components at the time of resetting in the Nth frame.

[0226] At time t7, TX_A1 φ202 and TX_B1 φ203 become Lo, terminating the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t7 are transferred approximately evenly to the first charge holding unit 204 and the second charge holding unit 205.

[0227] At time t8, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the charge addition unit 206 begins.

[0228] At time t9, TX_A2 φ306 goes to Lo, and the readout of the charges from the first charge holding unit 204 to the charge addition unit 206 ends.

[0229] At time t10, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the charge addition unit 206 begins.

[0230] At time t11, TX_B2 φ307 goes to Lo, and the readout of charge from the second charge holding unit 205 to the charge addition unit 206 ends.

[0231] At time t12, the SEL φ311 goes high, and the electric signal of the charge accumulated in the charge adder 206 starts to be read out to the vertical output line 312.

[0232] At time t13, SEL φ311 goes to Lo, and the readout of the electric signals of the charges accumulated in the charge adder 206 to the vertical output line 312 ends.

[0233] That is, charges are transferred from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205, except for the period when the first charge holding unit 204, the second charge holding unit 205, and the charge summation unit 206 are reset, and the period when charges and electrical signals are read out from the first charge holding unit 204, the second charge holding unit 205, and the charge summation unit 206.

[0234] That is, in the seventh embodiment, since electric charges are constantly transferred to the multiple signal holding units except for a predetermined period, electric charges are stored up to the saturation capacity of the multiple signal holding units, not the saturation capacity of the photoelectric conversion unit, which makes it possible to reduce the possibility of the photoelectric conversion unit 201 becoming saturated.

[0235] <Embodiment 8> Next, FIG. 21 is a timing chart showing an example of control over a predetermined pixel of the image sensor 102 according to the eighth embodiment, and FIG. 22 is a continuation of the timing chart in FIG.

[0236] 21 and 22 show examples of timing charts of control for a predetermined pixel of the image sensor 102, which can reduce the possibility of saturating the photoelectric conversion unit when using the configurations shown in Figs. 7 and 8 described in embodiment 2. A description will be given for the Nth frame, which is a predetermined frame.

[0237] At time t0, RES_A φ809 and RES_B φ812 go to Hi, and the reset unit 809 and the reset unit 812 turn on, thereby starting to reset the floating diffusion layer 807 and the floating diffusion layer 808.

[0238] At time t1, TX_A2 φ306 and TX_B2 φ307 go high, and resetting of the first charge holding unit 204 and the second charge holding unit 205 begins.

[0239] At time t2, TX_A2 φ306 and TX_B2 φ307 go to Lo, and resetting of the first charge holding unit 204 and the second charge holding unit 205 ends.

[0240] At time t3, RES_A φ809 and RES_B φ812 go to Lo, and the reset unit 809 and the reset unit 812 turn off, thereby completing the reset of the floating diffusion layer 807 and the floating diffusion layer 808.

[0241] At time t4, TX_A1 φ202 and TX_B1 φ203 go high, and charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205 begins.

[0242] At time t5, TX_A1 φ202 and TX_B1 φ203 become Lo, terminating the charge transfer from the photoelectric conversion unit 201 to the first charge holding unit 204 and the second charge holding unit 205. At this time, the charges accumulated in the photoelectric conversion unit 201 during the period (Tnd) from time t-1 to t9 are transferred approximately evenly to the first charge holding unit 204 and the second charge holding unit 205.

[0243] At time t6, SEL_A φ811 goes high, and the noise components of the floating diffusion layer 807 after the reset is completed at time t3 are started to be read out to the vertical output line 312.

[0244] At time t7, SEL_A φ811 becomes Lo, completing the readout of the noise components in the floating diffusion layer 807 immediately after the reset at time t3 to the vertical output line 312. This operation completes the readout of the noise components in the floating diffusion layer 807 immediately after the reset of the Nth frame.

[0245] At time t8, TX_A2 φ306 goes high, and reading of charges from the first charge holding unit 204 to the floating diffusion layer 807 begins.

[0246] At time t9, TX_A2 φ306 goes to Lo, and the readout of charges from the first charge holding unit 204 to the floating diffusion layer 807 ends.

[0247] At time t10, SEL_A φ811 goes high, and reading out the electrical signal of the charge stored in the first charge holding unit 204 to the vertical output line 312 begins.

[0248] At time t11, SEL_A φ811 goes to Lo, completing the readout of the electric signal of the charge accumulated in the first charge holding unit 204 to the vertical output line 312. This operation completes the readout of the signal component (including the noise component) of the first charge holding unit 204 in the Nth frame.

[0249] At time t12, SEL_B φ814 goes high, and the noise components of the floating diffusion layer 808, which was reset at time t3, start to be read out to the vertical output line 312.

[0250] At time t13, SEL_B φ814 goes Lo, completing the readout of the noise components of the floating diffusion layer 808 that was reset at time t3 to the vertical output line 312. This operation completes the readout of the noise components at the time of reset in the floating diffusion layer 808 of the Nth frame.

[0251] At time t14, TX_B2 φ307 goes high, and reading of charges from the second charge holding unit 205 to the floating diffusion layer 808 begins.

[0252] At time t15, TX_B2 φ307 goes to Lo, and the readout of charges from the second charge holding unit 205 to the floating diffusion layer 808 ends.

[0253] At time t16, SEL_B φ814 goes high, and reading out of the electrical signal of the charge stored in the second charge holding unit 205 to the vertical output line 312 begins.

[0254] At time t17, SEL_B φ814 goes to Lo, completing the readout of the electrical signal of the charge accumulated in the second charge holding unit 205 to the vertical output line 312. This operation completes the readout of the signal components (including noise components) from the second charge holding unit 205 in the Nth frame.

[0255] 19 and 20, charge is constantly transferred to the multiple signal holding units except for a specific period, so that charge is stored up to the saturation capacity of the multiple signal holding units, not the saturation capacity of the photoelectric conversion unit. This makes it possible to reduce the possibility of the photoelectric conversion unit 201 becoming saturated.

[0256] In the seventh and eighth embodiments, the charge is transferred to each charge storage unit once, but the number of transfers is not limited to one, and the possibility of saturating the photoelectric conversion unit may be reduced by time-division transfer at simultaneous timing. When time-division transfer is performed, it may be determined arbitrarily according to the control clock or control sequence of the image sensor, the moving speed of the subject, the timing of the occurrence of a flash, etc.

[0257] Also, similarly to the fifth and sixth embodiments, an example in which there are two charge holding portions has been described, but the number of charge holding portions is not limited to two, and the number of charge holding portions may be three or more.

[0258] In addition, when there are three or more charge holding sections, the same effect as in this embodiment can be obtained by transferring charges to two or more specified charge holding sections, so it is not necessarily necessary to transfer charges to all of the charge holding sections that the photoelectric conversion section has.

[0259] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to the above-mentioned embodiments, and various modifications and combinations of the above-mentioned embodiments are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention includes the following combinations.

[0260] (Configuration 1) An imaging device comprising: a photoelectric conversion unit; a plurality of signal holding units for respectively holding signals photoelectrically converted by the photoelectric conversion units; an adder unit for adding up the signals of the plurality of signal holding units; and a division unit for dividing the signal added up by the adder unit by the number of the plurality of signal holding units added up by the adder unit.

[0261] (Configuration 2) The imaging device according to configuration 1, wherein the plurality of signal holding sections simultaneously receive and hold the signals photoelectrically converted by the photoelectric conversion sections from the photoelectric conversion sections.

[0262] (Configuration 3) The imaging device described in Configuration 2, wherein the multiple signal holding units simultaneously receive and hold the signals photoelectrically converted by the photoelectric conversion units from the photoelectric conversion units during a period in which the photoelectric conversion units are performing photoelectric conversion.

[0263] (Configuration 4) The plurality of signal holding units hold the signals photoelectrically converted by the photoelectric conversion unit. 2. The imaging device according to configuration 1, wherein the image is received from the conversion unit a plurality of times and held. (Configuration 5) The imaging device according to any one of configurations 1 to 3, wherein the adder performs analog addition of the signals from the signal holding unit.

[0264] (Configuration 6) The imaging device according to configuration 5, wherein the adder section includes a floating diffusion layer for adding up the charges photoelectrically converted by the photoelectric conversion section.

[0265] (Configuration 7) The imaging device according to any one of configurations 1 to 6, wherein the adder adds the signals from the signal holding units after AD conversion.

[0266] (Method) An imaging method using an imaging device having a photoelectric conversion unit, a plurality of signal holding units for respectively holding signals photoelectrically converted by the photoelectric conversion units, and an adder unit for adding up the signals of the plurality of signal holding units, the imaging method comprising a division step of dividing the signal added up in the adder unit by the number of the plurality of signal holding units added up in the adder unit as a divisor.

[0267] (Program) A computer program for controlling each unit of the imaging device according to any one of configurations 1 to 7 by a computer.

[0268] In order to realize a part or all of the control in the above-mentioned embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to an imaging device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the imaging device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0269] 101 Optical Lens 102 Imaging element 103...Image processing unit 104 Signal recording unit 105...Control section 201...Photoelectric conversion unit 202 First transfer section 203 Second transfer section 204...First charge holding section 205...Second charge holding section 206 Charge summing section 207...Division section 306...Third transfer section 307 4th Transfer Section 309 Reset section 310 Source follower section 311···Select section 312 Vertical output line 701 Signal addition section 807 Floating diffusion layer 808 Floating diffusion layer 809 Reset section 810 Source follower section 811···Select section 812 Reset section 813 Source follower section 814···Select section

Claims

1. A photoelectric conversion unit; a plurality of signal holding units for holding the signals photoelectrically converted by the photoelectric conversion units; an adder that adds up the signals of the plurality of signal holding units; a division unit that divides the signal added by the addition unit by the number of the signal holding units added by the addition unit as a divisor; An imaging device comprising:

2. 2. The imaging device according to claim 1, wherein the plurality of signal holding sections simultaneously receive and hold the signals photoelectrically converted by the photoelectric conversion section from the photoelectric conversion section.

3. 3. The imaging device according to claim 2, wherein the plurality of signal holding units simultaneously receive and hold the signals photoelectrically converted by the photoelectric conversion units from the photoelectric conversion units during a period in which the photoelectric conversion units are performing photoelectric conversion.

4. The plurality of signal holding units hold the signals photoelectrically converted by the photoelectric conversion unit from the photoelectric conversion unit.

2. The imaging device according to claim 1, wherein each of the signals is received and held a plurality of times.

5. 2. The image pickup apparatus according to claim 1, wherein the adder performs analog addition of the signals from the signal holding units.

6. 6. The imaging device according to claim 5, wherein the adder section includes a floating diffusion layer for adding up the charges photoelectrically converted by the photoelectric conversion section.

7. 2. The image pickup apparatus according to claim 1, wherein the adder adds the signals from the signal holding units after AD conversion.

8. A photoelectric conversion unit; a plurality of signal holding units for holding the signals photoelectrically converted by the photoelectric conversion units; an adder that adds up the signals of the plurality of signal holding units, a division step of dividing the signal added by the adder using as a divisor the number of the plurality of signal holding units added by the adder.

9. A computer program for controlling each unit of the imaging device according to any one of claims 1 to 7 by a computer.

Images