Photoelectric conversion apparatus, control method thereof, image capturing apparatus, program, and storage medium

Abstract

To bring a total charge accumulation period close to a charge accumulation period which is arbitrarily set, in a photoelectric conversion apparatus that transfers charges from a charge storage section to a plurality of charge storage sections a plurality of times.SOLUTION: A photoelectric conversion apparatus includes: a plurality of pixels each including photoelectric conversion means that converts incident light into charges and accumulates the charges and a plurality of first holding means; control means that controls a charge accumulation period in the photoelectric conversion means in predetermined time units; and setting means that sets a charge accumulation period in one frame period. The control means performs control to transfer charges from the photoelectric conversion means to the plurality of first holding means in turn a plurality of times during each frame period, and controls the charge accumulation period in the photoelectric conversion means such that a difference between the set charge accumulation period and a total of divided charge accumulation periods each for accumulating charges to be transferred the plurality of times in each frame period becomes shorter than the time units.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device and a control method thereof, an imaging device, a program, and a storage medium, and more particularly to a control technique for charge accumulation time in a photoelectric conversion device. [Background technology]

[0002] Some so-called CMOS image sensors have a global shutter (GS) function by having a charge storage section in each pixel. The pixels of such image sensors have a gate that transfers the signal charge accumulated in the photoelectric conversion section to the charge storage section. In other words, the GS function is realized by transferring the signal charge from the photoelectric conversion section to the charge storage section simultaneously for all pixels, and making the timing of the start and end of charge accumulation in the photoelectric conversion section the same for all pixels.

[0003] On the other hand, there is an image sensor that configures multiple charge storage units for one photoelectric conversion unit and transfers charges to each charge storage unit multiple times during one frame period, making it possible to obtain multiple images with different total accumulation times of the charges transferred to each charge storage unit.The dynamic range can be improved by synthesizing the multiple images obtained.

[0004] Patent Document 1 discloses a configuration in which a single photoelectric conversion unit has a plurality of charge holding units. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US Patent Application Publication No. 2013 / 0135486 Summary of the Invention [Problem to be solved by the invention]

[0006] In the configuration described in Patent Document 1, charges are transferred from the photoelectric conversion unit to the multiple charge storage units in multiple times, but the charge storage time in the photoelectric conversion unit for each charge transfer from the photoelectric conversion unit to each of the multiple charge storage units is constant. In this case, the total charge storage time of the charges transferred from the photoelectric conversion unit to the multiple charge storage units is the sum of the integral multiples of the charge storage time in each of the multiple charge storage units depending on the number of transfers. Since the charge storage time in the photoelectric conversion unit is, for example, an integral multiple of the minimum unit time that can be controlled by a clock signal, there is a problem that the total charge storage time may have an error of more than the minimum unit time with respect to the charge storage time desired by the user.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to bring the total charge accumulation time in a photoelectric conversion device in which charges are transferred from a photoelectric conversion unit to multiple charge accumulation units in multiple batches closer to an arbitrarily set charge accumulation time. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the photoelectric conversion device of the present invention includes a plurality of pixels, each pixel having a photoelectric conversion means that converts incident light into an electric charge and accumulates it, and a plurality of first holding means, and includes a control means that controls the charge accumulation time in the photoelectric conversion means in a predetermined time unit, and a setting means that sets the charge accumulation time in one frame period, and the control means controls the transfer of electric charge from the photoelectric conversion means to the plurality of first holding means in sequence in a plurality of transfers during one frame period, and controls the charge accumulation time in the photoelectric conversion means so that the difference between the total of the divided charge accumulation times for accumulating each of the electric charges transferred in a plurality of transfers during the one frame period and the set charge accumulation time is smaller than the time unit. Effect of the Invention

[0009] According to the present invention, in a photoelectric conversion device in which charges are transferred from a photoelectric conversion section to a plurality of charge accumulation sections in a plurality of times, the total charge accumulation time can be made to approach an arbitrarily set charge accumulation time. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a photoelectric conversion device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is an equivalent circuit diagram showing an example of the configuration of a pixel according to the first embodiment. [Diagram 3] 4 is a timing chart showing a concept of charge accumulation control in the first embodiment. [Figure 4] 5 is a timing chart showing charge read control in the first embodiment. [Diagram 5] 5 is a timing chart showing charge accumulation control in the first and second embodiments. [Figure 6] FIG. 11 is an equivalent circuit diagram showing an example of the configuration of a pixel according to a second embodiment. [Figure 7] FIG. 11 is a block diagram showing a schematic configuration of an imaging system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] <First embodiment> A first embodiment of the present invention will now be described. FIG. 1 is a block diagram showing a schematic configuration of a photoelectric conversion device 100 according to the first embodiment.

[0013] The photoelectric conversion device 100 has a pixel region 10, a vertical scanning circuit 20, a readout circuit 30, a horizontal scanning circuit 40, an output circuit 50, and a control circuit 60. The pixel region 10 is electrically connected to the vertical scanning circuit 20 and the readout circuit 30. The readout circuit 30 is electrically connected to the horizontal scanning circuit 40 and the output circuit 50. The control circuit 60 is electrically connected to the vertical scanning circuit 20, the readout circuit 30, and the horizontal scanning circuit 40.

[0014] The pixel region 10 includes a plurality of pixels 12 arranged in a matrix. Each of the pixels 12 includes a photoelectric conversion unit formed of a photoelectric conversion element such as a photodiode, and outputs a pixel signal according to the amount of incident light. The number of rows and columns of the pixel array arranged in the pixel region 10 is not particularly limited. In addition to effective pixels that output pixel signals according to the amount of incident light, the pixel region 10 may also include optical black pixels whose photoelectric conversion units are shielded from light, dummy pixels that do not output signals, and the like.

[0015] In each row of the pixel region 10, a control line 14 is arranged, extending in a first direction (the horizontal direction in FIG. 1). Each of the control lines 14 is connected to the pixels 12 aligned in the first direction, and serves as a signal line common to these pixels 12. In the following description, the first direction in which the control lines 14 extend may be referred to as the row direction or horizontal direction. Each of the control lines 14 may include a plurality of signal lines. The control lines 14 are connected to a vertical scanning circuit 20.

[0016] In each column of the pixel region 10, a vertical output line 16 is arranged, which extends in a second direction (the vertical direction in FIG. 1) intersecting the first direction. Each of the vertical output lines 16 is connected to the pixels 12 aligned in the second direction, and serves as a common signal line for these pixels 12. In the following description, the second direction in which the vertical output lines 16 extend may be referred to as the column direction or vertical direction. The vertical output lines 16 are connected to a readout circuit 30.

[0017] The vertical scanning circuit 20 has a function of receiving a control signal from the control circuit 60, generating a control signal for driving the pixels 12, and outputting the control signal to the pixels 12 via the control line 14. The vertical scanning circuit 20 may include logic circuits such as a shift register and an address decoder. The pixel region 10 in this embodiment can be driven in a global shutter (GS) mode and a rolling shutter (RS) mode, and either mode can be selected from outside the photoelectric conversion device 100, for example, by an operation unit (not shown).

[0018] In the GS mode, the vertical scanning circuit 20 outputs control signals to the control lines 14 of each row, and drives all the pixels 12 in the pixel region 10 to accumulate charge at the same timing. On the other hand, in the RS mode, the vertical scanning circuit 20 outputs control signals to the control lines 14 of each row in sequence, and drives the pixels 12 in the pixel region 10 to accumulate charge sequentially row by row. In either mode, signals are read out from the pixels 12 row by row, and the signals read out from the pixels 12 are input to the readout circuit 30 via the vertical output lines 16 arranged in each column of the pixel region 10.

[0019] The readout circuit 30 has a function of performing predetermined signal processing, such as amplification and addition, on the signals read out from the pixels 12. The readout circuit 30 may include a signal holding unit, a column amplifier, a correlated double sampling (CDS) circuit, an addition circuit, etc. The readout circuit 30 may further include other processing circuits, such as an analog-to-digital (A / D) conversion circuit, as necessary.

[0020] The control circuit 60 supplies control signals for controlling the operations and timings of the vertical scanning circuit 20, the readout circuit 30, and the horizontal scanning circuit 40. At least some of the control signals supplied to the vertical scanning circuit 20, the readout circuit 30, and the horizontal scanning circuit 40 may be supplied from outside the photoelectric conversion device 100.

[0021] Next, each pixel of the photoelectric conversion device 100 in this embodiment will be described with reference to FIG. FIG. 2 is an equivalent circuit diagram showing an example of the configuration of each pixel of the photoelectric conversion device 100. As shown in FIG. As shown in Fig. 2, each of the pixels 12 has a photoelectric conversion unit PD and transfer transistors M1L1, M1L2, M1S1, M1S2, M3L1, M3L2, M3S1, and M3S2. Each of the pixels 12 further has a reset transistor M4, an amplification transistor M5, a selection transistor M6, and a charge discharge transistor M7. When electrons are used as the signal charge, each of the transistors may be configured as an N-type MOS transistor. Note that each of the transistors does not necessarily have to be an N-type MOS transistor, and each of the transistors may be configured as a P-type MOS transistor and holes may be used as the signal charge.

[0022] The photoelectric conversion unit PD may be composed of a photoelectric conversion element such as a photodiode. The photoelectric conversion unit PD converts incident light into an amount of charge corresponding to the amount of light (photoelectric conversion) and accumulates the generated charge. The photoelectric conversion unit PD has an anode connected to the ground node and a cathode connected to the sources of the transfer transistors M1L1, M1L2, M1S1, and M1S2 and the charge discharging transistor M7.

[0023] The drain of the transfer transistor M1L1 is connected to the source of the transfer transistor M3L1. The connection node between the drain of the transfer transistor M1L1 and the source of the transfer transistor M3L1 includes a capacitance component and functions as a charge holding unit (charge holding unit MEM_L1). The transfer transistor M1L1 functions as a transfer unit that performs a transfer operation of transferring the charge held by the photoelectric conversion unit PD to the charge holding unit MEM_L1 when it is turned on.

[0024] Similarly, the drain of the transfer transistor M1L2 is connected to the source of the transfer transistor M3L2. The connection node between the drain of the transfer transistor M1L2 and the source of the transfer transistor M3L2 includes a capacitance component and functions as a charge holding unit (charge holding unit MEM_L2). The transfer transistor M1L2 functions as a transfer unit that performs a transfer operation of transferring the charge held by the photoelectric conversion unit PD to the charge holding unit MEM_L2 by being turned on.

[0025] The drain of the transfer transistor M1S1 is connected to the source of the transfer transistor M3S1. The connection node between the drain of the transfer transistor M1S1 and the source of the transfer transistor M3S1 includes a capacitance component and functions as a charge holding unit (charge holding unit MEM_S1). The transfer transistor M1S1 functions as a transfer unit that performs a transfer operation of transferring the charge held by the photoelectric conversion unit PD to the charge holding unit MEM_S1 when it is turned on.

[0026] Similarly, the drain of the transfer transistor M1S2 is connected to the source of the transfer transistor M3S2. The connection node between the drain of the transfer transistor M1S2 and the source of the transfer transistor M3S2 includes a capacitance component and functions as a charge holding unit (charge holding unit MEM_S2). The transfer transistor M1S2 functions as a transfer unit that performs a transfer operation of transferring the charge held by the photoelectric conversion unit PD to the charge holding unit MEM_S2 by being turned on.

[0027] The drains of the transfer transistors M3L1, M3L2, M3S1, and M3S2 are connected to the source of the reset transistor M4 and the gate of the amplification transistor M5. The connection node between the drains of the transfer transistors M3L1, M3L2, M3S1, and M3S2, the source of the reset transistor M4, and the gate of the amplification transistor M5 is a so-called floating diffusion portion FD. The floating diffusion portion FD includes a capacitance component (floating diffusion capacitance) and functions as a charge holding portion.

[0028] The transfer transistor M3L1 has a function as a transfer unit that performs a transfer operation of transferring the charge held in the charge holding unit MEM_L1 to the floating diffusion unit FD when it is turned on. The transfer transistor M3L2 has a function as a transfer unit that performs a transfer operation of transferring the charge held in the charge holding unit MEM_L2 to the floating diffusion unit FD when it is turned on. The transfer transistor M3S1 has a function as a transfer unit that performs a transfer operation of transferring the charge held in the charge holding unit MEM_S1 to the floating diffusion unit FD when it is turned on. The transfer transistor M3S2 has a function as a transfer unit that performs a transfer operation of transferring the charge held in the charge holding unit MEM_S2 to the floating diffusion unit FD when it is turned on.

[0029] The reset transistor M4 functions as a reset unit that performs a reset operation of resetting the floating diffusion unit FD to a voltage corresponding to the voltage VDD when the reset transistor M4 is turned on.

[0030] The charge drain transistor M7 functions as an overflow drain unit that drains the charge held by the photoelectric conversion unit PD when it is turned on. That is, the charge drain transistor M7 functions as a reset unit that performs a reset operation to reset the photoelectric conversion unit PD to a voltage corresponding to the voltage VDD when it is turned on.

[0031] The selection transistor M6 functions as a selection section that selects whether or not to output a signal corresponding to the source voltage of the amplification transistor M5 to the vertical output line 16 as a pixel signal.

[0032] The drain of the reset transistor M4, the drain of the amplifying transistor M5, and the drain of the charge discharging transistor M7 are connected to a power supply voltage line (voltage VDD). Any two or three of the voltages supplied to the drain of the reset transistor M4, the voltages supplied to the drain of the amplifying transistor M5, and the voltages supplied to the drain of the charge discharging transistor M7 may be the same, or all of them may be different. The source of the amplifying transistor M5 is connected to the drain of the selection transistor M6, and the source of the selection transistor M6 is connected to the vertical output line 16.

[0033] The amplifier transistor M5 is configured such that a voltage VDD is supplied to the drain, and a bias current is supplied to the source from a current source (not shown) via the selection transistor M6, forming an amplifier section (source follower circuit) with the gate as an input node. As a result, the amplifier transistor M5 outputs a signal according to the potential of the floating diffusion section FD to the vertical output line 16 via the selection transistor M6. In this sense, the floating diffusion section FD, the amplifier transistor M5, and the selection transistor M6 can be said to form an output section that outputs a signal according to the amount of charge held by the floating diffusion section FD. Note that, although FIG. 2 shows a configuration in which four transfer sections (transfer transistors M3L1, M3L2, M3S1, and M3S2) share one floating diffusion section FD or output section, a floating diffusion section FD or output section may be formed for each of the four transfer sections.

[0034] 1 includes eleven signal lines connected to the gates of the transfer transistors M1L1, M1L2, M1S1, M1S2, M3L1, M3L2, M3S1, and M3S2, the reset transistor M4, the selection transistor M6, and the charge discharging transistor M7. A control signal GS_L1 is output from the vertical scanning circuit 20 to the signal line connected to the gate of the transfer transistor M1L1, and a control signal GS_L2 is output from the vertical scanning circuit 20 to the signal line connected to the gate of the transfer transistor M1L2. A control signal GS_S1 is output from the vertical scanning circuit 20 to the signal line connected to the gate of the transfer transistor M1S1, and a control signal GS_S2 is output from the vertical scanning circuit 20 to the signal line connected to the gate of the transfer transistor M1S2.

[0035] A control signal TX_L1 is output from the vertical scanning circuit 20 to a signal line connected to the gate of the transfer transistor M3L1, and a control signal TX_L2 is output from the vertical scanning circuit 20 to a signal line connected to the gate of the transfer transistor M3L2. A control signal TX_S1 is output from the vertical scanning circuit 20 to a signal line connected to the gate of the transfer transistor M3S1, and a control signal TX_S2 is output from the vertical scanning circuit 20 to a signal line connected to the gate of the transfer transistor M3S2.

[0036] A control signal RES is output from the vertical scanning circuit 20 to a signal line connected to the gate of the reset transistor M4. A control signal SEL is output from the vertical scanning circuit 20 to a signal line connected to the gate of the selection transistor M6. A control signal OFG is output from the vertical scanning circuit 20 to a signal line connected to the gate of the charge discharging transistor M7.

[0037] When each transistor is an N-type transistor, when a high-level control signal is supplied from the vertical scanning circuit 20, the corresponding transistor is turned on, and when a low-level control signal is supplied from the vertical scanning circuit 20, the corresponding transistor is turned off.

[0038] Next, a driving method of the photoelectric conversion device 100 in this embodiment in the GS mode will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a timing chart showing the concept of pixel charge accumulation control, and Fig. 4 is a timing chart showing charge read control.

[0039] 3 shows the time changes of control signals GS_L1, GS_L2, GS_S1, GS_S2, and OFG supplied to the transfer transistors M1L1, M1L2, M1S1, and M1S2 and the charge discharging transistor M7 in the (2N-1)th frame and the 2Nth frame. Here, N is an integer equal to or greater than 1. When each control signal is at a high level, the corresponding transistor is active (on state). Note that because this is control in the GS mode, the drive and timing related to the charge accumulation control are the same for the pixels 12 in all rows.

[0040] In each frame, charge accumulation control is executed using multiple (Ks) charge accumulation times Ts (split charge accumulation times) and multiple (Kl) charge accumulation times Tl (split charge accumulation times). The charge accumulation time Ts is a period for accumulating charge in the charge holding unit MEM_S1 or the charge holding unit MEM_S2, and the charge accumulation time Tl is a period for accumulating charge in the charge holding unit MEM_L1 or the charge holding unit MEM_L2.

[0041] In FIG. 3, the charge accumulation time Ts i and the (i+1)th charge accumulation time Ts i+1 and the j-th charge accumulation time Tl j and the (j+1)th charge accumulation time Tl j+1 Here, i is an integer greater than or equal to 1 and less than or equal to Ks-2, and j is an integer greater than or equal to 1 and less than or equal to Kl-1. Note that the numbers of times Ks and Kl can be set appropriately depending on the total charge accumulation time within one frame period, and the numbers of times Ks and Kl may be the same or different.

[0042] First, the charge accumulation control in the odd-numbered frame (the (2N-1)th frame) will be described. In the odd-numbered frame, the charge generated in the photoelectric conversion unit PD is accumulated in the charge storage units MEM_L1 and MEM_S1, and signals based on the charge accumulated in the charge storage units MEM_L2 and MEM_S2 are read out.

[0043] First, immediately before time t10, the control signal OFG is set to a high level. The charge discharging transistor M7 is turned on in response to the high-level control signal OFG, and the photoelectric conversion unit PD is reset to a potential corresponding to the voltage VDD.

[0044] At time t10, the vertical scanning circuit 20 controls the control signal OFG to change from high level to low level. This turns off the charge discharging transistor M7, and the reset state of the photoelectric conversion unit PD is released. That is, at the timing when the control signal OFG changes from high level to low level, the charge accumulation time Ts i While the charge drain transistor M7 is off, the photoelectric conversion unit PD converts the incident light into an electric charge and accumulates the generated electric charge.

[0045] In the period from a predetermined timing after time t10 to time t11, the vertical scanning circuit 20 controls the control signal GS_S1 to a high level. This turns on the transfer transistor M1S1, and the charge accumulated in the photoelectric conversion unit PD is transferred to the charge holding unit MEM_S1. The time t11 at which the transfer transistor M1S1 is turned off corresponds to the charge accumulation time Ts i That is, the period from time t10 to time t11 is the end time of the charge accumulation time Ts i It is.

[0046] After time t11, the vertical scanning circuit 20 changes the control signal OFG from low level to high level, which turns on the charge discharging transistor M7 and resets the photoelectric conversion unit PD to a potential corresponding to the voltage VDD.

[0047] At the next time t12, the vertical scanning circuit 20 controls the control signal OFG to change from high level to low level. This turns off the charge discharging transistor M7, and the reset state of the photoelectric conversion unit PD is released. That is, at the timing when the control signal OFG changes from high level to low level, the charge accumulation time Tl in the photoelectric conversion unit PD is j While the charge drain transistor M7 is off, the photoelectric conversion unit PD converts the incident light into an electric charge and accumulates the generated electric charge.

[0048] Here, the photoelectric conversion unit PD is reset by turning on the charge drain transistor M7. However, in the case of a configuration in which no charge remains in the photoelectric conversion unit PD during transfer, resetting the photoelectric conversion unit PD by the charge drain transistor M7 may be omitted. In that case, the timing (time t11) when the transfer transistor (here, the transfer transistor M1S1) is turned off is the start time of the charge storage time Tlj in the photoelectric conversion unit PD. The same applies to the other charge storage times Ts and Tl. The complete transfer of charges from the photoelectric conversion unit PD to the charge holding units MEM_L1, MEM_L2, MEM_S1, and MEM_S2 can be realized by the potential design of the photoelectric conversion unit PD, the transfer transistors M1L1, M1L2, M1S1, and M1S2, and the charge holding units MEM_L1, MEM_L2, MEM_S1, and MEM_S2.

[0049] In the period from a predetermined timing after time t12 to time t13, the vertical scanning circuit 20 controls the control signal GS_L1 to a high level. This turns on the transfer transistor M1L1, and the charge stored in the photoelectric conversion unit PD is transferred to the charge holding unit MEM_L1. The time t13 at which the transfer transistor M1L1 is turned off is within the charge storage time Tl in the photoelectric conversion unit PD. j That is, the period from time t12 to time t13 is the end time of the charge accumulation time Tl j It is.

[0050] After that, the charge accumulation time Ts and the charge accumulation time Tl are repeated a predetermined number of times, similarly to the period from time t10 to time t13. For example, as shown in FIG. 3, the period from time t14 to time t15 is the charge accumulation time Ts i+1 The period from time t16 to time t17 is the charge accumulation time Tl j+1 From time t18, the charge storage time Ts i+2 It becomes.

[0051] In this way, the signal charges are divided and transferred from the photoelectric conversion unit PD to the charge storage units MEM_L1, MEM_L2, MEM_S1, and MEM_S2 during one frame period and this is hereinafter referred to as "time division transfer." In this way, in odd-numbered frames, charge accumulation control is executed using Ks charge accumulation times Ts and Kl charge accumulation times Tl. As a result, the charge storage unit MEM_S1 has a charge accumulation time Ts 1 From charge storage time Ts Ks The charge generated in the photoelectric conversion unit PD is held for a charge storage time Tshort, which is the total length of the Ks periods from the charge storage time Tl1 to the charge storage time Tl Kl The charge generated in the photoelectric conversion unit PD is held for a charge accumulation time Tlong, which is the total length of the Kl periods up to the end of the charge accumulation time Tlong.

[0052] Next, the charge accumulation control in an even frame (2Nth frame) will be described. In an even frame, the charge generated in the photoelectric conversion unit PD is accumulated in the charge storage units MEM_L2 and MEM_S2, and a signal based on the charge accumulated in the charge storage units MEM_L1 and MEM_S1 is read out.

[0053] First, immediately before time t20, the control signal OFG is set to a high level, the charge discharging transistor M7 is turned on in response to the high-level control signal OFG, and the photoelectric conversion unit PD is reset to a potential corresponding to the voltage VDD.

[0054] At time t20, the vertical scanning circuit 20 controls the control signal OFG to change from high level to low level. This turns off the charge discharging transistor M7, and the reset state of the photoelectric conversion unit PD is released. That is, at the timing when the control signal OFG changes from high level to low level, the charge accumulation time Ts i While the charge drain transistor M7 is off, the photoelectric conversion unit PD converts the incident light into an electric charge and accumulates the generated electric charge.

[0055] In the period from a predetermined timing after time t20 to time t21, the vertical scanning circuit 20 controls the control signal GS_S2 to a high level. This turns on the transfer transistor M1S2, and the charge stored in the photoelectric conversion unit PD is transferred to the charge holding unit MEM_S2. The time t21 at which the transfer transistor M1S2 turns off is within the charge storage time Ts i That is, the period from time t20 to time t21 is the end time of the charge accumulation time Ts i It is.

[0056] After time t21, the vertical scanning circuit 20 changes the control signal OFG from low level to high level, which turns on the charge discharging transistor M7 and resets the photoelectric conversion unit PD to a potential corresponding to the voltage VDD.

[0057] At the next time t22, the vertical scanning circuit 20 controls the control signal OFG to change from high level to low level. This turns off the charge discharging transistor M7, and the reset state of the photoelectric conversion unit PD is released. That is, at the timing when the control signal OFG changes from high level to low level, the charge accumulation time Tl in the photoelectric conversion unit PD is j While the charge drain transistor M7 is off, the photoelectric conversion unit PD converts the incident light into an electric charge and accumulates the generated electric charge.

[0058] In the period from a predetermined timing after time t22 to time t23, the vertical scanning circuit 20 controls the control signal GS_L2 to a high level. This turns on the transfer transistor M1L2, and the charge stored in the photoelectric conversion unit PD is transferred to the charge holding unit MEM_L2. The time t23 at which the transfer transistor M1L2 is turned off is within the charge storage time T1 in the photoelectric conversion unit PD. j That is, the period from time t22 to time t23 is the end time of the charge accumulation time Tl. j It is.

[0059] After that, the charge accumulation time Ts and the charge accumulation time Tl are repeated a predetermined number of times, similarly to the period from time t20 to time t23. For example, as shown in FIG. 3, the period from time t24 to time t25 is the charge accumulation time Ts i+1 The period from time t26 to time t27 is the charge accumulation time Tl j+1 From time t28, the charge storage time Ts i+2 It becomes.

[0060] In this manner, charge accumulation control is executed for Ks charge accumulation times Ts and Kl charge accumulation times Tl in the even-numbered frames. Ks The charge generated in the photoelectric conversion unit PD is held for a charge storage time Tshort, which is the total length of the Ks periods from the charge storage time Tl1 to the charge storage time Tl2. Kl The signal charges generated in the photoelectric conversion unit PD are held during a charge accumulation time Tlong, which is the total length of the Kl periods up to the first period.

[0061] FIG. 4 shows the time change of the control signals TX_L1, TX_L2, TX_S1, TX_S2, RES, and SEL supplied to the transfer transistors M3L1, M3L2, M3S1, and M3S2, the reset transistor M4, and the selection transistor M6 in the (2N-1)th frame and the 2Nth frame. When each control signal is at a high level, the corresponding transistor is active (on state). Note that even in the GS mode, the readout operation of the pixel signals is executed sequentially for each row, and FIG. 4 shows the control signals supplied to the pixels 12 in the nth row and the control signals supplied to the pixels 12 in the (n+1)th row among the control signals corresponding to each of the multiple rows constituting the pixel area 10. The control signals supplied to the pixels 12 in the nth row are marked with (n), and the control signals supplied to the pixels 12 in the (n+1)th row are marked with (n+1).

[0062] First, the read operation in the odd frame (the (2N-1)th frame) will be described. In the odd frame, as described above, a read operation is performed on a signal based on the charge stored in the charge storage unit MEM_L2 and the charge storage unit MEM_S2. In the (2N-1)th frame, the charge stored in the (2N-2)th frame (not shown) is stored in the charge storage unit MEM_L2 and the charge storage unit MEM_S2 of each pixel 12. The read operation in each frame can be performed in parallel with the exposure operation described above.

[0063] Immediately before time t30, the control signals TX_L1(n), TX_L2(n), TX_S1(n), TX_S2(n), and SEL(n) are at low level, and the control signal RES(n) is at high level.

[0064] At time t30, the vertical scanning circuit 20 controls the control signal SEL(n) to change from low level to high level. As a result, the selection transistor M6 of the pixel 12 in the nth row is turned on, and the amplification transistor M5 of the pixel 12 in each column in the nth row is connected to the vertical output line 16 of the corresponding column via the selection transistor M6, resulting in a selected state in which pixel signals can be read out. At this time, the reset transistor M4 is on, and the floating diffusion FD is reset to a potential corresponding to the voltage VDD. As a result, a signal corresponding to the reset potential of the floating diffusion FD is output to the vertical output line 16.

[0065] At the next time t31, the vertical scanning circuit 20 controls the control signal RES(n) to change from high level to low level. This turns off the reset transistor M4, and the reset state of the floating diffusion portion FD is released. The voltage of the vertical output line 16 that settles after turning off the reset transistor M4 is the reset level voltage VRES of the pixel 12. In this way, the reset level voltage VRES of the pixel 12 is read out to the vertical output line 16.

[0066] In the subsequent period from time t32 to time t33, the vertical scanning circuit 20 controls the control signal TX_L2(n) to change from low level to high level. This turns on the transfer transistor M3L2 of the pixel 12 in the nth row, and the charge held in the charge holding unit MEM_L2 is transferred to the floating diffusion unit FD. The floating diffusion unit FD then has a potential corresponding to the amount of charge transferred from the charge holding unit MEM_L2, and a voltage corresponding to the potential of the floating diffusion unit FD is output to the vertical output line 16. The voltage of the vertical output line 16 that settles after the transfer transistor M3L2 is turned off at time t33 is the signal level voltage VSIG of the pixel 12. Thus, the signal level voltage VSIG of the pixel 12 based on the charge held in the charge holding unit MEM_L2 is read out to the vertical output line 16.

[0067] The difference between the reset level voltage VRES and the signal level voltage VSIG thus obtained, that is, |VSIG-VRES|, is a physical quantity corresponding to the amount of charge held in the charge holding unit MEM_L2.

[0068] At the next time t34, the vertical scanning circuit 20 changes the control signal RES(n) from low to high. This turns on the reset transistor M4 of the pixel 12 in the nth row, and resets the floating diffusion FD to a potential corresponding to the voltage VDD. A signal corresponding to the reset potential of the floating diffusion FD is output to the vertical output line 16.

[0069] At the next time t35, the vertical scanning circuit 20 controls the control signal RES(n) to change from high level to low level. This turns off the reset transistor M4, and the reset state of the floating diffusion portion FD is released. The voltage of the vertical output line 16 that settles after turning off the reset transistor M4 is the reset level voltage VRES of the pixel 12. In this way, the reset level voltage VRES of the pixel 12 is read out to the vertical output line 16.

[0070] In the subsequent period from time t36 to time t37, the vertical scanning circuit 20 controls the control signal TX_S2(n) to change from low level to high level. This turns on the transfer transistor M3S2 of the pixel 12 in the nth row, and the charge held in the charge holding unit MEM_S2 is transferred to the floating diffusion unit FD. The floating diffusion unit FD then has a potential corresponding to the amount of charge transferred from the charge holding unit MEM_S2, and a voltage corresponding to the potential of the floating diffusion unit FD is output to the vertical output line 16. The voltage of the vertical output line 16 that settles after the transfer transistor M3S2 is turned off at time t37 is the signal level voltage VSIG of the pixel 12. Thus, the signal level voltage VSIG of the pixel 12 based on the charge held in the charge holding unit MEM_S2 is read out to the vertical output line 16.

[0071] The difference between the reset level voltage VRES and the signal level voltage VSIG thus obtained, that is, |VSIG-VRES|, is a physical quantity corresponding to the amount of signal charge held in the charge holding unit MEM_S2.

[0072] At the next time t38, the vertical scanning circuit 20 changes the control signal RES(n) from low level to high level, which turns on the reset transistor M4 of the pixel 12 in the nth row and resets the floating diffusion portion FD to a potential corresponding to the voltage VDD.

[0073] At the next time t39, the vertical scanning circuit 20 changes the control signal SEL(n) from high to low, thereby turning off the selection transistors M6 of the pixels 12 in the nth row, and the nth row is deselected.

[0074] Furthermore, in the period from time t39 to the following time t40, similarly to the period from time t30 to time t39, signals based on the charges stored in the charge retention units MEM_L2 and MEM_S2 are read out from the pixels 12 in the (n+1)th row. The readout operation of the pixels 12 in the other rows is also performed in a similar manner.

[0075] Next, the read operation in the even frame (2Nth frame) will be described. In the even frame, as described above, a read operation of a signal based on the charge stored in the charge storage unit MEM_L1 and the charge storage unit MEM_S1 is performed. In the 2Nth frame, the signal charge stored in the (2N-1)th frame is stored in the charge storage unit MEM_L1 and the charge storage unit MEM_S1 of each pixel 12.

[0076] Immediately before time t50, the control signals TX_L1(n), TX_L2(n), TX_S1(n), TX_S2(n), and SEL(n) are at low level, and the control signal RES(n) is at high level.

[0077] At time t50, the vertical scanning circuit 20 controls the control signal SEL(n) to change from low level to high level. As a result, the selection transistor M6 of the pixel 12 in the nth row is turned on, and the amplification transistor M5 of the pixel 12 in each column in the nth row is connected to the vertical output line 16 of the corresponding column via the selection transistor M6, resulting in a selected state in which pixel signals can be read out. At this time, the reset transistor M4 is on, and the floating diffusion FD is reset to a potential corresponding to the voltage VDD. As a result, a signal corresponding to the reset potential of the floating diffusion FD is output to the vertical output line 16.

[0078] At the next time t51, the vertical scanning circuit 20 controls the control signal RES(n) to change from high level to low level. This turns off the reset transistor M4, and the reset state of the floating diffusion portion FD is released. The voltage of the vertical output line 16 that settles after turning off the reset transistor M4 is the reset level voltage VRES of the pixel 12. In this way, the reset level voltage VRES of the pixel 12 is read out to the vertical output line 16.

[0079] In the subsequent period from time t52 to time t53, the vertical scanning circuit 20 controls the control signal TX_L1(n) to change from low level to high level. This turns on the transfer transistor M3L1 of the pixel 12 in the nth row, and the charge held in the charge holding unit MEM_L1 is transferred to the floating diffusion unit FD. The floating diffusion unit FD then has a potential corresponding to the amount of charge transferred from the charge holding unit MEM_L1, and a voltage corresponding to the potential of the floating diffusion unit FD is output to the vertical output line 16. The voltage of the vertical output line 16 that settles after the transfer transistor M3L1 is turned off at time t53 is the signal level voltage VSIG of the pixel 12. In this way, the signal level voltage VSIG of the pixel 12 based on the charge held in the charge holding unit MEM_L1 is read out to the vertical output line 16.

[0080] The difference between the reset level voltage VRES and the signal level voltage VSIG thus obtained, that is, |VSIG-VRES|, is a physical quantity according to the amount of signal charge held in the charge holding unit MEM_L1.

[0081] At the next time t54, the vertical scanning circuit 20 changes the control signal RES(n) from low to high. This turns on the reset transistor M4 of the pixel 12 in the nth row, and resets the floating diffusion FD to a potential corresponding to the voltage VDD. A signal corresponding to the reset potential of the floating diffusion FD is output to the vertical output line 16.

[0082] At the next time t55, the vertical scanning circuit 20 controls the control signal RES(n) to change from high level to low level. This turns off the reset transistor M4, and the reset state of the floating diffusion FD is released. The voltage of the vertical output line 16 that settles after turning off the reset transistor M4 is the reset level voltage VRES of the pixel 12. In this way, the reset level voltage VRES of the pixel 12 is read out to the vertical output line 16.

[0083] In the subsequent period from time t56 to time t57, the vertical scanning circuit 20 controls the control signal TX_S1(n) to change from low level to high level. This turns on the transfer transistor M3S1 of the pixel 12 in the nth row, and the charge held in the charge holding unit MEM_S1 is transferred to the floating diffusion unit FD. The floating diffusion unit FD then has a potential corresponding to the amount of charge transferred from the charge holding unit MEM_S1, and a voltage corresponding to the potential of the floating diffusion unit FD is output to the vertical output line 16. The voltage of the vertical output line 16 that settles after the transfer transistor M3S1 is turned off at time t57 is the signal level voltage VSIG of the pixel 12. Thus, the signal level voltage VSIG of the pixel 12 based on the signal charge held in the charge holding unit MEM_S1 is read out to the vertical output line 16.

[0084] The difference between the reset level voltage VRES and the signal level voltage VSIG thus obtained, that is, |VSIG-VRES|, is a physical quantity according to the amount of signal charge held in the charge holding unit MEM_S1.

[0085] At the next time t58, the vertical scanning circuit 20 changes the control signal RES(n) from low level to high level, which turns on the reset transistor M4 of the pixel 12 in the nth row and resets the floating diffusion portion FD to a potential corresponding to the voltage VDD.

[0086] At the following time t59, the vertical scanning circuit 20 changes the control signal SEL(n) from high to low, thereby turning off the selection transistors M6 of the pixels 12 in the nth row, and the selection of the nth row is released.

[0087] Furthermore, in the period from time t59 to the following time t60, similarly to the period from time t50 to time t59, signals based on the charges stored in the charge retention units MEM_L1 and MEM_S1 are read out from the pixels 12 in the (n+1)th row. The readout operation of the pixels 12 in the other rows is also performed in a similar manner.

[0088] In the driving example described with reference to FIG. 3 and FIG. 4, the length of the charge accumulation time Tlong and the length of the charge accumulation time Tshort may be the same, but are preferably different. By changing the length of the charge accumulation time Tlong and the length of the charge accumulation time Tshort, it becomes possible to acquire two types of images with different effective exposure amounts in the same frame. By correcting the signal of one of the two types of images acquired in this way according to the ratio of the length of the charge accumulation time and combining it with the signal of the other image to synthesize it into one image, an image with a wide dynamic range (HDR image) can be obtained. The synthesis process of the HDR image may be performed in a signal processing unit in the photoelectric conversion device or in an external signal processing device.

[0089] In this embodiment, the charge accumulation time Tshort is divided into Ks charge accumulation times Ts, and the charge accumulation time Tlong is divided into Kl charge accumulation times Tl, and the pixels 12 are driven alternately by these charge accumulation times. By distributing the charge accumulation time in this manner within the frame period, it is possible to suppress the period during which the charge is accumulated from being biased within the frame period. This makes it possible to suppress jitter between frame images during video shooting, compared to the case in which the pixel 12 is driven with the charge accumulation time Tshort and the charge accumulation time Tlong each being a continuous time. In particular, a remarkable effect can be obtained when shooting a subject that moves at high speed within the screen or a blinking light source. In addition, since the positions of the temporal centers of gravity of the charge accumulation time Tshort and the charge accumulation time Tlong can be brought closer together, a good quality composite image can be obtained.

[0090] Furthermore, the pixels 12 of the photoelectric conversion device 100 in this embodiment include a charge storage section that stores charge in addition to a charge storage section that stores charge in the previous frame. Therefore, even while storing charge, it is possible to read a signal based on the charge stored in the previous frame, reducing the time during which a signal cannot be obtained during each frame period, and thus enabling seamless video to be obtained.

[0091] In this way, according to this embodiment, it is possible to obtain high-quality moving images with a high dynamic range.

[0092] Up to this point, an example has been described in which the charge accumulation time Ts and the charge accumulation time Tl are constant in one frame period. However, when the charge accumulation time Ts and the charge accumulation time Tl are constant, the charge accumulation time in one frame period may not be the desired charge accumulation time. Below, an example in which the desired charge accumulation time is not achieved and a method for adjusting the charge accumulation time will be described.

[0093] Fig. 5 is a timing diagram showing the relationship between the total charge storage time in one frame period and the desired charge storage time set by the user. In each of Fig. 5(a) to (d), the horizontal axis is the time axis, and MEM_S in the upper row represents the charge storage time corresponding to the charge holding units MEM_S1 and MEM_S2 in Fig. 2. Also, MEM_L in the lower row represents the charge storage time corresponding to the charge holding units MEM_L1 and MEM_L2 in Fig. 2. Note that for ease of explanation, Fig. 5 excludes the time during which the charge discharging transistor M7 is turned on by the control signal OFG.

[0094] Furthermore, Texp represents a desired charge accumulation time set by the user in the photoelectric conversion device 100. Note that, although an example is given here in which the charge accumulation time is set by the user, for example, in an embedded system or the like, a control system or the like may set the charge accumulation time by a predetermined method based on, for example, a photometric value or the like.

[0095] The charge accumulation time is generally set to an integer multiple of the minimum unit Tmin (a predetermined time unit). This minimum unit Tmin is, for example, a system clock signal or a horizontal synchronization period. When the charge accumulation time for one charge holding section is constant and a time-division transfer is performed in which the charge accumulation time Ts and the charge accumulation time Tl are repeated, the product of the number of times of time-division transfer and this minimum unit Tmin becomes the minimum control unit in the charge accumulation control involving the time-division transfer. Therefore, even if the charge accumulation time Ts and the charge accumulation time Tl are set so as to be as close as possible to the desired charge accumulation time Texp, an error exceeding the minimum unit Tmin may occur.

[0096] 5A shows an example in which the charge accumulation time for transferring charge to the charge retention units MEM_S1 and MEM_S2 is Tsa, and the charge accumulation time for transferring charge to the charge retention units MEM_L1 and MEM_L2 is Tla. In this case, the total charge accumulation time in one frame period is Tta, and it is assumed that the total charge accumulation time in one frame period Tta is longer than the desired charge accumulation time Texp, resulting in an error Tdiffa that exceeds the minimum unit Tmin.

[0097] 5B shows an example in which the charge accumulation time for transferring charge to the charge holding units MEM_L1 and MEM_L2 is set to Tlb by shortening the charge accumulation time Tla shown in FIG. 5A by the minimum unit Tmin. In this case, since transfer to the charge holding units MEM_L1 and MEM_L2 is performed five times, the total charge accumulation time in one frame period is Ttb, which is five times the minimum unit Tmin shorter than the total charge accumulation time Tta. However, the total charge accumulation time Ttb in one frame period is shorter than the desired charge accumulation time Texp, and an error Tdiffb that exceeds the minimum unit Tmin occurs.

[0098] FIG. 5(c) is a diagram showing an example of the control of the charge storage time in this embodiment. The charge storage time for at least one of the multiple charge storage units is adjusted so that at least one charge storage time is different from the other charge storage times to approach the desired charge storage time Texp. In FIG. 5(c), the charge storage time control is performed so that the charge storage times for the charge storage units MEM_S1 and MEM_S2 are two types, charge storage time Tsa and charge storage time Tsb. Furthermore, the charge storage time control is performed so that the charge storage times for the charge storage units MEM_L1 and MEM_L2 are two types, charge storage time Tla and charge storage time Tlb.

[0099] In the example of FIG. 5C, the charge storage time Tsb is shorter than the charge storage time Tsa. iAmong them, Ts1, Ts2, Ts5, and Ts6 are charge storage times Tsa, Ts3, and Ts4 are charge storage times Ts b The charge accumulation control is performed as follows.

[0100] Furthermore, the charge storage time Tl j Among these, Tl1, Tl2, Tl4, and Tl5 are charge accumulation times Tla, and T3 is charge accumulation time Tlb for performing charge accumulation control.

[0101] By controlling in this way, the total charge accumulation time in one frame period becomes Ttc, and it can be made to approach the desired charge accumulation time Texp until the difference becomes smaller than the minimum unit Tmin. In this way, since the charge accumulation time for one charge holding unit can be freely set, the total charge accumulation time can be made to approach the desired charge accumulation time Texp regardless of the number of time-division transfers.

[0102] 5(c) shows an example in which one charge storage section has two types of charge storage time, but it is sufficient that at least one charge storage section among the multiple charge storage sections has two or more types of charge storage time. Also, the number of time-division transfers may be changed depending on the total charge storage time, etc.

[0103] Fig. 5(d) shows another example of this embodiment, in which the charge storage times for the charge storage units MEM_S1 and MEM_S2 are controlled by three types of charge storage times Tsa, Tsb, and Tsc, and the charge storage times for the charge storage units MEM_L1 and MEM_L2 are controlled by two types of charge storage times Tla and Tlb. Note that in the example of Fig. 5(d), the charge storage time Tsb is shorter than the charge storage time Tsa, and the charge storage time Tsc is longer than either of them.

[0104] Charge storage time Ts for charge storage unit MEM_S1 and charge storage unit MEM_S2 iAmong them, Ts1 and Ts6 are charge storage times Tsb, Ts2 and Ts5 are charge storage times Tsc, and Ts3 and Ts4 are charge storage times Tsa, and charge storage control is performed. j Among them, Tl1, Tl3, and Tl5 are controlled with a charge accumulation time Tlb, and Tl2 and Tl4 are controlled with a charge accumulation time Tla. Even when the charge accumulation time is controlled in this manner, the total charge accumulation time Ttd in one frame period can be brought close to the desired charge accumulation time Texp until the difference becomes smaller than the minimum unit Tmin, as in FIG. 5(c).

[0105] As shown in Figures 5(c) and 5(d), the charge accumulation times can be symmetrically distributed with respect to the midpoint between the accumulation start point and the accumulation end point to make the temporal center of gravity of the charge accumulation time Tshort and the charge accumulation time Tlong coincide. By controlling the charge accumulation time in this way, it is possible to obtain a high-quality video with a high dynamic range even when there is a jolt between frames during video shooting, or when shooting a fast-moving subject or a blinking light source.

[0106] In addition, there is no restriction on the length or order of each charge storage time for one charge storage unit, and the charge storage times may be gradually shortened from the storage start point to the midpoint between the storage end point as shown in Fig. 5(c).Furthermore, the charge storage times may be nested regardless of the length of each charge storage time from the storage start point to the midpoint between the storage end point as shown in Fig. 5(d).The number of charge storage times used may be changed for each of the multiple charge storage units.

[0107] As described above, according to the first embodiment, the total charge accumulation time can be made close to an arbitrarily set charge accumulation time.

[0108] <Second embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, a configuration having four charge holding units for one photoelectric conversion unit has been described. In contrast, in the second embodiment, as shown in Fig. 6, a configuration having two charge holding units for one photoelectric conversion unit is shown. In the configuration in Fig. 6, the same reference numerals are used for the same parts as in the configuration in Fig. 2, and the description thereof will be omitted.

[0109] Fig. 6 is a configuration in which the transfer transistor M1L2, the transfer transistor M3L2, the transfer transistor M1S2, and the transfer transistor M3S2 are omitted from the configuration shown in Fig. 2. In the configuration of the second embodiment, similarly to the configuration of the first embodiment, the floating diffusion region FD or the output unit may be individually connected to each of the two transfer units.

[0110] In the configuration shown in FIG. 6, when driving in GS mode, signals cannot be read out during the period in which charges are accumulated, and there are times during each frame period when charges cannot be acquired, making it impossible to acquire seamless moving images. However, for example, odd-numbered frames may be controlled as charge accumulation periods for accumulating charges in the charge storage units MEM_L1 and MEM_S1, and even-numbered frames may be controlled as read periods for reading charges from the charge storage units MEM_L1 and MEM_S1. In this way, by alternating between periods in which charges are accumulated and periods in which signals based on charges are read out, it is possible to obtain high-quality moving images with a high dynamic range, even though they are not seamless.

[0111] 5(c) and 5(d), the upper MEM_S can be the charge storage time for the charge storage unit MEM_S1 in Fig. 6, and the lower MEM_L can be the charge storage time for the charge storage unit MEM_L1 in Fig. 6. As a result, in the configuration shown in Fig. 6, as in the first embodiment, charge storage control is performed using two or more types of charge storage times for one charge storage unit, so that the charge storage time can be brought close to the desired charge storage time until the difference becomes smaller than the minimum unit Tmin.

[0112] In the first embodiment, a configuration in which one photoelectric conversion unit has four charge holding units and in the second embodiment, a configuration in which one photoelectric conversion unit has two charge holding units are described, but the present invention is not limited to these. As long as there is a configuration in which one photoelectric conversion unit has two or more charge holding units, the above-mentioned processing can be performed in a similar manner.

[0113] <Third embodiment> Next, a third embodiment of the present invention will be described. 7 is a block diagram showing a schematic configuration of an imaging system 200 according to the third embodiment. The imaging system 200 of this embodiment has an imaging section 201 using the photoelectric conversion device 100 described in the first or second embodiment. In addition to the imaging section 201, the imaging system 200 includes an aperture 204, a lens unit 202, a barrier 206, an overall control / calculation section 218, a timing generation section 220, and a signal processing section 208. Furthermore, the imaging system 200 includes a memory section 210, a recording medium control interface (I / F) section 216, and an external interface (I / F) section 212.

[0114] The lens unit 202 forms an optical image of a subject on the photoelectric conversion device 100 of the imaging section 201. The aperture 204 adjusts the amount of light that passes through the lens unit 202, and the barrier 206 protects the lens unit 202. The lens unit 202 and the aperture 204 form an optical system that collects light on the imaging section 201.

[0115] The imaging unit 201 converts the optical image formed by the lens unit 202 into a pixel signal as described above by the photoelectric conversion device 100, and outputs the pixel signal to the signal processing unit 208. The imaging unit 201 may include an AD conversion unit that generates a digital signal, in which case the imaging unit 201 outputs the digital signal. The AD conversion unit may be formed in a semiconductor layer (semiconductor substrate) on which the photoelectric conversion device 100 is formed, or may be formed on a semiconductor substrate different from the semiconductor layer on which the photoelectric conversion device 100 is formed.

[0116] The signal processing unit 208 processes pixel signals output from the imaging unit 201 to generate image data. At that time, the signal processing unit 208 performs various corrections and compression as necessary and outputs the image data. The signal processing unit 208 may be formed on the same semiconductor substrate as the imaging unit 201.

[0117] Furthermore, the signal processing unit 208 of this embodiment generates an image with an expanded dynamic range by using two types of pixel signals output from the imaging unit 201. This synthesis process is performed based on the ratio between the charge accumulation times Tlong and Tshort described above. For example, if the pixel signal obtained in the charge accumulation time Tlong is a non-saturated low-luminance signal, the signal obtained in the charge accumulation time Tlong is used. If the pixel signal obtained in the charge accumulation time Tlong is a saturated high-luminance signal, the signal obtained in the charge accumulation time Tshort is multiplied by the synthesis ratio.

[0118] As described in the first and second embodiments, the error to be adjusted to the desired charge accumulation time differs depending on the total charge accumulation time, and therefore the adjustment amount when matching with the desired charge accumulation time Texp differs. Therefore, by using the ratio of the adjusted charge accumulation times (Tlong / Tshort) as the composition ratio, a good image with a high dynamic range can be obtained.

[0119] The overall control and calculation unit 218 controls the entire imaging system 200 and performs various calculations. The various calculations include a calculation to adjust the charge accumulation time for each of the above-mentioned transfers in order to match the total charge accumulation time of the charge accumulation time Tlong and the charge accumulation time Tshort to the desired charge accumulation time Texp. Therefore, the combination ratio may be calculated by the overall control and calculation unit 218, or may be calculated in the signal processing unit 208 by outputting the charge accumulation time Tlong and the charge accumulation time Tshort to the signal processing unit 208. The timing generation unit 220 outputs various timing signals to the imaging unit 201 and the signal processing unit 208 based on the control by the overall control and calculation unit 218.

[0120] Note that the timing signal and the like may be input from the outside, and the imaging system 200 only needs to include at least the imaging unit 201 and a signal processing unit 208 that processes an output signal output from the imaging unit 201 .

[0121] The memory unit 210 is used to temporarily store image data, and the external I / F unit 212 is used to communicate with an external computer, etc. The recording medium control interface unit (recording medium control I / F unit) 216 controls recording and reading of imaging data to a recording medium 214 such as a semiconductor memory. The recording medium 214 may be built into the imaging system 200, or may be removable.

[0122] As described above, according to the third embodiment, a high-quality moving image with a high dynamic range can be obtained by combining charge signals captured in the GS mode with different charge accumulation times.

[0123] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0124] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0125] <Summary> The disclosure of this embodiment includes the following configuration.

[0126] (Item 1) A photoelectric conversion device including a plurality of pixels, each pixel having a photoelectric conversion means for converting incident light into an electric charge and storing the electric charge, and a plurality of first holding means, A control means for controlling a charge accumulation time in the photoelectric conversion means in a predetermined time unit; A setting means for setting a charge accumulation time in one frame period, The control means Control is performed so that charges are transferred from the photoelectric conversion means to the first holding means in sequence in a plurality of times during one frame period, and The charge accumulation time in the photoelectric conversion means is controlled so that the difference between the total of divided charge accumulation times for accumulating the charges transferred in the plurality of times during the one frame period and the set charge accumulation time is smaller than the time unit. A photoelectric conversion device comprising: (Item 2) 2. The photoelectric conversion device according to item 1, wherein the control means controls at least one of the number of times the charge is transferred and the divided charge accumulation time. (Item 3) The photoelectric conversion device according to item 1 or 2, characterized in that the control means controls, for each of the plurality of first holding means, the allocation of the divided charge accumulation time for accumulating the transferred charges in the one frame period so as to be symmetrical with respect to a midpoint from the start to the end of the charge accumulation time in the photoelectric conversion means. (Item 4) The photoelectric conversion device described in any one of items 1 to 3, characterized in that the control means controls the divided charge accumulation time for accumulating the charge to be transferred to at least one of the multiple first holding means using at least two types of time. (Item 5) Each of the pixels further includes a plurality of second holding means; The photoelectric conversion device described in any one of items 1 to 4, characterized in that in a frame next to when electric charges are transferred to the plurality of first holding means, the control means performs the same control on the plurality of second holding means as the control performed for the transfer of electric charges to the plurality of first holding means. (Item 6) a readout means for reading out the charges stored in the first holding means and the second holding means, The reading means is reading out the charges stored in the second holding means in a frame in which the charges are transferred to the first holding means; In a frame in which charges are transferred to the second holding means, the charges stored in the first holding means are read out. 6. The photoelectric conversion device according to item 5, (Item 7) The photoelectric conversion device described in any one of items 1 to 6, characterized in that the control means controls each of the multiple first holding means so that the total of the divided charge accumulation time for accumulating the transferred charge is different from each other. (Item 8) The photoelectric conversion device according to any one of items 1 to 7, a lens unit for forming an optical image of a subject on the photoelectric conversion means; An imaging device comprising: (Item 9) Item 7. The photoelectric conversion device according to item 7, a processing means for processing a signal corresponding to the charge read out from the photoelectric conversion device, The imaging apparatus according to claim 1, wherein the processing means performs processing for expanding a dynamic range for each of the pixels based on the signal. (Item 10) 10. The imaging device according to item 9, wherein the processing means expands a dynamic range based on a ratio of charge accumulation times corresponding to the signals. (Item 11) A method for controlling a photoelectric conversion device including a plurality of pixels, each pixel having a photoelectric conversion means that converts incident light into an electric charge and accumulates the electric charge, and a plurality of first holding means, comprising the steps of: a setting step of setting a charge accumulation time in one frame period; A control step of controlling a charge accumulation time in the photoelectric conversion means in a predetermined time unit, In the control step, Control is performed so that charges are transferred from the photoelectric conversion means to the first holding means in sequence in a plurality of times during one frame period, and The charge accumulation time in the photoelectric conversion means is controlled so that the difference between the total of divided charge accumulation times for accumulating the charges transferred in the plurality of times during the one frame period and the set charge accumulation time is smaller than the time unit. A control method comprising: (Item 12) A program for causing a computer to execute each step of the control method described in item 11. (Item 13) Item 13. A computer-readable storage medium storing the program according to item 12.

[0127] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0128] PD...photoelectric conversion section, M1L1, M1L2, M1S1, M1S2, M3L1, M3L2, M3S1, M3S2...transfer transistor, M4...reset transistor, M5...amplification transistor, M6...selection transistor, M7...charge discharge transistor, MEM_L1, MEM_L2, MEM_S1, MEM_S2...charge retention section, FD...floating diffusion section, 10...pixel region, 12...pixel, 14...control line, 16...vertical output line, 20...vertical scanning circuit, 30...readout circuit, 40...horizontal scanning circuit, 50...output circuit, 60...control circuit, 100...photoelectric conversion device, 200...imaging system, 201...imaging section, 208...signal processing section, 210...overall control / arithmetic section, 220...timing generation section

Claims

1. A photoelectric conversion device including a plurality of pixels, each pixel having a photoelectric conversion means for converting incident light into an electric charge and storing the electric charge, and a plurality of first holding means, A control means for controlling a charge accumulation time in the photoelectric conversion means in a predetermined time unit; a setting means for setting a charge accumulation time in one frame period; The control means Controlling the transfer of charges from the photoelectric conversion means to the first holding means in sequence in a plurality of batches during one frame period, The charge accumulation time in the photoelectric conversion means is controlled so that the difference between the total of divided charge accumulation times for accumulating the charges transferred in a plurality of times during the one frame period and the set charge accumulation time is smaller than the time unit. A photoelectric conversion device comprising:

2. 2. The photoelectric conversion device according to claim 1, wherein the control means controls at least one of the number of times the electric charges are transferred and the divided electric charge accumulation time.

3. The photoelectric conversion device according to claim 1, characterized in that the control means controls, for each of the multiple first holding means, the allocation of the divided charge accumulation time for accumulating the transferred charges during the one frame period so as to be symmetrical with respect to a midpoint from the start to the end of the charge accumulation time in the photoelectric conversion means.

4. 2. The photoelectric conversion device according to claim 1, wherein the control means controls the divided charge accumulation time for accumulating the charges to be transferred to at least one of the plurality of first holding means by at least two types of time.

5. Each of the pixels further includes a plurality of second holding means, The photoelectric conversion device according to claim 1, characterized in that, in the frame next after the control means has transferred electric charges to the plurality of first holding means, the control means performs the same control on the plurality of second holding means as the control performed for the transfer of electric charges to the plurality of first holding means.

6. a readout means for reading out the charges stored in the first holding means and the second holding means, The reading means is reading out the charges stored in the second holding means in a frame in which the charges are transferred to the first holding means; In a frame in which charges are transferred to the second holding means, the charges stored in the first holding means are read out.

6. The photoelectric conversion device according to claim 5.

7. 2. The photoelectric conversion device according to claim 1, wherein the control means controls the plurality of first holding means so that the totals of the divided charge accumulation times for accumulating the transferred charges are different from one another.

8. The photoelectric conversion device according to claim 1 , a lens unit for forming an optical image of a subject on the photoelectric conversion means; An imaging device comprising:

9. The photoelectric conversion device according to claim 7 ; a processing means for processing a signal corresponding to the charge read out from the photoelectric conversion device, The imaging apparatus according to claim 1, wherein the processing means performs processing for expanding a dynamic range for each of the pixels based on the signal.

10. 10. The imaging apparatus according to claim 9, wherein the processing means expands a dynamic range based on a ratio of charge accumulation times corresponding to the signals.

11. A method for controlling a photoelectric conversion device including a plurality of pixels, each pixel having a photoelectric conversion unit that converts incident light into an electric charge and accumulates the electric charge, and a plurality of first holding units, comprising the steps of: a setting step of setting a charge accumulation time in one frame period; A control step of controlling a charge accumulation time in the photoelectric conversion means in a predetermined time unit, In the control step, Controlling the transfer of charges from the photoelectric conversion means to the first holding means in sequence in a plurality of batches during one frame period, The charge accumulation time in the photoelectric conversion means is controlled so that the difference between the total of divided charge accumulation times for accumulating the charges transferred in a plurality of times during the one frame period and the set charge accumulation time is smaller than the time unit. A control method comprising:

12. A program for causing a computer to execute each step of the control method according to claim 11.

13. A computer-readable storage medium storing the program according to claim 12.

Images