Imaging apparatus
Patent Information
- Application Number
- JP2023017355
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-02-08
- Publication Date
- 2026-02-10
AI Technical Summary
Global shutter (GS) sensors experience noise due to large current fluctuations when control signals are simultaneously output, particularly at high gain settings, affecting video signals.
An imaging device with a photoelectric conversion section and at least two charge units that accumulate charges, allowing charge transfer to the charge storage section in multiple times during one frame period in a first mode, and once in a second mode, based on exposure settings, to suppress noise and enhance dynamic range.
The solution effectively suppresses noise and enhances sensitivity, enabling high dynamic range imaging by controlling charge transfer timing and accumulation times in GS sensors.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging device. [Background technology]
[0002] Among CMOS image sensors, which are imaging elements, there are image sensors (hereafter referred to as "GS sensors") that have a global shutter (hereafter referred to as "GS") function by having a memory section (charge storage section) in each pixel. The pixels of this GS sensor are equipped with a gate that transfers the signal charge accumulated in the photoelectric conversion section to the charge storage section. In a GS sensor, the signal charge is transferred from the photoelectric conversion section to the charge storage section for all pixels at the same time, so that the timing of the start and end of signal accumulation in the photoelectric conversion section is the same for all pixels, thereby realizing the GS function.
[0003] Also, as disclosed in Patent Document 1, a GS sensor is known that has multiple charge storage units for one photoelectric conversion unit and transfers signal charges to each charge storage unit multiple times during one frame period. This GS sensor can improve the dynamic range by acquiring multiple images in which the total accumulation time of the signal charges transferred to each charge storage unit is changed. For example, by acquiring multiple images with different accumulation times and combining them, a single image with a high dynamic range can be obtained. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] US Patent Publication No. 2013 / 0135486 Summary of the Invention [Problem to be solved by the invention]
[0005] However, in a GS sensor, each control signal is output simultaneously for all pixels. Therefore, when a control signal is input, the current fluctuation is large, and noise may occur in the video signal at the timing when the control signal is input. For example, noise may occur at the timing when the signal charge is transferred to each charge storage section. In particular, the above noise is noticeable when the gain, which is one of the parameters that determines the exposure of the camera, is set high.
[0006] An object of the present invention is to suppress noise caused by global shutter driving of an image sensor. [Means for solving the problem]
[0007] In order to achieve the above-mentioned object, the imaging device of the present invention is characterized in having an imaging element having a plurality of pixels each having a photoelectric conversion unit that generates an electric charge by photoelectric conversion and at least two charge accumulation units connected to the photoelectric conversion unit and that accumulate the electric charge transferred from the photoelectric conversion unit, and a control means that controls the driving mode of the imaging element to be switched between a first mode in which the electric charge is transferred from the photoelectric conversion unit to the charge accumulation unit in multiple batches during one frame period, and a second mode in which the electric charge is transferred from the photoelectric conversion unit to the charge accumulation unit only once during one frame period, depending on an exposure setting. Effect of the Invention
[0008] According to the present invention, it is possible to suppress noise caused by global shutter driving of an image sensor. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a block diagram of an imaging device. [Diagram 2] FIG. 2 is an equivalent circuit diagram of one pixel. [Diagram 3] 4 is a timing chart of exposure driving in a first mode. [Figure 4] 11 is a timing chart of read driving in the first mode. [Diagram 5] 13 is a timing chart of exposure driving in a second mode. [Figure 6] 5 is a flowchart showing a drive mode switching process. [Figure 7] FIG. 2 is an equivalent circuit diagram of one pixel. [Figure 8] 4 is a timing chart of exposure driving in a first mode. [Figure 9] 11 is a timing chart of read driving in the first mode. [Figure 10] 13 is a timing chart of exposure driving in a second mode. [Figure 11] FIG. 2 is an equivalent circuit diagram of one pixel. [Figure 12] 4 is a timing chart of exposure driving in a first mode. [Figure 13] 11 is a timing chart of read driving in the first mode. [Figure 14] 13 is a timing chart of exposure driving in a second mode. [Figure 15] FIG. 2 is an equivalent circuit diagram of one pixel. [Figure 16] 4 is a timing chart of exposure driving in a first mode. [Figure 17] 11 is a timing chart of read driving in the first mode. [Figure 18] 13 is a timing chart of exposure driving in a second mode. [Figure 19] 5 is a flowchart showing a drive mode switching process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0011] (First embodiment) 1 is a block diagram of an imaging device equipped with an image sensor according to a first embodiment of the present invention. The imaging device 1000 includes an image sensor 111 and an image processing unit 112. The image sensor 111, which is a solid-state image sensor, includes a pixel unit 101, a vertical scanning circuit 102, a column amplifier circuit 103, a horizontal scanning circuit 104, an output circuit 105, a timing control circuit 106, and a controller circuit 113.
[0012] The controller circuit 113 is an interface between the image sensor 111 and the image processing unit 112, and communicates with the image processing unit 112 via serial communication. The controller circuit 113 receives a control signal to the image sensor 111 output from the image processing unit 112 via serial communication. The image processing unit 112, for example, calculates brightness based on a pixel signal output from the image sensor 111, and determines exposure settings such as an aperture and an exposure time (charge accumulation time) based on the calculated brightness. The determined exposure settings are then transmitted to the controller circuit 113 as a control signal, and the controller circuit 113 transmits the received control signal to the timing control circuit 106.
[0013] In a plan view of the substrate, the pixel section 101 is a pixel array including a plurality of pixels 107 arranged two-dimensionally including a plurality of rows and a plurality of columns.
[0014] The vertical scanning circuit 102 supplies control signals to a plurality of transistors included in the pixels 107, and controls the on (conducting state) or off (non-conducting state) of these transistors. A signal line 108 is provided for each column of the pixels 107, and a signal from the pixels 107 is output to the signal line 108 for each column. The column amplification circuit 103 includes an amplifier for amplifying the pixel signal output to the signal line 108, and an AD conversion circuit for converting an analog signal into a digital signal. The column amplification circuit 103 performs processing such as correlated double sampling on the pixel signal output to the signal line 108 based on the signal at the time of reset release of the pixel 107 and the signal at the time of photoelectric conversion.
[0015] The horizontal scanning circuit 104 supplies a control signal to a switch connected to the column amplifier circuit 103 of each column to control the switch to be on or off. A timing control circuit 106 controls the vertical scanning circuit 102, the column amplifier circuit 103, and the horizontal scanning circuit 104. The output circuit 105 has a serializer function, and converts the pixel signal from the column amplifier circuit 103 into a serial signal and outputs it.
[0016] The pixel signal output from the output circuit 105 is input to the image processing unit 112, where development processing such as various adjustment / correction processing is performed on the pixel signal, and the developed pixel signal is output to a monitor or recorded on a recording medium. Thus, the image processing unit 112 processes the signal output from the imaging element 111 to generate image data.
[0017] FIG. 2 is an equivalent circuit diagram of one pixel 107.
[0018] The photoelectric conversion unit PD1 is a photodiode that generates electric charges by photoelectric conversion of incident photons. The charge transfer units GS_LA2, GS_LB3, GS_SA4, and GS_SB5 transfer the signal charges generated by the photoelectric conversion unit PD1 to the subsequent charge storage units MEM_LA6, MEM_LB7, MEM_SA8, and MEM_SB9. The charge storage units MEM_LA6, MEM_LB7, MEM_SA8, and MEM_SB9 each hold the transferred signal charges. The transfer units TX_LA10, TX_LB11, TX_SA12, and TX_SB13 transfer the signal charges held in the preceding charge storage units MEM_LA6, MEM_LB7, MEM_SA8, and MEM_SB9 to the subsequent circuit elements.
[0019] The FD14 is an input node of the amplifier section described later, and holds the signal charge transferred from the charge storage section in the previous stage via the transfer section. For the FD14, for example, a floating diffusion region (FD region) arranged on a semiconductor substrate can be used.
[0020] RES15 is a reset unit that supplies a reference voltage to FD14, which is the input node of the amplifier unit. SF16 is an amplifier unit that amplifies a signal based on the signal charge transferred to the FD region and outputs it to the outside. As an example, SF16 is a source follower circuit using a MOS transistor. A configuration in which the gate of this MOS transistor is electrically connected to the floating diffusion region can be used. Although multiple transfer units (TX_LA10, TX_LB11, TX_SA12, TX_SB13) share the input nodes FD14 and SF16, a circuit configuration in which they do not share them may also be adopted.
[0021] SEL17 is a selection unit that can select each pixel and read out a signal (pixel signal) based on the signal charge for each pixel or for each pixel row to the outside. OFG18 is a charge discharge control unit that discharges unnecessary charges from the photoelectric conversion unit PD1. For example, a MOS transistor can be used for OFG18. In this case, a semiconductor region of the same polarity as the signal charge constituting part of the photoelectric conversion unit serves as the source, and a semiconductor region (overflow drain region: OFD region) to which a power supply voltage VDD19 is supplied serves as the drain. Note that MOS transistors can be used for each transfer unit, reset unit, selection unit, and charge discharge control unit.
[0022] When the charge transfer unit GS_LA2 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM_LA6. When the charge transfer unit GS_LB3 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM_LB7. When the charge transfer unit GS_SA4 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM_SA8. When the charge transfer unit GS_SB5 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM_SB9.
[0023] When the transfer unit TX_LA10 is turned on, it transfers the signal charge held in the charge accumulation unit MEM_LA6 to the FD14. When the transfer unit TX_LB11 is turned on, it transfers the signal charge held in the charge accumulation unit MEM_LB7 to the FD14. When the transfer unit TX_SA12 is turned on, it transfers the signal charge held in the charge accumulation unit MEM_SA8 to the FD14. When the transfer unit TX_SB13 is turned on, it transfers the signal charge held in the charge accumulation unit MEM_SB9 to the FD14.
[0024] In this embodiment, the "at least two charge storage units that store charges transferred from the photoelectric conversion units" corresponds to "a pair of MEM_LA6 and MEM_SA8" or "a pair of MEM_LB7 and MEM_SB9." For example, MEM_LA6 and MEM_SA8 are connected in parallel to the rear stage of the photoelectric conversion unit PD1. MEM_LB7 and MEM_SB9 are connected in parallel to the rear stage of the photoelectric conversion unit PD1.
[0025] In the following description, the terms "charge storage section" such as charge storage section MEM_LA6, "charge transfer section" such as charge transfer section GS_LA2, and "transfer section" such as transfer section TX_LA10 may be omitted as appropriate.
[0026] Next, a method of driving the pixel portion 101 will be described with reference to FIGS.
[0027] The timing control circuit 106 can transfer charges from the photoelectric conversion unit to the charge accumulation unit for all pixels at once. There are a first mode and a second mode as pixel drive modes that the timing control circuit 106 can execute. The first mode is a mode in which charges are transferred from the photoelectric conversion unit to the charge accumulation unit in multiple batches (intermittently) during one frame period. The second mode is a mode in which charges are transferred from the photoelectric conversion unit to the charge accumulation unit only once during one frame period.
[0028] Figures 3 to 5 are timing charts showing the time series transition of drive pulses supplied to each control electrode and transfer electrode. In particular, Figures 3 and 5 show drive related to exposure of one pixel, and Figure 4 shows drive related to signal readout of multiple pixels. Figures 3 and 4 show drive in the first mode, and Figure 5 shows drive in the second mode.
[0029] In Fig. 4, the subscript n is the number of pixel rows. The subscripts of each control line indicate the nth row and the n+1th row. Here, two rows are described, but this drive pattern can be repeated even when there are three or more rows. In a global shutter type imaging device, the drive timing for exposure-related drives is the same for all rows, so subscripts and notations for multiple rows are not used. The high pulse is active.
[0030] The drive timings of the read-out related drive and transfer units TX_LA10, TX_LB11, TX_SA12, TX_SB13, SEL17, and RES15 differ depending on the row, so subscripts or notations for multiple rows are used.
[0031] First, the driving related to exposure in the first mode will be described with reference to Fig. 3. As shown in Fig. 3, in the first mode, in an even frame (2N-th frame), the signal charge generated in the photoelectric conversion unit PD1 is transferred to the charge storage unit MEM_LA6 and stored therein by turning on the charge transfer unit GS_LA2. Also, the signal charge is transferred to the storage unit MEM_SA8 and stored therein by turning on the charge transfer unit GS_SA4.
[0032] In the first mode, the timing control circuit 106 controls the charge transfer from the photoelectric conversion unit to each charge accumulation unit at different timings during one frame period. The charge transfer drive can be performed multiple times during one frame period. In the example shown in FIG. 3, the signal charge generated in the photoelectric conversion unit PD1 is transferred alternately to the charge accumulation unit MEM_LA6 and the charge accumulation unit MEM_SA8.
[0033] 3 indicates the accumulation time of the i-th signal charge among the charge transfers repeated multiple times (e.g., Nshort times) in an even-numbered frame (2N-th frame). The accumulation time Tshorti is the time from when OFG18 is turned on and the photoelectric conversion unit PD1 is reset to when the charge transfer unit GS_SA4 is turned on and the signal charge is transferred from the photoelectric conversion unit PD1 to the charge accumulation unit MEM_SA8 until the charge transfer unit GS_SA4 is turned off.
[0034] Although the photoelectric conversion unit PD1 is reset by the OFG 18, if the signal charge is configured not to remain in the photoelectric conversion unit PD1 during transfer, the reset by the OFG 18 may be omitted. In that case, the accumulation time Tshorti is the time from when the immediately preceding charge transfer drive is completed to when the charge transfer unit GS_SA4 is turned off.
[0035] In an even-numbered frame (2N-th frame), the total accumulation time Tshort of the signal charges accumulated in the charge accumulation unit MEM_SA8 is the sum of the accumulation times Tshort for Nshort times. The accumulation times Tshort of the signal charges for the Nshort times may be the same or different.
[0036] 3 indicates the accumulation time of the i-th signal charge among the charge transfers repeated multiple times (e.g., Nlong times) in an even-numbered frame (2N-th frame). The accumulation time Tlong is the time from when OFG18 is turned on and the photoelectric conversion unit PD1 is reset to when the charge transfer unit GS_LA2 is turned on and the signal charge is transferred from the photoelectric conversion unit PD1 to the charge accumulation unit MEM_LA6 until when the charge transfer unit GS_LA2 is turned off.
[0037] Although the photoelectric conversion unit is reset by OFG 18, if the signal charge is configured not to remain in the photoelectric conversion unit PD1 during transfer, the reset by OFG 18 may be omitted. In that case, the accumulation time Tlongi is the time from when the immediately preceding charge transfer drive is completed to when the charge transfer unit GS_LA2 is turned off.
[0038] In an even-numbered frame (2N-th frame), the total accumulation time Tlong of the signal charges accumulated in the charge accumulation unit MEM_LA6 is the sum of the accumulation times Tlong for Nlong times. The accumulation times Tlong of the signal charges for Nlong times may be the same or different.
[0039] In this way, in the first mode, the signal charge is controlled to be transferred from the photoelectric conversion unit to each charge accumulation unit in multiple batches at different timings during one frame period. At that time, by controlling the charge accumulation times in each charge accumulation unit during one frame to be different from each other, it is possible to output an image with a high dynamic range.
[0040] Signal readout in the first mode will be described with reference to Fig. 4. First, signal readout in the even-numbered frame (2N-th frame) shown in Fig. 4 will be described.
[0041] Here, the signal charges stored in MEM_LB7 and MEM_SB9 in the odd-numbered frame (not shown) one frame before are read out. A signal readout process in the n-th row when read out row by row will be described.
[0042] The timing control circuit 106 turns on SEL17(n) to enable reading out of signals based on the charges of the FD14 of the pixels in the nth row. By turning on SEL17(n), the timing control circuit 106 turns off RES15(n), which resets the FD14, and reads out the reset release level voltage VRES of the FD14 (time t0). Next, the timing control circuit 106 turns on TX_LB11(n), transfers the signal charge held in MEM_LB7 to the FD14, and reads out the signal level VSIG of the FD14 (time t1). The difference between these two signal levels, |VSIG-VRES|, is the signal level proportional to the amount of signal charge in MEM_LB7.
[0043] The timing control circuit 106 again turns on RES15(n) and then turns it off to reset the FD14, and reads out the reset release level voltage VRES of the FD14 (time t2). Next, the timing control circuit 106 turns on TX_SB13(n), transfers the signal charge held in MEM_SB9 to the FD14, and reads out the signal level VSIG of the FD14 (time t3). The difference between these two signal levels, |VSIG-VRES|, is a signal level proportional to the amount of signal charge in MEM_SB9. The timing control circuit 106 repeats this row-sequentially in all areas or in the area to be acquired, and acquires signals based on the accumulated charges in MEM_LB7 and MEM_SB9 as pixel signals.
[0044] 3, in the first mode, in an odd-numbered frame ((2N+1)-th frame), the signal charge generated in the photoelectric conversion unit PD1 is transferred to and stored in the charge storage unit MEM_LB7 by turning on the charge transfer unit GS_LB3. Also, the signal charge is transferred to and stored in the storage unit MEM_SB9 by turning on the charge transfer unit GS_SB5.
[0045] The drive timing of the charge transfer units GS_LB3 and GS_SB5 in the odd-numbered frame (2N+1-th frame) may be the same as that of the charge transfer units GS_LA2 and GS_SA4 in the even-numbered frame (2N-th frame). The drive timing of the OFG18 in the odd-numbered frame (2N+1-th frame) may be the same as that of the OFG18 in the even-numbered frame (2N-th frame).
[0046] Next, signal readout in the odd-numbered frame ((2N+1)-th frame) shown in FIG. 4 will be described.
[0047] Here, the signal charges stored in MEM_LA6 and MEM_SA8 in the previous even-numbered frame (2N-th frame) are read out. The signal readout process in the n-th row when read out row by row will be described.
[0048] The timing control circuit 106 turns on SEL17(n) to enable reading out signals based on the charges of the FD14 of the pixels in the nth row. The timing control circuit 106 turns on SEL17(n) and turns off RES15(n), which resets the FD14, and reads out the reset release level voltage VRES of the FD14 (time t4). Next, the timing control circuit 106 turns on TX_LA10(n), transfers the signal charge held in MEM_LA6 to FD14, and reads out the signal level VSIG of the FD14 (time t5). The difference between these two signal levels, |VSIG-VRES|, is the signal level proportional to the amount of signal charge in MEM_LA6.
[0049] The timing control circuit 106 again turns on RES15(n) and then turns it off to reset the FD14, and reads out the reset release level voltage VRES of the FD14 (time t6). Next, the timing control circuit 106 turns on TX_SA12(n), transfers the signal charge held in MEM_SA8 to the FD14, and reads out the signal level VSIG of the FD14 (time t7). The difference between these two signal levels, |VSIG-VRES|, is a signal level proportional to the amount of signal charge in MEM_SA8. The timing control circuit 106 repeats this row-sequentially in all areas or in the area to be acquired, and acquires signals based on the accumulated charges in MEM_LA6 and MEM_SA8 as pixel signals.
[0050] The value of the accumulation time Tlong and the value of the accumulation time Tshort may be different from each other. This makes it possible to obtain two types of images with different effective exposure amounts in the same frame. By correcting the signal of one of these two types of images by the accumulation time and then combining them to form a single image, a single image with a high dynamic range can be obtained.
[0051] That is, the image processing unit 112 as a processing means can read out signals generated based on the signal charges held in the multiple charge holding units to generate and synthesize images. The timing control circuit 106 sets the ratio of the accumulation times Tlong and Tshort to a predetermined ratio, and controls the exposure time so that charges are accumulated in the multiple charge holding units at the predetermined ratio. The image processing unit 112 then synthesizes the signals from the at least two charge storage units to generate one image. At that time, the image processing unit 112 performs a calculation based on the predetermined ratio of the exposure times before synthesizing the images.
[0052] In this way, the pixel 107 has two charge storage units that store the signal charges transferred to one photoelectric conversion unit PD1, and two charge storage units that hold the charges until the signal of the next frame is read out. This makes it possible to capture seamless moving images with a high dynamic range and no time regions that cannot be captured.
[0053] On the other hand, for example, if one attempts to read out the signal charges of the charge storage units MEM_LB7 and MEM_SB9 at the timing when the charges are transferred to the charge storage units MEM_LA6 and MEM_SA8, there is a possibility that noise will be introduced into the signal due to the influence of current fluctuations, etc. Also, in the first mode, charges are transferred intermittently to one charge storage unit within one frame period, so the sensitivity is lower than when the charges are not transferred intermittently.
[0054] Next, driving related to exposure in the second mode will be described with reference to Fig. 5. The second mode is suitable, for example, when improved sensitivity is required or when a high dynamic range is not required.
[0055] Tlongi in Fig. 5 indicates the accumulation time of signal charges that is performed once each in an even frame (2N-th frame) and an odd frame ((2N+1)-th frame). As shown in Fig. 5, in the second mode, in an even frame (2N-th frame), the signal charges generated in the photoelectric conversion unit PD1 are transferred and accumulated only in the charge accumulation unit MEM_LA6 by turning on the charge transfer unit GS_LA2. This charge transfer drive to the charge accumulation unit MEM_LA6 is performed only once per frame.
[0056] In an odd-numbered frame ((2N+1)-th frame), the signal charge generated in the photoelectric conversion unit PD1 is transferred to and accumulated only in the charge storage unit MEM_LB7 by turning on the charge transfer unit GS_LB3. This charge transfer drive to the charge storage unit MEM_LB7 is performed only once per frame.
[0057] The drive timing of the charge transfer unit GS_LB3 and OFG18 in the odd-numbered frame (2N+1-th frame) may be the same as that of the charge transfer unit GS_LA2 and OFG18 in the even-numbered frame (2N-th frame).
[0058] The operation related to the readout in the second mode may be basically the same as that shown in Fig. 4. That is, the signal charge of the charge storage unit MEM_LB7 is read out in the even frame (2N-th frame), and the signal charge of the charge storage unit MEM_LA6 is read out in the odd frame (2N+1-th frame). However, since no charge is transferred to MEM_SA8 or MEM_SB9, the readout operation of the charge from MEM_SA8 or MEM_SB9 is not performed.
[0059] In this way, in the second mode, signal charge is not accumulated in the photoelectric conversion unit PD1 multiple times during one frame period, so it is possible to prevent current fluctuations caused by transferring charge to the charge accumulation unit from affecting the readout of the signal charge. Also, since charge is not transferred intermittently to the charge accumulation unit within one frame period, sensitivity is higher than when charge is transferred intermittently.
[0060] Next, the switching of the drive mode will be explained with reference to FIG.
[0061] 6 is a flowchart showing a drive mode switching process. This process is realized by a CPU (not shown) included in the image processing unit 112 reading and executing a program stored in a storage unit (not shown) such as a ROM in the imaging device 1000, and transmitting a control signal to the controller circuit 113 of the imaging element 111. This process is started, for example, in response to the exposure for photographing being set in the imaging device 1000.
[0062] First, in S101, the CPU of the image processing unit 112 serving as a control unit acquires the exposure settings set in the imaging device 1000. The exposure settings include, for example, the ISO sensitivity setting and the gain setting in signal processing.
[0063] In S102, the CPU of the image processing unit 112 determines whether the acquired exposure setting satisfies a predetermined condition. Here, the predetermined condition is that the set ISO sensitivity is not higher than a first predetermined value and the set gain value is not higher than a second predetermined value. Therefore, if the set ISO sensitivity is higher than the first predetermined value or the set gain value is higher than the second predetermined value, the predetermined condition is not satisfied. If the acquired exposure setting satisfies the predetermined condition, the CPU of the image processing unit 112 proceeds to S103, and if the acquired exposure setting does not satisfy the predetermined condition, the CPU proceeds to S104.
[0064] In S103, since the set ISO sensitivity and gain are low, this is a scene in which a high dynamic range is more important than improved sensitivity. Therefore, the CPU of the image processing unit 112 transmits a control signal to the controller circuit 113 of the image sensor 111 to execute the first mode (FIG. 3). As a result, the CPU controls the image sensor 111 to transfer charges to the charge storage unit multiple times during one frame period and read out a signal that can generate an image with a high dynamic range.
[0065] On the other hand, in S104, the ISO sensitivity or gain is set high, so this is a scene that requires high sensitivity. Therefore, the timing control circuit 106 transmits a control signal to the controller circuit 113 of the image sensor 111 to execute the second mode (FIG. 5). As a result, the CPU controls the image sensor 111 to transfer charges to the charge storage unit only once, thereby increasing the sensitivity and suppressing the effect of noise generated by the charge transfer. After S103 and S104, the drive mode switching process ends.
[0066] According to this embodiment, the CPU of the image processing unit 112 switches the drive mode between a first mode and a second mode depending on the exposure setting. For example, in the first mode, charges are transferred in multiple batches from the photoelectric conversion unit PD1 to the charge accumulation unit (MEM_LA6 or MEM_SA8) during one frame period. In the second mode, charges are transferred only once from the photoelectric conversion unit PD1 to the charge accumulation unit during one frame period.
[0067] In particular, when the acquired exposure setting satisfies a predetermined condition, that is, when the set ISO sensitivity or gain is low, the camera is controlled in the first mode. This makes it possible to obtain an image with a high dynamic range as required. For example, by controlling the charge accumulation time in each charge accumulation unit to be different from each other, it becomes possible to output an image with a high dynamic range. If the acquired exposure setting does not satisfy the predetermined conditions, that is, if the set ISO sensitivity or gain is high, the camera is controlled in the second mode. This improves sensitivity and suppresses noise. In particular, in a situation where the exposure setting is high and noise is easily noticeable, the second mode is adopted to effectively reduce noise. This makes it possible to suppress noise caused by the global shutter drive of the image sensor.
[0068] Second embodiment 7 is an equivalent circuit diagram of one pixel in an image sensor according to the second embodiment of the present invention. In the following description, the same components as those in the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.
[0069] This embodiment differs from the first embodiment in that there are two sets of charge storage units connected in series between the photoelectric conversion unit PD1 and the FD 14. In the first embodiment, four transfer units were connected to the FD 14, but in this embodiment, only two transfer units are required to be connected to the FD 14, so the capacity of the FD 14 can be reduced, resulting in a circuit configuration that is resistant to noise.
[0070] The charge transfer units GS1_L20, GS2_L24, GS1_S21, and GS2_S25 transfer the signal charges generated by the photoelectric conversion unit PD1 to the charge storage unit in the subsequent stage. The charge storage units MEM1_L22, MEM2_L26, MEM1_S23, and MEM2_S27 hold the transferred signal charges. The transfer units TX_L28 and TX_S29 transfer the signal charges held in the charge storage units in the previous stages to the circuit elements in the subsequent stages.
[0071] When the charge transfer unit GS1_L20 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM1_L22. When the charge transfer unit GS2_L24 is turned on, it transfers the signal charge held in the charge accumulation unit MEM1_L22 to the charge accumulation unit MEM2_L26. When the charge transfer unit GS1_S21 is turned on, it transfers the signal charge generated in the photoelectric conversion unit PD1 to the charge accumulation unit MEM1_S23. When the charge transfer unit GS2_S25 is turned on, it transfers the signal charge held in the charge accumulation unit MEM1_S23 to the charge accumulation unit MEM2_S27.
[0072] In this embodiment, the "at least two charge accumulation units that accumulate charges transferred from the photoelectric conversion unit" corresponds to the "set of MEM1_L22 and MEM1_S23." MEM1_L22 and MEM1_S23 are connected in parallel to the rear stage of the photoelectric conversion unit PD1.
[0073] Next, a method for driving the pixel portion 101 in this embodiment will be described with reference to FIGS.
[0074] Figures 8 to 10 are timing charts showing the time series transition of drive pulses supplied to each control electrode and transfer electrode. In particular, Figures 8 and 10 show drive related to exposure of one pixel, and Figure 9 shows drive related to signal readout of multiple pixels. Figures 8 and 9 show drive in a first mode, and Figure 10 shows drive in a second mode.
[0075] The meaning of the subscript n is the same as that explained in the first embodiment. Note that, although the first embodiment illustrates both the even-numbered and odd-numbered frames, in the present embodiment, there is no difference in the driving method between the even-numbered and odd-numbered frames, so only one frame is illustrated.
[0076] 8 and 9, in the first mode, the signal charge generated in the photoelectric conversion unit PD1 is transferred to and accumulated in the charge accumulation unit MEM1_L22 by turning on the charge transfer unit GS1_L20. Also, the signal charge is transferred to and accumulated in the charge accumulation unit MEM1_S23 by turning on the charge transfer unit GS1_S21. The signal charges in the charge accumulation units MEM2_L26 and MEM2_S27 are read out.
[0077] In the first mode, the timing control circuit 106 transfers charges from the photoelectric conversion unit to each charge accumulation unit at different timings during one frame period. Charge transfer driving can be performed multiple times during one frame period. In the example shown in FIG. 8, the signal charges generated in the photoelectric conversion unit PD1 are alternately transferred to the charge accumulation unit MEM1_L22 and the charge accumulation unit MEM1_S23.
[0078] 8 indicates the accumulation time of the i-th signal charge among the charge transfers repeated multiple times (e.g., Nshort times) in the N-th frame. The accumulation time Tshort is the time from when OFG18 is turned on and the photoelectric conversion unit PD1 is reset to when the charge transfer unit GS1_S21 is turned on and the signal charge is transferred from the photoelectric conversion unit PD1 to the charge accumulation unit MEM1_S23, until when the charge transfer unit GS1_S21 is turned off. The total accumulation time Tshort of the signal charge accumulated in the charge accumulation unit MEM1_S23 is the time obtained by adding up the accumulation times Tshort for Nshort times up to Nshort.
[0079] 8 indicates the accumulation time of the i-th signal charge among the charge transfers repeated multiple times (e.g., Nlong times) in the N-th frame. The accumulation time Tlong is the time from when the OFG18 is turned on and the photoelectric conversion unit PD1 is reset to when the charge transfer unit GS_1L20 is turned on and the signal charge is transferred from the photoelectric conversion unit PD1 to the charge accumulation unit MEM1_L22 until when the charge transfer unit GS1_L20 is turned off. The total accumulation time Tlong of the signal charge accumulated in the charge accumulation unit MEM1_L22 is the sum of the accumulation times Tlong of the signal charge for Nlong times.
[0080] When each charge transfer in the N-th frame (the Nshort-th charge transfer in FIG. 8) is completed, the charge in the charge storage unit MEM1_S23 can be transferred to the subsequent charge storage unit MEM2_S27. At this time, it is assumed that the signal charges acquired in the previous frame, the (N-1)-th frame, and held in the charge storage unit MEM2_S27 have been read out for all rows.
[0081] The charge transfer unit GS2_S25 turns on, and charge is transferred from the charge accumulation unit MEM1_S23 to the charge accumulation unit MEM2_S27. Similarly, for the charge accumulation unit MEM1_L22, the charge transfer unit GS2_L24 turns on, and charge is transferred from the charge accumulation unit MEM1_L22 to the charge accumulation unit MEM2_L26. As a result, the charge accumulation units MEM1_S23 and MEM1_L22 become empty, and are able to accumulate signal charge again in the next frame.
[0082] Signal readout in the N-th frame in the first mode will be described with reference to FIG.
[0083] Here, the signal charges stored in MEM2_L26 and MEM2_S27 in the previous frame ((N-1)-th frame) are read out. The signal readout process in the n-th row when read out row by row will be described.
[0084] The timing control circuit 106 turns on SEL17(n) to enable reading out signals based on the charges of the FD14 of the pixels in the nth row. The timing control circuit 106 turns on SEL17(n) and turns off RES15(n), which resets the FD14, and reads out the reset release level voltage VRES of the FD14 (time t0). Next, the timing control circuit 106 turns on TX_L28(n), transfers the signal charge held in MEM2_L26 to FD14, and reads out the signal level VSIG of the FD14 (time t1). The difference between these two signal levels, |VSIG-VRES|, is the signal level proportional to the amount of signal charge in MEM2_L26.
[0085] The timing control circuit 106 again turns on RES15(n) and then turns it off to reset the FD14, and reads out the reset release level voltage VRES of the FD14 (time t2). Next, the timing control circuit 106 turns on TX_S29(n), transfers the signal charge held in MEM2_S27 to the FD14, and reads out the signal level VSIG of the FD14 (time t3). The difference between these two signal levels, |VSIG-VRES|, is a signal level proportional to the amount of signal charge of MEM2_S27. The timing control circuit 106 repeats this row-by-row in all areas or in an area to be acquired, and acquires the signals of MEM2_L26 and MEM2_S27 as pixel signals.
[0086] In this way, the pixel 107 has two charge storage units that store the signal charges transferred to one photoelectric conversion unit PD1, and two charge storage units that hold the charges until the signal of the next frame is read out. This makes it possible to capture seamless moving images with a high dynamic range and no time regions that cannot be captured.
[0087] On the other hand, for example, if one attempts to read out the signal charges of the charge accumulation units MEM2_L26 and MEM2_S27 at the timing when the charges are transferred to the charge accumulation units MEM1_L22 and MEM1_S23, there is a possibility that noise will be introduced into the signal due to the influence of current fluctuations, etc. In addition, in the first mode, charges are intermittently transferred to one charge accumulation unit within one frame period, so the sensitivity is lower than when the charges are not transferred intermittently.
[0088] Next, the driving related to exposure in the second mode will be described with reference to Fig. 10. Tlongi in Fig. 10 indicates the accumulation time of the signal charge performed once in the N-th frame. Charge transfer to the charge accumulation unit MEM1_S23 is performed in the first mode (Fig. 8), but is not performed in the second mode.
[0089] After the charge accumulation in MEM1_L22 is completed, the charge is transferred to MEM2_L26. The operation related to the readout in the second mode is basically the same as that shown in Fig. 9. However, since the charge is not accumulated in MEM1_S23 or MEM2_S27, the charge is not transferred or read out from MEM1_S23 or MEM2_S27.
[0090] The drive mode switching process is the same as that in the first embodiment, as shown in Fig. 6. Therefore, if the acquired exposure setting satisfies a predetermined condition, control is performed in the first mode, and if the acquired exposure setting does not satisfy the predetermined condition, control is performed in the second mode.
[0091] According to this embodiment, it is possible to achieve the same effect as the first embodiment in terms of suppressing noise caused by global shutter driving of the image sensor.
[0092] (Third embodiment) 11 is an equivalent circuit diagram of one pixel in an image sensor according to the third embodiment of the present invention. In the following description, the same components as those in the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.
[0093] In the first embodiment, four charge accumulation units are connected to the photoelectric conversion unit PD1, but in this embodiment, two charge accumulation units are connected to the photoelectric conversion unit PD1. The pixel 107 in this embodiment corresponds to the pixel in the first embodiment (FIG. 2) in which the charge transfer units GS_SA4 and GS_SB5, the charge accumulation units MEM_SA8 and MEM_SB9, and the transfer units TX_SA12 and TX_SB13 are eliminated.
[0094] In comparison with the first embodiment (FIG. 2), the charge transfer units GS_A30 and GS_B31, and the charge storage units MEM_A32 and MEM_B33 are configured similarly to the charge transfer units GS_LA2 and GS_LB3, and the charge storage units MEM_LA6 and MEM_LB7, respectively. Also, the transfer units TX_A38 and TX_B39 are configured similarly to the transfer units TX_LA10 and TX_LB11, respectively.
[0095] In this embodiment, the "at least two charge accumulation units that accumulate charges transferred from the photoelectric conversion unit" corresponds to the "set of MEM_A32 and MEM_B33." MEM_A32 and MEM_B33 are connected in parallel to the rear stage of the photoelectric conversion unit PD1.
[0096] Next, a method for driving the pixel portion 101 in this embodiment will be described with reference to FIGS.
[0097] Fig. 12 to Fig. 14 are timing charts showing the time series transition of drive pulses supplied to each control electrode and transfer electrode. In particular, Fig. 12 and Fig. 14 show drive related to exposure of one pixel, and Fig. 13 shows drive related to signal readout of multiple pixels. Fig. 12 and Fig. 13 show drive in the first mode, and Fig. 14 shows drive in the second mode. The meaning of the subscript n is the same as that explained in the first embodiment.
[0098] As shown in Fig. 12, unlike the first embodiment, in the first mode, there is only one charge accumulation unit (either MEM_A32 or MEM_B33) to which charges are transferred from the photoelectric conversion unit PD1 during one frame period. As shown in Fig. 13, unlike the first embodiment, in the first mode, there is only one charge accumulation unit (either MEM_A32 or MEM_B33) from which signal charges are read out during one frame period. In the first mode, charges are intermittently transferred to one charge accumulation unit during one frame period, resulting in low sensitivity.
[0099] The operation of driving related to exposure in the second mode is as shown in Fig. 14. Ti in Fig. 14 is the same as Tlongi shown in Fig. 5. In the second mode, the operation in each of the even-numbered frames and odd-numbered frames is the same as that described in Fig. 5.
[0100] The drive mode switching process is the same as that in the first embodiment, as shown in Fig. 6. Therefore, if the acquired exposure setting satisfies a predetermined condition, control is performed in the first mode, and if the acquired exposure setting does not satisfy the predetermined condition, control is performed in the second mode.
[0101] According to this embodiment, it is possible to achieve the same effect as the first embodiment in terms of suppressing noise caused by global shutter driving of the image sensor.
[0102] (Fourth embodiment) 15 is an equivalent circuit diagram of one pixel in an image sensor according to the fourth embodiment of the present invention. In the following description, the same components as those in the second embodiment are given the same reference numerals, and detailed description thereof will be omitted.
[0103] In the second embodiment, two sets of charge accumulation units are connected in series between the photoelectric conversion units PD1 and FD 14, but in this embodiment, one set of charge accumulation units is connected in series between the photoelectric conversion units PD1 and FD 14. The pixel 107 in this embodiment corresponds to the pixel in the second embodiment (FIG. 7) in which the charge transfer units GS1_S21 and GS2_S25, the charge accumulation units MEM1_S23 and MEM2_S27, and the transfer unit TX_S29 are eliminated.
[0104] In comparison with the second embodiment (FIG. 7), the charge transfer units GS_A40 and GS_B44 are configured similarly to the charge transfer units GS1_L20 and GS2_L24, respectively. Also, the charge storage units MEM_A42 and MEM_B46, and the transfer unit TX48 are configured similarly to the charge storage units MEM1_L22, MEM2_L26, and the transfer unit TX_L28, respectively.
[0105] In this embodiment, the "at least two charge accumulation units that accumulate the charges transferred from the photoelectric conversion unit" corresponds to the "set of MEM_A42 and MEM_B46." MEM_A42 and MEM_B46 are connected in series to the rear stage of the photoelectric conversion unit PD1.
[0106] Next, a method of driving the pixel portion 101 in this embodiment will be described with reference to FIGS.
[0107] Fig. 16 to Fig. 18 are timing charts showing the time series transition of driving pulses supplied to each control electrode and transfer electrode. In particular, Fig. 16 and Fig. 18 show driving related to exposure of one pixel, and Fig. 17 shows driving related to signal readout of multiple pixels. Fig. 16 and Fig. 17 show driving in the first mode, and Fig. 18 shows driving in the second mode. The meaning of the subscript n is the same as that explained in the first embodiment. In this embodiment, there is no distinction between the driving method for even frames and odd frames, so only one frame is shown.
[0108] In the first mode, the timing control circuit 106 controls the charge accumulation unit (MEM_A42) in the preceding stage to transfer charges from the photoelectric conversion unit PD1 in multiple times during one frame period. In this embodiment, as shown in FIG. 16, there is only one charge accumulation unit (MEM_A42) to which charges are transferred from the photoelectric conversion unit PD1 during one frame period. Also, as shown in FIG. 17, there is only one charge accumulation unit (MEM_B46) from which signal charges are read during one frame period. Therefore, the charges accumulated in the charge accumulation unit (MEM_A42) in the preceding stage are transferred to the charge accumulation unit (MEM_B46) in the succeeding stage only once. In the first mode, the sensitivity is low because charges are intermittently transferred to one charge accumulation unit during one frame period.
[0109] The driving operation for exposure in the second mode is as shown in Fig. 18. Ti in Fig. 18 is the same as Tlongi shown in Fig. 10. In the second mode, the read operation is the same as that described in Fig. 10.
[0110] The drive mode switching process is the same as that in the first embodiment, as shown in Fig. 6. Therefore, if the acquired exposure setting satisfies a predetermined condition, control is performed in the first mode, and if the acquired exposure setting does not satisfy the predetermined condition, control is performed in the second mode.
[0111] According to this embodiment, it is possible to achieve the same effect as the first embodiment in terms of suppressing noise caused by global shutter driving of the image sensor.
[0112] In the first to fourth embodiments, the predetermined condition used in S102 is not limited to the exemplified one. Also, the predetermined condition may be defined by only one of the ISO sensitivity and the gain. That is, the predetermined condition may be that the ISO sensitivity is not higher than a first predetermined value. Alternatively, the predetermined condition may be that the gain value is not higher than a second predetermined value.
[0113] Fifth embodiment In the fifth embodiment of the present invention, a configuration will be described in which each pixel of the image sensor 111 is provided with a filter having a different sensitivity. Although not shown in the figure, each pixel of the pixel unit 101 is provided with a red, green, and blue color filter. That is, a Bayer array of color filters is arranged on a plurality of pixels in the pixel unit 101. In such a configuration, the optimal exposure setting differs for each pixel. Therefore, the exposure setting that is the basis for determining the pixel drive mode is a value that takes into account a gain value for performing white balance adjustment.
[0114] Fig. 19 is a flowchart showing the drive mode switching process. The execution body and start conditions of this process are the same as those of the drive mode switching process of Fig. 6. This process is performed for each of the red, green, and blue pixels.
[0115] In S201, the CPU of the image processing unit 112 executes the same process as S101 in Fig. 6. In S202, the CPU of the image processing unit 112 acquires a white balance (WB) gain to be applied in the imaging device 1000. For example, when a double gain is applied to a red pixel based on a green pixel, a numerical value of "2x" is acquired. In S203, the CPU of the image processing unit 112 calculates a total exposure setting for each pixel from the exposure setting acquired in S201 and the white balance gain acquired in S202. The exposure setting calculated in S203 is a value taking into account a gain value for performing white balance adjustment.
[0116] In S204, the CPU of the image processing unit 112 determines whether the exposure setting calculated in S203 satisfies a predetermined condition. The predetermined condition is that the exposure setting calculated in S203 is equal to or less than a predetermined value, which may be the same as that described in Fig. 6. In S205 and S206, the CPU of the image processing unit 112 executes the same processes as in S103 and S104.
[0117] According to this embodiment, in terms of suppressing noise caused by global shutter driving of the image sensor, the same effect as that of the first embodiment can be achieved. In particular, even when a Bayer array filter is arranged, noise caused by global shutter driving of the image sensor can be suppressed.
[0118] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.
[0119] The disclosure of this embodiment includes the following configuration. (Configuration 1) An imaging element including a plurality of pixels, each of which has a photoelectric conversion unit that generates electric charges by photoelectric conversion, and at least two charge accumulation units that are connected to the photoelectric conversion unit and accumulate electric charges transferred from the photoelectric conversion unit; a control means for controlling the driving mode of the imaging element to be switched between a first mode in which the charge is transferred from the photoelectric conversion unit to the charge accumulation unit in a plurality of times during one frame period and a second mode in which the charge is transferred from the photoelectric conversion unit to the charge accumulation unit only once during one frame period in accordance with an exposure setting; An imaging element comprising: (Configuration 2) The exposure setting includes an ISO sensitivity setting, The imaging device according to configuration 1, wherein the control means controls the imaging element to be driven in the first mode when the ISO sensitivity is not higher than a predetermined value, and controls the imaging element to be driven in the second mode when the ISO sensitivity is higher than the predetermined value. (Configuration 3) The exposure setting includes a gain setting in signal processing, The imaging device according to configuration 1, wherein the control means controls the imaging element to be driven in the first mode when the gain value is not higher than a predetermined value, and controls the imaging element to be driven in the second mode when the gain value is higher than the predetermined value. (Configuration 4) The imaging device according to any one of configurations 1 to 3, wherein the at least two charge accumulation sections are connected in parallel to a rear stage of the photoelectric conversion section. (Configuration 5) The imaging device described in Configuration 4, wherein in the first mode, the control means controls the imaging element so as to transfer the charges from the photoelectric conversion section to each charge accumulation section at mutually different timings during one frame period. (Configuration 6) The imaging device according to configuration 4, wherein in the first mode, the control means controls the imaging element so that the charge accumulation times in each of the charge accumulation sections during one frame period are different from each other. (Configuration 7) The at least two charge accumulation units are connected in series to a rear stage of the photoelectric conversion unit, The imaging device according to any one of configurations 1 to 3, wherein in the first mode, the control means controls the imaging element to transfer the charge from the photoelectric conversion unit to the preceding charge accumulation unit in a plurality of transfers during one frame period. (Configuration 8) A Bayer array color filter is arranged on the plurality of pixels, 4. The imaging device according to any one of configurations 1 to 3, wherein the exposure setting is a value taking into account a gain value for performing white balance adjustment. (Configuration 9) The imaging device according to any one of configurations 1 to 8, further comprising a processing means for processing a signal output from the imaging element to generate an image. (Configuration 10) The imaging device according to configuration 9, wherein the processing means generates one image by combining signals from the at least two charge storage sections of each pixel in the imaging element. [Explanation of symbols]
[0120] PD1 Photoelectric conversion unit MEM_LA6, MEM_LB7, MEM_SA8, MEM_SB9 Charge storage section 101 Pixel section 106 Control circuit 107 pixels 111 Image sensor 1000 Imaging device
Claims
1. an imaging element including a plurality of pixels, each of which has a photoelectric conversion unit that generates electric charges by photoelectric conversion, and at least two charge accumulation units that are connected to the photoelectric conversion unit and accumulate electric charges transferred from the photoelectric conversion unit; a control means for controlling the imaging device to switch between a first mode in which the charge is transferred from the photoelectric conversion unit to the charge accumulation unit in multiple transfers during one frame period and a second mode in which the charge is transferred from the photoelectric conversion unit to the charge accumulation unit only once during one frame period in accordance with an exposure setting; An imaging device comprising:
2. The exposure settings include ISO sensitivity settings, 2. The imaging device according to claim 1, wherein the control means controls the imaging element to be driven in the first mode when the ISO sensitivity is not higher than a predetermined value, and controls the imaging element to be driven in the second mode when the ISO sensitivity is higher than the predetermined value.
3. The exposure setting includes a gain setting in signal processing, 2. The imaging device according to claim 1, wherein the control means controls the imaging element to be driven in the first mode when the gain value is not higher than a predetermined value, and controls the imaging element to be driven in the second mode when the gain value is higher than the predetermined value.
4. 2. The imaging device according to claim 1, wherein the at least two charge storage sections are connected in parallel to a rear stage of the photoelectric conversion section.
5. 5. The imaging device according to claim 4, wherein in the first mode, the control means controls the imaging element so as to transfer the charges from the photoelectric conversion unit to each charge accumulation unit at different timings during one frame period.
6. 5. The imaging device according to claim 4, wherein in the first mode, the control means controls the imaging element so that the charge accumulation times in the charge accumulation sections during one frame period are different from each other.
7. the at least two charge accumulation units are connected in series to a subsequent stage of the photoelectric conversion unit; 2. The imaging device according to claim 1, wherein, in the first mode, the control means controls the imaging element so that the charge is transferred from the photoelectric conversion unit to the preceding charge accumulation unit in multiple batches during one frame period.
8. Bayer color filters are arranged on the plurality of pixels, 2. The imaging device according to claim 1, wherein the exposure setting is a value that takes into account a gain value for performing white balance adjustment.
9. 2. The imaging device according to claim 1, further comprising processing means for processing a signal output from said imaging element to generate an image.
10. 10. The imaging device according to claim 9, wherein the processing means generates one image by combining signals from the at least two charge storage portions of each pixel in the imaging element.
11. An imaging element having a plurality of pixels each having a photoelectric conversion unit that generates electric charges by photoelectric conversion, first and second charge storage units connected in parallel to each other downstream of the photoelectric conversion unit and each storing electric charges transferred from the photoelectric conversion unit, and third and fourth charge storage units connected in parallel to each other downstream of the photoelectric conversion unit and each storing electric charges transferred from the photoelectric conversion unit; a control means for controlling the driving mode of the imaging element to be switched between a first mode and a second mode; The control means transfers the charge from the photoelectric conversion unit to the first and second charge storage units, or the third and fourth charge storage units, in multiple batches during one frame period in the first mode, and transfers the charge from the photoelectric conversion unit to the first or second charge storage unit, or the third or fourth charge storage unit, only once during one frame period in the second mode.
12. The imaging device described in Claim 11, characterized in that in the first mode, the control means controls the transfer of the charges from the photoelectric conversion unit to the first and second charge storage units, or to the third and fourth charge storage units, at different times during one frame period.
13. The imaging device described in Claim 11, characterized in that in the first mode, the control means controls so that the charge accumulation times during one frame period are different between the first charge accumulation unit and the second charge accumulation unit, or between the third charge accumulation unit and the fourth charge accumulation unit.
14. The imaging device according to claim 11, wherein the control means processes the signal output from the imaging element to generate an image.
15. The imaging device described in Claim 14, characterized in that the control means generates a single image by combining the signals of the first and second charge storage sections, or the third and fourth charge storage sections, of each pixel in the imaging element.
16. An imaging element having a plurality of pixels each having a photoelectric conversion unit that generates electric charges by photoelectric conversion, first and second charge storage units connected in parallel to each other downstream of the photoelectric conversion unit and that respectively store electric charges transferred from the photoelectric conversion unit, and third and fourth charge storage units that respectively store electric charges transferred from the first and second charge storage units; a control means for controlling the driving mode of the imaging element to be switched between a first mode and a second mode; The control means transfers the charge from the photoelectric conversion unit to the first and second charge storage units in multiple batches during one frame period in the first mode, and transfers the charge from the photoelectric conversion unit to the first or second charge storage unit only once during one frame period in the second mode.
17. The imaging device described in Claim 16, characterized in that in the first mode, the control means controls the transfer of the charges from the photoelectric conversion unit to the first and second charge storage units at different times during one frame period.
18. The imaging device described in Claim 16, characterized in that in the first mode, the control means controls the first charge storage unit and the second charge storage unit so that the charge storage times during one frame period are different from each other.
19. The imaging device according to claim 16, wherein the control means processes the signal output from the imaging element to generate an image.
20. The imaging device described in Claim 19, characterized in that the control means generates a single image by combining signals from the third and fourth charge storage sections of each pixel in the imaging element.