Imaging element and imaging device

Abstract

To solve a problem in which it is difficult to capture an image by changing the accumulation times of two photoelectric conversion units in one pixel.SOLUTION: An imaging element comprises a first control line for outputting a control signal to a fifth transistor and a sixth transistor. The imaging element comprises a second control line for outputting a control signal to a seventh transistor and an eighth transistor. The imaging element comprises a first current source for supplying a current to a first transistor and a third transistor via a first signal line. The imaging element comprises a second current source for supplying a current to a second transistor and a fourth transistor via a second signal line.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an imaging element and an imaging device. [Background technology]

[0002] CMOS image sensors having multiple photoelectric conversion units in one pixel are known (see, for example, Patent Document 1). However, it has been difficult to capture an image by changing the accumulation times of the two photoelectric conversion units in one pixel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-250931 Summary of the Invention [Means for solving the problem]

[0004] One aspect of the present invention is an imaging element. The imaging element includes a first microlens onto which light is incident. The imaging element includes a second microlens arranged alongside the first microlens in the column direction and onto which light is incident. The imaging element includes a first filter having a first spectral characteristic onto which light from the first microlens is incident. The imaging element includes a second filter having a second spectral characteristic different from the first spectral characteristic onto which light from the second microlens is incident. The imaging element includes a first photoelectric conversion unit that converts light transmitted through the first filter into electric charges. The imaging element includes a second photoelectric conversion unit that converts light transmitted through the first filter into electric charges. The imaging element includes a third photoelectric conversion unit that converts light transmitted through the second filter into electric charges. The imaging element includes a fourth photoelectric conversion unit that converts light transmitted through the second filter into electric charges. The imaging element includes a first transfer unit that transfers the electric charges converted by the first photoelectric conversion unit. The imaging element includes a second transfer unit that transfers the electric charges converted by the second photoelectric conversion unit. The imaging element includes a third transfer unit that transfers the charges converted by the third photoelectric conversion unit. The imaging element includes a fourth transfer unit that transfers the charges converted by the fourth photoelectric conversion unit. The imaging element includes a first storage unit to which charges from the first photoelectric conversion unit are transferred by the first transfer unit. The imaging element includes a second storage unit to which charges from the second photoelectric conversion unit are transferred by the second transfer unit. The imaging element includes a third storage unit to which charges from the third photoelectric conversion unit are transferred by the third transfer unit. The imaging element includes a fourth storage unit to which charges from the fourth photoelectric conversion unit are transferred by the fourth transfer unit. The imaging element includes a first transistor that includes a gate unit electrically connected to the first storage unit and outputs a first signal. The imaging element includes a second transistor that includes a gate unit electrically connected to the second storage unit and outputs a second signal. The imaging element includes a third transistor that includes a gate unit electrically connected to the third storage unit and outputs a third signal. The imaging element includes a fourth transistor including a gate portion electrically connected to the fourth accumulation portion and outputting a fourth signal. The imaging element includes a fifth transistor for outputting the first signal from the first transistor to a first signal line. The imaging element includes a sixth transistor for outputting the second signal from the second transistor to a second signal line.The imaging element includes a seventh transistor for outputting a third signal from the third transistor to a first signal line. The imaging element includes an eighth transistor for outputting a fourth signal from the fourth transistor to a second signal line. The imaging element includes a first control line for outputting control signals to the fifth and sixth transistors. The imaging element includes a second control line for outputting control signals to the seventh and eighth transistors. The imaging element includes a first current source for supplying current to the first and third transistors via the first signal line. The imaging element includes a second current source for supplying current to the second and fourth transistors via the second signal line.

[0005] One aspect of the present invention is an imaging device including the imaging element of the above aspect. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an imaging element according to an embodiment. [Figure 2] FIG. 2 is a diagram showing a circuit configuration of an image sensor according to an embodiment. [Figure 3] 4 is a timing chart showing an example of the operation of the image sensor according to the embodiment. [Figure 4] 10 is a timing chart showing another example of the operation of the image sensor according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating another example of a plurality of photoelectric conversion units of the image sensor according to the embodiment. [Figure 6] FIG. 2 is a diagram showing a circuit configuration of an image sensor according to an embodiment. [Figure 7] 4 is a timing chart showing an example of the operation of the image sensor according to the embodiment. [Figure 8] 1 is a block diagram showing an imaging apparatus according to an embodiment; [Figure 9] 4 is a flowchart illustrating an example of the operation of the imaging device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] [First embodiment] FIG. 1(A) is a plan view showing an image sensor 1 according to this embodiment, FIG. 1(B) is a plan view showing an enlarged view of a pixel, and FIG. 1(C) is a cross-sectional view taken along line AA in FIG. 1(B). As shown in FIG. 1(A), the image sensor 1 has a plurality of pixels P arranged in a pixel region 1a. The plurality of pixels P are arranged two-dimensionally in a grid pattern. One of the directions in which the pixels P are arranged is called the horizontal scanning direction, and the other is called the vertical scanning direction. In this embodiment, directions perpendicular to the horizontal scanning direction and the vertical scanning direction are appropriately referred to as normal directions of the pixel region 1a (image sensor 1). The image sensor 1 is roughly plate-shaped, and the normal direction of the pixel region 1a corresponds to the thickness direction of the image sensor 1.

[0008] As shown in FIG. 1B, the pixel region 1a is provided with a light-blocking portion 2 that blocks light. The light-blocking portion 2 is, for example, a black matrix. The light-blocking portion 2 is formed in a lattice pattern and extends in both the horizontal scanning direction and the vertical scanning direction. The area surrounded by the light-blocking portion 2 is a pixel aperture Pa through which light passes. In this embodiment, a pixel P is an area surrounded by the center line of the light-blocking portion 2 when viewed from the normal direction of the pixel region 1a in FIG. 1, and includes the pixel aperture Pa and the light-blocking portion 2 around it.

[0009] Each of the plurality of pixels P is provided with a photoelectric conversion unit 3 and a photoelectric conversion unit 4. When viewed from the normal direction of the pixel region 1a, at least a part of the photoelectric conversion unit 3 and at least a part of the photoelectric conversion unit 4 are disposed inside one pixel aperture Pa. The photoelectric conversion unit 3 and the photoelectric conversion unit 4 each convert light that has passed through the pixel aperture Pa into electric charges.

[0010] In this embodiment, the image sensor 1 is capable of capturing a full-color image. The plurality of pixels P includes red pixels R, green pixels Gr, green pixels Gb, and blue pixels B. In FIG. 1(B), the pixels P are arranged in a Bayer array, in which the red pixels R and green pixels Gr are alternately arranged in the horizontal scanning direction, and the green pixels Gb and blue pixels B are alternately arranged. In addition, the red pixels R and green pixels Gb are alternately arranged in the vertical scanning direction, and the green pixels Gr and blue pixels B are alternately arranged. In this embodiment, a photoelectric conversion unit 3 and a photoelectric conversion unit 4 are provided in each color pixel.

[0011] The imaging element 1 includes an imaging section 5, a signal processing section 6, and a storage section 7, which are stacked on top of each other. The signal processing section 6 is stacked on the storage section 7 and is electrically connected to the storage section 7 by conductive bumps or the like (not shown). The imaging section 5 is stacked on the signal processing section 6 and is electrically connected to the signal processing section 6 by conductive bumps or the like (not shown).

[0012] The imaging unit 5 is, for example, a back-illuminated CMOS image sensor. The imaging unit 5 includes an element layer 11, a light-receiving layer 12, a color filter layer 13, and a lens layer 14. A passivation film, a planarization film, an anti-reflection film, etc. (not shown) may be provided between these layers. When capturing an image, the imaging unit 5 is positioned with the lens layer 14 facing the object to be captured. Light from the object to be captured enters the light-receiving layer 12 via the lens layer 14 and the color filter layer 13.

[0013] The lens layer 14 is laminated on the color filter layer 13. The lens layer 14 is, for example, a microlens array and includes a plurality of lens elements 14a. The lens elements 14a correspond, for example, one-to-one to the pixels P and are provided for each pixel P. The optical axis 14b of the lens elements 14a is set, for example, to pass through the center of the pixel aperture Pa (pixel P). Each of the plurality of lens elements 14a focuses light incident from the outside onto the light receiving layer 12 of the pixel P in which the lens element 14a is provided.

[0014] The color filter layer 13 is stacked on the light receiving layer 12. The color filter layer 13 includes a first filter 15, a second filter 16, and a light-shielding portion 2. The first filter 15 is provided in the pixel aperture Pa of the red pixel R. The first filter 15 transmits light in the red wavelength band and absorbs light other than the red wavelength band. The red wavelength band is, for example, a wavelength band of 620 nm to 750 nm, including 700 nm. The second filter 16 is provided in the pixel aperture Pa of the green pixel Gr. The second filter 16 transmits light in the green wavelength band and absorbs light other than the green wavelength band. The green wavelength band is, for example, a wavelength band of 495 nm to 570 nm, including 546.1 nm.

[0015] Although not shown in FIG. 1C, a second filter 16 is provided in the pixel aperture Pa of the green pixel Gb, and a third filter is provided in the pixel aperture Pa of the blue pixel B. The third filter transmits light in the blue wavelength band and absorbs light other than the blue wavelength band. The blue wavelength band is, for example, a wavelength band of 450 nm to 495 nm, including 435.8 nm.

[0016] The light-shielding portion 2 absorbs light in the wavelength band to which the photoelectric conversion portion 3 and the photoelectric conversion portion 4 are sensitive. For example, in the case of an image sensor 1 used in a visible light camera or the like, the light-shielding portion 2 is formed of a material that absorbs light in the wavelength band of 380 nm or more and 780 nm or less. The light-shielding portion 2 is provided, for example, to suppress crosstalk between two adjacent pixels. For example, the thickness of the light-shielding portion 2 is set so as to block light that has passed through the pixel aperture Pa of one pixel P and is directed toward the light-receiving layer 12 of another pixel P.

[0017] The light-receiving layer 12 is laminated on the element layer 11. The light-receiving layer 12 includes a photoelectric conversion unit 3 and a photoelectric conversion unit 4. The photoelectric conversion unit 3 and the photoelectric conversion unit 4 include, for example, a photodiode, but may also include, for example, a phototransistor as long as they convert light into an electric charge. In FIG. 1(C), the photoelectric conversion unit 4 is provided symmetrically to the photoelectric conversion unit 3 with respect to the optical axis of the lens element 14a and a plane parallel to the vertical scanning direction (a plane perpendicular to the horizontal scanning direction).

[0018] The photoelectric conversion unit 4 is provided near the photoelectric conversion unit 3 of the same pixel P. For example, the distance d1 between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 in the same pixel P is set to be narrower than the distance d2 between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of the adjacent pixel P, but may be set to be the same as the distance d2 or may be set to be longer than the distance d2.

[0019] The photoelectric conversion units 3 and 4 are insulated from each other. For example, a PN junction suppresses charge transfer between the photoelectric conversion units 3 and 4 in the same pixel P. An insulating unit such as an oxide film or trench structure formed by a LOCOS method or the like is provided between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of an adjacent pixel P. The structure between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 in the same pixel P may be the same as the structure between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of an adjacent pixel P.

[0020] The element layer 11 is provided on the side opposite to the light incident side of the light receiving layer 12. The element layer 11 is provided with circuits (to be shown later in FIG. 2 and the like) for reading out charges from each of the photoelectric conversion units 3 and 4. The element layer 11 may also be provided with a circuit for generating a signal based on charges read out from at least one of the photoelectric conversion units 3 and 4. These circuits include at least one of electronic components such as switching elements such as transistors, various wirings, capacitors, and terminals. These circuits are formed in a multilayer structure using, for example, interlayer insulating films and vias. At least some of the electronic components constituting these circuits may be provided in the light receiving layer 12, or may be provided between the light receiving layer 12 and the lens layer 14.

[0021] The signal processing unit 6 has a first surface 6a facing the imaging unit 5 and a second surface 6b facing the storage unit 7. At least one of the first surface 6a and the second surface 6b is provided with a circuit (not shown) that processes signals output from the imaging unit 5. At least one of the first surface 6a and the second surface 6b may be provided with routing wiring used for rewiring or the like. The signal processing unit 6 is provided with a conductive portion that electrically connects the first surface 6a side and the second surface 6b side. This conductive portion includes, for example, a TSV (Through-Silicon Via) and is capable of transmitting signals between the first surface 6a and the second surface 6b. The storage unit 7 includes, for example, at least one of a non-volatile memory and a volatile memory, and stores signals processed by the signal processing unit 6.

[0022] Note that instead of being stacked on the imaging unit 5, the signal processing unit 6 may be provided, for example, around the pixel region 1a shown in FIG. 1. Furthermore, the imaging element 1 does not need to include at least a part of the signal processing unit 6 and the storage unit 7. At least a part of the signal processing unit 6 and the storage unit 7 may be provided in a device external to the imaging element 1. This external device may be provided in, for example, a camera, a measuring device, or other device in which the imaging element 1 is implemented.

[0023] Fig. 2 is a diagram showing the circuit configuration of the image sensor 1 according to this embodiment. Fig. 2 shows an equivalent circuit diagram for one pixel. The image sensor 1 includes a first circuit system that reads out a first output signal based on the charge accumulated in the photoelectric conversion unit 3 of the pixel P, and a second circuit system that reads out a second output signal based on the charge accumulated in the photoelectric conversion unit 4 of the pixel P.

[0024] First, the first circuit system will be described. The first circuit system includes a transfer unit 21, a transfer control line 22, a charge-voltage conversion unit 23, a reset unit 24, a reset control line 25, a selection unit 26, a selection control line 27, an amplifier unit 28, a power supply line 29, and a vertical signal line 30. The transfer unit 21, the reset unit 24, the selection unit 26, and the amplifier unit 28 each include a switching element such as an n-type MOS transistor.

[0025] The photoelectric conversion unit 3 has an anode grounded and a cathode connected to the source of the transfer unit 21. The transfer unit 21 is used to transfer charges from the photoelectric conversion unit 3. The drain of the transfer unit 21 is electrically connected to a node 31. The gate of the transfer unit 21 is connected to a transfer control line 22. The transfer control line 22 can transmit a control signal TX1 to the gate of the transfer unit 21.

[0026] The charge-voltage conversion unit 23 is, for example, a floating diffusion, and converts the charge from the transfer unit 21 into a voltage. The charge-voltage conversion unit 23 includes a capacitance 32. The first electrode of the capacitance 32 is grounded, and the second electrode thereof is connected to a node 31. The capacitance 32 may include at least a portion of the capacitance connected to the node 31. The capacitance connected to the node 31 includes, for example, the capacitance of the drain of the transfer unit 21, the coupling capacitance between the wiring connected to the node 31 and other wiring, and the capacitance of the source of the reset unit 24.

[0027] The reset unit 24 resets the voltage converted by the charge-voltage conversion unit 23. The source of the reset unit 24 is connected to a node 31. The drain of the reset unit 24 is connected to a power supply line 29 that supplies a predetermined potential relative to a reference potential (ground potential). The gate of the reset unit 24 is connected to a reset control line 25. The reset control line 25 can transmit a control signal RST1 to the gate of the reset unit 24.

[0028] The selection unit 26 is used to select a pixel P from which a signal is to be read out, from among the multiple pixels P arranged in the vertical scanning line direction shown in FIG. 1. The source of the selection unit 26 is connected to a power supply line 29. The drain of the selection unit 26 is connected to the source of the amplifier unit 28. The gate of the selection unit 26 is connected to a selection control line 27. The selection control line 27 can transmit a control signal SEL to the gate of the selection unit 26.

[0029] The amplifier 28 amplifies the voltage converted by the charge-voltage converter 23. The source of the amplifier 28 is connected to a power supply line 29 via the selector 26. A power supply voltage VDD can be supplied to the source of the amplifier 28 via the selector 26 and the power supply line 29. The drain of the amplifier 28 is connected to a vertical signal line 30. The gate of the amplifier 28 is connected to a node 31.

[0030] Next, we will explain the second circuit system that reads out the second output signal based on the charge accumulated in the photoelectric conversion unit 4 of the pixel P. The second circuit system includes a transfer unit 41, a transfer control line 42, a charge-voltage conversion unit 43, a reset unit 44, a reset control line 45, a selection unit 46, an amplifier unit 47, a selection control line 27, a power supply line 29, and a vertical signal line 48. The transfer unit 41, the reset unit 44, the selection unit 46, and the amplifier unit 47 each include a switching element such as an n-type MOS transistor.

[0031] The photoelectric conversion unit 4 has an anode that is grounded and a cathode that is connected to the source of the transfer unit 41. The transfer unit 41 is used to transfer charges from the photoelectric conversion unit 4. The drain of the transfer unit 41 is electrically connected to a node 50. The gate of the transfer unit 41 is connected to a transfer control line 42. The transfer control line 42 is provided separately from the transfer control line 22 and is insulated from the transfer control line 22. The transfer control line 42 can transmit a control signal TX2 to the gate of the transfer unit 41.

[0032] The charge-voltage conversion unit 43 is, for example, a floating diffusion, and converts the charge from the transfer unit 41 into a voltage. The charge-voltage conversion unit 43 includes a capacitance 49. A first electrode of the capacitance 49 is grounded, and a second electrode of the capacitance 49 is connected to a node 50. The capacitance 49 may include at least a portion of a capacitance connected to the node 50. The capacitance connected to the node 50 includes, for example, a capacitance of the drain of the transfer unit 41, a coupling capacitance between a wiring connected to the node 50 and another wiring, and a capacitance of the source of the reset unit 44.

[0033] The reset unit 44 resets the voltage converted by the charge-voltage conversion unit 43. The source of the reset unit 44 is connected to a node 50. The drain of the reset unit 44 is connected to the power supply line 29. The gate of the reset unit 44 is connected to a reset control line 45. The reset control line 45 can transmit a control signal RST2 to the gate of the reset unit 44.

[0034] The source of the selection unit 46 is connected to the power supply line 29. The drain of the selection unit 46 is connected to the source of the amplification unit 47. The gate of the selection unit 46 is connected to a selection control line 27. The selection control line 27 can transmit a control signal SEL to the gate of the selection unit 46.

[0035] The amplifier 47 amplifies the voltage converted by the charge-voltage converter 43. The source of the amplifier 47 is connected to the power supply line 29 via the selector 46. A power supply voltage VDD can be supplied to the source of the amplifier 47 via the selector 46 and the power supply line 29. The drain of the amplifier 47 is connected to the vertical signal line 30. The gate of the amplifier 47 is connected to a node 50.

[0036] Here, the circuit configuration of one pixel out of the plurality of pixels P shown in FIG. 1 has been described, but all of the plurality of pixels P have the same circuit configuration. For example, the transfer control lines 22 extend roughly in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. Each of the plurality of transfer control lines 22 is connected to the gate of the transfer unit 21 of two or more pixels P arranged in the horizontal scanning direction. Furthermore, the transfer control lines 42 extend roughly in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. Each of the plurality of transfer control lines 42 is connected to the gate of the transfer unit 41 of two or more pixels P arranged in the horizontal scanning direction.

[0037] The reset control lines 25 extend generally in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. Each of the reset control lines 25 is connected to the gate of the reset unit 24 of two or more pixels P arranged in the horizontal scanning direction. The reset control lines 45 extend generally in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. Each of the reset control lines 45 is connected to the gate of the reset unit 44 of two or more pixels P arranged in the horizontal scanning direction.

[0038] The selection control lines 27 extend generally in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. Each of the selection control lines 27 is connected to the gate of the selection unit 26 and the gate of the selection unit 46 of two or more pixels P arranged in the horizontal scanning direction. The power supply line 29 is connected to the source of the selection unit 26, the source of the selection unit 46, the source of the reset unit 24, and the source of the reset unit 44 of two or more pixels P. The vertical signal lines 30 extend generally in the vertical scanning direction and are periodically arranged in the horizontal scanning direction. Each of the vertical signal lines 30 is connected to the drain of the amplifier unit 47 of two or more pixels P arranged in the vertical scanning direction.

[0039] 1C. That is, the transfer control lines 22 and 42 are arranged on the side opposite to the light incident side with respect to the photoelectric conversion units 3 and 4. Similarly, the reset control lines 25, 45, selection control lines 27, power supply lines 29, and vertical signal lines 30 are also arranged on the side opposite to the light incident side with respect to the photoelectric conversion units 3 and 4.

[0040] At least some of the transfer control lines 22, 42, reset control lines 25, 45, selection control lines 27, power supply lines 29, and vertical signal lines 30 may be arranged on the same side as the light incident side with respect to the photoelectric conversion units 3 and 4. At least some of these wirings may be provided in the light receiving layer 12 shown in FIG. 1(C), or may be provided between the light receiving layer 12 and the lens layer 14.

[0041] The imaging element 1 is provided with a vertical scanning circuit 51, a horizontal scanning circuit 52, a constant current source 53, and a constant current source 54. The vertical scanning circuit 51 supplies a control signal for reading out a signal based on electric charge from each of the plurality of pixels P.

[0042] First, an example of the operation of reading out a first output signal based on the charge accumulated in the photoelectric conversion unit 3 will be described. The vertical scanning circuit 51 supplies a control signal SEL to the selection control line 27. When the vertical scanning circuit 51 sets the control signal SEL for the selection control line 27 to an H level (high level), the source and drain of the selection unit 26 and the source and drain of the selection unit 46 connected to this selection control line 27 become electrically conductive (hereinafter referred to as an ON state). As a result, the power supply voltage VDD is supplied to the source of the amplification unit 28 via the power supply line 29 and the selection unit 26.

[0043] When the selection unit 26 reads out charges from pixels P in other rows, the control signal SEL for the selection control line 27 is set to an L level (low level), thereby disabling current flow between the source and drain of the selection unit 26 (hereinafter referred to as an "off state"). The vertical scanning circuit 51 can also turn off the selection unit 26 during operation of a circuit (e.g., an amplifier 55 described later) arranged downstream of the vertical signal line 30 in signal transmission. In this case, transmission of noise from the vertical signal line 30 to the amplifier unit 28 can be suppressed.

[0044] The vertical scanning circuit 51 also supplies a control signal TX1 to the transfer control line 22. When the vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to an H level, the transfer unit 21 connected to this transfer control line 22 is turned on, and the charge accumulated in the photoelectric conversion unit 3 is transferred to the charge-voltage conversion unit 23. The capacitor 32 is charged with the charge transferred from the photoelectric conversion unit 3 via the transfer unit 21, and the voltage between its first electrode and second electrode becomes a voltage corresponding to the charge. This voltage is applied to the gate of the amplifier unit 28 via the node 31, and the resistance value between the source and drain of the amplifier unit 28 becomes a value corresponding to the voltage applied to the gate of the amplifier unit 28. Furthermore, a current (signal) corresponding to the predetermined voltage applied to the source via the power supply line 29 and the selection unit 26 and the resistance value between the source and drain of the amplifier unit 28 flows between the source and drain of the amplifier unit 28. This signal is output to the vertical scanning circuit 51 via a vertical signal line 30.

[0045] Furthermore, the vertical scanning circuit 51 supplies a control signal RST1 to the reset control line 25. When the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to an H level, the reset unit 24 connected to this reset control line 25 is turned on, and the charge accumulated in the capacitor 32 of the charge-voltage conversion unit 23 is discharged via the reset unit 24 and the power supply line 29.

[0046] Next, an example of the operation of reading out the second output signal based on the charge accumulated in the photoelectric conversion unit 4 will be described. When the vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to H level, the power supply voltage VDD is supplied to the source of the amplifier unit 47 via the power supply line 29 and the selector unit 46. The vertical scanning circuit 51 supplies a control signal TX2 to the transfer control line 42. When the vertical scanning circuit 51 sets the control signal TX2 for the transfer control line 42 to H level, the transfer unit 41 connected to this transfer control line 42 is turned on, and the charge accumulated in the photoelectric conversion unit 4 is transferred to the charge-voltage converter 43.

[0047] The capacitor 49 is charged by the charge transferred from the photoelectric conversion unit 4 via the transfer unit 41, and the voltage between its first electrode and second electrode becomes a voltage corresponding to the charge. This voltage is applied to the gate of the amplifier unit 47 via a node 50, and the resistance value between the source and drain of the amplifier unit 47 becomes a value corresponding to the voltage applied to the gate of the amplifier unit 47. Furthermore, a current (signal) corresponding to the predetermined voltage applied to the source via the power supply line 29 and the selection unit 46 and the resistance value between the source and drain of the amplifier unit 47 flows between the source and drain of the amplifier unit 47. This signal is output to a vertical scanning circuit 51 via a vertical signal line 48.

[0048] Furthermore, the vertical scanning circuit 51 supplies a control signal RST2 to the reset control line 45. When the vertical scanning circuit 51 sets the control signal RST2 for the reset control line 45 to an H level, the reset unit 44 connected to this reset control line 45 is turned on, and the charge accumulated in the capacitance 49 of the charge-voltage conversion unit 43 is discharged via the reset unit 44 and the power supply line 29.

[0049] In this embodiment, a constant current source 53 is connected to the vertical signal line 30, and a constant current source 54 is connected to the vertical signal line 48. When the selection unit 26 is in the on state, a source follower circuit is formed by the amplifier unit 28, the selection unit 26, and the constant current source 53 connected to the vertical signal line 30. In this case, a signal from a pixel P belonging to the row selected by the selection unit 26 is output to the vertical signal line 30. Similarly, when the selection unit 46 is in the on state, a source follower circuit is formed by the amplifier unit 47, the selection unit 46, and the constant current source 54 connected to the vertical signal line 48.

[0050] The horizontal scanning circuit 52 converts the signals output from each of the plurality of pixels P into image data in a predetermined format. The horizontal scanning circuit 52 reduces noise in the signals output from the pixels P by correlated double sampling.

[0051] The horizontal scanning circuit 52 includes an amplifier 55, an amplifier 56, and an analog-to-digital converter (not shown). The amplifier 55 is an amplifier that amplifies a signal corresponding to the output from the photoelectric conversion unit 3. The amplifier 55 is, for example, a column amplifier, and is provided for each vertical signal line 30. The amplifier 55 amplifies a signal output via the vertical signal line 30. The amplifier 56 is an amplifier that amplifies a signal corresponding to the output from the photoelectric conversion unit 4. The amplifier 56 is, for example, a column amplifier, and is provided for each vertical signal line 48. The amplifier 56 amplifies a signal output via the vertical signal line 48.

[0052] In this embodiment, the amplification factor (gain) of amplifier 55 and the amplification factor of amplifier 56 are each variable. The amplification factor of amplifier 55 can be set to the same value as or a different value from the amplification factor of amplifier 56. The amplification factors of amplifier 55 and amplifier 56 can be set to, for example, about several tenths to several tens of times. When adjusting the amplification factors of amplifier 55 and amplifier 56, the amplification factors of amplifier 55 and amplifier 56 may be adjusted in conjunction with each other, or the amplification factors of amplifier 55 and amplifier 56 may be adjusted independently. For example, the ratio between the amplification factor of amplifier 55 and the amplification factor of amplifier 56 may be constant or fixed.

[0053] The analog-to-digital converter of the horizontal scanning circuit 52 converts the analog signals output from the amplifiers 55 and 56 into digital signals. These digital signals include, for example, gradation values ​​(pixel values) that represent the output of each pixel P in 8 bits. The horizontal scanning circuit 52 arranges the pixel values ​​representing the output of each pixel P according to the pixel arrangement, and outputs them in a predetermined data format. Note that the horizontal scanning circuit 52 may include at least a part of the constant current sources 53 and 54.

[0054] Next, the timing for reading out charges from the photoelectric conversion unit will be described with reference to a timing chart of FIG.

[0055] The image sensor 1 performs dummy readout after startup and before capturing the first image. As shown in FIG. 3 , at time t1, which is before the first frame period, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to an H level and sets the control signal RST2 for the reset control line 45 to an H level. This resets the voltage between the electrodes of the capacitor 32 of the charge-voltage converter 23 and the voltage between the electrodes of the capacitor 49 of the charge-voltage converter 43. At time t2, the vertical scanning circuit 51 sets the control signals RST1 and RST2 to an L level (low level). At time t3, the vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to an H level and sets the control signal TX2 for the transfer control line 42 to an H level. As a result, if charge has accumulated in the photoelectric conversion unit 3, the charge is transferred to the charge-voltage conversion unit 23. Furthermore, if charge has accumulated in the photoelectric conversion unit 4, the charge is transferred to the charge-voltage conversion unit 43. At time t4, the vertical scanning circuit 51 sets the control signals TX1 and TX2 to the L level.

[0056] Here, dummy readout from the photoelectric conversion unit 3 is performed in parallel with dummy readout from the photoelectric conversion unit 4, but the dummy readout from the photoelectric conversion unit 3 and the dummy readout from the photoelectric conversion unit 4 may be performed in periods that do not overlap. For example, the vertical scanning circuit 51 may set the period in which the control signal RST1 is set to H level and the period in which the control signal RST2 is set to H level so that they at least partially overlap, or may not overlap. Furthermore, the vertical scanning circuit 51 may set the period in which the control signal TX1 is set to H level and the period in which the control signal TX2 is set to H level so that they at least partially overlap, or may not overlap.

[0057] The image sensor 1 starts imaging processing for the first frame at time t5. The vertical scanning circuit 51 maintains the control signal SEL for the selection unit 26 at H level. The vertical scanning circuit 51 also performs a reset operation on the photoelectric conversion unit 3 during the period from time t6 to time t7. During this period, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to H level and then to L level. Next, the vertical scanning circuit 51 sets the control signal TX1 for the transfer unit 21 to H level and then to L level. Here, the vertical scanning circuit 51 sets each of the control signals RST1 and TX1 to three pulses, but the number of pulses may be one, two, or four or more. After the control signal TX1 is set to L level at time t7, electric charges corresponding to the amount of exposure are accumulated in the photoelectric conversion unit 3.

[0058] The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 3 during the period from time t8 to time t10. The vertical scanning circuit 51 sets the control signal RST1 to an H level at time t8 and then to an L level. The vertical scanning circuit 51 also sets the control signal TX1 to an H level at time t9, which is later than time t8, and then to an L level at time t10. Exposure of the photoelectric conversion unit 3 during the first frame period ends at time t9. The photoelectric conversion unit 3 outputs the charges accumulated during the exposure period from time t7 to time t9 to the charge-voltage conversion unit 23 via the transfer unit 21. The amplification unit 28 also outputs a signal corresponding to the voltage converted by the charge-voltage conversion unit 23 to the vertical signal line 30.

[0059] In this embodiment, the image sensor 1 exposes the photoelectric conversion unit 4 during a period that overlaps at least a portion of the exposure period for the photoelectric conversion unit 3. The vertical scanning circuit 51 resets the photoelectric conversion unit 4 during the first frame period, from time t11 to time t12, which is after time t7. During this period, the vertical scanning circuit 51 sets the control signal RST2 for the reset control line 45 to a high level and then to a low level. Next, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to a high level and then to a low level. Here, the vertical scanning circuit 51 sets each of the control signals RST2 and TX2 to three pulses, but the number of pulses may be one, two, or four or more. After the control signal TX2 is set to a low level at time t12, electric charges corresponding to the amount of exposure are accumulated in the photoelectric conversion unit 4.

[0060] The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 4 when the exposure period of the photoelectric conversion unit 4 ends. Here, the vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 4 in parallel with reading out charges from the photoelectric conversion unit 3. The vertical scanning circuit 51 sets the control signal RST2 to an H level at time t8 and then to an L level. The vertical scanning circuit 51 also sets the control signal TX2 to an H level at time t9 and to an L level at time t10. The photoelectric conversion unit 4 outputs the charges accumulated during the exposure period from time t12 to time t9 to the charge-voltage conversion unit 43 via the transfer unit 41. The amplifier 47 also outputs a signal corresponding to the voltage converted by the charge-voltage conversion unit 43 to a vertical signal line 48.

[0061] In this way, the first frame period ends at time t10, and the next second frame period begins. In Figure 3, the same operations as those in the first frame period are repeated in the second frame period.

[0062] In the example of Figure 3, during the first frame, the period during which charge is accumulated in photoelectric conversion unit 3 (exposure period from time t7 to time t9) is set to be longer than the period during which charge is accumulated in photoelectric conversion unit 4 (exposure period from time t12 to time t9).

[0063] In this way, the image sensor 1 can generate a first output signal based on the charge accumulated in the photoelectric conversion unit 3 and a second output signal based on the charge accumulated in the photoelectric conversion unit 4 by varying the length of the exposure period between the photoelectric conversion unit 3 and the photoelectric conversion unit 4. Using such first and second output signals, two images with different exposure periods can be generated. This allows, for example, to generate an image with a wide dynamic range. For example, if the exposure period of the first output signal is longer than the exposure period of the second output signal, the first output signal is used to generate an image of a relatively dark area of ​​the imaged subject, and the second output signal is used to generate an image of a relatively bright area of ​​the imaged subject. By combining these images, it is possible to brighten the dark areas while suppressing saturation in the bright areas.

[0064] One way to expand the dynamic range is to use an image sensor with one photoelectric conversion unit per pixel and alternate between frames with long and short exposure times. This method requires two frames to obtain one frame of image, resulting in a slower frame rate. Another method uses a signal from an image with a short exposure time and a signal with a long exposure time. This method requires two pixel signals to form one pixel of the image, resulting in a lower resolution.

[0065] In this embodiment, multiple photoelectric conversion units (photoelectric conversion unit 3 and photoelectric conversion unit 4) are arranged in one pixel, thereby widening the dynamic range without reducing the resolution of the captured image. Furthermore, the transfer control line 22, which transmits a control signal TX1 to the transfer unit 21 for transferring electric charges from the photoelectric conversion unit 3, is provided separately from the transfer control line 42, which transmits a control signal TX2 to the transfer unit 41 for transferring electric charges from the photoelectric conversion unit 4. This allows exposure of the photoelectric conversion unit 3 and exposure of the photoelectric conversion unit 4 to be performed in parallel, thereby preventing a decrease in frame rate. In such a case, by arranging the transfer control line 22 and the transfer control line 42 on the opposite side of the light incident side of the photoelectric conversion unit 3 and the photoelectric conversion unit 4, it is possible to prevent the light receiving areas of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 from being narrowed. The dynamic range can also be widened by adjusting the ratio between the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56.

[0066] 3, the time t9 at which exposure of the photoelectric conversion unit 4 ends is set to be approximately the same as the time t9 at which exposure of the photoelectric conversion unit 3 ends. This makes it possible to avoid complicating the control of reading out signals from the pixel P.

[0067] 4A to 4D are timing charts showing other examples of the operation of the image sensor 1. Each of Fig. 4A to Fig. 4D shows a timing chart for one frame period. Note that in each of Fig. 4A to Fig. 4D, the operation of reading out charges from the photoelectric conversion unit 3 is the same as the example shown in Fig. 3, and therefore the description thereof will be omitted or simplified.

[0068] 4A, the time at which the exposure period for photoelectric conversion unit 4 ends is set to a time different from the time at which the exposure period for photoelectric conversion unit 3 ends. During the period from time t6 to time t7, vertical scanning circuit 51 resets photoelectric conversion unit 3 and photoelectric conversion unit 4 in parallel. At time t7, exposure of photoelectric conversion unit 3 starts, and exposure of photoelectric conversion unit 4 starts.

[0069] At time t15, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. At time t16, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then to L level. The exposure period (charge accumulation period) for the photoelectric conversion unit 4 in the first frame period is the period from time t7 to time t16.

[0070] 4(B), the time at which the exposure period for the photoelectric conversion unit 4 starts is set to be different from the time at which the exposure period for the photoelectric conversion unit 3 starts. In addition, the time at which the exposure period for the photoelectric conversion unit 4 ends is set to be different from the time at which the exposure period for the photoelectric conversion unit 3 ends. The vertical scanning circuit 51 resets the photoelectric conversion unit 4 during the period from time t17 to time t18, which is after time t6 when resetting of the photoelectric conversion unit 3 starts. The exposure period for the photoelectric conversion unit 4 starts at time t18.

[0071] At time t19, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. At time t20, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then to L level. The exposure period (charge accumulation period) for the photoelectric conversion unit 4 in the first frame period is the period from time t18 to time t20. Here, the exposure period for the photoelectric conversion unit 4 is set so that its center time coincides with the center time of the exposure period for the photoelectric conversion unit 3.

[0072] In FIG. 4C, a plurality of exposure periods are set for the photoelectric conversion unit 4. The vertical scanning circuit 51 resets the photoelectric conversion unit 4 during the period from time t6 to time t7. A first exposure period for the photoelectric conversion unit 4 starts at time t7. At time t21, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to an H level and then to an L level. At time t22, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to an H level and then to an L level. The first exposure period for the photoelectric conversion unit 4 during the first frame period is the period from time t7 to time t22. Next, the vertical scanning circuit 51 resets the photoelectric conversion unit 4 during the period from time t23 to time t24. A second exposure period for the photoelectric conversion unit 4 starts at time t24. At time t8, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. At time t9, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then to L level. The second exposure period for the photoelectric conversion unit 4 in the first frame period is the period from time t24 to time t9.

[0073] The length of the first exposure period for the photoelectric conversion unit 4 in one frame may be set to be the same as or different from the second exposure period. When performing multiple exposures in one frame period, a signal based on the charges accumulated in the photoelectric conversion unit 4 during the first exposure period and a signal based on the charges accumulated in the photoelectric conversion unit 4 during the second exposure period may be used to perform calculations such as averaging these signals. Furthermore, when the lengths of the first and second exposure periods for the photoelectric conversion unit 4 are different, the dynamic range can be further expanded by using a signal based on the charges accumulated in the photoelectric conversion unit 4 during the first exposure period, a signal based on the charges accumulated in the photoelectric conversion unit 4 during the second exposure period, and a signal based on the charges accumulated in the photoelectric conversion unit 4 during this frame period.

[0074] 4(D), the exposure period for the photoelectric conversion unit 4 is set to be the same length as the exposure period for the photoelectric conversion unit 3. Here, the time t7 when the exposure period for the photoelectric conversion unit 4 starts is set to be the same as the time t7 when the exposure period for the photoelectric conversion unit 3 starts. In addition, the time t9 when the exposure period for the photoelectric conversion unit 4 ends is set to be the same as the time t9 when the exposure period for the photoelectric conversion unit 3 ends.

[0075] When the exposure periods of the photoelectric conversion units 3 and 4 are set to the same length, information regarding focus at the image sensor 1 may be detected using a first output signal based on charges accumulated in the photoelectric conversion unit 3 during a first frame period and a second output signal based on charges accumulated in the photoelectric conversion unit 4 during the first frame period. For example, when the levels of the first output signal and the second output signal are equivalent, it may be determined that the light receiving layer 12 of the image sensor 1 is in focus. When the level of one of the first output signal and the second output signal is higher than the other, it may be determined that the focus is closer to the object plane than the light receiving layer 12 (front focus). When the level of one of the first output signal and the second output signal is lower than the other, it may be determined that the light receiving layer 12 is closer to the object plane than the focus (back focus).

[0076] When the exposure periods of the photoelectric conversion units 3 and 4 are set to different lengths, information regarding focus at the image sensor 1 may be detected using the first and second output signals. In this case, information regarding focus can be detected in the same manner as when the exposure periods of the photoelectric conversion units 3 and 4 are set to the same length, using an estimated value of the signal level when the exposure periods are set to the same length. For example, when the exposure period for the photoelectric conversion unit 4 is half that of the exposure period for the photoelectric conversion unit 3, the estimated value of the signal level when the exposure periods are set to the same length can be obtained by halving the level of the first output signal or doubling the level of the second output signal. The level of the first output signal may be adjusted by adjusting the amplification factor of the amplifier 55 shown in FIG. 2, and the level of the second output signal may be adjusted by adjusting the amplification factor of the amplifier 56. At least one of the levels of the first and second output signals may be adjusted by arithmetic processing performed by the horizontal scanning circuit 52 shown in FIG. 2 or an external device.

[0077] In this embodiment, when multiple photoelectric conversion units (photoelectric conversion unit 3 and photoelectric conversion unit 4) are arranged in one pixel, the output per photoelectric conversion unit is smaller than when a single photoelectric conversion unit having a light-receiving area equal to the total light-receiving area of ​​the multiple photoelectric conversion units is used. Therefore, if the level of the first output signal or the second output is insufficient, the level of the first output signal or the second output signal may be adjusted by adjusting the amplification factor of amplifier 55 or amplifier 56. For example, by setting the amplification factor of amplifier 55 to 2x, the level of the first output signal can be made equivalent to that when a single photoelectric conversion unit having a light-receiving area equal to the total light-receiving area of ​​photoelectric conversion unit 3 and photoelectric conversion unit 4 is used.

[0078] It should be noted that the example shown in FIG. 3 and the examples shown in FIGS. 4(A) to 4(D) can be combined. For example, the image sensor 1 can switch between a first mode (see FIG. 4(D)) in which the exposure time of the photoelectric conversion unit 3 and the exposure time of the photoelectric conversion unit 4 are the same, and a second mode in which the exposure time of the photoelectric conversion unit 3 and the exposure time of the photoelectric conversion unit 4 are different. In the first mode, the vertical scanning circuit 51 sets, for example, the waveform of the control signal TX1 for the transfer unit 21 to be the same as the waveform of the control signal TX2 for the transfer unit 41. In the second mode, the vertical scanning circuit 51 sets, for example, the waveform of the control signal TX1 for the transfer unit 21 to be different from the waveform of the control signal TX2 for the transfer unit 41. The second mode may include one of the examples shown in FIG. 3 and the examples shown in FIGS. 4(A) to 4(C), or may include a combination of two or more of them.

[0079] In the above example, the length of the exposure period for the photoelectric conversion unit 3 is set to be equal to or longer than the length of the exposure period for the photoelectric conversion unit 4, but it may be set to be shorter than the length of the exposure period for the photoelectric conversion unit 4. Of the exposure time for the photoelectric conversion unit 3 and the exposure time for the photoelectric conversion unit 4, one exposure time may be set to be 1 / 10 or more and 1 time or less than 1 / 10 of the other exposure time.

[0080] Next, another example of a plurality of photoelectric conversion units arranged in one pixel will be described below. Fig. 5 is a diagram showing another example of a plurality of photoelectric conversion units.

[0081] In FIG. 5(A), the photoelectric conversion unit 3 is arranged asymmetrically with the photoelectric conversion unit 4 when viewed from the normal direction of the pixel region 1a. The light receiving area of ​​the photoelectric conversion unit 3 is set larger than the light receiving area of ​​the photoelectric conversion unit 4. If the light receiving areas of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are different, the dynamic range can be further widened. In this way, the photoelectric conversion unit 3 may be different from the photoelectric conversion unit 4 in at least one of the shape and dimensions.

[0082] In FIG. 5(B), a photoelectric conversion unit 3, a photoelectric conversion unit 4, and a photoelectric conversion unit 60 are provided within one pixel. The light receiving area of ​​the photoelectric conversion unit 3 is set to be larger than the light receiving area of ​​the photoelectric conversion unit 4. The light receiving area of ​​the photoelectric conversion unit 60 is set to be approximately the same as the light receiving area of ​​the photoelectric conversion unit 4. The photoelectric conversion unit 60 is provided symmetrically to the photoelectric conversion unit 4 with respect to the center line of the pixel P. As such, the number of photoelectric conversion units provided within one pixel may be three, or may be four or more. Furthermore, at least one of the shape and dimensions of any one of the multiple photoelectric conversion units may be the same as or different from the other photoelectric conversion units.

[0083] When such an image sensor 1 is used, for example, it is possible to detect information relating to focus at the image sensor 1 by using the second output signal based on the charge accumulated in the photoelectric conversion unit 4 and the third output signal based on the charge accumulated in the photoelectric conversion unit 60. Furthermore, it is also possible to widen the dynamic range by using at least two of the first output signal based on the charge accumulated in the photoelectric conversion unit 3, the second output signal, and the third output signal.

[0084] [Second embodiment] Next, a second embodiment will be described. In this embodiment, the same components as those in the above-described embodiment will be denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0085] FIG. 6 is a diagram showing the circuit configuration of the image sensor 1 according to this embodiment. FIG. 6 shows an equivalent circuit diagram for two pixels. The circuit configuration of pixel P1 is the same as that of the embodiment described above (see FIG. 2). Pixel P2 is a pixel disposed adjacent to pixel P1. For example, pixel P is the red pixel R in FIG. 1(B), and pixel P2 is the green pixel Gb.

[0086] Pixel P2 includes a photoelectric conversion unit 61 and a photoelectric conversion unit 62. The photoelectric conversion unit 61 and the photoelectric conversion unit 62 may include, for example, a photodiode, but may also include, for example, a phototransistor as long as they convert light into an electric charge. The imaging element 1 includes a third circuit system that reads out an output signal based on the electric charge accumulated in the photoelectric conversion unit 61 of pixel P2, and a fourth circuit system that reads out an output signal based on the electric charge accumulated in the photoelectric conversion unit 62 of pixel P2.

[0087] First, the third circuit system will be described. The third circuit system includes a transfer unit 63, a transfer control line 64, a charge-voltage conversion unit 23, a reset unit 24, a reset control line 25, a selection unit 26, a selection control line 27, an amplifier unit 28, a power supply line 29, and a vertical signal line 30. In other words, the third circuit system is shared with the first circuit system except for the transfer unit 63 and the transfer control line 64. The transfer unit 63 includes a switching element such as an n-type MOS transistor.

[0088] The photoelectric conversion unit 61 has an anode grounded and a cathode connected to the source of the transfer unit 63. The transfer unit 63 is used to transfer charges from the photoelectric conversion unit 61. The drain of the transfer unit 63 is electrically connected to the node 31. The gate of the transfer unit 63 is connected to a transfer control line 64. The transfer control line 64 can transmit a control signal TX3 to the gate of the transfer unit 63. The charge-voltage conversion unit 23 converts the charges from the transfer unit 21 into a voltage, and also converts the charges from the transfer unit 63 into a voltage.

[0089] Next, the fourth circuit system will be described. The fourth circuit system includes a transfer unit 65, a transfer control line 66, a charge-voltage conversion unit 43, a reset unit 44, a reset control line 45, a selection unit 46, a selection control line 27, an amplifier unit 47, a power supply line 29, and a vertical signal line 30. In other words, the fourth circuit system is shared with the second circuit system except for the transfer unit 65 and the transfer control line 66. The transfer unit 65 includes a switching element such as an n-type MOS transistor.

[0090] The photoelectric conversion unit 62 has an anode grounded and a cathode connected to the source of the transfer unit 65. The transfer unit 65 is used to transfer charges from the photoelectric conversion unit 62. The drain of the transfer unit 65 is electrically connected to the node 50. The gate of the transfer unit 65 is connected to a transfer control line 66. The transfer control line 66 can transmit a control signal TX4 to the gate of the transfer unit 65. The charge-voltage conversion unit 43 converts the charges from the transfer unit 41 into a voltage, and also converts the charges from the transfer unit 65 into a voltage.

[0091] In this embodiment, the transfer control lines 64 and 66 are provided in the element layer 11 shown in FIG. 1C. That is, the transfer control lines 64 and 66 are arranged on the side opposite to the light incident side with respect to the photoelectric conversion units 61 and 62. Note that at least a portion of the transfer control lines 64 and 66 may be arranged on the same side as the light incident side with respect to the photoelectric conversion units 61 and 62. At least a portion of these wirings may be provided in the light receiving layer 12 shown in FIG. 1C, or may be provided between the light receiving layer 12 and the lens layer 14.

[0092] Next, the timing for reading out charges from the photoelectric conversion unit will be described. Fig. 7 is a timing chart showing an example of the operation of the image sensor 1.

[0093] The image sensor 1 performs dummy readout in the same manner as in the first embodiment after startup and before capturing the first image. The image sensor 1 also reads out a signal from pixel P during the first frame period in the same manner as in the first embodiment. The image sensor 1 reads out a signal from pixel P2 during the second frame period following the first frame period.

[0094] The image sensor 1 starts imaging processing for the second frame at time t10. The vertical scanning circuit 51 maintains the control signal SEL for the selection unit 26 at H level. The vertical scanning circuit 51 also performs a reset operation on the photoelectric conversion unit 61 during the period from time t30 to time t31. During this period, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to H level and then to L level. Next, the vertical scanning circuit 51 sets the control signal TX3 for the transfer unit 21 to H level and then to L level. After the control signal TX3 is set to L level at time t31, charge corresponding to the amount of exposure is accumulated in the photoelectric conversion unit 61.

[0095] The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 61 during the period from time t32 to time t33. The vertical scanning circuit 51 sets the control signal RST1 to an H level at time t32 and then to an L level. The vertical scanning circuit 51 also sets the control signal TX3 to an H level at time t33, which is later than time t32, and then to an L level at time t34. Exposure of the photoelectric conversion unit 3 during the second frame period ends at time t33. The photoelectric conversion unit 61 outputs the charges accumulated during the exposure period from time t31 to time t33 to the charge-voltage conversion unit 23 via the transfer unit 63. The amplification unit 28 also outputs a signal corresponding to the voltage converted by the charge-voltage conversion unit 23 to the vertical signal line 30.

[0096] In this embodiment, the image sensor 1 can expose the photoelectric conversion unit 62 during a period that overlaps with at least a portion of the exposure period for the photoelectric conversion unit 61. The vertical scanning circuit 51 performs a reset operation on the photoelectric conversion unit 62 during the period from time t35 to time t36, which is after time t31 during the second frame period. During this period, the vertical scanning circuit 51 sets a control signal RST2 for the reset control line 45 to an H level and then to an L level. Next, the vertical scanning circuit 51 sets a control signal TX4 for the transfer unit 65 to an H level and then to an L level. After the control signal TX4 is set to an L level at time t36, electric charges corresponding to the amount of exposure are accumulated in the photoelectric conversion unit 62.

[0097] The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 62 when the exposure period of the photoelectric conversion unit 62 ends. Here, the vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 62 in parallel with the reading out of charges from the photoelectric conversion unit 62. The vertical scanning circuit 51 sets the control signal RST2 to a high level at time t32 and then to a low level. The vertical scanning circuit 51 also sets the control signal TX4 to a high level at time t33 and to a low level at time t34. The photoelectric conversion unit 62 outputs the charges accumulated during the exposure period from time t36 to time t33 to the charge-voltage conversion unit 43 via the transfer unit 65. The amplifier 47 also outputs a signal corresponding to the voltage converted by the charge-voltage conversion unit 43 to the vertical signal line 48. In this way, the second frame period ends at time t34 and the next frame period begins. In the frame period following the second frame period, exposure to the pixel P1 and reading of a signal from the pixel P1 are performed in the same manner as in the first frame period.

[0098] In the above-described embodiment, the image sensor 1 is capable of detecting light in three wavelength bands, but it may also be capable of detecting light in only one wavelength band (monochromatic light), or light in only two wavelength bands, or light in four or more wavelength bands. The pixel arrangement does not have to be a Bayer arrangement and can be selected arbitrarily. The image sensor 1 may also be sensitive to the invisible light region, and may detect, for example, at least one of infrared light and ultraviolet light.

[0099] 2 and 6 may be omitted. If the selection unit 26 and the selection control line 727 are not provided, the timing of reading out the charge from the pixel P can be controlled by the timing of turning on the transfer unit 21, the transfer unit 41, etc.

[0100] [Third embodiment] Next, an imaging device using the imaging element 1 described in the above embodiment will be described. FIG. 8 is a block diagram showing the configuration of an imaging device 70 according to this embodiment. The imaging device 70 shown in FIG. 8 is at least a part of an electronic device such as a digital camera, a smartphone, a mobile phone, a personal computer, or a measuring device. The imaging device 70 includes a lens unit 71, a lens driving unit 72, the imaging element 1, a control unit 73, a work memory 74, a display unit 75, an operation unit 76, and a recording unit 77.

[0101] The lens unit 71 includes, for example, an imaging optical system including multiple lenses and a lens barrel. The imaging optical system of the lens unit 71 collects light from a subject and guides it to the image sensor 1. The imaging optical system of the lens unit 71 forms an image of the subject on the light receiving layer 12 of the image sensor 1 (see FIG. 1(C)). The lens unit 71 may be integrated with a body that houses at least the control unit 73 of the image sensor 70, or may be an interchangeable lens that is detachable from the body. The lens unit 71 may include a focus lens or a zoom lens. The lens driving unit 72 drives the lens of the lens unit 71 to adjust the focus of the lens unit 71.

[0102] The control unit 73 controls each unit of the imaging device 70. The control unit 73 includes an imaging control unit 80, an image processing unit 81, and a focus detection unit 82.

[0103] The imaging control unit 80 sends a command to the imaging element 1, causing the imaging element 1 to perform imaging processing. For example, the imaging control unit 80 controls the timing to start and end exposure of each photoelectric conversion unit by sending a command to the vertical scanning circuit 51 of the imaging element 1 shown in FIG. 2, etc. The imaging control unit 80 also sends a command to the vertical scanning circuit 51, controlling the timing to read out a signal from each pixel to the horizontal scanning circuit 52. The imaging control unit 80 also sends a command to the imaging element 1, causing it to output (transmit) the imaging result to the control unit 73. For example, the imaging control unit 80 sends a command to the horizontal scanning circuit 52, causing the horizontal scanning circuit 52 to output the imaging result to the control unit 73.

[0104] The image processing unit 81 acquires the imaging results from the imaging element 1 and performs various image processing. The image processing unit 81 uses the work memory 74 as a workspace to perform image processing on RAW data consisting of pixel signals for each pixel, and generates image data. The work memory 74 temporarily stores image data and the like when the image processing unit 81 performs image processing.

[0105] In this embodiment, the image processing unit 81 performs HDR processing to widen the dynamic range using a first output signal based on the charges accumulated in the photoelectric conversion unit 3 and a second output signal based on the charges accumulated in the photoelectric conversion unit 4. In this case, the imaging control unit 80 sets the period (exposure period) during which charges are accumulated in the photoelectric conversion unit 3 and the period (exposure period) during which charges are accumulated in the photoelectric conversion unit 4 to different lengths within one frame period.

[0106] For example, as shown in FIG. 3, the imaging control unit 80 sets the exposure period for the photoelectric conversion unit 3 to be longer than the exposure period for the photoelectric conversion unit 4. Then, the image processing unit 81 generates an image of a relatively dark part of the imaging target using a first output signal with a long exposure period (high exposure amount). Also, the image processing unit 81 generates an image of a relatively bright part of the imaging target using a second output signal with a short exposure period (low exposure amount). The image processing unit 81 generates an image in which the dark part is brightened while suppressing saturation in the bright part by combining the image of the dark part and the image of the bright part.

[0107] The imaging device 70 can switch between a mode in which image data is generated by performing HDR processing and a mode in which image data is generated without performing HDR processing. When HDR processing is not performed, the imaging control unit 80 sets the period in which charge is accumulated in the photoelectric conversion unit 3 and the period in which charge is accumulated in the photoelectric conversion unit 4 to different lengths. The image processing unit 81 generates image data by, for example, adding together a first output signal based on the charge accumulated in the photoelectric conversion unit 3 and a second output signal based on the charge accumulated in the photoelectric conversion unit 4.

[0108] The image processing unit 81 can perform various image processing in addition to the above-mentioned HDR processing. For example, the image processing unit 81 generates an RGB image signal by performing color signal processing (color correction) on a signal obtained from a Bayer array. The image processing unit 81 also performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. The image processing unit 81 also performs compression processing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The control unit 73 can output the image data generated by the image processing unit 81 to the recording unit 77. The control unit 73 can also output the image data generated by the image processing unit 81 to the display unit 75.

[0109] The image processing unit 81 performs processing to detect a main subject from the image data, as described in, for example, Japanese Patent Application Laid-Open Publication No. 2010-16621 (US2010 / 0002940). Here, the term "main subject" refers to a subject that is the object of imaging and that is focused on by the user (photographer) or is presumed to be focused on by the user. The image data may contain not only one main subject, but also multiple main subjects. Furthermore, the image processing unit 81 detects a human body included in the image data as the main subject, as described in, for example, Japanese Patent Application Laid-Open Publication No. 2010-16621 (US2010 / 0002940).

[0110] The parameters referenced when the image processing unit 81 performs image processing are included in the control parameters (imaging conditions). For example, the control parameters include parameters for color signal processing (color correction), white balance adjustment, gradation adjustment, compression rate, etc. The signal read from the imaging element 1 changes depending on the charge accumulation time, and the parameters referenced when performing image processing also change depending on the change in the signal.

[0111] The image processing unit 81 extracts frames at predetermined intervals from among the multiple frames obtained in time series from the image sensor 1. Alternatively, the image processing unit 81 discards frames at predetermined intervals from among the multiple frames obtained in time series from the image sensor 1. This reduces the amount of data, thereby reducing the load on subsequent processing. Furthermore, the image processing unit 81 calculates one or more frames to be interpolated between each frame based on the multiple frames obtained in time series from the image sensor 1. The image processing unit 81 then adds the calculated one or more frames between each frame. This allows for smoother motion when playing back a video.

[0112] The focus detection unit 82 detects focus information (information related to focusing) of the imaging optical system of the lens unit 71 using a signal output from the image sensor 1. The focus detection unit 82 performs focus detection using a phase difference method, for example. When the focus detection unit 82 detects focus information, the imaging control unit 80 sets the period in which charge is accumulated in the photoelectric conversion unit 3 and the period in which charge is accumulated in the photoelectric conversion unit 4 to be the same length, as shown in FIG. 4(D), for example. The focus detection unit 82 compares a first output signal based on the charge accumulated in the photoelectric conversion unit 3 with a second output signal based on the charge accumulated in the photoelectric conversion unit 4 to detect focus information.

[0113] For example, when the difference between the level of the first output signal and the level of the second output signal is equal to or smaller than a threshold, the focus detection unit 82 determines that the lens unit 71 is focused on the light receiving layer 12 of the image sensor 1. When the difference between the level of the first output signal and the level of the second output signal is greater than the threshold and one of the first output signal and the second output signal is higher than the other, the focus detection unit 82 determines that the focus is on the object plane side of the light receiving layer 12 (front focus). When the difference between the level of the first output signal and the level of the second output signal is greater than the threshold and the other of the first output signal and the second output signal is higher than the other, the focus detection unit 82 determines that the focus is on the object plane side of the light receiving layer 12 (front focus). Note that the focus detection unit 82 may detect the amount of deviation between the focus of the lens unit 71 and the photoelectric conversion unit (light receiving layer 12) as focus information based on the difference between the level of the first output signal and the level of the second output signal.

[0114] The focus detection unit 82 supplies the detected focus information to the imaging control unit 80. The imaging control unit 80, for example, calculates the drive amount by the lens drive unit 72 required to focus on the photoelectric conversion unit using the focus information detected by the focus detection unit 82. The imaging control unit 80 controls the focus of the lens unit 71 by sending a command to the lens drive unit 72 based on the calculated drive amount.

[0115] When the focus detection unit 82 detects focus information, the imaging control unit 80 may set the period during which charge is accumulated in the photoelectric conversion unit 3 to be different from the period during which charge is accumulated in the photoelectric conversion unit 4. In this case, the focus detection unit 82 may detect focus information based on an estimated value of the level of the output signal when the lengths of the exposure periods of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are made the same.

[0116] For example, the imaging control unit 80 may control at least one of the amplification factors of the amplifiers 55 and 56 by sending a command to the horizontal scanning circuit 52 of the image sensor 1, causing it to output a signal corresponding to a state in which the lengths of the exposure periods for the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are the same. For example, if the length of the exposure period for the photoelectric conversion unit 3 is twice the length of the exposure period for the photoelectric conversion unit 4, the imaging control unit 80 may adjust the amplification factor of the amplifier 55 to half the amplification factor of the amplifier 56. Furthermore, the control unit 73 (e.g., the image processing unit 81, the focus detection unit 82) may calculate the level of a signal corresponding to a state in which the lengths of the exposure periods for the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are the same.

[0117] The display unit 75 includes, for example, a liquid crystal display panel, and displays images according to image data supplied from the control unit 73. The image data supplied from the control unit 73 includes image data based on images (e.g., still images, videos, live view images) captured by the image sensor 1, image data of images indicating settings such as shooting conditions of the image capture device 70, and the like. A live view image is an image displayed on the display unit 75 by sequentially outputting image data generated by the image processing unit 81 to the display unit 75. The live view image is used by the user to check the image of the subject being captured by the image sensor 1. The live view image is also called a through image or a preview image. The display unit 75 may include an input unit such as a touch panel.

[0118] For example, if the imaging device 70 is a camera, the operation unit 76 is a release switch, a video switch, various operation switches, etc. that are operated by the user. The operation unit 76 outputs a signal according to the operation by the user to the control unit 73. For example, the control unit 73 causes the imaging element 1 to perform imaging processing according to the operation of the operation unit 76. In addition, the control unit 73 causes the display unit 75 to display an image indicating the shooting conditions, etc., and the operation unit 76 can receive input from the user.

[0119] The recording unit 77 has a card slot into which a storage medium 83 such as a memory card can be inserted. The recording unit 77 stores image data and various other data generated by the image processing unit 81 in the storage medium 83 inserted in the card slot. For example, the control unit 73 stores data based on an image captured by the image sensor 1 in response to an operation of the operation unit 76 in the storage medium 83. The control unit 73 may store, for example, image data generated by HDR processing in the storage medium 83, or image data generated without HDR processing in the storage medium 83. The control unit 73 may also store, for example, first image data generated based on a first output signal based on charges accumulated in the photoelectric conversion unit 3 and second image data generated based on a second output signal based on charges accumulated in the photoelectric conversion unit 4 in the storage medium 83 separately. The first image data may be image data generated without using the second output signal, and the second image data may be image data generated without using the first output signal.

[0120] Next, an operation when the imaging device 70 is a camera will be described. Fig. 9 is a flowchart showing an example of the operation of the imaging device 70. In step S1, the imaging control unit 80 determines whether the still image mode is set.

[0121] First, a case where the still image mode is set will be described. If the imaging control unit 80 determines in step S1 that the still image mode is set (step S1; Yes), it determines in step S2 whether the HDR mode is set. If the imaging control unit 80 determines in step S2 that the HDR mode is set (step S2; Yes), it sets an exposure period in step S3. Here, it is assumed that the exposure periods for the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are set to be the same in the initial state. In step S3, the imaging control unit 80 sets the length of the exposure period for the photoelectric conversion unit 3 to be different from the exposure period for the photoelectric conversion unit 4.

[0122] If the imaging control unit 80 determines in step S2 that the HDR mode is not set (step S2; No) or if the processing of step S3 is completed, the imaging control unit 80 determines in step S4 whether or not a release switch (hereinafter abbreviated as release SW) has been operated. If the imaging control unit 80 determines in step S4 that the release SW has been half-pressed (step S4; Yes), the imaging control unit 80 performs auto exposure processing (AE processing) in step S5. If the imaging control unit 80 determines in step S4 that the release SW has not been operated (step S4; No), the imaging control unit 80 continues to monitor the operation of the release SW. If the imaging control unit 80 determines in step S4 that the release SW has been fully pressed, the imaging control unit 80 causes the image sensor 1 to perform imaging processing in step S8, which will be described later.

[0123] After completing the process of step S5, the imaging control unit 80 performs autofocus processing (AF processing) in step S6. In step S5, the imaging control unit 80 causes the image sensor 1 to perform imaging processing. Furthermore, the focus detection unit 82 detects focus information using a first output signal based on the charge accumulated in the photoelectric conversion unit 3 and a second output signal based on the charge accumulated in the photoelectric conversion unit 4. Furthermore, the imaging control unit 80 controls the lens driving unit 72 to adjust the focus of the lens unit 71 based on the focus information detected by the focus detection unit 82.

[0124] When the HDR mode is selected, the imaging control unit 80 may display a live-view image on the display unit 75 after the process of step S5 or step S6 is completed. The imaging control unit 80 may also receive an input requesting a change in at least one of the length of the exposure period for the photoelectric conversion unit 3 and the length of the exposure period for the photoelectric conversion unit 4. For example, the user can change the brightness setting for HDR processing while viewing the live-view image. The imaging control unit 80 may also automatically change at least one of the length of the exposure period for the photoelectric conversion unit 3 and the length of the exposure period for the photoelectric conversion unit 4, for example, based on the distribution of pixel values ​​in the live-view image. The imaging control unit 80 may also perform this exposure period change process while monitoring the operation of the release switch in step S4.

[0125] Furthermore, the imaging control unit 80 may use the results of the AE processing to adjust at least one of the amplification factors of the amplifiers 55 and 56. For example, the imaging control unit 80 may set the aperture so that dark areas of the object to be imaged can be detected according to which of the photoelectric conversion units 3 and 4 has the longer exposure time, and adjust the amplification factor so that the output does not become saturated with the photoelectric conversion unit 3 or 4 with the shorter exposure time under these conditions.

[0126] After completing the process of step S6, the imaging control unit 80 determines in step S7 whether the release SW has been operated. If the imaging control unit 80 determines in step S7 that the release SW has been fully pressed (step S7; Yes), the imaging control unit 80 causes the image sensor 1 to execute imaging processing in step S8. If the imaging control unit 80 determines in step S7 that the release SW has not been operated (step S7; No), the imaging control unit 80 continues to monitor the operation of the release SW. If the imaging control unit 80 determines in step S7 that the release SW has been half-pressed, the imaging control unit 80 again executes the AE process of step S5 and the AF process of step S6, and then continues to monitor the operation of the release SW.

[0127] If the HDR mode is set, the imaging control unit 80 sets the various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in Fig. 3 in step S8. If the HDR mode is not set, the imaging control unit 80 sets the various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in Fig. 4(D) in step S8. After the period of one frame has ended, the imaging control unit 80 causes the horizontal scanning circuit 52 to output the imaging result.

[0128] After the process of step S8 is completed, the control unit 73 causes the image processing unit 81 to generate image data according to the imaging result in step S9. If the HDR mode is set, the image processing unit 81 performs HDR processing to generate image data in step S9. On the other hand, if the HDR mode is not set, the image processing unit 81 generates image data in step S9 without performing HDR processing. After the process of step S9 is completed, the control unit 73 outputs the image data generated in step S8 to at least one of the storage unit 7 and the display unit 75 in step S10. In this way, the series of processes is completed.

[0129] Next, a case where the video mode is set (when the still image mode is not set) will be described. When the video mode is set, the imaging control unit 80 focuses, for example, on the center of the angle of view or a position specified by the user. When the imaging control unit 80 determines in step S1 that the still image mode is not set (step S1; No), it determines in step S11 whether the HDR mode is set. When the imaging control unit 80 determines in step S11 that the HDR mode is set (step S11; Yes), it sets an exposure period in step S12. Here, it is assumed that the exposure periods for the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are set to the same in the initial state. In step S12, the imaging control unit 80 sets the length of the exposure period for the photoelectric conversion unit 3 to be different from the exposure period for the photoelectric conversion unit 4.

[0130] If the imaging control unit 80 determines in step S11 that the HDR mode is not set (step S11; No) or if the processing of step S12 has ended, the imaging control unit 80 determines in step S13 whether or not a video switch (hereinafter abbreviated as video SW) has been operated to request the start of recording. If the imaging control unit 80 determines in step S13 that the video SW has been operated (step S13; Yes), the imaging control unit 80 causes the imaging element 1 to execute imaging processing in step S14. If the imaging control unit 80 determines in step S13 that the video SW has not been operated (step S13; No), the imaging control unit 80 continues to monitor the operation of the video SW.

[0131] If the HDR mode is set, the imaging control unit 80 sets the various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in Fig. 3 in step S14. If the HDR mode is not set, the imaging control unit 80 sets the various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in Fig. 4(D) in step S14. After the period of one frame has ended, the imaging control unit 80 causes the horizontal scanning circuit 52 to output the imaging result.

[0132] After the processing of step S14 is completed, the control unit 73 causes the image processing unit 81 to generate image data corresponding to the imaging result. If the HDR mode is set, the image processing unit 81 performs HDR processing to generate image data. If the HDR mode is not set, the image processing unit 81 generates image data without performing HDR processing. The control unit 73 outputs the image data generated by the image processing unit 81 to at least one of the storage unit 7 and the display unit 75.

[0133] When the processing of step S14 is completed, the imaging control unit 80 determines in step S15 whether or not an operation of the video switch has been performed to request the end of recording. When the imaging control unit 80 determines in step S15 that the video switch has been operated (step S15; Yes), it ends the series of processes. When the imaging control unit 80 determines in step S15 that the video switch has not been operated (step S15; No), it determines in step S16 whether or not imaging of a predetermined number of frames has been completed. When the imaging control unit 80 determines in step S16 that imaging of the predetermined number of frames has not been completed (step S16; No), it returns to step S14 and causes the imaging element 1 to perform imaging processing for the next frame.

[0134] If the imaging control unit 80 determines in step S16 that imaging of a predetermined number of frames has been completed (step S15; Yes), it executes AF processing in step S17. In step S16, the imaging control unit 80 causes the image sensor 1 to execute imaging processing. Furthermore, the focus detection unit 82 detects focus information using a first output signal based on the charge accumulated in the photoelectric conversion unit 3 and a second output signal based on the charge accumulated in the photoelectric conversion unit 4. Furthermore, the imaging control unit 80 controls the lens driving unit 72 to adjust the focus of the lens unit 71 based on the focus information detected by the focus detection unit 82.

[0135] In step S17, the imaging control unit 80 may execute imaging processing during the period between frames of the output video, and may execute AF processing using the results. Also, in step S17, the imaging control unit 80 may execute AF processing using image data of frames of the output video. Also, the imaging control unit 80 may execute AF processing using the outputs of each of the multiple pixels P, or may execute AF processing using signals output from some of the multiple pixels P. When executing AF processing using signals output from some of the pixels P, the some of the pixels P may be fixed among the multiple pixels P, or may be changed to different pixels for each AF processing. Also, when HDR mode is set, the length of the exposure period for some of the pixels P used in AF processing may be set to be the same for the photoelectric conversion unit 3 and the photoelectric conversion unit 4.

[0136] The predetermined number of frames is set depending on the response speed of the lens driver 72. For example, if the time required for the lens driver 72 to achieve the drive amount specified by the imaging control unit 80 is 1 second and the frame rate is 30 FPS, the predetermined number of frames is set to any value with 30 frames as the lower limit. The predetermined number of frames is set to, for example, a default value, and can be changed by the user to a value equal to or greater than the lower limit.

[0137] After completing the process of step S17, the imaging control unit 80 returns to step S13 and causes the imaging element 1 to perform imaging processing for the next frame. In this way, the imaging control unit 80 repeatedly performs the imaging processing of step S13 until it is determined in step S14 that an operation of the video switch requesting the suspension of recording has been performed. Furthermore, the imaging control unit 80 performs the AF processing of step S17 every time imaging processing for a predetermined number of frames has been performed.

[0138] The control unit 73 as described above may include, for example, a CPU (Central Processing Unit), and the CPU may execute at least one process according to an imaging control program. The imaging control program may cause the control unit 73 to transfer charges from the photoelectric conversion unit 3, of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 arranged in the first pixel and converting light into electric charges, during a first period of one frame period, and to transfer charges from the photoelectric conversion unit 4 during a second period of one frame period that is different from the first period. The control unit 73 may also include an arithmetic circuit such as an ASIC. For example, the image processing unit 81 may include an arithmetic circuit, and the arithmetic circuit may execute at least one process.

[0139] In the imaging device 70 according to this embodiment, multiple photoelectric conversion units are arranged per pixel of the image sensor 1, thereby widening the dynamic range without reducing the resolution of the captured image. Furthermore, since the transfer unit 21 for transferring charges from the photoelectric conversion unit 3 and the transfer unit 41 for transferring charges from the photoelectric conversion unit 4 can be controlled independently, a decrease in the frame rate can be suppressed. Furthermore, the imaging device 70 can also perform AF processing using a first output signal based on the charges accumulated in the photoelectric conversion unit 3 and a second output signal based on the charges accumulated in the photoelectric conversion unit 4.

[0140] The technical scope of the present invention is not limited to the above-described embodiments or modifications. For example, one or more of the features described in the above-described embodiments or modifications may be omitted. Furthermore, the features described in the above-described embodiments or modifications may be combined as appropriate. [Explanation of symbols]

[0141] 1 imaging element, 3 photoelectric conversion unit, 4 photoelectric conversion unit, P pixel, 21 transfer unit, 22 transfer control line, 23 charge-voltage conversion unit, 24 reset unit, 25 reset control line, 41 transfer unit, 42 transfer control line, 43 charge-voltage conversion unit, 44 reset unit, 45 reset control line, 55 amplifier, 56 amplifier, 60 photoelectric conversion unit, 61 photoelectric conversion unit, 62 photoelectric conversion unit, 63 transfer unit, 64 transfer control line, 65 transfer unit, 66 transfer control line, 70 imaging device, 80 imaging control unit, 81 image processing unit, 82 focus detection unit, P pixel, P1 pixel, P2 pixel, TX1 control signal, TX2 control signal, TX3 control signal, TX4 control signal, RST1 control signal, RST2 control signal

Claims

1. a first microlens onto which light is incident; a second microlens arranged next to the first microlens in the column direction and onto which light is incident; a first filter having a first spectral characteristic onto which light from the first microlens is incident; a second filter having a second spectral characteristic different from the first spectral characteristic onto which light from the second microlens is incident; a first photoelectric conversion unit that converts light transmitted through the first filter into an electric charge; a second photoelectric conversion unit that converts light transmitted through the first filter into an electric charge; a third photoelectric conversion unit that converts light transmitted through the second filter into an electric charge; a fourth photoelectric conversion unit that converts light transmitted through the second filter into an electric charge; a first transfer unit that transfers the charges converted by the first photoelectric conversion unit; a second transfer unit that transfers the charges converted by the second photoelectric conversion unit; a third transfer unit that transfers the charges converted by the third photoelectric conversion unit; a fourth transfer unit that transfers the charges converted by the fourth photoelectric conversion unit; a first accumulation unit to which the charges from the first photoelectric conversion unit are transferred by the first transfer unit; a second accumulation unit to which the charges from the second photoelectric conversion unit are transferred by the second transfer unit; a third accumulation unit to which the charges from the third photoelectric conversion unit are transferred by the third transfer unit; a fourth accumulation unit to which the charges from the fourth photoelectric conversion unit are transferred by the fourth transfer unit; a first transistor including a gate portion electrically connected to the first storage portion and outputting a first signal; a second transistor including a gate portion electrically connected to the second storage portion and outputting a second signal; a third transistor including a gate portion electrically connected to the third storage portion and outputting a third signal; a fourth transistor including a gate portion electrically connected to the fourth storage portion and outputting a fourth signal; a fifth transistor for outputting the first signal from the first transistor to a first signal line; a sixth transistor for outputting the second signal from the second transistor to a second signal line; a seventh transistor for outputting the third signal from the third transistor to the first signal line; an eighth transistor for outputting the fourth signal from the fourth transistor to the second signal line; a first control line that outputs a control signal to the fifth transistor and the sixth transistor; a second control line that outputs a control signal to the seventh transistor and the eighth transistor; a first current source that supplies a current to the first transistor and the third transistor via the first signal line; a second current source that supplies a current to the second transistor and the fourth transistor via the second signal line; An imaging element comprising:

2. 2. The imaging device according to claim 1, an imaging element including a scanning circuit that outputs control signals to the first transfer unit, the second transfer unit, the third transfer unit, and the fourth transfer unit;

3. 3. The imaging device according to claim 2, a first reset unit that resets the potential of the first storage unit; a second reset unit that resets the potential of the second storage unit; a third reset unit that resets the potential of the third storage unit; a fourth reset unit that resets the potential of the fourth storage unit; An imaging element comprising:

4. 4. The imaging device according to claim 3, the scanning circuit outputs control signals to the first reset unit, the second reset unit, the third reset unit, and the fourth reset unit. Image sensor.

5. 5. The imaging device according to claim 4, the fifth transistor is electrically connected to a supply unit to which a predetermined voltage is supplied and the first transistor; the sixth transistor is electrically connected to the supply unit and the second transistor; the seventh transistor is electrically connected to the supply unit and the third transistor; the eighth transistor is electrically connected to the supply unit and the fourth transistor; Image sensor.

6. 6. The imaging device according to claim 5, An imaging element including a light-shielding portion disposed between the first filter and the second filter.

7. 7. The imaging device according to claim 6, a light receiving area of ​​the second photoelectric conversion unit is different from a light receiving area of ​​the first photoelectric conversion unit; Image sensor.

8. 8. The imaging device according to claim 7, a light receiving area of ​​the fourth photoelectric conversion unit is different from a light receiving area of ​​the third photoelectric conversion unit; Image sensor.

9. An imaging device comprising the imaging element according to claim 1 .

10. 10. The imaging device according to claim 9, an imaging device comprising an image processing unit electrically connected to the imaging element and generating image data;

11. The imaging device according to claim 10, An imaging device including a storage unit that stores the image data in a storage medium.

12. 12. The imaging device according to claim 10, An imaging device comprising a display unit that displays an image of the image data.

Images