Imaging device and imaging apparatus

Abstract

To shorten time until the start of exposure for generating a static image after reception of an imaging instruction.SOLUTION: An imaging device comprises: a first photoelectric conversion unit and a second photoelectric conversion unit provided in a column direction, which perform photoelectric conversion of light having passed through an optical system to generate electric charges; a first signal line which outputs a first signal to use to generate a first image based on the electric charges generated by the first photoelectric conversion unit; a second signal line which outputs a second signal to use to detect a photographic state based on the electric charges generated by the second photoelectric conversion unit; a first output unit which outputs first data based on the first signal; and a second output unit which outputs second data based on the second signal.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an imaging element and an imaging device. [Background technology]

[0002] BACKGROUND ART In an imaging device, it is desirable to reduce the time from when an instruction to capture a still image is received until when exposure for generating the still image is started (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-108176 Summary of the Invention

[0004] According to the first aspect of the disclosure, the imaging element generates electric charges by photoelectrically converting light that has passed through an optical system, and includes a first photoelectric conversion unit and a second photoelectric conversion unit arranged in a column direction, a first signal line that outputs a first signal used to generate a first image based on the electric charges generated by the first photoelectric conversion unit, a second signal line that outputs a second signal used to detect a shooting state based on the electric charges generated by the second photoelectric conversion unit, a first output unit that outputs first data based on the first signal, and a second output unit that outputs second data based on the second signal.

[0005] According to a second aspect of the disclosure, the imaging device includes the imaging element, a display unit, a first processing unit that generates the first image to be displayed on the display unit based on the first data output from the first output unit, and a second processing unit that detects the shooting state based on the second data output from the second output unit.

[0006] The configurations of the embodiments described below may be modified as appropriate, and at least a portion of the configuration may be replaced with other components. Furthermore, components that are not particularly limited in terms of their placement may be placed in any position that allows them to achieve their function, not limited to the placement disclosed in the embodiments. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram schematically illustrating an example of the configuration of a digital camera equipped with an image sensor according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a schematic configuration of an imaging element according to the embodiment. [Figure 3] FIG. 3 is a circuit diagram showing the configuration of a pixel included in the imaging element according to the embodiment. [Figure 4] FIG. 4(A) is a sequence diagram showing an example of the operation of the camera when a live view image is displayed on the display unit, and FIG. 4(B) is a sequence diagram showing an example of the operation of the camera when flicker is detected. [Figure 5] FIG. 5 is a timing chart for explaining the operation of the image sensor when generating live view image data. [Figure 6] FIG. 6 is a timing chart for explaining the operation of the imaging element when detecting flicker. [Figure 7] FIG. 10 is a sequence diagram showing an example of the operation of the camera when an instruction to capture a still image is given. [Figure 8] FIG. 8 is a diagram illustrating a schematic configuration of an imaging element according to a comparative example. [Figure 9] FIG. 9 is a circuit diagram showing the configuration of a pixel included in an imaging element according to a comparative example. [Figure 10] FIG. 10 is a sequence diagram illustrating an example of the operation of a camera including an image sensor according to a comparative example. DETAILED DESCRIPTION OF THE INVENTION

[0008] 1 is a diagram schematically illustrating an example configuration of a digital camera 1 (hereinafter referred to as camera 1) equipped with an image sensor 21 according to one embodiment. Camera 1 includes an interchangeable lens 3 and a camera body 2. Interchangeable lens 3 is attached to camera body 2 via a lens mount (not shown). Note that camera 1 may also be configured as an integrated lens camera rather than an interchangeable lens camera.

[0009] The interchangeable lens 3 includes an imaging optical system 31 including, for example, a zoom lens, a focus lens, an aperture, an anti-vibration lens, etc., and a lens control unit 32. The lens control unit 32 includes peripheral components such as a CPU (Central Processing Unit) and memory. The lens control unit 32 performs drive control of the focus lens and aperture, detects the positions of the zoom lens and focus lens, sends lens information to the camera body 2, and receives camera information from the camera body 2.

[0010] The camera body 2 includes, for example, an image sensor 21, an image processing unit 22, a body control unit 23, a display unit 24, an operation unit 25, and a recording unit 26.

[0011] The operation unit 25 includes a shutter button, operation members for various settings, etc. The display unit 24 is, for example, a liquid crystal monitor (also called a rear monitor) mounted on the rear surface of the camera body 2.

[0012] The body control unit 23 includes a CPU and peripheral components such as memory. The body control unit 23 controls the operation of the camera 1, such as driving and controlling the image sensor 21, reading out image signals from the image sensor 21, performing focus detection calculations and focusing the interchangeable lens 3, and displaying and recording image data. The body control unit 23 also communicates with the lens control unit 32, receiving lens information and transmitting camera information (such as defocus amount and aperture value).

[0013] The recording unit 26 has a card slot into which a storage medium such as a memory card can be inserted. The recording unit 26 stores the image data and various data generated by the image processing unit 22 in the storage medium inserted in the card slot. The recording unit 26 has an internal memory. In this case, the recording unit 26 can also record the image data and various data generated by the image processing unit 22 in the internal memory.

[0014] The image processing unit 22 is an image processing engine that performs various image processes on the pixel data input from the imaging element 21. In this embodiment, the image processing unit 22 includes a first image processing unit 22A and a second image processing unit 22B.

[0015] The first image processing unit 22A performs various image processes on the pixel data input from the imaging element 21, and generates data (live view image data) for displaying a live view image on the display unit 24. The generated live view image data is displayed on the display unit 24 under the control of the body control unit 23.

[0016] The second image processing unit 22B detects flicker of light sources such as fluorescent lamps, mercury lamps, and LEDs using pixel data input from the imaging element 21. The second image processing unit 22B also performs various image processes on the pixel data input from the imaging element 21 to generate image data (still image data) to be stored in a storage medium such as a memory card. The generated still image data is stored in the storage medium by the recording unit 26 under the control of the body control unit 23.

[0017] The first image processing unit 22A and the second image processing unit 22B can each execute processing independently (asynchronously).

[0018] The imaging element 21 is disposed on the planned imaging plane (planned focal plane) of the interchangeable lens 3, and photoelectrically converts the subject image formed by the interchangeable lens 3. The imaging element 21 according to this embodiment is a stacked imaging element. Specifically, the imaging element 21 includes an imaging chip that outputs pixel signals corresponding to incident light, a signal processing chip that processes the pixel signals, and a memory chip that stores the pixel signals. The imaging chip, signal processing chip, and memory chip are stacked and electrically connected to one another by conductive bumps such as Cu.

[0019] Fig. 2 is a diagram illustrating a schematic configuration of the image sensor 21 according to this embodiment. Fig. 3 is a circuit diagram showing the configuration of a pixel 51 included in the image sensor 21.

[0020] 2, the image sensor 21 includes a pixel array 50 including a plurality of pixels 51 arranged in a matrix of M rows and N columns, a vertical drive unit 42, a drive control unit 43, a horizontal drive unit 44, a signal processing unit 45, and a memory unit 46. In this embodiment, M is a multiple of 39, and N is an arbitrary integer, but is not limited to this.

[0021] The pixel 51 is assigned one of four pixels, namely, a green pixel Gb, a green pixel Gr, a blue pixel B, and a red pixel R, according to, for example, the Bayer array.

[0022] 3, pixel 51(m,n) has a photoelectric conversion unit 62 such as a photodiode (PD) and a readout unit 60. Pixel 51(m,n) refers to the pixel 51 located in the mth row and nth column. m is an integer from 1 to M, and n is an integer from 1 to N.

[0023] The photoelectric conversion unit 62 has a function of converting incident light into electric charges and accumulating the photoelectrically converted electric charges. The readout unit 60 includes a transfer unit 63, a discharge unit 64, a floating diffusion (FD) 65, an amplifier unit 66, a first selection switch unit 67, and a second selection switch unit 68.

[0024] The transfer unit 63 is controlled by a signal Tx_m and transfers the charges photoelectrically converted by the photoelectric conversion unit 62 to the floating diffusion 65. That is, the transfer unit 63 forms a charge transfer path between the photoelectric conversion unit 62 and the floating diffusion 65. The floating diffusion 65 holds (accumulates) the charges. The amplifier 66 amplifies and outputs a signal based on the charges held in the floating diffusion 65. The amplifier 66 is connected to the first vertical signal line 52A(n) via a first selection switch unit 67 and to the second vertical signal line 52B(n) via a second selection switch unit 68.

[0025] The discharge unit (reset unit) 64 is controlled by a signal Rst_m to discharge the charge in the floating diffusion 65 and reset the potential of the floating diffusion 65 to a reset potential (reference potential). The first selection switch unit 67 is controlled by a signal Sel_A_m to output the signal from the amplification unit 66 to the first vertical signal line 52A(n). The second selection switch unit 68 is controlled by a signal Sel_B_m to output the signal from the amplification unit 66 to the second vertical signal line 52B(n). The transfer unit 63, the discharge unit 64, the amplification unit 66, the first selection switch unit 67, and the second selection switch unit 68 are each configured by, for example, a transistor M1, a transistor M2, a transistor M3, a transistor M4, and a transistor M5, respectively.

[0026] The readout unit 60 reads out a signal (noise signal) when the potential of the floating diffusion 65 is reset to the reset potential by the discharge unit 64 to the first vertical signal line 52A(n) via the first selection switch unit 67. The readout unit 60 also reads out a signal (photoelectric conversion signal) corresponding to the charge transferred from the photoelectric conversion unit 62 to the floating diffusion 65 by the transfer unit 63 to the first vertical signal line 52A(n) via the first selection switch unit 67.

[0027] Furthermore, the readout unit 60 reads out a signal (noise signal) when the potential of the floating diffusion 65 is reset to the reset potential by the discharge unit 64 to the second vertical signal line 52B(n) via the second selection switch unit 68. The readout unit 60 also reads out a signal (photoelectric conversion signal) corresponding to the charge transferred from the photoelectric conversion unit 62 to the floating diffusion 65 by the transfer unit 63 to the second vertical signal line 52B(n) via the second selection switch unit 68.

[0028] Returning to Figure 2, the drive control unit 43 generates clock signals and control signals that serve as the basis for the operation of the vertical drive unit 42, signal processing unit 45, memory unit 46, horizontal drive unit 44, etc., based on a master clock input from outside and a control signal input from the body control unit 23, and provides these signals to the vertical drive unit 42, signal processing unit 45, memory unit 46, horizontal drive unit 44, etc.

[0029] The vertical drive unit 42 supplies control signals such as a signal Rst_m (where m is an integer from 1 to M), a signal Tx_m, a signal Sel_A_m, and a signal Sel_B_m to N pixels 51 in the mth row, and controls the operation of each pixel 51.

[0030] The signal processing unit 45 includes analog-to-digital conversion units (ADCs) 451a1-451aN and 451b1-451bN and correlated double sampling units (CDSs) 452a1-452aN and 452b1-452bN. The ADCs 451a1-451aN convert noise signals and photoelectrically converted signals input via the first vertical signal line 52A(n) (n is an integer from 1 to N) into digital signals and output them to the CDSs 452a1-452aN, respectively. The ADCs 451b1-451bN convert noise signals and photoelectrically converted signals input via the second vertical signal line 52B(n) into digital signals and output them to the CDSs 452b1-452bN, respectively.

[0031] The CDSs 452a1 to 452aN and 452b1 to 452bN perform correlated double sampling on the input digital signals, remove noise from the digital signals, and output the noise-removed digital signals to the memory unit 46. In the following description, a signal corresponding to the charge accumulated in the photoelectric conversion unit 62 of the pixel 51(m,n) may be referred to as a pixel signal Sig(m,n).

[0032] The memory unit 46 includes memories 461a1 to 461aN and 461b1 to 461bN. The memories 461a1 to 461aN store the pixel signals output from the CDSs 452a1 to 452aN, respectively, and the memories 461b1 to 461bN store the pixel signals output from the CDSs 452b1 to 452bN, respectively.

[0033] In response to a scanning signal from the drive control unit 43, the horizontal drive unit 44 outputs the pixel signals stored in memories 461a1 to 461aN to the first image processing unit 22A or the buffer memory 47, and outputs the pixel signals stored in memories 461b1 to 461bN to the buffer memory 47.

[0034] The pixel signals stored in the memories 461a1 to 461aN and the pixel signals stored in the memories 461b1 to 461bN can be output asynchronously. The pixel signals stored in the memories 461a1 to 461aN and the pixel signals stored in the memories 461b1 to 461bN can be output simultaneously or at different times.

[0035] In this embodiment, pixels 51(3j+1,1) to 51(3j+1,N) on the 3j+1th row (j is an integer from 0 to M / 3-1) are used to generate a live view image to be displayed on the display unit 24, and pixels 51(39k+3,1) to 51(3k+3,N) on the 39k+3th row (k is an integer from 0 to M / 39-1) are used to detect the period and peak of the flicker.

[0036] 4(A) and 4(B) are sequence diagrams showing an example of the operation of camera 1. Fig. 4(A) is a sequence diagram showing an example of the operation of camera 1 when a live view image is displayed on display unit 24.

[0037] The image sensor 21 accumulates electric charges in the photoelectric conversion units 62 provided in the pixels 51(3j+1,1) to 51(3j+1,N) on the (3j+1) row, starting from the pixels 51(1,1) to 51(1,N) on the first row, and reads out pixel signals corresponding to the accumulated electric charges to the signal processing unit 45 via the first vertical signal lines 52A(1) to 52A(N), respectively. In FIG. 4A, the accumulation period of electric charges in the photoelectric conversion units 62 is T2.

[0038] A set of pixel signals from pixels 51(3j+1,1) to 51(3j+1,N) in the 3j+1th row processed by signal processing unit 45 is output as pixel data to first image processing unit 22A.

[0039] Here, for example, the operation of the image sensor 21 when creating a live view image LV2 will be described using the timing chart of Fig. 5. In Fig. 5, a transistor to which a high-level (e.g., power supply potential) control signal is input is turned on, and a transistor to which a low-level (e.g., ground potential) control signal is input is turned off.

[0040] At time t1, when the signal Rst_1 goes high, the transistors M2 of the reset units 64 of the pixels 51(1,1) to 51(1,N) in the first row are turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0041] Furthermore, the transistor M2 of the reset unit 64 is kept on, and the signal Tx_1 is set to high level from time t2 to time t3 to turn on the transistor M1 of the transfer unit 63. As a result, the charge accumulated in the photoelectric conversion unit 62 is discarded.

[0042] When the signal Rst_1 goes low at time t4, the transistor M2 of the reset unit 64 of the pixels 51(1,1) to 51(1,N) on the first row is turned off. When the signal Sel_A_1 goes high at time t5, the transistor M4 of the first selection switch unit 67 is turned on. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the first vertical signal lines 52A(1) to 52A(N).

[0043] At time t6, which is a predetermined accumulation (exposure) time (T2) after time t3, signal Tx_1 is set to high level, thereby turning on transistor M1 of transfer unit 63. Thereafter, signal Tx_1 is set to low level, thereby turning off transistor M1. As a result, the charge accumulated in photoelectric conversion unit 62 from time t3 to time t6 (accumulation period T2) is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0044] Because signal Sel_A_1 is at a high level, pixel signals Sig(1,1) to Sig(1,N) corresponding to the charges transferred from each floating diffusion 65 of pixels 51(1,1) to 51(1,N) are generated by transistor M3 of amplifier unit 66 and output to first vertical signal lines 52A(1) to 52A(N), respectively, via transistor M4 of first selection switch unit 67.

[0045] The pixel signals Sig(1,1) to Sig(1,N) are input to the signal processing unit 45 via the first vertical signal lines 52A(1) to 52A(N), processed by the signal processing unit 45, and then stored in the memories 461a1 to 461aN, respectively. The set of pixel signals stored in the memories 461a1 to 461aN is output as pixel data to the first image processing unit 22A.

[0046] At time t11, when the signal Rst_4 goes high, the transistors M2 of the reset units 64 of the pixels 51(4,1) to 51(4,N) on the fourth row are turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0047] Furthermore, while keeping the transistor M2 of the reset unit 64 in the on state, the signal Tx_4 is set to a high level from time t12 to time t13 to turn on the transistor M1 of the transfer unit 63. As a result, the charge accumulated in the photoelectric conversion unit 62 is discarded.

[0048] At time t7, signal Rst_4 goes low, turning off transistor M2 of the reset unit 64 of pixels 51(1,1) to 51(1,N) on the fourth row. At time t15, signal Sel_A_4 goes high, turning on transistor M4 of the first selection switch unit 67. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the first vertical signal lines 52A(1) to 52A(N).

[0049] At time t16, a predetermined accumulation period (T2) after time t13, signal Tx_4 is set to high level, thereby turning on transistor M1 of transfer unit 63. Thereafter, signal Tx_4 is set to low level, thereby turning off transistor M1. As a result, the charge accumulated in photoelectric conversion unit 62 between time t13 and time t16 is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0050] Because signal Sel_A_4 is at a high level, pixel signals Sig(4,1) to Sig(4,N) corresponding to the charges transferred from each floating diffusion 65 of pixels 51(4,1) to 51(4,N) are generated by transistor M3 of amplifier unit 66 and output to first vertical signal lines 52A(1) to 52A(N), respectively, via transistor M4 of first selection switch unit 67.

[0051] The pixel signals Sig(4,1) to Sig(4,N) are input to the signal processing unit 45 via the first vertical signal lines 52A(1) to 52A(N), processed by the signal processing unit 45, and then stored in memories 461a1 to 461aN. The set of pixel signals stored in the memories 461a1 to 461aN is output as pixel data to the first image processing unit 22A.

[0052] At time t21, when the signal Rst_3j+1 goes high, the transistor M2 of the reset unit 64 of the pixels 51(3j+1,1) to 51(3j+1,N) on the (3j+1)th row is turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0053] Furthermore, while keeping the transistor M2 of the reset unit 64 in the on state, the signal Tx_3j+1 is set to a high level from time t22 to time t23 to turn on the transistor M1 of the transfer unit 63. This causes the charge accumulated in the photoelectric conversion unit 62 to be discarded.

[0054] At time t24, the signal Rst_3j+1 goes low, turning off the transistor M2 of the reset unit 64 of the pixels 51(3j+1,1) to 51(3j+1,N) on the (3j+1)th row. At time t25, the signal Sel_A_3j+1 goes high, turning on the transistor M4 of the first selection switch unit 67. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the first vertical signal lines 52A(1) to 52A(N).

[0055] At time t26, a predetermined accumulation period (T2) after time t23, signal Tx_3j+1 is set to high level to turn on and then off transistor M1 of transfer unit 63. As a result, the charge accumulated in photoelectric conversion unit 62 between time t23 and time t26 is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0056] Because signal Sel_A_3j+1 is at a high level, pixel signals Sig(3j+1,1) to Sig(3j+1,N) corresponding to the charges transferred from each floating diffusion 65 of pixels 51(3j+1,1) to 51(3j+1,N) are generated by transistor M3 of amplifier unit 66 and output to first vertical signal lines 52A(1) to 52A(N) via transistor M4 of first selection switch unit 67.

[0057] The pixel signals Sig(3j+1,1) to Sig(3j+1,N) are input to the signal processing unit 45 via the first vertical signal lines 52A(1) to 52A(N), processed by the signal processing unit 45, and then stored in memories 461a1 to 461aN. The set of pixel signals stored in the memories 461a1 to 461aN is output as pixel data to the first image processing unit 22A.

[0058] In this way, the first image processing unit 22A performs various image processes on the pixel data input from the imaging element 21 to the first image processing unit 22A, and generates live view image data (LV2) (FIG. 4(A)). Thereafter, the body control unit 23 displays a live view image (LV2) based on the live view image data (LV2) in synchronization with a live view vertical synchronization signal.

[0059] In FIG. 5, the time interval T1 during which each of the signals Rst_1 to Rst_3j+1 is at low level is the same as the period T1 (FIG. 4(A)) of the live view vertical synchronization signal.

[0060] FIG. 4B is a sequence diagram showing an example of the operation of camera 1 when detecting flicker.

[0061] To detect flicker, the image sensor 21 accumulates charge in the photoelectric conversion units 62 of the pixels 51(39k+3,1) to 51(39k+3,N) in the (39k+3) row, starting with the pixels 51(3,1) to 51(3,N) in the third row, for an accumulation period T4, and reads out pixel signals corresponding to the accumulated charge to the signal processing unit 45 via the second vertical signal lines 52B(1) to 52B(N). The set of pixel signals (39k+3,1) to (39k+3,N) from the pixels 51(39k+3,1) to 51(39k+3,N) in the (39k+3) row, processed by the signal processing unit 45, is output as pixel data to the buffer memory 47. The pixel data output to the buffer memory 47 is output to the second image processing unit 22B.

[0062] Here, for example, the operation of the image sensor 21 when detecting flicker will be described using the timing chart of Fig. 6. In Fig. 6, a transistor to which a high-level (e.g., power supply potential) control signal is input is turned on, and a transistor to which a low-level (e.g., ground potential) control signal is input is turned off.

[0063] At time t31, when the signal Rst_3 goes high, the transistors M2 of the reset units 64 of the pixels 51(3,1) to 51(3,N) on the third row are turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0064] Furthermore, while the transistor M2 of the reset unit 64 is kept on, the signal Tx_3 is set to a high level from time t32 to time t33 to turn on the transistor M1 of the transfer unit 63. As a result, the charge accumulated in the photoelectric conversion unit 62 is discarded.

[0065] At time t34, signal Rst_3 goes low, turning off transistor M2 of the reset unit 64 of the pixels 51(3,1) to 51(3,N) on the third row. At time t35, signal Sel_B_3 goes high, turning on transistor M5 of the second selection switch unit 68. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the second vertical signal lines 52B(1) to 52B(N).

[0066] At time t36, a predetermined accumulation period (T4) after time t33, signal Tx_3 is set to high level to turn on transistor M1 of transfer unit 63, and then signal Tx_3 is set to low level to turn off transistor M1. As a result, the charge accumulated in photoelectric conversion unit 62 from time t33 to time t36 is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0067] Because signal Sel_B_3 is at a high level, pixel signals Sig(3,1) to Sig(3,N) corresponding to the charges transferred from each floating diffusion 65 of pixels 51(3,1) to 51(3,N) are generated by transistor M3 of amplifier unit 66 and output to second vertical signal lines 52B(1) to 52B(N) via transistor M5 of second selection switch unit 68.

[0068] The pixel signals Sig(3,1) to Sig(3,N) are input to the signal processing unit 45 via the second vertical signal lines 52B(1) to 52B(N), processed by the signal processing unit 45, and then stored in memories 461b1 to 461bN. The set of pixel signals stored in memories 461b1 to 461bN is temporarily stored as pixel data in the buffer memory 47, and then output to the second image processing unit 22B.

[0069] At time t41, when the signal Rst_42 goes high, the transistor M2 of the reset unit 64 of the pixels 51(42,1) to 51(42,N) on the 42nd row is turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0070] Furthermore, while keeping the transistor M2 of the reset unit 64 in the on state, the signal Tx_42 is set to a high level from time t42 to time t43 to turn on the transistor M1 of the transfer unit 63. As a result, the charge accumulated in the photoelectric conversion unit 62 is discarded.

[0071] At time t44, the signal Rst_42 goes low, turning off the transistor M2 of the reset unit 64 of the pixels 51(42,1) to 51(42,N) on the 42nd row. At time t45, the signal Sel_B_42 goes high, turning on the transistor M5 of the second selection switch unit 68. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the second vertical signal lines 52B(1) to 52B(N).

[0072] At time t46, a predetermined accumulation period (T4) after time t43, signal Tx_42 is set to high level to turn on and then off transistor M1 of transfer unit 63. As a result, the charge accumulated in photoelectric conversion unit 62 from time t43 to time t46 is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0073] Because signal Sel_B_42 is at a high level, pixel signals Sig(42,1) to Sig(42,N) corresponding to the charges transferred from each floating diffusion 65 of pixels 51(42,1) to 51(42,N) are generated by transistor M3 of amplifier unit 66 and output to second vertical signal lines 52B(1) to 52B(N) via transistor M5 of second selection switch unit 68.

[0074] The pixel signals Sig(42,1) to Sig(42,N) are input to the signal processing unit 45 via the second vertical signal lines 52B(1) to 52B(N), processed by the signal processing unit 45, and then stored in memories 461b1 to 461bN. The set of pixel signals stored in memories 461b1 to 461bN is temporarily stored as pixel data in the buffer memory 47, and then output to the second image processing unit 22B.

[0075] At time t51, when the signal Rst_39k+3 goes high, the transistor M2 of the reset unit 64 of the pixels 51(39k+3,1) to 51(39k+3,N) on the 39k+3th row is turned on, thereby resetting the charge in the floating diffusion 65 and setting the potential of the floating diffusion 65 to the reset potential.

[0076] Furthermore, while keeping the transistor M2 of the reset unit 64 in the on state, the signal Tx_39k+3 is set to a high level from time t522 to time t53 to turn on the transistor M1 of the transfer unit 63. As a result, the charge accumulated in the photoelectric conversion unit 62 is discarded.

[0077] At time t54, the signal Rst_39k+3 goes low, turning off the transistor M2 of the reset unit 64 of the pixels 51(39k+3,1) to 51(39k+3,N) on the (39k+3)th row. At time t55, the signal Sel_B_39k+3 goes high, turning on the transistor M5 of the second selection switch unit 68. As a result, a signal (noise signal) obtained when the potential of the floating diffusion 65 is reset to the reset potential is read out to the second vertical signal lines 52B(1) to 52B(N).

[0078] At time t56, a predetermined accumulation period (T4) after time t53, signal Tx_39k+3 is set to high level to turn on transistor M1 of transfer unit 63, and then signal Tx_39k+3 is set to low level to turn off transistor M1. As a result, the charge accumulated in photoelectric conversion unit 62 from time t53 to time t56 is transferred to the floating diffusion 65 via transistor M1 of transfer unit 63, and then transferred to the gate of transistor M3 of amplifier unit 66.

[0079] Since the signal Sel_B_39k+3 is at a high level, pixel signals Sig(39k+3,1) to Sig(39k+3,N) corresponding to the charges transferred by the transistor M1 of the transfer unit 63 are generated by the transistor M3 of the amplifier unit 66 and output to the second vertical signal lines 52B(1) to 52B(N) via the transistor M5 of the second selection switch unit 68.

[0080] The pixel signals Sig(39k+3,1) to Sig(39k+3,N) are input to the signal processing unit 45 via the second vertical signal lines 52B(1) to 52B(N), processed by the signal processing unit 45, and then stored in memories 461b1 to 461bN. The set of pixel signals stored in memories 461b1 to 461bN is temporarily stored as pixel data in the buffer memory 47, and then output to the second image processing unit 22B.

[0081] The second image processing unit 22B detects flicker using pixel data input to the second image processing unit 22B from the imaging element 21. Specifically, it detects the cycle in which the light amount of the light source that causes the flicker phenomenon increases and decreases, and the timing at which the light amount reaches its peak.

[0082] In FIG. 6, the time interval T3 during which each of the signals Rst_3 to Rst_39k+3 is at low level is the same as the period T3 (FIG. 4(B)) of the vertical synchronization signal for flicker.

[0083] In this embodiment, the image sensor 21 has two vertical signal lines 52A(n) and 52B(n), and ADCs 451a1-451aN and 451b1-451bN, CDSs 452a1-452aN and 452b1-452bN, and memories 461a1-461aN and 461b1-461bN corresponding to the two vertical signal lines 52A(n) and 52B(n), respectively. Therefore, the processing of Fig. 4(A) and the processing of Fig. 4(B) can be executed asynchronously. That is, the processing of Fig. 4(A) and the processing of Fig. 4(B) can be executed in parallel at the same time.

[0084] Next, the operation of the camera 1 equipped with the image sensor 21 according to this embodiment when an instruction to capture a still image is given (when the release button is pressed all the way down) will be described with reference to the sequence diagram of FIG.

[0085] As described above, in camera 1 according to this embodiment, first image processing unit 22A executes the process of generating live view image data, and second image processing unit 22B executes the process of detecting flicker and the process of generating still image data. In this embodiment, second image processing unit 22B executes the process of detecting flicker at a predetermined cycle (time interval) until the release button is fully pressed.

[0086] As shown in FIG. 7, the processing by the first image processing unit 22A to display a live view image on the display unit 24 and the processing by the second image processing unit 22B to detect a flicker peak (peak detection) are executed in parallel (asynchronously).

[0087] For example, suppose the release button is pressed all the way down during the exposure period for live view image LV7. In this case, because second image processing unit 22B repeatedly executes peak detection processing, body control unit 23 adjusts the timing to start exposure for generating a still image based on the result of the most recent peak detection. Specifically, the exposure start timing is adjusted so that exposure of pixels 51(1,1) to 51(1,N) on the first row and pixels 51(2,1) to 51(2,N) on the second row begins around the time when the light intensity of the light source reaches its peak.

[0088] The imaging element 21 starts exposing the pixels 51(1,1) to 51(1,N) in the first row and the pixels 51(2,1) to 51(2,N) in the second row at the adjusted timing. The image sensor 21 outputs pixel signals Sig(1,1) to Sig(1,N) of the pixels 51(1,1) to 51(1,N) in the first row to the signal processing unit 45 (ADCs 451a1 to 451aN and CDSs 452a1 to 452aN) via first vertical signal lines 52A(1) to 52A(N), and outputs pixel signals Sig(2,1) to Sig(2,N) of the pixels 51(2,1) to 51(2,N) in the second row to the signal processing unit 45 (ADCs 451b1 to 451bN and CDSs 452b1 to 452bN) via second vertical signal lines 52B(1) to 52B(N).

[0089] The pixel signals Sig(1,1) to Sig(1,N) of pixels 51(1,1) to 51(1,N) processed by signal processing unit 45 are stored in memories 461a1 to 461aN, and the pixel signals Sig(2,1) to Sig(2,N) of pixels 51(2,1) to 51(2,N) are stored in memories 461b1 to 461bN. The pixel signals stored in memories 461a1 to 461aN and the pixel signals stored in memories 461b1 to 461bN are output to buffer memory 47.

[0090] At this time, pixel data including pixel signals output from the pixels 51 in the first and second rows is stored in the buffer memory 47. When the exposure of the pixels in the first and second rows is completed, the exposure of the pixels in the third and fourth rows begins.

[0091] In this way, the pixel signals of the pixels in the two rows are stored in the memory unit 46, and pixel data including the pixel signals of the two rows is accumulated in the buffer memory 47.

[0092] The pixel data stored in the buffer memory 47 is output to the second image processing unit 22B, which then performs various image processes to create still image data. Thereafter, under the control of the body control unit 23, the created still image data is stored in a storage medium by the recording unit 26.

[0093] In this way, in the image sensor 21 according to this embodiment, two vertical signal lines 52A and 52B are provided for each pixel 51, and therefore, two rows of pixel data required for generating still image data can be created at a time, thereby shortening the time from when exposure for generating a still image begins to when the recording process of the still image data begins.

[0094] Furthermore, in camera 1 according to this embodiment, instead of starting exposure to create flicker image data after the release button is fully pressed, flicker peak detection is performed at predetermined time intervals. Therefore, the timing to start exposure to create a still image can be determined using the result of the most recent peak detection. Therefore, compared to when exposure to create flicker image data is started and flicker is detected after the release button is fully pressed (after an image capture command is received), the time from when the release button is fully pressed to when exposure to create a still image is started can be shortened.

[0095] Furthermore, because the time until exposure for generating a still image begins can be shortened, the time from the end of exposure for generating a live view image (LV7) to the start of exposure for generating the next live view image (LV8) can be shortened, thereby shortening the period (blackout period) from the end of displaying live view image LV7 to the next display of live view image LV8.

[0096] (Comparative Example) FIG. 8 is a diagram illustrating a schematic configuration of an image sensor 21' according to a comparative example, and FIG. 9 is a circuit diagram illustrating a configuration of a pixel 51' according to the comparative example.

[0097] As shown in FIG. 8, the imaging element 21′ includes a pixel array 50′ including a plurality of pixels 51′ arranged in a matrix of M rows and N columns, a vertical driving unit 42′, a driving control unit 43′, a horizontal driving unit 44′, a signal processing unit 45′, and a memory unit 46′.

[0098] In the comparative example, pixels 51' are arranged in M ​​rows and N columns. The pixels 51' are assigned one of four pixels: green pixels Gb, Gr, blue pixels B, and red pixels R, for example, according to the Bayer array.

[0099] 9, the pixel 51'(m,n) has a photoelectric conversion unit 62 such as a photodiode (PD) and a readout unit 60'. The photoelectric conversion unit 62 converts incident light into electric charges and accumulates the photoelectrically converted electric charges. The readout unit 60' has a transfer unit 63, a discharge unit 64, a floating diffusion (FD) 65, an amplifier unit 66, and a selection switch unit 69.

[0100] The transfer unit 63 is controlled by a signal Tx_m and transfers the charges photoelectrically converted by the photoelectric conversion unit 62 to the floating diffusion 65. That is, the transfer unit 63 forms a charge transfer path between the photoelectric conversion unit 62 and the floating diffusion 65. The floating diffusion 65 holds (accumulates) the charges. The amplifier unit 66 amplifies and outputs a signal based on the charges held in the floating diffusion 65. The amplifier unit 66 is connected to the vertical signal line 52(n) via a selection switch unit 69.

[0101] The discharge unit (reset unit) 64 is controlled by a signal Rst_m to discharge the charge in the floating diffusion 65 and reset the potential of the floating diffusion 65 to a reset potential (reference potential). The selection switch unit 69 is controlled by a signal Sel_m to output the signal from the amplification unit 66 to the vertical signal line 52(n). The transfer unit 63, the discharge unit 64, the amplification unit 66, and the selection switch unit 69 are each configured by, for example, a transistor M1, a transistor M2, a transistor M3, and a transistor M6, respectively.

[0102] The readout unit 60′ reads out a signal (noise signal) when the potential of the floating diffusion 65 is reset to the reset potential by the discharge unit 64 onto the vertical signal line 52(n) (n is an integer from 1 to N) via the selection switch unit 69. The readout unit 60′ also reads out a signal (photoelectric conversion signal) corresponding to the charge transferred from the photoelectric conversion unit 62 to the floating diffusion 65 by the transfer unit 63 onto the vertical signal line 52(n) via the selection switch unit 69.

[0103] Returning to FIG. 8, the drive control unit 43′ generates clock signals and control signals that serve as the basis for the operation of the vertical drive unit 42′, signal processing unit 45′, memory unit 46′, horizontal drive unit 44′, etc., based on a master clock input from the outside and a control signal input from the body control unit 23, and provides these signals to the vertical drive unit 42′, signal processing unit 45′, memory unit 46′, horizontal drive unit 44′, etc.

[0104] The vertical drive unit 42′ supplies control signals such as the signal Rst_m, the signal Tx_m, and the signal Sel_m to the N pixels 51′(m,1) to (m,N) in the mth row, and controls the operation of each of the pixels 51′(m,1) to (m,N).

[0105] The signal processing unit 45' includes analog-to-digital conversion units (ADC) 451(1) to 451(N) and correlated double sampling units (CDS) 452(1) to 452(N). The ADCs 451(1) to 451(N) convert noise signals and photoelectric conversion signals input via vertical signal lines 52(n) into digital signals and output the digital signals to the CDSs 452(1) to 452(N), respectively.

[0106] The CDSs 452(1) to 452(N) perform correlated double sampling on the input digital signals, remove noise from the pixel signals, and output the pixel signals from which the noise has been removed to the memory unit 46'.

[0107] The memory unit 46' includes memories 461(1) to 461(N). The memories 461(1) to 461(N) store the pixel signals output from the CDSs 452(1) to 452(N), respectively.

[0108] The horizontal drive unit 44' outputs the pixel signals stored in memories 461(1) to 461(N) to a buffer memory 47' in response to a scanning signal from the drive control unit 43'. The pixel signals temporarily stored in the buffer memory 47' are output to either the first image processing unit 22A' or the second image processing unit 22B'. In the comparative example, the first image processing unit 22A' performs various image processes on the pixel data to generate live view image data and to generate flicker image data from the pixel data and detect flicker. The second image processing unit 22B' performs various image processes on the pixel data output from the image sensor 21' and temporarily stored in the buffer memory 47' to create still image data.

[0109] 10 is a sequence diagram illustrating an example of the operation of a camera equipped with an image sensor 21' according to a comparative example. Note that the camera according to comparative example 1 is set to perform flicker detection when the release button is fully pressed.

[0110] 10, for example, when the camera is powered on, live view images (LV1 to LV4) generated based on pixel signals output from the photoelectric conversion units 62 of the pixels 51(3j+1,1) to 51(3j+1,N) in the above-mentioned 3j+1th row are displayed on the display unit 24. Here, for example, suppose the release button is pressed all the way down during the exposure period (accumulation period) for generating live view image LV4.

[0111] In this case, after the accumulation of charges in the photoelectric conversion units 62 of the pixels 51′(39k+1,1) to 51′(39k+1,N) in the 39k+3 row used for flicker detection and the output of pixel data to the first image processing unit 22A′ are performed multiple times at a predetermined timing in accordance with the flicker vertical synchronization signal, the first image processing unit 22A′ detects a peak. Then, the body control unit 23 adjusts the exposure start timing so that exposure for creating a still image begins around the time when the light intensity of the light source peaks. The image sensor 21′ sequentially starts exposure processing starting with the pixels in the first row at the adjusted timing and outputs the pixel data to the buffer memory 47′. The second image processing unit 22B′ performs various image processes on the pixel data stored in the buffer memory 47′ to generate still image data. The generated still image data is stored in a storage medium by the recording unit 26 under the control of the body control unit 23.

[0112] When the exposure process for creating a still image is completed, the exposure process for generating a live view image LV5 is restarted in synchronization with the live view vertical synchronization signal.

[0113] In this case, in the comparative example, after the display of live-view image LV4 has finished, flicker peak detection and exposure processing for still images have finished, exposure for generating live-view image LV5 begins. Therefore, the period from the end of display of live-view image LV4 to the start of display of the next live-view image LV5 (blackout period) is longer than the blackout period in camera 1 according to this embodiment (see FIG. 7).

[0114] In this embodiment, as described above, the time from when the release button is fully pressed until exposure to detect flicker begins can be shortened compared to when exposure to create a still image begins, so the time from when the display of live view image LV7 ends until display of live view image LV8 begins can be shortened, resulting in a shorter blackout period than in the comparative example.

[0115] As described above in detail, according to this embodiment, the image sensor 21 includes pixels 51(3j+1,1) to 51(3j+1,N) and pixels 51(39k+1,1) to 51(39k+1,N). The pixels 51(3j+1,1) to 51(3j+1,N) and pixels 51(39k+3,1) to 51(39k+3,N) are arranged in the column direction. The photoelectric conversion units 62 (first photoelectric conversion units) of the pixels 51(3j+1,1) to 51(3j+1,N) and the photoelectric conversion units 62 (second photoelectric conversion units) of the pixels 51(39k+3,1) to 51(39k+3,N) perform photoelectric conversion on light transmitted through the image pickup optical system 31 to generate electric charges. The image sensor 21 also includes first vertical signal lines 52A(1) to 52A(N) that output pixel signals used to generate a live view image based on the charges generated in the photoelectric conversion units 62 of the pixels 51(3j+1,1) to 51(3j+1,N), and second vertical signal lines 52B(1) to 52B(N) that output pixel signals used to detect flicker (shooting state) based on the charges generated in the photoelectric conversion units 62 of the pixels 51(39k+3,1) to 51(39k+3,N). Furthermore, the imaging element 21 includes a horizontal drive unit 44 (first output unit, second output unit) that outputs pixel data based on pixel signals based on charges generated in the photoelectric conversion units 62 of pixels 51(3j+1,1) to 51(3j+1,N) to the first image processing unit 22A, and outputs pixel data based on pixel signals based on charges generated in the photoelectric conversion units 62 of pixels 51(39k+3,1) to 51(39k+3,N) to the second image processing unit 22B (buffer memory 47).

[0116] This allows the output of pixel data based on pixel signals from pixels 51(3j+1,1) to 51(3j+1,N) to the first image processing unit 22A and the output of pixel data based on pixel signals from pixels 51(39k+3,1) to 51(39k+3,N) to the second image processing unit 22B to be performed asynchronously (in parallel), thereby allowing the display of a live view image to continue even while the second image processing unit 22B is detecting flicker.

[0117] Furthermore, in this embodiment, when a command to generate a still image is issued, pixel signals (third signals) based on charges generated by the photoelectric conversion units 62 of pixels 51(x,1) to 51(x,N) in the x-th row (x is an odd number between 1 and M) are output to the first vertical signal lines 52A(1) to 52A(N), and pixel signals (fourth signals) based on charges generated by pixels 51(x+1,1) to 51(x+1,N) in the x+1-th row are output to the second vertical signal lines 52B(1) to 52B(N). This allows pixel signals to be read out two rows at a time, thereby shortening the time required to generate still image data.

[0118] Furthermore, according to this embodiment, the camera 1 includes the image sensor 21, the display unit 24, a first image processing unit 22A that generates a live view image to be displayed on the display unit 24 based on pixel data derived from pixel signals from pixels 51(3j+1,1) to 51(3j+1,N), and a second image processing unit 22B that detects flicker based on pixel data derived from pixel signals from pixels 51(39k+3,1) to 51(39k+3,N). This allows the first image processing unit 22A to generate a live view image even while the second image processing unit 22B is detecting flicker, thereby allowing the live view image to continue to be displayed.

[0119] Furthermore, in this embodiment, camera 1 includes operation unit 25, and upon receiving an instruction to capture a still image via operation unit 25, second image processing unit 22B generates a still image based on pixel data that is based on pixel signals (third signals) from pixels 51(x,1) to 51(x,N) in the xth row (x is an odd number from 1 to N / 2) and pixel signals (fourth signals) from pixels 51(x+1,1) to 51(x+1,N) in the x+1th row. As a result, each pixel data includes pixel signals from two rows, thereby reducing the time required to create still image data.

[0120] In the above embodiment, pixel data is generated two rows at a time when generating still image data, but pixel data may be generated one row at a time. In this case, pixel signals may be output to the signal processing unit 45 using either the first vertical signal line 52A or the second vertical signal line 52B.

[0121] In the above embodiment, the second image processing unit 22B detects flicker until the release button is fully pressed, but this is not limiting. For example, the second image processing unit 22B may perform focus detection processing based on pixel data based on pixel signals from the pixels 51 used for focus detection.

[0122] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]

[0123] 1 camera 2 Camera body 3 Interchangeable lenses 21 Image sensor 22A First image processing unit 22B Second image processing section 25 Control section 31 Imaging optical system 44 Horizontal drive unit 51 pixels 52A 1st vertical signal line 52B 2nd vertical signal line 62 Photoelectric conversion unit

Claims

1. a first photoelectric conversion unit and a second photoelectric conversion unit that perform photoelectric conversion on light that has passed through the optical system to generate electric charges and are arranged in a column direction; a first signal line through which a first signal used to generate a first image based on the charges generated by the first photoelectric conversion unit is output; a second signal line through which a second signal used to detect an imaging state is output based on the electric charge generated by the second photoelectric conversion unit; a first output unit that outputs first data based on the first signal; a second output unit that outputs second data based on the second signal; An imaging element comprising:

2. the first output unit and the second output unit simultaneously output the first data and the second data; The imaging device according to claim 1 .

3. the first photoelectric conversion unit and the second photoelectric conversion unit are included in a plurality of photoelectric conversion units that are two-dimensionally arranged in row and column directions, When generation of a second image is instructed, a third signal used for generating the second image based on charges generated in a photoelectric conversion unit in an m-th row (m is an odd number from 1 to any integer M) among the plurality of photoelectric conversion units is output to the first signal line; a fourth signal used for generating the second image based on charges generated by a photoelectric conversion unit in the (m+1)th row among the plurality of photoelectric conversion units is output to the second signal line; 3. The imaging device according to claim 1.

4. the first photoelectric conversion unit and the second photoelectric conversion unit are included in a plurality of photoelectric conversion units that are two-dimensionally arranged in row and column directions, When generation of a second image is instructed, a signal used for generating the second image based on the charges generated in the plurality of photoelectric conversion units is output to either the first signal line or the second signal line in sequence starting from the photoelectric conversion units in a first row; The imaging device according to claim 1 .

5. The imaging device according to any one of claims 1 to 4, A display unit; a first processing unit that generates the first image to be displayed on the display unit based on the first data output from the first output unit; a second processing unit that detects the photographing state based on the second data output from the second output unit; An imaging device comprising:

6. the first processing unit and the second processing unit simultaneously execute the process of generating the first image and the process of detecting the photographing state. The imaging device according to claim 5 .

7. Equipped with an operating unit, the first photoelectric conversion unit and the second photoelectric conversion unit are included in a plurality of photoelectric conversion units that are two-dimensionally arranged in row and column directions, When generation of a second image is instructed via the operation unit, the second processing unit generates the second image based on fifth data that is based on a third signal that is output from the first output unit and is used to generate the second image based on charges generated in a photoelectric conversion unit in an m-th row (m is an odd number from 1 to any integer M) among the plurality of photoelectric conversion units, and a fourth signal that is output from the second output unit and is used to generate the second image based on charges generated in a photoelectric conversion unit in an m+1-th row among the plurality of photoelectric conversion units.

7. The imaging device according to claim 5.

Images