Image sensor, imaging device, imaging method, and computer program

Abstract

The present invention provides an image sensor, method, and program that enable high dynamic range (HDR) and phase-difference detection without degrading parasitic photosensitivity (PLS) performance. [Solution] An image sensor having a plurality of pixels 105 arranged in two dimensions, each having first and second photoelectric conversion units 201 and 202, wherein each pixel has a first charge holding unit 211 that holds charge converted photoelectrically for a first storage period and a second charge holding unit 212 that holds charge converted photoelectrically for a second storage period different from the first storage period, a third charge holding unit 213 that holds charge converted photoelectrically for a first storage period and a fourth charge holding unit 214 that holds charge converted photoelectrically for a second storage period, and the first and third charge holding units and the second and fourth charge holding units are each arranged in the diagonal peripheral part of the pixel, and the first charge holding unit or third charge holding unit of a pixel in each row is adjacent to the first charge holding unit or third charge holding unit of a pixel in an adjacent row, separated by a boundary line in the row direction.

Description

Technical Field

[0001] The present invention relates to an imaging device, an imaging apparatus, an imaging method, a computer program, and the like.

Background Art

[0002] Among the electronic shutter methods in a CMOS image sensor, the global electronic shutter (GS) method can perform exposure simultaneously for all pixels, and it is possible to take a picture without generating the moving object distortion that occurs in the rolling shutter method that performs exposure for each row.

[0003] In Patent Document 1, a configuration for realizing GS is disclosed, in which the signal charge generated according to the incident light amount is transferred and held in a charge holding portion covered with a light shielding film, and then read out for each row.

[0004] In this method, if complete transfer of the signal charge is possible, GS can be realized without introducing circuit elements accompanied by additional noise. On the other hand, a false signal may occur due to the influence of parasitic light sensitivity (PLS) caused by light leaking into the charge holding portion.

[0005] Also, in Patent Document 1, an example is disclosed in which two charge holding portions are provided to hold signal charges with different exposure times, thereby expanding the dynamic range and realizing GS and high dynamic range (HDR) imaging simultaneously.

[0006] In Patent Document 2, similar to Patent Document 1, a configuration is provided with two charge holding portions in one pixel, but an example is shown in which the photoelectric conversion portion is divided in the horizontal direction and two charge holding portions are used to acquire phase difference information.

[0007] Thereby, a configuration example enabling focus detection using phase difference information is shown. On the other hand, for a subject with a difficult-to-obtain parallax, such as a horizontal pattern, the accuracy of focus detection may decrease.

[0008] Patent Document 3 discloses an example of improving focus detection accuracy by mixing pixels whose division direction in the photoelectric conversion unit is horizontal (horizontally divided pixels) and pixels whose division direction is vertical (vertically divided pixels). [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2020-108022 [Patent Document 2] Japanese Patent Publication No. 2017-55359 [Patent Document 3] Japanese Patent Publication No. 2020-141122 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] However, in the technologies described in Patent Documents 1 to 3, for example, it is necessary to open contacts in the light-shielding film of the charge-holding part and wire control lines, which may worsen the PLS (Power Line Surgery).

[0011] One of the objectives of the present invention is to provide an image sensor that enables HDR and phase difference detection without degrading PLS performance. [Means for solving the problem]

[0012] An image sensor having a plurality of pixels arranged in a two-dimensional manner, each having a first photoelectric conversion unit and a second photoelectric conversion unit, The pixel has a first charge holding portion that holds charge photoelectrically converted for a first storage period in the first photoelectric conversion portion, a second charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the first photoelectric conversion portion, a third charge holding portion that holds charge photoelectrically converted for a first storage period in the second photoelectric conversion portion, and a fourth charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the second photoelectric conversion portion. In the aforementioned pixel, the first charge-holding portion and the third charge-holding portion are arranged in the diagonal peripheral portion of the pixel, and the second charge-holding portion and the fourth charge-holding portion are arranged in the other diagonal peripheral portion of the pixel. The first charge-holding portion or the third charge-holding portion of the pixel in each column is arranged so as to be adjacent to the first charge-holding portion or the third charge-holding portion of the pixel in an adjacent column, with the boundary line in the column direction in between. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide an image sensor that enables HDR and phase difference detection without degrading PLS performance. [Brief explanation of the drawing]

[0014] [Figure 1] This diagram schematically shows an example of the overall configuration of the image sensor 100 according to the embodiment. [Figure 2] This figure schematically shows an example of the equivalent circuit of pixel 105 according to the embodiment. [Figure 3] This figure schematically shows an example of the division direction of the photoelectric conversion unit and an example of the arrangement of the charge holding unit within a pixel according to the embodiment, over a 2x2 area of ​​4 pixels. [Figure 4] This figure schematically shows an example of a cross-section of a pixel according to the embodiment. [Figure 5A] This is a timing chart showing an example of pixel accumulation operation according to the embodiment. [Figure 5B] This is a timing chart showing an example of a readout operation after the pixel accumulation operation according to the embodiment. [Figure 6] This graph shows an example of the relationship between exposure amount and signal amount according to the embodiment. [Figure 7] This is a functional block diagram schematically showing an example of the configuration of an imaging device according to the embodiment. [Figure 8] This figure shows an example of the correspondence between the pixels 105 of the image sensor according to the embodiment and the pupil intensity distribution. [Figure 9] This figure schematically shows an example of pupil intensity distribution according to the embodiment. [Figure 10] It is a diagram for explaining an example of the correspondence between the image pickup device according to the embodiment and the pupil intensity distribution. [Figure 11] It is a diagram for explaining an example of pupil division in the imaging optical system and the image pickup device according to the embodiment.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same members or elements are given the same reference numerals, and duplicate explanations are omitted or simplified.

[0016] FIG. 1 is a diagram schematically showing an overall configuration example of an image pickup device 100 according to an embodiment of the present invention. The image pickup device 100 includes a pixel array unit 101, a vertical selection circuit 102, a column circuit 103, a horizontal selection circuit 104, and the like.

[0017] Pixels 105 are arranged in a two-dimensional array in the pixel array unit 101. Also, a reset signal is applied to all the pixels via a pixel control line group 107, and by turning off the reset signal, exposure (charge accumulation) starts.

[0018] Thereafter, by applying a transfer signal to the in-pixel charge holding unit to all the pixels via the pixel control line group 107, exposure (charge accumulation) is terminated. Thus, the timings of exposure start and end can be made the same for all the pixels.

[0019] Also, thereafter, when the output of the vertical selection circuit 102 is supplied to a pixel group of a predetermined row via the pixel control line group 107, the phase difference signal and the imaging signal generated in each pixel of that row can be output to the vertical signal lines 106, respectively. In the present embodiment, one vertical signal line 106 is arranged for each column, but the vertical signal lines 106 may be arranged one for each plurality of columns or a plurality may be arranged in one column.

[0020] The signal from each pixel is output to the column circuit 103 via the vertical signal line 106, where it undergoes processing such as noise reduction, amplification, and AD conversion. The signal is then output to the outside of the image sensor 100 via horizontal signal lines (not shown), starting with the column selected by the horizontal selection circuit 104.

[0021] By repeating the above operations, the imaging signal and phase difference signal of the two-dimensional array of pixels during the simultaneous exposure period for all pixels can be output from the image sensor 100.

[0022] Figure 2 is a schematic diagram showing an example of an equivalent circuit of a pixel 105 according to the embodiment. In Figure 2, 201 (PDA) and 202 (PDB) are two photodiodes, which are provided within a single pixel. Here, photodiodes 201 and 202 function as a first photoelectric conversion unit and a second photoelectric conversion unit, respectively. Thus, the image sensor of this embodiment has multiple pixels, each having a first photoelectric conversion unit and a second photoelectric conversion unit, arranged in a two-dimensional manner.

[0023] Furthermore, transistors 203 (GSAL) and 204 (GSAS) are used to transfer the signal charge generated by the photodiode 201 to the first charge holding section 211 (MEMAL) and the second charge holding section 212 (MEMAS), respectively.

[0024] In this embodiment, the first charge-holding portion 211 (MEMAL) and the second charge-holding portion 212 (MEMAS) are formed as potential wells under the gate electrodes of transistors 203 (GSAL) and 204 (GSAS). That is, the gate electrodes of transistors 203 (GSAL) and 204 (GSAS) correspond to the first charge-holding portion 211 (MEMAL) and the second charge-holding portion 212 (MEMAS).

[0025] Therefore, when transferring charge from the photodiode 201 to the first charge holding section 211 (MEMAL) and the second charge holding section 212 (MEMAS), a high-level voltage is applied to the gate electrodes of transistors 203 (GSAL) and 204 (GSAS), respectively. This lowers the potential well under the gate electrodes of transistors 203 (GSAL) and 204 (GSAS), thereby transferring the charge from the photodiode 201.

[0026] Furthermore, transistors 205 (GSBL) and 206 (GSBS) are used to transfer the signal charge generated by the photodiode 202 to the third charge holding section 213 (MEMBL) and the fourth charge holding section 214 (MEMBS), respectively.

[0027] In this embodiment, the third charge-holding portion 213 (MEMBL) and the fourth charge-holding portion 214 (MEMBS) are formed as potential wells under the gate electrodes of transistors 205 (GSBL) and 206 (GSBS). That is, the gate electrodes of transistors 205 (GSBL) and 206 (GSBS) correspond to the third charge-holding portion 213 (MEMBL) and the fourth charge-holding portion 214 (MEMBS).

[0028] Therefore, when transferring charge from the photodiode 202 to the third charge holding section 213 (MEMBL) and the fourth charge holding section 214 (MEMBS), a high-level voltage is applied to the gate electrodes of transistors 205 (GSBL) and 206 (GSBS), respectively. This lowers the potential well under the gate electrodes of transistors 205 (GSBL) and 206 (GSBS), thereby transferring the charge from the photodiode 202.

[0029] Furthermore, the first charge holding unit and the third charge holding unit are for holding the charge converted photoelectrically during the first storage period in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively. The second charge holding unit and the fourth charge holding unit are for holding the charge converted photoelectrically during the second storage period, which is different from the first storage period, in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively. In this embodiment, the first storage period is longer than the second storage period.

[0030] Furthermore, transistors 207 (TXAL) and 208 (TXAS) are used to transfer signal charge from the first charge holding unit 211 (MEMAL) and the second charge holding unit 212 (MEMAS) to the charge-voltage conversion unit 215 (FD), respectively.

[0031] Furthermore, 209 (TXBL) and 210 (TXBS) are transistors for transferring signal charge from the third charge holding unit 213 (MEMBL) and the fourth charge holding unit 214 (MEMBS) to the charge-voltage conversion unit 215, respectively.

[0032] Furthermore, 216(RES) is a reset transistor for resetting the potential of the charge-voltage conversion unit 215. 217(SF) is an amplification transistor for outputting the signal voltage to the vertical signal line 106.

[0033] Furthermore, 218 (SEL) is a selection transistor that selects pixels by receiving a selection signal from the vertical selection circuit 102, and 219 (OFGA) and 220 (OFGB) are discharge transistors that discharge charge from photodiodes 201 and 202, respectively.

[0034] Figure 3 is a schematic diagram showing an example of the division direction of the photoelectric conversion section and an example of the arrangement of the charge holding section within a pixel according to the embodiment, over a 2x2 area of ​​4 pixels. Specifically, Figure 3 shows an example of the division direction of the photoelectric conversion section and an example of the arrangement of the charge holding section for each of the four pixels 105.

[0035] In the example shown in Figure 3, of the four pixels in a 2x2 arrangement, the photoelectric conversion sections of the top left, top right, and bottom right pixels 105 each have two photodiodes 201 and 202 that are divided horizontally. In addition, the photoelectric conversion section of the bottom left pixel 105 has two photodiodes 201 and 202 that are divided vertically.

[0036] Here, a pixel having two photodiodes 201 and 202 with the photoelectric conversion section divided horizontally is called the first pixel, and a pixel having two photodiodes 201 and 202 with the photoelectric conversion section divided vertically is called the second pixel. Note that the ratio of the number of first pixels and second pixels within each pixel and their positional relationship are not limited to the example shown in Figure 3.

[0037] Thus, the plurality of pixels in this embodiment include a first pixel in which a first photoelectric conversion unit and a second photoelectric conversion unit are arranged horizontally, and a second pixel in which a first photoelectric conversion unit and a second photoelectric conversion unit are arranged vertically.

[0038] In Figure 3, 203G to 206G represent the gate electrodes of transistors 203 (GSAL), 204 (GSAS), 205 (GSBL), and 206 (GSBS), respectively, and 203G to 206G function as the first to fourth gate electrodes, respectively.

[0039] As mentioned above, the first to fourth charge-holding portions 211 to 214 are formed below the first to fourth gate electrodes. Therefore, in Figure 3, to show their correspondence, the gate electrodes 203G to 206G are labeled as 203G(211), 204G(212), 205G(213), and 206G(214).

[0040] Thus, in this embodiment, a first gate electrode and a second gate electrode for transferring the charge of the first photoelectric conversion unit to the first charge holding unit 211 and the second charge holding unit 212, respectively, are arranged on the first charge holding unit 211 and the second charge holding unit 212, respectively.

[0041] Furthermore, a third gate electrode and a fourth gate electrode, which transfer the charge of the second photoelectric conversion unit to the third charge holding unit 213 and the fourth charge holding unit 214, respectively, are positioned on the third charge holding unit 213 and the fourth charge holding unit 214.

[0042] Furthermore, in Figure 3, 301 is a first control line that supplies a common drive signal to the gate electrodes of transistors 203 (GSAL) and 205 (GSBL) for transferring charge to the first charge holding unit MEMAL 211 and the third charge holding unit MEMBL 213.

[0043] 302 is a second control line that supplies a common drive signal to the gate electrodes of transistors 204 (GSAS) and 206 (GSBS) for transferring charge to the second charge holding section MEMAS 212 and the fourth charge holding section MEMBS 214.

[0044] That is, the first control line connects the first gate electrode and the third gate electrode in common, and the second control line connects the second gate electrode and the fourth gate electrode in common.

[0045] 303 is a contact for connecting the first control line 301 or the second control line 302 to the gate electrodes of transistors 205-206, and 304 is a light-shielding film for blocking light leaking into the charge-holding section. 305 is a microlens for efficiently focusing the light incident on the pixel into the photoelectric conversion section.

[0046] Thus, in this embodiment, the photodiodes 201 and 202 receive light from different exit pupils of the photographic lens that are shifted horizontally or vertically via the microlens 305, and each generates a pupil division signal. Therefore, multiple photodiodes 201 and 202 can form two types of image signals (phase difference signals) that have a phase difference in the horizontal or vertical direction. Further details will be described later.

[0047] Figure 4 is a schematic diagram showing an example of the AA' cross-section in Figure 3. The metal wiring group 406 is used for supplying constant voltage and pulse signals, and for outputting pixel signals.

[0048] The control line passing over contact 303 is connected via contact 303 to the gate electrode of one of the transistors 203 (GSAL), 204 (GSAS), 205 (GSBL), or 206 (GSBS), which are made of polysilicon. In this embodiment, the light-shielding film 304 is made of aluminum, for example, but any material that does not transmit light of the wavelength at which photoelectric conversion occurs in the charge-holding portion under the gate electrode is acceptable, and is not limited to aluminum.

[0049] Furthermore, the charge holding portions under each gate electrode of the lower right pixel 105 in Figure 3 are arranged as follows, with the origin being the intersection point 308 of the first straight line 306 that divides the photoelectric conversion portion of the first pixel and the second straight line 307 that divides the photoelectric conversion portion of the second pixel.

[0050] Specifically, with the intersection point 308 as the origin, the gate electrode of transistor 206 (GSBS) and the fourth charge-holding portion 214 below it are located in the upper right first quadrant. In addition, the gate electrode of transistor 203 (GSAL) and the first charge-holding portion 211 below it are located in the upper left second quadrant.

[0051] Furthermore, the gate electrode of transistor 204 (GSAS) and the second charge-holding portion 212 below it are located in the third quadrant in the lower left, and the gate electrode of transistor 205 (GSBL) and the third charge-holding portion 213 below it are located in the fourth quadrant in the lower right.

[0052] Furthermore, the first control line 301 is wired such that when transistors 203 (GSAL) and 205 (GSBL) of a pixel in a certain column are turned on, transistors 203 (GSAL) and 205 (GSBL) of the pixels in the adjacent column of the same row are also turned on.

[0053] Furthermore, the second control line 302 is wired so that when transistors 204 (GSAS) and 206 (GSBS) of a pixel in a certain column are turned on, transistors 204 (GSAS) and 206 (GSBS) of the pixels in the adjacent column of the same row are also turned on.

[0054] Furthermore, as shown in Figure 3, the first charge-holding portion 211 and the third charge-holding portion 213 are arranged in the diagonal peripheral areas of each pixel, while the second charge-holding portion 212 and the fourth charge-holding portion 214 are arranged in the other diagonal peripheral areas of each pixel.

[0055] The first charge-holding portion 211 or third charge-holding portion 213 of a pixel in each column is arranged to be adjacent to the first charge-holding portion 211 or third charge-holding portion 213 of a pixel in an adjacent column, with the column boundary line in between.

[0056] With this arrangement and wiring, the charge holding portions under the gate electrodes of transistors that are simultaneously turned on to transfer charge from each divided photodiode can be placed adjacent to each other in both the first and second pixels of the same row. Therefore, the contacts 303 and their openings 401 for wiring the first and second control lines, respectively, can be shared between pixels in adjacent columns of the same row.

[0057] In other words, in this embodiment, the first gate electrode and the third gate electrode of adjacent pixels are connected to the first control line by a common contact provided in a common opening of the light-shielding film. Furthermore, the second gate electrode and the fourth gate electrode of adjacent pixels are connected to the second control line by another common contact provided in another common opening of the light-shielding film.

[0058] As a result, as shown in Figure 4, the apertures 401 of the light-shielding film 304 can be shared and combined into one for the charge-holding portions under the gate electrodes of transistors of adjacent pixels in the same row, allowing them to be positioned far from the light-receiving portion. Therefore, the amount of light leaking into the charge-holding portion through the aperture 401 can be minimized, enabling the coexistence of horizontally and vertically divided pixels without degrading PLS.

[0059] Furthermore, the two emission transistors 219 (OFGA) and 220 (OFGB) can be positioned adjacent to each divided photoelectric conversion section by arranging them on a side opposite to the division direction of the photoelectric conversion section relative to the center of the photoelectric conversion section.

[0060] Figure 5A is a timing chart showing an example of pixel accumulation operation according to the embodiment, and Figure 5B is a timing chart showing an example of readout operation after pixel accumulation according to the embodiment. The signals shown in the timing charts of Figures 5A and 5B are supplied from the timing generation unit 4 in Figure 7, which will be described later. In this embodiment, the drive timing of each control signal is represented as (t501).

[0061] In Figures 5A and 5B, φ represents the drive signal supplied to the gate electrode of the transistor corresponding to the following code number. For example, φGSAS corresponds to the signal applied to the gate electrode of transistor GSAS. The number in parentheses after the code number indicates the row number. That is, φRES(1) represents the drive signal supplied to the gate electrode of switch RES in the first row.

[0062] Furthermore, in this embodiment, each switch is an n-channel MOS transistor, and when the signal waveform of the drive signal in Figures 5A and 5B is high, it is in an ON (conducting) state, and when it is low, it is in an OFF (non-conducting) state. In addition, the operation of the pixels of the image sensor in this embodiment is divided into an accumulation period shown in Figure 5A and a subsequent output period shown in Figure 5B.

[0063] First, let's explain the accumulation operation shown in Figure 5A. Note that all drive signals other than the drive signal shown in Figure 5A remain unchanged. Furthermore, the accumulation operation shown in Figure 5A is performed simultaneously at all pixels of the image sensor, thereby enabling the acquisition of images captured at the same timing across all pixels.

[0064] First, at t501 to t502, the discharge transistors 219 (OFGA) and 220 (OFGB) are turned on. This discharges the remaining charge in the photoelectric conversion section (photodiodes 201 and 202) to the fixed potential lines connected to the discharge transistors 219 (OFGA) and 220 (OFGB).

[0065] From t502, the accumulation of signal charge converted photoelectrically by the photoelectric conversion unit (photodiodes 201 and 202) begins. Subsequently, at t503 to t504, transistors 204 (GSAS) and 206 (GSBS) are turned on.

[0066] As a result, the signal charges with a relatively short storage time Tint_S, which have been accumulated in the divided photoelectric conversion units (photodiodes 201 and 202), are transferred and held in the second charge holding unit 212 (MEMAS) and the fourth charge holding unit 214 (MEMBS), respectively.

[0067] Furthermore, in the above explanation, the time from time t502 when the emission transistors 219 (OFGA) and 220 (OFGB) are turned off to time t504 when transistors 204 (GSAS) and 206 (GSBS) are turned off is a relatively short storage time Tint_S.

[0068] Subsequently, at t505~t506, the discharge transistors 219 (OFGA) and 220 (OFGB) are turned on again to discharge any remaining charge in the photoelectric conversion section (photodiodes 201 and 202).

[0069] Next, from t506, the photoelectric conversion section (photodiodes 201 and 202) starts accumulating new signal charge. Also, transistors 203 (GSAL) and 205 (GSBL) are turned on from t507 to t508. Then, the signal charge accumulated in photodiodes 201 and 202 over a relatively long accumulation period Tint_L is transferred and held in the first charge holding section 211 (MEMAL) and the third charge holding section 213 (MEMBL), respectively.

[0070] In this embodiment, the operation of one cycle from t501 to t508 is repeated from t509 to t516. Therefore, the signal charges transferred to and held in the second charge holding unit 212 (MEMAS) and the fourth charge holding unit 214 (MEMBS) at t512 are signal charges corresponding to an accumulation time of (Tint_S) × 2.

[0071] Furthermore, at t516, the signal charges transferred to and held in the first charge holding unit 211 (MEMAL) and the third charge holding unit 213 (MEMBL), respectively, are signal charges corresponding to an accumulation time of (Tint_L) × 2.

[0072] Furthermore, the lengths of Tint_S and Tint_L are not limited to the example shown in Figure 5A, and may be in different ratios. In this embodiment, it is possible to acquire two types of signals with different exposure times (charge storage times) generated by photodiode 201 in one frame period. Also, it is possible to acquire two types of signals with different exposure times (charge storage times) generated by photodiode 202 in the same one frame period.

[0073] Figure 6 is a graph showing an example of the relationship between exposure amount and signal amount according to the embodiment, illustrating the relationship between illuminance (logarithmic) and signal level (logarithmic) for two types of signals with different accumulation times. By correcting the two types of signals obtained in this way according to the difference in accumulation time and then combining them, it is possible to obtain an image with a wide dynamic range.

[0074] Furthermore, the dynamic range can be varied by changing the ratio of the storage time Tint_S to Tint_L. Note that Figures 5A and 5B show two cycles of the operation consisting of storage and transfer, but the above cycle may be repeated three or more times within a single frame.

[0075] In this embodiment, the operation of transferring and holding signal charges with a relatively short storage time Tint_S and signal charges with a relatively long storage time Tint_L within one frame period is repeated for two or more cycles. This makes it possible to reduce the influence on the image from flashing light sources and fast-moving subjects.

[0076] Next, the readout operation of the signal charge transferred and held as described above will be explained using Figure 5B. Note that the drive signals other than those shown in Figure 5B will remain unchanged.

[0077] A selection signal (e.g., φSEL(1)) from the vertical selection circuit 102 is applied to the selection transistor 218(SEL). As a result, the selection transistor 218(SEL) is turned on during the readout period of the pixel group of the selected row (e.g., the first row), and the charge-voltage conversion unit 215(FD) of each pixel in the selected row is connected to the vertical signal line 106.

[0078] At this time, the amplification transistor 217(SF) within the pixel, together with a constant current source (not shown) connected to the vertical signal line 106, forms a source follower circuit, and the potential of the vertical signal line 106 becomes a potential corresponding to the potential of the charge-voltage conversion unit 215(FD) within the pixel. (V in Figure 5B) VLINE reference)

[0079] In this state, the reset transistor 216(RES) is turned on by φRES(1), thereby resetting the potential of the charge-voltage conversion unit 215(FD). Next, the reset transistor 216(RES) is turned off, and the reset level voltage VRES_S of the charge-voltage conversion unit 215(FD) is read out at t517. Here, the reset level voltage VRES_S corresponds to the noise component for a short storage time.

[0080] Next, φTXAS(1) turns on transistor 208 (TXAS), transferring the signal charge held in the second charge holding unit 212 (MEMAS) to the charge-voltage conversion unit 215 (FD). This causes a voltage change in the potential of the vertical signal line 106, corresponding to the amount of charge. At t518, the voltage level VA_S at that time is read out. Here, the voltage level VA_S corresponds to the signal charge with a short storage time in the photodiode 201.

[0081] Next, transistor 210 (TXBS) is turned on by φTXAB(1). This transfers the signal charge held in the fourth charge holding unit 214 (MEMBS) to the charge-voltage conversion unit 215 (FD), where it is added to the signal charge already present in the charge-voltage conversion unit 215 (FD). At t519, the voltage level VA+B_S is read out. Here, the voltage level VA+B_S corresponds to the signal charge obtained by adding the signal charges accumulated for short storage times in photodiodes 201 and 202.

[0082] After t519, reset transistor 216(RES) is turned on again by φRES(1) to reset the voltage of charge-to-voltage conversion unit 215(FD), and at t520, the voltage level VRES_L is read out. At this time, the voltage level VRES_L corresponds to the noise component for the long storage time.

[0083] Next, φTXAL(1) turns on transistor 207(TXAL), transferring the charge held in the first charge holding unit 211(MEMAL) to the charge-voltage conversion unit 215(FD), and t521 reads out the voltage level VA_L. Here, the voltage level VA_L corresponds to the signal charge with a long storage time in the photodiode 201.

[0084] Similarly, φTXBL(1) turns on transistor 209(TXBL). This transfers the charge held in the third charge holding unit 213(MEMBL) to the charge-voltage conversion unit 215(FD), where it is added to the signal charge already present in the charge-voltage conversion unit 215(FD), and the voltage level VA+B_L is read out at t522.

[0085] Here, the voltage level VA+B_L corresponds to the signal charge obtained by adding the signal charges accumulated over long storage times in photodiodes 201 and 202. Note that the operation up to t522 in Figure 5B is a read operation for the first row, and after time t522, the read operation is performed sequentially for all pixels by repeating this for the next row.

[0086] Based on the signal obtained in this way, the signal processing unit 7 in Figure 7, described later, calculates |VA_S - VRES_S| to obtain a signal component obtained by removing the noise component from the amount of charge held in the second charge holding unit 212 (MEMAS). Here, |VA_S - VRES_S| corresponds to the signal component obtained by removing the noise component from the signal charge with a short storage time in the photodiode 201.

[0087] Furthermore, by calculating |VA+B_S―VRES_S| in the signal processing unit 7, a signal component is obtained by removing the noise component from the signal obtained by adding the charge amounts held in the second charge holding unit 212 (MEMAS) and the fourth charge holding unit 214 (MEMBS). That is, |VA+B_S―VRES_S| corresponds to the signal component obtained by removing the noise component from the signal charge obtained by adding the signal charges with short storage times in photodiodes 201 and 202.

[0088] Similarly, by calculating |VA_L - VRES_L|, the signal component obtained by removing the noise component from the amount of charge held in the first charge holding unit 211 (MEMAL) is obtained. Here, |VA_L - VRES_L| corresponds to the signal component obtained by removing the noise component from the signal charge with a long storage time in the photodiode 201. Furthermore, by calculating |VA+B_L―VRES_L| in the signal processing unit 7, a signal component is obtained by removing the noise component from the signal obtained by adding the charge amounts held in the first charge holding unit 211 (MEMAL) and the third charge holding unit 213 (MEMBL). That is, |VA+B_L―VRES_L| corresponds to the signal component obtained by removing the noise component from the signal charge obtained by adding the signal charges of long storage times in photodiodes 201 and 202.

[0089] In the signal processing unit 7, by performing the above-described readout and calculation, it is possible to cancel out residual noise immediately after resetting the charge-voltage conversion unit 215 (FD), and offset variations caused by differences in the threshold voltage of the amplification transistor 217 for each pixel.

[0090] Furthermore, by sequentially repeating this operation for all rows, |VA_S―VRES_S|, |VA+B_S―VRES_S|, |VA_L―VRES_L|, and |VA+B_L―VRES_L| can be calculated for all pixels, and four types of images can be obtained.

[0091] In other words, the signal processing unit 7 can acquire a first image corresponding to the signal component of the short storage time in photodiode 201, and a second image corresponding to the signal component of the added charge obtained by adding the charges of the short storage times in photodiodes 201 and 202.

[0092] Furthermore, the signal processing unit 7 can acquire a third image corresponding to the signal component with a long storage time in photodiode 201, and a fourth image corresponding to the added charge obtained by adding the charges with long storage times in photodiodes 201 and 202. Note that noise has been removed from the first to fourth images.

[0093] According to this embodiment, the arrangement of the charge holding section and the exposure / readout operation described above make it possible to acquire images with relatively short storage times and images with relatively long storage times at the same timing in the photoelectric conversion section (photodiodes 201, 202) of all pixels.

[0094] Moreover, without compromising PLS degradation, the first to fourth images can be acquired from the first pixel having horizontally separated photodiodes and the second pixel having vertically separated photodiodes, respectively.

[0095] Furthermore, the signal processing unit 7 can generate a composite image signal for display with a wide dynamic range by, for example, combining a second image obtained from all pixels with a fourth image. The composite image signal obtained in this way can be displayed by the display unit 8 shown in Figure 7, which will be described later. Moreover, it can also be recorded by the recording unit 9 shown in Figure 7, which will be described later.

[0096] In other words, in the imaging device (imaging method) of this embodiment, a second image generated based on the charges held in the first charge holding unit and the third charge holding unit, respectively, and a fourth image generated based on the charges held in the second charge holding unit and the fourth charge holding unit, respectively, are combined. This combination process is realized by the CPU, which acts as the computer of the overall control and calculation unit shown in Figure 7, executing a computer program stored in memory.

[0097] On the other hand, the signal processing unit 7 can obtain a fifth image signal with a short storage time formed by the photodiode 202 group by subtracting the first image from the second image, for example. Furthermore, it can generate a sixth image signal with a long storage time formed by the photodiode 202 group by subtracting the third image from the fourth image.

[0098] Furthermore, since the first image and the fifth image have parallax (phase difference), the signal processing unit 7 can calculate the distance DS1 to the subject based on the phase difference between the first image and the fifth image. Similarly, since the third image and the sixth image also have parallax (phase difference), the signal processing unit 7 can calculate the distance DL1 to the subject based on the phase difference between the third image and the sixth image.

[0099] Furthermore, the signal processing unit 7 may calculate the average distance DAV1 to the subject by weighting and averaging distance DS1 and distance DL1, for example, based on their respective reliability. In this case, the weight of distance DL1 may be relatively increased for relatively dark subjects, and the weight of distance DS1 may be relatively increased for relatively bright subjects before combining them.

[0100] Furthermore, the distances corresponding to the above-mentioned distances DS1 and DL1 obtained from the first pixels obtained by horizontally dividing the photodiodes 201 and 202 may be defined as distance DSH1 and distance DLH1, respectively.

[0101] Alternatively, the distances corresponding to the above-mentioned distances DS1 and DL1 obtained from the second pixels obtained by vertically dividing the photodiodes 201 and 202 may be defined as distance DSV1 and distance DLV1, respectively.

[0102] Then, in the signal processing unit 7, the average distance DAV2 to the subject may be calculated by weighting and averaging distance DSV1 and distance DLV1, for example, based on their respective reliability. This allows for accurate distance calculation even for subjects with horizontal stripes. In addition, the weight of distance DLV1 may be relatively increased for relatively dark subjects, and the weight of distance DSV1 may be relatively increased for relatively bright subjects before combining them.

[0103] Alternatively, the average distance DAV1 obtained from multiple first pixels, which are horizontally divided in the photodiodes 201 and 202, and the average distance DAV2 obtained from multiple second pixels may be weighted and averaged based on their respective reliability. The final distance DVF may then be calculated.

[0104] In this case, the signal processing unit 7 may perform image recognition to determine the ratio of vertical and horizontal stripes in the subject, and if there are more vertical stripes, it may relatively increase the weight of DAV1, and if there are more horizontal stripes, it may relatively increase the weight of distance DAV2 and add them together.

[0105] Thus, in this embodiment, the signal processing unit 7 calculates the distance to the subject based on the charge photoelectrically converted during the first storage period and the charge photoelectrically converted during the second storage period in the first and second photoelectric conversion units, respectively.

[0106] Next, an example of an imaging device using the image sensor according to the above-described embodiment will be explained. Figure 7 is a schematic functional block diagram showing an example of the configuration of the imaging device according to the embodiment. Note that some of the functional blocks shown in Figure 7 are realized by having a CPU or other computer (not shown) included in the imaging device execute a computer program stored in memory (not shown) which is a storage medium.

[0107] However, some or all of these may be implemented in hardware. Hardware options include dedicated circuits (ASICs) and processors (reconfigurable processors, DSPs). Furthermore, each functional block shown in Figure 7 does not necessarily have to be housed in the same enclosure; they may be composed of separate devices connected to each other via signal paths.

[0108] As shown in Figure 7, the imaging device of this embodiment includes an image sensor 100, an overall control and calculation unit 2, an instruction unit 3, a timing generation unit 4, a shooting lens unit 5, a lens drive unit 6, a signal processing unit 7, a display unit 8, and a recording unit 9.

[0109] The imaging lens unit 5 forms an optical image of the subject onto the light-receiving surface of the image sensor 100. Although it is shown as a single lens in Figure 7, the imaging lens unit 5 may include multiple lenses, such as a focus lens and a zoom lens, as well as an aperture, and may be detachable from the main body of the imaging device or integrated into the main body.

[0110] The image sensor 100 has the configuration described in the embodiment above, and converts light incident on the light-receiving surface of the image sensor 100 via the imaging lens unit 5 into an electrical signal and outputs it.

[0111] The overall control and calculation unit 2 incorporates a CPU and memory that stores computer programs, and controls each part of the imaging device by executing the computer programs on the CPU.

[0112] Furthermore, the image signal processed by the signal processing unit 7 is used to perform the aforementioned calculations for focus detection. In addition, based on the image signal, calculations for exposure control related to aperture and storage time are performed, as well as predetermined signal processing such as development and compression to generate images for recording and display.

[0113] The lens drive unit 6 drives the shooting lens unit 5 and performs focus control, zoom control, aperture control, etc., on the shooting lens unit 5 according to the control signals from the overall control and calculation unit 2.

[0114] The instruction unit 3 receives inputs such as shooting execution instructions, imaging device drive mode settings, and various other settings and selections from an external source via user operation, and transmits them to the overall control / calculation unit 2. The timing generation unit 4 generates timing signals to drive the image sensor 100 and the signal processing unit 7 according to the control signals from the overall control / calculation unit 2.

[0115] The display unit 8 displays information such as preview images, playback images, and the drive mode of the imaging device. The recording unit 9 is equipped with a recording medium (not shown) on which the recording image signal is recorded. Examples of recording media include flash memory. The recording medium may be detachable from the recording unit 9 or may be built into the recording unit 9.

[0116] Next, referring to Figures 8 to 11, we will explain the calculation performed in the overall control and calculation unit 2 to calculate the defocus amount from the pupil division signal. Note that the calculation of the defocus amount from the horizontal phase difference signal and the calculation of the defocus amount from the vertical phase difference signal are fundamentally the same in principle, so we will explain the calculation from the horizontal phase difference signal.

[0117] Figure 8 shows an example of the correspondence between the pixels 105 of the image sensor according to this embodiment and the pupil intensity distribution, and Figure 9 is a schematic diagram showing an example of the pupil intensity distribution according to this embodiment. In Figure 8, an example of a lateral cross-section of a pixel where the division direction of the photodiodes 201 and 202 is the horizontal direction (x-axis direction), and the pupil surface of the imaging lens unit 5 is shown at a distance Ds away from the light-receiving surface 800 of the image sensor 100 in the negative z-axis (optical axis) direction.

[0118] The pupil surface of the imaging lens unit 5 and the light-receiving surface (second surface) of the image sensor are in a roughly conjugate relationship via the microlens 305. Therefore, the light beam that passes through the partial pupil region 801 is received by the roughly divided photodiode 201.

[0119] Furthermore, the light beam that passes through the partial pupil region 802 is generally received by the photodiode 202. The signal charge that is photoelectrically converted near the center of photodiodes 201 and 202 probabilistically moves into the potential well of one of the photodiodes.

[0120] Therefore, as the xp coordinate increases, the signal intensity of the pupil intensity distribution in photodiode 201 and photodiode 202 gradually switches at the boundary between partial pupil region 801 and partial pupil region 802. The xp direction dependence of the pupil intensity distribution takes on the shape illustrated in Figure 9. Here, the pupil intensity distribution corresponding to photodiode 201 is denoted as the first pupil intensity distribution 901, and the pupil intensity distribution corresponding to photodiode 202 is denoted as the second pupil intensity distribution 902.

[0121] Figure 10 illustrates an example of the correspondence between the image sensor and pupil intensity distribution according to the embodiment. As shown in Figure 10, in the image sensor 100 of this embodiment, the optical axis of the microlens 305 is arranged to be shifted from the center of the pixel according to the image height.

[0122] In other words, the optical axis of each microlens is positioned so that as the image height increases, it is offset from the center of each pixel towards the center of the light-receiving surface of the image sensor. Furthermore, if the imaging device has an image stabilization mechanism, the relationship between the center of the light-receiving surface of the image sensor and the optical axis of the imaging optical system changes slightly due to the driving of the imaging optical system or image sensor by the image stabilization mechanism, but this is negligible.

[0123] By configuring as shown in Figure 10, the first pupil intensity distribution 901 and the second pupil intensity distribution 902 in the photodiodes 201 and 202 of each pixel will be approximately the same, even if the image height of each pixel in the image sensor 100 is different.

[0124] Hereinafter, the first pupil intensity distribution 901 and the second pupil intensity distribution 902 will be referred to as the "sensor entrance pupil" of the image sensor 100, and the distance Ds will be referred to as the "sensor pupil distance" of the image sensor 100. It is not necessary for all pixels to have a single entrance pupil distance; for example, the entrance pupil distances of pixels up to 80% of the image height may be made approximately the same, or pixels may be configured to have different entrance pupil distances for each row or detection area.

[0125] Figure 11 illustrates an example of pupil division in the imaging optical system and image sensor according to the embodiment, showing the schematic relationship between the amount of image shift and the amount of defocus between disparity images. The light-receiving surface 800 corresponds to the light-receiving surface of the image sensor according to this embodiment, and similar to Figure 8, the exit pupil of the imaging optical system is divided into two parts: a partial pupil region 801 and a partial pupil region 802.

[0126] The amount of defocus d is defined as the distance from the imaging plane to the image source of the subject, with magnitude |d|. A negative sign (d<0) indicates a front focus state where the imaging source of the subject is on the subject side of the image source, and a positive sign (d>0) indicates a back focus state where the imaging source of the subject is on the opposite side of the image source.

[0127] The in-focus state, where the image of the subject is located on the imaging plane, is d=0. In Figure 11, subject 1101 shows an example of the in-focus state (d=0), and subject 1102 shows an example of the front-focus state (d<0). The front-focus state (d<0) and the back-focus state (d>0) together are called the defocus state (|d|>0).

[0128] In the front-focused state (d<0), the light beam from the subject 1102 that passes through the partial pupil region 801 is focused once, then spreads out with a width Γ1 (Γ2) centered on the centroid position G1 (G2) of the light beam, resulting in a blurred image on the light-receiving surface 800. The blurred image is received by photodiodes 201 and 202, and a disparity image (phase difference signal) is generated.

[0129] Therefore, the generated parallax image is formed with the subject 1102 blurred to a width Γ1 (Γ2) at the centroid position G1 (G2). The blur width Γ1 (Γ2) of the subject image increases roughly proportionally with the increase in the magnitude of the defocus amount d |d|.

[0130] Similarly, the magnitude of the image displacement p (=G2-G1) between the disparity images, |p|, increases roughly proportionally to the increase in the magnitude of the defocus amount d, |d|. The same principle applies in the back-focused state (d>0), although the direction of the image displacement between the disparity images is opposite to that of the front-focused state.

[0131] In the focused state (d=0), the centroid positions of the subject images between the disparity images coincide (p=0), and no image shift occurs. Therefore, in the two phase difference signals obtained using the signals from photodiode 201 and photodiode 202, as the amount of defocus in the disparity image increases, the amount of image shift in the x-direction between the two phase difference signals increases.

[0132] Based on this relationship, the amount of image displacement in the x-direction between disparity images is calculated by correlation calculation and converted into a defocus amount, thereby performing focus detection using a phase difference detection method.

[0133] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible in accordance with the spirit of the present invention, and these are not excluded from the scope of the present invention.

[0134] Furthermore, the present invention includes, for example, a system that realizes the functions of the above embodiment using at least one processor such as a CPU, memory, and circuitry (e.g., an ASIC). Alternatively, multiple processors may be used for distributed processing.

[0135] Furthermore, in order to implement some or all of the control in the above embodiment, a computer program that implements the functions of the above embodiment may be supplied to the imaging device, etc., via a network or various storage media.

[0136] The computer (or CPU or MPU, etc.) in the imaging device may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention. The present invention includes the following combinations.

[0137] (Configuration 1) An image sensor having a plurality of pixels arranged in a two-dimensional manner, each having a first photoelectric conversion unit and a second photoelectric conversion unit, wherein each pixel has a first charge holding unit that holds a charge photoelectrically converted for a first storage period in the first photoelectric conversion unit, a second charge holding unit that holds a charge photoelectrically converted for a second storage period different from the first storage period in the first photoelectric conversion unit, a third charge holding unit that holds a charge photoelectrically converted for a first storage period in the second photoelectric conversion unit, and a second charge holding unit that holds a charge photoelectrically converted for a second storage period different from the first storage period in the second photoelectric conversion unit An image sensor having a fourth charge-holding portion that holds a charge converted photoelectrically for a storage period of 2, wherein in the pixel, the first charge-holding portion and the third charge-holding portion are arranged in the diagonal peripheral portion of the pixel, and the second charge-holding portion and the fourth charge-holding portion are arranged in the other diagonal peripheral portion of the pixel, and the first charge-holding portion or the third charge-holding portion of the pixel in each row is arranged to be adjacent to the first charge-holding portion or the third charge-holding portion of the pixel in an adjacent row, with the row boundary line in between.

[0138] (Configuration 2) The image sensor according to Configuration 1, characterized in that the pixel includes a first pixel in which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged horizontally, and a second pixel in which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged vertically.

[0139] (Configuration 3) The image sensor according to Configuration 1 or 2, characterized in that a first gate electrode and a second gate electrode for transferring the charge of the first photoelectric conversion unit to the first charge holding unit and the second charge holding unit, respectively, are arranged on the first charge holding unit and the second charge holding unit, respectively, and a third gate electrode and a fourth gate electrode for transferring the charge of the second photoelectric conversion unit to the third charge holding unit and the fourth charge holding unit, respectively, are arranged on the third charge holding unit and the fourth charge holding unit, respectively.

[0140] (Configuration 4) An image sensor according to any one of Configurations 1 to 3, characterized by having a first control line that connects the first gate electrode and the third gate electrode in common, and a second control line that connects the second gate electrode and the fourth gate electrode in common.

[0141] (Configuration 5) The image sensor according to Configuration 4, characterized in that the first gate electrode and the third gate electrode of adjacent pixels are connected to the first control line by a common contact provided in a common opening of the light-shielding film, and the second gate electrode and the fourth gate electrode of adjacent pixels are connected to the second control line by another common contact provided in another common opening of the light-shielding film.

[0142] (Configuration 6) An imaging device using an image sensor described in any one of Configurations 1 to 5, characterized in that it has a signal processing unit that synthesizes a second image generated based on the charges held in the first charge holding unit and the third charge holding unit, respectively, and a fourth image generated based on the charges held in the second charge holding unit and the fourth charge holding unit, respectively.

[0143] (Configuration 7) The imaging apparatus according to Configuration 6, characterized in that the signal processing unit calculates the distance to the subject based on the charge photoelectrically converted during the first storage period and the charge photoelectrically converted during the second storage period in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively.

[0144] (Method) An imaging method using an image sensor, wherein the image sensor has a plurality of pixels arranged in two dimensions, each having a first photoelectric conversion unit and a second photoelectric conversion unit, and each pixel has a first charge holding unit and a third charge holding unit that each hold the charge photoelectrically converted for a first storage period in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively, and a second charge holding unit and a fourth charge holding unit that each hold the charge photoelectrically converted for a second storage period different from the first storage period in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively, and in the pixel, the first charge holding unit and the third charge holding unit are located diagonally to each pixel. An imaging method characterized by compositing a second image generated based on the charges held in the first and third charge holding parts, respectively, and a fourth image generated based on the charges held in the second and fourth charge holding parts, respectively, and combining a second image generated based on the charges held in the first and fourth charge holding parts, respectively, and a fourth image generated based on the charges held in the second and fourth charge holding parts, respectively.

[0145] (Program) A computer program for executing the above imaging method using a computer. [Explanation of symbols]

[0146] 100: Image sensor 105: Pixels 203~206: Transistors 211: First charge holding part 212: Second charge retention section 213: Third charge retention section 214: Fourth charge retention section 301: First control line 302: Second control line

Claims

1. An image sensor having a plurality of pixels arranged in a two-dimensional manner, each having a first photoelectric conversion unit and a second photoelectric conversion unit, The pixel has a first charge holding portion that holds charge photoelectrically converted for a first storage period in the first photoelectric conversion portion, a second charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the first photoelectric conversion portion, a third charge holding portion that holds charge photoelectrically converted for a first storage period in the second photoelectric conversion portion, and a fourth charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the second photoelectric conversion portion. In the aforementioned pixel, the first charge-holding portion and the third charge-holding portion are arranged in the diagonal peripheral area of ​​the pixel, and the second charge-holding portion and the fourth charge-holding portion are arranged in the other diagonal peripheral area of ​​the pixel. An image sensor characterized in that the first charge-holding portion or the third charge-holding portion of the pixel in each row is arranged adjacent to the first charge-holding portion or the third charge-holding portion of the pixel in an adjacent row, with a boundary line in the row direction in between.

2. The image sensor according to claim 1, characterized in that the pixel includes a first pixel in which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged horizontally side by side, and a second pixel in which the first photoelectric conversion unit and the second photoelectric conversion unit are arranged vertically side by side.

3. A first gate electrode and a second gate electrode for transferring the charge of the first photoelectric conversion unit to the first charge holding unit and the second charge holding unit, respectively, are arranged on the first charge holding unit and the second charge holding unit, respectively. The image sensor according to claim 1, characterized in that a third gate electrode and a fourth gate electrode for transferring the charge of the second photoelectric conversion unit to the third charge holding unit and the fourth charge holding unit, respectively, are arranged on the third charge holding unit and the fourth charge holding unit, respectively.

4. A first control line that connects the first gate electrode and the third gate electrode in common, The image sensor according to claim 3, further comprising a second control line that connects the second gate electrode and the fourth gate electrode in common.

5. The first gate electrode and the third gate electrode of adjacent pixels are connected to the first control line by a common contact provided in a common aperture of the light-shielding film. The image sensor according to claim 4, characterized in that the second gate electrode and the fourth gate electrode of adjacent pixels are connected to the second control line by another common contact provided in another common aperture of the light-shielding film.

6. An imaging device using an image sensor according to any one of claims 1 to 5, characterized in that it has a signal processing unit that synthesizes a second image generated based on the charges held in the first charge holding unit and the third charge holding unit, respectively, and a fourth image generated based on the charges held in the second charge holding unit and the fourth charge holding unit, respectively.

7. The imaging apparatus according to claim 6, characterized in that the signal processing unit calculates the distance to the subject based on the charge photoelectrically converted during the first storage period and the charge photoelectrically converted during the second storage period in the first photoelectric conversion unit and the second photoelectric conversion unit, respectively.

8. An imaging method using an image sensor, The image sensor has a plurality of pixels arranged in a two-dimensional manner, each having a first photoelectric conversion unit and a second photoelectric conversion unit. The pixel has a first charge holding portion that holds charge photoelectrically converted for a first storage period in the first photoelectric conversion portion, a second charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the first photoelectric conversion portion, a third charge holding portion that holds charge photoelectrically converted for a first storage period in the second photoelectric conversion portion, and a fourth charge holding portion that holds charge photoelectrically converted for a second storage period different from the first storage period in the second photoelectric conversion portion. In the aforementioned pixel, the first charge-holding portion and the third charge-holding portion are arranged in the diagonal peripheral area of ​​the pixel, and the second charge-holding portion and the fourth charge-holding portion are arranged in the other diagonal peripheral area of ​​the pixel. The first charge-holding portion or the third charge-holding portion of the pixel in each column is arranged to be adjacent to the first charge-holding portion or the third charge-holding portion of the pixel in an adjacent column, with the column boundary line in between. An imaging method characterized by combining a second image generated based on the charges held in the first charge-holding unit and the third charge-holding unit, respectively, with a fourth image generated based on the charges held in the second charge-holding unit and the fourth charge-holding unit, respectively.

9. A computer program for executing the imaging method described in claim 8 using a computer.

Images