Imaging device, electronic apparatus, and method for manufacturing imaging device

Abstract

An imaging device according to one embodiment of the present disclosure comprises a pixel array unit having a plurality of pixels each including a color filter, wherein the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color, the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the refractive indices of the first lens region and the second lens region are different.

Description

Imaging device, electronic device, and method of manufacturing imaging device

[0001] The present disclosure relates to an imaging device, an electronic device, and a method for manufacturing an imaging device.

[0002] Some imaging devices obtain image plane phase difference to achieve autofocus, for example. For example, Patent Document 1 discloses an imaging device in which one lens is provided at a position corresponding to a pixel pair including two light-receiving pixels. Deca-Octa Bayer Coding (DOBC), which is one type of pixel arrangement for imaging devices, differs from conventional Quad Bayer Coding (QBC) in that, as in Patent Document 1, a pixel block including 10 green pixels, a pixel block including 8 red pixels, and a pixel block including 8 blue pixels are repeatedly arranged.

[0003] International Publication No. 2022 / 130888

[0004] However, in an array such as the DOBC, the difference in refractive index between color filters makes the sensitivity difference between pixels of the same color, which is larger than that of conventional arrays such as QBC, even among pixels using color filters of the same color, larger than that of conventional arrays such as QBC.This causes color differences even among pixels of the same color, resulting in a decrease in image quality.

[0005] Therefore, the present disclosure proposes an imaging device, an electronic device, and a method for manufacturing an imaging device that can achieve improved image quality.

[0006] An imaging device according to one embodiment of the present disclosure includes a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices.

[0007] An electronic device according to one embodiment of the present disclosure includes an imaging device, the imaging device including a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section including a lens provided in common to the first pixel and the second pixel, the lens including a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices.

[0008] A manufacturing method for an imaging device according to one embodiment of the present disclosure includes forming a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices.

[0009] 1 is a diagram illustrating an example configuration of an imaging device according to an embodiment of the present disclosure; FIG. 2 is a plan view illustrating an example configuration of a pixel array unit according to an embodiment of the present disclosure; FIG. 3 is a cross-sectional view illustrating an example cross-sectional structure of a pixel array unit according to an embodiment of the present disclosure; FIG. 4 is a cross-sectional view illustrating an example cross-sectional structure of a pixel array unit according to a comparative example of an embodiment of the present disclosure; FIG. 5 is a diagram illustrating a sensitivity difference between same colors in a pixel array unit according to an embodiment of the present disclosure; FIG. 6 is a diagram illustrating a pixel array of 1×1 QBC; FIG. 7 is a diagram illustrating a pixel array of 2×2 QBC; FIG. 8 is a diagram illustrating a pixel array of DOBC; FIG. 9 is a diagram illustrating a sensitivity difference for each light receiving pixel in a pixel array unit according to an embodiment of the present disclosure; FIG. 10 is a plan view illustrating an example arrangement of lenses in a pixel array unit according to an embodiment of the present disclosure; FIG. 11 is a diagram illustrating an example configuration of a green pixel block according to an embodiment of the present disclosure; FIG. 12 is a diagram illustrating an example configuration of a red pixel block according to an embodiment of the present disclosure; FIG. 13 is a diagram illustrating an example connection of a plurality of pixel blocks according to an embodiment of the present disclosure; FIG. 14 is a diagram illustrating an example configuration of a readout unit according to an embodiment of the present disclosure; FIG. 15 is a diagram illustrating an example image signal according to an embodiment of the present disclosure; FIG. 16 is a diagram illustrating an example number of effective pixels in an imaging device according to an embodiment of the present disclosure; FIG. 17 is a diagram illustrating an example of zoom operation in an imaging device according to an embodiment of the present disclosure; FIG. 18 is a diagram illustrating an example operation of an imaging device in a first imaging mode according to an embodiment of the present disclosure. FIG. 1 is a diagram illustrating an example of a readout operation in an imaging device according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of image processing by a signal processing unit in a first imaging mode according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of an operation of an imaging device in a second imaging mode according to an embodiment of the present disclosure. FIG. 4 is a diagram illustrating more specifically an example of an operation of an imaging device in the second imaging mode according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of image processing by a signal processing unit in the second imaging mode according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example of re-mosaic processing in the second imaging mode according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating re-mosaic processing for a first pixel block in the second imaging mode according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating re-mosaic processing for a second pixel block and a third pixel block in the second imaging mode according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating re-mosaic processing for a fourth pixel block in the second imaging mode according to an embodiment of the present disclosure.10 is a diagram illustrating an example of operation of the imaging device in a third imaging mode according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating more specifically an example of operation of the imaging device in the third imaging mode according to an embodiment of the present disclosure. FIG. 12 is a diagram illustrating an example of another readout operation in the imaging device according to an embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of image processing by a signal processing unit in the third imaging mode according to an embodiment of the present disclosure. FIG. 14 is a diagram illustrating an example of re-mosaic processing in the third imaging mode according to an embodiment of the present disclosure. FIG. 15 is a diagram illustrating re-mosaic processing for a first pixel block in the third imaging mode according to an embodiment of the present disclosure. FIG. 16 is a diagram illustrating re-mosaic processing for a second pixel block and a second pixel block in the third imaging mode according to an embodiment of the present disclosure. FIG. 17 is a diagram illustrating re-mosaic processing for a fourth pixel block in the third imaging mode according to an embodiment of the present disclosure. FIG. 18 is a plan view illustrating the refractive index of each lens in a pixel array unit according to a first modification of an embodiment of the present disclosure. FIG. 19 is a plan view illustrating the refractive index of each lens in a pixel array unit according to a second modification of an embodiment of the present disclosure. FIG. 19 is a plan view illustrating the refractive index of each lens in a pixel array unit according to a third modification of an embodiment of the present disclosure. FIG. 19 is a diagram illustrating an example cross-sectional structure of a pixel array unit according to a third modification of an embodiment of the present disclosure. FIG. 19 is a plan view illustrating the refractive index of each lens in a pixel array unit according to a fourth modification of an embodiment of the present disclosure. Fig. 10 is a plan view showing the position of a desired lens in a pixel array unit according to Modification 5 of an embodiment of the present disclosure. Fig. 11 is a diagram for explaining the refractive index of a desired lens according to Modification 5 of an embodiment of the present disclosure. Fig. 12 is a diagram showing an application example using an imaging device according to an embodiment of the present disclosure. Fig. 13 is a diagram showing an example configuration of an imaging device according to an application example. Fig. 14 is a diagram showing an example configuration of a distance measuring device according to an application example.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the embodiments do not limit the devices, equipment, methods, etc. according to the present disclosure. Furthermore, in the following embodiments, essentially the same components are designated by the same reference numerals, and redundant explanations will be omitted. The following embodiments also include examples and modified examples.

[0011] One or more of the following embodiments can be implemented independently. However, at least a portion of the following embodiments may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from one another. Therefore, each embodiment may contribute to solving different purposes or problems, and may provide different effects. The effects of each embodiment are merely examples and are not intended to be limiting, and other effects may also be provided.

[0012] The present disclosure will be described in the following order: 1. Embodiment 1-1. Configuration example of imaging device 1-2. Configuration example of pixel array section 1-3. Configuration example of pixel block 1-4. Configuration example of readout section 1-5. Operation example of imaging device 1-6. Imaging mode MA (first imaging mode) 1-7. Imaging mode MB (second imaging mode) 1-8. Imaging mode MC (third imaging mode) 2. Modifications 2-1. Modification 1 2-2. Modification 2 2-3. Modification 3 2-4. Modification 4 2-5. Modification 5 3. Actions and effects 4. Other embodiments 5. Application examples 5-1. Imaging device 5-2. Distance measuring device 6. Supplementary notes

[0013] 1. Embodiment 1-1. Configuration Example of Imaging Apparatus An example of the configuration of an imaging apparatus 1 according to this embodiment will be described with reference to FIG.

[0014] FIG. 1 is a diagram showing an example of the configuration of an imaging device 1 according to this embodiment.

[0015] 1, the imaging device 1 includes a pixel array unit 11, a drive unit 12, a readout unit 20, a signal processing unit 15, and an imaging control unit 18. The imaging device 1 is, for example, a solid-state imaging device that generates image data of a subject.

[0016] The pixel array unit 11 has a plurality of light-receiving pixels (pixels) P. These light-receiving pixels P are arranged, for example, in a matrix. Each light-receiving pixel P is configured to generate a signal SIG including a pixel voltage Vpix corresponding to the amount of light received.

[0017] The drive unit 12 is configured to drive a plurality of light-receiving pixels P in the pixel array unit 11 based on instructions from the imaging control unit 18. A plurality of control lines TRGL, RSTL, and SELL are provided in the pixel array unit 11. Specifically, the drive unit 12 drives the plurality of light-receiving pixels P in the pixel array unit 11 by supplying a plurality of control signals STRG to the plurality of control lines TRGL, a plurality of control signals SRST to the plurality of control lines RSTL, and a plurality of control signals SSEL to the plurality of control lines SELL (this will be described in detail later).

[0018] The reference signal generation unit 13 is configured to generate a reference signal RAMP based on instructions from the imaging control unit 18. The reference signal RAMP has a so-called ramp waveform in which the voltage level gradually changes over time during the periods (P-phase period TP and D-phase period TD) when the readout unit 20 performs AD conversion. The reference signal generation unit 13 supplies this reference signal RAMP to the readout unit 20.

[0019] The readout unit 20 is configured to generate an image signal Spic0 by performing AD conversion based on the signal SIG supplied from the pixel array unit 11 via the signal line VSL, based on instructions from the imaging control unit 18 (details will be described later).

[0020] The signal processing unit 15 is configured to generate an image signal Spic by performing predetermined signal processing based on the image signal Spic0 and instructions from the imaging control unit 18. The signal processing unit 15 has an image data generation unit 16 and a phase difference data generation unit 17. The image data generation unit 16 is configured to generate image data DP indicative of a captured image by performing predetermined image processing based on the image signal Spic0. The phase difference data generation unit 17 is configured to generate phase difference data DF indicative of an image plane phase difference by performing predetermined image processing based on the image signal Spic0. The signal processing unit 15 generates an image signal Spic including the image data DP generated by the image data generation unit 16 and the phase difference data DF generated by the phase difference data generation unit 17.

[0021] The imaging control unit 18 is configured to supply control signals to the drive unit 12, the reference signal generation unit 13, the readout unit 20, and the signal processing unit 15, and to control the operation of these circuits, thereby controlling the operation of the imaging device 1. A control signal Sctl is supplied to the imaging control unit 18 from an external device. This control signal Sctl includes, for example, information about the zoom magnification of so-called electronic zoom. The imaging control unit 18 is configured to control the operation of the imaging device 1 based on the control signal Sctl.

[0022] <1-2. Configuration Example of Pixel Array Unit> A configuration example of the pixel array unit 11 according to this embodiment will be described with reference to FIGS. 2 to 11. FIG.

[0023] Fig. 2 is a plan view showing an example of the configuration of the pixel array section 11 according to this embodiment. Fig. 3 is a cross-sectional view showing an example of the cross-sectional structure of the pixel array section 11 according to this embodiment. Fig. 3 shows a cross section corresponding to the Y1-Y1 line shown in Fig. 2.

[0024] 2, the pixel array section 11 has a plurality of pixel units U and a plurality of lenses 101. In the example of Fig. 2, one pixel unit U is clearly shown, but in reality there are a plurality of pixel units U.

[0025] Each pixel unit U includes a pixel block 100R, a pixel block 100Gr, a pixel block 100Gb, and a pixel block 100B. In the pixel array section 11, the plurality of light-receiving pixels P are arranged with four pixel blocks 100R, 100Gr, 100Gb, and 100B as a minimum repeating unit (pixel unit U). When there is no need to distinguish between the four pixel blocks 100R, 100Gr, 100Gb, and 100B, they may be simply referred to as pixel blocks 100.

[0026] The pixel block 100R has, for example, eight light-receiving pixels P (light-receiving pixels PR) that include red (R) color filters 131 (see FIG. 3 ). The pixel block 100Gr has, for example, ten light-receiving pixels P (light-receiving pixels PGr) that include green (G) color filters 131. The pixel block 100Gb has, for example, ten light-receiving pixels P (light-receiving pixels PGb) that include green (G) color filters 131. The pixel block 100B has, for example, eight light-receiving pixels P (light-receiving pixels PB) that include blue (B) color filters 131. Note that when there is no need to distinguish between the four light-receiving pixels PR, PGr, PGb, and PB, they may be simply referred to as light-receiving pixels P.

[0027] In the example of FIG. 2 , differences in the colors of the color filters 131 are represented by hatching. The arrangement pattern of the light-receiving pixels PR in the pixel block 100R and the arrangement pattern of the light-receiving pixels PB in the pixel block 100B are the same. The arrangement pattern of the light-receiving pixels PGr in the pixel block 100Gr and the arrangement pattern of the light-receiving pixels PGb in the pixel block 100Gb are the same. In the pixel block 100R or the pixel block 100B, two, four, and two light-receiving pixels PR or PB arranged in parallel in the X-axis direction are arranged in a cross-shaped pattern, with the pixels arranged in parallel in the Y-axis direction. Furthermore, in the pixel block 100Gr or the pixel block 100Gb, four, two, and four light-receiving pixels PGr or PGb arranged in parallel in the X-axis direction are arranged in an H-shaped pattern, with the pixels arranged in parallel in the Y-axis direction.

[0028] In the pixel unit U, the pixel blocks 100R, 100Gr, 100Gb, and 100B are arranged in, for example, two rows and two columns. Specifically, the pixel blocks 100R and 100B, and the pixel blocks 100Gr and 100Gb, which have the same arrangement pattern, are arranged on diagonal lines that intersect with each other. As an example, the pixel block 100Gr is arranged in the upper left, the pixel block 100R is arranged in the upper right, the pixel block 100B is arranged in the lower left, and the pixel block 100Gb is arranged in the lower right. In this way, the pixel blocks 100R, 100Gr, 100Gb, and 100B are arranged in a so-called Bayer array, with the pixel block 100 serving as a unit. This array is called, for example, DOBC (Deca-Octa Bayer Coding).

[0029] As shown in FIG. 3, the pixel array section 11 includes a semiconductor substrate 111 , a light receiving section 112 , a separation section 113 , a multilayer wiring layer 121 , a color filter 131 , and a light blocking film 132 .

[0030] The semiconductor substrate 111 is a support substrate on which the imaging device 1 is formed. The semiconductor substrate 111 is, for example, a P-type semiconductor substrate, and has a pair of opposing surfaces (a front surface 111S1 and a back surface 111S2).

[0031] The light receiving units 112 are semiconductor regions embedded in the semiconductor substrate 111 at positions corresponding to the plurality of light receiving pixels P. The semiconductor regions are doped with, for example, N-type impurities to form photodiodes (PDs).

[0032] The isolation portion 113 is provided at the boundary between adjacent light-receiving pixels P in the XY plane within the semiconductor substrate 111. In the example of Fig. 3, the isolation portion 113 is a deep trench isolation (DTI) formed using an insulating material such as an oxide film, but may have another structure.

[0033] The multilayer wiring layer 121 is provided on the surface 111S1 of the semiconductor substrate 111, which is the surface opposite to the light incident side S of the pixel array unit 11. The multilayer wiring layer 121 includes, for example, a plurality of wiring layers 122 to 127 and an interlayer insulating layer 128. The wiring layers 122 to 127 are provided in order from the back surface (light incident surface) 111S2 side of the semiconductor substrate 111. Each of the wiring layers 122 to 127 is provided with a plurality of wirings, such as control lines, that connect a plurality of transistors provided on the surface 111S1 of the semiconductor substrate 111 to the driving unit 12, the readout unit 20, etc.

[0034] The color filter 131 is provided on the back surface (light incident surface) 111S2 of the semiconductor substrate 111, which is the light incident side S of the pixel array section 11. The color filter 131 is any one of red (R), green (G), and blue (B) color filters. The color filter 131 may be provided for each light receiving pixel P, or may be provided for each pair of light receiving pixels P adjacent to each other in the X-axis direction (pixel pair 90). In either case, however, each light receiving pixel P (e.g., light receiving pixel PR, light receiving pixel PGr, light receiving pixel PGb, and light receiving pixel PB) will have a color filter 131.

[0035] The light-shielding film 132 is provided on the rear surface (light incident surface) 111S2 of the semiconductor substrate 111, for example, for every two light-receiving pixels P (pixel pair 90) adjacent in the X-axis direction so as to surround the pixel pair 90. The two light-receiving pixels P adjacent in the X-axis direction function as the pixel pair 90 (see FIG. 2).

[0036] Each lens 101 is, for example, an on-chip lens, and is provided on a color filter 131 on the light incident side S of the pixel array unit 11. The lenses 101 are provided above two light-receiving pixels P (pixel pairs 90) adjacent to each other in the X-axis direction.

[0037] As shown in FIG. 2 , four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100R. Five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gr. Five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gb. Four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100B. The lenses 101 are arranged side by side in the X-axis direction and the Y-axis direction. The lenses 101 arranged in the Y-axis direction are shifted by one light receiving pixel P in the X-axis direction. In other words, the pixel pairs 90 arranged in the Y-axis direction are shifted by one light receiving pixel P in the X-axis direction.

[0038] With this configuration, the images of the two light-receiving pixels P in a pixel pair 90 corresponding to one lens 101 are shifted from each other. The imaging device 1 generates phase difference data DF based on the so-called image plane phase difference detected by the multiple pixel pairs 90. For example, a camera equipped with the imaging device 1 determines the amount of defocus based on this phase difference data DF, and moves the position of the photographing lens based on this defocus amount. In this way, the camera is able to achieve autofocus.

[0039] As shown in FIG. 3 , the lens 101 in the specific pixel pair 90 has a lens region 101a and a lens region 101b. The specific pixel pair 90 is, for example, a pixel pair having a sensitivity difference between the same colors. In the example of FIG. 3 , the specific pixel pair 90 is a pair of light-receiving pixels PGr. The lens region 101a is a first lens region corresponding to one light-receiving pixel PGr of the pixel pair 90. This lens region 101a faces one light-receiving pixel PGr of the pixel pair 90. The lens region 101b is a second lens region corresponding to the other light-receiving pixel PGr of the pixel pair 90. This lens region 101b faces the other light-receiving pixel PGr of the pixel pair 90.

[0040] The lens regions 101a and 101b have different refractive indices. In the example of Fig. 3, in a pair of light receiving pixels PGr, the lens regions 101a and 101b have different refractive indices, with the refractive index of the lens region 101a being higher than the refractive index of the lens region 101b. The refractive indices of the lens regions 101a and 101b are set so as to suppress the difference in sensitivity between the same colors.

[0041] The lens 101 is formed from a plurality of materials with different refractive indices (e.g., high-refractive index material, low-refractive index material, etc.) so that the lens region 101a and the lens region 101b have different refractive indices. The plurality of materials may include materials with different composition ratios. Examples of materials with different composition ratios include SiO2 and Si3N4, with SiOxNy filling the gaps. The composition ratios of the materials are determined, for example, according to the colors (refractive indexes) of the color filters 131 of the surrounding light-receiving pixels P surrounding the target light-receiving pixel P.

[0042] Comparative Example FIG. 4 is a cross-sectional view showing an example of the cross-sectional structure of the pixel array section 11 according to a comparative example of the present embodiment.

[0043] As shown in FIG. 4 , in the comparative example, the refractive index of each lens 101 is the same, and there is no difference in the refractive index within a single lens 101. The refractive index of each color filter 131 is, for example, R / G / B = 1.72 / 1.67 / 1.49. Normally, light is attracted to the color filter 131 with a higher refractive index. For example, in the light-receiving pixel PGr to which light L2 in FIG. 4 is incident, light L2 incident on the color filter 131 travels straight ahead. However, in the light-receiving pixel PGr to which light L1 in FIG. 4 is incident, light L1 incident on the color filter 131 is attracted to the right side and leaks toward the adjacent light-receiving pixel PR. In this case, the light-receiving pixel PGr to which light L1 in FIG. 4 is incident has lower sensitivity than the light-receiving pixel PGr to which light L2 in FIG. 4 is incident.

[0044] To suppress this decrease in sensitivity, as described above, the refractive index of the lens region 101a is made higher than the refractive index of the lens region 101b. In the example of FIG. 3, in the light-receiving pixel PGr into which the light L1 in FIG. 3 is incident, the light L1 incident on the lens 101 bends to the left and then enters the color filter 131. The light L1 then enters the color filter 131 and is shifted to the right, but leakage toward the adjacent light-receiving pixel PR is suppressed. Therefore, by making the refractive index of the lens region 101a higher than that of the lens region 101b, it is possible to suppress light leakage toward the adjacent light-receiving pixel PR. In other words, by making the refractive index of the lens region 101a higher than that of the lens region 101b, it is possible to suppress light from traveling toward the adjacent color filter 131.

[0045] Since it is usually difficult to make the refractive index of each color filter 131 uniform, it is desirable to correct the sensitivity using the total refractive index of the lens 101 and the color filter 131. Therefore, by adjusting the refractive index of each of the lens region 101a and the lens region 101b, the sensitivity can be corrected accurately, thereby achieving improved image quality.

[0046] (Same-Color Sensitivity Difference) Here, the same-color sensitivity difference due to the influence of the eight surrounding pixels (light-receiving pixels P) surrounding the subject pixel (light-receiving pixel P) will be described with reference to Figures 5 to 8. For example, the same-color sensitivity difference is a sensitivity difference that occurs depending on the individual colors (refractive indexes of the color filters 131) of the multiple light-receiving pixels P that exist in the vicinity surrounding the subject pixel.

[0047] FIG. 5 is a diagram for explaining the sensitivity difference between the same colors in the pixel array section 11 according to this embodiment.

[0048] The photosensitive pixel PGr designated A1 in FIG. 5 receives light from two photosensitive pixels PB and loses light to one photosensitive pixel PR. This is due to the relationship in refractive index between the photosensitive pixels PR, PB, and PGr (R / G / B = 1.72 / 1.67 / 1.49), and the same applies below. The photosensitive pixel PGr designated A2 in FIG. 5 receives light from one photosensitive pixel PB and loses light to three photosensitive pixels PR, resulting in lower sensitivity and a difference in sensitivity between same colors compared to the photosensitive pixel PGr designated A1 in FIG. 5. For this reason, the refractive index of the lens 101 described above is adjusted for the photosensitive pixels PGr designated A1 and A2 in FIG. 5. That is, the refractive index of the lens region 101a corresponding to the photosensitive pixel PGr designated A2 in FIG. 5 is made higher than the refractive index of the lens region 101b corresponding to the photosensitive pixel PGr designated A1 in FIG. 5.

[0049] 5, the number of red, green, and blue (R / G / B) pixels surrounding the A3 and A4 light receiving pixels PB is the same, and because the refractive index of the green color filter 131 is higher than that of the blue color filter 131, light is attracted to the green side (reduced sensitivity), but no difference in sensitivity between the same colors occurs. For this reason, the refractive index of the lens 101 described above is not adjusted for the A3 and A4 light receiving pixels PB in FIG.

[0050] In this way, the refractive index of one of the lens regions 101a and 101b is made higher than that of the other depending on the color of the light-receiving pixel P (i.e., the refractive index of the color filter 131) adjacent to the light-receiving pixel P, and the lens 101 is provided with the lens regions 101a and 101b having different refractive indices. This makes it possible to correct in advance by the lens 101 the bending of light due to the refractive index difference of the color filter 131 and perform sensitivity correction, thereby suppressing the difference in sensitivity between the same colors.

[0051] FIG. 6 is a diagram showing a pixel array of a 1×1 QBC. As shown in FIG. 6, when a subject pixel (photosensitive pixel P) is R, the number of surrounding eight pixels (photosensitive pixels P) surrounding the subject pixel is R=0, G=4, and B=4. When a subject pixel is G, the number of surrounding eight pixels surrounding the subject pixel is R=2, G=2, and B=4. When a subject pixel is B, the number of surrounding eight pixels surrounding the subject pixel is R=4, G=4, and B=0. In the example of FIG. 6, the number of colors of the surrounding eight pixels surrounding the subject pixel is the same for all the same colors. Therefore, the sensitivity differences between the same colors resulting from the surrounding pixels are qualitatively the same.

[0052] FIG. 7 is a diagram showing a pixel array of a 2×2 QBC. As shown in FIG. 7, when a subject pixel (photosensitive pixel P) is R, the number of surrounding eight pixels (photosensitive pixels P) surrounding the subject pixel is R=3, G=4, and B=1. When the subject pixel is G, the number of surrounding eight pixels surrounding the subject pixel is R=2, G=4, and B=2. When the subject pixel is B, the number of surrounding eight pixels surrounding the subject pixel is R=1, G=4, and B=3. In the example of FIG. 7, the number of colors of the surrounding eight pixels surrounding the subject pixel is the same for all the same colors. Therefore, the sensitivity differences between the same colors resulting from the surrounding pixels are qualitatively the same.

[0053] 8 is a diagram showing a pixel arrangement of DOBC. As shown in FIG. 8, when a pixel (light-receiving pixel P) is R, G, or B, the number of each color among the eight surrounding pixels (light-receiving pixels P) is not constant but varies depending on the pixel. In other words, in the case of DOBC, even pixels of the same RGB color have different numbers of each color among the eight surrounding pixels, resulting in a difference in sensitivity among the same colors due to the arrangement. For this reason, the refractive index of the lens 101 described above is adjusted for the pixel arrangement of DOBC.

[0054] Fig. 9 is a diagram showing the sensitivity difference for each light-receiving pixel P in the pixel array unit 11 according to this embodiment. Fig. 10 is a plan view showing an example of the arrangement of the lenses 101 in the pixel array unit 11 according to this embodiment. In the example of Fig. 9, the sensitivity difference is shown in %.

[0055] 9 , in the pixel array of DOBC, there is a difference in sensitivity between each light-receiving pixel P. Also, there are pairs of light-receiving pixels P (pixel pairs 90) with the same sensitivity difference, and pairs of light-receiving pixels P (pixel pairs 90) with different sensitivity differences. Lenses 101 are provided depending on whether these sensitivity differences match or mismatch.

[0056] For example, as shown in Fig. 10, lenses 101 with no refractive index difference are provided for a pair of light-receiving pixels P (pixel pair 90) with the same sensitivity difference. On the other hand, as shown in Fig. 9, lenses 101 with a refractive index difference (lens 101 having lens regions 101a and 101b) are provided for a pair of light-receiving pixels P (pixel pair 90) with different sensitivity differences.

[0057] In the pixel pair 90, the refractive index of the lens region 101a corresponding to the light-receiving pixel P with a large sensitivity difference is set higher than the refractive index of the lens region 101b corresponding to the light-receiving pixel P with a small sensitivity difference. For example, in the pixel pair 90, the refractive index of the lens region 101a corresponding to the light-receiving pixel P with a sensitivity difference of 12% is set higher than the refractive index of the lens region 101b corresponding to the light-receiving pixel P with a sensitivity difference of 11%. This makes it possible to suppress the sensitivity difference between the light-receiving pixels P in the pixel pair 90, i.e., to suppress the sensitivity difference between the same colors.

[0058] 10, the lens 101 is divided into two in a plan view, and the planar areas (areas in a plan view) of the lens regions 101a and 101b are the same. The planar shapes (shapes in a plan view) of the lens regions 101a and 101b are half ellipses. The lens region 101a functions as a high refractive index region, and the lens region 101b functions as a low refractive index region.

[0059] In order to improve the sensitivity difference between the same colors in a DOBC pixel array, it is possible to perform signal processing correction or aperture correction, but signal processing correction does not improve the physical optical characteristics. Furthermore, aperture correction makes it more difficult to control the aperture width as the light-receiving pixel P shrinks. Therefore, the lens 101 is formed using a lens material with a different refractive index depending on the refractive index of the color filters 131 of the light-receiving pixels P adjacent to the pixel itself (light-receiving pixel P). This makes it possible to correct the bending of light due to the refractive index difference of the color filters 131 in advance using the lens 101 and perform sensitivity correction, thereby suppressing the sensitivity difference between the same colors.

[0060] 1-3. Example Configuration of Pixel Blocks Example configurations of the pixel blocks 100Gr, 100Gb, 100R, and 100B according to this embodiment will be described with reference to FIGS. 11 to 13. FIG. 11 is a diagram showing an example configuration of a green pixel block 100Gr according to this embodiment. FIG. 12 is a diagram showing an example configuration of a red pixel block 100R according to this embodiment. FIG. 13 is a diagram showing an example connection of multiple pixel blocks 100R, 100Gr, 100Gb, and 100B according to this embodiment.

[0061] As shown in FIGS. 11 and 12, the pixel array section 11 has a plurality of control lines TRGL, a plurality of control lines RSTL, a plurality of control lines SELL, and a plurality of signal lines VSL.

[0062] The control line TRGL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12 (see FIG. 1 ). A control signal STRG is supplied to this control line TRGL by the drive unit 12. The control line RSTL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12. A control signal SRST is supplied to this control line RSTL by the drive unit 12. The control line SELL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12. A control signal SSEL is supplied to this control line SELL by the drive unit 12. The signal line VSL extends, for example, in the Y-axis direction, and one end is connected to the readout unit 20. This signal line VSL transmits a signal SIG generated by the light-receiving pixel P to the readout unit 20 (see FIG. 1 ).

[0063] (Green Pixel Block) As shown in FIG. 11, the pixel block 100Gr has, for example, ten photodiodes PD, ten transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors.

[0064] The ten photodiodes PD and ten transistors TRG correspond to the ten light-receiving pixels PGr included in the pixel block 100Gr. Each of the transistors TRG, RST, AMP, and SEL is, for example, an N-type metal oxide semiconductor (MOS) transistor. The floating diffusion layer FD and each of the transistors TRG, RST, AMP, and SEL are provided on the surface 111S1 of the semiconductor substrate 111.

[0065] The photodiode PD is a photoelectric conversion element that generates an amount of charge corresponding to the amount of light received and accumulates the generated charge internally. The anode of the photodiode PD is grounded, and the cathode is connected to the source of the transistor TRG.

[0066] The transistor TRG transfers the charge generated in the photodiode PD to the floating diffusion layer FD. The gate of the transistor TRG is connected to a control line TRGL, the source is connected to the cathode of the photodiode PD, and the drain is connected to the floating diffusion layer FD. The gates of the ten transistors TRG are connected to different control lines TRGL out of the ten control lines TRGL (in the example of FIG. 11 , the control lines TRGL1 to TRGL6 and TRGL9 to TRGL12).

[0067] The floating diffusion layer FD is configured to accumulate charges transferred from the photodiode PD via the transistor TRG. The floating diffusion layer FD is configured using, for example, a diffusion layer formed on the surface of a semiconductor substrate.

[0068] The gate of the transistor RST is connected to the control line RSTL, the drain is supplied with the power supply voltage VDD, and the source is connected to the floating diffusion layer FD. The gate of the transistor AMP is connected to the floating diffusion layer FD, the drain is supplied with the power supply voltage VDD, and the source is connected to the drain of the transistor SEL. The gate of the transistor SEL is connected to the control line SELL, the drain is connected to the source of the transistor AMP, and the source is connected to the signal line VSL.

[0069] With this configuration, in the light-receiving pixel P, for example, the transistors TRG and RST are turned on based on the control signals STRG and SRST, thereby discharging the charge accumulated in the photodiode PD. Then, when the transistors TRG and RST are turned off, an exposure period T begins, and an amount of charge corresponding to the amount of received light is accumulated in the photodiode PD. After the exposure period T ends, the light-receiving pixel P outputs a signal SIG including a reset voltage Vreset and a pixel voltage Vpix to the signal line VSL.

[0070] Specifically, first, the transistor SEL is turned on based on the control signal SSEL, electrically connecting the photosensitive pixel P to the signal line VSL. This connects the transistor AMP to a constant current source (e.g., the constant current source 21 described later) of the readout unit 20 and operates as a so-called source follower. Then, as described below, during a P-phase (pre-charge phase) period TP after the transistor RST is turned on and the voltage of the floating diffusion layer FD is reset, the photosensitive pixel P outputs a voltage corresponding to the voltage of the floating diffusion layer FD at that time as a reset voltage Vreset. Furthermore, during a D-phase (data phase) period TD after the transistor TRG is turned on and charge is transferred from the photodiode PD to the floating diffusion layer FD, the photosensitive pixel P outputs a voltage corresponding to the voltage of the floating diffusion layer at that time as a pixel voltage Vpix. The difference voltage between the pixel voltage Vpix and the reset voltage Vreset corresponds to the amount of light received by the photosensitive pixel P during the exposure period T. In this way, the light-receiving pixel P outputs a signal SIG including the reset voltage Vreset and the pixel voltage Vpix to the signal line VSL.

[0071] Like the pixel block 100Gr (see FIG. 11 ), the pixel block 100Gb includes ten photodiodes PD, ten transistors TRG, a floating diffusion layer FD, and three pixel transistors RST, AMP, and SEL. The ten photodiodes PD and ten transistors TRG correspond to the ten light-receiving pixels PGb included in the pixel block 100Gb. The gates of the ten transistors TRG are connected to different control lines TRGL among the ten control lines TRGL. The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are each provided on the surface 111S1 of the semiconductor substrate 111.

[0072] (Red Pixel Block) As shown in FIG. 12, the pixel block 100R has, for example, eight photodiodes PD, eight transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors.

[0073] The eight photodiodes PD and the eight transistors TRG correspond to the eight light-receiving pixels PR included in the pixel block 100R. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL (in the example of FIG. 12, control lines TRGL1, TRGL2, TRGL5 to TRGL10). The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are each provided on the surface 111S1 of the semiconductor substrate 111.

[0074] (Blue Pixel Block) Similar to the pixel block 100R (see FIG. 12), the pixel block 100B includes eight photodiodes PD, eight transistors TRG, a floating diffusion layer FD, and three pixel transistors RST, AMP, and SEL. The eight photodiodes PD and the eight transistors TRG correspond to the eight light-receiving pixels PB included in the pixel block 100B. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL. The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are each provided on the surface 111S1 of the semiconductor substrate 111.

[0075] (Connection Example of Each Pixel Block) As shown in Figure 13, pixel blocks 100B and 100Gb that belong to the same row and are aligned in the X direction are connected to multiple control lines TRGL out of the same 12 control lines TRGL. Also, although not shown, pixel blocks 100B and 100Gb that belong to the same row and are aligned in the X direction are connected to one control line RSTL and one control line SELL. Also, as shown in Figure 13, pixel blocks 100B that belong to the same column and are aligned in the Y direction are connected to one signal line VSL. Similarly, pixel blocks 100Gb that belong to the same column and are aligned in the Y direction are connected to one signal line VSL.

[0076] <1-4. Configuration example of readout unit> An example of the configuration of the readout unit 20 according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a diagram showing an example of the configuration of the readout unit 20 according to this embodiment. Note that in the example of Fig. 14, in addition to the readout unit 20, a reference signal generation unit 13, a signal processing unit 15, and an imaging control unit 18 are shown.

[0077] As shown in FIG. 14, the readout unit 20 includes a plurality of constant current sources 21, a plurality of AD (Analog to Digital) conversion units ADC, and a transfer control unit 27.

[0078] Each constant current source 21 and each AD conversion unit ADC are provided corresponding to a plurality of signal lines VSL. The constant current source 21 and AD conversion unit ADC corresponding to one signal line VSL will be described below.

[0079] The constant current source 21 is configured to pass a predetermined current through the corresponding signal line VSL. One end of the constant current source 21 is connected to the corresponding signal line VSL, and the other end is grounded.

[0080] The AD conversion unit ADC has a plurality of capacitive elements 22 and 23, a comparison circuit 24, a counter 25, and a latch 26. This AD conversion unit ADC is configured to perform AD conversion based on a signal SIG on a corresponding signal line VSL.

[0081] One end of the capacitive element 22 is connected to the signal line VSL, and the other end of the capacitive element 22 is connected to the comparison circuit 24. A signal SIG is supplied to the one end of the capacitive element 22 connected to the signal line VSL. A reference signal RAMP supplied from the reference signal generating unit 13 is supplied to one end of the capacitive element 23, and the other end of the capacitive element 23 is connected to the comparison circuit 24.

[0082] The comparator circuit 24 is configured to generate a signal CP by performing a comparison operation based on the signal SIG supplied from the light-receiving pixel P via the signal line VSL and the capacitive element 22 and the reference signal RAMP supplied from the reference signal generator 13 via the capacitive element 23. The comparator circuit 24 sets an operating point by setting the voltages of the capacitive elements 22, 23 based on a control signal AZ supplied from the imaging controller 18. Thereafter, the comparator circuit 24 performs a comparison operation in which the reset voltage Vreset included in the signal SIG is compared with the voltage of the reference signal RAMP during the P-phase period TP, and performs a comparison operation in which the pixel voltage Vpix included in the signal SIG is compared with the voltage of the reference signal RAMP during the D-phase period TD.

[0083] The counter 25 is configured to perform a counting operation of counting pulses of the clock signal CLK supplied from the imaging control unit 18 based on the signal CP supplied from the comparison circuit 24. Specifically, the counter 25 counts pulses of the clock signal CLK during the P-phase period TP until the signal CP transitions to generate a count value CNTP, and outputs this count value CNTP as a digital code having multiple bits. Also, the counter 25 counts pulses of the clock signal CLK during the D-phase period TD until the signal CP transitions to generate a count value CNTD, and outputs this count value CNTD as a digital code having multiple bits.

[0084] The latch 26 is configured to temporarily hold the digital code supplied from the counter 25 and to output the digital code to the bus line BUS based on an instruction from the transfer control unit 27 .

[0085] The transfer control unit 27 is configured to control the latches 26 of the plurality of AD conversion units ADC to sequentially output digital codes to the bus wiring BUS based on a control signal CTL supplied from the imaging control unit 18. The readout unit 20 uses this bus wiring BUS to sequentially transfer the plurality of digital codes supplied from the plurality of AD conversion units ADC to the signal processing unit 15 as an image signal Spic0.

[0086] The signal processing unit 15 generates the image signal Spic by performing predetermined signal processing based on the image signal Spic0 and instructions from the imaging control unit 18. FIG. 15 is a diagram showing an example of the image signal Spic according to this embodiment. As shown in FIG. 15, the signal processing unit 15 generates the image signal Spic by, for example, alternately arranging image data DP related to multiple rows of light-receiving pixels P and phase difference data DF related to multiple rows of light-receiving pixels P. The signal processing unit 15 then outputs this image signal Spic.

[0087] 1-5. Example of Operation of Imaging Apparatus An example of operation of the imaging apparatus 1 according to this embodiment will be described. First, the overall operation of the imaging apparatus 1 will be described with reference to FIG.

[0088] (Overall Operation) In the example of FIG. 1 , the drive unit 12 sequentially drives a plurality of light-receiving pixels P in the pixel array unit 11 based on instructions from the imaging control unit 18. The reference signal generation unit 13 generates a reference signal RAMP based on instructions from the imaging control unit 18. The light-receiving pixels P output a reset voltage Vreset as a signal SIG during a P-phase period TP, and output a pixel voltage Vpix corresponding to the amount of received light as a signal SIG during a D-phase period TD. The readout unit 20 generates an image signal Spic0 based on the signal SIG supplied from the pixel array unit 11 via a signal line VSL and instructions from the imaging control unit 18. In the signal processing unit 15, the image data generation unit 16 performs predetermined image processing on the image signal Spic0 to generate image data DP representing a captured image, and the phase difference data generation unit 17 performs predetermined image processing on the image signal Spic0 to generate phase difference data DF representing an image plane phase difference. The signal processing unit 15 then generates an image signal Spic including the image data DP and the phase difference data DF. The imaging control unit 18 supplies control signals to the drive unit 12, the reference signal generation unit 13, the readout unit 20, and the signal processing unit 15, and controls the operations of these circuits, thereby controlling the operation of the imaging device 1.

[0089] (Zooming Operation) The imaging control unit 18 controls the operation of the imaging device 1 based on a control signal Sctl that includes information about the zoom magnification of the electronic zoom. The zooming operation of the imaging device 1 will be described below.

[0090] Fig. 16 is a diagram showing an example of the number of effective pixels (the number of light-receiving pixels P) according to this embodiment when the zoom magnification is changed from 1x to 10x. In the example of Fig. 16, the solid line indicates the number of effective pixels of the imaging device 1. Fig. 17 is a diagram for explaining an example of zoom operation in the imaging device 1 according to this embodiment. In the example of Fig. 17, (A) shows operation when the zoom magnification is 1x, (B) shows operation when the zoom magnification is 2x, and (C) shows operation when the zoom magnification is 3x.

[0091] 17 , the imaging device 1 has three imaging modes MA, MB, and MC. The imaging control unit 18 selects one of the three imaging modes MA, MB, and MC based on information about the zoom magnification included in the control signal Sctl. Specifically, the imaging control unit 18 selects the imaging mode MA when the zoom magnification is less than 2, selects the imaging mode MB when the zoom magnification is equal to or greater than 2 but less than 3, and selects the imaging mode MC when the zoom magnification is equal to or greater than 3.

[0092] In imaging mode MA (see (A) in FIG. 17 ), the imaging device 1 obtains four pixel values ​​V (e.g., four pixel values ​​VR, VGr, VGb, and VB) for each of the multiple pixel units U. Specific operations will be described later. In this manner, the imaging device 1 generates image data DP by generating four pixel values ​​V for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array section 11 is 108 [Mpix], pixel values ​​V for 12 [Mpix] are calculated. As a result, the number of effective pixels becomes 12 [Mpix].

[0093] 16, in this imaging mode MA, when the zoom magnification is increased from 1, the number of effective pixels decreases in accordance with the magnification. Then, when the zoom magnification becomes 2, the imaging mode changes to imaging mode MB. The number of effective pixels when the zoom magnification is 2 is the same as the number of effective pixels when the zoom magnification is 1.

[0094] In imaging mode MB (see (B) in FIG. 17 ), the imaging device 1 obtains 16 pixel values ​​V for each of the plurality of pixel units U. Specific operations will be described later. In this manner, the imaging device 1 generates image data DP by generating pixel values ​​V at a ratio of 16 for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array section 11 is 108 Mpix, pixel values ​​V for 48 Mpix are calculated. In reality, because the zoom magnification is 2x, the imaging range is narrowed to 1 / 4, and the number of effective pixels is 12 Mpix (= 48 Mpix / 4).

[0095] 16 , in this image capture mode MB, when the zoom magnification is increased from 2, the number of effective pixels decreases in accordance with the magnification. Then, when the zoom magnification becomes 3, the image capture mode changes to image capture mode MC. The number of effective pixels when the zoom magnification is 3 is the same as the number of effective pixels when the zoom magnification is 2.

[0096] In imaging mode MC (see (C) in FIG. 17 ), the imaging device 1 obtains 36 pixel values ​​V in each of the plurality of pixel units U. Specific operations will be described later. In this manner, the imaging device 1 generates image data DP by generating pixel values ​​V at a ratio of 36 for each of the 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array section 11 is 108 Mpix, a captured image of 108 Mpix can be obtained. In reality, because the zoom magnification is 3x, the imaging range is narrowed to 1 / 9, and the number of effective pixels is 12 Mpix (= 108 Mpix / 9).

[0097] As described above, the imaging device 1 provides three imaging modes MA, MB, and MC, thereby reducing changes in the image quality of captured images when the zoom magnification is changed. For example, if the imaging mode MB is omitted and only two imaging modes MA and MC are provided, and the imaging mode MA is selected when the zoom magnification is less than 2x and the imaging mode MC is selected when the zoom magnification is 2x or greater, the number of effective pixels will change significantly, as indicated by the dashed line in FIG. 16 . In this example, when the zoom magnification is 2x, the imaging mode MC is selected, and the number of effective pixels is 27 Mpix (=108 Mpix / 4). Therefore, there is a significant difference between the number of effective pixels when the zoom magnification is, for example, 1.9x and the number of effective pixels when the zoom magnification is 2x, which can significantly change the image quality of captured images when the zoom magnification is around 2x. On the other hand, the imaging device 1 provides three imaging modes MA, MB, and MC, thereby reducing changes in the number of effective pixels when the zoom magnification is changed, thereby minimizing changes in the image quality of captured images.

[0098] <1-6. Image capture mode MA (first image capture mode)> The image capture mode MA according to this embodiment will be described with reference to FIGS. 18 to 20. FIG.

[0099] 18 is a diagram showing an example of the operation of the imaging device 1 in the imaging mode MA according to this embodiment. In the example of Fig. 18, the light-receiving pixels P marked with "o" indicate the light-receiving pixels P that are the target of the readout operation.

[0100] First, as shown in Figure 18 (A), the imaging device 1 generates image data DT1 by calculating a pixel value V corresponding to the amount of light received by the left light-receiving pixel P in the pixel pair 90 in which the lens 101 is provided in each of the multiple pixel blocks 100.

[0101] Specifically, the imaging device 1 performs a readout operation on five light receiving pixels PGr located on the left of five pixel pairs 90 out of the ten light receiving pixels PGr in the pixel block 100Gr, thereby calculating a pixel value VGr1 at the center of gravity of these five light receiving pixels PGr. The imaging device 1 also performs a readout operation on four light receiving pixels PR located on the left of four pixel pairs 90 out of the eight light receiving pixels PR in the pixel block 100R, thereby calculating a pixel value VR1 at the center of gravity of these four light receiving pixels PR. The imaging device 1 performs a readout operation on four light receiving pixels PB located on the left of four pixel pairs 90 out of the eight light receiving pixels PB in the pixel block 100B, thereby calculating a pixel value VB1 at the center of gravity of these four light receiving pixels PB. The imaging device 1 calculates a pixel value VGb1 at the center of gravity of the five light-receiving pixels PGb by selecting, as the target of a readout operation, the five light-receiving pixels PGb arranged on the left of the five pixel pairs 90 out of the ten light-receiving pixels PGb in the pixel block 100Gb. In this way, the imaging device 1 generates image data DT1 (see FIG. 18A) including the pixel values ​​VGr1, VR1, VB1, and VGb1.

[0102] Next, the imaging device 1 generates image data DT2 by calculating pixel values ​​V corresponding to the amount of light received by all light receiving pixels P in each of the multiple pixel blocks 100Gr, 100R, 100B, and 100Gb, as shown in Figure 18 (B).

[0103] Specifically, the imaging device 1 performs a readout operation on ten light-receiving pixels PGr in pixel block 100Gr, thereby calculating a pixel value VGr2 at the center of gravity of these ten light-receiving pixels PGr. The imaging device 1 also performs a readout operation on eight light-receiving pixels PR in pixel block 100R, thereby calculating a pixel value VR2 at the center of gravity of these eight light-receiving pixels PR. The imaging device 1 performs a readout operation on eight light-receiving pixels PB in pixel block 100B, thereby calculating a pixel value VB2 at the center of gravity of these eight light-receiving pixels PB. The imaging device 1 performs a readout operation on ten light-receiving pixels PGb in pixel block 100Gb, thereby calculating a pixel value VGb2 at the center of gravity of these ten light-receiving pixels PGb. In this way, the imaging device 1 generates image data DT2 (see FIG. 18B) including the pixel values ​​VGr2, VR2, VB2, and VGb2.

[0104] Focusing on a certain pixel block 100Gr, the readout operation for the ten light-receiving pixels PGr in this pixel block 100Gr will be described below.

[0105] 19 is a diagram showing an example of a readout operation in the imaging device 1 according to the present embodiment. In Fig. 19, (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG (control signal STRGL) supplied to the light receiving pixel PGr arranged on the left in the pixel pair 90, (D) shows the waveform of the control signal STRG (control signal STRGR) supplied to the light receiving pixel PGr arranged on the right in the pixel pair 90, (E) shows the waveform of the control signal AZ, (F) shows the waveform of the reference signal RAMP, (G) shows the waveform of the signal SIG, and (H) shows the waveform of the signal CP.

[0106] 19(F) and 19(G), the waveforms of the reference signal RAMP and the signal SIG are shown using the same voltage axis. Note that the waveform of the reference signal RAMP shown in FIG. 19(F) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitive element 23, and the waveform of the signal SIG shown in FIG. 19(G) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitive element 22.

[0107] First, at timing t11, the horizontal period H begins. This causes the drive unit 12 to change the voltage of the control signal SSEL from low to high (see FIG. 19A). This turns on the transistor SEL in the pixel block 100Gr, electrically connecting the pixel block 100Gr to the signal line VSL. Also at timing t11, the drive unit 12 changes the voltage of the control signal SRST from low to high (see FIG. 19B). This turns on the transistor RST in the pixel block 100Gr, setting the voltage of the floating diffusion layer FD to the power supply voltage VDD (reset operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusion layer FD at this time.

[0108] Also, at timing t11, the imaging control unit 18 changes the voltage of the control signal AZ from low to high (see FIG. 19(E)). As a result, the comparison circuit 24 of the AD conversion unit ADC sets the operating point by setting the voltages of the capacitance elements 22 and 23. In this way, the voltage of the signal SIG is set to the reset voltage Vreset, and the voltage of the reference signal RAMP is set to the same voltage as the voltage of the signal SIG (reset voltage Vreset) (see FIGS. 19(F) and (G)).

[0109] Then, after a predetermined time has elapsed since timing t11, the drive unit 12 changes the voltage of the control signal SRST from high to low (see FIG. 19B ), turning off the transistor RST in the pixel block 100Gr and completing the reset operation.

[0110] Next, at timing t12, the imaging control unit 18 changes the voltage of the control signal AZ from high to low (see FIG. 19(E)), which causes the comparison circuit 24 to finish setting the operating point.

[0111] At timing t12, the reference signal generator 13 sets the voltage of the reference signal RAMP to voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG, and the comparator circuit 24 changes the voltage of the signal CP from low to high (see FIG. 19(H)).

[0112] Then, during the period from timing t13 to t15 (P-phase period TP), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t13, the reference signal generation unit 13 begins to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate (see FIG. 19(F)). Also at this timing t13, the imaging control unit 18 begins generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0113] Then, at timing t14, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (reset voltage Vreset) (see (F) and (G) of FIG. 16). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see (H) of FIG. 19). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTP) is a value corresponding to the reset voltage Vreset. The latch 26 holds this count value CNTP. The counter 25 then resets its count value.

[0114] Next, at timing t15, the imaging control unit 18 stops generating the clock signal CLK as the P-phase period TP ends. At timing t15, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). From timing t15 onward, the readout unit 20 supplies the count value CNTP held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0115] Next, at timing t16, the imaging control unit 18 sets the voltage of the reference signal RAMP to voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (reset voltage Vreset), and the comparator circuit 24 changes the voltage of the signal CP from low to high (see FIG. 19(H)).

[0116] Next, at timing t17, the drive unit 12 changes the voltage of the control signal STRGL from low to high (see FIG. 19C). As a result, in the five light-receiving pixels PGr arranged on the left of the pixel pair 90, the transistors TRG are turned on, and the charge generated in the photodiodes PD is transferred to the floating diffusion layers FD (charge transfer operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusion layers FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix1 (see FIG. 19G).

[0117] Then, at a timing when a predetermined time has elapsed from timing t17, the drive unit 12 changes the voltage of the control signal STRGL from high to low (see FIG. 19C ), which turns off the transistors TRG in the five light-receiving pixels PGr arranged to the left of the pixel pair 90, and the charge transfer operation ends.

[0118] Then, during the period from timing t18 to t20 (D-phase period TD1), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t18, the reference signal generation unit 13 begins to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate (see FIG. 19(F)). Also at this timing t18, the imaging control unit 18 begins generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0119] Then, at timing t19, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix1) (see Figures 19(F) and 19(G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see Figure 19(H)). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD1) is a value corresponding to the pixel voltage Vpix1. The latch 26 holds this count value CNTD1. The counter 25 then resets its count value.

[0120] Next, at timing t20, the imaging control unit 18 stops generating the clock signal CLK as the D-phase period TD1 ends. At timing t20, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). Then, from timing t20 onward, the readout unit 20 supplies the count value CNTD1 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0121] Next, at timing t21, the imaging control unit 18 sets the voltage of the reference signal RAMP to voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (pixel voltage Vpix1), and the comparator circuit 24 changes the voltage of the signal CP from low to high (see FIG. 19(H)).

[0122] Next, at timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from low to high (see FIGS. 19C and 19D). As a result, in the ten light-receiving pixels PGr in the pixel block 100Gr, the transistors TRG are turned on, and the charges generated in the photodiodes PD are transferred to the floating diffusion layers FD (charge transfer operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusion layers FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix2 (see FIG. 19G).

[0123] Then, at a timing when a predetermined time has elapsed from timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from high to low (see FIGS. 19C and 19D), thereby turning off the transistors TRG in the ten light-receiving pixels PGr and ending the charge transfer operation.

[0124] Then, during the period from timing t23 to t25 (D-phase period TD2), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t23, the reference signal generation unit 13 begins to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate (see FIG. 19(F)). Also at this timing t23, the imaging control unit 18 begins generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0125] Then, at timing t24, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix2) (see (F) and (G) of FIG. 19). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see (H) of FIG. 19). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD2) is a value corresponding to the pixel voltage Vpix2. The latch 26 holds this count value CNTD2. The counter 25 then resets its count value.

[0126] Next, at timing t25, the imaging control unit 18 stops generating the clock signal CLK as the D-phase period TD2 ends. At timing t25, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). Then, from timing t25 onward, the readout unit 20 supplies the count value CNTD2 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0127] Next, at timing t26, the drive unit 12 changes the voltage of the control signal SSEL from high to low (see FIG. 19A), which turns off the transistor SEL in the pixel block 100Gr and electrically disconnects the pixel block 100Gr from the signal line VSL.

[0128] In this way, the readout unit 20 supplies the image signal Spic0 including the count values ​​CNTP, CNTD1, and CNTD2 to the signal processing unit 15. The signal processing unit 15 generates the pixel value VGr1 shown in Fig. 18(A) and the pixel value VGr2 shown in Fig. 18(B) by using the principle of correlated double sampling based on the count values ​​CNTP, CNTD1, and CNTD2 included in the image signal Spic0, for example.

[0129] Specifically, the signal processing unit 15 generates the pixel value VGr1 by, for example, subtracting the count value CNTP from the count value CNTD1. The count value CNTD1 is a value corresponding to the sum of the amounts of light received by the five light-receiving pixels PGr located on the left of the five pixel pairs 90 in the pixel block 100Gr, so the signal processing unit 15 can generate the pixel value VGr1 shown in FIG. 18A based on this count value CNTD1. Similarly, the signal processing unit 15 generates the pixel value VGr2 by, for example, subtracting the count value CNTP from the count value CNTD2. The count value CNTD2 is a value corresponding to the sum of the amounts of light received by the ten light-receiving pixels PGr in the pixel block 100Gr, so the signal processing unit 15 can generate the pixel value VGr2 shown in FIG. 18B based on this count value CNTD2.

[0130] The above has been described for pixel block 100Gr, but the same applies to each of pixel blocks 100R, 100Gb, and 100B. In this way, the signal processing unit 15 generates image data DT1 including pixel values ​​VR1, VGr1, VGb1, and VB1, and image data DT2 including pixel values ​​VR2, VGr2, VGb2, and VB2, as shown in FIG.

[0131] FIG. 20 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MA according to this embodiment.

[0132] First, the signal processing unit 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0133] Specifically, the signal processing unit 15 calculates the pixel value VGr3 by subtracting the pixel value VGr1 in the image data DT1 from the pixel value VGr2 in the image data DT2. This pixel value VGr3 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the right of the five pixel pairs 90 in the pixel block 100Gr. In other words, the pixel value VGr1 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the left of the five pixel pairs 90 in the pixel block 100Gr, and the pixel value VGr2 is a value corresponding to the sum of the amounts of light received by the ten light receiving pixels PGr in the pixel block 100Gr. Therefore, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the right of the five pixel pairs 90 in the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value corresponding to the sum of the amounts of light received at the five light-receiving pixels PGr located on the right of the five pixel pairs 90, and therefore, as shown in Figure 20, the pixel value VGr3 is located at the center of gravity of these five light-receiving pixels PGr.

[0134] Similarly, the signal processing unit 15 calculates a pixel value VR3 by subtracting the pixel value VR1 in the image data DT1 from the pixel value VR2 in the image data DT2. This pixel value VR3 is a value corresponding to the sum of the amounts of light received by the four light receiving pixels PR located on the right of the four pixel pairs 90 in the pixel block 100R. The pixel value VR3 is located at the center of gravity of the four light receiving pixels PR located on the right of the four pixel pairs 90 in the pixel block 100R.

[0135] The signal processing unit 15 calculates a pixel value VB3 by subtracting the pixel value VB1 in the image data DT1 from the pixel value VB2 in the image data DT2. This pixel value VB3 is a value corresponding to the sum of the amounts of light received by the four light-receiving pixels PB located on the right of the four pixel pairs 90 in the pixel block 100B. The pixel value VB3 is located at the center of gravity of the four light-receiving pixels PB located on the right of the four pixel pairs 90 in the pixel block 100B.

[0136] The signal processing unit 15 calculates a pixel value VGb3 by subtracting the pixel value VGb1 in the image data DT1 from the pixel value VGb2 in the image data DT2. This pixel value VGb3 is a value corresponding to the sum of the amounts of light received by the five light-receiving pixels PGb located on the right of the five pixel pairs 90 in the pixel block 100Gb. The pixel value VGb3 is located at the center of gravity of the five light-receiving pixels PGb located on the right of the five pixel pairs 90 in the pixel block 100Gb.

[0137] Then, the image data generating section 16 of the signal processing section 15 performs predetermined image processing based on the image data DT2 to generate image data DP (see FIG. 17A) representing the captured image.

[0138] Furthermore, the phase difference data generator 17 of the signal processor 15 performs predetermined image processing based on the image data DT1 and DT3 to generate phase difference data DF indicating the image plane phase difference. That is, the image data DT1 has a pixel value V of the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V of the light-receiving pixel P located on the right of the plurality of pixel pairs 90. Thus, the phase difference data generator 17 can generate the phase difference data DF based on the image data DT1 and DT3.

[0139] In the imaging device 1, the lenses 101 are arranged in parallel in the X direction and the Y direction in the pixel array unit 11 (see FIG. 2), so that the phase difference data DF can be generated with high resolution over the entire surface of the pixel array unit 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus, thereby improving image quality.

[0140] <1-7. Image Capture Mode MB (Second Image Capture Mode)> The image capture mode MB according to this embodiment will be described with reference to FIGS. 21 to 27. FIG.

[0141] Fig. 21 is a diagram showing an example of operation of the imaging device 1 in imaging mode MB according to this embodiment. Fig. 22 is a diagram showing the operation shown in Fig. 21 in more detail. In Fig. 21 and Fig. 22, the light-receiving pixels P marked with "o" indicate light-receiving pixels P that are the target of the readout operation.

[0142] 22A, in the pixel block 100Gr, the imaging device 1 performs a readout operation on the photosensitive pixel PGr located on the left of one of the five pixel pairs 90 in the pixel block 100Gr, and calculates a pixel value VGr1 at the position of this photosensitive pixel PGr. Next, as shown in FIG. 22B, the imaging device 1 performs a readout operation on two photosensitive pixels PGr in that pixel pair 90, and calculates a pixel value VGr2 at the center of gravity of these two photosensitive pixels PGr.

[0143] 22A, in the pixel block 100R, the imaging device 1 performs a readout operation on the photosensitive pixel PR located on the left of one of the four pixel pairs 90 in the pixel block 100R, and calculates a pixel value VR1 at the position of this photosensitive pixel PR. Next, as shown in FIG. 22B, the imaging device 1 performs a readout operation on two photosensitive pixels PR in that pixel pair 90, and calculates a pixel value VR2 at the center of gravity of these two photosensitive pixels PR.

[0144] 22A, in the pixel block 100B, the imaging device 1 performs a readout operation on the photosensitive pixel PB located on the left of one of the four pixel pairs 90 in the pixel block 100B, and calculates a pixel value VB1 at the position of this photosensitive pixel PB. Next, as shown in FIG. 22B, the imaging device 1 performs a readout operation on two photosensitive pixels PB in that pixel pair 90, and calculates a pixel value VB2 at the center of gravity of these two photosensitive pixels PB.

[0145] 22A, in pixel block 100Gb, the imaging device 1 performs a readout operation on the photosensitive pixel PGb located on the left of one of the five pixel pairs 90 in pixel block 100Gb, and calculates a pixel value VGb1 at the position of this photosensitive pixel PGb. Next, as shown in FIG. 22B, the imaging device 1 performs a readout operation on two photosensitive pixels PGb in that pixel pair 90, and calculates a pixel value VGb2 at the center of gravity of these two photosensitive pixels PGb.

[0146] The readout operations in Figures 22(A) and 22(B) are the same as the readout operation in the imaging mode MA described above (see Figure 19). The imaging device 1 then performs the operations in Figures 22(C) and 22(D), 22(E) and 22(F), 22(G) and 22(H), 22(I) and 22(J), and 22(K) and 22(L). In this manner, the imaging device 1 generates image data DT1 (see Figure 21(A)) including pixel values ​​VGr1, VR1, VB1, and VGb1, and image data DT2 (see Figure 21(B)) including pixel values ​​VGr2, VR2, VB2, and VGb2.

[0147] FIG. 23 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MB according to this embodiment.

[0148] First, the signal processing unit 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0149] Specifically, the signal processing unit 15 calculates five pixel values ​​VGr3 by subtracting five pixel values ​​VGr1 in the image data DT1 from five pixel values ​​VGr2 in the image data DT2, respectively. These pixel values ​​VGr3 correspond to the amount of light received by the light receiving pixel PGr located on the right in the pixel pair 90 of the pixel block 100Gr. That is, the pixel value VGr1 corresponds to the amount of light received by the light receiving pixel PGr located on the left in the pixel pair 90 of the pixel block 100Gr, and the pixel value VGr2 corresponds to the sum of the amounts of light received by the two light receiving pixels PGr in this pixel pair 90. Therefore, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the amount of light received by the light receiving pixel PGr located on the right in the pixel pair 90 of the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value corresponding to the amount of light received at the light-receiving pixel PGr located on the right in the pixel pair 90, and therefore, as shown in Figure 23, the pixel value VGr3 is located at the position of the light-receiving pixel PGr located on the right in the pixel pair 90.

[0150] Similarly, the signal processing unit 15 calculates four pixel values ​​VR3 by subtracting the four pixel values ​​VR1 in the image data DT1 from the four pixel values ​​VR2 in the image data DT2, respectively. These pixel values ​​VR3 correspond to the amount of light received by the light-receiving pixel PR located on the right in the pixel pair 90 of the pixel block 100R. These pixel values ​​VR3 are located at the position of the light-receiving pixel PR located on the right in this pixel pair 90.

[0151] The signal processing unit 15 calculates four pixel values ​​VB3 by subtracting the four pixel values ​​VB1 in the image data DT1 from the four pixel values ​​VB2 in the image data DT2, respectively. These pixel values ​​VB3 correspond to the amount of light received by the light-receiving pixel PB located on the right of the pixel pair 90 in the pixel block 100B. These pixel values ​​VB3 are located at the positions of the four light-receiving pixels PB located on the right of the pixel pair 90.

[0152] The signal processing unit 15 calculates five pixel values ​​VGb3 by subtracting the five pixel values ​​VGb1 in the image data DT1 from the five pixel values ​​VGb2 in the image data DT2, respectively. These pixel values ​​VGb3 correspond to the amount of light received by the light-receiving pixel PGb located on the right in the pixel pair 90 of the pixel block 100Gb. These pixel values ​​VGb3 are located at the position of the light-receiving pixel PGb located on the right in the pixel pair 90.

[0153] Then, the image data generating unit 16 of the signal processing unit 15 generates image data DP (FIG. 17B) representing the captured image by performing predetermined image processing based on the image data DT2, as shown in Fig. 23. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V.

[0154] 24 is a diagram showing an example of re-mosaic processing in imaging mode MB according to this embodiment. In Fig. 24, (A) shows image data DT2, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on the image data DT2. In Fig. 24(B), "◯" indicates the position of pixel value V in the image data DT2, and "□" indicates the position of pixel value V in the image data DT4.

[0155] Fig. 25 is a diagram showing the re-mosaic processing for the pixel block 100R (first pixel block) in the imaging mode MB according to this embodiment. Fig. 26 is a diagram showing the re-mosaic processing for each of the pixel blocks 100Gr and 100Gb (second and third pixel blocks) in the imaging mode MB according to this embodiment. Fig. 27 is a diagram showing the re-mosaic processing for the pixel block 100B (fourth pixel block) in the imaging mode MB according to this embodiment. In Figs. 25 to 27, (A) shows image data DT2, and (B) shows image data DT4 generated by the re-mosaic processing.

[0156] As shown in FIG. 24A, the image data DT2 includes 18 pixel values ​​V for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array unit 11 is 108 Mpix, the image data DT2 includes 54 Mpix worth of pixel values ​​V. On the other hand, as shown in FIG. 24C, the image data DT4 includes 16 pixel values ​​V for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array unit 11 is 108 Mpix, the image data DT4 includes 48 Mpix worth of pixel values ​​V. Furthermore, in the image data DT4, four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generation unit 16 generates this image data DT4 by modifying the pixel values ​​V in the image data DT2 and rearranging the pixel values ​​V.

[0157] Specifically, as shown in Fig. 25, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VR2 in the image data DT2, for example, by performing an interpolation process, and generates a pixel value VR4 based on the pixel value V for one surface. Similarly, as shown in Fig. 26, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VGr2, VGb2 in the image data DT2, for example, by performing an interpolation process, and generates pixel values ​​VGr4, VGb4 based on the pixel value V for one surface. As shown in Fig. 27, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VB2 in the image data DT2, for example, by performing an interpolation process, and generates a pixel value VB4 based on the pixel value V for one surface.

[0158] In this way, the image data generating unit 16 generates the image data DT4 shown in Fig. 24C based on the image data DT2 shown in Fig. 24A. Then, the image data generating unit 16 performs predetermined image processing based on this image data DT4 to generate image data DP (see Fig. 17B) representing the captured image.

[0159] 23, the phase difference data generation unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing predetermined image processing based on the image data DT1 and DT3. That is, the image data DT1 has a pixel value V of the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V of the light-receiving pixel P located on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generation unit 17 can generate the phase difference data DF based on the image data DT1 and DT3.

[0160] The phase difference data generation unit 17, like the image data generation unit 16, generates phase difference data DF by rearranging pixel values ​​V based on the image data DT1 and DT3. That is, in each of the image data DT1 and DT3, the left and right pixel values ​​V of the pixel pair 90 are included at a rate of 18 out of 36 light-receiving pixels P. Therefore, the phase difference data generation unit 17 rearranges the pixel values ​​V so that the left and right pixel values ​​V of the pixel pair 90 are included at a rate of 16 out of 36 light-receiving pixels P. In this way, the phase difference data generation unit 17 can generate phase difference data DF corresponding to the image data DP generated by the image data generation unit 16.

[0161] <1-8. Image capture mode MC (third image capture mode)> The image capture mode MC according to this embodiment will be described with reference to FIGS. 28 to 35. FIG.

[0162] Fig. 28 is a diagram showing an example of operation of the imaging device 1 in imaging mode MC according to this embodiment. Fig. 29 is a diagram showing more specifically the operation shown in Fig. 28. In Figs. 28 and 29, the light-receiving pixels P marked with "◯" indicate light-receiving pixels P that are the target of the readout operation.

[0163] In pixel block 100Gr, as shown in Figure 29 (A), the imaging device 1 targets one of the ten light-receiving pixels PGr in pixel block 100Gr as the target of the readout operation, and calculates the pixel value VGr1 at the position of this light-receiving pixel PGr.

[0164] FIG. 30 is a diagram showing an example of a readout operation in the imaging device 1 according to this embodiment. In FIG. 30, (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG supplied to the light-receiving pixel PGr to be read, (D) shows the waveform of the control signal AZ, (E) shows the waveform of the reference signal RAMP, (F) shows the waveform of the signal SIG, and (G) shows the waveform of the signal CP. This readout operation is the same as the readout operation in each imaging mode MA or MB (see FIG. 19 ), except that the operations at times t21 to t26 are omitted. In this manner, the readout unit 20 supplies the image signal Spic0, which includes the count values ​​CNTP and CNTD1, to the signal processing unit 15. The signal processing unit 15 generates a pixel value VGr1, for example, using the principle of correlated double sampling based on the count values ​​CNTP and CNTD1 included in the image signal Spic0. Specifically, the signal processing unit 15 generates the pixel value VGr1 by subtracting the count value CNTP from the count value CNTD1, for example.

[0165] Similarly, in pixel block 100R, as shown in Figure 29 (A), the imaging device 1 targets one of the eight photosensitive pixels PR in pixel block 100R as the target for the readout operation, and calculates the pixel value VR1 at the position of this photosensitive pixel PR.

[0166] Similarly, in pixel block 100B, as shown in Figure 29 (A), the imaging device 1 targets one of the eight photosensitive pixels PB in pixel block 100B as the target of the readout operation, and calculates the pixel value VB1 at the position of this photosensitive pixel PB.

[0167] Similarly, in pixel block 100Gb, as shown in Figure 29 (A), the imaging device 1 targets one of the ten photosensitive pixels PGb in pixel block 100Gb as the target of the readout operation and calculates the pixel value VGb1 of this photosensitive pixel PGb.

[0168] The imaging device 1 then performs the operations shown in Figures 29(B) to 29(L) In this way, the imaging device 1 generates image data DT1 (see Figure 28) including pixel values ​​VGr1, VR1, VB1, and VGb1.

[0169] FIG. 31 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MC according to this embodiment.

[0170] The image data generating unit 16 of the signal processing unit 15 performs predetermined image processing based on the image data DT1 to generate image data DP (see FIG. 17C) representing the captured image. This predetermined image processing includes a re-mosaic process that modifies the pixel values ​​V and rearranges the pixel values ​​V.

[0171] 32 is a diagram showing an example of re-mosaic processing in imaging mode MC according to this embodiment. In Fig. 32, (A) shows image data DT1, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on the image data DT1. In Fig. 32(B), "◯" indicates the position of pixel value V in the image data DT1, and "□" indicates the position of pixel value V in the image data DT4.

[0172] Fig. 33 is a diagram showing the re-mosaic processing for the pixel block 100R in the imaging mode MC according to this embodiment. Fig. 34 is a diagram showing the re-mosaic processing for each of the pixel blocks 100Gr and 100Gb in the imaging mode MC according to this embodiment. Fig. 35 is a diagram showing the re-mosaic processing for the pixel block 100B in the imaging mode MC according to this embodiment. In Figs. 33 to 35, (A) shows image data DT1, and (B) shows image data DT4 generated by the re-mosaic processing.

[0173] In the image data DT4, four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generator 16 generates this image data DT4 by modifying the pixel values ​​V in the image data DT1 and rearranging the pixel values ​​V.

[0174] Specifically, as shown in Fig. 33, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VR1 in the image data DT1, for example, by performing an interpolation process, and generates a pixel value VR4 based on the pixel value V for one surface. Similarly, as shown in Fig. 34, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VGr1, VGb1 in the image data DT1, for example, by performing an interpolation process, and generates pixel values ​​VGr4, VGb4 based on the pixel value V for one surface. As shown in Fig. 35, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VB1 in the image data DT1, for example, by performing an interpolation process, and generates a pixel value VB4 based on the pixel value V for one surface.

[0175] In this way, the image data generating unit 16 generates the image data DT4 shown in Fig. 32C based on the image data DT1 shown in Fig. 32A. Then, the image data generating unit 16 performs predetermined image processing based on this image data DT4 to generate image data DP (see Fig. 17C) representing the captured image.

[0176] 31, the phase difference data generation unit 17 of the signal processing unit 15 performs predetermined image processing based on the image data DT1 to generate phase difference data DF indicating the image plane phase difference. That is, the image data DT1 includes image data (image data DT11) having a pixel value V at the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and image data (image data DT12) having a pixel value V at the light-receiving pixel P located on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generation unit 17 can generate the phase difference data DF based on this image data DT1 (each of the image data DT11 and DT12).

[0177] As described above, the imaging device 1 includes a plurality of pixel blocks 100, each of which includes a plurality of light-receiving pixels P including color filters 131 of the same color. The plurality of light-receiving pixels P are divided into a plurality of pixel pairs 90, each of which includes two light-receiving pixels P. A plurality of lenses 101 are provided at positions corresponding to each of these plurality of pixel pairs 90. This allows the imaging device 1 to generate phase difference data DF with high resolution across the entire surface of the pixel array section 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus at various zoom magnifications. As a result, the imaging device 1 can improve image quality.

[0178] Furthermore, in the imaging device 1, the number of light receiving pixels P in a given pixel block 100 is set to be greater than the number of light receiving pixels P in another pixel block 100. Specifically, for example, the number of light receiving pixels PGr in pixel block 100Gr and the number of light receiving pixels PGb in pixel block 100Gb are set to be greater than the number of light receiving pixels PR in pixel block 100R and the number of light receiving pixels PB in pixel block 100B. This makes it possible to increase, for example, green light receiving sensitivity, thereby improving the image quality of the captured image.

[0179] Furthermore, the imaging device 1 is provided with three imaging modes MA, MB, and MC, and re-mosaic processing is performed in each of the imaging modes MB and MC. As a result, in the imaging device 1, for example, the number of effective pixels in imaging mode MB can be adjusted by re-mosaic processing in imaging mode MB, and it is possible to reduce changes in the number of effective pixels when the zoom magnification is changed, thereby suppressing changes in the image quality of the captured image.

[0180] 2. Modifications Modifications 1 to 5 according to this embodiment will be described with reference to FIGS. 36 to 42.

[0181] <2-1. Modification 1> FIG. 36 is a plan view for explaining the refractive index of each lens 101 in the pixel array section 11 according to Modification 1 of this embodiment.

[0182] As shown in Figure 36, the refractive index of the lens region 101a or the lens region 101b may be different for each color. In other words, although the high refractive index material or low refractive index material of the lens 101 is the same regardless of the color (R, G, B) in the above description, it may be different for each color. Because the degree of sensitivity difference between the same colors may differ for each color (see Figure 9), the refractive index of one or both of the lens region 101a and the lens region 101b may be different for each color. For example, the high refractive index material or low refractive index material may be different for each red (R) or blue (B) light-receiving pixel P.

[0183] 36, seven types of materials, three types of high refractive index materials and four types of low refractive index materials, are used to form the lens 101. The lens region 101a is formed from one of the three types of high refractive index materials, and the lens region 101b is formed from one of the four types of high refractive index materials. Note that there may be more types of each of the high refractive index materials and the low refractive index materials.

[0184] For example, a green (G) light-receiving pixel P adjacent to a red (R) light-receiving pixel P and a green (G) light-receiving pixel P adjacent to a blue (B) light-receiving pixel P may use a high-refractive index material or a low-refractive index material with the same refractive index. However, since the green (G) light-receiving pixel P adjacent to a red (R) light-receiving pixel P and the green (G) light-receiving pixel P adjacent to a blue (B) light-receiving pixel P are affected differently by the adjacent color filters 131, a high-refractive index material or a low-refractive index material with a different refractive index may be used for each of these light-receiving pixels P.

[0185] According to variant example 1, in addition to adjusting the refractive index of each of lens area 101a and lens area 101b, by changing the refractive index of lens area 101a or lens area 101b for each color, it becomes possible to accurately correct sensitivity, thereby achieving improved image quality.

[0186] <2-2. Modification 2> FIG. 37 is a plan view for explaining the refractive index of each lens 101 in the pixel array section 11 according to Modification 1 of this embodiment.

[0187] As shown in Fig. 37 , a pixel pair 90 that does not produce a same-color sensitivity difference is provided with a lens 101 with a different refractive index for each color. In the example of Fig. 37 , the lens 101 for the red (R) pixel pair 90 that does not produce a same-color sensitivity difference is filled with a left-diagonal line, the lens 101 for the green (G) pixel pair 90 that does not produce a same-color sensitivity difference is filled with a right-diagonal line, and the lens 101 for the blue (B) pixel pair 90 that does not produce a same-color sensitivity difference is not filled with a line.

[0188] That is, in the above description, lenses with the same refractive index regardless of color are used as lenses 101 for pixel pairs 90 that do not cause a same-color sensitivity difference, but lenses with different refractive indexes may be used to match the sensitivity differences for each color of pixel pairs 90. However, for Gr and Gb, it is desirable to use lenses with the same refractive index as lenses 101 that do not cause a same-color sensitivity difference.

[0189] According to variant example 2, in addition to adjusting the refractive index of each of lens area 101a and lens area 101b, by changing the refractive index of lens 101 for pixel pairs 90 that do not cause a sensitivity difference between the same colors for each color, it becomes possible to accurately correct the sensitivity, thereby achieving improved image quality.

[0190] <2-3. Modification 3> Fig. 38 is a plan view for explaining the refractive index of each lens 101 in a pixel array unit 11 according to Modification 3 of this embodiment. Fig. 39 is a diagram showing an example of the cross-sectional structure of the pixel array unit 11 according to Modification 3 of this embodiment. Note that Fig. 39 shows a cross section corresponding to the Y2-Y2 line shown in Fig. 38.

[0191] 38 and 39 , the ratio of the lens region 101a (high refractive index region) to the lens region 101b (low refractive index region) of the lens 101 changes depending on the position of the lens 101 with respect to the pixel array unit 11. In other words, the planar areas (areas in a plan view) of the lens region 101a and the lens region 101b each change depending on the position of the lens 101 with respect to the pixel array unit 11.

[0192] 38, the planar area of ​​lens region 101a increases as lens 101 is positioned closer to the outer periphery of the angle of view. Conversely, the planar area of ​​lens region 101b decreases as lens 101 is positioned closer to the outer periphery of the angle of view. Note that when lens 101 is positioned near the center of the angle of view, lens region 101a and lens region 101b have the same planar area.

[0193] According to variant example 3, in addition to adjusting the refractive index of each of lens area 101a and lens area 101b, the ratio of lens area 101a (high refractive area) to lens area 101b (low refractive area) can be changed depending on the position of lens 101 relative to pixel array section 11, thereby making it possible to accurately correct sensitivity and thereby achieving improved image quality.

[0194] <2-4. Modification 4> FIG. 40 is a plan view for explaining the refractive index of each lens 101 in the pixel array section 11 according to Modification 4 of this embodiment.

[0195] 40 , the planar division pattern (division pattern in plan view) of the lens region 101a (high refractive index region) and the lens region 101b (low refractive index region) of the lens 101 changes depending on the position of the lens 101 with respect to the pixel array unit 11. In other words, the planar shapes of the lens region 101a and the lens region 101b change depending on the position of the lens 101 with respect to the pixel array unit 11. Note that, as in the above-described fourth modification, for example, the planar area of ​​the lens region 101a increases as the lens 101 is positioned closer to the outer periphery of the angle of view.

[0196] In the example of Figure 40, the planar division pattern changes as the lens 101 is positioned closer to the periphery of the angle of view (outside the angle of view). For example, since the influence of oblique incidence becomes stronger as the lens 101 is positioned closer to the four corners of the angle of view, the lens 101 may be divided, for example, according to the position of the hands of a clock, rather than being divided vertically. In the example of Figure 40, in the lens 101 positioned on the four corner sides, the planar shape of the lens region 101b (low refractive region) is fan-shaped, but this is not limited thereto and may be changed as appropriate depending on the position of the lens 101 relative to the pixel array unit 11, and the planar area of ​​the fan shape may also be changed as appropriate depending on the position of the lens 101 relative to the pixel array unit 11.

[0197] According to variant example 4, in addition to adjusting the refractive index of each of lens area 101a and lens area 101b, the planar division pattern of lens area 101a (high refractive index area) and lens area 101b (low refractive index area) can be changed depending on the position of lens 101 relative to pixel array section 11, thereby making it possible to accurately correct sensitivity and thereby achieving improved image quality.

[0198] <2-5. Modification 5> Fig. 41 is a plan view showing the position of the desired lens 101 in the pixel array unit 11 according to Modification 5 of the present embodiment. Fig. 42 is a diagram for explaining the refractive index of the desired lens 101 according to Modification 5 of the present embodiment.

[0199] As shown in Figure 41, the sensitivity of a desired pixel pair 90 (photosensitive pixel PGr A1 and photosensitive pixel PGr A2 in Figure 41) may be affected differently for each photosensitive pixel P (six pixels a to f in Figure 41) that borders the desired pixel pair 90 at its side. Note that photosensitive pixels P that border the pixel pair 90 diagonally are not relevant because they border at a point. In the example of Figure 41, pixel a is photosensitive pixel PB, pixel b is photosensitive pixel PGb, pixel c is photosensitive pixel PGr, pixel d is photosensitive pixel PR, pixel e is photosensitive pixel PGr, and pixel f is photosensitive pixel PR.

[0200] In such a case, as shown in Fig. 42, one lens 101 is divided into six parts in a planar view. In the example of Fig. 42, the lens 101 is radially divided into six parts, each part being a to f. Part a is a low-refractive-index lens part 101b, parts b, c, and e are medium-refractive-index lens parts 101c, and parts d and f are high-refractive-index lens parts 101a. The refractive index of the high-refractive-index lens part 101a is higher than the refractive index of the low-refractive-index lens part 101b and the medium-refractive-index lens part 101c, and the refractive index of the medium-refractive-index lens part 101c is higher than the refractive index of the low-refractive-index lens part 101b.

[0201] According to the fifth modification, by dividing one lens 101 into six, it becomes possible to combine lens materials according to the sensitivity of the light-receiving pixel P adjacent to a desired pixel pair 90. This makes it possible to design the lens 101 taking into consideration the influence that each light-receiving pixel P of each pixel pair 90 receives from the surrounding light-receiving pixels P.

[0202] The number of divisions of the lens 101 may be 4, 8, 3 or 5, other than 6. In other words, the lens 101 for the pixel pair 90 may be divided into two regions, one with a high refractive index and one with a low refractive index, or may be divided into three or more regions with different refractive indices depending on the color of the light-receiving pixel P adjacent to the pixel pair 90 (the refractive index of the color filter 131 of the light-receiving pixel P).

[0203] 3. Functions and Effects As described above, the imaging device 1 according to this embodiment includes a pixel array unit 11 having a plurality of pixels (light-receiving pixels P) each including a color filter 131. The plurality of pixels include first and second pixels each including the color filter 131 of the same color. The pixel array unit 11 has a lens 101 provided in common to the first and second pixels. The lens 101 has a first lens region (e.g., lens region 101a) corresponding to the first pixel and a second lens region (e.g., lens region 101b) corresponding to the second pixel. The first lens region and the second lens region have different refractive indices (see, for example, FIGS. 3 and 10 ). This makes it possible to correct the sensitivity of the first pixel and the second pixel and reduce the difference in sensitivity between the first pixel and the second pixel of the same color, thereby achieving improved image quality.

[0204] Furthermore, the planar areas of the first lens region and the second lens region may be the same (see, for example, FIGS. 10, 36, and 37), which allows the sensitivity of the first pixel and the second pixel to be appropriately corrected, thereby reliably suppressing the difference in sensitivity between the first pixel and the second pixel for the same color.

[0205] The planar areas of the first lens region and the second lens region may be different (see FIGS. 38 to 40 ), which allows the sensitivity of the first pixel and the second pixel to be appropriately corrected, thereby reliably suppressing the difference in sensitivity between the first pixel and the second pixel for the same color.

[0206] Furthermore, the planar areas of the first lens region and the second lens region may be changed depending on the position of the lens 101 relative to the pixel array unit 11 (see FIGS. 38 to 40 ). This makes it possible to appropriately correct the sensitivities of the first pixel and the second pixel, thereby reliably suppressing the difference in sensitivity between the first pixel and the second pixel for the same color.

[0207] Furthermore, the refractive index of the first lens region may be higher than the refractive index of the second lens region, and the planar area of ​​the first lens region may be larger as the lens 101 is positioned closer to the outer periphery of the pixel array unit 11 (see FIGS. 38 to 40 ). This makes it possible to appropriately correct the sensitivities of the first and second pixels according to the angle of view of the pixel array unit 11, thereby reliably suppressing the difference in sensitivity between the first and second pixels of the same color.

[0208] Furthermore, the planar shapes of the first lens region and the second lens region may be the same (see, for example, FIGS. 10, 36, and 37), which allows the sensitivity of the first pixel and the second pixel to be appropriately corrected, thereby reliably suppressing the difference in sensitivity between the first pixel and the second pixel for the same color.

[0209] The first lens region and the second lens region may have different planar shapes (see FIGS. 38 to 40 ), which allows the sensitivity of the first pixel and the second pixel to be appropriately corrected, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0210] Furthermore, the planar shapes of the first lens region and the second lens region may be changed depending on the position of the lens 101 relative to the pixel array unit 11 (see FIG. 40 ). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel depending on the angle of view of the pixel array unit 11, thereby reliably suppressing the difference in sensitivity between the first pixel and the second pixel of the same color.

[0211] Furthermore, the lens 101 has a plurality of lens regions (e.g., lens region 101a, lens region 101b, and lens region 101c) including a first lens region and a second lens region, and the refractive indexes of any three or more of the plurality of lens regions may be different (see FIGS. 41 and 42). This makes it possible to more appropriately correct the sensitivity of the first pixel and the second pixel, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel for the same color.

[0212] Furthermore, the lens regions may be provided in accordance with the arrangement of the pixels surrounding the first pixel and the second pixel (see FIGS. 41 and 42 ), which makes it possible to more appropriately correct the sensitivity of the first pixel and the second pixel, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0213] Furthermore, the plurality of lens regions may be divided so that the lenses are radially separated in plan view (see FIG. 42 ), which allows for more appropriate correction of the sensitivities of the first and second pixels, thereby reliably suppressing the sensitivity difference between the first and second pixels of the same color.

[0214] Furthermore, the lens 101 may be formed from a plurality of materials with different refractive indices, which makes it possible to easily manufacture the lens 101 having a plurality of lens regions with different refractive indices.

[0215] The plurality of materials may also contain materials with different composition ratios, which makes it possible to easily manufacture the lens 101 having a plurality of lens regions with different refractive indices.

[0216] The first pixel and the second pixel may be adjacent to each other, which makes it possible to more appropriately correct the sensitivities of the first pixel and the second pixel in a pixel pair 90 in which the first pixel and the second pixel are adjacent to each other, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0217] Furthermore, the pixels may be arranged in Deca-Octa Bayer Coding (DOBC). When the pixel arrangement is DOBC, the sensitivity of the first pixel and the second pixel can be corrected to reduce the sensitivity difference between the first pixel and the second pixel of the same color, thereby improving image quality.

[0218] Furthermore, the first pixel and the second pixel may have a sensitivity difference between the same colors, and the refractive indexes of the first lens region and the second lens region may be set to suppress the sensitivity difference between the same colors. Even in such a case, the sensitivity of the first pixel and the second pixel can be corrected to suppress the sensitivity difference between the same colors of the first pixel and the second pixel, thereby achieving improved image quality.

[0219] The plurality of pixels include a third pixel and a fourth pixel, each having a color filter 131 of the same color. The pixel array unit 11 has a second lens (e.g., lens 101) provided in common to the third pixel and the fourth pixel. The second lens has a third lens region (e.g., lens region 101a) corresponding to the third pixel and a fourth lens region (e.g., lens region 101b) corresponding to the fourth pixel. The refractive indices of the first lens region, the second lens region, the third lens region, and the fourth lens region may be different from each other (see FIG. 36 ). This allows accurate sensitivity correction by changing the refractive indices of the first lens region, the second lens region, the third lens region, and the fourth lens region, thereby improving image quality.

[0220] The plurality of pixels include a plurality of pixel pairs 90 each having a pair of pixels each having a color filter 131 of the same color, and the pixel array unit 11 has a plurality of lenses 101 provided for each pixel pair 90, and the refractive index of each of the plurality of lenses 101 may be different for each color filter 131 of the same color (see FIG. 37 ). This makes it possible to accurately correct the sensitivity by changing the refractive index of the lens 101 for each color for pixel pairs 90 that do not cause a sensitivity difference between the same colors, thereby achieving improved image quality.

[0221] 4. Other Embodiments The configurations and processes according to the above-described embodiments (including examples and modified examples) may be implemented in various different forms other than the above-described embodiments. For example, the configurations and processes are not limited to the above-described examples and may be implemented in various forms. Furthermore, for example, the configurations, processing procedures, specific names, or information including various data and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified.

[0222] Furthermore, the components and processes according to the above-described embodiments (including examples and modifications) do not necessarily have to be physically configured as shown in the drawings. In other words, the specific form of distribution and integration of the components and processes is not limited to that shown in the drawings, and all or part of them may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0223] Furthermore, the configurations and processes of the above-described embodiments (including examples and modified examples) may be combined as appropriate. For example, at least a part of an embodiment may be combined as appropriate with at least a part of another embodiment. Furthermore, the effects of the embodiments are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0224] 5. Application Examples An application example of the imaging device 1 according to this embodiment (including modifications) will be described with reference to Fig. 43. Fig. 43 is a diagram showing an application example in which the imaging device 1 according to this embodiment is used.

[0225] The imaging device 1 described above can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as follows. For example, as shown in FIG. 43 , the imaging device 1 described above can be used in a variety of applications, including "devices for capturing images for viewing, such as digital cameras and portable devices with camera functions," "devices for traffic applications, such as in-vehicle sensors for capturing images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and for recognizing the driver's state, surveillance cameras for monitoring moving vehicles and roads, and distance measuring sensors for measuring distances between vehicles," "devices for home appliances, such as TVs, refrigerators, and air conditioners, for capturing images of user gestures and operating the device in accordance with the gestures," "devices for medical and healthcare applications, such as endoscopes and devices for capturing blood vessel images by receiving infrared light," "devices for security applications, such as surveillance cameras for crime prevention and cameras for person authentication," "devices for beauty applications, such as skin measuring devices for capturing images of the skin and microscopes for capturing images of the scalp," "devices for sports applications, such as action cameras and wearable cameras for sports applications," and "devices for agricultural applications, such as cameras for monitoring the condition of fields and crops."

[0226] The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as an electronic device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor). Furthermore, for example, the technology according to the present disclosure may be realized as an electronic device mounted on an endoscopic surgery system, a microsurgery system, or the like.

[0227] <5-1. Imaging Device> An imaging device 300 according to an application example will be described with reference to Fig. 44. Fig. 44 is a diagram showing an example of the configuration of the imaging device 300 according to an application example. This imaging device 300 is an example of an electronic device to which the above-described imaging device 1 is applied. Examples of the imaging device 300 include electronic devices such as digital still cameras, video cameras, smartphones and mobile phones with imaging functions.

[0228] 44, the imaging device 300 includes an optical system 301, a shutter device 302, an imaging element (solid-state imaging device) 303, a control circuit (drive circuit) 304, a signal processing circuit 305, a monitor 306, and a memory 307. The imaging device 300 is capable of capturing still images and moving images.

[0229] The optical system 301 includes one or more lenses. The optical system 301 guides light from a subject (incident light) to the image sensor 303, and forms an image on the light receiving surface of the image sensor 303.

[0230] The shutter device 302 is disposed between the optical system 301 and the image sensor 303. The shutter device 302 controls the light irradiation period and the light blocking period for the image sensor 303 under the control of the control circuit 304.

[0231] The image sensor 303 accumulates signal charges for a certain period of time in response to light that is focused on the light receiving surface via the optical system 301 and the shutter device 302. The signal charges accumulated in the image sensor 303 are transferred in accordance with a drive signal (timing signal) supplied from the control circuit 304. The image sensor 303 may be, for example, the image sensor 1 described above.

[0232] The control circuit 304 outputs a drive signal that controls the transfer operation of the image sensor 303 and the shutter operation of the shutter device 302 , thereby driving the image sensor 303 and the shutter device 302 .

[0233] The signal processing circuit 305 performs various signal processing on the signal charges output from the image sensor 303. The image (image data) obtained by the signal processing performed by the signal processing circuit 305 is supplied to a monitor 306 and also to a memory 307.

[0234] The monitor 306 displays moving or still images captured by the image sensor 303 based on the image data supplied from the signal processing circuit 305. As the monitor 306, for example, a panel-type display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel is used.

[0235] The memory 307 stores image data supplied from the signal processing circuit 305 , that is, image data of moving images or still images captured by the image sensor 303 .

[0236] In the imaging device 300 configured as above, by applying the imaging device 1 described above to the imaging element 303, it is possible to achieve an improvement in image quality.

[0237] <5-2. Distance Measuring Device> A distance measuring device 400 according to an application example will be described with reference to Fig. 45. Fig. 45 is a diagram showing an example of the configuration of the distance measuring device 400 according to an application example. This distance measuring device 400 is an example of an electronic device to which the above-described imaging device 1 is applied.

[0238] 45, a distance measuring device (distance image sensor) 400 includes a light source unit 401, an optical system 402, an image sensor (solid-state image sensor) 403, a control circuit (drive circuit) 404, a signal processing circuit 405, a monitor 406, and a memory 407. This distance measuring device 400 projects light from the light source unit 401 toward an object and receives light (modulated light or pulsed light) reflected from the surface of the object, thereby obtaining a distance image according to the distance to the object.

[0239] The light source unit 401 projects light toward the subject. For example, a vertical cavity surface emitting laser (VCSEL) array that emits laser light as a surface light source, or a laser diode array in which laser diodes are arranged in a line, is used as the light source unit 401. The laser diode array is supported by a predetermined drive unit (not shown) and scanned in a direction perpendicular to the direction in which the laser diodes are arranged.

[0240] The optical system 402 includes one or more lenses. The optical system 402 guides light from a subject (incident light) to the image sensor 403, and forms an image on the light receiving surface (sensor portion) of the image sensor 403.

[0241] The image sensor 403 accumulates signal charges in response to light that is imaged on the light receiving surface via the optical system 402. A distance signal indicating a distance determined from a light receiving signal (APD OUT) output from the image sensor 403 is supplied to a signal processing circuit 405. The image sensor 403 may be, for example, the image sensor 1 described above.

[0242] The control circuit 404 outputs a drive signal (control signal) that controls the operation of the light source unit 401, the image sensor 403, etc., and drives the light source unit 401, the image sensor 403, etc.

[0243] The signal processing circuit 405 performs various signal processing on the distance signal supplied from the image sensor 403. For example, the signal processing circuit 405 performs image processing (e.g., histogram processing, peak detection processing, etc.) to construct a distance image based on the distance signal. The image (image data) obtained by the signal processing performed by the signal processing circuit 405 is supplied to a monitor 406 and also to a memory 407.

[0244] The monitor 406 displays the distance image captured by the image sensor 403 based on the image data supplied from the signal processing circuit 405. The monitor 406 may be, for example, a panel display device such as a liquid crystal panel or an organic EL panel.

[0245] The memory 407 stores the image data supplied from the signal processing circuit 405 , that is, the image data of the distance image captured by the image sensor 403 .

[0246] In the distance measuring device 400 configured as above, by applying the imaging device 1 described above to the imaging element 403, it is possible to achieve an improvement in image quality.

[0247] As described above, the imaging device 1 according to this embodiment can be implemented in various electronic devices. For example, the imaging device 1 described above may be installed in various electronic devices such as a notebook PC (Personal Computer), a mobile device (e.g., a smartphone or a tablet PC), a PDA (Personal Digital Assistant), a wearable device, a game device, or a music device, in addition to the imaging device 300 or the distance measuring device 400.

[0248] <6. Supplementary Notes> Note that the present technology may also have the following configurations. (1) An imaging device including a pixel array unit having a plurality of pixels each including a color filter, wherein the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color, wherein the pixel array unit has a lens provided in common to the first pixel and the second pixel, wherein the lens has a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and wherein the first lens region and the second lens region have different refractive indices. (2) The imaging device according to (1), wherein the first lens region and the second lens region have the same planar area. (3) The imaging device according to (1), wherein the first lens region and the second lens region have different planar areas. (4) The imaging device according to any one of (1) to (3), wherein the planar areas of the first lens region and the second lens region are changed depending on the position of the lens with respect to the pixel array unit. (5) The imaging device according to (4), wherein the refractive index of the first lens region is higher than the refractive index of the second lens region, and wherein the planar area of ​​the first lens region increases as the lens is positioned closer to the outer periphery of the pixel array unit. (6) The imaging device according to any one of (1) to (5), wherein the planar shapes of the first lens region and the second lens region are the same. (7) The imaging device according to any one of (1) to (5), wherein the planar shapes of the first lens region and the second lens region are different. (8) The imaging device according to any one of (1) to (7), wherein the planar shapes of the first lens region and the second lens region are changed depending on the position of the lens with respect to the pixel array unit. (9) The imaging device according to any one of (1) to (8), wherein the lens has a plurality of lens regions including the first lens region and the second lens region, and wherein the refractive indices of three or more of the plurality of lens regions are different.(10) The imaging device according to (9), wherein the plurality of lens regions are provided according to the arrangement of a plurality of pixels present around the first pixel and the second pixel among the plurality of pixels. (11) The imaging device according to (9) or (10), wherein the plurality of lens regions are divided so that the lenses are divided radially in a planar view. (12) The imaging device according to any one of (1) to (11), wherein the lenses are formed of a plurality of materials having different refractive indices. (13) The imaging device according to (12), wherein the plurality of materials include materials with different composition ratios. (14) The imaging device according to any one of (1) to (13), wherein the first pixel and the second pixel are adjacent to each other. (15) The imaging device according to any one of (1) to (14), wherein the plurality of pixels are arranged using Deca-Octa Bayer Coding (DOBC). (16) The imaging device according to any one of (1) to (15), wherein the first pixel and the second pixel have a sensitivity difference between the same colors, and the refractive indexes of the first lens region and the second lens region are set to suppress the sensitivity difference between the same colors. (17) The imaging device according to any one of (1) to (16), wherein the plurality of pixels include a third pixel and a fourth pixel having color filters of the same color, the pixel array unit has a second lens provided in common to the third pixel and the fourth pixel, the second lens has a third lens region corresponding to the third pixel and a fourth lens region corresponding to the fourth pixel, and the refractive indexes of the first lens region, the second lens region, the third lens region, and the fourth lens region are different from each other. (18) The imaging device according to any one of (1) to (17), wherein the plurality of pixels include a plurality of pixel pairs each having a pair of pixels each having a color filter of the same color, the pixel array unit has a plurality of lenses provided for each of the pixel pairs, and the refractive index of each of the plurality of lenses is different for each of the color filters of the same color.(19) An electronic device comprising: an imaging device; the imaging device comprising: a pixel array unit having a plurality of pixels each including a color filter; the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color; the pixel array unit having a lens provided in common to the first pixel and the second pixel; the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel; and the first lens region and the second lens region having different refractive indices. (20) A method for manufacturing an imaging device, comprising forming a pixel array unit having a plurality of pixels each including a color filter, wherein the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color, the pixel array unit having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices. (21) An electronic device comprising the imaging device according to any one of (1) to (18). (22) A method for manufacturing an imaging device, comprising manufacturing the imaging device according to any one of (1) to (18).

[0249] REFERENCE SIGNS LIST 1 Imaging device 11 Pixel array section 12 Driving section 13 Reference signal generating section 15 Signal processing section 16 Image data generating section 17 Phase difference data generating section 18 Imaging control section 20 Readout section 21 Constant current source 22 Capacitor element 23 Capacitor element 24 Comparator circuit 25 Counter 26 Latch 27 Transfer control section 90 Pixel pair 100 Pixel block 100R Pixel block 100Gr Pixel block 100Gb Pixel block 100B Pixel block 101 Lens 101a Lens area 101b Lens area 101c Lens area 111 Semiconductor substrate 111S1 Front surface 111S2 Back surface 112 Light receiving section 113 Separation section 121 Multilayer wiring layer 122 Wiring layer 123 Wiring layer 124 Wiring layer 125 Wiring layer 126 Wiring layer 127 Wiring layer 128 Interlayer insulating layer 131 Color filter 132 Light-shielding film 300 Imaging device 400 Range-finding device L1 Light L2 Light P Light-receiving pixel PB Light-receiving pixel PD Photodiode PGb Light-receiving pixel PGr Light-receiving pixel PR Light-receiving pixel U Pixel unit

Claims

1. An imaging device comprising: a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section having a lens provided in common to the first pixel and the second pixel, the lens having a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel, and the first lens area and the second lens area having different refractive indices.

2. The imaging device according to claim 1, wherein the planar areas of the first lens area and the second lens area are the same.

3. The imaging device according to claim 1, wherein the first lens area and the second lens area have different planar areas.

4. The imaging device according to claim 1, wherein the planar areas of the first lens region and the second lens region are changed depending on the position of the lens relative to the pixel array section.

5. The imaging device according to claim 4, wherein the refractive index of the first lens region is higher than the refractive index of the second lens region, and the planar area of ​​the first lens region becomes larger as the lens is positioned closer to the outer periphery of the pixel array section.

6. The imaging device according to claim 1, wherein the first lens area and the second lens area have the same planar shape.

7. The imaging device according to claim 1, wherein the first lens region and the second lens region have different planar shapes.

8. The imaging device according to claim 1, wherein the planar shapes of the first lens region and the second lens region are changed depending on the position of the lens relative to the pixel array section.

9. The imaging device according to claim 1, wherein the lens has a plurality of lens regions including the first lens region and the second lens region, and the refractive indices of any three or more of the plurality of lens regions are different from each other.

10. The imaging device according to claim 9, wherein the plurality of lens regions are provided according to an arrangement of a plurality of pixels present around the first pixel and the second pixel among the plurality of pixels.

11. The imaging device according to claim 9, wherein the plurality of lens regions are divided so that the lenses are radially separated in a plan view.

12. The imaging device according to claim 1, wherein the lens is formed from a plurality of materials having different refractive indices.

13. The imaging device according to claim 12, wherein the plurality of materials include materials having different composition ratios.

14. The imaging device of claim 1, wherein the first pixel and the second pixel are adjacent to each other.

15. The imaging device according to claim 1, wherein the plurality of pixels are arranged in a Deca-Octa Bayer Coding (DOBC) format.

16. The imaging device according to claim 1, wherein the first pixel and the second pixel have a sensitivity difference between the same colors, and the refractive indexes of the first lens region and the second lens region are set so as to suppress the sensitivity difference between the same colors.

17. The imaging device of claim 1, wherein the plurality of pixels include a third pixel and a fourth pixel each having a color filter of the same color, the pixel array portion has a second lens provided in common to the third pixel and the fourth pixel, the second lens has a third lens area corresponding to the third pixel and a fourth lens area corresponding to the fourth pixel, and the first lens area, the second lens area, the third lens area and the fourth lens area have different refractive indices.

18. The imaging device of claim 1, wherein the plurality of pixels include a plurality of pixel pairs each having a pair of pixels each having a color filter of the same color, the pixel array section has a plurality of lenses provided for each of the pixel pairs, and the refractive index of each of the plurality of lenses is different for each of the color filters of the same color.

19. An electronic device comprising: an imaging device, the imaging device comprising a pixel array portion having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array portion having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices.

20. A method for manufacturing an imaging device, comprising: forming a pixel array portion having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array portion having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices.

Images