Imaging apparatus, electronic apparatus, and method for manufacturing imaging apparatus

Abstract

To achieve the improvement of image quality.SOLUTION: An imaging apparatus includes a pixel array part having a plurality of pixels each including a color filter. The plurality of pixels include a first pixel and a second pixel each having a color filter of the same color. The pixel array part has a lens provided in common in the first pixel and the second pixel. The lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel. The respective refractive indexes of the first lens area and the second lens area are different from each other.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an imaging device, an electronic device, and a method for manufacturing an imaging device. [Background technology]

[0002] Some imaging devices obtain an image plane phase difference to realize autofocus, for example. For example, Patent Document 1 discloses an imaging device in which one lens is provided at a position corresponding to a pixel pair including two light receiving pixels. DOBC (Deca-Octa Bayer Coding), which is one of the pixel arrangements of the imaging device, is different from conventional QBC (Quad Bayer Coding) in that, as in Patent Document 1, a pixel block including 10 green pixels, a pixel block including 8 red pixels, and a pixel block including 8 blue pixels are repeatedly arranged. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 130888 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in an array such as the DOBC described above, the difference in refractive index between color filters causes a larger difference in sensitivity between pixels using color filters of the same color, which means that the sensitivity varies from pixel to pixel, compared to conventional arrays such as QBC. This causes color differences even in pixels of the same color, resulting in reduced image quality.

[0005] In view of this, the present disclosure proposes an imaging device, an electronic device, and a method for manufacturing an imaging device that can achieve improved image quality. [Means for solving the problem]

[0006] An imaging device according to one embodiment of the present disclosure includes a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel, and the first lens area and the second lens area have different refractive indices.

[0007] An electronic device according to one embodiment of the present disclosure includes an imaging device, the imaging device including a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section having a lens provided in common to the first pixel and the second pixel, the lens having a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel, and the first lens area and the second lens area having different refractive indices.

[0008] A manufacturing method for an imaging device according to one embodiment of the present disclosure includes forming a pixel array section having a plurality of pixels each including a color filter, the plurality of pixels including a first pixel and a second pixel each having a color filter of the same color, the pixel array section having a lens provided in common to the first pixel and the second pixel, the lens having a first lens region corresponding to the first pixel and a second lens region corresponding to the second pixel, and the first lens region and the second lens region having different refractive indices. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment of the present disclosure. [Diagram 2] 1 is a plan view illustrating a configuration example of a pixel array unit according to an embodiment of the present disclosure. [Diagram 3]1 is a cross-sectional view showing an example of a cross-sectional structure of a pixel array unit according to an embodiment of the present disclosure. [Figure 4] 1 is a cross-sectional view showing an example of a cross-sectional structure of a pixel array unit according to a comparative example of an embodiment of the present disclosure. [Diagram 5] 10A and 10B are diagrams for explaining sensitivity differences between the same colors in a pixel array unit according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a diagram showing a pixel array of a 1×1 QBC. [Figure 7] FIG. 1 is a diagram showing a pixel array of a 2×2 QBC. [Figure 8] FIG. 1 is a diagram showing a pixel arrangement of a DOBC. [Figure 9] 5 is a diagram showing a difference in sensitivity between light receiving pixels in a pixel array section according to an embodiment of the present disclosure. FIG. [Figure 10] 1 is a plan view showing an example of an arrangement of lenses in a pixel array section according to an embodiment of the present disclosure. [Figure 11] FIG. 13 is a diagram illustrating an example of the configuration of a green pixel block according to an embodiment of the present disclosure. [Figure 12] FIG. 2 is a diagram illustrating an example of the configuration of a red pixel block according to an embodiment of the present disclosure. [Figure 13] FIG. 2 is a diagram illustrating an example of a connection between a plurality of pixel blocks according to an embodiment of the present disclosure. [Figure 14] FIG. 2 is a diagram illustrating a configuration example of a reading unit according to an embodiment of the present disclosure. [Figure 15] FIG. 2 is a diagram illustrating an example of an image signal according to an embodiment of the present disclosure. [Figure 16] 1 is a diagram illustrating an example of the number of effective pixels in an imaging device according to an embodiment of the present disclosure. [Figure 17] 1A to 1C are diagrams for explaining an example of a zoom operation in an imaging device according to an embodiment of the present disclosure. [Figure 18] 4A to 4C are diagrams illustrating an example of the operation of an imaging device in a first imaging mode according to an embodiment of the present disclosure. [Figure 19] 11A and 11B are diagrams illustrating an example of a readout operation in the imaging device according to an embodiment of the present disclosure. [Figure 20]6A to 6C are diagrams illustrating an example of image processing by a signal processing unit in a first imaging mode according to an embodiment of the present disclosure. [Figure 21] 11A to 11C are diagrams illustrating an example of the operation of the imaging device in a second imaging mode according to an embodiment of the present disclosure. [Figure 22] 11A to 11C are diagrams illustrating in more detail an example of the operation of the imaging device in the second imaging mode according to the embodiment of the present disclosure. [Figure 23] 11 is a diagram illustrating an example of image processing by a signal processing unit in a second imaging mode according to an embodiment of the present disclosure. FIG. [Figure 24] 13A to 13C are diagrams illustrating an example of a re-mosaic process in a second imaging mode according to an embodiment of the present disclosure. [Diagram 25] FIG. 13 is a diagram illustrating a re-mosaic process for a first pixel block in a second imaging mode according to an embodiment of the present disclosure. [Figure 26] 13A to 13C are diagrams illustrating re-mosaic processing for a second pixel block and a third pixel block in a second imaging mode according to an embodiment of the present disclosure. [Figure 27] FIG. 13 is a diagram illustrating a re-mosaic process for a fourth pixel block in the second imaging mode according to the embodiment of the present disclosure. [Figure 28] 13A to 13C are diagrams illustrating an example of the operation of the imaging device in a third imaging mode according to an embodiment of the present disclosure. [Figure 29] 13A to 13C are diagrams illustrating in more detail an example of operation of the imaging device in the third imaging mode according to the embodiment of the present disclosure. [Diagram 30] 11A and 11B are diagrams illustrating an example of another readout operation in the imaging device according to the embodiment of the present disclosure. [Diagram 31] 13A to 13C are diagrams illustrating an example of image processing by a signal processing unit in a third imaging mode according to an embodiment of the present disclosure. [Diagram 32] 13A to 13C are diagrams illustrating an example of a re-mosaic process in a third imaging mode according to an embodiment of the present disclosure. [Diagram 33] FIG. 13 is a diagram illustrating a re-mosaic process for a first pixel block in a third imaging mode according to an embodiment of the present disclosure. [Diagram 34] 13A to 13C are diagrams illustrating a second pixel block and a re-mosaic process for the second pixel block in the third imaging mode according to an embodiment of the present disclosure. [Diagram 35] FIG. 13 is a diagram illustrating a re-mosaic process for a fourth pixel block in the third imaging mode according to an embodiment of the present disclosure. [Diagram 36] FIG. 11 is a plan view for explaining the refractive index of each lens in a pixel array portion according to the first modified example of the embodiment of the present disclosure. [Figure 37] FIG. 11 is a plan view for explaining the refractive index of each lens in a pixel array portion according to a second modified example of the embodiment of the present disclosure. [Figure 38] FIG. 11 is a plan view for explaining the refractive index of each lens in a pixel array portion according to a third modified example of the embodiment of the present disclosure. [Figure 39] 13 is a diagram illustrating an example of a cross-sectional structure of a pixel array unit according to a third modified example of the embodiment of the present disclosure. FIG. [Diagram 40] FIG. 11 is a plan view for explaining the refractive index of each lens in a pixel array portion according to a fourth modified example of the embodiment of the present disclosure. [Diagram 41] FIG. 13 is a plan view showing a desired position of a lens in a pixel array portion according to a fifth modified example of the embodiment of the present disclosure. [Diagram 42] 13 is a diagram for explaining a desired refractive index of a lens according to Modification 5 of the embodiment of the present disclosure. FIG. [Diagram 43] 1A to 1C are diagrams illustrating application examples using an imaging device according to an embodiment of the present disclosure. [Diagram 44] FIG. 13 is a diagram illustrating a configuration example of an imaging device according to an application example. [Diagram 45] FIG. 13 is a diagram illustrating a configuration example of a distance measuring device according to an application example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the embodiments do not limit the devices, equipment, methods, etc. related to the present disclosure. In addition, in the following embodiments, the same reference numerals are used to basically designate the same parts, and redundant explanations will be omitted. The following embodiments also include examples and modified examples.

[0011] One or more of the following embodiments can be implemented independently. On the other hand, at least a part of the following embodiments may be implemented in appropriate combination with at least a part of other embodiments. These embodiments may include novel features different from each other. Therefore, each embodiment may contribute to solving different purposes or problems, and may have different effects. Note that the effects in each embodiment are merely examples and are not limited to the above, and other effects may also be provided.

[0012] The present disclosure will be described in the following order. 1. Embodiment 1-1. Example of imaging device configuration 1-2. Example of pixel array configuration 1-3. Example of pixel block configuration 1-4. Example of the readout section configuration 1-5. Example of imaging device operation 1-6. Imaging mode MA (first imaging mode) 1-7. Imaging mode MB (second imaging mode) 1-8. Imaging mode MC (third imaging mode) 2. Variations 2-1. Variation 1 2-2. Variation 2 2-3. Variation 3 2-4. Variation 4 2-5. Variation 5 3. Actions and Effects 4. Other embodiments 5. Application Examples 5-1. Imaging device 5-2. Distance measuring device 6. Notes

[0013] <1. Embodiment> <1-1. Configuration example of imaging device> An example of the configuration of an imaging device 1 according to this embodiment will be described with reference to FIG.

[0014] FIG. 1 is a diagram showing an example of the configuration of an imaging device 1 according to this embodiment.

[0015] 1, the imaging device 1 includes a pixel array section 11, a driving section 12, a readout section 20, a signal processing section 15, and an imaging control section 18. The imaging device 1 is, for example, a solid-state imaging device that generates image data of a subject.

[0016] The pixel array section 11 has a plurality of light receiving pixels (pixels) P. These light receiving pixels P are arranged, for example, in a matrix. The light receiving pixels P are configured to generate a signal SIG including a pixel voltage Vpix according to the amount of received light.

[0017] The drive unit 12 is configured to drive a plurality of light receiving pixels P in the pixel array unit 11 based on an instruction from the imaging control unit 18. A plurality of control lines TRGL, RSTL, and SELL are provided in the pixel array unit 11. Specifically, the drive unit 12 drives a plurality of light receiving pixels P in the pixel array unit 11 by supplying a plurality of control signals STRG to the plurality of control lines TRGL, a plurality of control signals SRST to the plurality of control lines RSTL, and a plurality of control signals SSEL to the plurality of control lines SELL (described in detail later).

[0018] The reference signal generating unit 13 is configured to generate a reference signal RAMP based on an instruction from the imaging control unit 18. The reference signal RAMP has a so-called ramp waveform in which the voltage level gradually changes over time during the period (P phase period TP and D phase period TD) in which the readout unit 20 performs AD conversion. The reference signal generating unit 13 is configured to supply such a reference signal RAMP to the readout unit 20.

[0019] The readout unit 20 is configured to generate an image signal Spic0 by performing AD conversion based on a signal SIG supplied from the pixel array unit 11 via a signal line VSL based on instructions from the imaging control unit 18 (details will be described later).

[0020] The signal processing unit 15 is configured to generate an image signal Spic by performing a predetermined signal processing based on the image signal Spic0 and an instruction from the imaging control unit 18. The signal processing unit 15 has an image data generation unit 16 and a phase difference data generation unit 17. The image data generation unit 16 is configured to generate image data DP indicating a captured image by performing a predetermined image processing based on the image signal Spic0. The phase difference data generation unit 17 is configured to generate phase difference data DF indicating an image plane phase difference by performing a predetermined image processing based on the image signal Spic0. The signal processing unit 15 generates an image signal Spic including the image data DP generated by the image data generation unit 16 and the phase difference data DF generated by the phase difference data generation unit 17.

[0021] The imaging control unit 18 is configured to supply control signals to the driving unit 12, the reference signal generating unit 13, the readout unit 20, and the signal processing unit 15, and to control the operations of these circuits, thereby controlling the operation of the imaging device 1. A control signal Sctl is supplied to the imaging control unit 18 from an external device. This control signal Sctl includes, for example, information on the zoom magnification of so-called electronic zoom. The imaging control unit 18 is configured to control the operation of the imaging device 1 based on the control signal Sctl.

[0022] <1-2. Example of pixel array configuration> An example of the configuration of the pixel array unit 11 according to this embodiment will be described with reference to FIGS. 2 to 11. FIG.

[0023] Fig. 2 is a plan view showing an example of the configuration of the pixel array section 11 according to this embodiment. Fig. 3 is a cross-sectional view showing an example of the cross-sectional structure of the pixel array section 11 according to this embodiment. Fig. 3 shows a cross section corresponding to the line Y1-Y1 shown in Fig. 2.

[0024] As shown in Fig. 2, the pixel array section 11 has a plurality of pixel units U and a plurality of lenses 101. In the example of Fig. 2, one pixel unit U is clearly shown, but in reality there are a plurality of pixel units U.

[0025] Each pixel unit U includes a pixel block 100R, a pixel block 100Gr, a pixel block 100Gb, and a pixel block 100B. In the pixel array section 11, the multiple light receiving pixels P are arranged with the four pixel blocks 100R, 100Gr, 100Gb, and 100B as a minimum repeating unit (pixel unit U). When there is no need to distinguish between the four pixel blocks 100R, 100Gr, 100Gb, and 100B, they may be simply referred to as pixel blocks 100.

[0026] The pixel block 100R has, for example, eight light receiving pixels P (light receiving pixels PR) each including a red (R) color filter 131 (see FIG. 3). The pixel block 100Gr has, for example, ten light receiving pixels P (light receiving pixels PGr) each including a green (G) color filter 131. The pixel block 100Gb has, for example, ten light receiving pixels P (light receiving pixels PGb) each including a green (G) color filter 131. The pixel block 100B has, for example, eight light receiving pixels P (light receiving pixels PB) each including a blue (B) color filter 131. Note that when there is no need to distinguish between the four light receiving pixels PR, PGr, PGb, and PB, they may simply be referred to as light receiving pixels P.

[0027] In the example of FIG. 2, the difference in color of the color filter 131 is expressed by using shading. The arrangement pattern of the light receiving pixels PR in the pixel block 100R and the arrangement pattern of the light receiving pixels PB in the pixel block 100B are the same. The arrangement pattern of the light receiving pixels PGr in the pixel block 100Gr and the arrangement pattern of the light receiving pixels PGb in the pixel block 100Gb are the same. In the pixel block 100R or the pixel block 100B, 2, 4, and 2 light receiving pixels PR or light receiving pixels PB arranged in parallel in the X-axis direction are arranged in a cross pattern arranged in parallel in the Y-axis direction. Also, in the pixel block 100Gr or the pixel block 100Gb, 4, 2, and 4 light receiving pixels PGr or light receiving pixels PGb arranged in parallel in the X-axis direction are arranged in an H-shaped pattern arranged in parallel in the Y-axis direction.

[0028] In the pixel unit U, the pixel block 100R, the pixel block 100Gr, the pixel block 100Gb, and the pixel block 100B are arranged in, for example, 2 rows x 2 columns. Specifically, the pixel block 100R and the pixel block 100B, and the pixel block 100Gr and the pixel block 100Gb, which have the same arrangement pattern, are arranged on diagonal lines that intersect with each other. As an example, the pixel block 100Gr is arranged in the upper left, the pixel block 100R is arranged in the upper right, the pixel block 100B is arranged in the lower left, and the pixel block 100Gb is arranged in the lower right. In this way, the pixel block 100R, the pixel block 100Gr, the pixel block 100Gb, and the pixel block 100B are arranged in a so-called Bayer array with the pixel block 100 as a unit. Such an array is called, for example, DOBC (Deca-Octa Bayer Coding).

[0029] As shown in FIG. 3, the pixel array section 11 includes a semiconductor substrate 111, a light receiving section 112, a separation section 113, a multilayer wiring layer 121, a color filter 131, and a light shielding film 132.

[0030] The semiconductor substrate 111 is a support substrate on which the imaging device 1 is formed. The semiconductor substrate 111 is, for example, a P-type semiconductor substrate, and has a pair of opposing surfaces (a front surface 111S1 and a back surface 111S2).

[0031] The light receiving section 112 is a semiconductor region embedded in the semiconductor substrate 111 at positions corresponding to the plurality of light receiving pixels P. The semiconductor region is doped with, for example, an N-type impurity to form a photodiode (PD).

[0032] The isolation portion 113 is provided at the boundary between adjacent light receiving pixels P in the XY plane within the semiconductor substrate 111. In the example of Fig. 3, the isolation portion 113 is a DTI (Deep Trench Isolation) formed using an insulating material such as an oxide film, but may have another structure.

[0033] The multilayer wiring layer 121 is provided on a surface 111S1 of the semiconductor substrate 111, which is the surface opposite to the light incident side S of the pixel array section 11. The multilayer wiring layer 121 includes, for example, a plurality of wiring layers 122-127 and an interlayer insulating layer 128. The wiring layers 122-127 are provided in order from the back surface (light incident surface) 111S2 side of the semiconductor substrate 111. In each of the wiring layers 122-127, for example, a plurality of wirings such as control lines that connect a plurality of transistors provided on the surface 111S1 of the semiconductor substrate 111 to the driving section 12, the readout section 20, etc. are provided.

[0034] The color filter 131 is provided on the back surface (light incident surface) 111S2 of the semiconductor substrate 111, which is the light incident side S of the pixel array section 11. The color filter 131 is any one of red (R), green (G), and blue (B) color filters. The color filter 131 may be provided for each light receiving pixel P, or may be provided for each two light receiving pixels P adjacent to each other in the X-axis direction (pixel pair 90). However, in either case, each light receiving pixel P (for example, the light receiving pixel PR, the light receiving pixel PGr, the light receiving pixel PGb, and the light receiving pixel PB) has a color filter 131.

[0035] The light-shielding film 132 is provided on the rear surface (light incident surface) 111S2 of the semiconductor substrate 111, for example, for every two light-receiving pixels P (pixel pair 90) adjacent in the X-axis direction so as to surround the pixel pair 90. The two light-receiving pixels P adjacent in the X-axis direction function as the pixel pair 90 (see FIG. 2).

[0036] Each lens 101 is, for example, an on-chip lens, and is provided on a color filter 131 on the light incident side S of the pixel array section 11. The lens 101 is provided above two light receiving pixels P (pixel pair 90) adjacent to each other in the X-axis direction.

[0037] As shown in FIG. 2, four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100R. Five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gr. Five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gb. Four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100B. The lenses 101 are arranged side by side in the X-axis direction and the Y-axis direction. The lenses 101 arranged in the Y-axis direction are shifted by one light receiving pixel P in the X-axis direction. In other words, the pixel pairs 90 arranged in the Y-axis direction are shifted by one light receiving pixel P in the X-axis direction.

[0038] With this configuration, the images of two light receiving pixels P in a pixel pair 90 corresponding to one lens 101 are shifted from each other. The imaging device 1 generates phase difference data DF based on what is called an image plane phase difference detected by a plurality of pixel pairs 90. For example, a camera equipped with the imaging device 1 determines a defocus amount based on this phase difference data DF, and moves the position of the photographing lens based on this defocus amount. In this way, the camera is able to realize autofocus.

[0039] As shown in FIG. 3, the lens 101 in the specific pixel pair 90 has a lens region 101a and a lens region 101b. The specific pixel pair 90 is, for example, a pixel pair having a sensitivity difference between the same colors. In the example of FIG. 3, the specific pixel pair 90 is a pair of light receiving pixels PGr. The lens region 101a is a first lens region corresponding to one light receiving pixel PGr of the pixel pair 90. The lens region 101a faces one light receiving pixel PGr of the pixel pair 90. The lens region 101b is a second lens region corresponding to the other light receiving pixel PGr of the pixel pair 90. The lens region 101b faces the other light receiving pixel PGr of the pixel pair 90.

[0040] The lens regions 101a and 101b have different refractive indices. In the example of Fig. 3, in a pair of light receiving pixels PGr, the lens regions 101a and 101b have different refractive indices, and the refractive index of the lens region 101a is higher than the refractive index of the lens region 101b. The refractive indices of the lens regions 101a and 101b are set so as to suppress the sensitivity difference between the same colors.

[0041] The lens 101 is formed of a plurality of materials (e.g., high refractive index material and low refractive index material) with different refractive indexes so that the lens region 101a and the lens region 101b have different refractive indexes. The plurality of materials may include materials with different composition ratios. Examples of materials with different composition ratios include SiO2 and Si3N4, with SiOxNy filling the gaps therebetween. The composition ratio of the materials is determined, for example, according to the color (refractive index) of the color filters 131 of the surrounding light receiving pixels P surrounding the target light receiving pixel P.

[0042] (Comparative Example) FIG. 4 is a cross-sectional view showing an example of a cross-sectional structure of the pixel array section 11 according to a comparative example of this embodiment.

[0043] As shown in FIG. 4, in the comparative example, the refractive index of each lens 101 is the same, and there is no difference in the refractive index within one lens 101. The refractive index of each color filter 131 is, for example, R / G / B=1.72 / 1.67 / 1.49. Normally, light is attracted to the side with a higher refractive index. For example, in the light receiving pixel PGr into which the light L2 in FIG. 4 is incident, the light L2 incident on the color filter 131 travels straight, but in the light receiving pixel PGr into which the light L1 in FIG. 4 is incident, the light L1 incident on the color filter 131 is attracted to the right side and leaks to the adjacent light receiving pixel PR side. In this case, the light receiving pixel PGr into which the light L1 in FIG. 4 is incident has a lower sensitivity than the light receiving pixel PGr into which the light L2 in FIG. 4 is incident.

[0044] In order to suppress this decrease in sensitivity, as described above, the refractive index of the lens region 101a is made higher than that of the lens region 101b. In the example of FIG. 3, in the light receiving pixel PGr to which the light L1 in FIG. 3 is incident, the light L1 incident on the lens 101 is bent to the left and then incident on the color filter 131. After that, the light L1 incident on the color filter 131 is shifted to the right, but leakage to the adjacent light receiving pixel PR is suppressed. Therefore, by making the refractive index of the lens region 101a higher than that of the lens region 101b, it is possible to suppress leakage of light to the adjacent light receiving pixel PR. In other words, by making the refractive index of the lens region 101a higher than that of the lens region 101b, it is possible to suppress light heading toward the adjacent color filter 131.

[0045] Since it is usually difficult to make the refractive index of each color filter 131 uniform, it is desirable to correct the sensitivity with the total refractive index of the lens 101 and the color filter 131. Therefore, the sensitivity can be corrected accurately by adjusting the refractive index of each of the lens area 101a and the lens area 101b. This makes it possible to improve the image quality.

[0046] (sensitivity difference between same colors) Here, the same-color sensitivity difference caused by the influence of the surrounding eight pixels (light-receiving pixels P) surrounding the own pixel (light-receiving pixel P) will be described with reference to Fig. 5 to Fig. 8. For example, the same-color sensitivity difference is a sensitivity difference that occurs according to the individual colors (refractive indexes of the color filters 131) of the multiple light-receiving pixels P that exist in the surroundings surrounding the own pixel.

[0047] FIG. 5 is a diagram for explaining the sensitivity difference between the same colors in the pixel array section 11 according to the present embodiment.

[0048] The light receiving pixel PGr of A1 in FIG. 5 receives light from two light receiving pixels PB and loses the light to one light receiving pixel PR. This is due to the relationship in the refractive index between the light receiving pixel PR, the light receiving pixel PB, and the light receiving pixel PGr (R / G / B=1.72 / 1.67 / 1.49), and the same applies below. The light receiving pixel PGr of A2 in FIG. 5 receives light from one light receiving pixel PB and loses the light to three light receiving pixels PR, so the sensitivity is lower than that of the light receiving pixel PGr of A1 in FIG. 5, and a sensitivity difference between the same colors occurs. For this reason, the refractive index adjustment of the lens 101 described above is performed for the light receiving pixel PGr of A1 in FIG. 5 and the light receiving pixel PGr of A2 in FIG. 5. That is, the refractive index of the lens region 101a corresponding to the light receiving pixel PGr of A2 in FIG. 5 is made higher than the refractive index of the lens region 101b corresponding to the light receiving pixel PGr of A1 in FIG. 5.

[0049] 5, the number of red, green, and blue (R / G / B) pixels surrounding the pixel are the same, and the refractive index of the green color filter 131 is higher than that of the blue color filter 131, so light is attracted to the green side (sensitivity reduction), but no sensitivity difference between the same colors occurs. For this reason, the refractive index adjustment of the lens 101 described above is not performed for the light receiving pixel PB of A3 and the light receiving pixel PB of A4 in FIG.

[0050] In this way, for example, the refractive index of one of the lens regions 101a and 101b is made higher than that of the other according to the color of the light receiving pixel P adjacent to the periphery of the light receiving pixel P (i.e., the refractive index of the color filter 131), and a lens 101 having the lens regions 101a and 101b with different refractive indices is provided. This makes it possible to correct the bending of light due to the refractive index difference of the color filter 131 in advance with the lens and perform sensitivity correction, thereby suppressing the sensitivity difference between the same colors.

[0051] FIG. 6 is a diagram showing a pixel arrangement of a 1×1 QBC. As shown in FIG. 6, when the subject pixel (light receiving pixel P) is R, the number of surrounding 8 pixels (light receiving pixels P) surrounding the subject pixel is R=0, G=4, and B=4. When the subject pixel is G, the number of surrounding 8 pixels surrounding the subject pixel is R=2, G=2, and B=4. When the subject pixel is B, the number of surrounding 8 pixels surrounding the subject pixel is R=4, G=4, and B=0. In the example of FIG. 6, the number of colors of the surrounding 8 pixels surrounding the subject pixel is the same for the same color. Therefore, the sensitivity difference between the same colors originating from the surrounding pixels is qualitatively the same.

[0052] FIG. 7 is a diagram showing a pixel arrangement of a 2×2 QBC. As shown in FIG. 7, when the subject pixel (light receiving pixel P) is R, the number of surrounding 8 pixels (light receiving pixels P) surrounding the subject pixel is R=3, G=4, and B=1. When the subject pixel is G, the number of surrounding 8 pixels surrounding the subject pixel is R=2, G=4, and B=2. When the subject pixel is B, the number of surrounding 8 pixels surrounding the subject pixel is R=1, G=4, and B=3. In the example of FIG. 7, the number of colors of the surrounding 8 pixels surrounding the subject pixel is the same for the same color. Therefore, the sensitivity difference between the same colors originating from the surrounding pixels is qualitatively the same.

[0053] Fig. 8 is a diagram showing a pixel arrangement of DOBC. As shown in Fig. 8, when a pixel (light receiving pixel P) is R, G, or B, the number of each color in the surrounding eight pixels (light receiving pixels P) surrounding the pixel is not constant, but differs depending on the pixel. In other words, in the case of DOBC, even pixels of the same RGB have different numbers of each color in the surrounding eight pixels, so that a sensitivity difference between the same colors occurs in the arrangement. For this reason, the refractive index of the lens 101 described above is adjusted for the pixel arrangement of DOBC.

[0054] Fig. 9 is a diagram showing the sensitivity difference for each light receiving pixel P in the pixel array section 11 according to this embodiment. Fig. 10 is a plan view showing an example of the arrangement of the lenses 101 in the pixel array section 11 according to this embodiment. In the example of Fig. 9, the sensitivity difference is shown in %.

[0055] 9, in the pixel array of the DOBC, there is a sensitivity difference between each light receiving pixel P. Also, there are a pair of light receiving pixels P (pixel pairs 90) that have the same sensitivity difference, and a pair of light receiving pixels P (pixel pairs 90) that have different sensitivity differences. Lenses 101 are provided according to whether these sensitivity differences match or mismatch.

[0056] For example, lenses 101 with no refractive index difference are provided for a pair of light receiving pixels P (pixel pair 90) having the same sensitivity difference as shown in Fig. 10. On the other hand, lenses 101 with a refractive index difference (lens 101 having lens area 101a and lens area 101b) are provided for a pair of light receiving pixels P (pixel pair 90) having different sensitivity differences as shown in Fig. 9.

[0057] In the pixel pair 90, the refractive index of the lens region 101a corresponding to the light receiving pixel P with a large sensitivity difference is set higher than the refractive index of the lens region 101b corresponding to the light receiving pixel P with a small sensitivity difference. For example, in the pixel pair 90, the refractive index of the lens region 101a corresponding to the light receiving pixel P with a sensitivity difference of 12% is set higher than the refractive index of the lens region 101b corresponding to the light receiving pixel P with a sensitivity difference of 11%. This makes it possible to suppress the sensitivity difference between each light receiving pixel P in the pixel pair 90, i.e., to suppress the sensitivity difference between the same colors.

[0058] In the example of Fig. 10, the lens 101 is divided into two in a plan view, and the planar areas (areas in a plan view) of the lens regions 101a and 101b are the same. The planar shapes (shapes in a plan view) of the lens regions 101a and 101b are half ellipses. The lens region 101a functions as a high refractive region, and the lens region 101b functions as a low refractive region.

[0059] In order to improve the sensitivity difference between the same colors in the pixel array of DOBC, it is possible to perform correction by signal processing or aperture correction, but correction by signal processing does not improve the physical optical characteristics. Furthermore, in aperture correction, it becomes more difficult to control the aperture width as the light receiving pixel P shrinks. Therefore, the lens 101 is formed using a lens material with a different refractive index according to the refractive index of the color filter 131 of the light receiving pixel P adjacent to the periphery of the own pixel (light receiving pixel P). This makes it possible to correct the bending of light due to the refractive index difference of the color filter 131 in advance with the lens 101 and perform sensitivity correction, thereby suppressing the sensitivity difference between the same colors.

[0060] <1-3. Example of pixel block configuration> An example of the configuration of each pixel block 100Gr, 100Gb, 100R, and 100B according to this embodiment will be described with reference to Figs. 11 to 13. Fig. 11 is a diagram showing an example of the configuration of a green pixel block 100Gr according to this embodiment. Fig. 12 is a diagram showing an example of the configuration of a red pixel block 100R according to this embodiment. Fig. 13 is a diagram showing an example of a connection between a plurality of pixel blocks 100R, 100Gr, 100Gb, and 100B according to this embodiment.

[0061] As shown in FIGS. 11 and 12, the pixel array section 11 has a plurality of control lines TRGL, a plurality of control lines RSTL, a plurality of control lines SELL, and a plurality of signal lines VSL.

[0062] The control line TRGL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12 (see FIG. 1). A control signal STRG is supplied to this control line TRGL by the drive unit 12. The control line RSTL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12. A control signal SRST is supplied to this control line RSTL by the drive unit 12. The control line SELL extends, for example, in the X-axis direction, and one end is connected to the drive unit 12. A control signal SSEL is supplied to this control line SELL by the drive unit 12. The signal line VSL extends, for example, in the Y-axis direction, and one end is connected to the readout unit 20. This signal line VSL transmits a signal SIG generated by the light receiving pixel P to the readout unit 20 (see FIG. 1).

[0063] (Green pixel block) As shown in FIG. 11, a pixel block 100Gr has, for example, ten photodiodes PD, ten transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors.

[0064] The ten photodiodes PD and the ten transistors TRG correspond to the ten light receiving pixels PGr included in the pixel block 100Gr. Each of the transistors TRG, RST, AMP, and SEL is, for example, an N-type MOS (Metal Oxide Semiconductor) transistor. The floating diffusion layer FD and each of the transistors TRG, RST, AMP, and SEL are provided on the surface 111S1 of the semiconductor substrate 111.

[0065] The photodiode PD is a photoelectric conversion element that generates an amount of charge according to the amount of light received and accumulates the generated charge inside. The anode of the photodiode PD is grounded, and the cathode is connected to the source of the transistor TRG.

[0066] The transistor TRG transfers the charge generated in the photodiode PD to the floating diffusion layer FD. The gate of the transistor TRG is connected to a control line TRGL, the source is connected to the cathode of the photodiode PD, and the drain is connected to the floating diffusion layer FD. The gates of the ten transistors TRG are connected to different control lines TRGL among the ten control lines TRGL (in the example of FIG. 11, the control lines TRGL1 to TRGL6, TRGL9 to TRGL12).

[0067] The floating diffusion layer FD is configured to accumulate charges transferred from the photodiode PD via the transistor TRG. The floating diffusion layer FD is configured, for example, using a diffusion layer formed on the surface of a semiconductor substrate.

[0068] The gate of the transistor RST is connected to the control line RSTL, the drain is supplied with the power supply voltage VDD, and the source is connected to the floating diffusion layer FD. The gate of the transistor AMP is connected to the floating diffusion layer FD, the drain is supplied with the power supply voltage VDD, and the source is connected to the drain of the transistor SEL. The gate of the transistor SEL is connected to the control line SELL, the drain is connected to the source of the transistor AMP, and the source is connected to the signal line VSL.

[0069] With this configuration, in the light receiving pixel P, for example, the transistor TRG and the transistor RST are turned on based on the control signal STRG and the control signal SRST, and the charge accumulated in the photodiode PD is discharged. Then, the transistor TRG and the transistor RST are turned off, and an exposure period T is started, and an amount of charge according to the amount of received light is accumulated in the photodiode PD. Then, after the exposure period T ends, the light receiving pixel P outputs a signal SIG including a reset voltage Vreset and a pixel voltage Vpix to the signal line VSL.

[0070] Specifically, first, the transistor SEL is turned on based on the control signal SSEL, and the light receiving pixel P is electrically connected to the signal line VSL. As a result, the transistor AMP is connected to a constant current source (for example, a constant current source 21 described later) of the readout section 20, and operates as a so-called source follower. Then, as described later, in a P-phase (Pre-charge phase) period TP after the voltage of the floating diffusion layer FD is reset by the transistor RST being turned on, the light receiving pixel P outputs a voltage corresponding to the voltage of the floating diffusion layer FD at that time as a reset voltage Vreset. Also, in a D-phase (Data phase) period TD after the charge is transferred from the photodiode PD to the floating diffusion layer FD by the transistor TRG being turned on, the light receiving pixel P outputs a voltage corresponding to the voltage of the floating diffusion layer at that time as a pixel voltage Vpix. The difference voltage between the pixel voltage Vpix and the reset voltage Vreset corresponds to the amount of light received by the light receiving pixel P during the exposure period T. In this manner, the light-receiving pixel P outputs a signal SIG including the reset voltage Vreset and the pixel voltage Vpix to the signal line VSL.

[0071] Similarly to the pixel block 100Gr (see FIG. 11), the pixel block 100Gb has ten photodiodes PD, ten transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors. The ten photodiodes PD and the ten transistors TRG correspond to the ten light receiving pixels PGb included in the pixel block 100Gb, respectively. The gates of the ten transistors TRG are connected to different control lines TRGL among the ten control lines TRGL. The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are provided on the surface 111S1 of the semiconductor substrate 111.

[0072] (Red pixel block) As shown in FIG. 12, the pixel block 100R includes, for example, eight photodiodes PD, eight transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors.

[0073] The eight photodiodes PD and the eight transistors TRG correspond to the eight light receiving pixels PR included in the pixel block 100R. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL (in the example of FIG. 12, the control lines TRGL1, TRGL2, TRGL5 to TRGL10). The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are provided on the surface 111S1 of the semiconductor substrate 111.

[0074] (Blue pixel block) The pixel block 100B has eight photodiodes PD, eight transistors TRG, a floating diffusion layer FD, and three transistors RST, AMP, and SEL as pixel transistors, similar to the pixel block 100R (see FIG. 12). The eight photodiodes PD and the eight transistors TRG correspond to the eight light receiving pixels PB included in the pixel block 100B, respectively. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL. The floating diffusion layer FD and the transistors TRG, RST, AMP, and SEL are provided on the surface 111S1 of the semiconductor substrate 111.

[0075] (Example of connection of each pixel block) As shown in Fig. 13, pixel blocks 100B and 100Gb that are aligned in the same row in the X direction are connected to multiple control lines TRGL out of the same 12 control lines TRGL. Although not shown, pixel blocks 100B and 100Gb that are aligned in the same row in the X direction are connected to one control line RSTL and one control line SELL. As shown in Fig. 13, pixel blocks 100B that are aligned in the same column in the Y direction are connected to one signal line VSL. Similarly, pixel blocks 100Gb that are aligned in the same column in the Y direction are connected to one signal line VSL.

[0076] <1-4. Example of the readout unit configuration> A configuration example of the readout unit 20 according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a diagram showing a configuration example of the readout unit 20 according to this embodiment. In the example of Fig. 14, in addition to the readout unit 20, a reference signal generating unit 13, a signal processing unit 15, and an imaging control unit 18 are shown.

[0077] As shown in FIG. 14, the readout unit 20 includes a plurality of constant current sources 21, a plurality of AD (Analog to Digital) conversion units ADC, and a transfer control unit 27.

[0078] Each of the constant current sources 21 and each of the AD conversion units ADC are provided corresponding to a plurality of signal lines VSL. The constant current source 21 and the AD conversion unit ADC corresponding to one of the signal lines VSL will be described below.

[0079] The constant current source 21 is configured to supply a predetermined current to the corresponding signal line VSL. One end of the constant current source 21 is connected to the corresponding signal line VSL, and the other end is grounded.

[0080] The AD conversion unit ADC has a plurality of capacitive elements 22 and 23, a comparison circuit 24, a counter 25, and a latch 26. This AD conversion unit ADC is configured to perform AD conversion based on a signal SIG on a corresponding signal line VSL.

[0081] One end of the capacitive element 22 is connected to the signal line VSL, and the other end of the capacitive element 22 is connected to the comparison circuit 24. A signal SIG is supplied to one end of the capacitive element 22 connected to the signal line VSL. A reference signal RAMP supplied from the reference signal generating unit 13 is supplied to one end of the capacitive element 23, and the other end of the capacitive element 23 is connected to the comparison circuit 24.

[0082] The comparator circuit 24 is configured to generate a signal CP by performing a comparison operation based on the signal SIG supplied from the light receiving pixel P via the signal line VSL and the capacitive element 22 and the reference signal RAMP supplied from the reference signal generator 13 via the capacitive element 23. The comparator circuit 24 sets an operating point by setting the voltages of the capacitive elements 22 and 23 based on a control signal AZ supplied from the imaging controller 18. Thereafter, the comparator circuit 24 performs a comparison operation in which a reset voltage Vreset included in the signal SIG is compared with the voltage of the reference signal RAMP during the P phase period TP, and performs a comparison operation in which a pixel voltage Vpix included in the signal SIG is compared with the voltage of the reference signal RAMP during the D phase period TD.

[0083] The counter 25 is configured to perform a counting operation of counting pulses of the clock signal CLK supplied from the imaging control unit 18 based on the signal CP supplied from the comparison circuit 24. Specifically, the counter 25 generates a count value CNTP by counting pulses of the clock signal CLK until the signal CP transitions during the P-phase period TP, and outputs the count value CNTP as a digital code having multiple bits. Also, the counter 25 generates a count value CNTD by counting pulses of the clock signal CLK until the signal CP transitions during the D-phase period TD, and outputs the count value CNTD as a digital code having multiple bits.

[0084] The latch 26 is configured to temporarily hold the digital code supplied from the counter 25 and to output the digital code to the bus line BUS based on an instruction from the transfer control unit 27 .

[0085] The transfer control unit 27 is configured to control the latches 26 of the multiple AD conversion units ADC to sequentially output digital codes to the bus wiring BUS based on a control signal CTL supplied from the imaging control unit 18. The readout unit 20 sequentially transfers the multiple digital codes supplied from the multiple AD conversion units ADC to the signal processing unit 15 as an image signal Spic0 using this bus wiring BUS.

[0086] The signal processing unit 15 generates the image signal Spic by performing a predetermined signal processing based on the above-mentioned image signal Spic0 and an instruction from the imaging control unit 18. FIG. 15 is a diagram showing an example of the image signal Spic according to this embodiment. As shown in FIG. 15, the signal processing unit 15 generates the image signal Spic by, for example, alternately arranging image data DP related to a plurality of rows of light receiving pixels P and phase difference data DF related to a plurality of rows of light receiving pixels P. Then, the signal processing unit 15 is configured to output such an image signal Spic.

[0087] <1-5. Operational example of imaging device> An example of the operation of the imaging device 1 according to this embodiment will be described below. First, the overall operation of the imaging device 1 will be described with reference to FIG.

[0088] (Overall operation) In the example of FIG. 1, the driving unit 12 sequentially drives a plurality of light receiving pixels P in the pixel array unit 11 based on an instruction from the imaging control unit 18. The reference signal generating unit 13 generates a reference signal RAMP based on an instruction from the imaging control unit 18. The light receiving pixel P outputs a reset voltage Vreset as a signal SIG in a P phase period TP, and outputs a pixel voltage Vpix according to the amount of received light as a signal SIG in a D phase period TD. The readout unit 20 generates an image signal Spic0 based on the signal SIG supplied from the pixel array unit 11 via a signal line VSL and an instruction from the imaging control unit 18. In the signal processing unit 15, the image data generating unit 16 performs a predetermined image processing based on the image signal Spic0 to generate image data DP indicating a captured image, and the phase difference data generating unit 17 performs a predetermined image processing based on the image signal Spic0 to generate phase difference data DF indicating an image surface phase difference. Then, the signal processing unit 15 generates an image signal Spic including the image data DP and the phase difference data DF. The imaging control unit 18 supplies control signals to the drive unit 12, the reference signal generation unit 13, the readout unit 20, and the signal processing unit 15, and controls the operation of these circuits, thereby controlling the operation of the imaging device 1.

[0089] (Zoom movement) The imaging control unit 18 controls the operation of the imaging device 1 based on a control signal Sctl that includes information about the zoom magnification of the electronic zoom. The zoom operation of the imaging device 1 will be described below.

[0090] Fig. 16 is a diagram showing an example of the number of effective pixels (the number of light receiving pixels P) according to this embodiment when the zoom magnification is changed from 1x to 10x. In the example of Fig. 16, the solid line indicates the number of effective pixels of the imaging device 1. Fig. 17 is a diagram for explaining an example of the zoom operation in the imaging device 1 according to this embodiment. In the example of Fig. 17, (A) shows the operation when the zoom magnification is 1x, (B) shows the operation when the zoom magnification is 2x, and (C) shows the operation when the zoom magnification is 3x.

[0091] 17, the imaging device 1 has three imaging modes MA, MB, and MC. The imaging control unit 18 selects one of the three imaging modes MA, MB, and MC based on information about the zoom magnification included in the control signal Sctl. Specifically, the imaging control unit 18 selects the imaging mode MA when the zoom magnification is less than 2, selects the imaging mode MB when the zoom magnification is equal to or greater than 2 and less than 3, and selects the imaging mode MC when the zoom magnification is equal to or greater than 3.

[0092] In the imaging mode MA (see (A) in FIG. 17), the imaging device 1 obtains four pixel values ​​V (e.g., four pixel values ​​VR, VGr, VGb and VB) in each of the multiple pixel units U. Specific operations will be described later. In this manner, the imaging device 1 generates image data DP by generating four pixel values ​​V for every 36 light receiving pixels P. For example, if the number of light receiving pixels P in the pixel array section 11 is 108 [Mpix], pixel values ​​V for 12 [Mpix] are calculated. As a result, the number of effective pixels becomes 12 [Mpix].

[0093] 16, in such imaging mode MA, when the zoom magnification is increased from 1, the number of effective pixels decreases according to the magnification. Then, when the zoom magnification becomes 2, the imaging mode changes to imaging mode MB. The number of effective pixels when the zoom magnification is 2 is the same as the number of effective pixels when the zoom magnification is 1.

[0094] In the imaging mode MB (see (B) in FIG. 17), the imaging device 1 obtains 16 pixel values ​​V in each of the multiple pixel units U. A specific operation will be described later. In this manner, the imaging device 1 generates image data DP by generating pixel values ​​V at a ratio of 16 for every 36 light receiving pixels P. For example, when the number of light receiving pixels P in the pixel array section 11 is 108 [Mpix], pixel values ​​V for 48 [Mpix] are calculated. In reality, since the zoom magnification is 2x, the imaging range is narrowed to 1 / 4, and the number of effective pixels is 12 [Mpix] (=48 [Mpix] / 4).

[0095] 16, in such an imaging mode MB, when the zoom magnification is increased from 2, the number of effective pixels decreases according to the magnification. Then, when the zoom magnification becomes 3, the imaging mode changes to imaging mode MC. The number of effective pixels when this zoom magnification is 3 is the same as the number of effective pixels when the zoom magnification is 2.

[0096] In the imaging mode MC (see (C) in FIG. 17), the imaging device 1 obtains 36 pixel values ​​V in each of the multiple pixel units U. A specific operation will be described later. In this manner, the imaging device 1 generates image data DP by generating pixel values ​​V at a ratio of 36 for every 36 light receiving pixels P. For example, when the number of light receiving pixels P in the pixel array section 11 is 108 [Mpix], an image of 108 [Mpix] can be obtained. In reality, since the zoom magnification is 3 times, the imaging range is narrowed to 1 / 9, and the number of effective pixels is 12 [Mpix] (=108 [Mpix] / 9).

[0097] In this way, the imaging device 1 is provided with three imaging modes MA, MB, and MC, so that the change in the image quality of the captured image can be reduced when the zoom magnification is changed. That is, for example, when the imaging mode MB is omitted and two imaging modes MA and MC are provided, and the imaging mode MA is selected when the zoom magnification is less than 2 times, and the imaging mode MC is selected when the zoom magnification is 2 times or more, the number of effective pixels changes significantly as shown by the broken line in FIG. 16. That is, in this example, when the zoom magnification is 2 times, the imaging mode MC is selected, and the number of effective pixels is 27 [Mpix] (= 108 [Mpix] / 4). Therefore, there is a large difference between the number of effective pixels when the zoom magnification is, for example, 1.9 times and the number of effective pixels when the zoom magnification is 2 times, so that the image quality of the captured image may change significantly when the zoom magnification is around 2 times. On the other hand, the imaging device 1 is provided with three imaging modes MA, MB, and MC, so that the change in the number of effective pixels can be reduced when the zoom magnification is changed, and the change in the image quality of the captured image can be suppressed.

[0098] <1-6. Imaging mode MA (first imaging mode)> The imaging mode MA according to this embodiment will be described with reference to FIGS.

[0099] Fig. 18 is a diagram showing an example of the operation of the imaging device 1 in the imaging mode MA according to this embodiment. In the example of Fig. 18, the light receiving pixels P marked with "o" indicate the light receiving pixels P that are the target of the readout operation.

[0100] First, as shown in FIG. 18(A), the imaging device 1 generates image data DT1 by calculating a pixel value V corresponding to the amount of light received by the left light receiving pixel P in a pixel pair 90 in which a lens 101 is provided, for each of a plurality of pixel blocks 100.

[0101] Specifically, the imaging device 1 performs a readout operation on five light receiving pixels PGr arranged on the left of five pixel pairs 90 out of the ten light receiving pixels PGr in the pixel block 100Gr, thereby calculating a pixel value VGr1 at the center of gravity of the five light receiving pixels PGr. The imaging device 1 also performs a readout operation on four light receiving pixels PR arranged on the left of four pixel pairs 90 out of the eight light receiving pixels PR in the pixel block 100R, thereby calculating a pixel value VR1 at the center of gravity of the four light receiving pixels PR. The imaging device 1 performs a readout operation on four light receiving pixels PB arranged on the left of four pixel pairs 90 out of the eight light receiving pixels PB in the pixel block 100B, thereby calculating a pixel value VB1 at the center of gravity of the four light receiving pixels PB. The imaging device 1 calculates a pixel value VGb1 at the center of gravity of the five light receiving pixels PGb by targeting the five light receiving pixels PGb arranged on the left of the five pixel pairs 90 out of the ten light receiving pixels PGb in the pixel block 100Gb for a readout operation. In this manner, the imaging device 1 generates image data DT1 (see FIG. 18(A)) including the pixel values ​​VGr1, VR1, VB1, and VGb1.

[0102] Next, as shown in FIG. 18B, the imaging device 1 generates image data DT2 by calculating pixel values ​​V according to the amount of light received by all light receiving pixels P in each of the plurality of pixel blocks 100Gr, 100R, 100B, and 100Gb.

[0103] Specifically, the imaging device 1 performs a read operation on ten light receiving pixels PGr in the pixel block 100Gr, thereby calculating a pixel value VGr2 at the center of gravity of the ten light receiving pixels PGr. The imaging device 1 performs a read operation on eight light receiving pixels PR in the pixel block 100R, thereby calculating a pixel value VR2 at the center of gravity of the eight light receiving pixels PR. The imaging device 1 performs a read operation on eight light receiving pixels PB in the pixel block 100B, thereby calculating a pixel value VB2 at the center of gravity of the eight light receiving pixels PB. The imaging device 1 performs a read operation on ten light receiving pixels PGb in the pixel block 100Gb, thereby calculating a pixel value VGb2 at the center of gravity of the ten light receiving pixels PGb. In this way, the imaging device 1 generates image data DT2 (see FIG. 18B) including the pixel values ​​VGr2, VR2, VB2, and VGb2.

[0104] Focusing on a certain pixel block 100Gr, the readout operation for the ten light-receiving pixels PGr in this pixel block 100Gr will be described below.

[0105] Fig. 19 is a diagram showing an example of a readout operation in the imaging device 1 according to the present embodiment. In Fig. 19, (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG (control signal STRGL) supplied to the light receiving pixel PGr arranged on the left in the pixel pair 90, (D) shows the waveform of the control signal STRG (control signal STRGR) supplied to the light receiving pixel PGr arranged on the right in the pixel pair 90, (E) shows the waveform of the control signal AZ, (F) shows the waveform of the reference signal RAMP, (G) shows the waveform of the signal SIG, and (H) shows the waveform of the signal CP.

[0106] 19(F) and (G), the waveforms of the reference signal RAMP and the signal SIG are shown using the same voltage axis. Note that the waveform of the reference signal RAMP shown in FIG. 19(F) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitance element 23, and the waveform of the signal SIG shown in FIG. 19(G) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitance element 22.

[0107] First, at timing t11, the horizontal period H starts. This causes the drive unit 12 to change the voltage of the control signal SSEL from low level to high level (see FIG. 19(A)). This causes the transistor SEL in the pixel block 100Gr to be turned on, and the pixel block 100Gr is electrically connected to the signal line VSL. Also, at this timing t11, the drive unit 12 changes the voltage of the control signal SRST from low level to high level (see FIG. 19(B)). This causes the transistor RST in the pixel block 100Gr to be turned on, and the voltage of the floating diffusion layer FD is set to the power supply voltage VDD (reset operation). Then, the pixel block 100Gr outputs a voltage corresponding to the voltage of the floating diffusion layer FD at this time.

[0108] Also, at this timing t11, the imaging control unit 18 changes the voltage of the control signal AZ from low level to high level (see FIG. 19(E)). As a result, the comparison circuit 24 of the AD conversion unit ADC sets the operating point by setting the voltages of the capacitance elements 22 and 23. In this way, the voltage of the signal SIG is set to the reset voltage Vreset, and the voltage of the reference signal RAMP is set to the same voltage as the voltage of the signal SIG (reset voltage Vreset) (see FIGS. 19(F) and (G)).

[0109] Then, when a predetermined time has elapsed from timing t11, the drive unit 12 changes the voltage of the control signal SRST from high to low (see FIG. 19(B)). As a result, in the pixel block 100Gr, the transistor RST is turned off, and the reset operation is completed.

[0110] Next, at timing t12, the imaging control section 18 changes the voltage of the control signal AZ from high level to low level (see FIG. 19(E)), which causes the comparison circuit 24 to finish setting the operating point.

[0111] Also, at this timing t12, the reference signal generating unit 13 sets the voltage of the reference signal RAMP to the voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG, so that the comparator circuit 24 changes the voltage of the signal CP from a low level to a high level (see FIG. 19(H)).

[0112] Then, during a period from timing t13 to t15 (P-phase period TP), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t13, the reference signal generation unit 13 starts decreasing the voltage of the reference signal RAMP from the voltage V1 at a predetermined rate of change (see FIG. 19(F)). Also, at this timing t13, the imaging control unit 18 starts generating a clock signal CLK. The counter 25 of the AD conversion unit ADC counts the pulses of this clock signal CLK by performing a counting operation.

[0113] Then, at timing t14, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (reset voltage Vreset) (see Figures 16 (F) and (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see Figure 19 (H)). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTP) is a value that corresponds to the reset voltage Vreset. The latch 26 holds this count value CNTP. Then, the counter 25 resets its count value.

[0114] Next, at timing t15, the imaging control unit 18 stops generating the clock signal CLK with the end of the P-phase period TP. Also, at this timing t15, the reference signal generating unit 13 stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). Then, during the period after this timing t15, the readout unit 20 supplies the count value CNTP held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0115] Next, at timing t16, the imaging control unit 18 sets the voltage of the reference signal RAMP to the voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (the reset voltage Vreset), so that the comparator circuit 24 changes the voltage of the signal CP from a low level to a high level (see FIG. 19(H)).

[0116] Next, at timing t17, the drive unit 12 changes the voltage of the control signal STRGL from low to high (see FIG. 19(C)). As a result, in the five light receiving pixels PGr arranged on the left of the pixel pair 90, the transistors TRG are turned on, and the charge generated in the photodiode PD is transferred to the floating diffusion layer FD (charge transfer operation). Then, the pixel block 100Gr outputs a voltage corresponding to the voltage of the floating diffusion layer FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix1 (see FIG. 19(G)).

[0117] Then, when a predetermined time has elapsed from timing t17, the drive unit 12 changes the voltage of the control signal STRGL from high to low (see FIG. 19C), which causes the transistors TRG in the five light receiving pixels PGr arranged to the left of the pixel pair 90 to be turned off, and the charge transfer operation ends.

[0118] Then, during a period from timing t18 to t20 (D-phase period TD1), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t18, the reference signal generation unit 13 starts decreasing the voltage of the reference signal RAMP from the voltage V1 at a predetermined rate of change (see FIG. 19(F)). Also, at this timing t18, the imaging control unit 18 starts generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count the pulses of this clock signal CLK.

[0119] Then, at timing t19, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix1) (see Figures 19(F) and (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see Figure 19(H)). The counter 25 of the AD conversion unit ADC stops its counting operation based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD1) is a value corresponding to the pixel voltage Vpix1. The latch 26 holds this count value CNTD1. Then, the counter 25 resets its count value.

[0120] Next, at timing t20, the imaging control unit 18 stops generating the clock signal CLK with the end of the D-phase period TD1. Also, at this timing t20, the reference signal generating unit 13 stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). Then, during the period after this timing t20, the readout unit 20 supplies the count value CNTD1 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0121] Next, at timing t21, the imaging control unit 18 sets the voltage of the reference signal RAMP to the voltage V1 (see FIG. 19(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (pixel voltage Vpix1), so that the comparator circuit 24 changes the voltage of the signal CP from a low level to a high level (see FIG. 19(H)).

[0122] Next, at timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from low to high, respectively (see Figures 19(C) and (D)). As a result, in the ten light receiving pixels PGr in the pixel block 100Gr, the transistors TRG are turned on, and the charges generated in the photodiodes PD are transferred to the floating diffusion layer FD (charge transfer operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusion layer FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix2 (see Figure 19(G)).

[0123] Then, at a timing when a predetermined time has elapsed from this timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from high level to low level, respectively (see FIGS. 19C and 19D). As a result, in the ten light receiving pixels PGr, the transistors TRG are turned off, and the charge transfer operation is terminated.

[0124] Then, during a period from timing t23 to t25 (D-phase period TD2), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t23, the reference signal generation unit 13 starts decreasing the voltage of the reference signal RAMP from the voltage V1 at a predetermined rate of change (see FIG. 19(F)). Also, at this timing t23, the imaging control unit 18 starts generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count the pulses of this clock signal CLK.

[0125] Then, at timing t24, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix2) (see Figures 19(F) and (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (see Figure 19(H)). The counter 25 of the AD conversion unit ADC stops its counting operation based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD2) is a value corresponding to the pixel voltage Vpix2. The latch 26 holds this count value CNTD2. Then, the counter 25 resets its count value.

[0126] Next, at timing t25, the imaging control unit 18 stops generating the clock signal CLK with the end of the D-phase period TD2. Also, at this timing t25, the reference signal generating unit 13 stops changing the voltage of the reference signal RAMP (see FIG. 19(F)). Then, during the period after this timing t25, the readout unit 20 supplies the count value CNTD2 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0127] Next, at timing t26, the drive unit 12 changes the voltage of the control signal SSEL from high to low (see FIG. 19A). As a result, in the pixel block 100Gr, the transistor SEL is turned off, and the pixel block 100Gr is electrically disconnected from the signal line VSL.

[0128] In this manner, the readout unit 20 supplies the image signal Spic0 including the count values ​​CNTP, CNTD1, and CNTD2 to the signal processing unit 15. The signal processing unit 15 generates the pixel value VGr1 shown in Fig. 18(A) and the pixel value VGr2 shown in Fig. 18(B) based on the count values ​​CNTP, CNTD1, and CNTD2 included in the image signal Spic0, for example, by utilizing the principle of correlated double sampling.

[0129] Specifically, the signal processing unit 15 generates the pixel value VGr1, for example, by subtracting the count value CNTP from the count value CNTD1. The count value CNTD1 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr arranged on the left side of the five pixel pairs 90 in the pixel block 100Gr, so the signal processing unit 15 can generate the pixel value VGr1 shown in FIG. 18(A) based on this count value CNTD1. Similarly, the signal processing unit 15 generates the pixel value VGr2, for example, by subtracting the count value CNTP from the count value CNTD2. The count value CNTD2 is a value corresponding to the sum of the amounts of light received by the ten light receiving pixels PGr in the pixel block 100Gr, so the signal processing unit 15 can generate the pixel value VGr2 shown in FIG. 18(B) based on this count value CNTD2.

[0130] The above has been described regarding the pixel block 100Gr, but the same is true for each of the pixel blocks 100R, 100Gb, and 100B. In this manner, the signal processing unit 15 generates image data DT1 including the pixel values ​​VR1, VGr1, VGb1, and VB1, and image data DT2 including the pixel values ​​VR2, VGr2, VGb2, and VB2, as shown in FIG.

[0131] FIG. 20 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MA according to the present embodiment.

[0132] First, the signal processor 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0133] Specifically, the signal processing unit 15 calculates a pixel value VGr3 by subtracting the pixel value VGr1 in the image data DT1 from the pixel value VGr2 in the image data DT2. This pixel value VGr3 is a value corresponding to the sum of the amount of light received by the five light receiving pixels PGr arranged on the right side of the five pixel pairs 90 in the pixel block 100Gr. That is, the pixel value VGr1 is a value corresponding to the sum of the amount of light received by the five light receiving pixels PGr arranged on the left side of the five pixel pairs 90 in the pixel block 100Gr, and the pixel value VGr2 is a value corresponding to the sum of the amount of light received by the ten light receiving pixels PGr in the pixel block 100Gr. Therefore, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the sum of the amount of light received by the five light receiving pixels PGr arranged on the right side of the five pixel pairs 90 in the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value that corresponds to the sum of the amounts of light received at the five light receiving pixels PGr located to the right of the five pixel pairs 90, and therefore, as shown in Figure 20, the pixel value VGr3 is located at the center of gravity of these five light receiving pixels PGr.

[0134] Similarly, the signal processing unit 15 calculates a pixel value VR3 by subtracting the pixel value VR1 in the image data DT1 from the pixel value VR2 in the image data DT2. This pixel value VR3 is a value corresponding to the sum of the amounts of light received by the four light receiving pixels PR arranged on the right of the four pixel pairs 90 in the pixel block 100R. The pixel value VR3 is located at the center of gravity of the four light receiving pixels PR arranged on the right of the four pixel pairs 90 in the pixel block 100R.

[0135] The signal processing unit 15 calculates a pixel value VB3 by subtracting the pixel value VB1 in the image data DT1 from the pixel value VB2 in the image data DT2. This pixel value VB3 is a value corresponding to the sum of the amounts of light received by the four light receiving pixels PB arranged on the right of the four pixel pairs 90 in the pixel block 100B. The pixel value VB3 is located at the center of gravity of the four light receiving pixels PB arranged on the right of the four pixel pairs 90 in the pixel block 100B.

[0136] The signal processing unit 15 calculates a pixel value VGb3 by subtracting the pixel value VGb1 in the image data DT1 from the pixel value VGb2 in the image data DT2. This pixel value VGb3 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGb arranged on the right of the five pixel pairs 90 in the pixel block 100Gb. The pixel value VGb3 is located at the center of gravity of the five light receiving pixels PGb arranged on the right of the five pixel pairs 90 in the pixel block 100Gb.

[0137] Then, the image data generating section 16 of the signal processing section 15 performs a predetermined image processing based on the image data DT2, thereby generating image data DP (see FIG. 17(A)) indicative of the captured image.

[0138] Furthermore, the phase difference data generating unit 17 of the signal processing unit 15 performs a predetermined image processing based on each of the image data DT1 and DT3 to generate phase difference data DF indicating the image plane phase difference. That is, the image data DT1 has a pixel value V of the light receiving pixel P arranged on the left of the multiple pixel pairs 90, and the image data DT3 has a pixel value V of the light receiving pixel P arranged on the right of the multiple pixel pairs 90. Therefore, the phase difference data generating unit 17 can generate the phase difference data DF based on each of the image data DT1 and DT3.

[0139] In the imaging device 1, the lenses 101 are arranged in parallel in the X direction and the Y direction in the pixel array section 11 (see FIG. 2), so that the phase difference data DF can be generated with high resolution over the entire surface of the pixel array section 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus, thereby improving image quality.

[0140] <1-7. Imaging mode MB (second imaging mode)> The imaging mode MB according to this embodiment will be described with reference to FIGS.

[0141] Fig. 21 is a diagram showing an example of the operation of the imaging device 1 in the imaging mode MB according to the present embodiment. Fig. 22 is a diagram showing more specifically the operation shown in Fig. 21. In Fig. 21 and Fig. 22, the light receiving pixel P marked with "◯" indicates the light receiving pixel P that is the target of the readout operation.

[0142] In the pixel block 100Gr, as shown in Fig. 22(A), the imaging device 1 performs a readout operation on the photoreceptor pixel PGr arranged on the left of one of the five pixel pairs 90 in the pixel block 100Gr, and calculates a pixel value VGr1 at the position of this photoreceptor pixel PGr. Next, as shown in Fig. 22(B), the imaging device 1 performs a readout operation on two photoreceptor pixels PGr in the pixel pair 90, and calculates a pixel value VGr2 at the center of gravity of these two photoreceptor pixels PGr.

[0143] Similarly, in the pixel block 100R, as shown in Fig. 22(A), the imaging device 1 performs a readout operation on the light receiving pixel PR located on the left of one of the four pixel pairs 90 in the pixel block 100R, and calculates a pixel value VR1 at the position of this light receiving pixel PR. Next, as shown in Fig. 22(B), the imaging device 1 performs a readout operation on two light receiving pixels PR in that pixel pair 90, and calculates a pixel value VR2 at the center of gravity of these two light receiving pixels PR.

[0144] Similarly, in pixel block 100B, as shown in Fig. 22(A), the imaging device 1 performs a readout operation on the light receiving pixel PB located on the left of one of the four pixel pairs 90 in pixel block 100B, and calculates a pixel value VB1 at the position of this light receiving pixel PB. Next, as shown in Fig. 22(B), the imaging device 1 performs a readout operation on two light receiving pixels PB in that pixel pair 90, and calculates a pixel value VB2 at the center of gravity of these two light receiving pixels PB.

[0145] Similarly, in pixel block 100Gb, as shown in Fig. 22(A), the imaging device 1 performs a readout operation on the photoreceptor pixel PGb located on the left of one of the five pixel pairs 90 in pixel block 100Gb, and calculates a pixel value VGb1 at the position of this photoreceptor pixel PGb. Next, as shown in Fig. 22(B), the imaging device 1 performs a readout operation on two photoreceptor pixels PGb in that pixel pair 90, and calculates a pixel value VGb2 at the center of gravity of these two photoreceptor pixels PGb.

[0146] The readout operation in Fig. 22(A) and Fig. 22(B) is the same as the readout operation in the imaging mode MA described above (see Fig. 19). The imaging device 1 then performs the operations in Fig. 22(C) and Fig. 22(D), Fig. 22(E) and Fig. 22(F), Fig. 22(G) and Fig. 22(H), Fig. 22(I) and Fig. 22(J), and Fig. 22(K) and Fig. 22(L). In this way, the imaging device 1 generates image data DT1 (see Fig. 21(A)) including the pixel values ​​VGr1, VR1, VB1, and VGb1, and image data DT2 (see Fig. 21(B)) including the pixel values ​​VGr2, VR2, VB2, and VGb2.

[0147] FIG. 23 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MB according to the present embodiment.

[0148] First, the signal processor 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0149] Specifically, the signal processing unit 15 calculates five pixel values ​​VGr3 by subtracting five pixel values ​​VGr1 in the image data DT1 from five pixel values ​​VGr2 in the image data DT2, respectively. The pixel value VGr3 is a value corresponding to the amount of light received by the light receiving pixel PGr arranged on the right in the pixel pair 90 of the pixel block 100Gr. In other words, the pixel value VGr1 is a value corresponding to the amount of light received by the light receiving pixel PGr arranged on the left in the pixel pair 90 of the pixel block 100Gr, and the pixel value VGr2 is a value corresponding to the sum of the amounts of light received by the two light receiving pixels PGr in this pixel pair 90. Thus, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the amount of light received by the light receiving pixel PGr arranged on the right in the pixel pair 90 of the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value that corresponds to the amount of light received at the photosensitive pixel PGr located to the right of the pixel pair 90, and therefore, as shown in FIG. 23, the pixel value VGr3 is positioned at the position of the photosensitive pixel PGr located to the right of the pixel pair 90.

[0150] Similarly, the signal processing unit 15 calculates four pixel values ​​VR3 by subtracting the four pixel values ​​VR1 in the image data DT1 from the four pixel values ​​VR2 in the image data DT2, respectively. The pixel value VR3 is a value corresponding to the amount of light received at the light receiving pixel PR located on the right of the pixel pair 90 in the pixel block 100R. The pixel value VR3 is located at the position of the light receiving pixel PR located on the right of the pixel pair 90.

[0151] The signal processing unit 15 calculates four pixel values ​​VB3 by subtracting the four pixel values ​​VB1 in the image data DT1 from the four pixel values ​​VB2 in the image data DT2, respectively. The pixel value VB3 is a value corresponding to the amount of light received at the light receiving pixel PB located on the right of the pixel pair 90 in the pixel block 100B. The pixel value VB3 is located at the position of the four light receiving pixels PB located on the right of the pixel pair 90.

[0152] The signal processing unit 15 calculates five pixel values ​​VGb3 by subtracting the five pixel values ​​VGb1 in the image data DT1 from the five pixel values ​​VGb2 in the image data DT2, respectively. The pixel value VGb3 is a value corresponding to the amount of light received at the light receiving pixel PGb located on the right of the pixel pair 90 in the pixel block 100Gb. The pixel value VGb3 is located at the position of the light receiving pixel PGb located on the right of the pixel pair 90.

[0153] Then, the image data generating unit 16 of the signal processing unit 15 generates image data DP (FIG. 17(B)) representing the captured image by performing a predetermined image processing based on the image data DT2 as shown in Fig. 23. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V.

[0154] Fig. 24 is a diagram showing an example of re-mosaic processing in imaging mode MB according to this embodiment. In Fig. 24, (A) shows image data DT2, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on image data DT2. In Fig. 24(B), "◯" indicates the position of pixel value V in image data DT2, and "□" indicates the position of pixel value V in image data DT4.

[0155] Fig. 25 is a diagram showing the re-mosaic processing for the pixel block 100R (first pixel block) in the imaging mode MB according to this embodiment. Fig. 26 is a diagram showing the re-mosaic processing for each of the pixel blocks 100Gr, 100Gb (second pixel block and third pixel block) in the imaging mode MB according to this embodiment. Fig. 27 is a diagram showing the re-mosaic processing for the pixel block 100B (fourth pixel block) in the imaging mode MB according to this embodiment. In Figs. 25 to 27, (A) shows image data DT2, and (B) shows image data DT4 generated by the re-mosaic processing.

[0156] In the image data DT2, as shown in FIG. 24(A), pixel values ​​V are included at a ratio of 18 out of 36 light receiving pixels P. For example, when the number of light receiving pixels P in the pixel array unit 11 is 108 [Mpix], the image data DT2 includes pixel values ​​V for 54 [Mpix]. On the other hand, in the image data DT4, pixel values ​​V are included at a ratio of 16 out of 36 light receiving pixels P as shown in FIG. 24(C). For example, when the number of light receiving pixels P in the pixel array unit 11 is 108 [Mpix], the image data DT4 includes pixel values ​​V for 48 [Mpix]. In the image data DT4, four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generating unit 16 generates such image data DT4 by correcting the pixel values ​​V in the image data DT2 and rearranging the pixel values ​​V.

[0157] Specifically, as shown in Fig. 25, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VR2 in the image data DT2, and generates a pixel value VR4 based on the pixel value V over the one surface. Similarly, as shown in Fig. 26, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VGr2, VGb2 in the image data DT2, and generates pixel values ​​VGr4, VGb4 based on the pixel value V over the one surface. As shown in Fig. 27, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VB2 in the image data DT2, and generates a pixel value VB4 based on the pixel value V over the one surface.

[0158] In this way, the image data generating unit 16 generates the image data DT4 shown in Fig. 24(C) based on the image data DT2 shown in Fig. 24(A). Then, the image data generating unit 16 performs a predetermined image processing based on this image data DT4 to generate image data DP (see Fig. 17(B)) indicating the captured image.

[0159] 23, the phase difference data generating unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing a predetermined image processing based on each of the image data DT1 and DT3. That is, the image data DT1 has a pixel value V of the light receiving pixel P arranged on the left of the multiple pixel pairs 90, and the image data DT3 has a pixel value V of the light receiving pixel P arranged on the right of the multiple pixel pairs 90. Therefore, the phase difference data generating unit 17 can generate the phase difference data DF based on each of the image data DT1 and DT3.

[0160] The phase difference data generating unit 17, like the image data generating unit 16, generates the phase difference data DF by rearranging the pixel values ​​V based on the image data DT1 and DT3. That is, in each of the image data DT1 and DT3, the left and right pixel values ​​V of the pixel pair 90 are included at a ratio of 18 out of 36 light receiving pixels P. Therefore, the phase difference data generating unit 17 rearranges the pixel values ​​V so as to include the left and right pixel values ​​V of the pixel pair 90 at a ratio of 16 out of 36 light receiving pixels P. In this way, the phase difference data generating unit 17 can generate the phase difference data DF corresponding to the image data DP generated by the image data generating unit 16.

[0161] <1-8. Imaging mode MC (third imaging mode)> The imaging mode MC according to this embodiment will be described with reference to FIGS.

[0162] Fig. 28 is a diagram showing an example of the operation of the imaging device 1 in the imaging mode MC according to this embodiment. Fig. 29 is a diagram showing more specifically the operation shown in Fig. 28. In Fig. 28 and Fig. 29, the light receiving pixel P marked with "o" indicates the light receiving pixel P that is the target of the readout operation.

[0163] In the pixel block 100Gr, as shown in FIG. 29(A), the imaging device 1 targets one photosensitive pixel PGr out of the ten photosensitive pixels PGr in the pixel block 100Gr for the readout operation, and calculates the pixel value VGr1 at the position of this photosensitive pixel PGr.

[0164] FIG. 30 is a diagram showing an example of a readout operation in the imaging device 1 according to the present embodiment. In FIG. 30, (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG supplied to the light receiving pixel PGr to be read out, (D) shows the waveform of the control signal AZ, (E) shows the waveform of the reference signal RAMP, (F) shows the waveform of the signal SIG, and (G) shows the waveform of the signal CP. This readout operation is the readout operation in each imaging mode MA, MB (see FIG. 19) with the operations at timings t21 to t26 omitted. In this way, the readout unit 20 supplies the image signal Spic0 including the count values ​​CNTP, CNTD1 to the signal processing unit 15. The signal processing unit 15 generates a pixel value VGr1 using the principle of correlated double sampling based on the count values ​​CNTP, CNTD1 included in the image signal Spic0, for example. Specifically, the signal processing unit 15 generates the pixel value VGr1 by subtracting the count value CNTP from the count value CNTD1, for example.

[0165] Similarly, in pixel block 100R, as shown in Figure 29 (A), the imaging device 1 targets one of the eight photosensitive pixels PR in pixel block 100R as the target of a readout operation, and calculates a pixel value VR1 at the position of this photosensitive pixel PR.

[0166] Similarly, in pixel block 100B, as shown in FIG. 29(A), the imaging device 1 targets one of the eight light receiving pixels PB in pixel block 100B as the target of the readout operation, and calculates a pixel value VB1 at the position of this light receiving pixel PB.

[0167] Similarly, in pixel block 100Gb, as shown in Figure 29 (A), the imaging device 1 targets one photosensitive pixel PGb out of the ten photosensitive pixels PGb in pixel block 100Gb as the target of the readout operation, and calculates the pixel value VGb1 of this photosensitive pixel PGb.

[0168] Thereafter, the imaging device 1 performs the operations shown in Figures 29(B) to 29(L). In this manner, the imaging device 1 generates image data DT1 (see Figure 28) including the pixel values ​​VGr1, VR1, VB1, and VGb1.

[0169] FIG. 31 is a diagram showing an example of image processing by the signal processing unit 15 in the imaging mode MC according to the present embodiment.

[0170] The image data generating unit 16 of the signal processing unit 15 performs a predetermined image processing based on the image data DT1 to generate image data DP (see FIG. 17C) representing the captured image. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V.

[0171] Fig. 32 is a diagram showing an example of re-mosaic processing in imaging mode MC according to this embodiment. In Fig. 32, (A) shows image data DT1, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on image data DT1. In Fig. 32(B), "◯" indicates the position of pixel value V in image data DT1, and "□" indicates the position of pixel value V in image data DT4.

[0172] Fig. 33 is a diagram showing the re-mosaic processing for the pixel block 100R in the imaging mode MC according to this embodiment. Fig. 34 is a diagram showing the re-mosaic processing for each of the pixel blocks 100Gr, 100Gb in the imaging mode MC according to this embodiment. Fig. 35 is a diagram showing the re-mosaic processing for the pixel block 100B in the imaging mode MC according to this embodiment. In Figs. 33 to 35, (A) shows image data DT1, and (B) shows image data DT4 generated by the re-mosaic processing.

[0173] In the image data DT4, the four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generator 16 generates such image data DT4 by modifying the pixel values ​​V in the image data DT1 and rearranging the pixel values ​​V.

[0174] Specifically, as shown in Fig. 33, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VR1 in the image data DT1, and generates a pixel value VR4 based on the pixel value V over the one surface. Similarly, as shown in Fig. 34, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VGr1, VGb1 in the image data DT1, and generates pixel values ​​VGr4, VGb4 based on the pixel value V over the one surface. As shown in Fig. 35, the image data generating unit 16 calculates a pixel value V over one surface by performing, for example, an interpolation process based on a plurality of pixel values ​​VB1 in the image data DT1, and generates a pixel value VB4 based on the pixel value V over the one surface.

[0175] In this way, the image data generating unit 16 generates the image data DT4 shown in Fig. 32(C) based on the image data DT1 shown in Fig. 32(A). Then, the image data generating unit 16 performs a predetermined image processing based on this image data DT4 to generate image data DP (see Fig. 17(C)) indicating the captured image.

[0176] 31, the phase difference data generating unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing a predetermined image processing based on the image data DT1. That is, the image data DT1 includes image data (image data DT11) having a pixel value V at a light receiving pixel P arranged on the left of the plurality of pixel pairs 90, and image data (image data DT12) having a pixel value V at a light receiving pixel P arranged on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generating unit 17 can generate the phase difference data DF based on this image data DT1 (each of the image data DT11, DT12).

[0177] In this way, the imaging device 1 is provided with a plurality of pixel blocks 100 each having a plurality of light receiving pixels P including color filters 131 of the same color. The plurality of light receiving pixels P are divided into a plurality of pixel pairs 90 each including two light receiving pixels P. A plurality of lenses 101 are provided at positions corresponding to the plurality of pixel pairs 90. This allows the imaging device 1 to generate phase difference data DF with high resolution over the entire surface of the pixel array section 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus at various zoom magnifications. As a result, the imaging device 1 can improve image quality.

[0178] Furthermore, in the imaging device 1, the number of the light receiving pixels P in a certain pixel block 100 is set to be greater than the number of the light receiving pixels P in another certain pixel block 100. Specifically, for example, the number of the light receiving pixels PGr in the pixel block 100Gr and the number of the light receiving pixels PGb in the pixel block 100Gb are set to be greater than the number of the light receiving pixels PR in the pixel block 100R and the number of the light receiving pixels PB in the pixel block 100B. This makes it possible to increase, for example, the light receiving sensitivity of green, thereby improving the image quality of the captured image.

[0179] Furthermore, the imaging device 1 is provided with three imaging modes MA, MB, and MC, and re-mosaic processing is performed in each of the imaging modes MB and MC. As a result, in the imaging device 1, for example, the number of effective pixels in the imaging mode MB can be adjusted by the re-mosaic processing in the imaging mode MB, and it becomes possible to reduce the change in the number of effective pixels when the zoom magnification is changed, and therefore it is possible to suppress the change in the image quality of the captured image.

[0180] <2. Modifications> Modifications 1 to 5 according to this embodiment will be described with reference to FIGS.

[0181] <2-1. Variation 1> FIG. 36 is a plan view for explaining the refractive index of each lens 101 in the pixel array portion 11 according to the first modification of the present embodiment.

[0182] As shown in Fig. 36, the refractive index of the lens region 101a or the lens region 101b may be different for each color. In other words, in the above description, the high refractive index material or the low refractive index material of the lens 101 is the same regardless of the color (R, G, B), but it may be different for each color. Since the degree of sensitivity difference between the same colors may differ for each color (see Fig. 9), the refractive index of one or both of the lens region 101a and the lens region 101b may be different for each color. For example, the high refractive index material or the low refractive index material may be different in each light receiving pixel P of red (R) or blue (B).

[0183] 36, seven types of materials, three types of high refractive index materials and four types of low refractive index materials, are used to form the lens 101. The lens region 101a is formed from one of the three types of high refractive index materials, and the lens region 101b is formed from one of the four types of high refractive index materials. Note that there may be more types of each of the high refractive index materials and the low refractive index materials.

[0184] For example, a green (G) light receiving pixel P adjacent to a red (R) light receiving pixel P and a green (G) light receiving pixel P adjacent to a blue (B) light receiving pixel P may use a high refractive index material or a low refractive index material with the same refractive index. However, since the green (G) light receiving pixel P adjacent to a red (R) light receiving pixel P and the green (G) light receiving pixel P adjacent to a blue (B) light receiving pixel P are affected differently by the adjacent color filters 131, a high refractive index material or a low refractive index material with a different refractive index may be used for each of these light receiving pixels P.

[0185] According to variant example 1, in addition to adjusting the refractive index of each of lens area 101a and lens area 101b, by changing the refractive index of lens area 101a or lens area 101b for each color, it becomes possible to accurately correct the sensitivity, thereby achieving improved image quality.

[0186] <2-2. Variation 2> FIG. 37 is a plan view for explaining the refractive index of each lens 101 in the pixel array portion 11 according to the first modification of the present embodiment.

[0187] As shown in Fig. 37, a pixel pair 90 that does not produce a same-color sensitivity difference is provided with lenses 101 with different refractive indices for each color. In the example of Fig. 37, the lens 101 for the red (R) pixel pair 90 that does not produce a same-color sensitivity difference is filled with a left diagonal line, the lens 101 for the green (G) pixel pair 90 that does not produce a same-color sensitivity difference is filled with a right diagonal line, and the lens 101 for the blue (B) pixel pair 90 that does not produce a same-color sensitivity difference is not filled.

[0188] In other words, in the above explanation, lenses with the same refractive index regardless of color are used as lenses 101 for pixel pairs 90 that do not cause same-color sensitivity differences, but lenses with different refractive indexes may be used to match the sensitivity differences of each pixel pair 90 for each color. However, for Gr and Gb, it is desirable to use lenses with the same refractive index as lenses 101 that do not cause same-color sensitivity differences.

[0189] According to variant example 2, in addition to adjusting the refractive index of each of lens areas 101a and 101b, the refractive index of lens 101 for pixel pairs 90 that do not have sensitivity differences between the same colors is changed for each color, making it possible to accurately correct the sensitivity, thereby achieving improved image quality.

[0190] <2-3. Variation 3> Fig. 38 is a plan view for explaining the refractive index of each lens 101 in the pixel array section 11 according to the modified example 3 of this embodiment. Fig. 39 is a diagram showing an example of the cross-sectional structure of the pixel array section 11 according to the modified example 3 of this embodiment. Note that Fig. 39 shows a cross section corresponding to the line Y2-Y2 shown in Fig. 38.

[0191] 38 and 39, the ratio of the lens region 101a (high refractive index region) to the lens region 101b (low refractive index region) of the lens 101 changes depending on the position of the lens 101 with respect to the pixel array section 11. In other words, the planar areas (areas in a plan view) of the lens region 101a and the lens region 101b change depending on the position of the lens 101 with respect to the pixel array section 11.

[0192] 38, the planar area of ​​lens region 101a becomes larger as lens 101 is positioned closer to the outer periphery of the angle of view. Conversely, the planar area of ​​lens region 101b becomes smaller as lens 101 is positioned closer to the outer periphery of the angle of view. Note that when lens 101 is positioned near the center of the angle of view, lens region 101a and lens region 101b have the same planar area.

[0193] According to variant example 3, in addition to adjusting the refractive indexes of lens area 101a and lens area 101b, the ratio of lens area 101a (high refractive index area) to lens area 101b (low refractive index area) is changed depending on the position of lens 101 relative to pixel array section 11, thereby making it possible to accurately correct sensitivity, thereby achieving improved image quality.

[0194] <2-4. Variation 4> FIG. 40 is a plan view for explaining the refractive index of each lens 101 in the pixel array portion 11 according to the fourth modification of the present embodiment.

[0195] 40, the planar division pattern (division pattern in plan view) of the lens region 101a (high refraction region) and the lens region 101b (low refraction region) of the lens 101 changes depending on the position of the lens 101 with respect to the pixel array section 11. That is, the planar shapes of the lens region 101a and the lens region 101b change depending on the position of the lens 101 with respect to the pixel array section 11. Note that, as in the above-described fourth modification, for example, the planar area of ​​the lens region 101a becomes larger as the lens 101 is positioned closer to the outer periphery of the angle of view.

[0196] In the example of Fig. 40, the plane division pattern changes as the lens 101 is positioned toward the periphery of the angle of view (outside the angle of view). For example, since the influence of oblique incidence becomes stronger as the lens 101 is positioned toward the four corners of the angle of view, the lens 101 may be divided, for example, according to the position of the hands of a clock, rather than being divided vertically. In the example of Fig. 40, in the lens 101 positioned on the four corner sides, the planar shape of the lens region 101b (low refractive region) is a sector, but this is not limited thereto, and may be changed as appropriate depending on the position of the lens 101 relative to the pixel array unit 11, and the planar area of ​​the sector may also be changed as appropriate depending on the position of the lens 101 relative to the pixel array unit 11.

[0197] According to variant example 4, in addition to adjusting the refractive indexes of lens region 101a and lens region 101b, the planar division patterns of lens region 101a (high refractive index region) and lens region 101b (low refractive index region) are changed depending on the position of lens 101 relative to pixel array section 11, thereby making it possible to accurately correct sensitivity, thereby achieving improved image quality.

[0198] <2-5. Variation 5> Fig. 41 is a plan view showing the position of a desired lens 101 in a pixel array unit 11 according to Modification 5 of the present embodiment. Fig. 42 is a diagram for explaining the refractive index of a desired lens 101 according to Modification 5 of the present embodiment.

[0199] As shown in Fig. 41, the sensitivity of a desired pixel pair 90 (photosensitive pixel PGr A1 and photosensitive pixel PGr A2 in Fig. 41) may be affected differently for each photosensitive pixel P (six pixels a-f in Fig. 41) that borders the desired pixel pair 90 at a side. Note that a photosensitive pixel P that borders the pixel pair 90 diagonally is not relevant because it borders at a point. In the example of Fig. 41, pixel a is photosensitive pixel PB, pixel b is photosensitive pixel PGb, pixel c is photosensitive pixel PGr, pixel d is photosensitive pixel PR, pixel e is photosensitive pixel PGr, and pixel f is photosensitive pixel PR.

[0200] In such a case, as shown in Fig. 42, one lens 101 is divided into six parts in a plan view. In the example of Fig. 42, the lens 101 is radially divided into six parts, each part being a to f. The part a is a lens part 101b with a low refractive index, the parts b, c, and e are lens parts 101c with a medium refractive index, and the parts d and f are lens parts 101a with a high refractive index. The refractive index of the lens part 101a with a high refractive index is higher than the refractive index of the lens part 101b with a low refractive index and the lens part 101c with a medium refractive index, and the refractive index of the lens part 101c with a medium refractive index is higher than the refractive index of the lens part 101b with a low refractive index.

[0201] According to the fifth modification, by dividing one lens 101 into six, it becomes possible to combine lens materials according to the sensitivity of the light receiving pixel P adjacent to a desired pixel pair 90. This makes it possible to design the lens 101 while taking into consideration the influence that each light receiving pixel P of each pixel pair 90 receives from the surrounding light receiving pixels P.

[0202] The number of divisions of the lens 101 may be 4, 8, 3 or 5, other than 6. In other words, the lens 101 for the pixel pair 90 may be divided into two regions, one with a high refractive index and one with a low refractive index, or may be divided into three or more regions with different refractive indexes depending on the color of the light receiving pixel P adjacent to the pixel pair 90 (the refractive index of the color filter 131 of the light receiving pixel P).

[0203] <3. Actions and Effects> As described above, the imaging device 1 according to this embodiment includes a pixel array section 11 having a plurality of pixels (light receiving pixels P) each including a color filter 131, the plurality of pixels including a first pixel and a second pixel each having a color filter 131 of the same color, the pixel array section 11 has a lens 101 provided in common to the first pixel and the second pixel, the lens 101 has a first lens area (for example, lens area 101a) corresponding to the first pixel and a second lens area (for example, lens area 101b) corresponding to the second pixel, and the refractive indexes of the first lens area and the second lens area are different (see FIG. 3, FIG. 10, etc.). This makes it possible to correct the sensitivity of the first pixel and the second pixel and suppress the sensitivity difference between the first pixel and the second pixel of the same color, thereby improving the image quality.

[0204] Furthermore, the planar areas of the first lens region and the second lens region may be the same (see FIG. 10, FIG. 36, FIG. 37, etc.). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel, and therefore makes it possible to reliably suppress the sensitivity difference between the first pixel and the second pixel for the same color.

[0205] The planar areas of the first lens region and the second lens region may be different from each other (see Figs. 38 to 40). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel, and therefore makes it possible to reliably suppress the sensitivity difference between the first pixel and the second pixel for the same color.

[0206] Furthermore, the planar areas of the first lens region and the second lens region may be changed according to the position of the lens 101 relative to the pixel array section 11 (see FIGS. 38 to 40). This makes it possible to appropriately correct the sensitivities of the first pixel and the second pixel, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel for the same color.

[0207] The refractive index of the first lens region may be higher than that of the second lens region, and the planar area of ​​the first lens region may be larger as the lens 101 is positioned closer to the outer periphery of the pixel array section 11 (see FIGS. 38 to 40). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel according to the angle of view of the pixel array section 11, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0208] Furthermore, the planar shapes of the first lens region and the second lens region may be the same (see FIG. 10, FIG. 36, FIG. 37, etc.). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel, and therefore makes it possible to reliably suppress the sensitivity difference between the first pixel and the second pixel for the same color.

[0209] The planar shapes of the first lens region and the second lens region may be different from each other (see Figs. 38 to 40). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel, and therefore makes it possible to reliably suppress the sensitivity difference between the first pixel and the second pixel for the same color.

[0210] Furthermore, the planar shapes of the first lens region and the second lens region may be changed according to the position of the lens 101 relative to the pixel array section 11 (see FIG. 40). This makes it possible to appropriately correct the sensitivity of the first pixel and the second pixel according to the angle of view of the pixel array section 11, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0211] Furthermore, the lens 101 has a plurality of lens regions including a first lens region and a second lens region (for example, lens region 101a, lens region 101b, and lens region 101c), and the refractive indexes of any three or more of the plurality of lens regions may be different (see FIGS. 41 and 42). This makes it possible to more appropriately correct the sensitivity of the first pixel and the second pixel, thereby reliably suppressing the sensitivity difference between the same colors of the first pixel and the second pixel.

[0212] Furthermore, the lens regions may be provided according to the arrangement of the pixels around the first pixel and the second pixel (see Figs. 41 and 42). This makes it possible to more appropriately correct the sensitivity of the first pixel and the second pixel, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0213] Furthermore, the lens regions may be divided so that the lenses are radially divided in plan view (see FIG. 42). This makes it possible to more appropriately correct the sensitivities of the first and second pixels, thereby reliably suppressing the sensitivity difference between the first and second pixels for the same color.

[0214] Furthermore, the lens 101 may be formed from a plurality of materials having different refractive indices, which makes it possible to easily manufacture the lens 101 having the first lens region 101a and the second lens region 101b having different refractive indices.

[0215] The plurality of materials may also include materials with different composition ratios, which makes it possible to easily manufacture lens 101 having first lens region 101a and second lens region 101b with different refractive indices.

[0216] The first pixel and the second pixel may be adjacent to each other. This makes it possible to more appropriately correct the sensitivity of the first pixel and the second pixel in the pixel pair 90 in which the first pixel and the second pixel are adjacent to each other, thereby reliably suppressing the sensitivity difference between the first pixel and the second pixel of the same color.

[0217] In addition, the pixels may be arranged in a Deca-Octa Bayer Coding (DOBC) format. When the pixel arrangement is in the DOBC format, the sensitivity of the first pixel and the second pixel can be corrected to suppress the sensitivity difference between the first pixel and the second pixel of the same color, thereby improving image quality.

[0218] The first pixel and the second pixel may have a sensitivity difference between the same colors, and the refractive index of each of the first lens region and the second lens region may be set to suppress the sensitivity difference between the same colors. Even in such a case, the sensitivity of the first pixel and the second pixel can be corrected to suppress the sensitivity difference between the same colors of the first pixel and the second pixel, thereby improving image quality.

[0219] The pixels include a third pixel and a fourth pixel each having a color filter 131 of the same color, the pixel array unit 11 has a second lens (for example, lens 101) provided in common to the third pixel and the fourth pixel, the second lens has a third lens area (for example, lens area 101a) corresponding to the third pixel and a fourth lens area (for example, lens area 101b) corresponding to the fourth pixel, and the refractive indexes of the first lens area, the second lens area, the third lens area, and the fourth lens area may be different (see FIG. 36). This makes it possible to accurately correct the sensitivity by changing the refractive indexes of the first lens area, the second lens area, the third lens area, and the fourth lens area, thereby improving the image quality.

[0220] The pixels include a plurality of pixel pairs 90 each having a pair of pixels each having a color filter 131 of the same color, and the pixel array unit 11 has a plurality of lenses 101 provided for each pixel pair 90, and the refractive index of each of the lenses 101 may be different for each color filter 131 of the same color (see FIG. 37). This makes it possible to accurately correct the sensitivity by changing the refractive index of the lens 101 for each pixel pair 90 that does not cause a sensitivity difference between the same colors, thereby improving image quality.

[0221] <4. Other embodiments> The configurations according to the above-mentioned embodiments (examples, modifications) may be implemented in various different forms other than the above-mentioned embodiments. For example, the configurations are not limited to the above-mentioned examples, and may be in various forms. In addition, for example, the information including the configurations, processing procedures, specific names, various data, and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified.

[0222] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0223] In addition, the above-mentioned embodiments (examples, modifications) can be appropriately combined within a range that does not cause contradictions in the processing contents. In addition, the effects described in this specification are merely examples and are not limited to the present invention, and other effects may also be obtained.

[0224] <5. Application Examples> An application example of the imaging device 1 according to this embodiment (including modifications) will be described with reference to Fig. 43. Fig. 43 is a diagram showing an application example in which the imaging device 1 according to this embodiment is used.

[0225] The imaging device 1 described above can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as described below. For example, as shown in FIG. 43, the imaging device 1 described above is used in "devices for capturing images for viewing, such as digital cameras and portable devices with camera functions", "devices for traffic use, such as in-vehicle sensors for capturing images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and recognition of the driver's state, surveillance cameras for monitoring moving vehicles and roads, and distance measuring sensors for measuring distances between vehicles, etc.", "devices for home appliances such as TVs, refrigerators, and air conditioners for capturing images of user gestures and operating devices according to the gestures", "devices for medical and healthcare use, such as endoscopes and devices for capturing images of blood vessels by receiving infrared light", "devices for security use, such as surveillance cameras for crime prevention and cameras for person authentication", "devices for beauty use, such as skin measuring devices for capturing images of the skin and microscopes for capturing images of the scalp", "devices for sports use, such as action cameras and wearable cameras for sports use, etc.", and "devices for agriculture use, such as cameras for monitoring the state of fields and crops".

[0226] The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as an electronic device mounted on any type of moving object such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, a robot, a construction machine, an agricultural machine (tractor), etc. Also, for example, the technology according to the present disclosure may be realized as an electronic device mounted on an endoscopic surgery system, a microsurgery system, etc.

[0227] <5-1. Imaging device> An imaging device 300 according to an application example will be described with reference to Fig. 44. Fig. 44 is a diagram showing a configuration example of the imaging device 300 according to an application example. This imaging device 300 is an example of an electronic device to which the above-mentioned imaging device 1 is applied. Examples of the imaging device 300 include electronic devices such as digital still cameras, video cameras, smartphones and mobile phones having imaging functions.

[0228] 44, the imaging device 300 includes an optical system 301, a shutter device 302, an imaging element (solid-state imaging device) 303, a control circuit (drive circuit) 304, a signal processing circuit 305, a monitor 306, and a memory 307. The imaging device 300 is capable of capturing still images and moving images.

[0229] The optical system 301 includes one or more lenses. The optical system 301 guides light from a subject (incident light) to the image sensor 303, and forms an image on the light receiving surface of the image sensor 303.

[0230] The shutter device 302 is disposed between the optical system 301 and the image sensor 303. The shutter device 302 controls the light irradiation period and the light blocking period for the image sensor 303 under the control of the control circuit 304.

[0231] The image sensor 303 accumulates signal charges for a certain period of time in response to light that is imaged on the light receiving surface via the optical system 301 and the shutter device 302. The signal charges accumulated in the image sensor 303 are transferred in accordance with a drive signal (timing signal) supplied from a control circuit 304. As the image sensor 303, for example, the above-mentioned image sensor 1 is used.

[0232] The control circuit 304 outputs drive signals that control the transfer operation of the image sensor 303 and the shutter operation of the shutter device 302 , thereby driving the image sensor 303 and the shutter device 302 .

[0233] The signal processing circuit 305 performs various signal processing on the signal charges output from the image sensor 303. An image (image data) obtained by the signal processing performed by the signal processing circuit 305 is supplied to a monitor 306 and also to a memory 307.

[0234] The monitor 306 displays a moving image or a still image captured by the image sensor 303 based on the image data supplied from the signal processing circuit 305. As the monitor 306, for example, a panel-type display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel is used.

[0235] The memory 307 stores image data supplied from the signal processing circuit 305 , that is, image data of a moving image or a still image captured by the imaging element 303 .

[0236] In the imaging device 300 configured as above, by applying the imaging device 1 described above to the imaging element 303, it is possible to achieve an improvement in image quality.

[0237] <5-2. Distance measuring device> A distance measuring device 400 according to an application example will be described with reference to Fig. 45. Fig. 45 is a diagram showing a configuration example of a distance measuring device 400 according to an application example. This distance measuring device 400 is an example of an electronic device to which the above-mentioned imaging device 1 is applied.

[0238] 45, a distance measuring device (distance image sensor) 400 includes a light source unit 401, an optical system 402, an image sensor (solid-state image sensor) 403, a control circuit (drive circuit) 404, a signal processing circuit 405, a monitor 406, and a memory 407. This distance measuring device 400 can obtain a distance image according to the distance to the subject by projecting light from the light source unit 401 toward the subject and receiving light (modulated light or pulsed light) reflected by the surface of the subject.

[0239] The light source unit 401 projects light toward the subject. For example, a vertical cavity surface emitting laser (VCSEL) array that emits laser light as a surface light source or a laser diode array in which laser diodes are arranged in a line is used as the light source unit 401. The laser diode array is supported by a predetermined driving unit (not shown) and is scanned in a direction perpendicular to the direction in which the laser diodes are arranged.

[0240] The optical system 402 includes one or more lenses. The optical system 402 guides light from a subject (incident light) to the image sensor 403, and forms an image on the light receiving surface (sensor portion) of the image sensor 403.

[0241] The image sensor 403 accumulates signal charges in response to light that is imaged on the light receiving surface via the optical system 402. A distance signal indicating a distance determined from a light receiving signal (APD OUT) output from the image sensor 403 is supplied to a signal processing circuit 405. As the image sensor 403, for example, the above-mentioned imaging device 1 is used.

[0242] The control circuit 404 outputs a drive signal (control signal) for controlling the operation of the light source unit 401, the image sensor 403, etc., and drives the light source unit 401, the image sensor 403, etc.

[0243] The signal processing circuit 405 performs various signal processing on the distance signal supplied from the imaging element 403. For example, the signal processing circuit 405 performs image processing (e.g., histogram processing, peak detection processing, etc.) to construct a distance image based on the distance signal. The image (image data) obtained by the signal processing performed by the signal processing circuit 405 is supplied to a monitor 406 and also to a memory 407.

[0244] The monitor 406 displays the distance image captured by the image sensor 303 based on the image data supplied from the signal processing circuit 405. As the monitor 406, for example, a panel type display device such as a liquid crystal panel or an organic EL panel is used.

[0245] The memory 407 stores image data supplied from the signal processing circuit 405 , that is, image data of the distance image captured by the image sensor 303 .

[0246] In the distance measuring device 400 configured as above, by applying the imaging device 1 described above to the imaging element 403, it is possible to improve the image quality.

[0247] As described above, the imaging device 1 according to this embodiment can be implemented in various electronic devices. For example, the imaging device 1 described above may be mounted on various electronic devices such as a notebook PC (Personal Computer), a mobile device (e.g., a smartphone or a tablet PC), a PDA (Personal Digital Assistant), a wearable device, a game device, and a music device, in addition to the imaging device 300 and the distance measuring device 400.

[0248] <6. Notes> The present technology can also be configured as follows. (1) a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. Imaging device. (2) The first lens area and the second lens area have the same planar area. The imaging device according to (1) above. (3) The first lens region and the second lens region have different planar areas. The imaging device according to (1) above. (4) A planar area of ​​each of the first lens region and the second lens region is changed depending on a position of the lens with respect to the pixel array unit. The imaging device according to any one of (1) to (3). (5) The refractive index of the first lens region is higher than the refractive index of the second lens region; a planar area of ​​the first lens region becomes larger as the lens is positioned closer to the outer periphery of the pixel array section; The imaging device according to (4) above. (6) The first lens region and the second lens region have the same planar shape. The imaging device according to any one of (1) to (5). (7) The first lens region and the second lens region have different planar shapes. The imaging device according to any one of (1) to (5). (8) a planar shape of each of the first lens region and the second lens region is changed depending on a position of the lens with respect to the pixel array unit; The imaging device according to any one of (1) to (7). (9) the lens has a plurality of lens regions including the first lens region and the second lens region; The refractive indices of any three or more of the plurality of lens regions are different. The imaging device according to any one of (1) to (8). (10) the lens regions are provided according to an arrangement of a plurality of pixels present around the first pixel and the second pixel among the plurality of pixels; The imaging device according to (9) above. (11) The plurality of lens regions are divided so that the lens is radially divided in a plan view. The imaging device according to (9) or (10) above. (12) The lens is formed of a plurality of materials having different refractive indices. The imaging device according to any one of (1) to (11). (13) The plurality of materials include materials having different composition ratios. The imaging device according to (12) above. (14) The first pixel and the second pixel are adjacent to each other. The imaging device according to any one of (1) to (13). (15) The plurality of pixels are arranged in a Deca-Octa Bayer Coding (DOBC) format. The imaging device according to any one of (1) to (14). (16) the first pixel and the second pixel have a same-color sensitivity difference; The refractive indexes of the first lens region and the second lens region are set so as to suppress the sensitivity difference between the same colors. The imaging device according to any one of (1) to (15). (17) the plurality of pixels include a third pixel and a fourth pixel each having a color filter of the same color; the pixel array unit has a second lens provided in common to the third pixel and the fourth pixel, the second lens has a third lens area corresponding to the third pixel and a fourth lens area corresponding to the fourth pixel; The first lens region, the second lens region, the third lens region, and the fourth lens region have different refractive indices. The imaging device according to any one of (1) to (16). (18) the plurality of pixels include a plurality of pixel pairs each including a pair of pixels having color filters of the same color; the pixel array unit has a plurality of lenses provided for each of the pixel pairs, The refractive index of each of the plurality of lenses is different for each of the color filters of the same color. The imaging device according to any one of (1) to (17). (19) An imaging device is provided, The imaging device includes: a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. electronic equipment. (20) forming a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. A method for manufacturing an imaging device. (twenty one) The imaging device according to any one of (1) to (18), electronic equipment. (twenty two) Manufacturing an imaging device according to any one of (1) to (18) above, A method for manufacturing a semiconductor device. [Explanation of symbols]

[0249] 1. Imaging device 11 Pixel array section 12 Drive unit 13 Reference signal generation section 15 Signal Processing Section 16 Image data generation unit 17 Phase difference data generator 18 Imaging control section 20 Reading part 21 Constant current source 22 Capacitive element 23 Capacitive element 24 Comparison circuit 25 Counter 26 Latch 27 Transfer control section 90 pixel pairs 100 pixel block 100R pixel block 100Gr pixel block 100Gb pixel block 100B pixel block 101 Lens 101a Lens area 101b Lens area 101c Lens area 111 Semiconductor substrate 111S1 Surface 111S2 back side 112 Light receiving part 113 Separation section 121 Multilayer wiring layer 122 wiring layer 123 Wiring layer 124 wiring layer 125 wiring layer 126 wiring layer 127 Wiring layer 128 Interlayer Insulation Layer 131 Color Filter 132 Light-shielding film 300 Imaging device 400 Rangefinder L1 light L2 light P Photosensitive pixel PB Photosensitive pixel PD Photodiode PGb Photosensitive pixel PGr Light receiving pixel PR light receiving pixel U Pixel Unit

Claims

1. a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. Imaging device.

2. The first lens area and the second lens area have the same planar area. The imaging device according to claim 1 .

3. The first lens region and the second lens region have different planar areas. The imaging device according to claim 1 .

4. a planar area of ​​each of the first lens region and the second lens region is changed depending on a position of the lens with respect to the pixel array unit; The imaging device according to claim 1 .

5. The refractive index of the first lens region is higher than the refractive index of the second lens region; a planar area of ​​the first lens region becomes larger as the lens is positioned closer to the outer periphery of the pixel array section; The imaging device according to claim 4.

6. The first lens region and the second lens region have the same planar shape. The imaging device according to claim 1 .

7. The first lens region and the second lens region have different planar shapes. The imaging device according to claim 1 .

8. a planar shape of each of the first lens region and the second lens region is changed depending on a position of the lens with respect to the pixel array unit; The imaging device according to claim 1 .

9. the lens has a plurality of lens regions including the first lens region and the second lens region; The refractive indices of any three or more of the plurality of lens regions are different. The imaging device according to claim 1 .

10. the lens regions are provided according to an arrangement of a plurality of pixels present around the first pixel and the second pixel among the plurality of pixels; The imaging device according to claim 9.

11. The plurality of lens regions are divided so that the lens is radially divided in a plan view. The imaging device according to claim 9.

12. The lens is formed of a plurality of materials having different refractive indices. The imaging device according to claim 1 .

13. The plurality of materials include materials having different composition ratios. The imaging device according to claim 12.

14. The first pixel and the second pixel are adjacent to each other. The imaging device according to claim 1 .

15. The plurality of pixels are arranged in a Deca-Octa Bayer Coding (DOBC) format. The imaging device according to claim 1 .

16. the first pixel and the second pixel have a same-color sensitivity difference; The refractive indexes of the first lens region and the second lens region are set so as to suppress the sensitivity difference between the same colors. The imaging device according to claim 1 .

17. the plurality of pixels include a third pixel and a fourth pixel each having a color filter of the same color; the pixel array unit has a second lens provided in common to the third pixel and the fourth pixel; the second lens has a third lens area corresponding to the third pixel and a fourth lens area corresponding to the fourth pixel; The first lens region, the second lens region, the third lens region and the fourth lens region have different refractive indices. The imaging device according to claim 1 .

18. the plurality of pixels include a plurality of pixel pairs each including a pair of pixels having color filters of the same color; the pixel array unit has a plurality of lenses provided for each of the pixel pairs, The refractive index of each of the plurality of lenses is different for each of the color filters of the same color. The imaging device according to claim 1 .

19. An imaging device is provided, The imaging device includes: a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. electronic equipment.

20. forming a pixel array portion having a plurality of pixels each including a color filter; the plurality of pixels include a first pixel and a second pixel each having a color filter of the same color; the pixel array unit has a lens provided in common to the first pixel and the second pixel, the lens has a first lens area corresponding to the first pixel and a second lens area corresponding to the second pixel; The first lens region and the second lens region have different refractive indices. A method for manufacturing an imaging device.

Images