Imaging device

Abstract

An image capturing device according to the present disclosure is provided with: a plurality of pixel blocks, each comprising a plurality of light receiving pixels including color filters having the same color as one another, wherein the plurality of light receiving pixels are divided into a plurality of pixel pairs each including two light receiving pixels; and a plurality of lenses provided respectively in positions corresponding to the plurality of pixel pairs.

Description

[Technical Field]

[0001] The present disclosure relates to an imaging device that captures an image of a subject. [Background technology]

[0002] Some imaging devices obtain an image plane phase difference to realize, for example, autofocus. For example, Patent Document 1 discloses an imaging device including normal pixels and phase difference detection pixels for obtaining the image plane phase difference. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016 / 098640 Summary of the Invention

[0004] In imaging devices, it is desired that the image quality of the captured image is high, and further improvement in image quality is expected.

[0005] It is desirable to provide an imaging device that can improve image quality.

[0006] In one embodiment of the present disclosure 1st The imaging device includes a plurality of pixel blocks and a plurality of lenses. Each of the plurality of pixel blocks includes a plurality of light-receiving pixels each including a color filter of the same color. The plurality of light-receiving pixels are divided into a plurality of pixel pairs, each including two light-receiving pixels. The plurality of lenses are provided at positions corresponding to the plurality of pixel pairs. The two light-receiving pixels are arranged side by side in a first direction. In each of the plurality of pixel blocks, two pixel pairs arranged side by side in a second direction intersecting the first direction are arranged to be shifted in the first direction. A second imaging device according to an embodiment of the present disclosure includes a plurality of pixel blocks and a plurality of lenses. Each of the plurality of pixel blocks includes a plurality of light receiving pixels including color filters of the same color. The plurality of light receiving pixels are divided into a plurality of pixel pairs, each including two light receiving pixels. The plurality of lenses are provided at positions corresponding to the plurality of pixel pairs. The plurality of pixel blocks include a first pixel block and a second pixel block. In the first pixel block, the plurality of light receiving pixels are arranged in a first arrangement pattern. In the second pixel block, the plurality of light receiving pixels are arranged in a second arrangement pattern.

[0007] In an imaging device according to an embodiment of the present disclosure, a plurality of pixel blocks are provided with a plurality of light-receiving pixels including color filters of the same color. The plurality of light-receiving pixels are divided into a plurality of pixel pairs, each including two light-receiving pixels. A plurality of lenses are provided at positions corresponding to the plurality of pixel pairs. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example configuration of an imaging device according to an embodiment of the present disclosure. [Figure 2] 2 is an explanatory diagram illustrating an example of the configuration of the pixel array shown in FIG. 1. [Figure 3] 3 is an explanatory diagram illustrating an example of the configuration of a light-receiving pixel shown in FIG. 2. FIG. [Figure 4] 3 is a circuit diagram illustrating an example of the configuration of the pixel block shown in FIG. 2. [Figure 5] 3 is a circuit diagram illustrating an example of the configuration of another pixel block shown in FIG. 2. FIG. [Figure 6] 3 is an explanatory diagram illustrating an example of connections between the plurality of pixel blocks shown in FIG. 2. FIG. [Figure 7] 2 is a block diagram illustrating an example of the configuration of a readout unit illustrated in FIG. 1. FIG. [Figure 8] 2 is an explanatory diagram illustrating an example of the configuration of an image signal shown in FIG. 1. [Figure 9] 2 is an explanatory diagram illustrating an example of the number of effective pixels in the imaging device shown in FIG. [Figure 10] 2 is an explanatory diagram illustrating an example of operation of a plurality of imaging modes in the imaging device shown in FIG. [Figure 11] 2 is an explanatory diagram illustrating an example of an operation of the imaging device shown in FIG. [Figure 12] 2 is a timing waveform diagram illustrating an example of a readout operation in the imaging device shown in FIG. 1. [Figure 13] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 14] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 15] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 16] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 17] 2 is an explanatory diagram illustrating an example of re-mosaic processing in the imaging device shown in FIG. [Figure 18A] FIG. 18 is an explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. 17. [Figure 18B] FIG. 18 is another explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. [Figure 18C] FIG. 18 is another explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. [Figure 19] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 20] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 21] 1. FIG. 4 is a timing waveform diagram illustrating an example of another readout operation in the imaging device shown in FIG. [Figure 22] 1. FIG. 4 is another explanatory diagram illustrating an example of the operation of the imaging device shown in FIG. [Figure 23] 1. FIG. 4 is an explanatory diagram illustrating another example of re-mosaic processing in the imaging device shown in FIG. [Figure 24A] FIG. 24 is an explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. 23. [Figure 24B] FIG. 24 is another explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. 23. [Figure 24C] FIG. 24 is another explanatory diagram illustrating an example of the re-mosaic processing shown in FIG. 23. [Figure 25] FIG. 10 is another explanatory diagram illustrating an example of an operation of the imaging device according to the modified example. [Figure 26] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a light-receiving pixel according to another modified example. [Figure 27] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a light-receiving pixel according to another modified example. [Figure 28] 5A to 5C are explanatory diagrams illustrating an example of an operation of a light-receiving pixel according to an embodiment. [Figure 29]FIG. 10 is another explanatory diagram illustrating an example of an operation of the pixel array according to the embodiment. [Figure 30] FIG. 10 is a block diagram illustrating an example of the configuration of an imaging device according to another modified example. [Figure 31] FIG. 31 is an explanatory diagram illustrating an example of the configuration of the pixel array shown in FIG. 30. [Figure 32] 32 is an explanatory diagram illustrating an example of connections between the plurality of pixel blocks shown in FIG. 31. FIG. [Figure 33] 32 is an explanatory diagram illustrating an example of an operation of the pixel array shown in FIG. 31. [Figure 34] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a pixel array according to another modified example. [Figure 35] 35 is an explanatory diagram illustrating an example of connections between the plurality of pixel blocks shown in FIG. 34. [Figure 36] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a pixel array according to another modified example. [Figure 37] FIG. 1 is an explanatory diagram illustrating an example of use of an imaging device. [Figure 38] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Figure 39] FIG. 2 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order. 1. Embodiment 2. Examples of use of imaging devices 3. Mobile application examples

[0010] <1. Embodiment> [Configuration example] 1 shows an example of the configuration of an imaging device (imaging device 1) according to an embodiment. The imaging device 1 includes a pixel array 11, a driver 12, a reference signal generator 13, a readout unit 20, a signal processor 15, and an imaging controller 18.

[0011] The pixel array 11 has a plurality of light-receiving pixels P arranged in a matrix. The light-receiving pixels P are configured to generate a signal SIG including a pixel voltage Vpix according to the amount of received light.

[0012] Fig. 2 shows an example of the arrangement of light-receiving pixels P in the pixel array 11. Fig. 3 shows an example of a schematic cross-sectional structure of the pixel array 11. The pixel array 11 has a plurality of pixel blocks 100 and a plurality of lenses 101.

[0013] The plurality of pixel blocks 100 includes pixel blocks 100R, 100Gr, 100Gb, and 100B. In the pixel array 11, the plurality of light receiving pixels P are arranged in units (units U) of four pixel blocks 100 (pixel blocks 100R, 100Gr, 100Gb, and 100B).

[0014] The pixel block 100R has eight photosensitive pixels P (photosensitive pixels PR) that include red (R) color filters 115, the pixel block 100Gr has ten photosensitive pixels P (photosensitive pixels PGr) that include green (G) color filters 115, the pixel block 100Gb has ten photosensitive pixels P (photosensitive pixels PGb) that include green (G) color filters 115, and the pixel block 100B has eight photosensitive pixels P (photosensitive pixels PB) that include blue (B) color filters 115. In FIG. 2, differences in the colors of the color filters are represented by hatching. The arrangement pattern of the photosensitive pixels PR in the pixel block 100R and the arrangement pattern of the photosensitive pixels PB in the pixel block 100B are the same, and the arrangement pattern of the photosensitive pixels PGr in the pixel block 100Gr and the arrangement pattern of the photosensitive pixels PGb in the pixel block 100Gb are the same. In unit U, pixel block 100Gr is arranged in the upper left, pixel block 100R is arranged in the upper right, pixel block 100B is arranged in the lower left, and pixel block 100Gb is arranged in the lower right. In this way, pixel blocks 100R, 100Gr, 100Gb, and 100B are arranged in a so-called Bayer array, with pixel block 100 as the unit.

[0015] As shown in FIG. 3 , the pixel array 11 includes a semiconductor substrate 111, a semiconductor region 112, an insulating layer 113, a multilayer wiring layer 114, a color filter 115, and a light-shielding film 116. The semiconductor substrate 111 is a support substrate on which the imaging device 1 is formed and is a P-type semiconductor substrate. The semiconductor region 112 is a semiconductor region provided in a position corresponding to each of the plurality of light-receiving pixels P within the semiconductor substrate 111, and is doped with N-type impurities to form a photodiode PD. The insulating layer 113 is provided at the boundary between the plurality of light-receiving pixels P arranged side by side in the XY plane within the semiconductor substrate 111, and in this example, is a DTI (Deep Trench Isolation) formed using an oxide film or the like. The multilayer wiring layer 114 is provided on the semiconductor substrate 111 on the surface opposite the light incident surface S of the pixel array 11, and includes a plurality of wiring layers and an interlayer insulating film. The wiring in the multilayer wiring layer 114 is configured to connect, for example, transistors (not shown) provided on the surface of the semiconductor substrate 111 to the driving unit 12 and the readout unit 20. The color filter 115 is provided on the semiconductor substrate 111 on the light incident surface S of the pixel array 11. The light-shielding film 116 is provided on the light incident surface S of the pixel array 11 so as to surround two light-receiving pixels P (hereinafter also referred to as pixel pair 90) arranged side by side in the X direction.

[0016] The multiple lenses 101 are so-called on-chip lenses, and are provided on the color filter 115 on the light incident surface S of the pixel array 11. The lenses 101 are provided above two light receiving pixels P (pixel pairs 90) arranged side by side in the X direction. Four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100R, five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gr, five lenses 101 are provided above the ten light receiving pixels P of the pixel block 100Gb, and four lenses 101 are provided above the eight light receiving pixels P of the pixel block 100B. The lenses 101 are arranged side by side in the X direction and the Y direction. The lenses 101 arranged in the Y direction are shifted by one light receiving pixel P in the X direction. In other words, the pixel pairs 90 arranged in the Y direction are shifted by one light receiving pixel P in the X direction.

[0017] With this configuration, the images of the two light-receiving pixels P in a pixel pair 90 corresponding to one lens 101 are shifted from each other. The imaging device 1 generates phase difference data DF based on the so-called image plane phase difference detected by the multiple pixel pairs 90. For example, a camera equipped with the imaging device 1 determines the amount of defocus based on this phase difference data DF, and moves the position of the photographing lens based on the defocus amount. In this way, the camera is able to achieve autofocus.

[0018] Fig. 4 shows an example of the configuration of pixel block 100Gr. Fig. 5 shows an example of the configuration of pixel block 100R. Fig. 6 shows an example of the wiring of pixel blocks 100R, 100Gr, 100Gb, and 100B. Note that in Fig. 6, for ease of explanation, multiple pixel blocks 100 are drawn spaced apart from each other.

[0019] The pixel array 11 has a plurality of control lines TRGL, a plurality of control lines RSTL, a plurality of control lines SELL, and a plurality of signal lines VSL. The control lines TRGL extend in the X direction (the horizontal direction in FIGS. 4 to 6), and one end is connected to the drive unit 12. A control signal STRG is supplied to the control line TRGL by the drive unit 12. The control line RSTL extends in the X direction, and one end is connected to the drive unit 12. A control signal SRST is supplied to the control line RSTL by the drive unit 12. The control line SELL extends in the X direction, and one end is connected to the drive unit 12. A control signal SSEL is supplied to the control line SELL by the drive unit 12. The signal lines VSL extend in the Y direction (the vertical direction in FIGS. 4 to 6), and one end is connected to the readout unit 20. The signal lines VSL transmit signals SIG generated by the light-receiving pixels P to the readout unit 20.

[0020] The pixel block 100Gr (FIG. 4) includes ten photodiodes PD, ten transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The ten photodiodes PD and ten transistors TRG correspond to the ten light-receiving pixels PGr included in the pixel block 100Gr. In this example, the transistors TRG, RST, AMP, and SEL are N-type MOS (Metal Oxide Semiconductor) transistors.

[0021] The photodiode PD is a photoelectric conversion element that generates an electric charge in an amount corresponding to the amount of light received and accumulates the generated electric charge internally. The anode of the photodiode PD is grounded, and the cathode is connected to the source of the transistor TRG.

[0022] The gate of the transistor TRG is connected to the control line TRGL, the source is connected to the cathode of the photodiode PD, and the drain is connected to the floating diffusion FD. The gates of the ten transistors TRG are connected to different control lines TRGL among the ten control lines TRGL (in this example, control lines TRGL1 to TRGL6, TRGL9 to TRGL12).

[0023] The floating diffusion FD is configured to accumulate the charge transferred from the photodiode PD via the transistor TRG. The floating diffusion FD is configured, for example, using a diffusion layer formed on the surface of a semiconductor substrate. In Figure 4, the floating diffusion FD is represented by a symbol for a capacitive element.

[0024] The gate of the transistor RST is connected to a control line RSTL, the power supply voltage VDD is supplied to the drain, and the source is connected to the floating diffusion FD.

[0025] The gate of the transistor AMP is connected to the floating diffusion FD, the power supply voltage VDDH is supplied to the drain, and the source is connected to the drain of the transistor SEL.

[0026] The gate of the transistor SEL is connected to a control line SELL, the drain is connected to the source of the transistor AMP, and the source is connected to a signal line VSL.

[0027] With this configuration, in the photosensitive pixel P, for example, the transistors TRG and RST are turned on based on the control signals STRG and SRST, thereby discharging the charge accumulated in the photodiode PD. Then, the transistors TRG and RST are turned off, starting the exposure period T, and an amount of charge corresponding to the amount of received light is accumulated in the photodiode PD. After the exposure period T ends, the photosensitive pixel P outputs a signal SIG including a reset voltage Vreset and a pixel voltage Vpix to the signal line VSL. Specifically, the transistor SEL is turned on based on the control signal SSEL, electrically connecting the photosensitive pixel P to the signal line VSL. This connects the transistor AMP to a constant current source 21 (described below) of the readout unit 20 and operates as a so-called source follower. Then, as described below, during a P-phase (pre-charge phase) period TP after the voltage of the floating diffusion FD is reset by the transistor RST being turned on, the photosensitive pixel P outputs a voltage corresponding to the voltage of the floating diffusion FD at that time as the reset voltage Vreset. Furthermore, during a D-phase (Data phase) period TD after the transistor TRG is turned on and charge is transferred from the photodiode PD to the floating diffusion FD, the photosensitive pixel P outputs a pixel voltage Vpix that corresponds to the voltage of the floating diffusion FD at that time. The difference voltage between the pixel voltage Vpix and the reset voltage Vreset corresponds to the amount of light received by the photosensitive pixel P during the exposure period T. In this way, the photosensitive pixel P outputs a signal SIG that includes the reset voltage Vreset and the pixel voltage Vpix to the signal line VSL.

[0028] The pixel block 100R (FIG. 5) includes eight photodiodes PD, eight transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The eight photodiodes PD and the eight transistors TRG correspond to the eight light-receiving pixels PR included in the pixel block 100R. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL (in this example, control lines TRGL1, TRGL2, TRGL5 to TRGL10).

[0029] As shown in FIG. 6, pixel blocks 100Gr and 100R that belong to the same row aligned in the X direction are connected to multiple control lines TRGL out of the same 12 control lines TRGL (control lines TRGL1 to TRGL12). In this example, the control lines TRGL1 to TRGL12 are aligned in this order from bottom to top in FIG. 6. The pixel block 100Gr is connected to 10 control lines TRGL (control lines TRGL1 to TRGL6, TRGL9 to TRGL12) out of the 12 control lines TRGL (control lines TRGL1 to TRGL12), and the pixel block 100R is connected to 8 control lines TRGL (control lines TRGL1, TRGL2, TRGL5 to TRGL10) out of the 12 control lines TRGL (control lines TRGL1 to TRGL12).

[0030] Although not shown, the pixel blocks 100Gr and 100R that are aligned in the X direction and belong to the same row are connected to one control line RSTL and one control line SELL.

[0031] 6, pixel blocks 100Gr that are aligned in the Y direction and belong to the same column are connected to one signal line VSL. Similarly, pixel blocks 100R that are aligned in the Y direction and belong to the same column are connected to one signal line VSL.

[0032] Like the pixel block 100R (FIG. 5), the pixel block 100B includes eight photodiodes PD, eight transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The eight photodiodes PD and the eight transistors TRG correspond to the eight light-receiving pixels PB included in the pixel block 100B. The gates of the eight transistors TRG are connected to different control lines TRGL among the eight control lines TRGL.

[0033] Like pixel block 100Gr (FIG. 4), pixel block 100Gb includes ten photodiodes PD, ten transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The ten photodiodes PD and ten transistors TRG correspond to the ten light-receiving pixels PGb included in pixel block 100Gb. The gates of the ten transistors TRG are connected to different control lines TRGL among the ten control lines TRGL.

[0034] As shown in FIG. 6, pixel blocks 100B and 100Gb that are aligned in the same row in the X direction are connected to multiple control lines TRGL out of the same 12 control lines TRGL. Although not shown, pixel blocks 100B and 100Gb that are aligned in the same row in the X direction are connected to one control line RSTL and one control line SELL. As shown in FIG. 6, pixel blocks 100B that are aligned in the same column in the Y direction are connected to one signal line VSL. Similarly, pixel blocks 100Gb that are aligned in the same column in the Y direction are connected to one signal line VSL.

[0035] The drive unit 12 (FIG. 1) is configured to drive a plurality of light receiving pixels P in the pixel array 11 based on instructions from the imaging control unit 18. Specifically, the drive unit 12 drives a plurality of light receiving pixels P in the pixel array 11 by supplying a plurality of control signals STRG to a plurality of control lines TRGL in the pixel array 11, a plurality of control signals SRST to a plurality of control lines RSTL, and a plurality of control signals SSEL to a plurality of control lines SELL.

[0036] The reference signal generation unit 13 is configured to generate a reference signal RAMP based on an instruction from the imaging control unit 18. The reference signal RAMP has a so-called ramp waveform in which the voltage level gradually changes over time during the period (P-phase period TP and D-phase period TD) when the readout unit 20 performs AD conversion. The reference signal generation unit 13 supplies such a reference signal RAMP to the readout unit 20.

[0037] The readout unit 20 is configured to perform AD conversion based on the signal SIG supplied from the pixel array 11 via the signal line VSL, based on an instruction from the imaging control unit 18, to generate an image signal Spic0.

[0038] FIG. 7 shows an example of the configuration of the readout unit 20. In addition to the readout unit 20, FIG. 6 also shows the reference signal generation unit 13, the signal processing unit 15, and the imaging control unit 18. The readout unit 20 has a plurality of constant current sources 21, a plurality of AD (Analog to Digital) conversion units ADC, and a transfer control unit 27. The plurality of constant current sources 21 and the plurality of AD conversion units ADC are provided corresponding to the plurality of signal lines VSL, respectively. Below, the constant current source 21 and the AD conversion unit ADC corresponding to one signal line VSL will be described.

[0039] The constant current source 21 is configured to pass a predetermined current through the corresponding signal line VSL. One end of the constant current source 21 is connected to the corresponding signal line VSL, and the other end is grounded.

[0040] The AD conversion unit ADC is configured to perform AD conversion based on the signal SIG on the corresponding signal line VSL. The AD conversion unit ADC has capacitive elements 22 and 23, a comparator circuit 24, a counter 25, and a latch 26.

[0041] One end of the capacitive element 22 is connected to the signal line VSL and is supplied with the signal SIG, and the other end is connected to the comparison circuit 24. One end of the capacitive element 23 is supplied with the reference signal RAMP supplied from the reference signal generation unit 13, and the other end is connected to the comparison circuit 24.

[0042] The comparator circuit 24 is configured to generate a signal CP by performing a comparison operation based on the signal SIG supplied from the light-receiving pixel P via the signal line VSL and the capacitive element 22, and the reference signal RAMP supplied from the reference signal generator 13 via the capacitive element 23. The comparator circuit 24 sets an operating point by setting the voltages of the capacitive elements 22 and 23 based on a control signal AZ supplied from the imaging controller 18. Thereafter, the comparator circuit 24 performs a comparison operation in which, during the P-phase period TP, it compares the reset voltage Vreset included in the signal SIG with the voltage of the reference signal RAMP, and during the D-phase period TD, it performs a comparison operation in which it compares the pixel voltage Vpix included in the signal SIG with the voltage of the reference signal RAMP.

[0043] The counter 25 is configured to perform a counting operation of counting pulses of the clock signal CLK supplied from the imaging control unit 18 based on the signal CP supplied from the comparison circuit 24. Specifically, the counter 25 counts pulses of the clock signal CLK during the P-phase period TP until the signal CP transitions to generate a count value CNTP, and outputs this count value CNTP as a digital code having multiple bits. Also, the counter 25 counts pulses of the clock signal CLK during the D-phase period TD until the signal CP transitions to generate a count value CNTD, and outputs this count value CNTD as a digital code having multiple bits.

[0044] The latch 26 is configured to temporarily hold the digital code supplied from the counter 25 and to output the digital code to the bus line BUS based on an instruction from the transfer control unit 27 .

[0045] The transfer control unit 27 is configured to control the latches 26 of the plurality of AD conversion units ADC to sequentially output digital codes to the bus wiring BUS based on a control signal CTL supplied from the imaging control unit 18. The readout unit 20 sequentially transfers the plurality of digital codes supplied from the plurality of AD conversion units ADC to the signal processing unit 15 as an image signal Spic0 using this bus wiring BUS.

[0046] The signal processing unit 15 (FIG. 1) is configured to generate an image signal Spic by performing predetermined signal processing based on the image signal Spic0 and instructions from the imaging control unit 18. The signal processing unit 15 has an image data generation unit 16 and a phase difference data generation unit 17. The image data generation unit 16 is configured to generate image data DP indicative of a captured image by performing predetermined image processing based on the image signal Spic0. The phase difference data generation unit 17 is configured to generate phase difference data DF indicative of an image plane phase difference by performing predetermined image processing based on the image signal Spic0. The signal processing unit 15 generates an image signal Spic including the image data DP generated by the image data generation unit 16 and the phase difference data DF generated by the phase difference data generation unit 17.

[0047] 8 shows an example of the image signal Spic. The signal processing unit 15 generates the image signal Spic by, for example, alternately arranging image data DP related to multiple rows of light-receiving pixels P and phase difference data DF related to multiple rows of light-receiving pixels P. The signal processing unit 15 then outputs this image signal Spic.

[0048] The imaging control unit 18 is configured to supply control signals to the drive unit 12, the reference signal generation unit 13, the readout unit 20, and the signal processing unit 15, and to control the operation of these circuits, thereby controlling the operation of the imaging device 1. A control signal Sctl is supplied to the imaging control unit 18 from an external device. This control signal Sctl includes, for example, information about the zoom magnification of so-called electronic zoom. The imaging control unit 18 is configured to control the operation of the imaging device 1 based on the control signal Sctl.

[0049] Here, the light receiving pixel P corresponds to a specific example of a "light receiving pixel" in the present disclosure. The pixel pair 90 corresponds to a specific example of a "pixel pair" in the present disclosure. The pixel block 100 corresponds to a specific example of a "pixel block" in the present disclosure. For example, the pixel block 100Gr corresponds to a specific example of a "first pixel block" in the present disclosure. For example, the pixel block 100R corresponds to a specific example of a "second pixel block" in the present disclosure. The lens 101 corresponds to a specific example of a "lens" in the present disclosure. The control line TRGL corresponds to a specific example of a "control line" in the present disclosure. The insulating layer 113 corresponds to a specific example of an "insulating layer" in the present disclosure.

[0050] [Actions and Actions] Next, the operation and function of the imaging device 1 of this embodiment will be described.

[0051] (Overview of overall operation) First, an overview of the overall operation of the imaging device 1 will be described with reference to FIGS. 1 and 7. The drive unit 12 sequentially drives a plurality of light-receiving pixels P in the pixel array 11 based on instructions from the imaging control unit 18. The reference signal generation unit 13 generates a reference signal RAMP based on instructions from the imaging control unit 18. The light-receiving pixels P output a reset voltage Vreset as a signal SIG during a P-phase period TP, and output a pixel voltage Vpix corresponding to the amount of received light as a signal SIG during a D-phase period TD. The readout unit 20 generates an image signal Spic0 based on the signal SIG supplied from the pixel array 11 via a signal line VSL and instructions from the imaging control unit 18. In the signal processing unit 15, the image data generation unit 16 performs predetermined image processing on the image signal Spic0 to generate image data DP representing a captured image, and the phase difference data generation unit 17 performs predetermined image processing on the image signal Spic0 to generate phase difference data DF representing an image plane phase difference. The signal processing unit 15 generates an image signal Spic including the image data DP and the phase difference data DF. The imaging control unit 18 supplies control signals to the drive unit 12, the reference signal generation unit 13, the readout unit 20, and the signal processing unit 15, and controls the operation of these circuits, thereby controlling the operation of the imaging device 1.

[0052] (Detailed operation) The imaging control unit 18 controls the operation of the imaging device 1 based on a control signal Sctl that includes information about the zoom magnification of the electronic zoom. The zoom operation of the imaging device 1 will be described below.

[0053] Fig. 9 shows an example of the number of light-receiving pixels P (number of effective pixels) related to a captured image when the zoom magnification is changed from 1 to 10. In Fig. 9, the solid line indicates the number of effective pixels of the imaging device 1. Fig. 10 shows an example of zoom operation in the imaging device 1, where (A) shows operation when the zoom magnification is 1, (B) shows operation when the zoom magnification is 2, and (C) shows operation when the zoom magnification is 3.

[0054] The imaging device 1 has three imaging modes M (imaging modes MA, MB, MC). The imaging control unit 18 selects one of the three imaging modes MA to MC based on information about the zoom magnification included in the control signal Sctl. Specifically, as shown in Fig. 9, the imaging control unit 18 selects imaging mode MA when the zoom magnification is less than 2, selects imaging mode MB when the zoom magnification is equal to or greater than 2 but less than 3, and selects imaging mode MC when the zoom magnification is 3 or greater.

[0055] In the imaging mode MA, as shown in FIG. 10A, the imaging device 1 obtains four pixel values ​​V (pixel values ​​VR, VGr, VGb, VB) for each of the plurality of base units U. Specific operations will be described later. In this manner, the imaging device 1 generates image data DP by generating pixel values ​​V at a ratio of four for every 36 light receiving pixels P. If the number of light receiving pixels P in the pixel array 11 is 108 [Mpix], pixel values ​​V for 12 [Mpix] are calculated. As a result, the number of effective pixels becomes 12 [Mpix], as shown in FIG. 9.

[0056] 9, in this imaging mode MA, when the zoom magnification is increased from 1, the number of effective pixels decreases in accordance with the magnification. Then, when the zoom magnification becomes 2, imaging mode M becomes imaging mode MB.

[0057] In imaging mode MB, as shown in FIG. 10(B), the imaging device 1 obtains 16 pixel values ​​V in each of the multiple base units U. Specific operations will be described later. In this manner, the imaging device 1 generates pixel values ​​V at a ratio of 16 for every 36 light-receiving pixels P, thereby generating image data DP. If the number of light-receiving pixels P in the pixel array 11 is 108 [Mpix], pixel values ​​V for 48 [Mpix] are calculated. In reality, because the zoom magnification is 2x, the imaging range is narrowed to 1 / 4 as shown in FIG. 10(B), and the number of effective pixels is 12 [Mpix] (=48 [Mpix] / 4).

[0058] 9, in this imaging mode MB, when the zoom magnification is increased from 2, the number of effective pixels decreases in accordance with the magnification. Then, when the zoom magnification becomes 3, imaging mode M becomes imaging mode MC.

[0059] In the imaging mode MC, as shown in FIG. 10(C), the imaging device 1 obtains 36 pixel values ​​V in each of the multiple units U. Specific operations will be described later. In this manner, the imaging device 1 generates pixel values ​​V at a ratio of 36 for each of the 36 light-receiving pixels P, thereby generating image data DP. If the number of light-receiving pixels P in the pixel array 11 is 108 [Mpix], a captured image of 108 [Mpix] can be obtained. In reality, since the zoom magnification is 3x, the imaging range is narrowed to 1 / 9 as shown in FIG. 10(C), and the number of effective pixels is 12 [Mpix] (=108 [Mpix] / 9).

[0060] As described above, the imaging device 1 is provided with three imaging modes M, which reduces changes in the quality of the captured image when the zoom magnification is changed. That is, for example, if imaging mode MB is omitted and two imaging modes MA and MC are provided, and imaging mode MA is selected when the zoom magnification is less than 2x and imaging mode MC is selected when the zoom magnification is 2x or greater, the number of effective pixels will change significantly, as indicated by the dashed line in FIG. 9 . That is, in this example, when the zoom magnification is 2x, imaging mode MC is selected and the number of effective pixels is 27 [Mpix] (=108 [Mpix] / 4). Therefore, there is a large difference between the number of effective pixels when the zoom magnification is, for example, 1.9x and the number of effective pixels when the zoom magnification is 2x, and therefore the quality of the captured image may change significantly when the zoom magnification is around 2x. On the other hand, the imaging device 1 is provided with three imaging modes M, which reduces changes in the number of effective pixels when the zoom magnification is changed, thereby suppressing changes in the quality of the captured image.

[0061] (imaging mode MA) Fig. 11 shows an example of the operation of the imaging device 1 in the imaging mode MA. In Fig. 11, the light-receiving pixels P marked with "◯" indicate the light-receiving pixels P that are the target of the readout operation.

[0062] 11A, the imaging device 1 generates image data DT1 by calculating a pixel value V corresponding to the amount of light received by the left photosensitive pixel P in each pixel pair 90 in which the lens 101 is provided, in each of the multiple pixel blocks 100. Specifically, the imaging device 1 performs a readout operation on five photosensitive pixels PGr located on the left of five pixel pairs 90 out of the ten photosensitive pixels PGr in the pixel block 100Gr, thereby calculating a pixel value VGr1 at the center of gravity of these five photosensitive pixels PGr. Furthermore, the imaging device 1 performs a readout operation on four photosensitive pixels PR located on the left of four pixel pairs 90 out of the eight photosensitive pixels PR in the pixel block 100R, thereby calculating a pixel value VR1 at the center of gravity of these four photosensitive pixels PR. The imaging device 1 performs a readout operation on four photosensitive pixels PB located on the left of four pixel pairs 90 out of the eight photosensitive pixels PB in the pixel block 100B, thereby calculating a pixel value VB1 at the center of gravity of these four photosensitive pixels PB. The imaging device 1 performs a readout operation on five photosensitive pixels PGb located on the left of five pixel pairs 90 out of the ten photosensitive pixels PGb in the pixel block 100Gb, thereby calculating a pixel value VGb1 at the center of gravity of these five photosensitive pixels PGb. In this way, the imaging device 1 generates image data DT1 (FIG. 11(A)) including pixel values ​​VGr1, VR1, VB1, and VGb1.

[0063] Next, as shown in FIG. 11B, the imaging device 1 generates image data DT2 by calculating a pixel value V corresponding to the amount of light received by all light receiving pixels P in each of the multiple pixel blocks 100. Specifically, the imaging device 1 performs a readout operation on the ten light receiving pixels PGr of the pixel block 100Gr, thereby calculating a pixel value VGr2 at the center of gravity of these ten light receiving pixels PGr. The imaging device 1 also performs a readout operation on the eight light receiving pixels PR of the pixel block 100R, thereby calculating a pixel value VR2 at the center of gravity of these eight light receiving pixels PR. The imaging device 1 performs a readout operation on the eight light receiving pixels PB of the pixel block 100B, thereby calculating a pixel value VB2 at the center of gravity of these eight light receiving pixels PB. The imaging device 1 performs a readout operation on the ten light receiving pixels PGb of the pixel block 100Gb, thereby calculating a pixel value VGb2 at the center of gravity of these ten light receiving pixels PGb. In this way, the imaging device 1 generates image data DT2 (FIG. 11(B)) including pixel values ​​VGr2, VR2, VB2, and VGb2.

[0064] Focusing on a certain pixel block 100Gr, the readout operation for the ten light-receiving pixels PGr in this pixel block 100Gr will be described below.

[0065] 12 illustrates an example of a readout operation, in which (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG (control signal STRGL) supplied to the left-side photosensitive pixel PGr in the pixel pair 90, (D) shows the waveform of the control signal STRG (control signal STRGR) supplied to the right-side photosensitive pixel PGr in the pixel pair 90, (E) shows the waveform of the control signal AZ, (F) shows the waveform of the reference signal RAMP, (G) shows the waveform of the signal SIG, and (H) shows the waveform of the signal CP. In FIGS. 12(F) and (G), the waveforms of the reference signal RAMP and the signal SIG are shown using the same voltage axis. In this description, the waveform of the reference signal RAMP shown in FIG. 12(F) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitance element 23, and the waveform of the signal SIG shown in FIG. 12(G) is the waveform of the voltage supplied to the input terminal of the comparison circuit 24 via the capacitance element 22.

[0066] First, at timing t11, the horizontal period H begins. This causes the driving unit 12 to change the voltage of the control signal SSEL from low to high (FIG. 12A). This causes the transistor SEL in the pixel block 100Gr to be turned on, electrically connecting the pixel block 100Gr to the signal line VSL. Also at timing t11, the driving unit 12 changes the voltage of the control signal SRST from low to high (FIG. 12B). This causes the transistor RST in the pixel block 100Gr to be turned on, and the voltage of the floating diffusion FD is set to the power supply voltage VDD (reset operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusion FD at this time. Also at timing t11, the imaging control unit 18 changes the voltage of the control signal AZ from low to high (FIG. 12E). This causes the comparison circuit 24 of the AD conversion unit ADC to set an operating point by setting the voltages of the capacitance elements 22 and 23. In this way, the voltage of the signal SIG is set to the reset voltage Vreset, and the voltage of the reference signal RAMP is set to the same voltage as the voltage of the signal SIG (reset voltage Vreset) (FIGS. 12(F) and 12(G)).

[0067] Then, after a predetermined time has elapsed since timing t11, the drive unit 12 changes the voltage of the control signal SRST from high to low (FIG. 12(B)), which turns off the transistor RST in the pixel block 100Gr and ends the reset operation.

[0068] Next, at timing t12, the imaging control unit 18 changes the voltage of the control signal AZ from high level to low level (FIG. 12(E)), which causes the comparison circuit 24 to finish setting the operating point.

[0069] Also, at timing t12, the reference signal generating unit 13 sets the voltage of the reference signal RAMP to voltage V1 (FIG. 12(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG, and the comparator circuit 24 changes the voltage of the signal CP from low to high (FIG. 12(H)).

[0070] Then, during a period from timing t13 to t15 (P-phase period TP), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t13, the reference signal generation unit 13 starts to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate of change (FIG. 12(F)). Also at this timing t13, the imaging control unit 18 starts generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0071] Then, at timing t14, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (reset voltage Vreset) (FIGS. 12(F), (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (FIG. 12(H)). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTP) is a value corresponding to the reset voltage Vreset. The latch 26 holds this count value CNTP. Then, the counter 25 resets its count value.

[0072] Next, at timing t15, the imaging control unit 18 stops generating the clock signal CLK as the P-phase period TP ends. At timing t15, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (FIG. 12(F)). Then, during the period after timing t15, the readout unit 20 supplies the count value CNTP held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0073] Next, at timing t16, the imaging control unit 18 sets the voltage of the reference signal RAMP to voltage V1 (FIG. 12(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (reset voltage Vreset), and the comparator circuit 24 changes the voltage of the signal CP from low to high (FIG. 12(H)).

[0074] Next, at timing t17, the drive unit 12 changes the voltage of the control signal STRGL from low to high (FIG. 12(C)). As a result, in the five light-receiving pixels PGr arranged on the left of the pixel pair 90, the transistors TRG are turned on, and the charge generated in the photodiodes PD is transferred to the floating diffusions FD (charge transfer operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusions FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix1 (FIG. 12(G)).

[0075] Then, after a predetermined time has elapsed from timing t17, the drive unit 12 changes the voltage of the control signal STRGL from high to low (FIG. 12C), which turns off the transistors TRG in the five light receiving pixels PGr arranged to the left of the pixel pair 90, and the charge transfer operation ends.

[0076] Then, during a period from timing t18 to t20 (D-phase period TD1), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t18, the reference signal generation unit 13 starts to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate of change (FIG. 12(F)). Also at this timing t18, the imaging control unit 18 starts to generate a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0077] Then, at timing t19, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix1) (FIGS. 12(F), (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (FIG. 12(H)). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD1) is a value corresponding to the pixel voltage Vpix1. The latch 26 holds this count value CNTD1. Then, the counter 25 resets its count value.

[0078] Next, at timing t20, the imaging control unit 18 stops generating the clock signal CLK as the D-phase period TD1 ends. At timing t20, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (FIG. 12(F)). Then, during the period after timing t20, the readout unit 20 supplies the count value CNTD1 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0079] Next, at timing t21, the imaging control unit 18 sets the voltage of the reference signal RAMP to voltage V1 (FIG. 12(F)). As a result, the voltage of the reference signal RAMP becomes higher than the voltage of the signal SIG (pixel voltage Vpix1), and the comparison circuit 24 changes the voltage of the signal CP from low to high (FIG. 12(H)).

[0080] Next, at timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from low to high, respectively (FIGS. 12(C) and 12(D)). As a result, in the ten light-receiving pixels PGr in the pixel block 100Gr, the transistors TRG are turned on, and the charge generated in the photodiodes PD is transferred to the floating diffusions FD (charge transfer operation). The pixel block 100Gr then outputs a voltage corresponding to the voltage of the floating diffusions FD at this time. In this way, the voltage of the signal SIG becomes the pixel voltage Vpix2 (FIG. 12(G)).

[0081] Then, after a predetermined time has elapsed from timing t22, the drive unit 12 changes the voltages of the control signals STRGL and STRGR from high to low, respectively (FIGS. 12C and 12D). As a result, the transistors TRG in the ten light-receiving pixels PGr are turned off, and the charge transfer operation is terminated.

[0082] Then, during the period from timing t23 to t25 (D-phase period TD2), the AD conversion unit ADC performs AD conversion based on the signal SIG. Specifically, first, at timing t23, the reference signal generation unit 13 starts to decrease the voltage of the reference signal RAMP from voltage V1 at a predetermined rate of change (FIG. 12(F)). Also at this timing t23, the imaging control unit 18 starts generating a clock signal CLK. The counter 25 of the AD conversion unit ADC performs a counting operation to count pulses of this clock signal CLK.

[0083] Then, at timing t24, the voltage of the reference signal RAMP falls below the voltage of the signal SIG (pixel voltage Vpix2) (FIGS. 12(F), (G)). This causes the comparison circuit 24 of the AD conversion unit ADC to change the voltage of the signal CP from high to low (FIG. 12(H)). The counter 25 of the AD conversion unit ADC stops counting based on this transition of the signal CP. The count value of the counter 25 at this time (count value CNTD2) is a value corresponding to the pixel voltage Vpix2. The latch 26 holds this count value CNTD2. Then, the counter 25 resets its count value.

[0084] Next, at timing t25, the imaging control unit 18 stops generating the clock signal CLK as the D-phase period TD2 ends. At timing t25, the reference signal generation unit 13 also stops changing the voltage of the reference signal RAMP (FIG. 12(F)). Then, during the period after timing t25, the readout unit 20 supplies the count value CNTD2 held in the latch 26 to the signal processing unit 15 as the image signal Spic0.

[0085] Next, at timing t26, the drive unit 12 changes the voltage of the control signal SSEL from high to low (FIG. 12(A)), which turns off the transistor SEL in the pixel block 100Gr and electrically disconnects the pixel block 100Gr from the signal line VSL.

[0086] In this manner, the readout unit 20 supplies the image signal Spic0, which includes the count values ​​CNTP, CNTD1, and CNTD2, to the signal processing unit 15. The signal processing unit 15 generates the pixel value VGr1 shown in FIG. 11A and the pixel value VGr2 shown in FIG. 11B, for example, based on the count values ​​CNTP, CNTD1, and CNTD2 included in the image signal Spic0, using the principle of correlated double sampling. Specifically, the signal processing unit 15 generates the pixel value VGr1, for example, by subtracting the count value CNTP from the count value CNTD1. Since the count value CNTD1 is a value corresponding to the sum of the amounts of light received by the five light-receiving pixels PGr located on the left of the five pixel pairs 90 in the pixel block 100Gr, the signal processing unit 15 can generate the pixel value VGr1 shown in FIG. 11A based on this count value CNTD1. Similarly, the signal processing unit 15 generates the pixel value VGr2, for example, by subtracting the count value CNTP from the count value CNTD2. Since the count value CNTD2 is a value corresponding to the sum of the amounts of light received at the 10 light receiving pixels PGr of the pixel block 100Gr, the signal processing unit 15 can generate the pixel value VGr2 shown in Figure 11(B) based on this count value CNTD2.

[0087] The above has been explained regarding pixel block 100Gr, but the same applies to pixel blocks 100R, 100Gb, and 100B. In this way, the signal processing unit 15 generates image data DT1 including pixel values ​​VR1, VGr1, VGb1, and VB1, and image data DT2 including pixel values ​​VR2, VGr2, VGb2, and VB2, as shown in FIG.

[0088] FIG. 13 shows an example of image processing by the signal processing unit 15 in the imaging mode MA.

[0089] First, the signal processing unit 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0090] Specifically, the signal processing unit 15 calculates the pixel value VGr3 by subtracting the pixel value VGr1 in the image data DT1 from the pixel value VGr2 in the image data DT2. This pixel value VGr3 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the right of the five pixel pairs 90 in the pixel block 100Gr. That is, the pixel value VGr1 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the left of the five pixel pairs 90 in the pixel block 100Gr, and the pixel value VGr2 is a value corresponding to the sum of the amounts of light received by the ten light receiving pixels PGr in the pixel block 100Gr. Therefore, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGr located on the right of the five pixel pairs 90 in the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value corresponding to the sum of the amounts of light received at the five light receiving pixels PGr located on the right of the five pixel pairs 90, and therefore, as shown in Figure 13, the pixel value VGr3 is located at the center of gravity of these five light receiving pixels PGr.

[0091] Similarly, the signal processing unit 15 calculates a pixel value VR3 by subtracting the pixel value VR1 in the image data DT1 from the pixel value VR2 in the image data DT2. This pixel value VR3 is a value corresponding to the sum of the amounts of light received by the four light receiving pixels PR located on the right of the four pixel pairs 90 in the pixel block 100R. The pixel value VR3 is located at the center of gravity of the four light receiving pixels PR located on the right of the four pixel pairs 90 in the pixel block 100R.

[0092] The signal processing unit 15 calculates a pixel value VB3 by subtracting the pixel value VB1 in the image data DT1 from the pixel value VB2 in the image data DT2. This pixel value VB3 is a value corresponding to the sum of the amounts of light received by the four light receiving pixels PB located on the right of the four pixel pairs 90 in the pixel block 100B. The pixel value VB3 is located at the center of gravity of the four light receiving pixels PB located on the right of the four pixel pairs 90 in the pixel block 100B.

[0093] The signal processing unit 15 calculates a pixel value VGb3 by subtracting the pixel value VGb1 in the image data DT1 from the pixel value VGb2 in the image data DT2. This pixel value VGb3 is a value corresponding to the sum of the amounts of light received by the five light receiving pixels PGb located on the right of the five pixel pairs 90 in the pixel block 100Gb. The pixel value VGb3 is located at the center of gravity of the five light receiving pixels PGb located on the right of the five pixel pairs 90 in the pixel block 100Gb.

[0094] Then, the image data generating section 16 of the signal processing section 15 performs predetermined image processing based on the image data DT2 to generate image data DP (FIG. 10(A)) representing the captured image.

[0095] Furthermore, the phase difference data generation unit 17 of the signal processing unit 15 performs predetermined image processing based on the image data DT1 and DT3 to generate phase difference data DF indicating the image plane phase difference. That is, the image data DT1 has a pixel value V of the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V of the light-receiving pixel P located on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generation unit 17 can generate the phase difference data DF based on the image data DT1 and DT3.

[0096] 2, in the imaging device 1, the lenses 101 are arranged in parallel in the X direction and the Y direction in the pixel array 11, and therefore it is possible to generate phase difference data DF with high resolution over the entire surface of the pixel array 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus, thereby improving image quality.

[0097] Here, the imaging mode MA corresponds to a specific example of a "first imaging mode" in the present disclosure. The pixel values ​​in the image data DT1 correspond to a specific example of a "first pixel value" in the present disclosure. The pixel values ​​in the image data DT2 correspond to a specific example of a "second pixel value" in the present disclosure. The pixel values ​​in the image data DT3 correspond to a specific example of a "third pixel value" in the present disclosure.

[0098] (Shooting mode MB) Fig. 14 shows an example of operation of the imaging device 1 in imaging mode MB. Fig. 15 shows more specifically the operation shown in Fig. 14. In Figs. 14 and 15, the photosensitive pixels P marked with "◯" indicate the photosensitive pixels P that are the target of the readout operation.

[0099] 15(A), in the pixel block 100Gr, the imaging device 1 performs a readout operation on the photosensitive pixel PGr located on the left of one of the five pixel pairs 90 in the pixel block 100Gr, and calculates a pixel value VGr1 at the position of this photosensitive pixel PGr. Next, as shown in FIG. 15(B), the imaging device 1 performs a readout operation on two photosensitive pixels PGr in that pixel pair 90, and calculates a pixel value VGr2 at the center of gravity of these two photosensitive pixels PGr.

[0100] 15(A), in the pixel block 100R, the imaging device 1 performs a readout operation on the photosensitive pixel PR located on the left of one of the four pixel pairs 90 in the pixel block 100R, and calculates a pixel value VR1 at the position of this photosensitive pixel PR. Next, as shown in FIG. 15(B), the imaging device 1 performs a readout operation on two photosensitive pixels PR in that pixel pair 90, and calculates a pixel value VR2 at the center of gravity of these two photosensitive pixels PR.

[0101] Similarly, in pixel block 100B, as shown in Fig. 15(A), the imaging device 1 performs a readout operation on the photosensitive pixel PB located on the left of one of the four pixel pairs 90 in pixel block 100B, and calculates a pixel value VB1 at the position of this photosensitive pixel PB. Next, as shown in Fig. 15(B), the imaging device 1 performs a readout operation on two photosensitive pixels PB in that pixel pair 90, and calculates a pixel value VB2 at the center of gravity of these two photosensitive pixels PB.

[0102] Similarly, in pixel block 100Gb, as shown in Fig. 15(A), the imaging device 1 performs a readout operation on the photosensitive pixel PGb located on the left of one of the five pixel pairs 90 in pixel block 100Gb, and calculates a pixel value VGb1 at the position of this photosensitive pixel PGb. Next, as shown in Fig. 15(B), the imaging device 1 performs a readout operation on two photosensitive pixels PGb in that pixel pair 90, and calculates a pixel value VGb2 at the center of gravity of these two photosensitive pixels PGb.

[0103] The readout operations in Figures 15(A) and (B) are the same as the readout operation in the imaging mode MA described above (Figure 12). The imaging device 1 then performs the operations in Figures 15(C) and (D), 15(E) and (F), 15(G) and (H), 15(I) and (J), and 15(K) and (L). In this way, the imaging device 1 generates image data DT1 (Figure 14(A)) including pixel values ​​VGr1, VR1, VB1, and VGb1, and image data DT2 (Figure 14(B)) including pixel values ​​VGr2, VR2, VB2, and VGb2.

[0104] FIG. 16 shows an example of image processing by the signal processing unit 15 in the imaging mode MB.

[0105] First, the signal processing unit 15 performs subtraction processing based on the image data DT1 and DT2 to generate image data DT3.

[0106] Specifically, the signal processing unit 15 calculates five pixel values ​​VGr3 by subtracting five pixel values ​​VGr1 in the image data DT1 from five pixel values ​​VGr2 in the image data DT2, respectively. These pixel values ​​VGr3 correspond to the amount of light received by the light receiving pixel PGr located on the right of each pixel pair 90 in the pixel block 100Gr. That is, the pixel value VGr1 corresponds to the amount of light received by the light receiving pixel PGr located on the left of each pixel pair 90 in the pixel block 100Gr, and the pixel value VGr2 corresponds to the sum of the amounts of light received by the two light receiving pixels PGr in this pixel pair 90. Therefore, by subtracting the pixel value VGr1 from the pixel value VGr2, a value corresponding to the amount of light received by the light receiving pixel PGr located on the right of each pixel pair 90 in the pixel block 100Gr is obtained. In this way, the pixel value VGr3 is a value corresponding to the amount of light received at the photosensitive pixel PGr located on the right in the pixel pair 90, and therefore, as shown in Figure 16, the pixel value VGr3 is located at the position of the photosensitive pixel PGr located on the right in the pixel pair 90.

[0107] Similarly, the signal processing unit 15 calculates four pixel values ​​VR3 by subtracting the four pixel values ​​VR1 in the image data DT1 from the four pixel values ​​VR2 in the image data DT2, respectively. These pixel values ​​VR3 correspond to the amount of light received by the light receiving pixel PR located on the right in the pixel pair 90 of the pixel block 100R. These pixel values ​​VR3 are located at the position of the light receiving pixel PR located on the right in the pixel pair 90.

[0108] The signal processing unit 15 calculates four pixel values ​​VB3 by subtracting the four pixel values ​​VB1 in the image data DT1 from the four pixel values ​​VB2 in the image data DT2, respectively. These pixel values ​​VB3 correspond to the amount of light received by the light-receiving pixel PB located on the right of the pixel pair 90 in the pixel block 100B. These pixel values ​​VB3 are located at the positions of the four light-receiving pixels PB located on the right of the pixel pair 90.

[0109] The signal processing unit 15 calculates five pixel values ​​VGb3 by subtracting the five pixel values ​​VGb1 in the image data DT1 from the five pixel values ​​VGb2 in the image data DT2, respectively. The pixel value VGb3 corresponds to the amount of light received at the photosensitive pixel PGb located on the right of the pixel pair 90 in the pixel block 100Gb. The pixel value VGb3 is located at the position of the photosensitive pixel PGb located on the right of the pixel pair 90.

[0110] Then, the image data generating unit 16 of the signal processing unit 15 generates image data DP (FIG. 10(B)) representing the captured image by performing predetermined image processing based on the image data DT2, as shown in Fig. 16. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V.

[0111] 17 shows an example of re-mosaic processing in imaging mode M2, where (A) shows image data DT2, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on the image data DT2. In Fig. 17(B), "◯" indicates the position of pixel value V in image data DT2, and "□" indicates the position of pixel value V in image data DT4.

[0112] Fig. 18A shows the re-mosaic processing for pixel block 100R, Fig. 18B shows the re-mosaic processing for pixel blocks 100Gr and 100Gb, and Fig. 18C shows the re-mosaic processing for pixel block 100B. In Figs. 18A to 18C, (A) shows image data DT2, and (B) shows image data DT4 generated by the re-mosaic processing.

[0113] As shown in FIG. 17A, the image data DT2 includes 18 pixel values ​​V for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array 11 is 108 [Mpix], the image data DT2 includes 54 [Mpix] worth of pixel values ​​V. On the other hand, as shown in FIG. 17C, the image data DT4 includes 16 pixel values ​​V for every 36 light-receiving pixels P. For example, if the number of light-receiving pixels P in the pixel array 11 is 108 [Mpix], the image data DT4 includes 48 [Mpix] worth of pixel values ​​V. Furthermore, in the image data DT4, four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generation unit 16 generates this image data DT4 by correcting the pixel values ​​V in the image data DT2 and rearranging the pixel values ​​V.

[0114] Specifically, as shown in Fig. 18A, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VR2 in the image data DT2, for example, by performing an interpolation process, and generates a pixel value VR4 based on the pixel value V for one surface. Similarly, as shown in Fig. 18B, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VGr2 and VGb2 in the image data DT2, for example, by performing an interpolation process, and generates pixel values ​​VGr4 and VGb4 based on the pixel value V for one surface. As shown in Fig. 18C, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VB2 in the image data DT2, for example, by performing an interpolation process, and generates a pixel value VB4 based on the pixel value V for one surface.

[0115] In this way, the image data generating unit 16 generates the image data DT4 shown in Fig. 17(C) based on the image data DT2 shown in Fig. 17(A). Then, the image data generating unit 16 performs predetermined image processing based on this image data DT4 to generate image data DP (Fig. 10(B)) representing the captured image.

[0116] 16, the phase difference data generation unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing predetermined image processing based on the image data DT1 and DT3. That is, the image data DT1 has a pixel value V of the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V of the light-receiving pixel P located on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generation unit 17 can generate the phase difference data DF based on the image data DT1 and DT3.

[0117] The phase difference data generation unit 17, like the image data generation unit 16, generates phase difference data DF by rearranging pixel values ​​V based on the image data DT1 and DT3. That is, the image data DT1 and DT3 include left and right pixel values ​​V of the pixel pair 90 at a ratio of 18 out of 36 light-receiving pixels P. Therefore, the phase difference data generation unit 17 rearranges the pixel values ​​V so that left and right pixel values ​​V of the pixel pair 90 are included at a ratio of 16 out of 36 light-receiving pixels P. In this way, the phase difference data generation unit 17 can generate phase difference data DF corresponding to the image data DP generated by the image data generation unit 16.

[0118] Here, imaging mode MB corresponds to a specific example of a "second imaging mode" in the present disclosure. The pixel values ​​in image data DT1 correspond to a specific example of a "first pixel value" in the present disclosure. The pixel values ​​in image data DT2 correspond to a specific example of a "second pixel value" in the present disclosure. The pixel values ​​in image data DT3 correspond to a specific example of a "third pixel value" in the present disclosure. The pixel values ​​in image data DT4 correspond to a specific example of a "fourth pixel value" in the present disclosure.

[0119] (imaging mode MC) Fig. 19 shows an example of operation of the imaging device 1 in imaging mode MC. Fig. 20 shows more specifically the operation shown in Fig. 19. In Figs. 19 and 20, the photosensitive pixels P marked with "◯" indicate the photosensitive pixels P that are the target of the readout operation.

[0120] In pixel block 100Gr, as shown in Figure 20(A), the imaging device 1 targets one photosensitive pixel PGr out of the ten photosensitive pixels PGr in pixel block 100Gr as the target of the readout operation, and calculates the pixel value VGr1 at the position of this photosensitive pixel PGr.

[0121] FIG. 21 illustrates an example of a readout operation, where (A) shows the waveform of the control signal SSEL, (B) shows the waveform of the control signal SRST, (C) shows the waveform of the control signal STRG supplied to the light-receiving pixel PGr to be readout, (D) shows the waveform of the control signal AZ, (E) shows the waveform of the reference signal RAMP, (F) shows the waveform of the signal SIG, and (G) shows the waveform of the signal CP. This readout operation is the same as the readout operation in the imaging modes MA and MB (FIG. 12), except that the operations at times t21 to t26 are omitted. In this manner, the readout unit 20 supplies the image signal Spic0, including the count values ​​CNTP and CNTD1, to the signal processing unit 15. The signal processing unit 15 generates a pixel value VGr1, for example, based on the count values ​​CNTP and CNTD1 included in the image signal Spic0, using the principle of correlated double sampling. Specifically, the signal processing unit 15 generates the pixel value VGr1, for example, by subtracting the count value CNTP from the count value CNTD1.

[0122] Similarly, in pixel block 100R, as shown in Figure 20(A), the imaging device 1 targets one of the eight light-receiving pixels PR in pixel block 100R as the target for the readout operation, and calculates the pixel value VR1 at the position of this light-receiving pixel PR.

[0123] Similarly, in pixel block 100B, as shown in Figure 20(A), the imaging device 1 targets one of the eight photosensitive pixels PB in pixel block 100B as the target of the readout operation, and calculates the pixel value VB1 at the position of this photosensitive pixel PB.

[0124] Similarly, in pixel block 100Gb, as shown in Figure 20(A), the imaging device 1 targets one photosensitive pixel PGb out of the ten photosensitive pixels PGb in pixel block 100Gb as the target of the readout operation, and calculates the pixel value VGb1 of this photosensitive pixel PGb.

[0125] The imaging device 1 then performs the operations shown in (B) to (L) of Fig. 20. In this way, the imaging device 1 generates image data DT1 (Fig. 19) including pixel values ​​VGr1, VR1, VB1, and VGb1.

[0126] FIG. 22 shows an example of image processing by the signal processing unit 15 in the imaging mode MC.

[0127] The image data generating unit 16 of the signal processing unit 15 performs predetermined image processing based on the image data DT1 to generate image data DP (FIG. 10C) representing the captured image. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V.

[0128] 23 shows an example of re-mosaic processing in imaging mode MC, where (A) shows image data DT1, (B) shows the positions of pixel values ​​V before and after re-mosaic processing, and (C) shows image data DT4 generated by re-mosaic processing based on image data DT1. In Fig. 23(B), "◯" indicates the position of pixel value V in image data DT1, and "□" indicates the position of pixel value V in image data DT4.

[0129] Fig. 24A shows the re-mosaic processing for pixel block 100R, Fig. 24B shows the re-mosaic processing for pixel blocks 100Gr and 100Gb, and Fig. 24C shows the re-mosaic processing for pixel block 100B. In Figs. 24A to 24C, (A) shows image data DT1, and (B) shows image data DT4 generated by the re-mosaic processing.

[0130] In the image data DT4, four pixel values ​​VGr4, VR4, VB4, and VGb4 are arranged in a Bayer array. The image data generation unit 16 generates such image data DT4 by correcting the pixel values ​​V in the image data DT1 and rearranging the pixel values ​​V.

[0131] Specifically, as shown in Fig. 24A, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VR1 in the image data DT1, for example, by performing an interpolation process, and generates a pixel value VR4 based on the pixel value V for one surface. Similarly, as shown in Fig. 24B, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VGr1 and VGb1 in the image data DT1, for example, by performing an interpolation process, and generates pixel values ​​VGr4 and VGb4 based on the pixel value V for one surface. As shown in Fig. 24C, the image data generation unit 16 calculates a pixel value V for one surface based on a plurality of pixel values ​​VB1 in the image data DT1, for example, by performing an interpolation process, and generates a pixel value VB4 based on the pixel value V for one surface.

[0132] In this way, the image data generation unit 16 generates the image data DT4 shown in Fig. 23(C) based on the image data DT1 shown in Fig. 23(A). Then, the image data generation unit 16 performs predetermined image processing based on this image data DT4 to generate image data DP (Fig. 10(C)) representing the captured image.

[0133] 22, the phase difference data generation unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing predetermined image processing based on the image data DT1. That is, the image data DT1 includes image data (image data DT11) having a pixel value V at the light-receiving pixel P arranged on the left of the plurality of pixel pairs 90, and image data (image data DT12) having a pixel value V at the light-receiving pixel P arranged on the right of the plurality of pixel pairs 90. Therefore, the phase difference data generation unit 17 can generate the phase difference data DF based on this image data DT1 (image data DT11, DT12).

[0134] Here, the imaging mode MC corresponds to a specific example of a "third imaging mode" in the present disclosure. The pixel values ​​in the image data DT1 correspond to a specific example of a "first pixel value" in the present disclosure. The pixel values ​​in the image data DT2 correspond to a specific example of a "second pixel value" in the present disclosure. The pixel values ​​in the image data DT3 correspond to a specific example of a "third pixel value" in the present disclosure. The pixel values ​​in the image data DT4 correspond to a specific example of a "fourth pixel value" in the present disclosure.

[0135] As described above, the imaging device 1 is provided with a plurality of pixel blocks 100, each of which has a plurality of light-receiving pixels P including color filters of the same color. These light-receiving pixels P are divided into a plurality of pixel pairs 90, each of which includes two light-receiving pixels P. A plurality of lenses 101 are provided at positions corresponding to these pixel pairs 90. This allows the imaging device 1 to generate phase difference data DF with high resolution across the entire surface of the pixel array 11. Therefore, for example, a camera equipped with such an imaging device 1 can achieve highly accurate autofocus at various zoom magnifications. As a result, the imaging device 1 can improve image quality.

[0136] Furthermore, in the imaging device 1, the number of light receiving pixels in a certain pixel block 100 is made greater than the number of light receiving pixels in another certain pixel block 100. Specifically, in this example, the number of light receiving pixels PGr in the pixel block 100Gr and the number of light receiving pixels PGb in the pixel block 100Gb are made greater than the number of light receiving pixels PR in the pixel block 100R and the number of light receiving pixels PB in the pixel block 100B. This makes it possible to increase, for example, green light receiving sensitivity, thereby improving the image quality of the captured image.

[0137] Furthermore, the imaging device 1 is provided with three imaging modes MA to MC, and re-mosaic processing is performed in imaging modes MB and MC. As a result, in the imaging device 1, the number of effective pixels in imaging mode MB is adjusted particularly by re-mosaic processing in imaging mode MB, and changes in the number of effective pixels when the zoom magnification is changed can be reduced, thereby suppressing changes in the image quality of the captured image.

[0138] [effect] As described above, in this embodiment, a plurality of pixel blocks are provided, each having a plurality of light-receiving pixels including color filters of the same color. These light-receiving pixels P are divided into a plurality of pixel pairs 90, each including two light-receiving pixels. A plurality of lenses are provided at positions corresponding to these pixel pairs 90. This makes it possible to achieve highly accurate autofocus, thereby improving image quality.

[0139] In this embodiment, the number of light-receiving pixels in a certain pixel block is greater than the number of light-receiving pixels in another certain pixel block, so that the quality of the captured image can be improved.

[0140] In this embodiment, three imaging modes MA to MC are provided, and re-mosaic processing is performed in imaging modes MB and MC, so that changes in the image quality of captured images can be reduced when the zoom magnification is changed.

[0141] [Variation 1] In the above embodiment, in imaging mode MC, the imaging device 1 individually performs a readout operation on each of the multiple light-receiving pixels P as shown in FIGS. 19 and 20 , but this is not limited to this. Instead, for example, in imaging mode MC, as in imaging mode MB ( FIGS. 14 and 15 ), a pixel value V may be generated by performing a readout operation on the light-receiving pixel P located on the left of the pixel pair 90, and then a pixel value V may be generated by performing a readout operation on the two light-receiving pixels P in the pixel pair 90. In this case, in imaging mode MC, the signal processing unit 15 generates the image data DT1 shown in FIG. 14(A) and the image data DT2 shown in FIG. 14(B) by a readout operation, as in imaging mode MB. The operation of the signal processing unit 15 in this case will be described below.

[0142] FIG. 25 shows an example of image processing by the signal processing unit 15 in the imaging mode MC.

[0143] First, the signal processing unit 15 performs subtraction processing based on the image data DT1 and DT2, in the same way as in the case of the imaging mode MB (FIG. 16), to generate image data DT3.

[0144] Then, as shown in FIG. 25, the image data generation unit 16 of the signal processing unit 15 performs predetermined image processing based on the image data DT1 and DT3 to generate image data DP (FIG. 10(C)) representing the captured image. That is, the image data DT1 has a pixel value V at the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V at the light-receiving pixel P located on the right of the plurality of pixel pairs 90, so the image data DT1 and DT3 have pixel values ​​V at all light-receiving pixels. Therefore, the image data generation unit 16 performs predetermined image processing based on the image data DT1 and DT3 to generate the image data DP. This predetermined image processing includes a re-mosaic process that corrects the pixel values ​​V and rearranges the pixel values ​​V. This re-mosaic process is the same as in the above-described embodiment (FIGS. 23, 24A to 24C).

[0145] 25, the phase difference data generation unit 17 of the signal processing unit 15 generates phase difference data DF indicating the image plane phase difference by performing predetermined image processing based on the image data DT1. That is, the image data DT1 has a pixel value V at the light-receiving pixel P located on the left of the plurality of pixel pairs 90, and the image data DT3 has a pixel value V at the light-receiving pixel P located on the right of the plurality of pixel pairs 90, so the phase difference data generation unit 17 can generate the phase difference data DF based on this image data DT1 and DT3.

[0146] As a result, in the imaging device according to this modification, the frame rate can be increased in the imaging mode MC. That is, in the above embodiment, the readout operation shown in FIG. 21 is performed in each of FIGS. 20(A) to 20(L), and therefore the readout operation shown in FIG. 21 is performed 12 times. On the other hand, in this modification, the readout operation shown in FIG. 12 is performed in each of FIGS. 15(A), (B), 15(C), (D), 15(E), (F), 15(G), (H), 15(I), (J), and 15(K), (L), and therefore the readout operation shown in FIG. 12 is performed six times. The time required to perform the readout operation shown in FIG. 12 six times is shorter than the time required to perform the readout operation shown in FIG. 21 12 times. As a result, in this modification, the frame rate can be increased.

[0147] [Variation 2] In the above embodiment, as shown in FIG. 3, the insulating layer 113 is provided at the boundary between the plurality of photosensitive pixels P within the semiconductor substrate 111. However, this is not limited to this. Instead, for example, as shown in FIG. 26, an insulating layer 123 may be provided at the boundary between the plurality of pixel pairs 90 within the semiconductor substrate 111, and an impurity layer 124 doped with, for example, a P-type impurity may be provided between the two photosensitive pixels P in the pixel pair 90. Note that this is not limited to this, and for example, as shown in FIG. 27, the impurity layer 124 may be provided in a portion between the two photosensitive pixels P in the pixel pair 90. This makes it possible to reduce the difference between the amount of light received by the photosensitive pixel PGr and the amount of light received by the photosensitive pixel PGb, as will be described below.

[0148] That is, in the pixel array 11 ( FIG. 3 ) according to the above embodiment, an insulating layer 113 made of, for example, an oxide film is provided between two light receiving pixels P in a pixel pair 90. Therefore, for example, as shown in FIG. 28 , light L incident on the pixel array 11 and collected by the lens 101 may be reflected, for example, at the upper part of the insulating layer 113, and the reflected light may enter the semiconductor region 112, travel straight, and then enter the semiconductor region 112 of an adjacent light receiving pixel P via the insulating layer 113. Therefore, as shown in FIG. 29 , for example, red light LR may enter a light receiving pixel PR of a pixel block 100R and then enter the semiconductor region 112 of a light receiving pixel PGr of a pixel block 100Gr adjacent to the pixel block 100R. In this case, the light receiving pixel PGr of the pixel block 100Gr may detect not only green light but also red light LR. Similarly, for example, blue light LB may enter the semiconductor region 112 of the photosensitive pixel PGb in the pixel block 100Gb adjacent to the pixel block 100B after entering the photosensitive pixel PB in the pixel block 100B. In this case, the photosensitive pixel PGb in the pixel block 100Gb detects not only green light but also blue light LB. In this way, the photosensitive pixel PGr associated with green detects not only green light but also red light LR, and the photosensitive pixel PGb associated with green detects not only green light but also blue light LB. Therefore, even when uniform green light is incident, a difference will occur between the amount of light received by the photosensitive pixel PGr and the amount of light received by the photosensitive pixel PGb depending on the balance between the light intensities of the red light LR and the blue light LB.

[0149] On the other hand, in this modified example (FIGS. 26 and 27), an impurity layer 124 is provided between the two photosensitive pixels P in the pixel pair 90. Because the difference between the refractive index of the semiconductor substrate 111 and the refractive index of the impurity layer 124 is small, the possibility of the incident light L being reflected can be reduced. This reduces, for example, the possibility that the photosensitive pixel PGr corresponding to green detects red light LR, and the possibility that the photosensitive pixel PGb corresponding to green detects blue light LB. This reduces the possibility of a difference occurring between the amount of light received by the photosensitive pixel PGr and the amount of light received by the photosensitive pixel PGb.

[0150] 26 and 27 , in this example, the impurity layer 124 is provided in place of the insulating layer 123 entirely or partially between the two light-receiving pixels P in all pixel pairs 90 in the pixel array 11, but this is not limited to this. For example, the impurity layer 124 may be provided in place of the insulating layer 123 entirely or partially between the two light-receiving pixels P only for pixel pairs 90 including a red color filter 115. Furthermore, the impurity layer 124 may be provided in place of the insulating layer 123 entirely or partially between the two light-receiving pixels P only for pixel pairs 90 including a red color filter 115 and pixel pairs 90 including a blue color filter 115.

[0151] [Variation 3] In the above embodiment, as shown in Fig. 2, pixel blocks 100Gr and 100Gb each including 10 light receiving pixels P and pixel blocks 100R and 100B each including 8 light receiving pixels P are provided, but the present invention is not limited to this. Some examples will be described in detail below.

[0152] 30 shows an example of the configuration of an imaging device 2 according to this modification. The imaging device 2 includes a pixel array 31, a drive unit 32, and a signal processing unit .

[0153] 31 shows an example of the arrangement of light-receiving pixels P in the pixel array 31. The pixel array 31 has a plurality of pixel blocks 300 and a plurality of lenses 101.

[0154] The plurality of pixel blocks 300 includes pixel blocks 300R, 300Gr, 300Gb, and 300B. In the pixel array 31, the plurality of light receiving pixels P are arranged in units (units U) of four pixel blocks 300 (pixel blocks 300R, 300Gr, 300Gb, and 300B).

[0155] The pixel block 300R has eight light receiving pixels P (light receiving pixels PR) that include red (R) color filters 115, the pixel block 300Gr has eight light receiving pixels P (light receiving pixels PGr) that include green (G) color filters 115, the pixel block 300Gb has ten light receiving pixels P (light receiving pixels PGb) that include green (G) color filters 115, and the pixel block 300B has eight light receiving pixels P (light receiving pixels PB) that include blue (B) color filters 115. The arrangement pattern of the light receiving pixels PR in the pixel block 300R, the arrangement pattern of the light receiving pixels PGr in the pixel block 300Gr, the arrangement pattern of the light receiving pixels PGb in the pixel block 300Gb, and the arrangement pattern of the light receiving pixels PB in the pixel block 300B are all the same. In unit U, pixel block 300Gr is arranged in the upper left, pixel block 300R is arranged in the upper right, pixel block 300B is arranged in the lower left, and pixel block 300Gb is arranged in the lower right. In this way, pixel blocks 300R, 300Gr, 300Gb, and 300B are arranged in a so-called Bayer array, with pixel block 300 as a unit.

[0156] In the pixel array 31, the plurality of light receiving pixels P are arranged side by side in a diagonal direction. That is, in the pixel array 11 (FIG. 2) according to the above embodiment, the plurality of light receiving pixels P are arranged side by side in the X direction and the Y direction, but in the pixel array 31 (FIG. 31), the plurality of light receiving pixels P are arranged side by side in a diagonal direction. As a result, the two light receiving pixels P in the pixel pair 90 are also arranged side by side in a diagonal direction. A lens 101 is provided above this pixel pair 90.

[0157] The pixel block 300 (pixel blocks 300R, 300Gr, 300Gb, and 300B) has the same configuration as the pixel block 100R (FIG. 5) according to the above embodiment, and includes eight photodiodes PD, eight transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The eight photodiodes PD and the eight transistors TRG correspond to the eight light-receiving pixels P included in the pixel block 300, respectively.

[0158] Fig. 32 shows an example of wiring for pixel blocks 300R, 300Gr, 300Gb, and 300B. For ease of explanation, the pixel blocks 300 are depicted separated from one another in Fig. 32.

[0159] Similar to the pixel array 11 according to the above embodiment, the pixel array 31 has a plurality of control lines TRGL, a plurality of control lines RSTL, a plurality of control lines SELL, and a plurality of signal lines VSL. The control lines TRGL extend in the X direction, and one end is connected to the drive unit 32. The control lines RSTL extend in the X direction, and one end is connected to the drive unit 32. The control lines SELL extend in the X direction, and one end is connected to the drive unit 32. The signal lines VSL extend in the Y direction, and one end is connected to the readout unit 20.

[0160] The pixel blocks 300Gr and 300R that are aligned in the same row in the X direction are connected to the same eight control lines TRGL. Although not shown, the pixel blocks 300Gr and 300R that are aligned in the same row in the X direction are connected to one control line RSTL and one control line SELL. The pixel blocks 300Gr that are aligned in the same column in the Y direction are connected to one signal line VSL. Similarly, the pixel blocks 300R that are aligned in the same column in the Y direction are connected to one signal line VSL.

[0161] Similarly, pixel blocks 300B and 300Gb that are aligned in the same row in the X direction are connected to the same eight control lines TRGL. Although not shown, pixel blocks 300B and 300Gb that are aligned in the same row in the X direction are connected to one control line RSTL and one control line SELL. Furthermore, pixel blocks 300B that are aligned in the same column in the Y direction are connected to one signal line VSL. Similarly, pixel blocks 300Gb that are aligned in the same column in the Y direction are connected to one signal line VSL.

[0162] As a result, in the pixel array 31, for example, the line density of the control lines TRGL can be reduced compared to the pixel array 11 according to the above embodiment. Specifically, in the pixel array 11 according to the above embodiment (FIGS. 2 and 6), the line density of the control lines TRGL is four lines per side of the length of one side of the light receiving pixel P. On the other hand, in the pixel array 31 according to this modification (FIGS. 31 and 32), the line density of the control lines TRGL is 2.83 lines (=2√2) per side of the length of one side of the light receiving pixel P. As a result, in the pixel array 31, for example, it is possible to make it easier to wire the control lines TRGL, or to increase the resolution by reducing the pitch of the light receiving pixels P.

[0163] The drive unit 32 (FIG. 30) is configured to drive a plurality of light-receiving pixels P in the pixel array 31 based on instructions from the imaging control unit 18. The signal processing unit 35 is configured to generate an image signal Spic by performing predetermined signal processing based on the image signal Spic0 and instructions from the imaging control unit 18. The signal processing unit 35 has an image data generation unit 36 ​​and a phase difference data generation unit 37.

[0164] For example, the imaging device 2 can select a first imaging mode M when the zoom magnification is less than 2, select a second imaging mode M when the zoom magnification is 2 or more and less than 2√2, and select a third imaging mode M when the zoom magnification is 2√2 or more.

[0165] In this way, by arranging the light receiving pixels P in the imaging device 2 as shown in Figure 31, even if light L is reflected by the insulating layer 113 as shown in Figure 28, the possibility of a difference occurring between the amount of light received by the light receiving pixel PGr and the amount of light received by the light receiving pixel PGb can be reduced.

[0166] That is, even in this case, as shown in FIG. 33, it is possible that the green-related photosensitive pixel PGr detects not only green light but also red light LR, and the green-related photosensitive pixel PGb detects not only green light but also blue light LB. However, the arrangement positions of the pixel block 300Gr and the pixel block 300Gb are symmetrical with respect to the pixel block 300R, and similarly, they are symmetrical with respect to the pixel block 300B. Therefore, for example, the amount of red light LR leaking from the pixel block 300R to the pixel block 300Gr is approximately the same as the amount of red light LR leaking from the pixel block 300R to the pixel block 300Gb. Similarly, for example, the amount of blue light LB leaking from the pixel block 300B to the pixel block 300Gr is approximately the same as the amount of blue light LB leaking from the pixel block 300B to the pixel block 300Gb. Therefore, for example, it is possible to reduce the possibility of a difference occurring between the amount of light received by the photosensitive pixel PGr and the amount of light received by the photosensitive pixel PGb depending on the balance between the light intensity of the red light LR and the light intensity of the blue light LB.

[0167] In this example, four pixel blocks 300 (pixel blocks 300R, 300Gr, 300Gb, and 300B) are arranged as a unit (unit U), but this is not limiting. An example in which eight pixel blocks are arranged as a unit will be described in detail below. Similar to the image pickup device 2 (FIG. 30), the image pickup device 2A according to this modification includes a pixel array 31A, a drive unit 32A, and a signal processing unit 35A.

[0168] 34 shows an example of the arrangement of the light receiving pixels P in the pixel array 31 A. The pixel array 31 A has a plurality of pixel blocks 300 and a plurality of lenses 101.

[0169] The pixel array 31A has a plurality of pixel blocks 300 and a plurality of lenses 101, similar to the pixel array 31 (FIG. 31).

[0170] The multiple pixel blocks 300 include pixel blocks 300R1, 300R2, 300Gr1, 300Gr2, 300Gb1, 300Gb2, 300B1, and 300B2. Each of the pixel blocks 300 (pixel blocks 300R1, 300R2, 300Gr1, 300Gr2, 300Gb1, 300Gb2, 300B1, and 300B2) has four photosensitive pixels P. Specifically, each of the pixel blocks 300R1 and 300R2 has four photosensitive pixels PR, each of the pixel blocks 300Gr1 and 300Gr2 has four photosensitive pixels PGr, each of the pixel blocks 300Gb1 and 300Gb2 has four photosensitive pixels PGb, and each of the pixel blocks 300B1 and 300B2 has four photosensitive pixels PB. The pixel block 300 includes four photodiodes PD, four transistors TRG, a floating diffusion FD, and transistors RST, AMP, and SEL. The four photodiodes PD and the four transistors TRG correspond to the four light-receiving pixels P included in the pixel block 300, respectively.

[0171] Fig. 35 shows an example of wiring for pixel blocks 300R1, 300R2, 300Gr1, 300Gr2, 300Gb1, 300Gb2, 300B1, and 300B2. Note that, for the sake of convenience, the pixel blocks 300 are depicted separated from one another in Fig. 34.

[0172] The pixel blocks 300Gr1, 300Gr2, 300R1, and 300R2 that are aligned in the same row in the X direction are connected to the same four control lines TRGL. Although not shown, the pixel blocks 300Gr1, 300Gr2, 300R1, and 300R2 that are aligned in the same row in the X direction are connected to one control line RSTL and one control line SELL. The pixel blocks 300Gr1 that are aligned in the same column in the Y direction are connected to one signal line VSL, and the pixel blocks 300Gr2 that are aligned in the same column in the Y direction are connected to one signal line VSL. Similarly, the pixel blocks 300R1 that are aligned in the same column in the Y direction are connected to one signal line VSL, and the pixel blocks 300R2 that are aligned in the same column in the Y direction are connected to one signal line VSL.

[0173] Similarly, pixel blocks 300B1, 300B2, 300Gb1, and 300Gb2 that belong to the same row and are aligned in the X direction are connected to the same four control lines TRGL. Although not shown, pixel blocks 300B1, 300B2, 300Gb1, and 300Gb2 that belong to the same row and are aligned in the X direction are connected to one control line RSTL and one control line SELL. Furthermore, pixel blocks 300B1 that belong to the same column and are aligned in the Y direction are connected to one signal line VSL, and pixel blocks 300B2 that belong to the same column and are aligned in the Y direction are connected to one signal line VSL. Similarly, pixel blocks 300Gb1 that belong to the same column and are aligned in the Y direction are connected to one signal line VSL, and pixel blocks 300Gb2 that belong to the same column and are aligned in the Y direction are connected to one signal line VSL.

[0174] As a result, in the pixel array 31A, for example, the line density of the control lines TRGL can be reduced compared to the pixel array 31 according to the above-described modified example. Specifically, in this pixel array 31A, the line density of the control lines TRGL is 1.41 lines (=√2) per side length of the light receiving pixel P. As a result, in the pixel array 31A, for example, it is possible to make it easier to wire the control lines TRGL or to reduce the pitch of the light receiving pixels P.

[0175] The drive unit 32A is configured to drive the plurality of light receiving pixels P in the pixel array 31 based on instructions from the imaging control unit 18. The signal processing unit 35A is configured to generate an image signal Spic by performing predetermined signal processing based on the image signal Spic0 and instructions from the imaging control unit 18. The signal processing unit 35A has an image data generation unit 36A and a phase difference data generation unit 37A.

[0176] In the above example, the two light receiving pixels P in the pixel pair 90 are arranged side by side in a diagonal direction, but this is not limited to this. Instead, for example, the two light receiving pixels P in the pixel pair 90 may be arranged side by side in the X direction, as in the pixel array 31B shown in FIG.

[0177] [Other variations] Two or more of these variations may also be combined.

[0178] <2. Examples of use of imaging device> 37 shows an example of use of the imaging device 1 according to the above embodiment. The imaging device 1 described above can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as follows:

[0179] ·Digital cameras, mobile devices with camera functions, and other devices that take images for viewing purposes - Devices used for traffic purposes, such as in-vehicle sensors that take pictures of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping, and for recognizing the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles. A device used in home appliances such as televisions, refrigerators, and air conditioners to capture user gestures and operate the appliances according to those gestures. -Medical and healthcare equipment, such as endoscopes and devices that take blood vessel images using infrared light - Security devices such as surveillance cameras for crime prevention and cameras for person authentication Cosmetic devices such as skin measuring devices that take pictures of the skin and microscopes that take pictures of the scalp Sports equipment such as action cameras and wearable cameras for sports purposes Agricultural equipment such as cameras for monitoring the condition of fields and crops

[0180] <3. Application examples for mobile devices> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0181] FIG. 38 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0182] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 38, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (interface) 12053.

[0183] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, etc.

[0184] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0185] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc., based on the received images.

[0186] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0187] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0188] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drivetrain control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including avoiding or mitigating collisions between vehicles, following based on the distance between vehicles, maintaining vehicle speed, warning of vehicle collisions, or warning of vehicle lane departure.

[0189] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0190] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0191] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 38, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0192] FIG. 39 is a diagram showing an example of the installation position of the imaging unit 12031.

[0193] In FIG. 39, a vehicle 12100 has imaging units 12101, 12102, 12103, 12104, and 12105 as the imaging unit 12031.

[0194] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The forward images acquired by the imaging units 12101 and 12105 are mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0195] 39 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

[0196] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0197] For example, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of automatic driving, which runs autonomously without relying on driver operation.

[0198] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drivetrain control unit 12010.

[0199] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. The pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104, which are infrared cameras, and then performing pattern matching on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0200] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 of the above-described configuration. The imaging device mounted on the vehicle can improve the image quality of the captured image. As a result, the vehicle control system 12000 can achieve, with high accuracy, functions such as a vehicle collision avoidance or collision mitigation function, a following driving function based on the following distance, a vehicle speed maintenance function, a vehicle collision warning function, and a vehicle lane departure warning function.

[0201] The present technology has been described above by giving the embodiments, modifications, and specific application examples thereof, but the present technology is not limited to these embodiments and can be modified in various ways.

[0202] For example, the arrangement of pixel blocks in the pixel array and the arrangement of light-receiving pixels P in the pixel blocks are not limited to those described in the above embodiments, and various arrangements are possible.

[0203] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0204] The present technology can be configured as follows: According to the present technology configured as follows, it is possible to improve the image quality of a captured image.

[0205] (1) a plurality of pixel blocks each including a plurality of light receiving pixels each including a color filter of the same color, the plurality of light receiving pixels being divided into a plurality of pixel pairs each including two light receiving pixels; a plurality of lenses respectively provided at positions corresponding to the plurality of pixel pairs; An imaging device comprising: (2) the two light-receiving pixels are arranged side by side in a first direction, In each of the plurality of pixel blocks, two of the pixel pairs aligned in a second direction intersecting the first direction are arranged with a shift in the first direction. The imaging device according to (1) above. (3) Further, two control lines are connected to the two light-receiving pixels, respectively, and extend in the first direction. The imaging device according to (2) above. (4) The display device further includes two control lines connected to the two light-receiving pixels, respectively, and extending in a third direction different from the first direction and the second direction. The imaging device according to (2) above. (5) the plurality of pixel blocks include a first pixel block and a second pixel block; In the first pixel block, the plurality of light receiving pixels are arranged in a first arrangement pattern, In the second pixel block, the plurality of light-receiving pixels are arranged in a second arrangement pattern. The imaging device according to any one of (1) to (4). (6) the number of the plurality of light-receiving pixels in the first pixel block is greater than the number of the plurality of light-receiving pixels in the second pixel block; The plurality of light-receiving pixels included in the first pixel block include the green color filter. The imaging device according to (5) above. (7) In each of the plurality of pixel blocks, the plurality of light-receiving pixels are arranged in a predetermined arrangement pattern. The imaging device according to any one of (1) to (4). (8) The pixel element further includes an insulating layer provided between adjacent pixel pairs among the plurality of pixel pairs. The imaging device according to any one of (1) to (7). (9) Each of the plurality of pixel blocks further includes an impurity layer provided in a portion between the two light receiving pixels in each of the plurality of pixel pairs, The insulating layer is provided in a portion between the two light-receiving pixels where the impurity layer is not provided. The imaging device according to (8) above. (10) In the pixel blocks among the plurality of pixel blocks, the pixel block has the plurality of light receiving pixels including the color filter of a first color, and further includes an impurity layer provided in a part between the two light receiving pixels in each of the plurality of pixel pairs, In the pixel block, the insulating layer is provided in a portion between the two light-receiving pixels where the impurity layer is not provided. The imaging device according to (8) above. (11) In each of the plurality of pixel blocks, an impurity layer is further provided between the two light-receiving pixels in each of the plurality of pixel pairs. The imaging device according to (8) above. (12) In the pixel blocks having the plurality of light receiving pixels including the color filter of the first color among the plurality of pixel blocks, an impurity layer is further provided between the two light receiving pixels in each of the plurality of pixel pairs. The imaging device according to (8) above. (13) The method further includes a signal processing unit that generates image data and phase difference data based on the amount of light received in the plurality of pixel blocks. The imaging device according to any one of (1) to (12). (14) the imaging device has a first imaging mode; In the first imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a first pixel value of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a second pixel value of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a third pixel value by subtracting the first pixel value from the second pixel value; generating the image data based on the second pixel value, and generating the phase difference data based on the first pixel value and the third pixel value; The imaging device according to (13) above. (15) the imaging device has a second imaging mode; In the second imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a plurality of first pixel values ​​of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a plurality of second pixel values ​​of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a plurality of third pixel values ​​by subtracting the plurality of first pixel values ​​from the plurality of second pixel values, respectively; generating the image data based on the plurality of second pixel values, and generating the phase difference data based on the plurality of first pixel values ​​and the plurality of third pixel values; The imaging device according to (13) above. (16) The signal processing unit generates the image data by performing an interpolation process based on the second pixel values ​​to generate a plurality of fourth pixel values ​​at a plurality of pixel positions different from the pixel positions of the second pixel values. The imaging device according to (15) above. (17) The number of the plurality of fourth pixel values ​​is less than the number of the plurality of second pixel values. The imaging device according to (16) above. (18) the imaging device has a third imaging mode; In the third imaging mode, the signal processing unit calculating a plurality of first pixel values ​​of the light receiving pixels for each of the plurality of pixel blocks based on the amount of light received by each of the plurality of light receiving pixels; The image data is generated based on the plurality of first pixel values, and the phase difference data is generated based on a plurality of pixel values ​​for one of the two light-receiving pixels and a plurality of pixel values ​​for the other of the two light-receiving pixels, among the plurality of first pixel values. The imaging device according to (13) above. (19) The signal processing unit generates the image data by performing an interpolation process based on the plurality of first pixel values ​​to generate a plurality of fourth pixel values ​​at a plurality of pixel positions different from the plurality of pixel positions of the plurality of first pixel values. The imaging device according to (18) above. (20) the imaging device has a third imaging mode; In the third imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a plurality of first pixel values ​​of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a plurality of second pixel values ​​of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a plurality of third pixel values ​​by subtracting the plurality of first pixel values ​​from the plurality of second pixel values, respectively; generating the image data based on the plurality of first pixel values ​​and the plurality of third pixel values, and generating the phase difference data based on the plurality of first pixel values ​​and the plurality of third pixel values; The imaging device according to (13) above.

[0206] This application claims priority based on Japanese Patent Application No. 2020-208541, filed on December 16, 2020, with the Japan Patent Office, the entire contents of which are incorporated herein by reference.

[0207] Those skilled in the art will recognize that various modifications, combinations, subcombinations, and variations may occur depending on design requirements and other factors, and are intended to be within the scope of the appended claims and their equivalents.

Claims

1. a plurality of pixel blocks each including a plurality of light receiving pixels each including a color filter of the same color, the plurality of light receiving pixels being divided into a plurality of pixel pairs each including two light receiving pixels; a plurality of lenses respectively provided at positions corresponding to the plurality of pixel pairs; Equipped with the two light-receiving pixels are arranged side by side in a first direction, In each of the plurality of pixel blocks, two of the pixel pairs aligned in a second direction intersecting the first direction are arranged with a shift in the first direction. Imaging device.

2. Further, two control lines are connected to the two light-receiving pixels, respectively, and extend in the first direction. The imaging device according to claim 1 .

3. The pixel electrode further includes two control lines respectively connected to the two light-receiving pixels and extending in a third direction different from the first direction and the second direction. The imaging device according to claim 1 .

4. the plurality of pixel blocks include a first pixel block and a second pixel block; In the first pixel block, the plurality of light receiving pixels are arranged in a first arrangement pattern, In the second pixel block, the plurality of light-receiving pixels are arranged in a second arrangement pattern. The imaging device according to claim 1 .

5. the number of the plurality of light-receiving pixels in the first pixel block is greater than the number of the plurality of light-receiving pixels in the second pixel block; The plurality of light-receiving pixels included in the first pixel block include the green color filter. The imaging device according to claim 4 .

6. In each of the plurality of pixel blocks, the plurality of light-receiving pixels are arranged in a predetermined arrangement pattern. The imaging device according to claim 1 .

7. The pixel element further includes an insulating layer provided between adjacent pixel pairs among the plurality of pixel pairs. The imaging device according to claim 1 .

8. Each of the plurality of pixel blocks further includes an impurity layer provided in a portion between the two light-receiving pixels in each of the plurality of pixel pairs, The insulating layer is provided in a portion between the two light-receiving pixels where the impurity layer is not provided. The imaging device according to claim 7 .

9. a pixel block having the plurality of light receiving pixels including the color filter of a first color among the plurality of pixel blocks, further including an impurity layer provided in a part between the two light receiving pixels in each of the plurality of pixel pairs; In the pixel block, the insulating layer is provided in a portion between the two light-receiving pixels where the impurity layer is not provided. The imaging device according to claim 7 .

10. In each of the plurality of pixel blocks, an impurity layer is further provided between the two light-receiving pixels in each of the plurality of pixel pairs. The imaging device according to claim 7 .

11. In the pixel blocks having the plurality of light receiving pixels including the color filter of the first color among the plurality of pixel blocks, an impurity layer is further provided between the two light receiving pixels in each of the plurality of pixel pairs. The imaging device according to claim 7 .

12. The method further includes a signal processing unit that generates image data and phase difference data based on the amount of light received in the plurality of pixel blocks. The imaging device according to claim 1 .

13. the imaging device has a first imaging mode; In the first imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a first pixel value of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a second pixel value of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a third pixel value by subtracting the first pixel value from the second pixel value; generating the image data based on the second pixel value, and generating the phase difference data based on the first pixel value and the third pixel value; The imaging device according to claim 12.

14. the imaging device has a second imaging mode; In the second imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a plurality of first pixel values ​​of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a plurality of second pixel values ​​of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a plurality of third pixel values ​​by subtracting the plurality of first pixel values ​​from the plurality of second pixel values, respectively; generating the image data based on the plurality of second pixel values, and generating the phase difference data based on the plurality of first pixel values ​​and the plurality of third pixel values; The imaging device according to claim 12.

15. The signal processing unit generates the image data by performing an interpolation process based on the second pixel values ​​to generate a plurality of fourth pixel values ​​at a plurality of pixel positions different from the pixel positions of the second pixel values. The imaging device according to claim 14.

16. The number of the plurality of fourth pixel values ​​is less than the number of the plurality of second pixel values. The imaging device according to claim 15.

17. the imaging device has a third imaging mode; In the third imaging mode, the signal processing unit calculating a plurality of first pixel values ​​of the light-receiving pixels for each of the plurality of pixel blocks based on the amount of light received by each of the plurality of light-receiving pixels; The image data is generated based on the plurality of first pixel values, and the phase difference data is generated based on a plurality of pixel values ​​for one of the two light-receiving pixels and a plurality of pixel values ​​for the other of the two light-receiving pixels, among the plurality of first pixel values. The imaging device according to claim 12.

18. The signal processing unit generates the image data by performing an interpolation process based on the plurality of first pixel values ​​to generate a plurality of fourth pixel values ​​at a plurality of pixel positions different from the plurality of pixel positions of the plurality of first pixel values. The imaging device according to claim 17.

19. the imaging device has a third imaging mode; In the third imaging mode, the signal processing unit for each of the plurality of pixel blocks, calculating a plurality of first pixel values ​​of the plurality of pixel pairs based on the amount of light received by one of the two light receiving pixels in each of the plurality of pixel pairs, and calculating a plurality of second pixel values ​​of the plurality of pixel pairs based on the amount of light received by both of the two light receiving pixels in each of the plurality of pixel pairs; calculating a plurality of third pixel values ​​by subtracting the plurality of first pixel values ​​from the plurality of second pixel values, respectively; generating the image data based on the plurality of first pixel values ​​and the plurality of third pixel values, and generating the phase difference data based on the plurality of first pixel values ​​and the plurality of third pixel values; The imaging device according to claim 12.

20. A photodiode comprising: a plurality of photosensitive pixels each including a color filter of the same color; and a plurality of pixel blocks divided into a plurality of pixel pairs each including two photosensitive pixels; a plurality of lenses respectively provided at positions corresponding to the plurality of pixel pairs; Equipped with the plurality of pixel blocks include a first pixel block and a second pixel block; In the first pixel block, the plurality of light receiving pixels are arranged in a first arrangement pattern, In the second pixel block, the plurality of light-receiving pixels are arranged in a second arrangement pattern. Imaging device.

Images