Image sensor and imaging device

Abstract

The present invention pertains to a floating diffusion (FD) sharing-type device where the number of elements per pixel is reduced.　An image sensor according to the present invention includes a plurality of main pixels, a plurality of sub-pixels, a floating diffusion, a plurality of first on-chip lenses, and a plurality of second on-chip lenses. The plurality of main pixels are arranged in a two-dimensional grid. The plurality of sub-pixels have lower sensitivity than the plurality of main pixels and are each disposed in a region surrounded by a plurality of the main pixels. The floating diffusion is disposed in a region surrounded by a plurality of the main pixels and is shared by said plurality of main pixels and a plurality of the sub-pixels.

Description

Image sensor and imaging device

[0001] The present technology relates to an image sensor, and more particularly to an image sensor and an imaging device that expand a dynamic range.

[0002] High-Dynamic-Range (HDR) compositing has been used to improve image quality in imaging devices, etc. For example, an FD (Floating Diffusion) sharing imaging device has been proposed in which two pixels with different sensitivities share one FD (Floating Diffusion) (see, for example, Patent Document 1).

[0003] JP 2023-011858 A

[0004] In the conventional technology described above, the pixel voltages of two pixels with different sensitivities are converted into analog-to-digital (AD) signals, and the converted signals are then combined to expand the dynamic range. However, in the image pickup device described above, it is difficult to further reduce the number of elements per pixel.

[0005] This technology was developed in light of these circumstances, and aims to reduce the number of elements per pixel in a shared FD device.

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof is an image sensor including a plurality of main pixels arranged in a two-dimensional lattice pattern, a plurality of sub-pixels each having a lower sensitivity than the plurality of main pixels and arranged in an area surrounded by the plurality of main pixels, and a floating diffusion arranged in the area surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels, thereby providing the effect of reducing the number of elements per pixel.

[0007] In addition, in the first aspect, the pixel may further include a plurality of first on-chip lenses each covering one of the plurality of main pixels and a plurality of second on-chip lenses arranged in an area surrounded by the plurality of first on-chip lenses, wherein the plurality of main pixels include first, second, third, and fourth main pixels, the plurality of sub-pixels include first, second, and third sub-pixels, and the floating diffusion is shared by the first, second, third, and fourth main pixels and the first, second, and third sub-pixels, thereby providing an effect of reducing the number of elements per pixel.

[0008] In addition, in this first aspect, the pixel may further include an FD gain transistor that opens and closes a path between a predetermined common node and the floating diffusion, the plurality of sub-pixels may include first and second sub-pixels, the first sub-pixel may include a first photodiode, and the second sub-pixel may include a second photodiode, thereby providing an effect of controlling charge-to-voltage conversion efficiency.

[0009] In addition, in this first aspect, the first subpixel may further include a first wiring capacitance, a first transfer transistor that transfers charge from the first photodiode to the first wiring capacitance, and a first gain transistor that opens and closes a path between the first wiring capacitance and the common node, and the second subpixel may further include a second wiring capacitance, a second transfer transistor that transfers charge from the second photodiode to the second wiring capacitance, and a second gain transistor that opens and closes a path between the second wiring capacitance and the common node, thereby providing an effect that charge-to-voltage conversion efficiency is controlled in three stages.

[0010] In addition, in this first aspect, the first subpixel may further include a first capacitance element, a first transfer transistor that transfers charge from the first photodiode to the first capacitance element, and a first gain transistor that opens and closes a path between the first wiring capacitance and the common node, and the second subpixel may further include a second capacitance element, a second transfer transistor that transfers charge from the second photodiode to the second capacitance element, and a second gain transistor that opens and closes a path between the second wiring capacitance and the common node, thereby providing an effect that charge-to-voltage conversion efficiency is controlled in three stages.

[0011] In the first aspect, visible light may be incident on the floating diffusion, which brings about an effect that the image sensor can be easily manufactured.

[0012] In addition, in the first aspect, a light-shielding portion that shields the floating diffusion from light may be further provided, thereby improving the image quality of image data.

[0013] In the first aspect, the light-shielding portion may be a black color filter, thereby improving the image quality of the image data.

[0014] In the first aspect, the light-shielding portion may be a plurality of overlapping color filters, thereby improving the image quality of the image data.

[0015] In the first aspect, the light blocking portion may be a metal plate, thereby improving the image quality of the image data.

[0016] In the first aspect, the metal plate may contain tungsten, thereby improving the image quality of the image data.

[0017] In addition, in this first aspect, the second on-chip lenses may be arranged in regions that cover the sub-pixels, respectively, and may not be arranged in a region that covers the floating diffusion, thereby improving the image quality of image data.

[0018] In the first aspect, the second on-chip lenses may be disposed in regions that cover the sub-pixels and the floating diffusion, respectively, thereby providing an effect of facilitating the manufacture of the image sensor.

[0019] In the first aspect, the plurality of main pixels may be arranged in a Bayer array, thereby providing an effect that color image data is captured.

[0020] In addition, in this first aspect, the plurality of main pixels may be arranged in a Quad Bayer array, thereby providing an effect that color image data is captured.

[0021] In the first aspect, the plurality of main pixels may include first, second, third, and fourth main pixels, the plurality of sub-pixels may include first, second, third, and fourth sub-pixels, and the floating diffusion may be shared by the first, second, third, and fourth main pixels and the first, second, third, and fourth sub-pixels, thereby eliminating missing sub-pixels.

[0022] In addition, in the first aspect, the image sensor may further include a signal processing unit that corrects pixel signals of the plurality of sub-pixels using pixel signals of surrounding sub-pixels, thereby improving image quality.

[0023] In the first aspect, the plurality of main pixels may be arranged in a Bayer array or a Quad Bayer array, thereby providing an effect that color image data is captured.

[0024] In the first aspect, each of the plurality of sub-pixels may include a capacitance element, thereby providing an effect of controlling charge-to-voltage conversion efficiency.

[0025] According to a second aspect of the present technology, there is provided an imaging device including: a plurality of main pixels arranged in a two-dimensional lattice pattern; a plurality of sub-pixels each having a lower sensitivity than the plurality of main pixels and arranged in an area surrounded by the plurality of main pixels; a floating diffusion arranged in the area surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels; a plurality of first on-chip lenses each covering one of the plurality of main pixels; a plurality of second on-chip lenses arranged in the area surrounded by the plurality of first on-chip lenses; and a column processing unit that converts pixel voltages of the plurality of main pixels and the plurality of sub-pixels into digital signals, thereby achieving an effect of reducing the number of elements per pixel.

[0026] 1 is a block diagram showing a configuration example of an imaging device according to a first embodiment of the present technology. FIG. 2 is a block diagram showing a configuration example of an image sensor according to the first embodiment of the present technology. FIG. 3 is a circuit diagram showing a configuration example of an FD shared block according to the first embodiment of the present technology. FIG. 4 is a plan view showing an example of a pixel array according to the first embodiment of the present technology. FIG. 5 is a plan view showing another example of a pixel array according to the first embodiment of the present technology. FIG. 6 is a plan view showing an example of an array of on-chip lenses according to the first embodiment of the present technology. FIG. 7 is an example cross-sectional view of a pixel array unit according to the first embodiment of the present technology. FIG. 8 is a plan view showing an example of a layout of elements in an FD shared block according to the first embodiment of the present technology. FIG. 9 is a timing chart showing an example of an operation of an image sensor when converting pixel voltages up to VP4 according to the first embodiment of the present technology. FIG. 10 is a timing chart showing an example of an operation of an image sensor when converting pixel voltages VP5 and onwards according to the first embodiment of the present technology. FIG. 11 is a circuit diagram showing a configuration example of an FD shared block according to a second embodiment of the present technology. FIG. 12 is a plan view showing an example of a layout of elements in an FD shared block according to the second embodiment of the present technology. FIG. 13 is a plan view showing an example of a pixel array according to a third embodiment of the present technology. FIG. 10 is a plan view showing an example of an arrangement of on-chip lenses according to a fourth embodiment of the present technology. FIG. 11 is a plan view showing an example of a pixel arrangement according to a fifth embodiment of the present technology. FIG. 12 is a diagram for explaining a correction method according to the fifth embodiment of the present technology. FIG. 13 is a circuit diagram showing an example of a configuration of an FD shared block according to the fifth embodiment of the present technology. FIG. 14 is a plan view showing an example of a layout of elements in the FD shared block according to the fifth embodiment of the present technology. FIG. 15 is a plan view showing an example of a layout of elements in the FD shared block when an MIM capacitor is used according to the fifth embodiment of the present technology. FIG. 16 is a plan view showing an example of a Quad Bayer arrangement according to the fifth embodiment of the present technology. FIG. 17 is a plan view showing another example of the Quad Bayer arrangement according to the fifth embodiment of the present technology. FIG. 18 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 19 is an explanatory diagram showing an example of installation positions of an outside vehicle information detection unit and an imaging unit.

[0027] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. First embodiment (an example in which four main pixels and three sub-pixels share an FD) 2. Second embodiment (an example in which four main pixels and three sub-pixels including a capacitive element share an FD) 3. Third embodiment (an example in which four main pixels and three sub-pixels share an FD and the FD is light-shielded) 4. Fourth embodiment (an example in which four main pixels and three sub-pixels share an FD and no on-chip lens is arranged on the FD) 5. Fifth embodiment (an example in which four main pixels and four sub-pixels share an FD) 6. Application example to a moving body

[0028] 1 is a block diagram showing an example of the configuration of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device for capturing image data, and includes an optical unit 110, an image sensor 200, and a DSP (Digital Signal Processing) circuit 120. The imaging device 100 further includes a display unit 130, an operation unit 140, a bus 150, a frame memory 160, a storage unit 170, and a power supply unit 180. The imaging device 100 is expected to be a camera mounted on a smartphone, an in-vehicle camera, or the like.

[0029] The optical unit 110 collects light from a subject and guides it to the image sensor 200. The image sensor 200 generates image data by photoelectric conversion. The image sensor 200 supplies the generated image data to the DSP circuit 120 via a signal line 209.

[0030] The DSP circuit 120 performs predetermined signal processing on the image data, and outputs the processed image data to a frame memory 160 or the like via a bus 150.

[0031] The display unit 130 displays image data. For example, a liquid crystal panel or an organic EL (Electro Luminescence) panel is assumed as the display unit 130. The operation unit 140 generates an operation signal in accordance with a user's operation.

[0032] The bus 150 is a common path for the optical unit 110, image sensor 200, DSP circuit 120, display unit 130, operation unit 140, frame memory 160, storage unit 170, and power supply unit 180 to exchange data with one another.

[0033] The frame memory 160 holds image data. The storage unit 170 stores various data such as image data. The power supply unit 180 supplies power to the image sensor 200, the DSP circuit 120, the display unit 130, and the like.

[0034] 2 is a block diagram showing an example of the configuration of an image sensor 200 according to the first embodiment of the present technology. The image sensor 200 includes a vertical drive unit 210, a system control unit 220, a DAC (Digital to Analog Converter) 230, and a pixel array unit 240. The image sensor 200 further includes a column processing unit 250, a horizontal drive unit 260, and a signal processing unit 270.

[0035] In the pixel array section 240, a plurality of pixels are arranged in a two-dimensional grid pattern. The pixel array section 240 is further divided into a plurality of FD sharing blocks 300. In each FD sharing block 300, a plurality of pixels (e.g., four pixels) that share an FD are arranged. Of these four pixels, three include a main pixel and a sub-pixel, and the remaining pixel includes a main pixel and an FD. The circuit configuration and element layout of the FD sharing block 300 will be described later.

[0036] The vertical driving section 210 sequentially drives the rows in the pixel array section 240 to generate pixel voltages.

[0037] The system control section 220 controls the operation timing of each of the vertical drive section 210, the DAC 230, the column processing section 250, and the horizontal drive section 260 in synchronization with a vertical synchronization signal.

[0038] The DAC 230 generates a sawtooth reference signal by DA (Digital to Analog) conversion and supplies it to the column processing unit 250 .

[0039] The column processing unit 250 has an ADC (Analog to Digital Converter) and the like arranged for each column of the FD shared block 300. These ADCs perform analog to digital (AD) conversion on the pixel voltages of the corresponding columns. The column processing unit 250 supplies the converted digital signals to the signal processing unit 270.

[0040] The signal processing unit 270 performs various signal processing such as CDS (Correlated Double Sampling) processing on the digital signals from the column processing unit 250. The signal processing unit 270 outputs an image in which the processed signals are arranged to the DSP circuit 120.

[0041] [Configuration Example of FD Shared Block] Figure 3 is a circuit diagram showing a configuration example of the FD shared block 300 according to the first embodiment of the present technology. The FD shared block 300 includes photodiodes 311-a, 311-b, 311-c, 311-d, 321-a, 321-b, and 321-c. The FD shared block 300 also includes transfer transistors 312-a, 312-b, 312-c, 312-d, 322-a, 322-b, and 322-c. The FD shared block 300 also includes FC gain transistors 324-a, 324-b, and 324-c, an FD 330, a reset transistor 341, an FD gain transistor 342, an amplification transistor 343, and a selection transistor 344. For example, nMOS (n-channel metal oxide semiconductor) transistors are used as the transistors in the FD shared block 300. In addition, in the pixel array section 240, vertical signal lines VSL are wired in the column direction for each column of the FD sharing block 300.

[0042] The wiring capacitance of the wiring between the transfer transistor 322-a and the FC gain transistor 324-a is defined as 323-a. The wiring capacitance of the wiring between the transfer transistor 322-b and the FC gain transistor 324-b is defined as 323-b. The wiring capacitance of the wiring between the transfer transistor 322-c and the FC gain transistor 324-c is defined as 323-c. Note that the wiring capacitances 323-a and 323-b are examples of the first wiring capacitance and the second wiring capacitance described in the claims.

[0043] Photodiodes 311-a, 311-b, 311-c, 311-d, 321-a, 321-b, and 321-c generate electric charges through photoelectric conversion. The photodiodes 311-a, 311-b, 311-c, and 311-d have the same light-receiving area. The photodiodes 321-a, 321-b, and 321-c also have the same light-receiving area, but these light-receiving areas are smaller than the photodiode 311-a, etc.

[0044] The photodiodes 321-a and 321-b are examples of the first photodiode and the second photodiode set forth in the claims.

[0045] The transfer transistor 312-a transfers charges from the photodiode 311-a to the FD 330 in accordance with a drive signal TGLa from the vertical drive unit 210. The transfer transistor 312-b transfers charges from the photodiode 311-b to the FD 330 in accordance with a drive signal TGLb from the vertical drive unit 210. The transfer transistor 312-c transfers charges from the photodiode 311-c to the FD 330 in accordance with a drive signal TGLc from the vertical drive unit 210. The transfer transistor 312-d transfers charges from the photodiode 311-d to the FD 330 in accordance with a drive signal TGLd from the vertical drive unit 210.

[0046] The transfer transistor 322-a transfers charges from the photodiode 321-a to the wiring capacitance 323-a and the FD 330 in accordance with a drive signal TGSa from the vertical drive unit 210. The transfer transistor 322-b transfers charges from the photodiode 321-b to the wiring capacitance 323-b and the FD 330 in accordance with a drive signal TGSb from the vertical drive unit 210. The transfer transistor 322-c transfers charges from the photodiode 321-c to the wiring capacitance 323-c and the FD 330 in accordance with a drive signal TGSc from the vertical drive unit 210.

[0047] The transfer transistors 322-a and 322-b are examples of the first transfer transistor and the second transfer transistor set forth in the claims.

[0048] The FC gain transistors 324-a, 324-b, and 324-c and the FD gain transistor 342 are transistors for controlling the charge-to-voltage conversion efficiency of the FD shared block 300. The FC gain transistors 324-a and 324-b are examples of the first gain transistor and the second gain transistor described in the claims.

[0049] The FC gain transistor 324-a opens and closes the path between the wiring capacitance 323-a and the common node 340 in accordance with a drive signal FCGa from the vertical drive unit 210. The FC gain transistor 324-b opens and closes the path between the wiring capacitance 323-b and the common node 340 in accordance with a drive signal FCGb from the vertical drive unit 210. The FC gain transistor 324-c opens and closes the path between the wiring capacitance 323-c and the common node 340 in accordance with a drive signal FCGc from the vertical drive unit 210.

[0050] The FD gain transistor 342 opens and closes the path between the common node 340 and the FD 330 in accordance with the drive signal FDG from the vertical drive section 210 .

[0051] With the above-described connection configuration, the vertical drive unit 210 can control any one of the FC gain transistors 324-a, 324-b, and 324-c and both the FD gain transistor 342 to the on state. In this case, charge is converted into voltage by the combined capacitance of the wiring capacitance, the FC gain transistor, the FD gain transistor 342, and the FD 330. The charge-to-voltage conversion efficiency at this time is called "LCG (Low Convert Gain)."

[0052] Furthermore, the vertical drive unit 210 can turn off all of the FC gain transistors 324-a, 324-b, and 324-c, and turn on the FD gain transistor 342. In this case, the combined capacitance of the FD gain transistor 342 and the FD 330 converts charge into voltage. The charge-to-voltage conversion efficiency at this time is higher than that of LCG, and is hereinafter referred to as "MCG (Middle Convert Gain)."

[0053] Furthermore, the vertical drive unit 210 can control all of the FC gain transistors 324-a, 324-b, and 324-c and the FD gain transistor 342 to be in an off state. In this case, the FD 330 converts the charge into a voltage. The charge-to-voltage conversion efficiency at this time is higher than that of MCG, and is hereinafter referred to as "HCG (High Convert Gain)."

[0054] The reset transistor 341 extracts charge from the capacitance of the FD 330 and the like to perform initialization in accordance with a drive signal RST from the vertical drive unit 210 .

[0055] The amplifying transistor 343 constitutes a source follower circuit, and outputs a pixel voltage corresponding to the voltage of the source (that is, the voltage of the FD 330) from the drain.

[0056] The selection transistor 344 outputs the pixel voltage signal from the amplification transistor 343 as a pixel signal SIG to the vertical signal line VSL in accordance with a drive signal SEL from the vertical drive unit 210 .

[0057] Although the wiring capacitances 323-a, 323-b, and 323-c are provided in the FD shared block 300, these may be reduced.

[0058] Furthermore, although the FC gain transistors 324-a, 324-b, and 324-c are provided in the FD shared block 300, if the charge-voltage conversion efficiency is controlled in two stages, these transistors can be eliminated.

[0059] Furthermore, although the FC gain transistors 324-a, 324-b, and 324-c and the FD gain transistor 342 are provided in the FD shared block 300, these transistors can be eliminated if the charge-voltage conversion efficiency is to be kept constant.

[0060] 4 is a plan view showing an example of a pixel array according to the first embodiment of the present technology. When viewed from the optical axis direction, main pixels 310-a, 310-b, 310-c, and 310-d are arranged in a two-dimensional lattice pattern.

[0061] Furthermore, sub-pixels such as sub-pixels 320-a, 320-b, and 320c have lower sensitivity than the main pixels and are arranged in an area surrounded by the main pixels. FD 330 is shared by main pixels 310-a, 310-b, 310-c, and 310-d and sub-pixels 320-a, 320-b, and 320c, and is arranged in an area surrounded by those main pixels. Note that sub-pixels 320-a and 320-b are examples of the first and second sub-pixels described in the claims.

[0062] The axis perpendicular to the plane on which the above-mentioned main pixels, sub-pixels, and other pixels are arranged (in other words, the optical axis) is defined as the "Z axis." A specific axis parallel to this plane is defined as the "X axis," and an axis perpendicular to the X axis and Z axis is defined as the "Y axis."

[0063] The main pixels 310-a, 310-b, 310-c, and 310-d, the sub-pixels 320-a, 320-b, and 320c, and the FD 330 illustrated in the figure constitute the FD shared block 300 having the circuit configuration illustrated in Fig. 3. For example, the photodiode 311-a, the transfer transistor 312-a, and the subsequent circuit (such as the amplification transistor 343) function as the main pixel 310-a. The photodiode 321-a, the transfer transistor 322-a, the wiring capacitance 323-a, the FC gain transistor 324-a, and the subsequent circuit function as the sub-pixel 320-a.

[0064] Because four main pixels and three sub-pixels share the FD 330, the number of elements per pixel can be reduced compared to the prior art where two pixels share an FD or no FD is shared. This makes it easier to secure layout space for elements and to increase the size of transistors and capacitors. It also improves the degree of freedom in element placement.

[0065] 4, the main pixel has an octagonal shape, for example, and the sub-pixel and FD 330 have a rectangular shape, for example. The main pixel receives visible light of any of R (Red), G (Green), and B (Blue). The letters "R," "G," and "B" in the figure indicate the color of light received. The main pixels are arranged, for example, in a Bayer array.

[0066] A color filter (not shown) is also disposed above the FD, and any one of R, G, and B visible light is incident thereon. The sub-pixels also receive any one of R, G, and B visible light. The color received by each of these sub-pixels and the FD 330 is the same as the color received by the nearby main pixel (for example, the lower left). Therefore, when the main pixels are arranged in a Bayer array, the sub-pixels and the FD 330 are also arranged in a Bayer array.

[0067] The arrangement of the main pixels is not limited to the Bayer arrangement.

[0068] For example, as shown in Fig. 5, the main pixels can be arranged in a quad Bayer array. The pixels (main pixels and sub-pixels) that receive R, G, and B light are designated as R, B, and G pixels, respectively. In the quad Bayer array, R pixels are arranged in 2 rows by 2 columns, G pixels are arranged in 2 rows by 2 columns, and B pixels are arranged in 2 rows by 2 columns. Focusing on a 4-row by 4-column area, 2 rows by 2 columns of R pixels and 2 rows by 2 columns of B pixels are arranged diagonally, and G pixels are arranged in the remaining area.

[0069] Furthermore, an array other than the Bayer array or the Quad Bayer array can also be used. For example, R, G, B, and W (white) pixels can be arranged.

[0070] In addition, although the number of pixels sharing the FD 330 is four in FIGS. 2 to 5, the number of pixels is not limited to this.

[0071] 6 is a plan view showing an example of an arrangement of on-chip lenses according to the first embodiment of the present technology. In the drawing, dotted lines indicate the peripheries of the main pixels, sub-pixels, or FDs 330, and solid lines indicate the peripheries of the on-chip lenses. The direction toward the optical unit 110 (not shown) is defined as the "up" direction, and on-chip lenses are provided above each of the main pixels, sub-pixels, and FDs 330.

[0072] In the block sharing the FD 330, on-chip lenses 351-a, 351-b, 351-c, 351-d, 352-a, 352-b, 352-c, and 352-d are arranged.

[0073] The on-chip lens 351-a is disposed in an area covering the main pixel 310-a, the on-chip lens 351-b is disposed in an area covering the main pixel 310-b, the on-chip lens 351-c is disposed in an area covering the main pixel 310-c, and the on-chip lens 351-d is disposed in an area covering the main pixel 310-d.

[0074] Furthermore, on-chip lens 352-a is disposed in an area covering sub-pixel 320-a, and on-chip lens 352-b is disposed in an area covering sub-pixel 320-b. On-chip lens 352-c is disposed in an area covering sub-pixel 320-c, and on-chip lens 352-d is disposed in an area covering FD 330. These on-chip lenses have smaller areas than the on-chip lenses that cover the main pixels.

[0075] As shown in the figure, each of the on-chip lenses 351-a, 351-b, 351-c, and 351-d covers one of the main pixels, and the on-chip lenses 352-a, 352-b, 352-c, and 352-d are disposed in an area surrounded by the on-chip lenses 351-a, etc.

[0076] The on-chip lenses 351-a, 351-b, 351-c, and 351-d are examples of the first on-chip lens described in the claims. The on-chip lenses 352-a, 352-b, 352-c, and 352-d are examples of the second on-chip lens described in the claims.

[0077] 7 is an example of a cross-sectional view of the pixel array unit 240 according to the first embodiment of the present technology. In the drawing, a and b are cross-sectional views as viewed from the Y-axis direction. In the drawing, a is an example of a cross-sectional view of a row in which R pixels and G pixels are arranged, and b is an example of a cross-sectional view of a row in which G pixels and B pixels are arranged.

[0078] As shown in the diagram (a), photodiodes 311-a and 311-b are arranged on top of wiring layer 370. Furthermore, a color filter layer 360 is formed on top of these photodiodes. Color filter layer 360 includes color filter 361-a, which transmits R visible light and guides it to photodiode 311-a, and color filter 361-b, which transmits G visible light and guides it to photodiode 311-b.

[0079] As shown in FIG. 1B, photodiodes 311-c and 311-d are arranged in rows where G and B pixels are arranged. Color filter layer 360 includes color filter 361-c that transmits B visible light and directs it to photodiode 311-c, and color filter 361-d that transmits G visible light and directs it to photodiode 311-d.

[0080] As shown by the examples a and b in the figure, a color filter 361-a and the like are provided above each photodiode of the main pixel. Note that the color filter 361-a also guides R visible light to the photodiode 321-a (not shown) of the nearby subpixel, and the color filter 361-b also guides G visible light to the photodiode 321-b (not shown) of the nearby subpixel. The color filter 361-c also guides B visible light to the photodiode 321-c (not shown) of the nearby subpixel, and the color filter 361-d also guides G visible light to the FD 330 (not shown).

[0081] 8 is a plan view showing an example of the layout of elements in the FD shared block 300 according to the first embodiment of the present technology. The dotted lines in the figure indicate the periphery of the photodiode. The elements illustrated in the figure are arranged in the wiring layer 370 below the photodiode, as illustrated in FIGS. 7A and 7B.

[0082] As shown in FIG. 8, in the row in which the photodiodes 321-a and 321-b are arranged, the transfer transistors 322-a and 322-b and the FC gain transistors 324-a and 324-b are arranged along the row direction.

[0083] In the row in which the FD 330 and photodiode 321-c are arranged, the reset transistor 341 and FD gain transistor 342, the transfer transistor 322-c and the FC gain transistor 324-c are arranged along the row direction.

[0084] Transfer transistors 312-a, 312-b, 312-c, and 312-d are disposed between the FD 330 and the photodiodes 311-a, 311-b, 311-c, and 311-d.

[0085] The amplification transistor 343 and the selection transistor 344 are disposed, for example, below the photodiodes 311-c and 311-d.

[0086] [Example of Operation of Image Sensor] Next, a method for converting pixel voltages will be described with reference to Fig. 9 and Fig. 10. Fig. 9 and Fig. 10 focus on the main pixel 310-a and the sub-pixel 320-a, and illustrate timing charts for converting the pixel voltages thereof.

[0087] 9 is a timing chart showing an example of the operation of the image sensor 200 when converting pixel voltages up to VP4 according to the first embodiment of the present technology. The thick dotted line in the figure indicates the voltage of the pixel signal SIG transmitted via the vertical signal line VSL.

[0088] When the cycle of the horizontal synchronization signal HSYNC starts at timing t1, the vertical drive section 210 selects a row of the FD sharing block 300 and sets the drive signal SEL to that row to high level at timing t2.

[0089] The vertical drive unit 210 then sets the drive signals FDG and RST to high level during the period from timing t11 to t12, thereby initializing the FD 330. The vertical drive unit 210 then sets the drive signal FDG to high level during the period from timing t13 to t21. During this period, the DAC 230 gradually reduces the voltage of the reference signal REF during a conversion period P1 from timing t14 to t16. At timing t15, the column processing unit 250 converts the pixel voltage VP1 generated by the MCG into a counter value CNT1.

[0090] After initializing the reference signal REF, the DAC 230 gradually reduces the voltage of the reference signal REF over a conversion period P2 from timing t22 to t24. At timing t23, the column processing unit 250 converts the pixel voltage VP2 generated by the HCG into a counter value CNT2.

[0091] Then, during the period from timing t31 to t32, the vertical drive unit 210 sets the drive signal TGLa to high level, thereby transferring charge from the photodiode 311-a to the FD 330.

[0092] After initializing the reference signal REF, the DAC 230 gradually decreases the voltage of the reference signal REF over a conversion period P3 from timing t33 ​​to t35. At timing t34, the column processing unit 250 converts the pixel voltage VP3 generated by the HCG into a counter value CNT3.

[0093] Then, at timing t41, the vertical drive section 210 sets the drive signal FDG to high level, thereby causing the charge-to-voltage conversion efficiency to become MCG.

[0094] After initializing the reference signal REF, the DAC 230 gradually reduces the voltage of the reference signal REF over a conversion period P4 from timing t42 to t44. At timing t43, the column processing unit 250 converts the pixel voltage VP4 generated by the MCG into a counter value CNT4.

[0095] The signal processing unit 270 at the subsequent stage subtracts the counter value CNT2 from the counter value CNT3, and sets the resulting value as the pixel value VAL1. Since the pixel voltage VP3 has a level corresponding to the exposure amount, and the pixel voltage VP2 is a level at the time of initialization, this processing corresponds to CDS processing.

[0096] Furthermore, the signal processing unit 270 subtracts the counter value CNT1 from the counter value CNT4, and sets the resulting value as the pixel value VAL2. Since the pixel voltage VP4 has a level corresponding to the exposure amount, and the pixel voltage VP1 is a level at the time of initialization, this process also corresponds to the CDS process.

[0097] 10 is a timing chart showing an example of the operation of the image sensor 200 when converting pixel voltages VP5 and onward according to the first embodiment of the present technology. After converting pixel voltage VP4, the vertical drive unit 210 sets the drive signal RST to a high level during a period from timing t51 to t52. This initializes the FD 330.

[0098] Then, the vertical drive unit 210 sets the drive signal FCGa to high level at timing t53, which causes the charge-to-voltage conversion efficiency to become LCG.

[0099] After initializing the reference signal REF, the DAC 230 gradually decreases the voltage of the reference signal REF over a conversion period P5 from timing t54 to t56. At timing t55, the column processing unit 250 converts the pixel voltage VP5 generated by the LCG into a counter value CNT5.

[0100] Then, during the period from timing t61 to t62, the vertical drive unit 210 sets the drive signal TGSa to high level, which causes the charge to be transferred from the photodiode 321-a to the combined capacitance.

[0101] After initializing the reference signal REF, the DAC 230 gradually reduces the voltage of the reference signal REF over a conversion period P6 from timing t63 to t65. At timing t64, the column processing unit 250 converts the pixel voltage VP6 generated by the LCG into a counter value CNT6.

[0102] The vertical drive unit 210 then sets the drive signal RST to high level during the period from timing t71 to t73, thereby initializing the FD 330. The vertical drive unit 210 also sets the drive signal FCGa to low level at timing t72, and to high level at timing t74.

[0103] After initializing the reference signal REF, the DAC 230 gradually decreases the voltage of the reference signal REF over a conversion period P7 from timing t75 to t77. At timing t76, the column processing unit 250 converts the pixel voltage VP7 generated by the LCG into a counter value CNT7.

[0104] Then, the vertical drive section 210 sets the drive signals FDG and FCGa to low level at timing t7, and sets the drive signal SEL to low level at timing t8.

[0105] The signal processing unit 270 at the subsequent stage subtracts the counter value CNT5 from the counter value CNT6, and sets the resulting value as the pixel value VAL3. Since the pixel voltage VP6 has a level corresponding to the exposure amount, and the pixel voltage VP5 is a level at the time of initialization, this processing corresponds to CDS processing.

[0106] Furthermore, the signal processing unit 270 subtracts the counter value CNT7 from the counter value CNT5, and sets the resulting value as the pixel value VAL4. Since the pixel voltages VP5 and VP7 are at the levels at the time of initialization, this process corresponds to DDS (Double Data Sampling).

[0107] The signal processing unit 270 combines the pixel values ​​VAL1, VAL2, VAL3, and VAL4 for each pixel to generate image data, which expands the dynamic range.

[0108] 9 and 10 show the process of converting the pixel voltages of the main pixel 310-a and sub-pixel 320-a. Next, the pixel voltages of the main pixel 310-b and sub-pixel 320-b are converted using a similar process. Next, the pixel voltages of the main pixel 310-c and sub-pixel 320-c are converted, and then the pixel voltage of the main pixel 310-d is converted. Here, there is no sub-pixel corresponding to the main pixel 310-d. Therefore, the signal processing unit 270 needs to interpolate the pixel value of that sub-pixel by, for example, averaging the pixel values ​​of surrounding sub-pixels of the same color.

[0109] Although the image sensor 200 converts the pixel voltages of the four pixels in the FD sharing block 300 in order without adding them, these can also be added together. Adding pixels can speed up readout and reduce the data rate. Adding pixels also increases the signal amount, improving the SNR (Signal-Noise Ratio).

[0110] When pixel addition is performed, for example, a quad Bayer arrangement is used, and while the drive signal TGLa is at a high level in Fig. 9, the drive signals TGLb, TGLc, and TGLd are also simultaneously controlled to a high level. Also, while the drive signal TGSa is at a high level in Fig. 10, the drive signals TGSb and TGSc are also simultaneously controlled to a high level. The same applies to the drive signals FCGa, FCGb, and FCGc.

[0111] However, when adding pixels, the number of main pixels to be added is four, while the number of sub-pixels to be added is three, so the signal processing unit 270 needs to correct the pixel values ​​and the ratio at the time of synthesis according to the ratio of these pixel numbers.

[0112] As described above, according to the first embodiment of the present technology, sub-pixels and FDs are arranged in an area surrounded by a main pixel, and the FD is shared by these pixels, thereby reducing the number of elements per pixel compared to when the FDs are not shared.

[0113] 2. Second Embodiment In the first embodiment described above, the transfer transistors 322-a etc. transfer charges to the wiring capacitances 323-a etc., but this configuration is not limited to this. The image sensor 200 in this second embodiment differs from the first embodiment in that a capacitive element is provided instead of the wiring capacitance.

[0114] 11 is a circuit diagram showing a configuration example of the FD shared block 300 according to the second embodiment of the present technology. The FD shared block 300 according to the second embodiment differs from the first embodiment in that capacitive elements 325-a, 325-b, and 325-c are arranged instead of the wiring capacitances 323-a, 323-b, and 323-c.

[0115] 12 is a plan view showing an example of the layout of elements in the FD shared block 300 according to the second embodiment of the present technology. In the second embodiment, for example, capacitive elements 325-a, 325-b, and 325-c are arranged directly below the photodiodes 311-a, 311-b, and 311-c, respectively. Also, an amplification transistor 343 and a selection transistor 344 are arranged directly below the photodiode 311-d. Note that the capacitive elements 325-a and 325-b are examples of the first capacitive element and the second capacitive element set forth in the claims.

[0116] As described above, according to the second embodiment of the present technology, since the capacitive elements 325-a, 325-b, and 325-c are arranged, the transfer transistors 322-a and the like can transfer charges thereto.

[0117] 3. Third Embodiment In the first embodiment described above, visible light is incident on the FD 330 in addition to the main pixels and sub-pixels. However, in this configuration, the visible light may cause charges to be generated in the FD 330. This excess charge may degrade the image quality of the image data. The image sensor 200 in this third embodiment differs from the first embodiment in that the FD 330 is light-shielded.

[0118] 13 is a plan view showing an example of a pixel arrangement according to the third embodiment of the present technology. The pixel array unit 240 according to the third embodiment differs from the first embodiment in that a light-shielding unit 380 that shields the FD 330 from light is further provided.

[0119] For example, a black color filter can be used as the light-shielding portion 380. Alternatively, a metal plate can be used as the light-shielding portion 380. Examples of materials for this metal plate include tungsten. Alternatively, a plurality of overlapping color filters can be used as the light-shielding portion 380. For example, an R color filter and a G color filter can be overlapped.

[0120] As illustrated in the figure, the light-shielding portion 380 shields the FD 330 from light, thereby preventing the generation of excess charge due to incident light on the FD 330 and improving the image quality of the image data.

[0121] When the FD 330 is not light-shielded as in the first embodiment, the color filter arrangement patterns can be aligned between rows and columns including the FD 300 and rows and columns not including the FD 330, resulting in reduced pattern misalignment. Furthermore, the process of forming the light-shielding portion 380 is not required. This facilitates the manufacture of the image sensor 200.

[0122] Moreover, the second embodiment can be applied to the third embodiment.

[0123] As described above, according to the third embodiment of the present technology, the light-shielding portion 380 shields the FD 330 from light, thereby preventing the generation of excess charge and improving the image quality of the image data.

[0124] 4. Fourth Embodiment In the first embodiment described above, the FD 330 was covered with the on-chip lens 352-d. However, in this configuration, light collected by the on-chip lens 352-d is incident on the FD 330, and this light may cause electric charge to be generated within the FD 330. This excess electric charge may degrade the image quality of the image data. The image sensor 200 in this fourth embodiment differs from the first embodiment in that the on-chip lens 352-d is not disposed above the FD 330.

[0125] 14 is a plan view showing an example of an arrangement of on-chip lenses according to a fourth embodiment of the present technology. In this fourth embodiment, the on-chip lenses 352-d above the FD 330 are eliminated. By not disposing on-chip lenses in the region covering the FD 330, it is possible to suppress the generation of excess charge due to incident light on the FD 330, and improve the image quality of image data.

[0126] When the FD 330 is also covered with the on-chip lens 352-d as in the first embodiment, the arrangement pattern of the on-chip lenses can be aligned between rows and columns that include the FD 300 and rows and columns that do not include the FD 330. This reduces pattern misalignment, making it easier to manufacture the image sensor 200.

[0127] Moreover, the second and third embodiments can be applied to the fourth embodiment.

[0128] As described above, according to the fourth embodiment of the present technology, an on-chip lens is not disposed in the area covering the FD 330, and therefore, it is possible to suppress the generation of excess charge due to incident light on the FD 330, thereby improving the image quality of the image data.

[0129] 5. Fifth Embodiment In the first embodiment described above, four main pixels and three sub-pixels share the FD 330, but with this arrangement, one sub-pixel is missing within the FD sharing block 300. The image sensor 200 in this fifth embodiment differs from the first embodiment in that four main pixels and four sub-pixels share the FD 330.

[0130] 15 is a plan view showing an example of a pixel array according to the fifth embodiment of the present technology. The pixel array unit 240 according to the fifth embodiment differs from the first embodiment in that a sub-pixel 320-d is further arranged in the FD shared block 300. The pixels including this sub-pixel 320-d are arranged in a Bayer array.

[0131] The FD 330 is disposed in the center of the FD sharing block 300. If the top of the page is the north direction, the main pixel 310-a is disposed northwest of the FD 330, and the sub-pixel 320-a is disposed northwest of the main pixel 310-a. The main pixel 310-b is disposed northeast of the FD 330, and the sub-pixel 320-b is disposed northeast of the main pixel 310-b. The main pixel 310-c is disposed southeast of the FD 330, and the sub-pixel 320-c is disposed southeast of the main pixel 310-c. The main pixel 310-d is disposed southwest of the FD 330, and the sub-pixel 320-d is disposed southwest of the main pixel 310-d. The FD 330 is shared by the four main pixels and four sub-pixels described above.

[0132] The main pixels 310-a, 310-b, 310-c, and 310-d are examples of the first, second, third, and fourth main pixels described in the claims. The sub-pixels 320-a, 320-b, 320-c, and 320-d are examples of the first, second, third, and fourth sub-pixels described in the claims.

[0133] As shown in the figure, there are no missing sub-pixels because four main pixels and four sub-pixels are arranged in the FD shared block 300. However, the position of the sub-pixels is shifted from the main pixels, which may result in a deterioration in image quality.

[0134] Therefore, as shown in the example of Figure 16, the signal processing unit 270 corrects the pixel signal of sub-pixel 320-a using the pixel signals of the three surrounding sub-pixels of the same color. The four sub-pixels enclosed in bold frames in the figure are used for correction. The signal processing unit 270 calculates the average value of these four sub-pixels and outputs it as the pixel signal for the position indicated by the dotted line, which includes main pixel 310-a and sub-pixel 320-b. This makes it possible to suppress degradation in image quality.

[0135] 17 is a circuit diagram showing a configuration example of the FD shared block 300 according to the fifth embodiment of the present technology. In the circuit of the same figure, a photodiode 321-d, a transfer transistor 322-d, a wiring capacitance 323-d, and an FC gain transistor 324-d are added compared to the circuit of FIG.

[0136] 18 is a plan view showing an example of the layout of elements in the FD sharing block 300 according to the fifth embodiment of the present technology. The dotted lines in the drawing indicate the periphery of the photodiode.

[0137] As illustrated in the figure, in the row in which photodiodes 321-a and 321-b are arranged, transfer transistors 322-a and 322-b and FC gain transistors 324-a and 324-b are arranged along the row direction. In the row in which photodiodes 321-c and 321-d are arranged, transfer transistors 322-c and 322-d and FC gain transistors 324-c and 324-d are arranged along the row direction.

[0138] Transfer transistors 312-a, 312-b, 312-c, and 312-d are disposed between the FD 330 and the photodiodes 311-a, 311-b, 311-c, and 311-d.

[0139] In the row in which the FDs 330 are arranged, a reset transistor 341, an FD gain transistor 342, an amplification transistor 343, and a selection transistor 344 are arranged along the row direction.

[0140] A wiring capacitance 323-a is arranged between the transfer transistor 322-a and the FD gain transistor 342, and a wiring capacitance 323-b is arranged between the transfer transistor 322-b and the amplifier transistor 343. In addition, a wiring capacitance 323-d is arranged between the transfer transistor 322-d and the FD gain transistor 342, and a wiring capacitance 323-c is arranged between the transfer transistor 322-c and the amplifier transistor 343.

[0141] As shown in FIG. 19, MIM (Metal-Insulator-Metal) capacitors 326-a, 326-b, 326-c, and 326-d can be used instead of the wiring capacitors.

[0142] In addition, although the pixels are arranged in a Bayer array in FIG. 15, the arrangement is not limited to this.

[0143] For example, as shown in FIG. 20, the pixels can be arranged in a quad Bayer arrangement.

[0144] Furthermore, as shown in Fig. 21, in a Quad Bayer arrangement, each sub-pixel can be arranged so that it is surrounded by four main pixels of the same color, thereby making it possible to suppress colored flare.

[0145] In this way, according to the fifth embodiment of the present technology, four main pixels and four sub-pixels that share the FD 330 are arranged within the FD sharing block 300, thereby eliminating missing sub-pixels within the block.

[0146] 4. Application Examples to Mobile Bodies The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0147] FIG. 22 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0148] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 22, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0149] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0150] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0151] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0152] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0153] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0154] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0155] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0156] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0157] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 22, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0158] FIG. 23 is a diagram showing an example of the installation position of the imaging unit 12031.

[0159] In FIG. 23, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0160] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0161] 23 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0162] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0163] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0164] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0165] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0166] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the image capturing unit 12031 of the above-described configuration. Specifically, the image capturing device 100 of FIG. 1 can be applied to the image capturing unit 12031. By applying the technology according to the present disclosure to the image capturing unit 12031, the number of elements per pixel can be reduced, facilitating miniaturization.

[0167] Note that the above-described embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specifying the invention in the claims correspond to each other. Similarly, the matters specifying the invention in the claims and the matters in the embodiment of the present technology having the same name correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist thereof.

[0168] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0169] The present technology may also be configured as follows: (1) An image sensor comprising: a plurality of main pixels arranged in a two-dimensional lattice pattern; a plurality of sub-pixels having lower sensitivity than the plurality of main pixels and each arranged in an area surrounded by the plurality of main pixels; and a floating diffusion arranged in the area surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels. (2) The image sensor according to (1), further comprising: a plurality of first on-chip lenses each covering one of the plurality of main pixels; and a plurality of second on-chip lenses arranged in an area surrounded by the plurality of first on-chip lenses, wherein the plurality of main pixels include first, second, third, and fourth main pixels, the plurality of sub-pixels include first, second, and third sub-pixels, and the floating diffusion is shared by the first, second, third, and fourth main pixels and the first, second, and third sub-pixels. (3) The image sensor according to (1) or (2), further comprising an FD gain transistor that opens and closes a path between a predetermined common node and the floating diffusion, the plurality of subpixels including first and second subpixels, the first subpixel comprising a first photodiode, and the second subpixel comprising a second photodiode. (4) The image sensor according to (3), further comprising: a first wiring capacitance, a first transfer transistor that transfers charge from the first photodiode to the first wiring capacitance, and a first gain transistor that opens and closes the path between the first wiring capacitance and the common node, and the second subpixel further comprising: a second wiring capacitance, a second transfer transistor that transfers charge from the second photodiode to the second wiring capacitance, and a second gain transistor that opens and closes the path between the second wiring capacitance and the common node.(5) The image sensor according to (3), wherein the first subpixel further comprises: a first capacitance element; a first transfer transistor that transfers charge from the first photodiode to the first capacitance element; and a first gain transistor that opens and closes a path between the first wiring capacitance and the common node; and the second subpixel further comprises: a second capacitance element; a second transfer transistor that transfers charge from the second photodiode to the second capacitance element; and a second gain transistor that opens and closes a path between the second wiring capacitance and the common node. (6) The image sensor according to any one of (1) to (5), wherein visible light is incident on the floating diffusion. (7) The image sensor according to any one of (1) to (5), further comprising a light-shielding portion that shields the floating diffusion from light. (8) The image sensor according to (7), wherein the light-shielding portion is a black color filter. (9) The image sensor according to (7), wherein the light-shielding portion is a plurality of overlapping color filters. (10) The image sensor according to (7), wherein the light-shielding portion is a metal plate. (11) The image sensor according to (10), wherein the metal plate contains tungsten. (12) The image sensor according to any one of (1) to (11), wherein the second on-chip lenses are arranged in regions covering the sub-pixels, respectively, and are not arranged in a region covering the floating diffusion. (13) The image sensor according to any one of (1) to (11), wherein the second on-chip lenses are arranged in regions covering the sub-pixels and the floating diffusion, respectively. (14) The image sensor according to any one of (1) to (13), wherein the main pixels are arranged in a Bayer array. (15) The image sensor according to any one of (1) to (13), wherein the main pixels are arranged in a quad-Bayer array.(16) The image sensor according to (1), wherein the plurality of main pixels include first, second, third, and fourth main pixels, the plurality of sub-pixels include first, second, third, and fourth sub-pixels, and the floating diffusion is shared by the first, second, third, and fourth main pixels and the first, second, third, and fourth sub-pixels. (17) The image sensor according to (16), further comprising a signal processing unit that corrects pixel signals of each of the plurality of sub-pixels using pixel signals of surrounding sub-pixels. (18) The image sensor according to (16) or (17), wherein the plurality of main pixels are arranged in a Bayer array or a Quad Bayer array. (19) The image sensor according to any of (16) to (18), wherein each of the plurality of sub-pixels includes a capacitive element. (20) An imaging device comprising: a plurality of main pixels arranged in a two-dimensional lattice; a plurality of sub-pixels each having a lower sensitivity than the plurality of main pixels and arranged in an area surrounded by the plurality of main pixels; a floating diffusion arranged in the area surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels; and a column processing unit that converts the pixel voltages of the plurality of main pixels and the plurality of sub-pixels into digital signals.

[0170] 100 Imaging device 110 Optical unit 120 DSP circuit 130 Display unit 140 Operation unit 150 Bus 160 Frame memory 170 Storage unit 180 Power supply unit 200 Image sensor 210 Vertical drive unit 220 System control unit 230 DAC 240 Pixel array unit 250 Column processing unit 260 Horizontal drive unit 270 Signal processing unit 300 FD shared block 310-a, 310-b, 310-c, 310-d, 310-e, 310-f, 310-g, 310-h Main pixels 311-a, 311-b, 311-c, 311-d, 321-a, 321-b, 321-c, 321-d Photodiodes 312-a, 312-b, 312-c, 312-d, 322-a, 322-b, 322-c, 322-d Transfer transistors 320-a, 320-b, 320-c, 320-d, 320-e, 320-f, 320-g Subpixels 323-a, 323-b, 323-c, 323-d Wiring capacitances 324-a, 324-b, 324-c, 324-d FC gain transistors 325-a, 325-b, 325-c Capacitor elements 326-a, 326-b, 326-c, 326-d MIM capacitances 330 FD 341 Reset transistor 342 FD gain transistor 343 Amplification transistor 344 Selection transistor 351-a, 351-b, 351-c, 351-d, 352-a, 352-b, 352-c, 352-d On-chip lens 360 Color filter layer 361-a, 361-b, 361-c, 361-d Color filter 370 Wiring layer 380 Light-shielding portion 12031 Imaging portion

Claims

1. An image sensor comprising a plurality of main pixels arranged in a two-dimensional grid pattern, a plurality of sub-pixels each having a lower sensitivity than the plurality of main pixels and disposed in regions surrounded by the plurality of main pixels, and a floating diffusion disposed in a region surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels.

2. The image sensor according to claim 1, further comprising a plurality of first on-chip lenses each covering any one of the plurality of main pixels, and a plurality of second on-chip lenses disposed in a region surrounded by the plurality of first on-chip lenses, wherein the plurality of main pixels include first, second, third, and fourth main pixels, the plurality of sub-pixels include first, second, and third sub-pixels, and the floating diffusion is shared by the first, second, third, and fourth main pixels and the first, second, and third sub-pixels.

3. The image sensor according to claim 1, further comprising an FD (Floating Diffusion) gain transistor that opens and closes a path between a predetermined common node and the floating diffusion, wherein the plurality of sub-pixels include first and second sub-pixels, the first sub-pixel includes a first photodiode, and the second sub-pixel includes a second photodiode.

4. The image sensor according to claim 3, wherein the first sub-pixel further comprises a first wiring capacitance, a first transfer transistor that transfers charges from the first photodiode to the first wiring capacitance, and a first gain transistor that opens and closes a path between the first wiring capacitance and the common node, and the second sub-pixel further comprises a second wiring capacitance, a second transfer transistor that transfers charges from the second photodiode to the second wiring capacitance, and a second gain transistor that opens and closes a path between the second wiring capacitance and the common node.

5. The first sub-pixel further includes a first capacitive element, a first transfer transistor that transfers charges from the first photodiode to the first capacitive element, and a first gain transistor that opens and closes a path between the first wiring capacitance and the common node. The second sub-pixel further includes a second capacitive element, a second transfer transistor that transfers charges from the second photodiode to the second capacitive element, and a second gain transistor that opens and closes a path between the second wiring capacitance and the common node. The image sensor according to claim 3.

6. The image sensor according to claim 1, wherein visible light is incident on the floating diffusion.

7. The image sensor according to claim 1, further comprising a light-shielding portion that shields the floating diffusion.

8. The image sensor according to claim 7, wherein the light-shielding portion is a black color filter.

9. The image sensor according to claim 7, wherein the light-shielding portion is a plurality of overlapping color filters.

10. The image sensor according to claim 7, wherein the light-shielding portion is a metal plate.

11. The image sensor according to claim 10, wherein the metal plate contains tungsten.

12. The image sensor according to claim 1, wherein the plurality of second on-chip lenses are arranged in regions covering respective ones of the plurality of sub-pixels and are not arranged in a region covering the floating diffusion.

13. The image sensor according to claim 1, wherein the plurality of second on-chip lenses are arranged in regions covering respective ones of the plurality of sub-pixels and the floating diffusion.

14. The image sensor according to claim 1, wherein the plurality of main pixels are arranged in a Bayer array.

15. The image sensor according to claim 1, wherein the plurality of main pixels are arranged in a quad Bayer array.

16. The image sensor according to claim 1, wherein the plurality of main pixels include first, second, third, and fourth main pixels, the plurality of sub-pixels include first, second, third, and fourth sub-pixels, and the floating diffusion is shared by the first, second, third, and fourth main pixels and the first, second, third, and fourth sub-pixels.

17. The image sensor according to claim 16, further comprising a signal processing unit that corrects the pixel signal of each of the plurality of sub-pixels using the pixel signals of surrounding sub-pixels.

18. The image sensor according to claim 16, wherein the plurality of main pixels are arranged in a Bayer array or a quad Bayer array.

19. The image sensor according to claim 16, wherein each of the plurality of sub-pixels includes a capacitive element.

20. An imaging device comprising: a plurality of main pixels arranged in a two-dimensional grid; a plurality of sub-pixels each having a lower sensitivity than the plurality of main pixels and disposed in an area surrounded by the plurality of main pixels; a floating diffusion disposed in an area surrounded by the plurality of main pixels and shared by the plurality of main pixels and the plurality of sub-pixels; and a column processing unit that converts the pixel voltages of the plurality of main pixels and the plurality of sub-pixels into digital signals.

Images