Light detection device and electronic instrument

Abstract

A light detection device according to an embodiment of the present disclosure comprises: a semiconductor substrate having a first surface and a second surface facing each other, and having a pixel array unit in which a plurality of pixels including first pixels and second pixels are arranged in an array; and a plurality of first microlenses arranged in an array on the pixel array unit on the first surface side of the semiconductor substrate, the first pixels and the second pixels having different numbers of first microlenses.

Description

Photodetector and electronic equipment

[0001] This disclosure relates to a photodetector and electronic equipment.

[0002] For example, Patent Document 1 discloses a solid-state imaging device that corrects the difference in light reception sensitivity between phase difference detection pixel pairs by arranging an incident light scatterer on the optical path connecting the pixel boundary portion that separates adjacent phase difference detection pixels and the center position of the pupil division microlens.

[0003] Japanese Patent Publication No. 2013-211413

[0004] By the way, in light detection devices, there is a need to achieve both high dynamic range and improved image quality.

[0005] It is desirable to provide an optical detection device and electronic equipment that can achieve both high dynamic range and improved image quality.

[0006] An optical detection device as one embodiment of the present disclosure comprises a semiconductor substrate having a pixel array portion having opposing first and second surfaces and a plurality of pixels including first and second pixels arranged in an array, and a plurality of first microlenses arranged in an array in the pixel array portion on the first surface side of the semiconductor substrate, wherein the first pixels and second pixels have a different number of first microlenses.

[0007] An electronic device as one embodiment of the present disclosure is equipped with the optical detection device of the above embodiment of the present disclosure.

[0008] In one embodiment of the present disclosure, a light detection device and an electronic device, the pixel array portion is configured such that the first microlenses arranged in the array consist of a mixture of first pixels and second pixels, each having different first microlenses. This allows for greater dispersion of the incident light collection position in pixels with a larger number of first microlenses, thereby reducing sensitivity while suppressing periodic changes in sensitivity due to the angle of incidence.

[0009] Figure 1 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to an embodiment of the present disclosure. Figure 2A is a schematic plan view showing an example of the layout of inner lenses arranged in the high-sensitivity pixels shown in Figure 1. Figure 2B is a schematic plan view showing an example of the layout of inner lenses arranged in the low-sensitivity pixels shown in Figure 1. Figure 3 is a block diagram showing the overall configuration of the photodetector shown in Figure 1. Figure 4 is an equivalent circuit diagram of a unit pixel shown in Figure 1. Figure 5 is a schematic cross-sectional view showing another example of the configuration of a photodetector according to an embodiment of the present disclosure. Figure 6 is a schematic cross-sectional view showing another example of the configuration of a photodetector according to an embodiment of the present disclosure. Figure 7 is a simulation diagram showing the relationship between the angle of incidence of light and the pixel output as a comparative example. Figure 8 is a simulation diagram showing the relationship between the angle of incidence of light and the pixel output as an example. Figure 9 is a schematic plan view showing an example of the layout of a plurality of microlenses arranged in a low-sensitivity pixel according to Modification 1 of the present disclosure. Figure 10 is a schematic plan view showing an example of the layout of a plurality of microlenses arranged in a low-sensitivity pixel according to Modification 1 of the present disclosure. Figure 11 is a schematic plan view showing an example of the layout of a plurality of microlenses arranged in a low-sensitivity pixel according to Modification 1 of the present disclosure. Figure 12 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 2 of the present disclosure. Figure 13 is a schematic plan view showing an example of the layout of the light-shielding layer shown in Figure 12. Figure 14 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 3 of the present disclosure. Figure 15 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 4 of the present disclosure. Figure 16 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 5 of the present disclosure. Figure 17 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 6 of the present disclosure. Figure 18 is a schematic cross-sectional view showing an example of the configuration of a photodetector according to Modification 6 of the present disclosure. Figure 19 is a block diagram showing an example of the configuration of an electronic device having the photodetector shown in Figure 1, etc. Figure 20A is a schematic diagram showing an example of the overall configuration of a photodetector system using the photodetector shown in Figure 1, etc. Figure 20B is a diagram showing an example of the circuit configuration of the photodetection system shown in Figure 20A. It is an explanatory diagram showing an example of the use of the photodetection device. Figure 22 is a diagram showing an example of a schematic configuration of an endoscopic surgical system.Figure 23 is a block diagram showing an example of the functional configuration of the camera head and CCU. Figure 24 is a block diagram showing an example of the schematic configuration of the vehicle control system. Figure 25 is an explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit.

[0010] Hereinafter, one embodiment of the present disclosure will be described in detail with reference to the drawings. The following description is a specific example of the present disclosure, and the present disclosure is not limited to the following embodiment. Furthermore, the present disclosure is not limited to the arrangement, dimensions, dimensional ratios, etc., of each component shown in each figure. The order of description is as follows: 1. Embodiment (An example in which a low-sensitivity pixel is configured by arranging multiple microlenses within a pixel in a photodetector that combines high-sensitivity and low-sensitivity pixels) 2. Modifications 2-1. Modification 1 (Another example of the layout of multiple microlenses arranged in a low-sensitivity pixel) 2-2. Modification 2 (Another example of the configuration of the photodetector) 2-3. Modification 3 (Another example of the configuration of the photodetector) 2-4. Modification 4 (Another example of the configuration of the photodetector) 2-5. Modification 5 (Another example of the configuration of the photodetector) 2-6. Modification 6 (Another example of the configuration of the photodetector) 3. Application Examples 4. Usage Examples 5. Application Examples

[0011] <1. Embodiments> Figure 1 schematically shows an example of a cross-sectional configuration of a photodetector (photodetector 1) according to one embodiment of the present disclosure. Figure 2A schematically shows an example of a planar layout of an inner lens (microlens 22L1) arranged in the high-sensitivity pixel P1 shown in Figure 1. Figure 2B schematically shows an example of a planar layout of an inner lens (microlens 22L2) arranged in the low-sensitivity pixel P2 shown in Figure 1. The photodetector 1 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor used in electronic devices such as digital still cameras and video cameras, and has a pixel section (pixel array section 100A) in which a plurality of pixels are arranged in a matrix in two dimensions as an imaging area. The photodetector 1 is, for example, a so-called back-illuminated photodetector in this CMOS image sensor.

[0012] The photodetector 1 has a pair of opposing surfaces (front surface 11S1 and back surface 11S2) and a pixel array section 100A in which a plurality of unit pixels P are arranged in an array. An inner lens (INL) layer 22 and an on-chip lens (OCL) layer 26 are laminated from the semiconductor substrate 11 side on the back surface 11S2 side of the semiconductor substrate 11. The photodetector 1 has a plurality of unit pixels P, including high-sensitivity pixels P1 and low-sensitivity pixels P2 which have lower sensitivity than the high-sensitivity pixels. Different numbers of microlenses 22L1 and 22L2 are provided on the high-sensitivity pixels P1 and the low-sensitivity pixels P2.

[0013] Here, the semiconductor substrate 11 corresponds to a specific example of the "semiconductor substrate" in one embodiment of the present disclosure, the back surface 11S2 corresponds to a specific example of the "first surface" in one embodiment of the present disclosure, and the front surface 11S1 corresponds to a specific example of the "second surface" in one embodiment of the present disclosure. The high-sensitivity pixel P1 corresponds to a specific example of the "first pixel" in one embodiment of the present disclosure, and the low-sensitivity pixel P2 corresponds to a specific example of the "second pixel" in one embodiment of the present disclosure. The microlenses 22L1 and 22L2 correspond to specific examples of the "first microlenses" as an embodiment of the present disclosure.

[0014] [Outline Configuration of the Light Detection Device] Figure 3 shows an example of the overall configuration of the light detection device 1 shown in Figure 1.

[0015] The light detection device 1 captures incident light (image light) from a subject via an optical lens system (for example, lens group 1001, see Figure 19), converts the amount of light of the incident light formed on the imaging surface into an electrical signal on a pixel-by-pixel basis, and outputs it as a pixel signal. The light detection device 1 has a pixel array section 100A as an imaging area on a semiconductor substrate 11, and in the peripheral region of this pixel array section 100A, it has, for example, a vertical drive circuit 111, a column signal processing circuit 112, a horizontal drive circuit 113, an output circuit 114, a control circuit 115, and input / output terminals 116.

[0016] In the pixel array section 100A, for example, multiple unit pixels P are arranged in a matrix in a two-dimensional manner. The multiple unit pixels P generate an image generation signal by photoelectric conversion of the subject image formed by the imaging lens in the photodiode PD. In order to achieve a high dynamic range, the unit pixels P have two types of pixels as described above: high-sensitivity pixels P1 and low-sensitivity pixels P2 which have lower sensitivity than the high-sensitivity pixels P1. The high-sensitivity pixels P1 and low-sensitivity pixels P2 are arranged periodically in the pixel array section 100A. When it is not necessary to distinguish between the high-sensitivity pixels P1 and low-sensitivity pixels P2, they will be described as unit pixels P.

[0017] Each unit pixel P is wired with, for example, a pixel drive line Lread (specifically, a row selection line and a reset control line) for each pixel row, and a vertical signal line Lsig for each pixel column. The pixel drive line Lread transmits drive signals for reading signals from the pixels. One end of the pixel drive line Lread is connected to the output terminal corresponding to each row of the vertical drive circuit 111.

[0018] The vertical drive circuit 111 is a pixel drive unit composed of a shift register, an address decoder, etc., which drives each unit pixel P of the pixel array section 100A, for example, in row units. The signals output from each unit pixel P of the pixel row selected and scanned by the vertical drive circuit 111 are supplied to the column signal processing circuit 112 through each of the vertical signal lines Lsig. The column signal processing circuit 112 is composed of amplifiers, horizontal selection switches, etc., provided for each vertical signal line Lsig.

[0019] The horizontal drive circuit 113 is composed of a shift register, an address decoder, etc., and sequentially drives each horizontal selection switch of the column signal processing circuit 112 while scanning it. Through this selection scanning by the horizontal drive circuit 113, the signals of each pixel transmitted through each of the vertical signal lines Lsig are sequentially output to the horizontal signal line 121 and transmitted to the outside of the semiconductor substrate 11 through the horizontal signal line 121.

[0020] The output circuit 114 performs signal processing on the signals sequentially supplied from each of the column signal processing circuits 112 via the horizontal signal lines 121 and outputs the processed signals. The output circuit 114 may, for example, only perform buffering, or may perform black level adjustment, column variation correction, and various digital signal processing operations.

[0021] The circuit portion including the vertical drive circuit 111, the column signal processing circuit 112, the horizontal drive circuit 113, the horizontal signal lines 121, and the output circuit 114 may be formed directly on the semiconductor substrate 11, or may be disposed in an external control IC. Further, those circuit portions may be formed on other substrates connected by a cable or the like.

[0022] The control circuit 115 receives a clock given from outside the semiconductor substrate 11, data for commanding an operation mode, etc., and outputs data such as internal information of the photodetection device 1. The control circuit 115 further has a timing generator that generates various timing signals, and performs drive control of peripheral circuits such as the vertical drive circuit 111, the column signal processing circuit 112, and the horizontal drive circuit 113 based on the various timing signals generated by the timing generator.

[0023] The input / output terminal 116 exchanges signals with the outside. <{0000091}>

[0024] [Circuit Configuration of Unit Pixel] FIG. 4 shows an example of the readout circuit of the unit pixel P of the photodetection device 1 shown in FIG. 3. The unit pixel P has, for example, as shown in FIG. 4, one photoelectric conversion unit 12, a transfer transistor TR, a floating diffusion FD, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL.

[0025] The photoelectric conversion unit 12 is a so-called buried-type photodiode (PD) in which an n-type impurity region is formed inside a p-type impurity region formed in the semiconductor substrate 11. The photoelectric conversion unit 12 generates charges according to the amount of received light and accumulates the generated charges to a certain amount. The anode of the photoelectric conversion unit 12 is connected to the ground voltage line, and the cathode is connected to the source of the transfer transistor TR.

[0026] The transfer transistor TR is connected between the photoelectric conversion unit 12 and the floating diffusion FD. A drive signal TRsig is applied to the gate electrode of the transfer transistor TR. When this drive signal TRsig becomes active, the transfer gate of the transfer transistor TR becomes conductive, and the signal charges accumulated in the photoelectric conversion unit 12 are transferred to the floating diffusion FD through the transfer transistor TR.

[0027] The floating diffusion FD is connected between the transfer transistor TR and the amplification transistor AMP. The floating diffusion FD performs charge-voltage conversion on the signal charges transferred by the transfer transistor TR into a voltage signal and outputs it to the amplification transistor AMP.

[0028] The reset transistor RST is connected between the floating diffusion FD and the power supply unit. A drive signal RSTsig is applied to the gate electrode of the reset transistor RST. When this drive signal RSTsig becomes active, the reset gate of the reset transistor RST becomes conductive, and the potential of the floating diffusion FD is reset to the level of the power supply unit.

[0029] The gate electrode of the amplification transistor AMP is connected to the floating diffusion FD, and the drain electrode is connected to the power supply unit, respectively, serving as the input part of a readout circuit for the voltage signal held by the floating diffusion FD, a so-called source follower circuit. That is, the source electrode of the amplification transistor AMP is connected to the vertical signal line Lsig through the selection transistor SEL, thereby constituting a constant current source and a source follower circuit connected to one end of the vertical signal line Lsig.

[0030] The selection transistor SEL is connected between the source electrode of the amplification transistor AMP and the vertical signal line Lsig. A drive signal SELsig is applied to the gate electrode of the selection transistor SEL. When this drive signal SELsig becomes active, the selection transistor SEL becomes conductive, and the unit pixel P becomes selected. As a result, the readout signal (pixel signal) output from the amplification transistor AMP is output to the vertical signal line Lsig via the selection transistor SEL.

[0031] [Configuration of Unit Pixels] As described above, the light detection device 1 is, for example, a back-illuminated light detection device, and each of the multiple unit pixels P arranged in a matrix in two dimensions in the pixel array section 100A has a configuration in which, for example, a light receiving section 10, a light collecting section 20 provided on the light incident side S1 of the light receiving section 10, and a multilayer wiring layer 30 provided on the side opposite to the light incident side S1 of the light receiving section 10 are stacked.

[0032] The light-receiving unit 10 includes a semiconductor substrate 11 having opposing front surface 11S1 and back surface 11S2, and a plurality of photoelectric conversion units 12 embedded in the semiconductor substrate 11. The semiconductor substrate 11 is made of, for example, a silicon (Si) substrate. The photoelectric conversion units 12 are, for example, PIN (Positive Intrinsic Negative) type photodiodes (PDs) and have a pn junction in a predetermined region of the semiconductor substrate 11. The photoelectric conversion units 12 are embedded for each unit pixel P.

[0033] The light-receiving section 10 further includes a fixed charge film 13 and a light-shielding section 14.

[0034] The fixed charge film 13 is provided on the back surface 11S2 of the semiconductor substrate 11 to suppress the generation of dark current caused by the interface state of the back surface 11S2, which is the light-receiving surface of the semiconductor substrate 11. The fixed charge film 13 may be a film with a positive fixed charge or a film with a negative fixed charge. The electric field induced by the fixed charge film 13 causes a hole accumulation layer to form near the back surface 11S2 of the semiconductor substrate 11. This hole accumulation layer suppresses the generation of electrons from the back surface 11S2.

[0035] As the constituent material of the fixed charge film 13, a semiconductor material or a conductive material having a band gap wider than that of the semiconductor substrate 11 can be mentioned. Specifically, for example, hafnium oxide (HfO ｘ , , , ｙ , ｙ , ｘ , ｘ , ｘ ,

[0037] ,

[0036] , ｘ , ｘ , ｘ ), aluminum oxide (AlO ｘ ), zirconium oxide (ZrO ｘ ), tantalum oxide (TaO ｘ ), titanium oxide (TiO ｘ ), lanthanum oxide (LaO ｘ ), praseodymium oxide (PrO ｘ ), cerium oxide (CeO ｘ ), neodymium oxide (NdO ｘ ), promethium oxide (PmO ｘ ), samarium oxide (SmO ｘ ), europium oxide (EuO ｘ ), gadolinium oxide (GdO ｘ ), terbium oxide (TbO ｘ ), dysprosium oxide (DyO ｘ ), holmium oxide (HoO ｘ ), thulium oxide (TmO ｘ ), ytterbium oxide (YbO ｘ ), lutetium oxide (LuO ｘ ), yttrium oxide (YO ｘ ), hafnium nitride (HfN ｘ ), aluminum nitride (AlN ｘ ), hafnium oxynitride (HfO​​​​​​​​​​​​​Examples of materials constituting the light-shielding portion 14 include light-shielding materials. Specifically, examples include tungsten (W), silver (Ag), copper (Cu), titanium (Ti), aluminum (Al), or alloys thereof. In addition, metal compounds such as TiN can be used as materials constituting the light-shielding portion 14. The light-shielding portion 14 may be formed as, for example, a single layer or a laminated layer. In the case of a laminated layer, in order to improve adhesion with the fixed charge film 13, a layer made of, for example, Ti, tantalum (Ta), W, cobalt (Co), or molybdenum (Mo), or alloys thereof, nitrides, oxides, or carbides can be provided as an underlayer.

[0038] The light-shielding portion 14 may also serve to shield the unit pixels P that determine the optical black level. Furthermore, the light-shielding portion 14 may also serve to shield the peripheral circuits provided in the peripheral region of the pixel array portion 100A to suppress the generation of noise. It is preferable that the light-shielding portion 14 is grounded so as not to be destroyed by plasma damage caused by accumulated charge during processing.

[0039] The light-collecting section 20 is provided on the light-incident side S1 of the light-receiving section 10, and for example, an interlayer film 21A, an inner lens layer 22, an interlayer film 21B, a sealing layer 24, a color filter 25, and an on-chip lens layer 26 are stacked in this order from the light-receiving section 10 side. The light-collecting section 20 further has a light-shielding wall 23 that extends in the Z-axis direction so as to penetrate the interlayer films 21A, 21B and the inner lens layer 22 between adjacent unit pixels P.

[0040] Interlayer film 21A maintains the gap between the semiconductor substrate 11 and the inner lens 22 layer and makes the surface on the inner lens layer 22 side a flat surface. Interlayer film 21B maintains the gap between the inner lens layer 22 and the on-chip lens layer 26 and makes the surface on the on-chip lens layer 26 side a flat surface. Interlayer films 21A and 21B correspond to one specific example of an "intermediate layer" as one embodiment of the present disclosure. Interlayer films 21A and 21B are, for example, silicon oxide (SiO2). ｘ ), silicon nitride (SiN ｘ ) and silicon oxynitride (SiO ｘ Nｙ It is formed using ) etc.

[0041] The inner lens layer 22 is for guiding light incident from above to the photoelectric conversion unit 12. The inner lens layer 22 is provided so as to cover the entire surface of the pixel array 100A, and has a plurality of microlenses 22L on its surface. The number of microlenses 22L on the surface of the inner lens layer 22 is different for the high-sensitivity pixels P1 and the low-sensitivity pixels P2. For example, the plurality of microlenses 22L include two types of microlenses 22L1 and 22L2 having different sizes. One microlens 22L1 is placed in the high-sensitivity pixel P1, and four microlenses 22L2, which are smaller than microlens 22L1, are placed in the low-sensitivity pixel P2 in a 2x2 arrangement, as shown in Figures 1 and 2B. When light incident on the low-sensitivity pixel P2 is collected, the focal point is dispersed by the four microlenses 22L2, and a portion of it is absorbed by the light-shielding wall 23. As a result, as will be explained in more detail later, in the light detection device 1, periodic changes in sensitivity due to the angle of incidence are suppressed, and low-sensitivity pixels (low-sensitivity pixels P2) are formed.

[0042] Figures 1 and 2 show an example in which a low-sensitivity pixel P2 is provided with four microlenses 22L arranged in a 2x2 grid, but the invention is not limited to this. The low-sensitivity pixel P2 may also be provided with nine microlenses 22L arranged in a 3x3 grid, as in the photodetector 1A shown in Figure 5, or with sixteen microlenses 22L arranged in a 4x4 grid, as in the photodetector 1B shown in Figure 6. By increasing the number of microlenses 22L arranged in the low-sensitivity pixel P2, the periodic change in sensitivity due to the angle of incidence is further reduced, and the sensitivity can be further lowered. However, the decrease in sensitivity becomes smaller with increasing numbers of microlenses 22L2 arranged in the pixel, and even if six rows x six columns or more of microlenses 22L2 are arranged, the sensitivity will be approximately the same as that of a low-sensitivity pixel P2 in which microlenses 22L2 are arranged in a 5x5 grid.

[0043] The inner lens layer 22 is constructed using, for example, a high refractive index material. Examples of materials used for the inner lens layer 22 include silicon oxide (SiO₂). ｘ ) and silicon nitride (SiN ｘ Examples of inorganic materials include those listed above. In addition, the inner lens layer 22 may be formed using organic materials with a high refractive index, such as episulfide resins, thietan compounds, or their resins.

[0044] The shape of the microlens 22L is not particularly limited, and various lens shapes such as hemispherical or semi-cylindrical can be used.

[0045] The light-shielding wall 23 is designed to prevent obliquely incident light from leaking into adjacent unit pixels P, and in a cross-sectional view, it is formed to penetrate the interlayer films 21A, 21B and the inner lens layer 22. Furthermore, similar to the light-shielding portion 14, the light-shielding wall 23 is formed to extend in a grid pattern across the entire pixel array portion 100A between adjacent unit pixels P.

[0046] Examples of materials that make up the light-shielding wall 23 include tungsten (W), silver (Ag), copper (Cu), titanium (Ti), aluminum (Al), or alloys thereof. In addition, metal compounds such as TiN can be used as materials that make up the light-shielding wall 23. The light-shielding wall 23 may be formed, for example, as a single layer or a laminated layer.

[0047] The sealing layer 24 is, for example, silicon oxide (SiO ｘ ), silicon nitride (SiN ｘ ) and silicon oxynitride (SiO ｘ N ｙ It is formed using ) etc.

[0048] The color filter 25 is configured to selectively transmit light in a predetermined wavelength range from the incident light. The color filter 25 includes, for example, a color filter that transmits red (R) light (red filter 25R), a color filter that transmits green (G) light (green filter 25G), and a color filter that transmits blue (B) light (blue filter 25B).

[0049] Multiple unit pixels P arranged in a two-dimensional array in the pixel array section 100A become red pixels Pr capable of generating an R component pixel signal, green pixels Pg capable of generating a G component pixel signal, and blue pixels Pb capable of generating a B component pixel signal, respectively, when the red filter 25R, green filter 25G, and blue filter 25B are arranged. In other words, the red pixels Pr, green pixels Pg, and blue pixels Pb described below are synonymous with the arrangement of the red filter 25R, green filter 25G, and blue filter 25B.

[0050] In the pixel array section 100A, red pixels Pr, green pixels Pg, and blue pixels Pb are arranged repeatedly. The red pixels Pr, green pixels Pg, and blue pixels Pb are arranged according to a Bayer array, for example. In the pixel array section 100A, four pixels arranged in a 2x2 configuration, each consisting of one red pixel Pr, two green pixels Pg, and one blue pixel Pb (hereinafter referred to as a 2x2 pixel), are repeatedly provided. In a 2x2 pixel, two green pixels Pg are arranged in one diagonal direction, and one red pixel Pr and one blue pixel Pb are arranged in the other diagonal direction. In other words, the pixel array section 100A has, for example, pixel rows in which green pixels Pg and red pixels Pr are arranged alternately, and pixel rows in which blue pixels Pb and green pixels Pg are arranged alternately.

[0051] Note that the arrangement of unit pixels P is not limited to the example described above and can be set arbitrarily. For example, red pixels Pr, green pixels Pg, and blue pixels Pb may each be arranged in 2x2 pixel units. In the pixel array section 100A, one set of red pixels Pr, two sets of green pixels Pg, and one set of unit pixels P, arranged in 2x2 pixel units, may be arranged according to the Bayer array.

[0052] Furthermore, the color filter 25 is not limited to primary color (RGB) color filters, but may also be complementary color filters such as Cy (cyan), Mg (magenta), and Ye (yellow). A filter corresponding to W (white), that is, a filter that transmits light across the entire wavelength range of incident light, may also be provided. The color filter 25 may also be a filter that transmits infrared light.

[0053] The color filter 25 can be formed, for example, using pigments or dyes. The thickness of the color filter 25 may vary for each color, taking into consideration the color reproducibility and sensor sensitivity based on its spectral distribution. In monochrome pixels, a layer made of a transparent material can be considered as the color filter 25. In infrared pixels, a layer made of a material that selectively transmits infrared light can be considered as the color filter 25.

[0054] The on-chip lens layer 26, like the inner lens layer 22, is for guiding light incident from above to the photoelectric conversion unit 12. The on-chip lens layer 26 is provided so as to cover the entire surface of the pixel array unit 100A, and its surface has, for example, a plurality of gapless microlenses 26L. The microlenses 26L correspond to a specific example of the "second microlens" as one embodiment of the present disclosure, and are provided for each unit pixel P as shown in Figure 1.

[0055] The on-chip lens layer 26, like the inner lens layer 22, is constructed using, for example, a high refractive index material. Examples of materials used for the on-chip lens layer 26 include silicon oxide (SiO₂). ｘ ) and silicon nitride (SiN ｘ Examples of inorganic materials include those listed above. In addition, the on-chip lens layer 26 may be formed using organic materials with a high refractive index, such as episulfide resins, thietan compounds, or their resins.

[0056] The shape of the microlens 26L is not particularly limited, and various lens shapes such as hemispherical or semi-cylindrical can be used.

[0057] The multilayer wiring layer 30 is provided on the side opposite to the light incident side S1 of the light receiving unit 10, specifically on the surface 11S1 side of the semiconductor substrate 11. The multilayer wiring layer 30 has a configuration in which multiple wiring layers 32, 33, and 34 are stacked with an interlayer insulating layer 31 in between. In addition to the readout circuit described above, the multilayer wiring layer 30 has, for example, a vertical drive circuit 111, a column signal processing circuit 112, a horizontal drive circuit 113, an output circuit 114, a control circuit 115, and input / output terminals 116 formed on it.

[0058] The interlayer insulating layer 341 is, for example, silicon oxide (SiO ｘ ), silicon nitride (SiN ｘ ) and silicon oxynitride (SiO ｘ N ｙ It is formed by a single layer film consisting of one of the following, or a laminated film consisting of two or more of these.

[0059] The wiring layers 32, 33, and 34 are formed using, for example, aluminum (Al), copper (Cu), or tungsten (W). Alternatively, the wiring layers 32, 33, and 34 may be formed using polysilicon (Poly-Si).

[0060] [Function and Effects] In the light detection device 1 of this embodiment, an inner lens layer 22 and an on-chip lens layer 26 are laminated from the semiconductor substrate 11 side to the back surface 11S2 side, which is the light-receiving surface of a semiconductor substrate 11 having a pixel array section 100A in which a plurality of unit pixels P are arranged in an array. The light detection device 1 has a plurality of unit pixels P, namely high-sensitivity pixels P1 and low-sensitivity pixels P2. The high-sensitivity pixels P1 have one microlens 22L1 and 26L as the inner lens and on-chip lens, respectively, and the low-sensitivity pixels P2 have an inner lens and an on-chip lens, namely n arranged in n rows × n columns. ２ A number of microlenses 22L2 (where n is an integer greater than or equal to 2) and one microlens 26L are arranged. This will be explained below.

[0061] In recent years, image sensors have been developed that incorporate pixels with intentionally reduced sensitivity in order to achieve a high dynamic range in image sensors. One proposed method for reducing sensitivity is to use a light-reducing structure having a light-shielding pattern in which light-transmitting regions that transmit light and light-shielding regions that partially block light are periodically arranged, such as the light-shielding layer 27 shown in Figures 12 and 13. This light-reducing structure is placed below or above the inner lens, but depending on its placement, it can cause undulations in the oblique incidence waveform.

[0062] Figure 7 is a simulation diagram showing the relationship between the angle of incidence of light and the pixel output for pixels without the light-reducing structure, pixels with the light-reducing structure placed on the light-receiving surface, pixels with the light-reducing structure placed on the lower surface of the inner lens (below the inner lens), and pixels with the light-reducing structure placed above the inner lens (below the color filter (CF)). As the angle of incidence of light changes, the light-gathering spot moves in one direction. In an image sensor with a light-reducing structure placed on the light-receiving surface, when the light-gathering spot overlaps with the light-shielding region, most of the incident light is blocked, and only a small amount of light that has passed through the surrounding transparent region enters the photoelectric conversion unit. Because the light-reducing structure has periodically arranged transparent and light-shielding regions, the amount of light incident on the photoelectric conversion unit also changes periodically according to the angle of incidence of light. In other words, as shown in Figure 7, in an image sensor with a light-reducing structure placed on the light-receiving surface, the pixel output changes periodically depending on the angle of incidence of light. If the decrease in sensitivity changes depending on the angle of incidence of light, the rate of decrease in sensitivity due to the lens F value will also change, which may lead to a decrease in the image quality of the captured image.

[0063] By placing the above-mentioned light-reducing structure on the lower surface of the inner lens, the change in the amount of incident light to the photoelectric conversion section according to the angle of incidence is reduced, but as shown in Figure 7, the oblique incidence waveform still contains undulation. When the above-mentioned light-reducing structure is placed above the inner lens, as shown in Figure 7, although the undulation is suppressed, there are concerns about flare due to reflection from the light-reducing structure and deterioration of image quality due to uneven sweeping during color filter formation.

[0064] In contrast, in this embodiment, as described above, the high-sensitivity pixel P1 has one microlens 22L1 as an inner lens, and the low-sensitivity pixel P2 has, for example, n arranged in n rows × n columns as an inner lens. ２ Microlenses 22L2 are arranged in this manner. As a result, in the low-sensitivity pixels P2, the effective position of the incident light is dispersed, and some of it is absorbed, for example, by the light-shielding wall 23.

[0065] Figure 8 is a simulation diagram showing the relationship between the angle of incidence of light and the pixel output for a high-sensitivity pixel P1, a low-sensitivity pixel P2 (P1_2×2) with microlenses 22L2 arranged in 2 rows and 2 columns, a low-sensitivity pixel P2 (P1_3×3) with microlenses 22L2 arranged in 3 rows and 3 columns, and a low-sensitivity pixel P2 (P1_4×4) with microlenses 22L2 arranged in 4 rows and 4 columns. Compared to the high-sensitivity pixel P1, the low-sensitivity pixel P2, which has multiple microlenses 22L2 arranged as inner lenses, has reduced sensitivity while suppressing the periodic change in pixel output due to the angle of incidence of light.

[0066] As described above, the light detection device 1 of this embodiment makes it possible to achieve both high dynamic range and improved image quality.

[0067] Next, modifications 1 to 6 of the present disclosure, as well as examples of application, use, and application, will be described. In the following, components similar to those in the above embodiments will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0068] <2. Modifications> (2-1. Modification 1) Figures 9 to 11 schematically represent an example of a planar layout of a plurality of microlenses 22L2 arranged in a low-sensitivity pixel P2 according to Modification 1 of the present disclosure.

[0069] In the above embodiment, an example was shown in which multiple microlenses 22L2 are arranged in an n x n row configuration on a low-sensitivity pixel P2, but the invention is not limited to this.

[0070] The multiple microlenses 22L2 arranged in the low-sensitivity pixel P2 may be arranged in an n row and m column configuration (where n is an integer greater than or equal to 1, and m is an integer greater than or equal to 2, different from n), as shown in Figure 9. By changing the number of microlenses 22L2 in the row direction (X-axis direction) and the column direction (Y-axis direction) in this way, an asymmetrical oblique incidence waveform can be obtained.

[0071] The multiple microlenses 22L2 arranged in the low-sensitivity pixel P2 may be arranged concentrically from the center of the low-sensitivity pixel P2, for example, as shown in Figures 10 and 11. By arranging the multiple microlenses 22L2 in such a highly optically symmetrical manner, a symmetrical oblique incidence waveform can be obtained in each angular direction in the XY plane.

[0072] (2-2. Modification 2) Figure 12 schematically shows an example of the cross-sectional configuration of a photodetector (photodetector 1C) according to Modification 2 of the present disclosure. Figure 13 schematically shows an example of the planar layout of the light-shielding layer 27 shown in Figure 12. The photodetector 1C is, for example, a CMOS image sensor used in electronic devices such as digital still cameras and video cameras, and is, like the photodetector 1 of the above embodiment, for example, a so-called back-illuminated photodetector.

[0073] In this modified example, the light detection device 1C has a light-shielding layer 27 positioned below the inner lens layer 22 in the low-sensitivity pixel P2. Except for this point, the light detection device 1C has substantially the same configuration as the light detection device 1 of the above embodiment.

[0074] The light-shielding layer 27 corresponds to a specific example of the "light-shielding layer" as one embodiment of the present disclosure, and has a plurality of apertures 27H that transmit light in the XY plane direction. The plurality of apertures 27H are formed in the same layout as the microlenses 22L2, for example, as shown in Figure 13. That is, the plurality of apertures 27H are each provided below the plurality of microlenses 22L2 arranged in the low-sensitivity pixel P2. The light-shielding layer 27 attenuates incident light according to the arrangement pattern of the plurality of apertures 27H.

[0075] Examples of materials constituting the light-shielding layer 27 include tungsten (W), silver (Ag), copper (Cu), titanium (Ti), aluminum (Al), or alloys thereof. Other materials constituting the light-shielding layer 27 include metal compounds such as TiN. The light-shielding layer 27 may be formed, for example, as a single layer or a multilayer film.

[0076] Thus, in this modified light detection device 1C, the light-shielding layer 27 is placed below the inner lens layer 22 in the low-sensitivity pixel P2, making it possible to further reduce the sensitivity of the low-sensitivity pixel P2. Therefore, in addition to the effects of the above embodiment, it is possible to provide a light detection device 1C with a higher dynamic range.

[0077] (2-3. Modification 3) Figure 14 schematically shows an example of the cross-sectional configuration of a photodetector (photodetector 1D) according to Modification 3 of the present disclosure. The photodetector 1D is, for example, a CMOS image sensor used in electronic devices such as digital still cameras and video cameras, and is, similar to the photodetector 1 of the above embodiment, for example, a so-called back-illuminated photodetector.

[0078] In this modified example, the photodetector 1D has an inter-pixel separation unit 15 provided between photoelectric conversion units 12, which are provided for each unit pixel P on the semiconductor substrate 11. Except for this point, the photodetector 1D has substantially the same configuration as the photodetector 1 of the above embodiment.

[0079] The inter-pixel isolation section 15 corresponds to a specific example of the "inter-pixel isolation section" as one embodiment of the present disclosure, and is for electrically and optically separating adjacent unit pixels P. The inter-pixel isolation section 15 is provided, for example, between adjacent unit pixels P. In other words, the inter-pixel isolation section 15 is provided around the unit pixels P and, like the light-shielding section 14 and the light-shielding wall 23, is formed to extend in a grid pattern over the entire pixel array section 100A. The inter-pixel isolation section 15 extends, for example, from the back surface 11S2 side of the semiconductor substrate 11 toward the front surface 11S1 side. The inter-pixel isolation section 15 may have a so-called FTI (Full Trench Isolation) structure that penetrates between the front surface 11S1 and the back surface 11S2 of the semiconductor substrate 11, or it may have a DTI (Deep Trench Isolation) structure with a bottom surface inside the semiconductor substrate 11, as shown in Figure 14, for example.

[0080] The inter-pixel separation section 15 is formed using a material containing, for example, at least one of a light-shielding elemental metal, metal alloy, metal nitride, and metal silicide. More specifically, the constituent materials include tungsten (W), silver (Ag), copper (Cu), titanium (Ti), aluminum (Al), or alloys thereof. Among these, Al is the most optically preferable constituent material. In addition, the inter-pixel separation section 15 can be formed, for example, by diffusing graphite, a low refractive index material, or p-type impurities.

[0081] Thus, in this modified light detection device 1D, an inter-pixel separation section 15 is provided between adjacent unit pixels P. Therefore, for example, it is possible to suppress the leakage of light dispersed by multiple microlenses 22L2 arranged in a low-sensitivity pixel P2 into adjacent pixels. Thus, in addition to the effects of the above embodiment, it is possible to suppress the occurrence of crosstalk.

[0082] (2-4. Modification 4) Figure 15 schematically shows an example of a cross-sectional configuration of a photodetector (photodetector 1E) according to Modification 4 of the present disclosure. The photodetector 1E is, for example, a CMOS image sensor used in electronic devices such as digital still cameras and video cameras, and is, similar to the photodetector 1 of the above embodiment, for example, a so-called back-illuminated photodetector.

[0083] In this modified example, the photodetector 1E has multiple inner lens layers 22 (in this case, two layers, inner lens layers 22A and 22B) stacked in the Z-axis direction on a low-sensitivity pixel P2. Except for this point, the photodetector 1E has substantially the same configuration as the photodetector 1 of the above embodiment.

[0084] In this modified light detection device 1E, multiple inner lens layers 22 are stacked on the low-sensitivity pixel P2. Therefore, in addition to the effects of the above embodiment, a further decrease in sensitivity can be expected due to greater light diffusion.

[0085] (2-5. Modification 5) Figure 16 schematically shows an example of a cross-sectional configuration of a photodetector (photodetector 1F) according to Modification 5 of the present disclosure. The photodetector 1F is, for example, a CMOS image sensor used in electronic devices such as digital still cameras and video cameras, and is, similar to the photodetector 1 of the above embodiment, for example, a so-called back-illuminated photodetector.

[0086] In this modified example, the photodetector 1F changes the number or shape of the microlenses 22L2 arranged in the low-sensitivity pixel P2 depending on the type of color filter 25 provided (for example, any of the red filter 25R, green filter 25G, and blue filter 25B). Except for this point, the photodetector 1F has substantially the same configuration as the photodetector 1 of the above embodiment.

[0087] Generally, the red pixel Pr, green pixel Pg, and blue pixel Pb, which detect wavelengths corresponding to RGB, have different sensitivities to each other depending on the detected wavelength. In the modified photodetector 1F, for example, as shown in Figure 16, the number of microlenses 22L2 arranged in the red pixel Pr2 and the green pixel Pg2 are made different. In this way, by changing the number or shape of the microlenses 22L2 arranged in the low-sensitivity pixel P2 according to the type of color filter 25 provided, the sensitivity of each color pixel can be made uniform. Therefore, in the modified photodetector 1G, in addition to the effects of the above embodiment, it is possible to further improve image quality.

[0088] (2-6. Modification 6) Figure 17 schematically shows an example of a cross-sectional configuration of a photodetector according to Modification 6 of the present disclosure (photodetector 1G). Figure 18 schematically shows another example of a cross-sectional configuration of a photodetector according to Modification 6 of the present disclosure (photodetector 1H). Photodetectors 1G and 1H are, for example, CMOS image sensors used in electronic devices such as digital still cameras and video cameras, and are, similar to photodetector 1 in the above embodiment, for example, so-called back-illuminated photodetectors.

[0089] In this modified example, the photodetector 1G changes the expansion width of the light-shielding portion 14 in the XY plane direction between high-sensitivity pixels P1 and low-sensitivity pixels P2 of the same color. In this modified example, the photodetector 1H changes the expansion width of the light-shielding portion 14 in the XY plane direction between low-sensitivity pixels P2 of different colors. Except for this point, the photodetectors 1G and 1H have substantially the same configuration as the photodetector 1 of the above embodiment.

[0090] In the light detection device 1G, the expanded width W2 of the light-shielding portion 14 of the low-sensitivity pixel P2 was made larger than the expanded width W1 of the light-shielding portion 14 of the high-sensitivity pixel P1. In the light detection device 1H, the expanded width W4 of the light-shielding portion 14 of the low-sensitivity red pixel Pr2 was made larger than the expanded width W3 of the light-shielding portion 14 of the low-sensitivity green pixel Pg2.

[0091] Thus, in the modified photodetector devices 1G and 1H, the expansion width of the light-shielding portion 14 is changed between high-sensitivity pixels P1 and low-sensitivity pixels P2 of the same color, or between low-sensitivity pixels P2 of different colors. In addition to the effects of the above embodiment, it is possible to adjust the sensitivity of the pixels.

[0092] <3. Application Examples> (Application Example 1) The above-mentioned light detection device 1 can be applied to any type of electronic device equipped with an imaging function, such as camera systems like digital still cameras and video cameras, and mobile phones with imaging capabilities. Figure 19 shows a schematic configuration of the electronic device 1000.

[0093] The electronic device 1000 includes, for example, a lens group 1001, a light detection device 1, a DSP (Digital Signal Processor) circuit 1002, a frame memory 1003, a display unit 1004, a storage unit 1005, an operation unit 1006, and a power supply unit 1007, all of which are interconnected via a bus line 1008.

[0094] The lens group 1001 captures incident light (image light) from the subject and forms an image on the imaging surface of the light detection device 1. The light detection device 1 converts the amount of incident light formed on the imaging surface by the lens group 1001 into an electrical signal on a pixel-by-pixel basis and supplies it as a pixel signal to the DSP circuit 1002.

[0095] The DSP circuit 1002 is a signal processing circuit that processes signals supplied from the light detection device 1. The DSP circuit 1002 outputs image data obtained by processing the signals from the light detection device 1. The frame memory 1003 temporarily holds the image data processed by the DSP circuit 1002 in frame units.

[0096] The display unit 1004 consists of a panel-type display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel, and the storage unit 1005 records the video or still image data captured by the light detection device 1 onto a recording medium such as a semiconductor memory or a hard disk.

[0097] The operation unit 1006 outputs operation signals for various functions possessed by the electronic device 1000 in accordance with user operations. The power supply unit 1007 appropriately supplies various power sources to the DSP circuit 1002, frame memory 1003, display unit 1004, storage unit 1005, and operation unit 1006, which serve as the operating power sources for these devices.

[0098] (Application Example 2) Figure 20A schematically shows an example of the overall configuration of a photodetection system 2000 equipped with a photodetector 1. Figure 20B shows an example of the circuit configuration of the photodetection system 2000. The photodetection system 2000 includes a light-emitting device 2001 as a light source that emits infrared light L2, and a photodetector 2002 as a light-receiving unit having a photoelectric conversion element. The photodetector 1 described above can be used as the photodetector 2002. The photodetection system 2000 may further include a system control unit 2003, a light source drive unit 2004, a sensor control unit 2005, a light source side optical system 2006, and a camera side optical system 2007.

[0099] The photodetector 2002 can detect light L1 and light L2. Light L1 is light reflected from ambient light from the outside by the subject (object to be measured) 2100 (Figure 20A). Light L2 is light that has been emitted by the light-emitting device 2001 and then reflected by the subject 2100. Light L1 is, for example, visible light, and light L2 is, for example, infrared light. Light L1 is detectable in the photoelectric conversion unit of the photodetector 2002, and light L2 is detectable in the photoelectric conversion region of the photodetector 2002. Image information of the subject 2100 can be obtained from light L1, and distance information between the subject 2100 and the photodetector system 2000 can be obtained from light L2. The photodetector system 2000 can be mounted on, for example, electronic devices such as smartphones or mobile devices such as cars. The light-emitting device 2001 can be, for example, a semiconductor laser, a surface-emitting semiconductor laser, or a vertical-cavity surface-emitting laser (VCSEL). As a detection method for the light L2 emitted from the light-emitting device 2001 by the photodetector 2002, for example, the iTOF method can be used, but is not limited to this. In the iTOF method, the photoelectric conversion unit can measure the distance to the subject 2100 by, for example, the time-of-flight (TOF). As a detection method for the light L2 emitted from the light-emitting device 2001 by the photodetector 2002, for example, the structured light method or the stereo vision method can also be used. For example, in the structured light method, the distance between the photodetector 2000 and the subject 2100 can be measured by projecting a predetermined pattern of light onto the subject 2100 and analyzing the degree of distortion of the pattern. In the stereo vision method, for example, the distance between the photodetector 2000 and the subject can be measured by using two or more cameras to acquire two or more images of the subject 2100 from two or more different viewpoints. Furthermore, the light-emitting device 2001 and the light-detecting device 2002 can be synchronously controlled by the system control unit 2003.

[0100] <4. Examples of Use> Figure 21 shows an example of use of the light detection device (for example, light detection device 1) according to the above embodiment. The light detection device 1 described above can be used in various cases to sense light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as follows.

[0101] - Devices that capture images for viewing purposes, such as digital cameras and portable devices with camera functions. - Devices used for traffic purposes, such as in-vehicle sensors that capture images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and recognition of the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles. - Devices used in televisions and home appliances such as refrigerators and air conditioners that capture user gestures and allow device operation according to those gestures. - Devices used for medical and healthcare purposes, such as endoscopes and devices that perform angiography using infrared light reception. - Devices used for security purposes, such as surveillance cameras for crime prevention and cameras for person recognition. - Devices used for beauty purposes, such as skin measuring devices that capture images of the skin and microscopes that capture images of the scalp. - Devices used for sports purposes, such as action cameras and wearable cameras for sports use. - Devices used for agriculture, such as cameras that monitor the condition of fields and crops.

[0102] <5. Application Examples> (Application Example to Endoscopic Surgical Systems) The technology disclosed herein (this technology) can be applied to various products. For example, the technology disclosed herein may be applied to an endoscopic surgical system.

[0103] Figure 22 is a diagram showing an example of a schematic configuration of an endoscopic surgical system to which the technology described herein (the technology) may be applied.

[0104] Figure 22 illustrates a surgeon (physician) 11131 performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgical system 11000. As shown in the figure, the endoscopic surgical system 11000 consists of an endoscope 11100, other surgical instruments 11110 such as a pneumoperitoneum tube 11111 and an energy treatment device 11112, a support arm device 11120 for supporting the endoscope 11100, and a cart 11200 equipped with various devices for endoscopic surgery.

[0105] The endoscope 11100 consists of a barrel 11101, the tip of which is inserted into the body cavity of the patient 11132 for a predetermined length, and a camera head 11102 connected to the base end of the barrel 11101. In the illustrated example, the endoscope 11100 is shown as a so-called rigid endoscope having a rigid barrel 11101, but the endoscope 11100 may also be configured as a so-called flexible endoscope having a flexible barrel.

[0106] An opening into which an objective lens is fitted is provided at the tip of the microscope tube 11101. A light source device 11203 is connected to the endoscope 11100, and the light generated by the light source device 11203 is guided to the tip of the microscope tube by a light guide extending inside the microscope tube 11101, and is irradiated through the objective lens towards the object to be observed inside the body cavity of the patient 11132. The endoscope 11100 may be a straight-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0107] The camera head 11102 contains an optical system and an image sensor. Reflected light from the object being observed (observation light) is focused onto the image sensor by the optical system. The image sensor converts the observation light into electrical signals, generating an electrical signal corresponding to the observation light, i.e., an image signal corresponding to the observed image. This image signal is transmitted as RAW data to the camera control unit (CCU) 11201.

[0108] The CCU 11201 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and other components, and comprehensively controls the operation of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102 and performs various image processing operations on that image signal, such as development processing (demosaic processing), to display an image based on that image signal.

[0109] The display device 11202 displays an image based on an image signal that has been processed by the CCU 11201, under control from the CCU 11201.

[0110] The light source device 11203 is composed of a light source such as an LED (light-emitting diode) and supplies illumination light to the endoscope 11100 when photographing the surgical area, etc.

[0111] The input device 11204 is an input interface for the endoscopic surgical system 11000. The user can input various types of information and instructions to the endoscopic surgical system 11000 via the input device 11204. For example, the user can input instructions to change the imaging conditions (type of light, magnification, focal length, etc.) of the endoscope 11100.

[0112] The treatment instrument control device 11205 controls the drive of the energy treatment instrument 11112 for purposes such as tissue cauterization, incision, or blood vessel sealing. The insufflation device 11206 injects gas into the body cavity of the patient 11132 via the insufflation tube 11111 to inflate the body cavity for the purpose of securing a field of view by the endoscope 11100 and securing the operator's workspace. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various formats such as text, images, or graphs.

[0113] The light source device 11203 that supplies illumination light to the endoscope 11100 when photographing the surgical area can be configured as a white light source consisting of, for example, an LED, a laser light source, or a combination thereof. When the white light source is configured as a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so the white balance of the captured image can be adjusted in the light source device 11203. In this case, it is also possible to capture images corresponding to each of the RGB colors in time-division by irradiating the observation target with laser light from each of the RGB laser light sources in time-division and controlling the drive of the image sensor of the camera head 11102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter on the image sensor.

[0114] Furthermore, the light source device 11203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the drive of the image sensor of the camera head 11102 in synchronization with the timing of the change in light intensity, images can be acquired in time-division order, and these images can be combined to generate high dynamic range images without so-called black crushing and white clipping.

[0115] Furthermore, the light source device 11203 may be configured to supply light in a predetermined wavelength range corresponding to special light observation. In special light observation, for example, so-called narrow-band imaging is performed, in which a predetermined tissue such as blood vessels on the surface of the mucosa is imaged with high contrast by irradiating with narrow-band light compared to the irradiation light used in normal observation (i.e., white light), utilizing the wavelength dependence of light absorption in body tissue. Alternatively, fluorescence observation may be performed in special light observation, in which an image is obtained from fluorescence generated by irradiation with excitation light. In fluorescence observation, fluorescence can be obtained by irradiating body tissue with excitation light and observing the fluorescence from the body tissue (autofluorescence observation), or by locally injecting a reagent such as indocyanine green (ICG) into body tissue and irradiating the body tissue with excitation light corresponding to the fluorescence wavelength of the reagent to obtain a fluorescence image. The light source device 11203 may be configured to supply narrow-band light and / or excitation light corresponding to such special light observation.

[0116] Figure 23 is a block diagram showing an example of the functional configuration of the camera head 11102 and CCU 11201 shown in Figure 22.

[0117] The camera head 11102 includes a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected to each other via a transmission cable 11400 so that they can communicate with each other.

[0118] The lens unit 11401 is an optical system provided at the connection point with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and then incident on the lens unit 11401. The lens unit 11401 is composed of a combination of multiple lenses, including a zoom lens and a focus lens.

[0119] The imaging unit 11402 may consist of one image sensor (a so-called single-chip type) or multiple image sensors (a so-called multi-chip type). If the imaging unit 11402 is configured as a multi-chip type, for example, each image sensor may generate image signals corresponding to RGB, and these may be combined to obtain a color image. Alternatively, the imaging unit 11402 may be configured to have a pair of image sensors for acquiring image signals for the right eye and left eye, respectively, corresponding to 3D (dimensional) display. By performing 3D display, the surgeon 11131 can more accurately grasp the depth of the biological tissue in the surgical area. In addition, if the imaging unit 11402 is configured as a multi-chip type, multiple lens units 11401 may be provided corresponding to each image sensor.

[0120] Furthermore, the imaging unit 11402 does not necessarily have to be located on the camera head 11102. For example, the imaging unit 11402 may be located inside the lens barrel 11101, directly behind the objective lens.

[0121] The drive unit 11403 is composed of actuators and, under control from the camera head control unit 11405, moves the zoom lens and focus lens of the lens unit 11401 along the optical axis by a predetermined distance. This allows the magnification and focus of the image captured by the imaging unit 11402 to be adjusted as appropriate.

[0122] The communication unit 11404 is composed of communication devices for sending and receiving various types of information with the CCU 11201. The communication unit 11404 transmits the image signal obtained from the imaging unit 11402 as RAW data to the CCU 11201 via the transmission cable 11400.

[0123] Furthermore, the communication unit 11404 receives a control signal from the CCU 11201 to control the drive of the camera head 11102 and supplies it to the camera head control unit 11405. The control signal includes information about imaging conditions, such as information to specify the frame rate of the captured image, information to specify the exposure value at the time of imaging, and / or information to specify the magnification and focus of the captured image.

[0124] The imaging conditions such as frame rate, exposure value, magnification, and focus may be specified by the user as appropriate, or they may be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal. In the latter case, the endoscope 11100 is equipped with so-called AE (Auto Exposure), AF (Auto Focus), and AWB (Auto White Balance) functions.

[0125] The camera head control unit 11405 controls the drive of the camera head 11102 based on the control signal received from the CCU 11201 via the communication unit 11404.

[0126] The communication unit 11411 is comprised of a communication device for sending and receiving various types of information with the camera head 11102. The communication unit 11411 receives image signals transmitted from the camera head 11102 via the transmission cable 11400.

[0127] Furthermore, the communication unit 11411 transmits control signals to the camera head 11102 to control the driving of the camera head 11102. Image signals and control signals can be transmitted by telecommunications, optical communications, etc.

[0128] The image processing unit 11412 performs various image processing operations on the image signal, which is RAW data transmitted from the camera head 11102.

[0129] The control unit 11413 performs various controls related to imaging the surgical area, etc., by the endoscope 11100, and the display of the images obtained from imaging the surgical area, etc. For example, the control unit 11413 generates a control signal to control the driving of the camera head 11102.

[0130] Furthermore, the control unit 11413 displays the captured image showing the surgical area, etc., on the display device 11202 based on the image signal processed by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition technologies. For example, the control unit 11413 can recognize surgical instruments such as forceps, specific biological sites, bleeding, mist when using the energy treatment device 11112, etc., by detecting the shape and color of the edges of objects included in the captured image. When the control unit 11413 displays the captured image on the display device 11202, it may use the recognition results to superimpose various surgical support information onto the image of the surgical area. By superimposing the surgical support information and presenting it to the surgeon 11131, the burden on the surgeon 11131 can be reduced, and the surgeon 11131 can proceed with the surgery reliably.

[0131] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 is an electrical signal cable compatible with electrical signal communication, an optical fiber compatible with optical communication, or a composite cable thereof.

[0132] In the illustrated example, communication was performed via a wired connection using a transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may be performed wirelessly.

[0133] The above describes an example of an endoscopic surgical system to which the technology described herein may be applied. The technology described herein can be applied to the imaging unit 11402 of the configuration described above. By applying the technology described herein to the imaging unit 11402, the detection accuracy is improved.

[0134] While an endoscopic surgical system has been described here as an example, the technology described herein may also be applied to other systems, such as microsurgical systems.

[0135] (Examples of application to mobile devices) The technology disclosed herein can be applied to a variety of products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device, such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, robots, construction machinery, or agricultural machinery (tractors).

[0136] Figure 24 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0137] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 24, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0138] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0139] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0140] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0141] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0142] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0143] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0144] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0145] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0146] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 24, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0147] Figure 25 shows an example of the installation position of the imaging unit 12031.

[0148] In Figure 25, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0149] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0150] Figure 25 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0151] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0152] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, etc., that drives autonomously without driver operation, can be performed.

[0153] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0154] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0155] The above describes an example of a mobile object control system to which the technology of this disclosure may be applied. The technology of this disclosure can be applied to the imaging unit 12031 of the configuration described above. Specifically, the light detection device (for example, light detection device 1) according to the above embodiment and its modified form can be applied to the imaging unit 12031. By applying the technology of this disclosure to the imaging unit 12031, high-resolution images with low noise can be obtained, so that high-precision control using the captured images can be performed in the mobile object control system.

[0156] The present disclosure has been described above with reference to embodiments and modifications 1 to 6, as well as application examples, usage examples, and application examples. However, the present technology is not limited to the above embodiments, and various modifications are possible. For example, the above embodiments show examples in which inner lenses (microlenses 22L1, 22L2) are placed in the high-sensitivity pixel P1 and the low-sensitivity pixel P2, respectively, but the technology is not limited to this. For example, the microlens 22L1 of the high-sensitivity pixel P1 may be omitted.

[0157] Furthermore, while the above-described modification 5 shows an example in which the number or shape of microlenses 22L2 arranged in the low-sensitivity pixel P2 is changed according to the type of color filter 25 provided (for example, any of the red filter 25R, green filter 25G, and blue filter 25B), the invention is not limited to this. The number or shape of microlenses 22L2 arranged in the low-sensitivity pixel P2 may be changed according to the position of the low-sensitivity pixel P2 arranged in the pixel array 100A (for example, the central part and the peripheral part of the pixel array 100A). This makes it possible for the light detection device 1F to reduce the image height dependence of light-gathering characteristics such as sensitivity.

[0158] Furthermore, the effects described herein are merely examples and are not limited to those described; other effects may also occur.

[0159] Furthermore, this disclosure can also take the following configuration. According to the technology with the following configuration, the pixel array portion is configured such that the first microlenses arranged in the array portion are mixed with first pixels and second pixels which are different from each other, so that the collection position of incident light is more dispersed in pixels which have a larger number of first microlenses. As a result, it is possible to reduce sensitivity while suppressing the periodic change in sensitivity due to the angle of incidence. Thus, it is possible to achieve both high dynamic range and improved image quality. (1) A light detection device comprising: a semiconductor substrate having a pixel array portion which has opposing first and second surfaces and which has a plurality of pixels including first pixels and second pixels arranged in an array; and a plurality of first microlenses which are arranged in an array portion on the first surface side of the semiconductor substrate, wherein the first pixels and the second pixels have a different number of the first microlenses. (2) The light detection device according to (1) above, wherein the first pixel has one of the first microlenses, and the second pixel has two or more of the first microlenses. (3) The photodetector according to (1) or (2), further comprising a plurality of second microlenses, wherein the plurality of second microlenses are provided on the side of the plurality of first microlenses opposite to the semiconductor substrate side, and one is arranged in each of the plurality of pixels. (4) The photodetector according to any one of (1) to (3), wherein the plurality of first microlenses are arranged in an n row and n column (where n is an integer of 2 or more) in the second pixel. (5) The photodetector according to any one of (1) to (3), wherein the plurality of first microlenses are arranged in an n row and m column (where n is an integer of 1 or more, and m is an integer of 2 or more different from n). (6) The photodetector according to any one of (1) to (3), wherein the plurality of first microlenses are arranged in a concentric circle from the center of the second pixel. (7) The photodetector according to any one of (1) to (6), wherein the plurality of first microlenses are stacked in the thickness direction of the semiconductor substrate in the second pixel.(8) The light detection device according to any one of (1) to (7), further comprising a light-shielding layer having a plurality of openings in its surface, wherein the light-shielding layer is disposed between the first surface of the semiconductor substrate of the second pixel and the plurality of first microlenses. (9) The light detection device according to any one of (1) to (8), further comprising an intermediate layer on the first surface side of the semiconductor substrate in which the plurality of first microlenses are embedded in the layer, wherein the intermediate layer further comprises a light-shielding wall extending between adjacent plurality of pixels. (10) The light detection device according to any one of (1) to (9), wherein the semiconductor substrate includes an inter-pixel separation portion extending from the first surface toward the second surface between adjacent pixels. (11) The light detection device according to (10), wherein the inter-pixel separation portion has light-shielding properties. (12) The light detection device according to any one of (1) to (11), further comprising a light-shielding portion provided on the first surface of the semiconductor substrate and extending between adjacent pixels in a plan view, wherein the light-shielding portion is formed to extend beyond the first pixel toward the second pixel. (13) The light detection device according to any one of (1) to (12), further comprising a color filter, wherein the color filter includes a first filter and a second filter having different transmission wavelengths, and the second pixel provided with the first filter and the second pixel provided with the second filter have different numbers of the first microlenses. (14) The light detection device according to any one of (1) to (13), wherein in the pixel array portion, the first pixels and the second pixels are periodically arranged as the plurality of pixels. (15) The light detection device according to (14), wherein the number of first microlenses arranged in the second pixel varies depending on the position of the second pixel arranged in the pixel array portion.(16) An electronic device including a light detection device, the light detection device having a semiconductor substrate having a pixel array portion having a plurality of pixels including a first pixel and a second pixel arranged in an array, and a plurality of first microlenses arranged in an array in the pixel array portion on the first surface side of the semiconductor substrate, wherein the first pixel and the second pixel have a different number of the first microlenses.

[0160] This application claims priority based on Japanese Patent Application No. 2024-220122, filed with the Japan Patent Office on 16 December 2024, and all contents of that application are incorporated herein by reference.

[0161] Those skilled in the art will understand that various modifications, combinations, subcombinations, and changes can be conceived depending on design requirements and other factors, and that these fall within the scope of the attached claims and their equivalents.

Claims

1. A light detection device comprising: a semiconductor substrate having a first surface and a second surface facing each other and a pixel array portion in which a plurality of pixels including a first pixel and a second pixel are arranged in an array; and a plurality of first microlenses arranged in an array in the pixel array portion on the first surface side of the semiconductor substrate, wherein the first pixel and the second pixel have a different number of the first microlenses.

2. The light detection device according to claim 1, wherein the first pixel has one first microlens, and the second pixel has two or more first microlenses.

3. The photodetector according to claim 1, further comprising a plurality of second microlenses, wherein the plurality of second microlenses are provided on the side of the plurality of first microlenses opposite to the semiconductor substrate side, and one is provided for each of the plurality of pixels.

4. The photodetector according to claim 1, wherein the plurality of first microlenses are arranged in an n x n (where n is an integer of 2 or more) arrangement in the second pixel.

5. The photodetector according to claim 1, wherein the plurality of first microlenses are arranged in the second pixel in an n x m row configuration (where n is an integer of 1 or more, and m is an integer of 2 or more different from n).

6. The photodetector according to claim 1, wherein the plurality of first microlenses are arranged concentrically around the second pixel from the pixel center.

7. The photodetector according to claim 1, wherein the plurality of first microlenses are stacked in the thickness direction of the semiconductor substrate in the second pixel.

8. The light detection device according to claim 1, further comprising a light-shielding layer having a plurality of openings in its plane, wherein the light-shielding layer is disposed between the first surface of the semiconductor substrate of the second pixel and the plurality of first microlenses.

9. The light detection device according to claim 1, further comprising an intermediate layer on the first surface side of the semiconductor substrate, wherein the plurality of first microlenses are embedded in the layer, and the intermediate layer further comprises light-shielding walls extending between adjacent plurality of pixels.

10. The photodetector according to claim 1, wherein the semiconductor substrate includes an inter-pixel separation portion extending from the first surface toward the second surface between adjacent pixels.

11. The light detection device according to claim 10, wherein the inter-pixel separation portion has light-shielding properties.

12. The light detection device according to claim 1, further comprising a light-shielding portion provided on the first surface of the semiconductor substrate and extending between adjacent pixels in a plan view, wherein the light-shielding portion is formed to extend beyond the first pixel towards the second pixel.

13. The photodetector according to claim 1, further comprising a color filter, the color filter comprising a first filter and a second filter having different transmission wavelengths, and the second pixel provided with the first filter and the second pixel provided with the second filter having different numbers of the first microlenses.

14. The photodetector according to claim 1, wherein in the pixel array section, the first pixels and the second pixels are periodically arranged as the plurality of pixels.

15. The photodetector according to claim 14, wherein the number of first microlenses arranged in the second pixel varies depending on the position of the second pixel arranged in the pixel array.

16. An electronic device including a light detection device, the light detection device having a semiconductor substrate having a pixel array portion having opposing first and second surfaces and a plurality of pixels including first pixels and second pixels arranged in an array, and a plurality of first microlenses arranged in an array portion on the first surface side of the semiconductor substrate, wherein the first pixels and second pixels have a different number of the first microlenses.

Images