Solid-state imaging device and recognition system

Abstract

The purpose of the present invention is to make it possible to carry out a more secure authentication. The solid-state imaging device according to an embodiment comprises a plurality of unit pixels (110) arranged in a matrix shape, and signal processing circuits (103A, 103B) for reading a signal from each of the unit pixels. Each of the unit pixels is provided with a first pixel (10) that is positioned on a first surface and that detects light in a first wavelength band, and a second pixel (20) that is positioned on a second surface parallel to the first surface and that detects light in a second wavelength band different from the first wavelength band. The signal processing circuits are each provided with a first conversion circuit (103a) that is connected to the first pixel and the second pixel of each of the unit pixels and that converts an analog signal outputted from each of the first pixels and the second pixels into a digital signal.

Description

[Technical Field]

[0001] The present disclosure relates to a solid-state imaging device and a recognition system. [Background technology]

[0002] In recent years, with the spread of portable devices such as smartphones and tablet terminals, there has been a demand for secure authentication systems. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-21855 [Patent Document 2] Japanese Patent Application Publication No. 2018-125848 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional authentication systems have generally been based on information obtained from a single sensor, leaving room for improvement in security against unauthorized access such as spoofing.

[0005] Therefore, the present disclosure proposes a solid-state imaging device and a recognition system that enable more secure authentication. [Means for solving the problem]

[0006] In order to solve the above problem, a solid-state imaging device according to one embodiment of the present disclosure includes a plurality of unit pixels arranged in a matrix, and a signal processing circuit that reads out signals from each of the unit pixels, each of which includes a first pixel arranged on a first surface that detects light in a first wavelength band, and a second pixel arranged on a second surface parallel to the first surface that detects light in a second wavelength band different from the first wavelength band, and the signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels that converts analog signals output from each of the first pixel and the second pixel into digital signals. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a schematic configuration example of an electronic device equipped with an image sensor according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the functional configuration of a recognition system according to a first embodiment. [Figure 3] 1 is a block diagram showing a schematic configuration example of an image sensor according to a first embodiment. [Figure 4] FIG. 10 is a block diagram showing a schematic configuration example of an image sensor according to a modified example of the first embodiment. [Figure 5] 1 is a schematic diagram illustrating an example of the schematic configuration of a pixel array unit according to a first embodiment. [Figure 6] 1 is a circuit diagram showing a schematic configuration example of a unit pixel according to a first embodiment. [Figure 7] FIG. 4 is a circuit diagram showing a schematic configuration example of a unit pixel according to a first modified example of the first embodiment. [Figure 8] FIG. 10 is a circuit diagram showing a schematic configuration example of a unit pixel according to a second modified example of the first embodiment. [Figure 9] FIG. 10 is a circuit diagram showing a schematic configuration example of a unit pixel according to a third modified example of the first embodiment. [Figure 10] 1 is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to a first embodiment. [Figure 11]FIG. 11 is a diagram showing an example of a planar layout of each layer of the pixel array unit according to the first embodiment. [Figure 12] 3 is a plan view showing an example of wiring of pixel drive lines for RGB pixels according to the first embodiment. FIG. [Figure 13] 3 is a plan view showing an example of wiring of pixel drive lines for IR pixels according to the first embodiment. FIG. [Figure 14] 2A and 2B are diagrams illustrating an example of a stacked structure of the image sensor according to the first embodiment. [Figure 15] 5 is a flowchart showing an example of a recognition operation according to the first embodiment. [Figure 16] FIG. 10 is a schematic diagram illustrating an example of the schematic configuration of a unit pixel according to a second embodiment. [Figure 17] FIG. 10 is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a second embodiment. [Figure 18] FIG. 10 is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to a second embodiment. [Figure 19] 10A and 10B are diagrams illustrating an example of a planar layout of each layer of a pixel array section according to a second embodiment. [Figure 20] 10A and 10B are diagrams illustrating an example of a planar layout of each layer of a pixel array unit according to a modified example of the on-chip lens of the second embodiment. [Figure 21] 10A and 10B are diagrams illustrating an example of a planar layout of each layer of a pixel array section according to a modified example of a color filter array of the second embodiment. [Figure 22] FIG. 10 is a block diagram showing an example of the functional configuration of a recognition system according to a third embodiment. [Figure 23] FIG. 10 is a schematic diagram illustrating an example of the configuration of an electronic device that realizes a recognition system according to a third embodiment. [Figure 24] FIG. 10 is a block diagram showing an example of the configuration of an electronic device that realizes a recognition system according to a third embodiment. [Figure 25] FIG. 10 is a block diagram showing a schematic configuration example of an image sensor according to a third embodiment. [Figure 26] FIG. 11 is a block diagram showing a schematic configuration example of an image sensor according to a modified example of the third embodiment. [Figure 27] FIG. 10 is a schematic diagram illustrating an example of the schematic configuration of a pixel array unit according to a third embodiment. [Figure 28] FIG. 10 is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a third embodiment. [Figure 29] FIG. 13 is a circuit diagram showing a schematic configuration example of a unit pixel according to a first modified example of the third embodiment. [Figure 30] FIG. 13 is a circuit diagram showing a schematic configuration example of a unit pixel according to a second modified example of the third embodiment. [Figure 31] FIG. 13 is a circuit diagram showing a schematic configuration example of a unit pixel according to a third modified example of the third embodiment. [Figure 32] FIG. 11 is a circuit diagram showing a schematic configuration example of a unit pixel according to a fourth modified example of the third embodiment. [Figure 33] FIG. 10 is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to a third embodiment. [Figure 34] 10A and 10B are diagrams illustrating an example of a planar layout of each layer of a pixel array unit according to a third embodiment. [Figure 35] 10 is a flowchart showing an example of a recognition operation according to the third embodiment. [Figure 36] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Figure 37] FIG. 2 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0009] The present disclosure will be described in the following order. 1. First embodiment 1.1 Example of electronic device configuration 1.2 Example of functional configuration of recognition system 1.3 Image sensor configuration example 1.3.1 Image sensor variations 1.4 Example of unit pixel configuration 1.5 Example of unit pixel circuit configuration 1.6 Circuit configuration variations 1.6.1 First variant 1.6.2 Second variant 1.6.3 Third variant 1.7 Example of cross-sectional structure of a unit pixel 1.8 Organic materials 1.9 Planar structure example 1.10 Pixel drive line wiring example 1.11 Example of image sensor stack structure 1.12 Recognition operation example 1.13 Actions and Effects 2. Second embodiment 2.1 Example of unit pixel configuration 2.2 Example of unit pixel circuit configuration 2.3 Example of cross-sectional structure of a unit pixel 2.4 Planar structure example 2.5 Modified on-chip lens 2.6 Modified color filter array 2.7 Actions and Effects 3. Third embodiment 3.1 Example of functional configuration of recognition system 3.2 System configuration example 3.3 Image sensor configuration example 3.3.1 Image sensor variations 3.4 Example of unit pixel configuration 3.5 Example of unit pixel circuit configuration 3.6 Circuit Configuration Variations 3.6.1 First variant 3.6.2 Second variant 3.6.3 Third variant 3.6.4 Fourth Variant 3.7 Example of cross-sectional structure of a unit pixel 3.8 Planar structure example 3.9 Recognition operation example 3.10 Actions and Effects 4. Mobile application examples

[0010] 1. First embodiment First, a solid-state imaging device (hereinafter referred to as an image sensor), an electronic device, and a recognition system according to a first embodiment will be described in detail with reference to the drawings. Note that, in this embodiment, a case where the technology according to this embodiment is applied to a CMOS (Complementary Metal-Oxide Semiconductor) type image sensor is illustrated, but the present invention is not limited thereto, and the technology according to this embodiment can be applied to various sensors equipped with a photoelectric conversion unit, such as a CCD (Charge-Coupled Device) type image sensor, a ToF (Time-of-Flight) sensor, and an EVS (Event Vision Sensor).

[0011] 1.1 Example of electronic device configuration Fig. 1 is a block diagram showing a schematic configuration example of an electronic device equipped with an image sensor according to embodiment 1. As shown in Fig. 1, the electronic device 1 includes, for example, an imaging lens 2, an image sensor 100, a storage unit 3, and a processor 4.

[0012] The imaging lens 2 is an example of an optical system that collects incident light and forms an image on the light-receiving surface of the image sensor 100. The light-receiving surface may be a surface on which photoelectric conversion units in the image sensor 100 are arranged. The image sensor 100 photoelectrically converts the incident light to generate image data. The image sensor 100 also performs predetermined signal processing, such as noise removal and white balance adjustment, on the generated image data.

[0013] The storage unit 3 is configured with, for example, a flash memory, a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like, and records image data input from the image sensor 100, and the like.

[0014] The processor 4 is configured using, for example, a CPU (Central Processing Unit) etc., and may include an application processor that executes an operating system, various application software etc., a GPU (Graphics Processing Unit), a baseband processor etc. The processor 4 performs various processes as necessary on image data input from the image sensor 100, image data read from the storage unit 3 etc., displays the data to the user, and transmits the data to the outside via a predetermined network.

[0015] In addition, the processor 4 can perform various processes such as distance measurement and recognition by integrally processing the color image read out from the RGB pixels 10 described below and the monochrome image (IR image) read out from the IR pixels 20.

[0016] 1.2 Example of functional configuration of recognition system Next, an example of the functional configuration of a recognition system configured using the electronic device 1 will be described. FIG. 2 is a block diagram showing an example of the functional configuration of the recognition system according to the first embodiment. As shown in FIG. 2, the recognition system 70 includes two types of sensor units: an RGB sensor unit 71 and an IR sensor unit 73. The recognition system 70 also includes an RGB image processing unit 72, an IR image processing unit 74, a recognition processing unit 75, and an interface (I / F) unit 76.

[0017] The RGB sensor unit 71 includes, for example, a plurality of pixels (hereinafter referred to as RGB pixels) each having a color filter that transmits the wavelength components of each of the three primary colors RGB, and generates a color image (hereinafter referred to as RGB image) made up of the color components of the three primary colors RGB. Note that, instead of the RGB sensor unit 71, a sensor unit or the like including a plurality of pixels each having a color filter that transmits the wavelength components of each of the three primary colors CMY may be used.

[0018] The IR sensor unit 73 includes, for example, a plurality of pixels (hereinafter referred to as IR pixels) each having an IR filter that transmits infrared (IR) light, and generates a monochrome image (hereinafter referred to as IR image) made up of the color components of IR light. Note that instead of the IR sensor unit 73, a sensor unit or the like including a plurality of pixels each having a color filter that transmits light of one or more other wavelength bands may be used.

[0019] The RGB image processing unit 72 performs predetermined signal processing such as noise removal, white balance adjustment, pixel interpolation, etc. on the RGB image data input from the RGB sensor unit 71 .

[0020] The IR image processing unit 74 performs predetermined signal processing such as noise removal and white balance adjustment on the IR image data input from the IR sensor unit 73 .

[0021] The recognition processing unit 75 uses the RGB image data input from the RGB image processing unit 72 and / or the IR image data input from the IR image processing unit 74 to perform recognition processing of objects and the like present within the angle of view of the RGB sensor unit 71 and / or the IR sensor unit 73. The recognition processing by the recognition processing unit 75 may be recognition processing such as pattern recognition or recognition processing by artificial intelligence (AI). For example, deep learning using a neural network such as a convolution neural network (CNN) or a recurrent neural network (RNN) may be applied to the recognition processing by AI. The recognition processing unit 75 may also perform part of the recognition processing and output the result (intermediate data, etc.).

[0022] The interface unit 76 outputs the recognition results (including intermediate data, etc.) obtained by the recognition processing unit 75 and the image data acquired by the RGB sensor unit 71 and / or the IR sensor unit 73 to an external device, such as the processor 4 and / or the memory unit 3.

[0023] The RGB image processing unit 72 may perform object region determination on the RGB image data and input information such as an address specifying a region of interest (ROI) obtained as a result of the determination (hereinafter simply referred to as ROI information) to the IR sensor unit 73 and / or the IR image processing unit 74. In response to this, the IR sensor unit 73 may operate to acquire IR image data of a region corresponding to the ROI information input from the RGB image processing unit 72. Alternatively, the IR image processing unit 74 may perform processing such as trimming of the region corresponding to the ROI information input from the RGB image processing unit 72 on the IR image data input from the IR sensor unit 73.

[0024] 1.3 Image sensor configuration example Fig. 3 is a block diagram showing a schematic configuration example of an image sensor according to the first embodiment. As shown in Fig. 3, the image sensor 100 according to this embodiment includes, for example, a pixel array unit 101, an RGB pixel drive circuit 102A, an IR pixel drive circuit 102B, an RGB signal processing circuit 103A, an IR signal processing circuit 103B, a column drive circuit 104, a system control circuit 105, an RGB data processing unit 108A, and an IR data processing unit 108B.

[0025] The pixel array unit 101, the RGB pixel drive circuit 102A, the RGB signal processing circuit 103A, the column drive circuit 104, and the system control circuit 105 constitute, for example, the RGB sensor unit 71 in FIG. 2, and the pixel array unit 101, the IR pixel drive circuit 102B, the IR signal processing circuit 103B, the column drive circuit 104, and the system control circuit 105 constitute, for example, the IR sensor unit 73 in FIG. 2. The RGB signal processing circuit 103A and the RGB data processing unit 108A constitute, for example, the RGB image processing unit 72 in FIG. 2, and the IR signal processing circuit 103B and the IR data processing unit 108B constitute, for example, the IR image processing unit 74 in FIG. 2. The recognition processing unit 75 in FIG. 2 may be implemented by the processor 4 alone, or may be implemented by linking the RGB data processing unit 108A and the IR data processing unit 108B with the processor 4, or may be implemented by linking the RGB data processing unit 108A and the IR data processing unit 108B with each other.

[0026] The pixel array section 101 has a configuration in which unit pixels 110 are arranged in row and column directions, i.e., in a two-dimensional lattice pattern (also called a matrix pattern). Here, the row direction refers to the direction in which pixels in a pixel row are arranged (the horizontal direction in the drawing), and the column direction refers to the direction in which pixels in a pixel column are arranged (the vertical direction in the drawing).

[0027] Each unit pixel 110 includes an RGB pixel 10 and an IR pixel 20. In the following description, when there is no need to distinguish between the RGB pixels 10 and the IR pixels 20, they may be simply referred to as pixels. Details of the specific circuit configuration and pixel structure of the unit pixel 110 will be described later, but each of the RGB pixels 10 and the IR pixel 20 includes a photoelectric conversion unit that generates and accumulates charge according to the amount of light received, and generates a pixel signal with a voltage according to the amount of incident light.

[0028] In the pixel array unit 101, pixel drive lines LD1 and LD2 are wired in the row direction for each pixel row, and vertical signal lines VSL1 and VSL2 are wired in the column direction for each pixel column, for a matrix-like pixel arrangement. For example, the pixel drive line LD1 is connected to the RGB pixels 10 in each row, and the pixel drive line LD2 is connected to the IR pixels 20 in each row. On the other hand, for example, the vertical signal line VSL1 is connected to the RGB pixels 10 in each column, and the vertical signal line VSL2 is connected to the IR pixels 20 in each column. However, this is not limited thereto, and the pixel drive lines LD1 and LD2 may be wired orthogonal to each other. Similarly, the vertical signal lines VSL1 and VSL2 may be wired orthogonal to each other. For example, the pixel drive line LD1 may be wired in the row direction, the pixel drive line LD2 may be wired in the column direction, the vertical signal line VSL1 may be wired in the column direction, and the vertical signal line VSL2 may be wired in the row direction.

[0029] The pixel drive line LD1 transmits a control signal for driving the RGB pixels 10 when pixel signals are read out. The pixel drive line LD2 transmits a control signal for driving the IR pixels 20 when pixel signals are read out. In FIG. 3, the pixel drive lines LD1 and LD2 are shown as one wiring each, but the number of wirings is not limited to one. One end of the pixel drive line LD1 is connected to an output terminal corresponding to each row of the RGB pixel drive circuit 102A, and one end of the pixel drive line LD2 is connected to an output terminal corresponding to each row of the IR pixel drive circuit 102B.

[0030] The RGB pixel drive circuit 102A and the IR pixel drive circuit 102B are each composed of a shift register, an address decoder, etc., and drive each pixel of the pixel array section 101 simultaneously or row by row, etc. That is, the RGB pixel drive circuit 102A, together with the system control circuit 105 that controls the RGB pixel drive circuit 102A, constitutes a drive unit that controls the operation of each RGB pixel 10 of the pixel array section 101, and the IR pixel drive circuit 102B, together with the system control circuit 105 that controls the IR pixel drive circuit 102B, constitutes a drive unit that controls the operation of each IR pixel 20 of the pixel array section 101. Although the specific configurations of the RGB pixel drive circuit 102A and the IR pixel drive circuit 102B are not shown in the figures, they generally have two scan systems: a readout scan system and a sweep scan system.

[0031] The readout scanning system sequentially selects and scans each pixel in the pixel array unit 101 row by row to read out a signal from each pixel. The pixel signal read out from each pixel is an analog signal. The sweep scanning system performs sweep scanning on the readout row to be read out by the readout scanning system, prior to the readout scanning by the exposure time.

[0032] The sweep scanning by this sweep scanning system sweeps out unnecessary charges from the photoelectric conversion units of each pixel in the readout row, thereby resetting the photoelectric conversion units. Then, by sweeping out (resetting) the unnecessary charges with this sweep scanning system, a so-called electronic shutter operation is performed. Here, the electronic shutter operation refers to the operation of discarding the charges in the photoelectric conversion units and starting a new exposure (starting the accumulation of charges).

[0033] The signal read by the readout scanning system corresponds to the amount of light received since the previous readout operation or electronic shutter operation. The period from the readout timing of the previous readout operation or the sweep timing of the electronic shutter operation to the readout timing of the current readout operation is the charge accumulation period (also called the exposure period) in each pixel.

[0034] The pixel signals output from each RGB pixel 10 in a pixel row selected and scanned by the RGB pixel drive circuit 102A are input to the RGB signal processing circuit 103A for each pixel column through each vertical signal line VSL1. The RGB signal processing circuit 103A performs predetermined signal processing on the pixel signals output from each RGB pixel 10 in the selected row through the vertical signal line VSL1 for each pixel column in the pixel array unit 101, and temporarily stores the pixel signals after signal processing.

[0035] Similarly, pixel signals output from each IR pixel 20 in a pixel row selected and scanned by the IR pixel drive circuit 102B are input to the IR signal processing circuit 103B through each vertical signal line VSL2 for each pixel column. The IR signal processing circuit 103B performs predetermined signal processing on the pixel signals output from each IR pixel 20 in the selected row through the vertical signal line VSL2 for each pixel column in the pixel array unit 101, and temporarily stores the pixel signals after signal processing.

[0036] Specifically, each of the RGB signal processing circuit 103A and the IR signal processing circuit 103B performs at least noise removal processing, such as correlated double sampling (CDS) processing or double data sampling (DDS) processing, as signal processing. For example, CDS processing removes pixel-specific fixed pattern noise, such as reset noise and threshold variations in the amplification transistors in the pixels. Each of the RGB signal processing circuit 103A and the IR signal processing circuit 103B also has, for example, an AD (analog-to-digital) conversion function, and converts analog pixel signals read from the photoelectric conversion units into digital signals and outputs them.

[0037] The column driving circuit 104 is configured with a shift register, an address decoder, etc., and sequentially selects readout circuits (hereinafter referred to as pixel circuits) corresponding to pixel columns in the RGB signal processing circuit 103 A and the IR signal processing circuit 103 B. Through selective scanning by this column driving circuit 104, pixel signals processed for each pixel circuit in the RGB signal processing circuit 103 A and the IR signal processing circuit 103 B are sequentially output.

[0038] The system control circuit 105 is composed of a timing generator that generates various timing signals, and controls the driving of the RGB pixel driving circuit 102A, the IR pixel driving circuit 102B, the RGB signal processing circuit 103A, the IR signal processing circuit 103B, and the column driving circuit 104 based on the various timings generated by the timing generator.

[0039] The RGB data processing unit 108A and the IR data processing unit 108B each have at least an arithmetic processing function, and perform various signal processing such as arithmetic processing on the image signals output from the RGB signal processing circuit 103A or the IR signal processing circuit 103B.

[0040] The image data output from the RGB data processing unit 108A or the IR data processing unit 108B may be subjected to predetermined processing, for example, in a processor 4 in the electronic device 1 equipped with the image sensor 100, or may be transmitted to the outside via a predetermined network.

[0041] The image sensor 100 may also include a storage unit for temporarily storing data necessary for signal processing in the RGB data processing unit 108A and the IR data processing unit 108B, and data processed by one or more of the RGB signal processing circuit 103A, the IR signal processing circuit 103B, the RGB data processing unit 108A, and the IR data processing unit 108B.

[0042] 1.3.1 Image sensor variations Fig. 4 is a block diagram showing a schematic configuration example of an image sensor according to a modified example of the first embodiment. While Fig. 3 above illustrates a case where separate signal processing circuits (RGB signal processing circuit 103A and IR signal processing circuit 103B) are provided for the RGB pixels 10 and the IR pixels 20, the present invention is not limited to this. As illustrated in Fig. 4, a common signal processing circuit 103 can be provided for the RGB pixels 10 and the IR pixels 20. In this case, the pixel signals from the RGB pixels 10 and the pixel signals from the IR pixels 20 can be read out in a time-division manner or in parallel.

[0043] 1.4 Example of unit pixel configuration Next, a configuration example of the unit pixel 110 will be described. Note that here, a case where the unit pixel 110 includes an RGB pixel 10 for acquiring an RGB image of the three RGB primary colors and an IR pixel 20 for acquiring an IR image of infrared (IR) light will be exemplified. Note that in FIG. 5 and below, when there is no need to distinguish between the color filters 31r, 31g, and 31b that transmit the light of each color component that makes up the three RGB primary colors, the reference numeral 31 is used.

[0044] FIG. 5 is a schematic diagram illustrating an example of a schematic configuration of a pixel array unit according to the first embodiment. As shown in FIG. 5, the pixel array unit 101 has a configuration in which unit pixels 110, each of which includes an RGB pixel 10 and an IR pixel 20, are arranged in a two-dimensional lattice pattern along the light incidence direction. That is, in this embodiment, the RGB pixels 10 and the IR pixels 20 are positioned perpendicular to the arrangement direction (planar direction) of the unit pixels 110, and light transmitted through an RGB pixel 10 located upstream in the optical path of incident light is incident on an IR pixel 20 located downstream of the RGB pixel 10. With this configuration, the photoelectric conversion unit PD2 of the IR pixel 20 is disposed on the surface of the photoelectric conversion unit PD1 of the RGB pixel 10 opposite to the incident surface of the incident light. As a result, in this embodiment, the optical axes of the incident light of the RGB pixels 10 and the IR pixels 20 arranged along the light incidence direction are aligned or substantially aligned.

[0045] In this embodiment, the photoelectric conversion unit PD1 constituting the RGB pixel 10 is made of an organic material, and the photoelectric conversion unit PD2 constituting the IR pixel 20 is made of a semiconductor material such as silicon, but this is not limiting. For example, both the photoelectric conversion unit PD1 and the photoelectric conversion unit PD2 may be made of a semiconductor material, or both the photoelectric conversion unit PD1 and the photoelectric conversion unit PD2 may be made of an organic material, or the photoelectric conversion unit PD1 may be made of a semiconductor material and the photoelectric conversion unit PD2 may be made of an organic material. Alternatively, at least one of the photoelectric conversion unit PD1 and the photoelectric conversion unit PD2 may be made of a photoelectric conversion material that is different from the organic material and the semiconductor material.

[0046] 1.5 Example of unit pixel circuit configuration Next, an example of the circuit configuration of the unit pixel 110 will be described. Fig. 6 is a circuit diagram showing a schematic configuration example of the unit pixel according to the first embodiment. As shown in Fig. 6, the unit pixel 110 includes one RGB pixel 10 and one IR pixel 20.

[0047] (RGB pixels 10) The RGB pixel 10 includes, for example, a photoelectric conversion unit PD1, a transfer gate 11, a floating diffusion region FD1, a reset transistor 12, an amplification transistor 13, and a selection transistor 14.

[0048] A selection control line included in pixel drive line LD1 is connected to the gate of selection transistor 14, a reset control line included in pixel drive line LD1 is connected to the gate of reset transistor 12, and a transfer control line included in pixel drive line LD1 is connected to a storage electrode (see storage electrode 37 in FIG. 10 described later) of transfer gate 11. In addition, a vertical signal line VSL1, one end of which is connected to RGB signal processing circuit 103A, is connected to the drain of amplification transistor 13 via selection transistor 14.

[0049] In the following description, the reset transistor 12, the amplification transistor 13, and the selection transistor 14 are collectively referred to as a pixel circuit, which may include a floating diffusion region FD1 and / or a transfer gate 11.

[0050] The photoelectric conversion unit PD1 is made of, for example, an organic material and performs photoelectric conversion on incident light. The transfer gate 11 transfers the charge generated in the photoelectric conversion unit PD1. The floating diffusion region FD1 accumulates the charge transferred by the transfer gate 11. The amplification transistor 13 generates a pixel signal on the vertical signal line VSL1 with a voltage value corresponding to the charge accumulated in the floating diffusion region FD1. The reset transistor 12 releases the charge accumulated in the floating diffusion region FD1. The selection transistor 14 selects the RGB pixel 10 to be read.

[0051] The anode of the photoelectric conversion unit PD1 is grounded, and the cathode is connected to a transfer gate 11. The transfer gate 11 will be described in detail later with reference to FIG. 10 , and includes, for example, a storage electrode 37 and a readout electrode 36. During exposure, a voltage is applied to the storage electrode 37 via a transfer control line to collect charges generated in the photoelectric conversion unit PD1 in the semiconductor layer 35 near the storage electrode 37. During readout, a voltage is applied to the storage electrode 37 via a transfer control line to cause the charges collected in the semiconductor layer 35 near the storage electrode 37 to flow out via the readout electrode 36.

[0052] The charge flowing out through the readout electrode 36 is accumulated in a floating diffusion region FD1 formed by a wiring structure connecting the readout electrode 36, the source of the reset transistor 12, and the gate of the amplification transistor 13. The drain of the reset transistor 12 may be connected to a power supply line that supplies, for example, a power supply voltage VDD or a reset voltage lower than the power supply voltage VDD.

[0053] The source of the amplification transistor 13 may be connected to a power supply line via, for example, a constant current circuit (not shown) etc. The drain of the amplification transistor 13 is connected to the source of the selection transistor 14, and the drain of the selection transistor 14 is connected to the vertical signal line VSL1.

[0054] The floating diffusion region FD1 converts the accumulated charge into a voltage having a value corresponding to the amount of charge. The floating diffusion region FD1 may be, for example, a capacitance to ground. However, the present invention is not limited to this. The floating diffusion region FD1 may also be a capacitance added by intentionally connecting a capacitor or the like to a node connecting the drain of the transfer gate 11, the source of the reset transistor 12, and the gate of the amplifier transistor 13.

[0055] The vertical signal line VSL1 is connected to an AD (Analog-to-Digital) conversion circuit 103a provided for each column (i.e., for each vertical signal line VSL1) in the RGB signal processing circuit 103A. The AD conversion circuit 103a includes, for example, a comparator and a counter, and converts the analog pixel signal into a digital pixel signal by comparing a reference voltage, such as a single slope or ramp voltage, input from an external reference voltage generation circuit (DAC (Digital-to-Analog Converter)), with the pixel signal appearing on the vertical signal line VSL1. The AD conversion circuit 103a may include, for example, a CDS (Correlated Double Sampling) circuit or the like, and may be configured to reduce kTC noise and the like.

[0056] (IR pixels 20) The IR pixel 20 includes, for example, a photoelectric conversion unit PD2, a transfer transistor 21, a floating diffusion region FD2, a reset transistor 22, an amplification transistor 23, a selection transistor 24, and a discharge transistor 25. That is, in the IR pixel 20, the transfer gate 11 in the RGB pixel 10 is replaced with the transfer transistor 21, and a discharge transistor 25 is added.

[0057] The connection relationship between the floating diffusion region FD2, the reset transistor 22, and the amplifier transistor 23 and the transfer transistor 21 may be similar to the connection relationship between the floating diffusion region FD1, the reset transistor 12, and the amplifier transistor 13 and the transfer gate 11 in the RGB pixel 10. In addition, the connection relationship between the amplifier transistor 23, the selection transistor 24, and the vertical signal line VSL2 may be similar to the connection relationship between the amplifier transistor 13, the selection transistor 14, and the vertical signal line VSL1 in the RGB pixel 10.

[0058] The source of the transfer transistor 21 is connected to, for example, the cathode of the photoelectric conversion unit PD2, and the drain is connected to the floating diffusion region FD2. In addition, the gate of the transfer transistor 21 is connected to a transfer control line included in the pixel drive line LD2.

[0059] The source of the discharge transistor 25 may be connected to, for example, the cathode of the photoelectric conversion unit PD2, and the drain may be connected to a power supply line that supplies a power supply voltage VDD or a reset voltage lower than the power supply voltage VDD. In addition, the gate of the discharge transistor 25 is connected to a discharge control line included in the pixel drive line LD2.

[0060] In the following description, the reset transistor 22, the amplification transistor 23, and the selection transistor 24 are collectively referred to as a pixel circuit, which may include one or more of a floating diffusion region FD2, a transfer transistor 21, and a drain transistor 25.

[0061] The photoelectric conversion unit PD2 is made of, for example, a semiconductor material and performs photoelectric conversion on incident light. The transfer transistor 21 transfers the charges generated in the photoelectric conversion unit PD2. The floating diffusion region FD2 accumulates the charges transferred by the transfer transistor 21. The amplification transistor 23 generates a pixel signal on the vertical signal line VSL2 with a voltage value corresponding to the charges accumulated in the floating diffusion region FD2. The reset transistor 22 releases the charges accumulated in the floating diffusion region FD2. The selection transistor 24 selects the IR pixel 20 to be read out.

[0062] The anode of the photoelectric conversion unit PD2 is grounded, and the cathode is connected to the transfer transistor 21. The drain of the transfer transistor 21 is connected to the source of the reset transistor 22 and the gate of the amplification transistor 23, and the wiring structure connecting these constitutes the floating diffusion layer FD2. Charges flowing out from the photoelectric conversion unit PD2 via the transfer transistor 21 are accumulated in the floating diffusion region FD2.

[0063] The floating diffusion region FD2 converts the accumulated charge into a voltage having a value corresponding to the amount of charge. The floating diffusion region FD2 may be, for example, a capacitance to ground. However, the present invention is not limited to this. The floating diffusion region FD2 may also be a capacitance added by intentionally connecting a capacitor or the like to a node connecting the drain of the transfer transistor 21, the source of the reset transistor 22, and the gate of the amplifier transistor 23.

[0064] The discharge transistor 25 is turned on when discharging the charge accumulated in the photoelectric conversion unit PD2 and resetting the photoelectric conversion unit PD2, causing the charge accumulated in the photoelectric conversion unit PD2 to flow out to the power supply line via the discharge transistor 25, and resetting the photoelectric conversion unit PD2 to an unexposed state.

[0065] The vertical signal line VSL2, like the vertical signal line VSL1, is connected to an AD conversion circuit 103a provided for each column (that is, for each vertical signal line VSL2) in the IR signal processing circuit 103B.

[0066] 1.6 Circuit configuration variations Next, several examples of modifications of the circuit configuration of the unit pixel 110 shown in FIG. 6 will be described.

[0067] 1.6.1 First variant FIG. 7 is a circuit diagram showing a schematic configuration example of a unit pixel according to a first modified example of the first embodiment. As shown in FIG. 7, the unit pixel 110-1 has the same configuration as the unit pixel 110 shown in FIG. 6, except that the vertical signal lines VSL1 and VSL2 are connected to a common AD conversion circuit 103a. Therefore, in the first modified example, a switch circuit 131 is provided to switch the vertical signal line connected to the AD conversion circuit 103a between the vertical signal lines VSL1 and VSL2. The switch circuit 131 may be included in, for example, the RGB signal processing circuit 103A and the IR signal processing circuit 103B or the common signal processing circuit 103. The switch circuit 131 may be provided on the same semiconductor substrate as the pixel circuits of the RGB pixels 10 and / or the IR pixels 20, on a semiconductor substrate on which the signal processing circuits are arranged, or on a different semiconductor substrate. Furthermore, the control signal for controlling the switch circuit 131 may be supplied from the RGB pixel drive circuit 102A or the IR pixel drive circuit 102B, from the column drive circuit 104, or from another component (such as the processor 4 in FIG. 1).

[0068] With this configuration, the RGB signal processing circuit 103A and the IR signal processing circuit 103B can be replaced with a common signal processing circuit, thereby making it possible to reduce the circuit scale, thereby improving area efficiency and making it possible to reduce the size of the image sensor 100 and increase its resolution.

[0069] 1.6.2 Second variant 8 is a circuit diagram showing a schematic configuration example of a unit pixel according to a second modified example of the first embodiment. As shown in FIG. 8, the unit pixel 110-2 has the same configuration as the unit pixel 110 shown in FIG. 6, but is configured so that each of the vertical signal lines VSL1 and VSL2 can be connected to one of two AD conversion circuits 103a. Therefore, the second modified example includes a switch circuit 132 that switches the vertical signal line connected to the AD conversion circuit 103a between the vertical signal lines VSL1 and VSL2, and a switch circuit 133 that switches the vertical signal line connected to the AD conversion circuit 103a between the vertical signal lines VSL1 and VSL2. The switch circuits 132 and 133 may be included in, for example, the RGB signal processing circuit 103A and the IR signal processing circuit 103B, or the common signal processing circuit 103. The switch circuits 132 and 133 may be provided on the same semiconductor substrate as the pixel circuits of the RGB pixels 10 and / or IR pixels 20, or on the same semiconductor substrate on which the RGB signal processing circuit 103A and the IR signal processing circuit 103B are arranged, or on a different semiconductor substrate. Control signals for controlling the switch circuits 132 and 133 may be supplied from the RGB pixel drive circuit 102A or the IR pixel drive circuit 102B, from the column drive circuit 104, or from another component (such as the processor 4 in FIG. 1).

[0070] With this configuration, it is possible to select from multiple (two in this example) AD conversion circuits 103a to be used for each column, thereby making it possible to suppress degradation of image quality due to noise such as streaking.

[0071] 1.6.3 Third variant 9 is a circuit diagram showing a schematic configuration example of a unit pixel according to a third modified example of the first embodiment. As shown in FIG. 9, the unit pixel 110-3 has a configuration similar to that of the unit pixel 110-2 shown in FIG. 8 for the RGB pixel 10, but has a so-called pixel-sharing circuit structure in which a floating diffusion region FD1, a reset transistor 12, an amplifier transistor 13, and a selection transistor 14 are shared among a plurality of RGB pixels 10-1 to 10-N (N is an integer equal to or greater than 2). Similarly, the unit pixel 110-3 has a configuration similar to that of the unit pixel 110-2 shown in FIG. 8 for the IR pixel 20, but has a so-called pixel-sharing circuit structure in which a floating diffusion region FD2, a reset transistor 22, an amplifier transistor 23, and a selection transistor 24 are shared among a plurality of IR pixels 20-1 to 20-N. The number of the RGB pixels 10-1 to 10-N does not necessarily have to match the number of the IR pixels 20-1 to 20-N.

[0072] With this configuration, it is possible to switch between HDR (High Dynamic Range) readout and LDR (Low Dynamic Range) readout depending on the situation, thereby suppressing deterioration of image quality at times of low illumination and high illumination. Note that, although this explanation has been given based on the second modified example explained using Fig. 8, it is not limited to this, and it is also possible to use the unit pixel 110 and unit pixel 110-1 shown in Fig. 6 and Fig. 7 as a base.

[0073] 1.7 Example of cross-sectional structure of a unit pixel Next, an example of the cross-sectional structure of the image sensor 100 according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a cross-sectional view showing an example of the cross-sectional structure of the image sensor according to the first embodiment. Here, the example of the cross-sectional structure will be described, focusing on a semiconductor chip on which photoelectric conversion units PD1 and PD2 in the unit pixel 110 are formed.

[0074] In addition, the following description will exemplify a so-called back-illuminated cross-sectional structure in which the light incident surface is the back side of the semiconductor substrate 50 (the side opposite to the element formation surface), but the present invention is not limited to this and may also be a so-called front-illuminated cross-sectional structure in which the light incident surface is the front side (element formation surface side) of the semiconductor substrate 50. Furthermore, in this description, an example is given in which an organic material is used for the photoelectric conversion unit PD1 of the RGB pixel 10, but as described above, one or both of an organic material and a semiconductor material (also called an inorganic material) may be used for the photoelectric conversion material of each of the photoelectric conversion units PD1 and PD2.

[0075] In addition, when semiconductor materials are used for both the photoelectric conversion material of the photoelectric conversion unit PD1 and the photoelectric conversion material of the photoelectric conversion unit PD2, the image sensor 100 may have a cross-sectional structure in which the photoelectric conversion unit PD1 and the photoelectric conversion unit PD2 are fabricated on the same semiconductor substrate 50, or may have a cross-sectional structure in which a semiconductor substrate in which the photoelectric conversion unit PD1 is fabricated and a semiconductor substrate in which the photoelectric conversion unit PD2 is fabricated are bonded together, or may have a cross-sectional structure in which one of the photoelectric conversion units PD1 and PD2 is fabricated in the semiconductor substrate 50 and the other is fabricated in a semiconductor layer formed on the back or front surface of the semiconductor substrate 50.

[0076] 10, in this embodiment, a photoelectric conversion unit PD2 of an IR pixel 20 is formed on a semiconductor substrate 50, and a photoelectric conversion unit PD1 of an RGB pixel 10 is provided on the back surface (the surface opposite to the element formation surface) of the semiconductor substrate 50. For convenience of explanation, in FIG. 10, the back surface of the semiconductor substrate 50 is positioned at the top of the page, and the front surface is positioned at the bottom.

[0077] The semiconductor substrate 50 may be made of a semiconductor material such as silicon (Si), but is not limited to this, and various semiconductor materials may be used, including compound semiconductors such as GaAs, InGaAs, InP, AlGaAs, InGaP, AlGaInP, and InGaAsP.

[0078] (RGB pixels 10) The photoelectric conversion unit PD1 of the RGB pixel 10 is provided on the back surface side of the semiconductor substrate 50, with an insulating layer 53 sandwiched therebetween. The photoelectric conversion unit PD1 includes, for example, a photoelectric conversion film 34 made of an organic material, and a transparent electrode 33 and a semiconductor layer 35 arranged to sandwich the photoelectric conversion film 34. The transparent electrode 33 provided on the upper side of the page (hereinafter, the upper side in the page will be referred to as the upper surface side and the lower side will be referred to as the lower surface side) of the photoelectric conversion film 34 functions, for example, as an anode of the photoelectric conversion unit PD1, and the semiconductor layer 35 provided on the lower surface side functions as a cathode of the photoelectric conversion unit PD1.

[0079] The semiconductor layer 35, which functions as a cathode, is electrically connected to a read electrode 36 formed in the insulating layer 53. The read electrode 36 is electrically drawn out to the front (lower) surface side of the semiconductor substrate 50 by connecting to wiring 61, 62, 63, and 64 that penetrate the insulating layer 53 and the semiconductor substrate 50. Although not shown in FIG. 10, the wiring 64 is electrically connected to the floating diffusion region FD1 shown in FIG.

[0080] A storage electrode 37 is provided on the lower surface side of the semiconductor layer 35 that functions as the cathode, with an insulating layer 53 sandwiched therebetween. Although not shown in Fig. 10, the storage electrode 37 is connected to a transfer control line in the pixel drive line LD1, and as described above, during exposure, a voltage is applied to collect the charges generated in the photoelectric conversion unit PD1 in the semiconductor layer 35 near the storage electrode 37, and during readout, a voltage is applied to cause the charges collected in the semiconductor layer 35 near the storage electrode 37 to flow out via the readout electrode 36.

[0081] The readout electrode 36 and the storage electrode 37 may be transparent conductive films, similar to the transparent electrode 33. For example, a transparent conductive film such as indium tin oxide (ITO) or zinc oxide (IZO) may be used for the transparent electrode 33, readout electrode 36, and storage electrode 37. However, the present invention is not limited to these, and various conductive films may be used as long as they are conductive films that can transmit light in the wavelength band that is the detection target of the photoelectric conversion unit PD2.

[0082] Furthermore, a transparent semiconductor layer such as IGZO may be used for the semiconductor layer 35. However, the present invention is not limited to this, and various semiconductor layers may be used as long as they are capable of transmitting light in the wavelength band that is the detection target of the photoelectric conversion unit PD2.

[0083] Furthermore, the insulating layer 53 may be made of an insulating film such as a silicon oxide film (SiO2) or a silicon nitride film (SiN), but is not limited to these, and various insulating films may be used as long as they are capable of transmitting light in the wavelength band that is the detection target of the photoelectric conversion unit PD2.

[0084] A color filter 31 is provided on the upper surface of the transparent electrode 33, which functions as an anode, with a sealing film 32 sandwiched therebetween. The sealing film 32 is made of an insulating material such as silicon nitride (SiN), and may contain atoms of aluminum (Al) or titanium (Ti) to prevent these atoms from diffusing from the transparent electrode 33.

[0085] The arrangement of the color filters 31 will be described later, but for example, a color filter 31 that selectively transmits light of a specific wavelength component is provided for one RGB pixel 10. However, if monochrome pixels that acquire luminance information are provided instead of RGB pixels 10 that acquire color information, the color filters 31 may be omitted.

[0086] (IR pixels 20) The photoelectric conversion unit PD2 of the IR pixel 20 includes, for example, a p-type semiconductor region 43 formed in a p-well region 42 in the semiconductor substrate 50, and an n-type semiconductor region 44 formed near the center of the p-type semiconductor region 43. The n-type semiconductor region 44 functions, for example, as a charge accumulation region that accumulates charges (electrons) generated by photoelectric conversion, and the p-type semiconductor region 43 functions as a region that forms a potential gradient for collecting the charges generated by photoelectric conversion within the n-type semiconductor region 44.

[0087] An IR filter 41 that selectively transmits IR light is disposed on the light incident surface side of the photoelectric conversion unit PD2. The IR filter 41 may be disposed, for example, in an insulating layer 53 provided on the back surface side of the semiconductor substrate 50. By disposing the IR filter 41 on the light incident surface of the photoelectric conversion unit PD2, it is possible to suppress the incidence of visible light on the photoelectric conversion unit PD2, thereby improving the S / N ratio of IR light relative to visible light. This makes it possible to obtain more accurate detection results for IR light.

[0088] In order to suppress reflection of incident light (IR light in this example), for example, a fine uneven structure is provided on the light incident surface of the semiconductor substrate 50. This uneven structure may be a structure known as a moth-eye structure, or may be an uneven structure having a size and pitch different from that of the moth-eye structure.

[0089] A vertical transistor 45 functioning as a transfer transistor 21 and a floating diffusion region FD2 functioning as a charge storage section are provided on the surface (lower surface in the drawing) side of the semiconductor substrate 50, i.e., the element formation surface side. The gate electrode of the vertical transistor 45 reaches from the surface of the semiconductor substrate 50 to the n-type semiconductor region 44, and is connected to the IR pixel drive circuit 102B via wiring 65 and 66 (part of the transfer control line of the pixel drive line LD2) formed in the interlayer insulating film 56.

[0090] The charge flowing out through the vertical transistor 45 is accumulated in the floating diffusion region FD2. The floating diffusion region FD2 is connected to the source of the reset transistor 22 and the gate of the amplifier transistor 23 via wirings 67 and 68 formed in the interlayer insulating film 56. The reset transistor 22, the amplifier transistor 23, and the select transistor 24 may be provided on the element formation surface of the semiconductor substrate 50, or may be provided on a semiconductor substrate different from the semiconductor substrate 50.

[0091] 10 illustrates an example in which two vertical transistors 45 (transfer transistors 21) are provided for one photoelectric conversion unit PD2, but the present invention is not limited to this and one vertical transistor 45 or three or more vertical transistors 45 may be provided. Similarly, although the present invention illustrates an example in which two floating diffusion regions FD2 are provided for one photoelectric conversion unit PD2, the present invention is not limited to this and one floating diffusion region FD2 or three or more floating diffusion regions FD2 may be provided.

[0092] (pixel isolation structure) The semiconductor substrate 50 is provided with pixel separators 54 that electrically separate the unit pixels 110, and a photoelectric conversion unit PD2 is provided in each of the regions partitioned by the pixel separators 54. For example, when the image sensor 100 is viewed from the rear surface (top surface in the figure) of the semiconductor substrate 50, the pixel separators 54 have, for example, a lattice shape interposed between the unit pixels 110, and each photoelectric conversion unit PD2 is formed in each of the regions partitioned by the pixel separators 54.

[0093] The pixel separator 54 may be formed using a light-reflecting reflective film made of, for example, tungsten (W) or aluminum (Al). This allows incident light that enters the photoelectric conversion unit PD2 to be reflected by the pixel separator 54, thereby lengthening the optical path length of the incident light within the photoelectric conversion unit PD2. In addition, by forming the pixel separator 54 into a light-reflecting structure, it is possible to reduce light leakage into adjacent pixels, thereby further improving image quality and distance measurement accuracy. Note that the pixel separator 54 may be formed using a light-reflecting structure without being limited to a structure using a reflective film. For example, the pixel separator 54 may be formed using a material with a refractive index different from that of the semiconductor substrate 50.

[0094] For example, a fixed charge film 55 is provided between the semiconductor substrate 50 and the pixel separation portion 54. The fixed charge film 55 is formed using a high dielectric material having a negative fixed charge so that a positive charge (hole) accumulation region is formed at the interface with the semiconductor substrate 50, thereby suppressing the generation of dark current. Since the fixed charge film 55 is formed to have a negative fixed charge, an electric field is applied to the interface with the semiconductor substrate 138 due to the negative fixed charge, and a positive charge (hole) accumulation region is formed.

[0095] The fixed charge film 55 can be formed of, for example, a hafnium oxide film (HfO2 film). The fixed charge film 55 can also be formed so as to contain at least one of oxides of other elements, such as hafnium, zirconium, aluminum, tantalum, titanium, magnesium, yttrium, and lanthanoid elements.

[0096] Note that FIG. 10 illustrates an example in which the pixel isolation portion 54 has a so-called FTI (Full Trench Isolation) structure in which the pixel isolation portion 54 reaches from the front surface to the back surface of the semiconductor substrate 50, but is not limited to this. For example, it is possible to adopt various element isolation structures, such as a so-called DTI (Deep Trench Isolation) structure in which the pixel isolation portion 54 is formed from the back surface or front surface of the semiconductor substrate 50 to near the middle of the semiconductor substrate 50.

[0097] (Eye correction) A planarization film 52 made of a silicon oxide film, a silicon nitride film, or the like is provided on the upper surface of the color filter 31. The upper surface of the planarization film 52 is planarized by, for example, CMP (Chemical Mechanical Polishing), and an on-chip lens 51 for each unit pixel 110 is provided on this planarized upper surface. The on-chip lens 51 of each unit pixel 110 has a curvature that focuses incident light onto the photoelectric conversion units PD1 and PD2. Note that the positional relationship between the on-chip lens 51, color filter 31, IR filter 41, and photoelectric conversion unit PD2 in each unit pixel 110 may be adjusted (pupil correction) according to, for example, the distance from the center of the pixel array unit 101 (image height).

[0098] 10, a light-shielding film may be provided to prevent obliquely incident light from leaking into adjacent pixels. The light-shielding film may be located above the pixel separation unit 54 provided inside the semiconductor substrate 50 (upstream in the optical path of the incident light). However, when pupil correction is performed, the position of the light-shielding film may be adjusted, for example, according to the distance (image height) from the center of the pixel array unit 101. Such a light-shielding film may be provided, for example, in the sealing film 32 or the planarization film 52. The light-shielding film may be made of a light-shielding material such as aluminum (Al) or tungsten (W).

[0099] 1.8 Organic materials In the first embodiment, when an organic semiconductor is used as the material of the photoelectric conversion film 34, the layer structure of the photoelectric conversion film 34 can be as follows: However, in the case of a stacked structure, the stacking order can be changed as appropriate. (1) Single layer structure of p-type organic semiconductor (2) Single-layer structure of n-type organic semiconductor (3-1) p-type organic semiconductor layer / n-type organic semiconductor layer stack structure (3-2) Layered structure of p-type organic semiconductor layer / mixed layer of p-type organic semiconductor and n-type organic semiconductor (bulk heterostructure) / n-type organic semiconductor layer (3-3) Layer structure of p-type organic semiconductor layer / mixed layer of p-type organic semiconductor and n-type organic semiconductor (bulk heterostructure) (3-4) Layer structure of n-type organic semiconductor layer / mixed layer of p-type organic semiconductor and n-type organic semiconductor (bulk heterostructure) (4) Mixed layer of p-type organic semiconductor and p-type organic semiconductor (bulk heterostructure)

[0100] Examples of p-type organic semiconductors include naphthalene derivatives, anthracene derivatives, phenanthrene derivatives, pyrene derivatives, perylene derivatives, tetracene derivatives, pentacene derivatives, quinacridone derivatives, thiophene derivatives, thienothiophene derivatives, benzothiophene derivatives, benzothienobenzothiophene derivatives, triallylamine derivatives, carbazole derivatives, perylene derivatives, picene derivatives, chrysene derivatives, fluoranthene derivatives, phthalocyanine derivatives, subphthalocyanine derivatives, subporphyrazine derivatives, metal complexes having a heterocyclic compound as a ligand, polythiophene derivatives, polybenzothiadiazole derivatives, and polyfluorene derivatives.

[0101] Examples of n-type organic semiconductors include fullerenes and fullerene derivatives (for example, fullerenes such as C60, C70, and C74 (higher fullerenes, endohedral fullerenes, etc.) or fullerene derivatives (for example, fullerene fluorides, PCBM fullerene compounds, fullerene polymers, etc.)), organic semiconductors with larger (deeper) HOMO and LUMO than p-type organic semiconductors, and transparent inorganic metal oxides.

[0102] Specific examples of n-type organic semiconductors include organic molecules, organometallic complexes, and subphthalocyanine derivatives having, as part of their molecular skeletons, heterocyclic compounds containing nitrogen atoms, oxygen atoms, or sulfur atoms, such as pyridine derivatives, pyrazine derivatives, pyrimidine derivatives, triazine derivatives, quinoline derivatives, quinoxaline derivatives, isoquinoline derivatives, acridine derivatives, phenazine derivatives, phenanthroline derivatives, tetrazole derivatives, pyrazole derivatives, imidazole derivatives, thiazole derivatives, oxazole derivatives, imidazole derivatives, benzimidazole derivatives, benzotriazole derivatives, benzoxazole derivatives, carbazole derivatives, benzofuran derivatives, dibenzofuran derivatives, subporphyrazine derivatives, polyphenylenevinylene derivatives, polybenzothiadiazole derivatives, and polyfluorene derivatives.

[0103] Examples of groups contained in the fullerene derivative include halogen atoms; linear, branched, or cyclic alkyl or phenyl groups; groups having linear or condensed aromatic compounds; groups having halides; partial fluoroalkyl groups; perfluoroalkyl groups; silylalkyl groups; silylalkoxy groups; arylsilyl groups; arylsulfanyl groups; alkylsulfanyl groups; arylsulfonyl groups; alkylsulfonyl groups; arylsulfide groups; alkylsulfide groups; amino groups; alkylamino groups; arylamino groups; hydroxy groups; alkoxy groups; acylamino groups; acyloxy groups; carbonyl groups; carboxy groups; carboxamido groups; carboalkoxy groups; acyl groups; sulfonyl groups; cyano groups; nitro groups; groups having chalcogenides; phosphine groups; phosphonic groups; and derivatives thereof.

[0104] The thickness of the photoelectric conversion film 34 made of the organic material as described above is not limited to the following value, but may be, for example, 1×10 -8 m (meters) to 5 x 10 -7 m, preferably 2.5 x 10 -8 m to 3×10 -7 m, more preferably 2.5 x 10 -8 m to 2×10 -7 m, more preferably 1×10 -7 m to 1.8×10 -7 Examples of suitable organic semiconductors include p-type and n-type. Organic semiconductors are often classified as p-type and n-type, but p-type means that they easily transport holes, and n-type means that they easily transport electrons, and are not limited to the interpretation that they have holes or electrons as thermally excited majority carriers like inorganic semiconductors.

[0105] Examples of materials that can be used to form the photoelectric conversion film 34 that photoelectrically converts light of a green wavelength include rhodamine-based dyes, melacyanine-based dyes, quinacridone derivatives, and subphthalocyanine-based dyes (subphthalocyanine derivatives).

[0106] Furthermore, examples of materials that can be used to form the photoelectric conversion film 34 that photoelectrically converts blue light include coumaric acid dyes, tris-8-hydroxyquinolialuminum (Alq3), and melacyanine dyes.

[0107] Furthermore, examples of materials that can be used to form the photoelectric conversion film 34 that photoelectrically converts red light include phthalocyanine dyes and subphthalocyanine dyes (subphthalocyanine derivatives).

[0108] Furthermore, as the photoelectric conversion film 34, a panchromatic photosensitive organic photoelectric conversion film that is sensitive to almost all visible light from the ultraviolet region to the red region can also be used.

[0109] 1.9 Planar structure example Next, an example of the planar structure of the pixel array unit according to this embodiment will be described. Fig. 11 is a diagram showing an example of the planar layout of each layer of the pixel array unit according to the first embodiment, where (A) shows an example of the planar layout of the on-chip lens 51, (B) shows an example of the planar layout of the color filter 31, (C) shows an example of the planar layout of the storage electrode 37, and (D) shows an example of the planar layout of the photoelectric conversion unit PD2. In Fig. 11, (A) to (D) show an example of the planar layout of a surface parallel to the element formation surface of the semiconductor substrate 50. In addition, in this description, a case where a unit array is a Bayer array of 2 × 2 pixels composed of pixels that selectively detect red (R) wavelength components (hereinafter referred to as R pixels 10r), pixels that selectively detect green (G) wavelength components (hereinafter referred to as G pixels 10g), and pixels that selectively detect blue (B) wavelength components (hereinafter referred to as B pixels 10b) will be described as an example.

[0110] 11(A) to 11(D), in this embodiment, one on-chip lens 51, one color filter 31, one storage electrode 37, and one photoelectric conversion unit PD2 are provided for one unit pixel 110. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one IR pixel 20.

[0111] In this way, by arranging one RGB pixel 10 and one IR pixel 20 in one unit pixel 110 along the traveling direction of incident light, it is possible to improve the coaxiality of the RGB pixels 10 and the IR pixels 20 with respect to the incident light, and therefore it is possible to suppress spatial misalignment that occurs between the RGB image and the IR image. This makes it possible to improve the accuracy of the results obtained by integrally processing information (RGB image and IR image) acquired by different sensors.

[0112] 1.10 Pixel drive line wiring example Next, an example of wiring of the pixel drive line LD1 connecting the RGB pixel 10 and the RGB pixel drive circuit 102A and the pixel drive line LD2 connecting the IR pixel 20 and the IR pixel drive circuit 102B will be described. Fig. 12 is a plan view showing an example of wiring of the pixel drive line for the RGB pixels according to the first embodiment, and Fig. 13 is a plan view showing an example of wiring of the pixel drive line for the IR pixel according to the first embodiment.

[0113] 12 and 13 , the RGB drive line LD1 connecting the RGB pixel drive circuit 102A to the transfer gate 11, reset transistor 12, and selection transistor 14 of the RGB pixel 10, and the IR drive line LD2 connecting the IR pixel drive circuit 102B to the transfer transistor 21, reset transistor 22, selection transistor 24, and discharge transistor 25 of the IR pixel 20, may be wired orthogonal to each other. However, this is not limiting, and the RGB drive line LD1 and the IR drive line LD2 may be wired in parallel. In this case, the RGB pixel drive circuit 102A and the IR pixel drive circuit 102B may supply various control signals to the pixel array unit 101 from the same side or from different sides.

[0114] 1.11 Example of image sensor stack structure Fig. 14 is a diagram showing an example of a stacked structure of the image sensor according to the first embodiment. As shown in Fig. 14, the image sensor 100 has a structure in which a pixel chip 140 and a circuit chip 150 are stacked one above the other. The pixel chip 140 is, for example, a semiconductor chip including a pixel array section 101 in which unit pixels 110 including RGB pixels 10 and IR pixels 20 are arranged, and the circuit chip 150 is, for example, a semiconductor chip in which the pixel circuits shown in Fig. 5 are arranged.

[0115] The pixel chip 140 and the circuit chip 150 can be bonded together by, for example, flattening their respective bonding surfaces and bonding them together by electron force, which is known as direct bonding. However, the present invention is not limited to this, and other bonding methods such as Cu-Cu bonding, which bonds together copper (Cu) electrode pads formed on the bonding surfaces of the two chips, or bump bonding can also be used.

[0116] The pixel chip 140 and the circuit chip 150 are electrically connected via a connection portion such as a TSV (Through-Silicon Via) that penetrates the semiconductor substrate. For the connection using the TSV, for example, a so-called twin TSV method can be adopted in which two TSVs, a TSV provided on the pixel chip 140 and a TSV provided from the pixel chip 140 to the circuit chip 150, are connected on the outer surface of the chip, or a so-called shared TSV method can be adopted in which the two are connected by a TSV that penetrates from the pixel chip 140 to the circuit chip 150.

[0117] However, when Cu-Cu bonding or bump bonding is used to bond the pixel chip 140 and the circuit chip 150, they may be electrically connected via the Cu-Cu bonding portion or the bump bonding portion.

[0118] 1.12 Recognition operation example Next, an example of the recognition operation performed by the electronic device 1 according to this embodiment will be described. Note that, here, an example of the recognition operation will be described using the recognition system illustrated in Fig. 2, but as described above, the recognition operation may be realized so as to be completed within the image sensor 100, or may be realized by processing image data acquired by the image sensor 100 within the processor 4, or may be realized by performing some of the processing on the image data acquired by the image sensor 100 within the image sensor 100 and the rest within the processor 4.

[0119] 15 is a flowchart showing an example of the recognition operation according to the first embodiment. As shown in FIG. 15, in this operation, first, the RGB sensor unit 71 drives the RGB pixels 10 to acquire RGB image data (step S101), and the IR sensor unit 73 drives the IR pixels 20 to acquire IR image data (step S102). The acquired RGB image data and IR image data are subjected to predetermined processes in the RGB image processing unit 72 and the IR image processing unit 74, and then input to the recognition processing unit 75. Note that, in step S102, if ROI information is input from the RGB image processing unit 72 to the IR sensor unit 73 or the IR image processing unit 74, the RGB image data and / or IR image data of the region corresponding to the ROI information may be input to the recognition processing unit 75.

[0120] Next, the recognition processing unit 75 uses the input RGB image data to perform a recognition process (first recognition process) of an object present within the angle of view of the image sensor 100 (step S103). The first recognition process may be a recognition process such as pattern recognition or a recognition process using artificial intelligence.

[0121] Next, the recognition processing unit 75 uses the input IR image data and the result of the first recognition processing to execute a recognition processing (second recognition processing) for more accurately recognizing objects present within the angle of view of the image sensor 100 (step S104). As with the first recognition processing, the second recognition processing may be a recognition processing such as pattern recognition or a recognition processing using artificial intelligence.

[0122] Next, the recognition processing unit 75 outputs the result of the second recognition processing obtained in step S104 to the outside, for example, via the interface unit 76 (step S105). Note that the recognition processing unit 75 may execute a part of the first recognition processing and output the result (intermediate data, etc.) to the outside, or may execute a part of the second recognition processing and output the result (intermediate data, etc.).

[0123] Thereafter, the recognition system determines whether or not to end this operation (step S106), and if not to end it (NO in step S106), returns to step S101. On the other hand, if to end it (YES in step S106), the recognition system ends this operation.

[0124] 1.13 Actions and Effects As described above, according to the first embodiment, it is possible to acquire multiple pieces of sensor information, namely, an RGB image acquired by the RGB pixel 10 and an IR image acquired by the IR pixel 20, thereby improving the accuracy of the recognition process using these pieces of sensor information. For example, as described above, by acquiring IR image data in addition to RGB image data, it is possible to more accurately identify unauthorized access, such as spoofing using a photograph in face authentication. This makes it possible to realize a solid-state imaging device and a recognition system that enable more secure authentication.

[0125] Furthermore, in this embodiment, the accuracy of the recognition process can be further improved by performing multi-stage recognition processing using information from multiple sensors, thereby realizing a solid-state imaging device and a recognition system that enable even more secure authentication.

[0126] 2. Second embodiment Next, a second embodiment will be described in detail with reference to the drawings. In the following description, configurations similar to those of the above-described embodiment will be referred to, and overlapping descriptions will be omitted.

[0127] The first embodiment described above exemplifies a case in which one IR pixel 20 is associated with one RGB pixel 10. In contrast, the second embodiment exemplifies a case in which a plurality of RGB pixels 10 are associated with one IR pixel 20.

[0128] 2.1 Example of unit pixel configuration First, a configuration example of a unit pixel 210 according to this embodiment will be described. As in the first embodiment, here, a case will be exemplified in which the unit pixel 210 includes an RGB pixel for acquiring an RGB image of the three primary colors of RGB and an IR pixel for acquiring an IR image of infrared (IR) light. Furthermore, it is assumed that the RGB pixels 10 are arranged according to, for example, a Bayer array.

[0129] 16 is a schematic diagram showing an example of the schematic configuration of a unit pixel according to the second embodiment. As shown in FIG. 16, the unit pixel 210 has a structure in which one IR pixel 20 is arranged in the light incident direction for four RGB pixels 10 arranged in two rows and two columns. That is, in this embodiment, one IR pixel 20 is arranged in a direction perpendicular to the arrangement direction (planar direction) of the unit pixels 210 for the four RGB pixels 10, and light transmitted through four RGB pixels 10 located upstream in the optical path of the incident light is incident on one IR pixel 20 located downstream of these four RGB pixels 10. Therefore, in this embodiment, the optical axes of the incident light of the unit array of the Bayer array composed of four RGB pixels 10 and the IR pixel 20 are aligned or approximately aligned.

[0130] 2.2 Example of unit pixel circuit configuration Fig. 17 is a circuit diagram showing a schematic configuration example of a unit pixel according to the second embodiment. Note that Fig. 17 is based on the unit pixel 110-2 according to the second modified example described with reference to Fig. 8 in the first embodiment, but is not limited to this and may be based on any of the unit pixels 110 to 110-3.

[0131] As shown in FIG. 17, a unit pixel 210 includes a plurality of RGB pixels 10-1 to 10-N (N is 4 in FIG. 17) and one IR pixel 20. When one unit pixel 210 includes a plurality of RGB pixels 10 in this manner, the plurality of RGB pixels 10 can share one pixel circuit (reset transistor 12, floating diffusion region FD1, amplification transistor 13, and selection transistor 14) (pixel sharing), as in the third modification described with reference to FIG. 9 in the first embodiment. Therefore, in this embodiment, the plurality of RGB pixels 10-1 to 10-N share a pixel circuit including the reset transistor 12, floating diffusion region FD1, amplification transistor 13, and selection transistor 14. That is, in this embodiment, a plurality of photoelectric conversion units PD1 and transfer gates 11 are connected to a common floating diffusion region FD1.

[0132] 2.3 Example of cross-sectional structure of a unit pixel Fig. 18 is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the second embodiment. Similar to Fig. 16, this description will take as an example a case where each unit pixel 210 is composed of four RGB pixels 10 arranged in two rows and two columns and one IR pixel 20. Similar to Fig. 10, the following description will focus on a semiconductor chip on which photoelectric conversion units PD1 and PD2 of the unit pixel 210 are formed, and will explain an example of the cross-sectional structure. Furthermore, in the following description, structures similar to the cross-sectional structure of the image sensor 100 described in the first embodiment using Fig. 10 will be cited, and redundant description will be omitted.

[0133] 18, in this embodiment, in a cross-sectional structure similar to that illustrated in Fig. 10, the on-chip lens 51, the color filter 31, and the storage electrode 37 are divided into four parts, 2 rows and 2 columns (however, Fig. 18 shows only two of the four), thereby constituting four RGB pixels 10. The four RGB pixels 10 in each unit pixel 210 may form a basic array of a Bayer array.

[0134] 2.4 Planar structure example 19 is a diagram showing an example of a planar layout of each layer of a pixel array unit according to the second embodiment, in which (A) shows an example of a planar layout of an on-chip lens 51, (B) shows an example of a planar layout of a color filter 31, (C) shows an example of a planar layout of a storage electrode 37, and (D) shows an example of a planar layout of a photoelectric conversion unit PD2. In FIG. 19, (A) to (D) show examples of a planar layout of a surface parallel to the element formation surface of a semiconductor substrate 50.

[0135] 19(A) to 19(D), in this embodiment, four on-chip lenses 51, four color filters 31, four storage electrodes 37, and one photoelectric conversion unit PD2 are provided for one unit pixel 210. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one IR pixel 20.

[0136] In this way, in one unit pixel 110, by arranging a basic Bayer array consisting of four RGB pixels 10 and one IR pixel 20 along the traveling direction of incident light, it is possible to improve the coaxiality of each RGB pixel 10 and IR pixel 20 with respect to the incident light, thereby suppressing spatial misalignment that occurs between the RGB image and the IR image. This makes it possible to improve the accuracy of the results obtained by integrally processing information (RGB image and IR image) acquired by different sensors.

[0137] 2.5 Modified on-chip lens In the second embodiment described above, an example has been given in which one on-chip lens 51 is provided for one RGB pixel 10, but the present invention is not limited to this, and one on-chip lens may be provided for a plurality of RGB pixels 10. Fig. 20 is a diagram showing an example of a planar layout of each layer of a pixel array unit according to a modified example of the on-chip lens of the second embodiment, in which, similar to Fig. 19, (A) shows an example of a planar layout of the on-chip lens 51, (B) shows an example of a planar layout of the color filter 31, (C) shows an example of a planar layout of the storage electrode 37, and (D) shows an example of a planar layout of the photoelectric conversion unit PD2.

[0138] 20A, in a modified example of the on-chip lens shown in FIG. 20A, two on-chip lenses 51 arranged in the row direction in some of the unit pixels 210 are replaced with one on-chip lens 251 of a 2×1 pixel that spans two RGB pixels 10. Also, as shown in FIG. 20B, the two RGB pixels 10 that share the on-chip lens 251 are provided with color filters 31 that selectively transmit the same wavelength component. In the example shown in FIG. 20B, in the upper left unit pixel 210, the color filter 31b that originally selectively transmits blue (B) wavelength components in the Bayer array is replaced with a color filter 31g that selectively transmits green (G) wavelength components, and as a result, the color filters 31 of the two RGB pixels 10 that share the on-chip lens 251 are unified into the color filter 31g.

[0139] For the RGB pixels 10 whose color filters 31 have been replaced in this way, the pixel values ​​of the wavelength components that should be detected according to the Bayer array may be interpolated from the pixel values ​​of surrounding pixels, for example. Various methods, such as linear interpolation, may be used for this pixel interpolation.

[0140] In addition, in the modified example of the on-chip lens, a case where two on-chip lenses 51 arranged in the row direction are shared is exemplified, but the present invention is not limited to this, and various modifications are possible, such as a configuration where two on-chip lenses 51 arranged in the column direction are shared, or a configuration where all four on-chip lenses 51 included in one unit pixel 210 are replaced with one on-chip lens. In this case, the color filters 31 of the RGB pixels 10 that share the on-chip lens may be color filters 31 that selectively transmit the same wavelength component.

[0141] Furthermore, sharing the on-chip lens 51 between adjacent RGB pixels 10 is not limited to the second embodiment, but can also be applied to the first embodiment.

[0142] 2.6 Modified color filter array In the above-described embodiment and its modified examples, the Bayer array has been exemplified as the filter array of the color filter 31, but this is not limiting. For example, various filter arrays may be used, such as a 3×3 pixel color filter array used in the X-Trans (registered trademark) CMOS sensor, a 4×4 pixel quad Bayer array (also called a quadra array), or a 4×4 pixel color filter array (also called a white RGB array) that combines a Bayer array with a white RGB color filter.

[0143] 21A and 21B are diagrams showing examples of the planar layout of each layer of the pixel array section according to a modified example of the color filter arrangement of the second embodiment. Similar to FIGS. 19 and 20, (A) shows an example of the planar layout of the on-chip lens 51, (B) shows an example of the planar layout of the color filter 31, (C) shows an example of the planar layout of the storage electrode 37, and (D) shows an example of the planar layout of the photoelectric conversion section PD2.

[0144] 21 shows a modified example of the color filter array, in which each color filter 31 in the 2×2 pixel Bayer array is divided into 2×2 pixels, resulting in a quadra array of 4×4 pixels overall, as shown in (B). In such a quadra array, even if an on-chip lens 51 is shared between two adjacent RGB pixels 10 as shown in (A) of FIG. 21, the color filters 31 in these RGB pixels 10 are originally aligned, so there is no need to change the array of the color filters 31, and therefore there is no need to perform pixel interpolation.

[0145] 2.7 Actions and Effects As described above, according to the second embodiment, four photoelectric conversion units PD1 of four RGB pixels 10 and one photoelectric conversion unit PD2 of one IR pixel 20 are arranged in the direction of light incidence. Even with such a configuration, as in the first embodiment, it is possible to acquire multiple pieces of sensor information for the RGB image and the IR image, thereby improving the accuracy of the recognition process using these pieces of information. This makes it possible to realize a solid-state imaging device and a recognition system that enable more secure authentication.

[0146] Furthermore, similar to the first embodiment, by performing multi-stage recognition processing using information from multiple sensors, it is possible to further improve the accuracy of the recognition processing, thereby realizing a solid-state imaging device and a recognition system that enable even more secure authentication.

[0147] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0148] 3. Third embodiment Next, a third embodiment will be described in detail with reference to the drawings. In the following description, configurations similar to those of the above-described embodiments will be referred to, and overlapping descriptions will be omitted.

[0149] In the first and second embodiments described above, examples of a solid-state imaging device and a recognition system that enable more secure authentication by combining an RGB sensor unit 71 that acquires RGB images and an IR sensor unit 73 that acquires IR images are described. In contrast, in the third embodiment, examples of a solid-state imaging device and a recognition system that enable more secure authentication by combining an RGB sensor unit 71 that acquires RGB images and a distance measurement sensor unit that acquires depth images are described.

[0150] In the following description, an indirect ToF (Time-of-Flight) sensor is used as an example of a distance measurement sensor, which detects a two-dimensional distance distribution to an object based on the phase of light emitted from a light source when it reflects off the object and returns. However, the present invention is not limited to this, and various distance measurement sensors may be used, such as a direct ToF sensor that detects a two-dimensional distance distribution to an object by measuring the time it takes for light emitted from a light source to reflect off the object and return, a structured light distance measurement sensor that projects a predetermined pattern of light onto an object and detects the distance distribution to the object from the distortion of that pattern, a stereo vision distance measurement sensor that detects the distance distribution to an object using two or more images captured from different positions, and other distance measurement sensors such as millimeter-wave radar, LIDER (Laser Imaging Detection and Ranging), and LADAR (Laser Detection and Ranging).

[0151] 3.1 Example of functional configuration of recognition system First, an example of the functional configuration of a recognition system according to this embodiment will be described. Fig. 22 is a block diagram showing an example of the functional configuration of a recognition system according to a third embodiment. As shown in Fig. 22, the recognition system 370 includes two types of sensor units: an RGB sensor unit 71 and a ToF sensor unit 373. The recognition system 370 also includes an RGB image processing unit 72, a distance measurement processing unit 374, a recognition processing unit 75, and an interface (I / F) unit 76.

[0152] The RGB sensor unit 71, the RGB image processing unit 72, the recognition processing unit 75, and the interface unit 76 may be the same as those exemplified in the first embodiment.

[0153] The ToF sensor unit 373 includes, for example, a light source and a light receiving unit (corresponding to an image sensor), and detects light emitted from the light source and reflected by an object and returned. As will be described later, the light source may be, for example, a laser light source such as a VCSEL (Vertical Cavity Surface Emitting Laser). Furthermore, the light receiving unit may be, for example, an image sensor in which the pixel circuits of the IR pixels 20 in the IR sensor unit 73 according to the above-described embodiment are changed to pixel circuits for a ToF sensor.

[0154] The ranging processing unit 374 detects the phase of the reflected light for each ToF pixel based on the emission timing of light from the light source (hereinafter referred to as irradiated light) from the pixel signal for each ToF pixel input from the ToF sensor unit 373, and generates a depth image showing a two-dimensional distance distribution based on the detected phase for each ToF pixel.

[0155] The recognition processing unit 75 uses the RGB image data input from the RGB image processing unit 72 and / or the depth image data input from the distance measurement processing unit 374 to perform recognition processing of objects and the like present within the angle of view of the RGB sensor unit 71 and / or the ToF sensor unit 373. As in the first embodiment, the recognition processing by the recognition processing unit 75 may be recognition processing such as pattern recognition or recognition processing by AI. Furthermore, the recognition processing unit 75 may perform part of the recognition processing and output the results (intermediate data, etc.).

[0156] The interface unit 76 outputs the recognition results (including intermediate data, etc.) obtained by the recognition processing unit 75 and the image data acquired by the RGB sensor unit 71 and / or the ToF sensor unit 373 to an external device, such as the processor 4 and / or the memory unit 3.

[0157] The RGB image processing unit 72 may perform object region determination on the RGB image data, and input information such as an address specifying a region of interest (ROI) obtained as a result (hereinafter simply referred to as ROI information) to the ToF sensor unit 373 and / or the distance measurement processing unit 374. In response to this, the ToF sensor unit 373 may operate to acquire depth image data of a region corresponding to the ROI information input from the RGB image processing unit 72. Alternatively, the distance measurement processing unit 374 may be configured to perform distance measurement processing on only ToF pixels of a region corresponding to the ROI information input from the RGB image processing unit 72, for image data made up of pixel signals for each ToF pixel input from the ToF sensor unit 373 (hereinafter referred to as ToF image data).

[0158] 3.2 System configuration example Next, an example of the system configuration of the recognition system according to this embodiment will be described. Fig. 23 is a schematic diagram showing an example of the configuration of an electronic device that realizes the recognition system according to the third embodiment, and Fig. 24 is a block diagram showing an example of the configuration of an electronic device that realizes the recognition system according to the third embodiment.

[0159] As shown in FIG. 23, an electronic device 80 according to this embodiment includes a laser light source 81, an illumination lens 83, an imaging lens 84, an image sensor 300, and a system control unit 85.

[0160] 24, the laser light source 81 is configured with, for example, a vertical cavity surface emitting laser (VCSEL) 382 and a light source driver 381 that drives the VCSEL 382. However, the light source is not limited to the VCSEL 382, ​​and various light sources such as an LED (Light Emitting Diode) may be used. Furthermore, the laser light source 81 may be any of a point light source, a surface light source, and a linear light source. In the case of a surface light source or a linear light source, the laser light source 81 may have a configuration in which, for example, a plurality of point light sources (e.g., VCSELs) are arranged one-dimensionally or two-dimensionally.

[0161] In this embodiment, the laser light source 81 may emit light of a wavelength band different from the detection wavelength band of the RGB pixels 10 in the image sensor 300, such as IR light.

[0162] The illumination lens 83 is disposed on the side of the emission surface of the laser light source 81, and converts the light emitted from the laser light source 81 into illumination light with a predetermined divergence angle.

[0163] The imaging lens 84 is disposed on the light receiving surface side of the image sensor 300, and forms an image on the light receiving surface of the image sensor 300 based on incident light (including reflected light of the irradiated light).

[0164] The image sensor 300 will be described in detail later, but as shown in FIG. 24, for example, it is made up of a light receiving unit 384 and a sensor control unit 383 that drives the light receiving unit 384 to read out RGB image data and ToF image data.

[0165] The system control unit 85 is configured by, for example, a processor (CPU), and drives the VCSEL 382 via the light source drive unit 381. The system control unit 85 also controls the image sensor 300 to acquire RGB image data, and controls the image sensor 300 in synchronization with the control of the laser light source 81 to acquire ToF image data for the light emitted from the laser light source 81.

[0166] In this configuration, illumination light emitted from the laser light source 81 passes through an illumination lens 83 and is projected onto a subject (also referred to as a measurement target or object) 901. This projected light is reflected by the subject 901. The light reflected by the subject 901 passes through an imaging lens 84 and enters the image sensor 300. The ToF sensor unit 373 in the image sensor 300 receives the light reflected by the subject 901 and generates ToF image data. Meanwhile, the RGB sensor unit 71 in the image sensor 300 receives, for example, visible light of the incident light and generates RGB image data. The RGB image data and ToF image data generated by the image sensor 300 are supplied to an application processor 86 of the electronic device 80. The application processor 86 performs predetermined processing, such as recognition processing, on the RGB image data and ToF image data input from the image sensor 300.

[0167] 3.3 Image sensor configuration example Next, a configuration example of the image sensor 300 according to this embodiment will be described. Fig. 25 is a block diagram showing a schematic configuration example of an image sensor according to a third embodiment.

[0168] The image sensor 300 according to this embodiment has a configuration similar to that of the image sensor 100 according to the first embodiment, but with the configuration corresponding to the IR sensor unit 73 replaced with a configuration corresponding to a ToF sensor unit 373. Specifically, for example, the unit pixel 110 is replaced with a unit pixel 310, the IR pixel drive circuit 102B is replaced with a ToF pixel drive circuit 302B, the IR signal processing circuit 103B is replaced with a ToF signal processing circuit 303B, and the IR data processing unit 108B is replaced with a ToF data processing unit 308B.

[0169] The pixel array unit 101, the ToF pixel drive circuit 302B, the ToF signal processing circuit 303B, the column drive circuit 104, and the system control circuit 105 constitute, for example, a ToF sensor unit 373 in Fig. 22. The ToF signal processing circuit 303B and the ToF data processing unit 308B constitute, for example, a distance measurement processing unit 374 in Fig. 22. The recognition processing unit 75 in Fig. 22 may be realized by the application processor 86 alone, or may be realized by linking the RGB data processing unit 108A and the ToF data processing unit 308B with the application processor 86, or may be realized by linking the RGB data processing unit 108A with the ToF data processing unit 308B.

[0170] Each unit pixel 310 includes an RGB pixel 10 and a ToF pixel 320. Each ToF pixel 320 includes a photoelectric conversion unit that generates and accumulates electric charges according to the amount of light received, and generates a pixel signal having a voltage according to the amount of incident light.

[0171] In this embodiment, one end of the pixel drive line LD2 is connected to an output terminal corresponding to each row of the ToF pixel drive circuit 302B, and transmits a control signal for driving the ToF pixels 320 when pixel signals are read out.

[0172] Similar to the RGB pixel drive circuit 102A, the ToF pixel drive circuit 302B is configured with a shift register, an address decoder, etc., and drives each pixel of the pixel array unit 101 simultaneously for all pixels or in row units, etc. That is, the ToF pixel drive circuit 302B, together with the system control circuit 105 that controls the ToF pixel drive circuit 302B, constitutes a drive unit that controls the operation of each ToF pixel 320 in the pixel array unit 101. Also, similar to the RGB pixel drive circuit 102A, the ToF pixel drive circuit 302B may be equipped with two scan systems: a readout scan system and a sweep scan system.

[0173] The pixel signals output from each ToF pixel 320 in a pixel row selected and scanned by the ToF pixel drive circuit 302B are input to the ToF signal processing circuit 303B through vertical signal lines VSL2 and VSL4 for each pixel column. For example, a pixel signal based on charge read from one readout terminal (also referred to as TapA) of the ToF pixel 320 appears on the vertical signal line VSL2, and a pixel signal based on charge read from the other readout terminal (also referred to as TapB) of the ToF pixel 320 appears on the vertical signal line VSL4. The ToF signal processing circuit 303B performs predetermined signal processing on the pixel signals output from each ToF pixel 320 in the selected row through the vertical signal line VSL2 or VSL4 for each pixel column of the pixel array unit 101, and temporarily stores the pixel signals after signal processing.

[0174] Similarly to the RGB signal processing circuit 103A, the ToF signal processing circuit 303B may perform noise removal processing on pixel signals, such as CDS (Correlated Double Sampling) processing or DDS (Double Data Sampling) processing. Similarly to the RGB signal processing circuit 103A, the ToF signal processing circuit 303B may also have, for example, an AD conversion function, and converts analog pixel signals read out from the photoelectric conversion units into digital signals and outputs the digital signals.

[0175] The system control circuit 105 controls the driving of the RGB pixel driving circuit 102A, the ToF pixel driving circuit 302B, the RGB signal processing circuit 103A, the ToF signal processing circuit 303B, the column driving circuit 104, etc., based on a synchronization control signal input from the system control unit 85. As a result, driving for acquiring RGB image data and / or ToF image data is executed in synchronization with the light emission control of the laser light source 81.

[0176] The ToF data processing unit 308B has, for example, an arithmetic processing function, and performs various signal processing such as arithmetic processing on the image signal output from the ToF signal processing circuit 303B. The ToF data processing unit 308B may generate a depth image showing a two-dimensional distance distribution based on a synchronization control signal input from the system control unit 85 (or the system control circuit 105) and pixel signals read out from each ToF pixel 320 at a predetermined sampling period.

[0177] The RGB image data or ToF image data (or depth image data) output from the RGB data processing unit 108A or the ToF data processing unit 308B may be subjected to predetermined processing, for example, in an application processor 86 in an electronic device 80 equipped with the image sensor 300, or may be transmitted to the outside via a predetermined network.

[0178] The image sensor 300 may also include a storage unit for temporarily storing data necessary for signal processing in the RGB data processing unit 108A and the ToF data processing unit 308B, and data processed by one or more of the RGB signal processing circuit 103A, the ToF signal processing circuit 303B, the RGB data processing unit 108A, and the ToF data processing unit 308B.

[0179] 3.3.1 Image sensor variations Fig. 26 is a block diagram showing a schematic configuration example of an image sensor according to a modified example of the third embodiment. While Fig. 25 above illustrates a case where separate signal processing circuits (RGB signal processing circuit 103A and ToF signal processing circuit 303B) are provided for the RGB pixels 10 and the ToF pixels 320, respectively, the present invention is not limited to this. As illustrated in Fig. 26, it is also possible to provide a common signal processing circuit 303 for the RGB pixels 10 and the ToF pixels 320. In this case, the readout of pixel signals from the RGB pixels 10 and the readout of pixel signals from the ToF pixels 320 may be performed in a time-division manner or in parallel.

[0180] 3.4 Example of unit pixel configuration Next, a configuration example of the unit pixel 310 will be described. Note that here, a case where the unit pixel 310 includes an RGB pixel for acquiring an RGB image of the three primary colors of RGB and a ToF pixel 320 for acquiring a ToF image of infrared (IR) light will be exemplified. Note that in FIG. 27 and below, when there is no need to distinguish between the color filters 31r, 31g, and 31b that transmit the light of each color component that makes up the three primary colors of RGB, the reference numeral 31 is used.

[0181] Fig. 27 is a schematic diagram showing an example of the outline configuration of a pixel array section according to the third embodiment. As shown in Fig. 27, the unit pixel 310 has a configuration similar to that of the unit pixel 210 described in the second embodiment with reference to Fig. 16, but with the IR pixel 20 replaced with a ToF pixel 320. However, without being limited thereto, the unit pixel 310 may have a configuration similar to that of the unit pixel 110 described in the first embodiment with reference to Fig. 5, but with the IR pixel 20 replaced with a ToF pixel 320.

[0182] 3.5 Example of unit pixel circuit configuration Fig. 28 is a circuit diagram showing a schematic configuration example of a unit pixel according to the third embodiment. Note that Fig. 28 is based on the unit pixel 110 described in the first embodiment using Fig. 6, but is not limited to this and may be based on any of the unit pixels 110 to 110-3 and 210.

[0183] As shown in FIG. 28, the unit pixel 310 includes two RGB pixels 10-1 and 10-2 and one ToF pixel 320.

[0184] (RGB pixels 10) Like the RGB pixels 10 according to the first embodiment, each RGB pixel 10 includes a transfer gate 11, a floating diffusion region FD1, a reset transistor 12, an amplification transistor 13, and a selection transistor 14. However, the drain of the selection transistor 14 of one RGB pixel 10-1 is connected to a vertical signal line VSL1, and the drain of the selection transistor 14 of the other RGB pixel 10-2 is connected to a vertical signal line VSL3. Furthermore, the two RGB pixels 10-1 and 10-2 may be, for example, two RGB pixels 10 adjacent to each other in the row direction in the pixel array section 101. However, the number of RGB pixels 10 included in one unit pixel 310 is not limited to two, and may be three or more.

[0185] (ToF pixels 320) The ToF pixel 320 includes, for example, a photoelectric conversion unit PD2, transfer transistors 21A and 21B, floating diffusion regions FD2A and FD2B, reset transistors 22A and 22B, amplification transistors 23A and 23B, selection transistors 24A and 24B, and a discharge transistor 25. The transfer transistor 21A, the floating diffusion region FD2A, the reset transistor 22A, the amplification transistor 23A, and the selection transistor 24A form a pixel circuit (hereinafter referred to as pixel circuit 320A) for reading out charges from one tap TapA of two taps provided in the photoelectric conversion unit PD2, and the transfer transistor 21B, the floating diffusion region FD2B, the reset transistor 22B, the amplification transistor 23B, and the selection transistor 24B form a pixel circuit (hereinafter referred to as pixel circuit 320B) for reading out charges from the other tap TapB of two taps provided in the photoelectric conversion unit PD2. The drain of the selection transistor 24A in the pixel circuit 320A is connected to the vertical signal line VSL2, and the drain of the selection transistor 24B in the pixel circuit 320B is connected to the vertical signal line VSL4.

[0186] The connection relationships between the floating diffusion region FD2A or FD2B, the reset transistor 22A or 22B, and the amplifier transistor 23A or 23B and the transfer transistor 21A or 21B may be similar to the connection relationships between the floating diffusion region FD1, the reset transistor 12, and the amplifier transistor 13 and the transfer gate 11 in the RGB pixel 10-1 or 10-2. The connection relationships between the amplifier transistor 23A or 23B, the selection transistor 24A or 24B, and the vertical signal line VSL2 or VSL4 may be similar to the connection relationships between the amplifier transistor 13, the selection transistor 14, and the vertical signal line VSL1 or VSL3 in the RGB pixel 10-1 or 10-2.

[0187] The photoelectric conversion unit PD2, the details of which will be described later, has two taps TapA and TapB, and outputs charges generated by photoelectric conversion from the two taps TapA and TapB in a time-division manner. Therefore, the pixel circuit 320A causes a pixel signal having a voltage value corresponding to the amount of charge output from the tap TapA to appear on the vertical signal line VSL2, and the pixel circuit 320B causes a pixel signal having a voltage value corresponding to the amount of charge output from the tap TapB to appear on the vertical signal line VSL4.

[0188] The vertical signal lines VSL2 and VSL4, like the vertical signal lines VSL1 and VSL3, are connected to AD conversion circuits 103a provided for each column (that is, for each vertical signal line VSL2 and VSL4) in the ToF signal processing circuit 303B.

[0189] 3.6 Circuit Configuration Variations Next, several examples of modifications of the circuit configuration of the unit pixel 310 shown in FIG. 28 will be described.

[0190] 3.6.1 First variant FIG. 29 is a circuit diagram showing a schematic configuration example of a unit pixel according to a first modified example of the third embodiment. As shown in FIG. 29, the unit pixel 310-1, unlike the unit pixel 310 shown in FIG. 28, includes one RGB pixel 10 and one ToF pixel 320. The pixel circuits of the RGB pixel 10 and the ToF pixel 320 may be similar to those described with reference to FIG. 28. However, in the first modified example, the vertical signal line VSL1 is shared by the RGB pixel 10 and the pixel circuit 320B in the ToF pixel 320, thereby eliminating the vertical signal lines VSL3 and VSL4. The AD conversion circuit 103a connected to the vertical signal line VSL1 is used in a time-division manner, for example, to read pixel signals from the RGB pixel 10 and the ToF pixel 320.

[0191] According to such a configuration, it is possible to reduce the number of vertical signal lines VSL and the number of AD conversion circuits 103a connected thereto, thereby making it possible to reduce the circuit scale, and thereby making it possible to reduce the size of the image sensor 100 and increase the resolution by improving area efficiency.

[0192] 3.6.2 Second variant Fig. 30 is a circuit diagram showing a schematic configuration example of a unit pixel according to a second modification of the third embodiment. As shown in Fig. 30, a unit pixel 310-2 has a configuration similar to that of the unit pixel 310-1 illustrated in Fig. 29, except that the RGB pixel 10 is connected to a vertical signal line VSL1, the pixel circuit 320A in the ToF pixel 320 is connected to a vertical signal line VSL2, and the pixel circuit 320B is connected to a vertical signal line VSL4. The vertical signal lines VSL1 and VSL2 are connected to a common AD conversion circuit 103a. Therefore, in the second modification, a switch circuit 131 is provided that switches the vertical signal line connected to the AD conversion circuit 103a to either the vertical signal line VSL1 or VSL2, as in the first modification of the first embodiment.

[0193] With this configuration, parts of the RGB signal processing circuit 103A and the ToF signal processing circuit 303B can be replaced with a common signal processing circuit, thereby making it possible to reduce the circuit scale, thereby improving area efficiency and enabling the image sensor 100 to be made smaller and have higher resolution.

[0194] 3.6.3 Third variant Fig. 31 is a circuit diagram showing a schematic configuration example of a unit pixel according to a third modified example of the third embodiment. As shown in Fig. 31, a unit pixel 310-3 has the same configuration as the unit pixel 310 shown in Fig. 28, except that the vertical signal lines VSL1 and VSL2 are connected to a common AD conversion circuit 103a, and the vertical signal lines VSL3 and VSL4 are connected to the common AD conversion circuit 103a. Therefore, in the third modified example, as in the second modified example, a switch circuit 131 is provided that switches the vertical signal line connected to one AD conversion circuit 103a to either the vertical signal lines VSL1 or VSL2, and a switch circuit 131 that switches the vertical signal line connected to the other AD conversion circuit 103a to either the vertical signal lines VSL3 or VSL4.

[0195] With this configuration, the RGB signal processing circuit 103A and the ToF signal processing circuit 303B can be replaced with a common signal processing circuit, which makes it possible to further reduce the circuit scale, thereby enabling the image sensor 100 to be made smaller and have higher resolution due to further improved area efficiency.

[0196] 3.6.4 Fourth Variant Fig. 32 is a circuit diagram showing a schematic configuration example of a unit pixel according to a fourth modified example of the third embodiment. As shown in Fig. 32, a unit pixel 310-4 has the same configuration as the unit pixel 310 shown in Fig. 28, but is configured such that each of the vertical signal lines VSL1 and VSL2 can be connected to one of two AD conversion circuits 103a, and each of the vertical signal lines VSL3 and VSL4 can be connected to one of two AD conversion circuits 103a. Therefore, in the fourth modified example, as in the third modified example of the first embodiment, switch circuits 132 and 133 are provided to switch the vertical signal lines connected to one of the two AD conversion circuits 103a to either the vertical signal lines VSL1 and VSL2, and switch circuits 132 and 133 are provided to switch the vertical signal lines connected to the other two AD conversion circuits 103a to either the vertical signal lines VSL3 and VSL4.

[0197] With this configuration, it is possible to select from multiple (two in this example) AD conversion circuits 103a to be used for each column, thereby making it possible to suppress degradation of image quality due to noise such as streaking.

[0198] 3.7 Example of cross-sectional structure of a unit pixel Next, an example of the cross-sectional structure of the image sensor 300 according to the third embodiment will be described with reference to Fig. 33. Fig. 33 is a cross-sectional view showing an example of the cross-sectional structure of the image sensor according to the third embodiment. Here, as in the above-described embodiments, the example of the cross-sectional structure will be described by focusing on a semiconductor chip on which photoelectric conversion units PD1 and PD2 in unit pixels 310 are formed.

[0199] In the following description, as in the above-described embodiment, a so-called back-illuminated cross-sectional structure is illustrated in which the light incident surface is the back side (opposite to the element formation surface) of the semiconductor substrate 50. However, the present invention is not limited to this, and a so-called front-illuminated cross-sectional structure in which the light incident surface is the front side (element formation surface side) of the semiconductor substrate 50 may also be used. Furthermore, in the present description, an organic material is used for the photoelectric conversion unit PD1 of the RGB pixel 10. However, as in the above-described embodiment, one or both of an organic material and a semiconductor material (also referred to as an inorganic material) may be used for the photoelectric conversion material of each of the photoelectric conversion units PD1 and PD2. Furthermore, in the present description, a case in which one unit pixel 310 includes two RGB pixels 10-1 and 10-2 (corresponding to the unit pixel 310 described above with reference to FIG. 28) is illustrated. However, the present invention is not limited to this, and one unit pixel 310 may include one or three or more RGB pixels 10.

[0200] 33, the image sensor 300 according to this embodiment has a cross-sectional structure similar to that of the image sensor 100 described in the second embodiment with reference to FIG. 18, but includes two taps 46 arranged on the element formation surface side (lower side in the drawing) of the photoelectric conversion unit PD2 for reading out charges of different phases. One tap 46 may correspond to tap TapA connected to the transfer transistor 21A (vertical transistor 45) of the pixel circuit 320A, and the other tap 46 may correspond to tap TapB connected to the transfer transistor 21B (vertical transistor 45) of the pixel circuit 320B. The floating diffusion region connected to the drain of the transfer transistor 21A may function as the floating diffusion region FD2A of the pixel circuit 320A, and the floating diffusion region connected to the drain of the transfer transistor 21B may function as the floating diffusion region FD2B of the pixel circuit 320B.

[0201] Other structures, materials, etc. may be the same as those in the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0202] 3.8 Planar structure example Next, an example of the planar structure of the pixel array unit according to this embodiment will be described. Fig. 34 is a diagram showing an example of the planar layout of each layer of the pixel array unit according to the third embodiment, where (A) shows an example of the planar layout of the on-chip lens 51, (B) shows an example of the planar layout of the color filter 31, (C) shows an example of the planar layout of the storage electrode 37, and (D) shows an example of the planar layout of the photoelectric conversion unit PD2. In Fig. 34, (A) to (D) show example planar layouts of a surface parallel to the element formation surface of the semiconductor substrate 50, as in the above-mentioned embodiments. In addition, in this description, a 2x2 pixel Bayer array consisting of a pixel that selectively detects red (R) wavelength components (hereinafter referred to as R pixel 10r), a pixel that selectively detects green (G) wavelength components (hereinafter referred to as G pixel 10g), and a pixel that selectively detects blue (B) wavelength components (hereinafter referred to as B pixel 10b) is used as a unit array, and one unit pixel 310 is provided with four RGB pixels 10 that constitute the unit array and one ToF pixel 320.

[0203] 34(A) to 34(D), in this embodiment, four on-chip lenses 51, four color filters 31, four storage electrodes 37, and one photoelectric conversion unit PD2 are provided for one unit pixel 310. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one ToF pixel 320.

[0204] In this way, by arranging a basic Bayer array consisting of four RGB pixels 10 and one ToF pixel 320 in one unit pixel 310 along the traveling direction of incident light, it is possible to increase the quantum efficiency of one ToF pixel 320, and therefore to improve the accuracy of the results obtained by integrally processing information acquired by different sensors (RGB image and depth image). As a result, it is possible to realize a solid-state imaging device and recognition system that enable more secure authentication.

[0205] 3.9 Recognition operation example Next, an example of the recognition operation performed by the recognition system according to this embodiment will be described. Note that, here, the example of the recognition operation will be described using the electronic device 80 described with reference to Figures 23 and 24, and also with reference to the recognition system 370 described with reference to Figure 22, but as described above, the recognition operation may be realized so as to be completed within the image sensor 300, or may be realized by processing image data acquired by the image sensor 300 within the application processor 86, or may be realized by performing some of the processing on the image data acquired by the image sensor 300 within the image sensor 300 and the rest within the application processor 86.

[0206] Fig. 35 is a flowchart showing an example of the recognition operation according to the third embodiment. As shown in Fig. 35, in this operation, first, the system control unit 85 acquires RGB image data by driving the RGB sensor unit 71 (see Fig. 22) in the image sensor 300 (step S301). The system control unit 85 also drives the laser light source 81 at a predetermined sampling period to cause the laser light source 81 to emit irradiation light at the predetermined sampling period (step S302), and also drives the ToF sensor unit 373 (see Fig. 22) in the image sensor 300 at the predetermined sampling period in synchronization with the driving of the laser light source 81 to acquire ToF image data at the predetermined sampling period (step S303).

[0207] The acquisition of RGB image data may be performed in parallel with the acquisition of ToF image data, or may be performed during a period separate from the acquisition of ToF image data. In this case, it does not matter which of the RGB image data and the ToF image data is acquired first. Furthermore, the acquisition of RGB image data may be performed once for every K (K is an integer equal to or greater than 1) times of acquisition of ToF image data.

[0208] Of the RGB image data and ToF image data acquired in this manner, the RGB image data is subjected to predetermined processing by the RGB image processing unit 72 and then input to the recognition processing unit 75. Note that in step S302 or S303, if ROI information is input from the RGB image processing unit 72 to the ToF sensor unit 373 or the ranging processing unit 374 in FIG. 22 , the RGB image data and / or ToF image data of the area corresponding to the ROI information may be input to the recognition processing unit 75.

[0209] Next, the recognition processing unit 75 uses the input RGB image data to perform a recognition process (first recognition process) of an object present within the angle of view of the image sensor 300 (step S304). As in the first embodiment, the first recognition process may be a recognition process such as pattern recognition or a recognition process using artificial intelligence.

[0210] Furthermore, for the ToF image data acquired in step S303, the ranging processing unit 374 (see FIG. 22) generates a depth image showing a two-dimensional distance distribution to an object existing within the angle of view from the phase difference between each pixel in the two ToF image data acquired from the taps TapA and TapB at a predetermined sampling period (step S305). Note that the generation of the depth image data may be executed in parallel with the first recognition process or may be executed prior to the first recognition process. Furthermore, the generation of the depth image data may be executed, for example, by the ToF data processing unit 308B inside the image sensor 300 or by the application processor 86.

[0211] Next, the recognition processing unit 75 executes a recognition process (second recognition process) for more accurately recognizing objects present within the angle of view using the result of the first recognition process and the depth image data (step S306). As with the first recognition process, the second recognition process may be a recognition process such as pattern recognition or a recognition process using artificial intelligence.

[0212] Next, the recognition processing unit 75 outputs the result of the second recognition processing obtained in step S306 to the outside, for example, via the interface unit 76 (step S307). Note that the recognition processing unit 75 may execute a part of the first recognition processing and output the result (intermediate data, etc.) to the outside, or may execute a part of the second recognition processing and output the result (intermediate data, etc.).

[0213] Thereafter, the recognition system 370 determines whether or not to end this operation (step S308), and if not to end it (NO in step S308), returns to step S301. On the other hand, if to end it (YES in step S308), the recognition system 370 ends this operation.

[0214] 3.10 Actions and Effects As described above, according to the third embodiment, it is possible to acquire multiple pieces of sensor information, including an RGB image acquired by the RGB pixel 10 and a depth image based on a ToF image acquired by the ToF pixel 320, thereby improving the accuracy of recognition processing using these pieces of sensor information. For example, as described above, acquiring depth image data in addition to RGB image data makes it possible to more accurately identify unauthorized access, such as spoofing using a photograph in face authentication. This makes it possible to realize a solid-state imaging device and a recognition system that enable more secure authentication.

[0215] Furthermore, in this embodiment, similar to the first embodiment, it is possible to further improve the accuracy of the recognition process by performing multi-stage recognition processing using information from a plurality of sensors, thereby realizing a solid-state imaging device and a recognition system that enable even more secure authentication.

[0216] The other configurations, operations, and effects may be the same as those of the above-described embodiment, and therefore detailed description thereof will be omitted here.

[0217] 4. Mobile application examples The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, or a robot.

[0218] FIG. 36 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0219] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 36, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0220] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, etc.

[0221] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0222] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc., based on the received images.

[0223] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0224] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0225] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drivetrain control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including avoiding or mitigating collisions between vehicles, following based on the distance between vehicles, maintaining vehicle speed, warning of vehicle collisions, or warning of vehicle lane departure.

[0226] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0227] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0228] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 36, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0229] FIG. 37 is a diagram showing an example of the installation position of the imaging unit 12031.

[0230] In FIG. 37, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0231] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0232] 37 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

[0233] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0234] For example, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of automatic driving, which runs autonomously without relying on driver operation.

[0235] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drivetrain control unit 12010.

[0236] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. The pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104, which are infrared cameras, and then performing pattern matching on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0237] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 of the above-described configuration. Specifically, the imaging units 12101, 12102, 12103, 12104, 12105, etc., illustrated in FIG. 37 may be mounted on the vehicle 12100. By applying the technology according to the present disclosure to the imaging units 12101, 12102, 12103, 12104, 12105, etc., it is possible to improve the accuracy of the results obtained by integrally processing information acquired by different sensors (for example, color images and monochrome images).

[0238] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0239] Furthermore, the effects of each embodiment described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0240] The present technology can also be configured as follows. (1) a plurality of unit pixels arranged in a matrix; a signal processing circuit that reads out a signal from each of the unit pixels; Equipped with Each of the unit pixels is a first pixel disposed on the first surface and configured to detect light in a first wavelength band; second pixels arranged on a second surface parallel to the first surface and configured to detect light in a second wavelength band different from the first wavelength band; Equipped with The signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels, and configured to convert analog signals output from the first pixel and the second pixel into digital signals. Solid-state imaging device. (2) The solid-state imaging device according to (1), wherein at least a portion of the first pixel overlaps with the second pixel in a direction perpendicular to the first surface and the second surface. (3) The solid-state imaging device according to (1) or (2), wherein the first pixel has an organic photoelectric conversion film. (4) The signal processing circuit further includes a first switch that switches the connection destination of the first conversion circuit between the first pixel and the second pixel. The solid-state imaging device according to any one of (1) to (3) above. (5) The signal processing circuit a second conversion circuit connected to the first pixel and the second pixel in each of the unit pixels, and configured to convert analog signals output from the first pixel and the second pixel into digital signals; a second switch that switches a connection destination of the second conversion circuit between the first pixel and the second pixel; The solid-state imaging device according to (4) further comprises: (6) Each of the unit pixels includes a plurality of the first pixels connected to the first conversion circuit. The solid-state imaging device according to any one of (1) to (5) above. (7) Each of the first pixels is a first photoelectric conversion unit that photoelectrically converts incident light; a first pixel circuit that reads out charges generated in the first photoelectric conversion unit; Equipped with The plurality of first pixels share at least a portion of the first pixel circuit. The solid-state imaging device according to (6) above. (8) Each of the unit pixels includes a plurality of the second pixels connected to the first conversion circuit. The solid-state imaging device according to any one of (1) to (7) above. (9) Each of the second pixels is a second photoelectric conversion unit that photoelectrically converts incident light; a second pixel circuit that reads out charges generated in the second photoelectric conversion unit; Equipped with The plurality of second pixels share at least a portion of the second pixel circuit. The solid-state imaging device according to (8) above. (10) The signal processing circuit generating first image data based on signals read out from the first pixels in the plurality of unit pixels; generating second image data based on signals read out from the second pixels in the plurality of unit pixels; The solid-state imaging device according to any one of (1) to (9) above. (11) The second pixel is a photoelectric conversion unit that converts incident light into an electric signal; a first pixel circuit that reads out charges generated in the photoelectric conversion unit; a second pixel circuit that reads out charges generated in the photoelectric conversion unit; Equipped with the signal processing circuit further includes a second conversion circuit connected to the first pixel circuit and converting an analog signal output from the first pixel circuit into a digital signal; The first conversion circuit is connected to the first pixel and the second pixel circuit. The solid-state imaging device according to any one of (1) to (10) above. (12) The signal processing circuit further includes a first switch that switches the connection destination of the first conversion circuit between the first pixel circuit and the second pixel circuit. The solid-state imaging device according to (11) above. (13) Each of the unit pixels includes a plurality of the first pixels, The second pixel is a photoelectric conversion unit that converts incident light into an electric signal; a first pixel circuit that reads out charges generated in the photoelectric conversion unit; a second pixel circuit that reads out charges generated in the photoelectric conversion unit; Equipped with the signal processing circuit further includes a second conversion circuit connected to one of the plurality of first pixels and the second pixel circuit, and configured to convert analog signals output from the one of the plurality of first pixels and the second pixel circuit into digital signals; The first conversion circuit is connected to another one of the plurality of first pixels and the first pixel circuit. The solid-state imaging device according to any one of (1) to (12) above. (14) The signal processing circuit a first switch that switches a connection destination of the first conversion circuit to either the other one of the plurality of first pixels or the first pixel circuit; a second switch that switches a connection destination of the second conversion circuit between the one of the plurality of first pixels and the second pixel circuit; The solid-state imaging device according to (13) above, further comprising: (15) The signal processing circuit a third conversion circuit connected to the other one of the plurality of first pixels and the first pixel circuit, and configured to convert analog signals output from the other one of the plurality of first pixels and the first pixel circuit into digital signals; a fourth conversion circuit connected to the one of the plurality of first pixels and the second pixel circuit, and configured to convert analog signals output from the one of the plurality of first pixels and the second pixel circuit into digital signals; a third switch that switches a connection destination of the first conversion circuit between the other one of the plurality of first pixels and the first pixel circuit; a fourth switch that switches a connection destination of the second conversion circuit between the one of the plurality of first pixels and the second pixel circuit; The solid-state imaging device according to (14) above, further comprising: (16) the light in the first wavelength band is light included in the wavelength band of visible light, The light in the second wavelength band is light included in the wavelength band of infrared light. The solid-state imaging device according to any one of (1) to (15) above. (17) a first driving circuit that drives the first pixel in each of the unit pixels; a second driving circuit that drives the second pixel in each of the unit pixels; The solid-state imaging device according to any one of (1) to (16) above, comprising: (18) the first pixel includes a first photoelectric conversion unit that photoelectrically converts incident light; the second pixel includes a second photoelectric conversion unit that photoelectrically converts incident light, The first photoelectric conversion unit and the second photoelectric conversion unit are arranged along the optical axis of the incident light. The solid-state imaging device according to any one of (1) to (17) above. (19) The solid-state imaging device according to any one of (1) to (18) above; a recognition processing unit that executes recognition processing based on first detection data acquired by the first pixel and second detection data acquired by the second pixel in the solid-state imaging device; A recognition system comprising: (20) a light source that emits light in the second wavelength band; a control unit that controls the light source and the solid-state imaging device; Furthermore, The control unit controls the light emission timing of the light source and the drive timing of the second pixel in the solid-state imaging device so as to synchronize them. The recognition system according to (19) above. (twenty one) a processing unit that generates depth information indicating a distance from the second pixel to an object, the depth information being acquired by the second pixel in the solid-state imaging device; The recognition processing unit performs the recognition processing based on the first detection data and the depth information. The recognition system according to (19) or (20). (twenty two) The recognition processing unit performing a first recognition process based on one of the first detection data and the second detection data; A second recognition process is performed based on a result of the first recognition process and the other of the first detection data and the second detection data. The recognition system according to any one of (19) to (21) above. [Explanation of symbols]

[0241] 1, 80 electronic equipment 2 Imaging lens 3 Storage section 4 processors 10, 10-1 to 10-N RGB pixels 11 Transfer Gate 12, 22, 22A, 22B Reset transistors 13, 23, 23A, 23B Amplifying transistors 14, 24, 24A, 24B select transistors 20 IR pixels 21, 21A, 21B Transfer transistors 25 Emission transistor 31, 31r, 31g, 31b color filters 32 Sealing film 33 Transparent electrode 34 Photoelectric conversion film 35 Semiconductor layer 36 readout electrode 37 Storage electrode 41 IR filter 42 p-well region 43 p-type semiconductor region 44 n-type semiconductor region 45 Vertical transistor 46 Tap (TapA, TapB) 50 Semiconductor substrate 51 On-chip lens 52 Planarization film 53 Insulating layer 54 Pixel separation section 55 Fixed charge membrane 56 Interlayer insulating film 61~68 Wiring 70, 370 Recognition System 71 RGB sensor section 72 RGB image processing section 73 IR sensor part 74 IR image processing unit 75 Recognition processing section 76 Interface section 81 Laser light source 83 Irradiation lens 84 Imaging Lens 85 System control section 86 Application Processors 100, 300 Solid-state imaging device (image sensor) 101 Pixel array section 102A RGB pixel driving circuit 102B IR pixel driving circuit 103, 303 Common signal processing circuit 103a AD conversion circuit 103A RGB signal processing circuit 103B IR signal processing circuit 104 Column driver circuit 105 System control circuit 108A RGB data processing unit 108B IR data processing unit 110, 110-1 to 110-3, 210, 310, 310-1 to 310-4 unit pixel 131, 132, 133 Switch circuits 140 pixel chip 150 circuit chips 303B ToF signal processing circuit 308B ToF data processing unit 320 ToF pixels 320A, 320B pixel circuit 373 ToF sensor part 374 Ranging processing unit 381 Light source driver 382 VCSEL 383 Sensor control unit 384 Light receiving part 901 Subject FD1, FD2, FD2A, FD2B Floating diffusion regions LD pixel drive line LD1 RGB drive line LD2 IR drive wire PD1, PD2 photoelectric conversion section VSL1, VSL2, VSL3, VSL4 Vertical signal line

Claims

1. a plurality of unit pixels arranged in a matrix; a signal processing circuit that reads out a signal from each of the unit pixels; Equipped with Each of the unit pixels is a first pixel disposed on the first surface and configured to detect light in a first wavelength band; second pixels arranged on a second surface parallel to the first surface and configured to detect light in a second wavelength band different from the first wavelength band; Equipped with the signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels and configured to convert analog signals output from the first pixel and the second pixel into digital signals, and a first switch configured to switch a connection destination of the first conversion circuit between the first pixel and the second pixel. Solid-state imaging device.

2. At least a portion of the first pixel overlaps with the second pixel in a direction perpendicular to the first surface and the second surface. The solid-state imaging device according to claim 1 .

3. The first pixel has an organic photoelectric conversion film. The solid-state imaging device according to claim 1 .

4. The signal processing circuit a second conversion circuit connected to the first pixel and the second pixel in each of the unit pixels, and configured to convert analog signals output from the first pixel and the second pixel into digital signals; a second switch that switches a connection destination of the second conversion circuit between the first pixel and the second pixel; 4. The solid-state imaging device according to claim 1, further comprising:

5. Each of the unit pixels includes a plurality of the first pixels connected to the first conversion circuit.

5. The solid-state imaging device according to claim 1.

6. Each of the first pixels comprises: a first photoelectric conversion unit that performs photoelectric conversion on incident light; a first pixel circuit that reads out charges generated in the first photoelectric conversion unit; Equipped with The plurality of first pixels share at least a portion of the first pixel circuit. The solid-state imaging device according to claim 5 .

7. Each of the unit pixels includes a plurality of the second pixels connected to the first conversion circuit.

7. The solid-state imaging device according to claim 1.

8. Each of the second pixels is a second photoelectric conversion unit that photoelectrically converts incident light; a second pixel circuit that reads out charges generated in the second photoelectric conversion unit; Equipped with The plurality of second pixels share at least a portion of the second pixel circuit. The solid-state imaging device according to claim 7 .

9. The signal processing circuit generating first image data based on signals read out from the first pixels in the plurality of unit pixels; generating second image data based on signals read out from the second pixels in the plurality of unit pixels; 9. The solid-state imaging device according to claim 1.

10. A plurality of unit pixels arranged in a matrix; a signal processing circuit that reads out a signal from each of the unit pixels; Equipped with Each of the unit pixels is a first pixel disposed on the first surface and configured to detect light in a first wavelength band; second pixels arranged on a second surface parallel to the first surface and configured to detect light in a second wavelength band different from the first wavelength band; Equipped with the signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels and configured to convert analog signals output from the first pixel and the second pixel into digital signals; The second pixel is a photoelectric conversion unit that converts incident light into an electric signal; a first pixel circuit that reads out charges generated in the photoelectric conversion unit; a second pixel circuit that reads out charges generated in the photoelectric conversion unit; Equipped with the signal processing circuit further includes a second conversion circuit connected to the first pixel circuit and converting an analog signal output from the first pixel circuit into a digital signal; The first conversion circuit is connected to the first pixel and the second pixel circuit. Solid-state imaging device.

11. The signal processing circuit further includes a first switch that switches the connection destination of the first conversion circuit between the first pixel circuit and the second pixel circuit. The solid-state imaging device according to claim 10.

12. A plurality of unit pixels arranged in a matrix; a signal processing circuit that reads out a signal from each of the unit pixels; Equipped with Each of the unit pixels is a first pixel disposed on the first surface and configured to detect light in a first wavelength band; second pixels arranged on a second surface parallel to the first surface and configured to detect light in a second wavelength band different from the first wavelength band; Equipped with the signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels and configured to convert analog signals output from the first pixel and the second pixel into digital signals; Each of the unit pixels includes a plurality of the first pixels, The second pixel is a photoelectric conversion unit that converts incident light into an electric signal; a first pixel circuit that reads out charges generated in the photoelectric conversion unit; a second pixel circuit that reads out charges generated in the photoelectric conversion unit; Equipped with the signal processing circuit further includes a second conversion circuit connected to one of the plurality of first pixels and the second pixel circuit, and configured to convert analog signals output from the one of the plurality of first pixels and the second pixel circuit into digital signals; The first conversion circuit is connected to another one of the plurality of first pixels and the first pixel circuit. Solid-state imaging device.

13. The signal processing circuit a first switch that switches a connection destination of the first conversion circuit to either the other one of the plurality of first pixels or the first pixel circuit; a second switch that switches a connection destination of the second conversion circuit between the one of the plurality of first pixels and the second pixel circuit; The solid-state imaging device according to claim 12 , further comprising:

14. The signal processing circuit a third conversion circuit connected to the other one of the plurality of first pixels and the first pixel circuit, and configured to convert analog signals output from the other one of the plurality of first pixels and the first pixel circuit into digital signals; a fourth conversion circuit connected to the one of the plurality of first pixels and the second pixel circuit, and configured to convert analog signals output from the one of the plurality of first pixels and the second pixel circuit into digital signals; a third switch that switches a connection destination of the first conversion circuit between the other one of the plurality of first pixels and the first pixel circuit; a fourth switch that switches a connection destination of the second conversion circuit between the one of the plurality of first pixels and the second pixel circuit; The solid-state imaging device according to claim 13 , further comprising:

15. the light in the first wavelength band is light included in a wavelength band of visible light, The light in the second wavelength band is light included in the wavelength band of infrared light. The solid-state imaging device according to any one of claims 1 to 14.

16. a first driving circuit that drives the first pixel in each of the unit pixels; a second driving circuit that drives the second pixel in each of the unit pixels; 16. The solid-state imaging device according to claim 1, comprising:

17. the first pixel includes a first photoelectric conversion unit that performs photoelectric conversion on incident light, the second pixel includes a second photoelectric conversion unit that performs photoelectric conversion on incident light, The first photoelectric conversion unit and the second photoelectric conversion unit are arranged along the optical axis of the incident light. The solid-state imaging device according to any one of claims 1 to 16.

18. A solid-state imaging device according to any one of claims 1 to 17, a recognition processing unit that executes recognition processing based on first detection data acquired by the first pixel and second detection data acquired by the second pixel in the solid-state imaging device; A recognition system comprising:

19. a light source that emits light in the second wavelength band; a control unit that controls the light source and the solid-state imaging device; Furthermore, The control unit controls the light emission timing of the light source and the drive timing of the second pixel in the solid-state imaging device so as to synchronize them. The recognition system of claim 18.

Images