Solid-state imaging device and identification system

Abstract

An object of the present application is to enable more secure authentication. A solid-state imaging device according to an embodiment includes a plurality of unit pixels (110) arranged in a matrix, and a signal processing circuit (103A, 103B) that reads out a signal from each unit pixel. Each unit pixel includes a first pixel (10) arranged on a first surface and detecting light of a first wavelength band, and a second pixel (20) arranged on a second surface parallel to the first surface and detecting light of a second wavelength band different from the first wavelength band. The signal processing circuit includes a first conversion circuit (103a) connected to the first pixel and the second pixel of each unit pixel and converting an analog signal output from each of the first pixel and the second pixel into a digital signal.

Description

Technical Field

[0001] This disclosure relates to a solid-state imaging device and a recognition system. Background Technology

[0002] In recent years, with the increasing popularity of portable devices such as smartphones and tablets, there is a need for secure authentication systems.

[0003] Citation List

[0004] Patent documents

[0005] Patent Document 1: JP 2020-21855A

[0006] Patent Document 2: JP 2018-125848A Summary of the Invention

[0007] Technical issues

[0008] However, since authentication systems based on information acquired by a single sensor are common in known circumstances, there is room for improvement in terms of security to prevent unauthorized access, such as identity theft.

[0009] In view of this, this disclosure proposes a solid-state imaging device and identification system that can achieve more secure authentication.

[0010] Solution to the problem

[0011] To address the aforementioned problems, a solid-state imaging device according to an embodiment of this disclosure includes: a plurality of unit pixels arranged in a matrix; and a signal processing circuit that reads signals from each of the unit pixels, wherein each of the unit pixels includes: a first pixel disposed on a first surface and detecting light of a first wavelength band; and a second pixel disposed on a second surface parallel to the first surface and detecting light of a second wavelength band different from the first wavelength band, and the signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel among the unit pixels, and converting an analog signal output from each of the first pixel and the second pixel into a digital signal. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating a schematic configuration example of an electronic device equipped with an image sensor according to the first embodiment.

[0013] Figure 2 This is a block diagram illustrating an example of the functional configuration of an identification system according to the first embodiment.

[0014] Figure 3 This is a block diagram illustrating a schematic configuration example of an image sensor according to the first embodiment.

[0015] Figure 4 This is a block diagram illustrating a schematic configuration example of an image sensor according to a variation of the first embodiment.

[0016] Figure 5 This is a schematic diagram showing a illustrative configuration example of the pixel array section according to the first embodiment.

[0017] Figure 6 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the first embodiment.

[0018] Figure 7 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a first variant of the first embodiment.

[0019] Figure 8 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a second variation of the first embodiment.

[0020] Figure 9 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a third variation of the first embodiment.

[0021] Figure 10 This is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the first embodiment.

[0022] Figure 11 This is a diagram illustrating an example of the planar layout of each layer of the pixel array section according to the first embodiment.

[0023] Figure 12 This is a plan view illustrating a wiring example for a pixel driving line for an RGB pixel according to a first embodiment.

[0024] Figure 13 This is a plan view illustrating a wiring example for a pixel driving line for an IR pixel according to a first embodiment.

[0025] Figure 14 This is a diagram illustrating an example of a stacked structure of an image sensor according to a first embodiment.

[0026] Figure 15 This is a flowchart illustrating an example of an identification operation according to the first embodiment.

[0027] Figure 16 This is a schematic diagram illustrating a illustrative configuration example of a unit pixel according to the second embodiment.

[0028] Figure 17 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the second embodiment.

[0029] Figure 18 This is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the second embodiment.

[0030] Figure 19 This is a diagram showing an example of the planar layout of each layer of the pixel array section according to the second embodiment.

[0031] Figure 20 This is a diagram showing an example of the planar layout of each layer of the pixel array section in a modified example of an on-chip lens according to the second embodiment.

[0032] Figure 21 This is a diagram showing an example of the planar layout of each layer of the pixel array section in a modified example of the color filter array according to the second embodiment.

[0033] Figure 22 This is a block diagram illustrating an example of the functional configuration of an identification system according to the third embodiment.

[0034] Figure 23 This is a schematic diagram illustrating an example of the configuration of an electronic device that implements an identification system according to the third embodiment.

[0035] Figure 24 This is a block diagram illustrating an example of the configuration of an electronic device that implements the identification system according to the third embodiment.

[0036] Figure 25 This is a block diagram illustrating a schematic configuration example of an image sensor according to a third embodiment.

[0037] Figure 26 This is a block diagram illustrating a schematic configuration example of an image sensor according to a variation of the third embodiment.

[0038] Figure 27 This is a schematic diagram illustrating a illustrative configuration example of the pixel array section according to the third embodiment.

[0039] Figure 28 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the third embodiment.

[0040] Figure 29 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a first variation of the third embodiment.

[0041] Figure 30 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a second variation of the third embodiment.

[0042] Figure 31 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a third variation of the third embodiment.

[0043] Figure 32This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a fourth variation of the third embodiment.

[0044] Figure 33 This is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the third embodiment.

[0045] Figure 34 This is a diagram illustrating an example of the planar layout of each layer of the pixel array section according to the third embodiment.

[0046] Figure 35 This is a flowchart illustrating an example of the identification operation according to the third implementation scheme.

[0047] Figure 36 This is a block diagram illustrating an example of the schematic configuration of a vehicle control system.

[0048] Figure 37 This is an example diagram illustrating the installation location of the vehicle exterior information detection unit and the imaging unit. Detailed Implementation

[0049] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. In each of the following embodiments, the same reference numerals denote the same parts, and repeated descriptions thereof will be omitted.

[0050] This disclosure will be described in the following order.

[0051] 1. First Implementation Plan

[0052] 1.1 Example of the structure of an electronic device

[0053] 1.2 Example of the functional composition of an identification system

[0054] 1.3 Example of Image Sensor Configuration

[0055] 1.3.1 Variations of Image Sensors 1.4 Examples of Unit Pixel Composition

[0056] 1.5 Example of circuit configuration per unit pixel

[0057] 1.6 Examples of circuit configuration variations

[0058] 1.6.1 First Variation Example

[0059] 1.6.2 Second Variation Example

[0060] 1.6.3 Third Variation Example

[0061] 1.7 Cross-sectional structure of a unit pixel Example 1.8 Organic materials

[0062] 1.9 Example of a planar structure

[0063] 1.10 Example of pixel driving line wiring; 1.11 Example of image sensor stack-up structure; 1.12 Example of recognition operation.

[0064] 1.13 Functions and Effects

[0065] 2. Second Implementation Plan

[0066] 2.1 Example of a unit pixel's composition

[0067] 2.2 Example of circuit configuration per unit pixel

[0068] 2.3 Example of cross-sectional structure per unit pixel

[0069] 2.4 Example of a planar structure

[0070] 2.5 Examples of on-chip lens deformation

[0071] 2.6 Variations of Color Filter Arrays

[0072] 2.7 Functions and Effects

[0073] 3. Third Implementation Plan

[0074] 3.1 Example of the functional composition of an identification system

[0075] 3.2 System Configuration Example

[0076] 3.3 Example of Image Sensor Configuration

[0077] 3.3.1 Examples of image sensor variations

[0078] 3.4 Example of a unit pixel

[0079] 3.5 unit pixel circuit configuration example

[0080] 3.6 Examples of circuit configuration variations

[0081] 3.6.1 First Variation Example

[0082] 3.6.2 Second variation

[0083] 3.6.3 Third variation example

[0084] 3.6.4 Fourth Variation Example

[0085] 3.7 Example of cross-sectional structure per unit pixel

[0086] 3.8 Example of a planar structure

[0087] 3.9 Example of Recognition Operation

[0088] 3.10 Functions and Effects

[0089] 4. Examples of applications of moving bodies

[0090] 1. First Implementation Plan

[0091] First, a solid-state imaging device (hereinafter referred to as an image sensor), electronic device, and recognition system according to a first embodiment will be described in detail with reference to the accompanying drawings. This embodiment will illustrate the application of the technology according to this embodiment to a complementary metal-oxide-semiconductor (CMOS) image sensor. However, the type of image sensor is not limited to this. For example, the technology according to this embodiment can be applied to various sensors including photoelectric conversion units, such as charge-coupled device (CCD) image sensors, time-of-flight (ToF) sensors, and event vision sensors (EVS).

[0092] 1.1 Example of the composition of electronic devices

[0093] Figure 1 This is a block diagram illustrating a schematic configuration example of an electronic device equipped with an image sensor according to the first embodiment. For example, such as Figure 1 As shown, the electronic device 1 includes an imaging lens 2, an image sensor 100, a storage unit 3, and a processor 4.

[0094] Imaging lens 2 is an example of an optical system that converges incident light and forms a converged light image on the light-receiving surface of image sensor 100. The light-receiving surface can be a surface on which photoelectric conversion units from image sensor 100 are arranged. Image sensor 100 performs photoelectric conversion of the incident light to generate image data. Furthermore, image sensor 100 performs predetermined signal processing on the generated image data, such as noise removal and white balance adjustment.

[0095] For example, the storage unit 3 includes a flash memory drive, dynamic random access memory (DRAM), static random access memory (SRAM), etc., and records image data input from the image sensor 100, etc.

[0096] For example, the processor 4 is configured using a central processing unit (CPU) and may include an application processor that executes an operating system and various types of application software, a graphics processing unit (GPU), a baseband processor, etc. The processor 4 performs various processing on image data input from the image sensor 100 and image data read from the storage unit 3 as needed, displays the data to the user, and sends the image data to the outside via a predetermined network.

[0097] In addition, the processor 4 processes the color image read from RGB pixel 10 and the monochrome image (IR image) read from IR pixel 20 as described later, thereby performing various types of processing such as distance measurement processing and recognition processing.

[0098] 1.2 Example of the functional composition of an identification system

[0099] Next, an example of the functional configuration of an identification system constructed using electronic device 1 will be described. Figure 2 This is a block diagram illustrating an example of the functional configuration of an identification system according to a first embodiment. For example... Figure 2 As shown, the recognition system 70 includes two types of sensor units: an RGB sensor unit 71 and an IR sensor unit 73. Furthermore, the recognition system 70 includes an RGB image processing unit 72, an IR image processing unit 74, a recognition processing unit 75, and an interface (I / F) unit 76.

[0100] For example, the RGB sensor unit 71 includes multiple pixels, each pixel including a color filter that transmits the wavelength components of each of the three primary colors of RGB (hereinafter referred to as an RGB pixel), and generates a color image including the color components of the three primary colors of RGB (hereinafter referred to as an RGB image). Instead of the RGB sensor unit 71, it is also permissible to use a sensor unit including multiple pixels, each pixel including a color filter that transmits the wavelength components of each of the three primary colors of CMY.

[0101] For example, the IR sensor unit 73 includes multiple pixels, each including an IR filter that transmits infrared (IR) light (hereinafter referred to as an IR pixel), and generates a monochrome image (hereinafter referred to as an IR image) that includes the color components of IR light. Instead of the IR sensor unit 73, it is also permissible to use a sensor unit or the like that includes multiple pixels, each including a color filter that transmits light in one or more other wavelength bands.

[0102] The RGB image processing unit 72 performs predetermined signal processing such as noise removal, white balance adjustment and pixel interpolation on the RGB image data input from the RGB sensor unit 71.

[0103] The IR image processing unit 74 performs predetermined signal processing such as noise removal and white balance adjustment on the IR image data input from the IR sensor unit 73.

[0104] The recognition processing unit 75 uses RGB image data input from the RGB image processing unit 72 and / or IR image data input from the IR image processing unit 74 to perform recognition processing of objects, etc., existing within the viewing angle of the RGB sensor unit 71 and / or the IR sensor unit 73. The recognition processing of the recognition processing unit 75 can use recognition processing such as pattern recognition, or recognition processing based on artificial intelligence (AI). For example, deep learning using neural networks such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) can be applied to AI recognition processing. Furthermore, the recognition processing unit 75 can perform a portion of the recognition processing and output results (intermediate data, etc.).

[0105] For example, the interface unit 76 outputs the recognition results (including intermediate data, etc.) obtained by the recognition processing unit 75 and the image data obtained by the RGB sensor unit 71 and / or the IR sensor unit 73 to the processor 4 and / or the storage unit 3, which are external devices.

[0106] Note that the RGB image processing unit 72 can perform object region determination on the RGB image data, and input information such as the address of the specified region of interest (ROI) obtained as the determination result (hereinafter referred to as ROI information) to the IR sensor unit 73 and / or the IR image processing unit 74. In response, the IR sensor unit 73 can be operated to acquire IR image data of the region corresponding to the ROI information input from the RGB image processing unit 72. Alternatively, the IR image processing unit 74 can perform processing on the IR image data input from the IR sensor unit 73, such as trimming the region corresponding to the ROI information input from the RGB image processing unit 72.

[0107] 1.3 Example of Image Sensor Configuration

[0108] Figure 3 This is a block diagram illustrating a schematic configuration example of an image sensor according to a first embodiment. For example, such as... Figure 3As shown, the image sensor 100 according to this embodiment includes a pixel array unit 101, an RGB pixel driving circuit 102A, an IR pixel driving circuit 102B, an RGB signal processing circuit 103A, an IR signal processing circuit 103B, a column driving circuit 104, a system control circuit 105, an RGB data processing unit 108A, and an IR data processing unit 108B.

[0109] For example, a pixel array unit 101, an RGB pixel driving circuit 102A, an RGB signal processing circuit 103A, a column driving circuit 104, and a system control circuit 105 are configured. Figure 2 The RGB sensor unit 71 is configured, for example, as a pixel array unit 101, an IR pixel driving circuit 102B, an IR signal processing circuit 103B, a column driving circuit 104, and a system control circuit 105. Figure 2 The IR sensor unit 73 is included. Furthermore, for example, an RGB signal processing circuit 103A and an RGB data processing unit 108A are configured. Figure 2 The RGB image processing unit 72 is configured, for example, as an IR signal processing circuit 103B and an IR data processing unit 108B. Figure 2 The IR image processing unit 74 in the middle. Figure 2 The recognition processing unit 75 can be implemented in the following ways: such as using the processor 4 alone; using the RGB data processing unit 108A and the IR data processing unit 108B in conjunction with the processor 4; or using the RGB data processing unit 108A and the IR data processing unit 108B in conjunction with each other, etc.

[0110] The pixel array section 101 has the following configuration: the unit pixels 110 are arranged in the row direction and the column direction, that is, in a two-dimensional grid pattern (also known as a matrix pattern). Here, the row direction refers to the direction in which pixels are arranged in a pixel row (the horizontal direction in the figure), and the column direction refers to the direction in which pixels are arranged in a pixel column (the vertical direction in the figure).

[0111] Each unit pixel 110 includes RGB pixels 10 and IR pixels 20. In the following description, when RGB pixels 10 and IR pixels 20 are not distinguished from each other, these pixels may be simply referred to as pixels. Each of the RGB pixels 10 and IR pixels 20 includes a photoelectric conversion unit that generates and accumulates charge according to the amount of received light, and generates a pixel signal of voltage according to the amount of incident light. The specific circuit configuration and pixel structure details of the unit pixel 110 will be described below.

[0112] The pixel array section 101 has pixel driving lines LD1 and LD2 routed in the row direction for each pixel row, and vertical signal lines VSL1 and VSL2 routed in the column direction for each pixel column relative to the pixel array in the matrix. For example, pixel driving line LD1 is connected to RGB pixels 10 in each row, while pixel driving line LD2 is connected to IR pixels 20 in each row. On the other hand, for example, vertical signal line VSL1 is connected to RGB pixels 10 in each column, while vertical signal line VSL2 is connected to IR pixels 20 in each column. However, this arrangement is not limited to this, and pixel driving lines LD1 and LD2 can be routed orthogonally to each other. Similarly, vertical signal lines VSL1 and VSL2 can be routed orthogonally to each other. For example, pixel driving line LD1 can be routed in the row direction, pixel driving line LD2 can be routed in the column direction, vertical signal line VSL1 can be routed in the column direction, and vertical signal line VSL2 can be routed in the row direction.

[0113] When a pixel signal is read from RGB pixel 10, pixel drive line LD1 transmits control signals for driving. When a pixel signal is read from IR pixel 20, pixel drive line LD2 transmits control signals for driving. Although Figure 3 The pixel driving lines LD1 and LD2 are shown as having a single-line wiring pattern, but the wiring pattern is not limited to using a single line. One end of pixel driving line LD1 is connected to the output terminal of the RGB pixel driving circuit 102A corresponding to each row, while one end of pixel driving line LD2 is connected to the output terminal of the IR pixel driving circuit 102B corresponding to each row.

[0114] Each of the RGB pixel driving circuit 102A and the IR pixel driving circuit 102B includes a shift register, an address decoder, etc., and drives each pixel of the pixel array section 101 simultaneously for all pixels or row by row. That is, the RGB pixel driving circuit 102A, together with the system control circuit 105 controlling the RGB pixel driving circuit 102A, constitutes a driving unit that controls the operation of each RGB pixel 10 of the pixel array section 101; while the IR pixel driving circuit 102B, together with the system control circuit 105 controlling the IR pixel driving circuit 102B, constitutes a driving unit that controls the operation of each IR pixel 20 of the pixel array section 101. The RGB pixel driving circuit 102A and the IR pixel driving circuit 102B typically each include two scanning systems, namely, a readout scanning system and a scanout scanning system, although their specific configurations are not shown here.

[0115] In order to read out signals from each pixel, the readout scanning system performs selective scanning of each pixel in the pixel array 101 line by line. The pixel signals read out from each pixel are analog signals. One exposure time before the readout scan, the scan system performs a scan on the readout lines for which the readout scan will be performed.

[0116] By scanning outwards using the scanning system, unwanted charges are removed from the photoelectric conversion units of each pixel in the target row, thereby resetting the photoelectric conversion units. By removing (resetting) unwanted charges in the scanning system, an electronic shutter operation is performed. Here, the electronic shutter operation refers to the operation of discarding the charges in the photoelectric conversion units and restarting exposure (starting charge accumulation).

[0117] The signal read out by the readout operation of the readout scanning system corresponds to the amount of light received immediately after the preceding readout operation or electronic shutter operation. The time interval from the readout timing of the preceding readout operation or the scan timing of the electronic shutter operation to the readout timing of the current readout operation corresponds to the charge accumulation period (also known as the exposure period) in each pixel.

[0118] Pixel signals output from each RGB pixel 10 of a pixel row selectively scanned by the RGB pixel driving circuit 102A are input to the RGB signal processing circuit 103A via each vertical signal line VSL1 for each pixel column. The RGB signal processing circuit 103A performs predetermined signal processing on the pixel signals output from each RGB pixel 10 of the selected row via the vertical signal line VSL1 for each pixel column of the pixel array section 101, and temporarily holds the pixel signals after signal processing.

[0119] Similarly, the pixel signals output by each IR pixel 20 of the pixel row selectively scanned by the IR pixel driving circuit 102B are input to the IR signal processing circuit 103B through each vertical signal line VSL2 for each pixel column. The IR signal processing circuit 103B performs predetermined signal processing on the pixel signals output from each IR pixel 20 of the selected row through the vertical signal line VSL2 for each pixel column of the pixel array section 101, and temporarily holds the pixel signals after the signal processing.

[0120] Specifically, each of the RGB signal processing circuit 103A and the IR signal processing circuit 103B performs signal processing including at least noise removal processing (e.g., correlated double sampling (CDS) processing or double data sampling (DDS) processing). For example, CDS processing removes fixed pattern noise inherent in pixels, such as reset noise and threshold variations of amplifying transistors in pixels. For example, each of the RGB signal processing circuit 103A and the IR signal processing circuit 103B also includes an analog-to-digital (AD) conversion function, converting the analog pixel signal obtained by reading from the photoelectric conversion unit into a digital signal and outputting the obtained digital signal.

[0121] The column driver circuit 104 includes a shift register, an address decoder, etc., and sequentially selects the readout circuits (hereinafter referred to as pixel circuits) corresponding to the pixel columns of the RGB signal processing circuit 103A and the IR signal processing circuit 103B. Through the selective scanning of the column driver circuit 104, the pixel signals that have undergone signal processing for each pixel circuit in the RGB signal processing circuit 103A and the IR signal processing circuit 103B are sequentially output.

[0122] The system control circuit 105 includes a timing generator that generates various timing signals, and executes drive control of the RGB pixel drive circuit 102A, IR pixel drive circuit 102B, RGB signal processing circuit 103A, IR signal processing circuit 103B, column drive circuit 104, etc., based on the various timing signals generated by the timing generator.

[0123] Each of the RGB data processing unit 108A and the IR data processing unit 108B has at least arithmetic processing function, and performs various signal processing such as arithmetic processing on the image signal output from the RGB signal processing circuit 103A or the IR signal processing circuit 103B.

[0124] For example, the image data output from the RGB data processing unit 108A and the IR data processing unit 108B can be processed in a predetermined manner in the processor 4 or the like in the electronic device 1 equipped with the image sensor 100, or can be sent to the outside via a predetermined network.

[0125] Note that the image sensor 100 may include a storage unit for temporarily storing data required for signal processing in the RGB data processing unit 108A and the IR data processing unit 108B, as well as data processed by any one or more of the RGB signal processing circuit 103A, the IR signal processing circuit 103B, the RGB data processing unit 108A, and the IR data processing unit 108B.

[0126] 1.3.1 Variations of Image Sensors

[0127] Figure 4 This is a block diagram illustrating a schematic configuration example of an image sensor according to a modified example of the first embodiment. (The above...) Figure 3 This refers to the case where separate signal processing circuits (RGB signal processing circuit 103A and IR signal processing circuit 103B) are provided for each of the RGB pixels 10 and IR pixels 20. However, the arrangement of the signal processing circuits is not limited to this, and it is also permissible to have circuits such as... Figure 4 The diagram shows a configuration where a common signal processing circuit 103 is provided for both RGB pixels 10 and IR pixels 20. In this configuration, the reading of pixel signals from RGB pixels 10 and IR pixels 20 can be performed in a time-division manner or in parallel.

[0128] 1.4 Example of a unit pixel

[0129] Here, an example of the configuration of unit pixel 110 will be described. The following is an exemplary case where unit pixel 110 includes RGB pixel 10 for acquiring an RGB image of the RGB three primary colors and IR pixel 20 for acquiring an IR image of infrared (IR) light. Figure 5 In the following description, when the color filters 31r, 31g, and 31b that transmit light through the individual color components that constitute the RGB primary colors are not distinguished, the filter is indicated by reference numeral 31.

[0130] Figure 5 This is a schematic diagram illustrating a illustrative configuration example of the pixel array section according to the first embodiment. For example... Figure 5 As shown, the pixel array 101 is formed of unit pixels 110, each unit pixel 110 having a structure in which RGB pixels 10 and IR pixels 20 are arranged in the light incident direction, and the pixel array 101 has a configuration in which the unit pixels 110 are arranged in a two-dimensional grid pattern. That is, this embodiment uses the following configuration: the RGB pixels 10 and IR pixels 20 are located in a direction perpendicular to the arrangement direction (planar direction) of the unit pixels 110, and the light transmitted by the RGB pixel 10 located upstream in the light path of the incident light is incident on the IR pixel 20 located downstream of the RGB pixel 10. With this configuration, the photoelectric conversion section PD2 of the IR pixel 20 is arranged on a surface side opposite to the incident light incident surface of the photoelectric conversion section PD1 of the RGB pixel 10. Therefore, in this embodiment, the optical axes of the incident light on the RGB pixels 10 arranged in the light incident direction and the optical axes of the incident light on the IR pixel 20 are completely or substantially aligned with each other.

[0131] While this embodiment is an exemplary case where the photoelectric conversion unit PD1 constituting the RGB pixel 10 is formed of an organic material and the photoelectric conversion unit PD2 constituting the IR pixel 20 is formed of a semiconductor material such as silicon, the materials are not limited thereto. For example, both photoelectric conversion units PD1 and PD2 may be formed of semiconductor materials, both photoelectric conversion units PD1 and PD2 may be formed of organic materials, or photoelectric conversion unit PD1 may be formed of a semiconductor material while photoelectric conversion unit PD2 may be formed of an organic material. Alternatively, at least one of the photoelectric conversion units PD1 and PD2 may be formed of a photoelectric conversion material that is neither an organic material nor a semiconductor material.

[0132] Example of circuit configuration for 1.5 unit pixels

[0133] Next, an example of the circuit configuration for a unit pixel 110 will be explained. Figure 6 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the first embodiment. For example... Figure 6 As shown, unit pixel 110 includes an RGB pixel 10 and an IR pixel 20.

[0134] (10 RGB pixels)

[0135] For example, RGB pixel 10 includes photoelectric conversion unit PD1, transmission gate 11, floating diffusion region FD1, reset transistor 12, amplification transistor 13 and selection transistor 14.

[0136] The gate of select transistor 14 is connected to the select control line included in pixel drive line LD1; the gate of reset transistor 12 is connected to the reset control line included in pixel drive line LD1; and the storage electrode of transmission gate 11 (see later description) Figure 10 The storage electrode 37) is connected to the transmission control line included in the pixel drive line LD1. In addition, the drain of the amplifying transistor 13 is connected to the vertical signal line VSL1 via the select transistor 14; one end of the vertical signal line VSL1 is connected to the RGB signal processing circuit 103A.

[0137] In the following description, reset transistor 12, amplification transistor 13, and selection transistor 14 are collectively referred to as pixel circuitry. Pixel circuitry may include floating diffusion region FD1 and / or transmission gate 11.

[0138] For example, the photoelectric conversion unit PD1 is formed of an organic material and performs photoelectric conversion of incident light. The transmission gate 11 transmits the charge generated in the photoelectric conversion unit PD1. The floating diffusion region FD1 accumulates the charge transmitted by the transmission gate 11. The amplification transistor 13 causes a pixel signal having a voltage value corresponding to the charge accumulated in the floating diffusion region FD1 to appear in the vertical signal line VSL1. The reset transistor 12 releases the charge accumulated in the floating diffusion region FD1. The selection transistor 14 selects the RGB pixel 10 as the readout target.

[0139] The anode of the photoelectric conversion unit PD1 is grounded, while the cathode is connected to the transmission gate 11. For example, the transmission gate 11 includes a storage electrode 37 and a readout electrode 36. (Refer to the following...) Figure 10 The details of transmission gate 11 are explained in detail. During exposure, a voltage for collecting the charge generated in the photoelectric conversion unit PD1 into the semiconductor layer 35 near the storage electrode 37 is applied to the storage electrode 37 via a transmission control line. During readout, a voltage for releasing the charge collected in the semiconductor layer 35 near the storage electrode 37 via the readout electrode 36 is applied to the storage electrode 37 via the transmission control line.

[0140] The charge released via the readout electrode 36 accumulates in the floating diffusion region FD1, which has a wiring structure connecting the readout electrode 36, the source of the reset transistor 12, and the gate of the amplification transistor 13. Note that the drain of the reset transistor 12 can be connected to the power supply voltage VDD, or to a power supply line that supplies it with a reset voltage lower than the power supply voltage VDD.

[0141] The source of amplifying transistor 13 can be connected to a power supply line via, for example, a constant current circuit (not shown). The drain of amplifying transistor 13 is connected to the source of select transistor 14, and the drain of select transistor 14 is connected to the vertical signal line VSL1.

[0142] The floating diffusion region FD1 converts the accumulated charge into a voltage value corresponding to the amount of charge. For example, the floating diffusion region FD1 can be a grounded capacitor. However, the configuration is not limited to this. The floating diffusion region FD1 can be a capacitance increased by intentionally connecting capacitors or the like to the node that connects the drain of the transmission gate 11, the source of the reset transistor 12, and the gate of the amplification transistor 13 to each other.

[0143] Vertical signal lines VSL1 are connected to analog-to-digital (AD) conversion circuits 103a configured for each column (i.e., for each vertical signal line VSL1) in the RGB signal processing circuit 103A. For example, the AD conversion circuit 103a includes a comparator and a counter, and converts analog pixel signals into digital pixel signals by comparing a reference voltage with a shape such as a single slope or ramp shape, input from an external reference voltage generation circuit (digital-to-analog converter (DAC)), with the pixel signals appearing in the vertical signal lines VSL1. For example, the AD conversion circuit 103a can be configured to reduce kTC noise, etc., by incorporating correlated double sampling (CDS) circuitry.

[0144] (IR pixel 20)

[0145] For example, the IR pixel 20 includes a photoelectric conversion unit PD2, a transmission transistor 21, a floating diffusion region FD2, a reset transistor 22, an amplification transistor 23, a selection transistor 24, and an exhaust transistor 25. That is, the IR pixel 20 has the following configuration: the transmission gate 11 in the RGB pixel 10 has been replaced by the transmission transistor 21, and an exhaust transistor 25 has been added.

[0146] The connection relationship between the floating diffusion region FD2, the reset transistor 22, and the amplification transistor 23 relative to the transmission transistor 21 can be similar to the connection relationship between the floating diffusion region FD1, the reset transistor 12, and the amplification transistor 13 relative to the transmission gate 11 in the RGB pixel 10. Furthermore, the connection relationship between the amplification transistor 23, the selection transistor 24, and the vertical signal line VSL2 can be similar to the connection relationship between the amplification transistor 13, the selection transistor 14, and the vertical signal line VSL1 in the RGB pixel 10.

[0147] For example, the source of the transmission transistor 21 is connected to the cathode of the photoelectric conversion unit PD2, and the drain is connected to the floating diffusion region FD2. Furthermore, the gate of the transmission transistor 21 is connected to the transmission control line included in the pixel driving line LD2.

[0148] For example, the source of the discharge transistor 25 can be connected to the cathode of the photoelectric conversion unit PD2, and the drain can be connected to the power supply voltage VDD or a power supply line that supplies a reset voltage lower than the power supply voltage VDD. Furthermore, the gate of the discharge transistor 25 is connected to the discharge control line included in the pixel drive line LD2.

[0149] In the following description, reset transistor 22, amplification transistor 23, and selection transistor 24 are also collectively referred to as pixel circuitry. Pixel circuitry may include one or more of the following: floating diffusion region FD2, transfer transistor 21, and discharge transistor 25.

[0150] For example, the photoelectric conversion unit PD2 is formed of a semiconductor material and performs photoelectric conversion of incident light. The transfer transistor 21 transfers the charge generated in the photoelectric conversion unit PD2. The floating diffusion region FD2 accumulates the charge transferred by the transfer transistor 21. The amplification transistor 23 causes a pixel signal having a voltage value corresponding to the charge accumulated in the floating diffusion region FD2 to appear in the vertical signal line VSL2. The reset transistor 22 releases the charge accumulated in the floating diffusion region FD2. The selection transistor 24 selects the IR pixel 20 as the readout target.

[0151] The anode of the photoelectric conversion unit PD2 is grounded, and the cathode is connected to the transmission transistor 21. The drain of the transmission transistor 21 is connected to the source of the reset transistor 22 and the gate of the amplification transistor 23. The wiring structure connecting these transistors forms the floating diffusion region FD2. The charge released from the photoelectric conversion unit PD2 via the transmission transistor 21 accumulates in the floating diffusion region FD2.

[0152] The floating diffusion region FD2 converts the accumulated charge into a voltage value corresponding to the amount of charge. For example, the floating diffusion region FD2 can be a grounded capacitor. However, the configuration is not limited to this, and the floating diffusion region FD2 can be a capacitance increased by intentionally connecting capacitors or the like to the node that connects the drain of the transmission transistor 21, the source of the reset transistor 22, and the gate of the amplification transistor 23 to each other.

[0153] When the accumulated charge in the photoelectric converter PD2 is discharged and the photoelectric converter PD2 is reset, the discharge transistor 25 is turned on. Through this operation, the accumulated charge in the photoelectric converter PD2 is released to the power line via the discharge transistor 25, and the photoelectric converter PD2 is reset to the unexposed state.

[0154] Similar to the vertical signal line VSL1, the vertical signal line VSL2 is connected to the AD conversion circuit 103a provided for each column (i.e., for each vertical signal line VSL2) in the IR signal processing circuit 103B.

[0155] 1.6 Examples of circuit configuration variations

[0156] The following examples will illustrate this. Figure 6 The circuit configuration of the unit pixel 110 shown is a modified example.

[0157] 1.6.1 First Variation Example

[0158] Figure 7 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a first variation of the first embodiment. For example... Figure 7 As shown, unit pixel 110-1 has the same Figure 6 The configuration of the unit pixel 110 shown is similar, wherein the vertical signal lines VSL1 and VSL2 are connected to a common AD conversion circuit 103a. Therefore, the first variation includes a switching circuit 131 that switches the vertical signal lines connected to the AD conversion circuit 103a to either vertical signal line VSL1 or VSL2. For example, the switching circuit 131 may be included in the RGB signal processing circuit 103A and the IR signal processing circuit 103B or in a common signal processing circuit 103. Furthermore, for example, the switching circuit 131 may be disposed on the same semiconductor substrate as the pixel circuits of the RGB pixel 10 and / or the IR pixel 20, on a semiconductor substrate on which the signal processing circuits are disposed, or on a different semiconductor substrate. Furthermore, the control signal for controlling the switching circuit 131 may be supplied from the RGB pixel driving circuit 102A or the IR pixel driving circuit 102B, from the column driving circuit 104, or from another configuration (e.g., Figure 1 The processor 4, etc., is supplied.

[0159] With this configuration, the RGB signal processing circuit 103A and the IR signal processing circuit 103B can be replaced by a common signal processing circuit, which reduces the circuit size and thus leads to miniaturization of the image sensor 100 and higher resolution due to improved area efficiency.

[0160] 1.6.2 Second Variation Example

[0161] Figure 8 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a second variation of the first embodiment. For example... Figure 8 As shown, unit pixel 110-2 has the same Figure 6The configuration of the unit pixel 110 shown is similar, wherein each of the vertical signal lines VSL1 and VSL2 can be connected to either of the two common AD conversion circuits 103a. Therefore, the second variation includes: a switching circuit 132 that switches the vertical signal lines connected to the AD conversion circuit 103a to either of the vertical signal lines VSL1 and VSL2; and a switching circuit 133 that switches the vertical signal lines connected to the AD conversion circuit 103a to either of the vertical signal lines VSL1 and VSL2. For example, switching circuits 132 and 133 can be included in the RGB signal processing circuit 103A and the IR signal processing circuit 103B or in the common signal processing circuit 103. Furthermore, for example, switching circuits 132 and 133 can be disposed on the same semiconductor substrate as the pixel circuits of the RGB pixel 10 and / or the IR pixel 20, on a semiconductor substrate on which the RGB signal processing circuit 103A and the IR signal processing circuit 103B are disposed, or on a different semiconductor substrate. Furthermore, the control signals used to control the switching circuits 132 and 133 can be supplied from the RGB pixel driving circuit 102A or the IR pixel driving circuit 102B, from the column driving circuit 104, or from another configuration (e.g., Figure 1 The processor 4, etc., is supplied.

[0162] For example, with this configuration, since the AD conversion circuit 103a for each column can be selected from multiple AD conversion circuits (two AD conversion circuits in this example), the image quality degradation caused by noise such as stripes can be suppressed.

[0163] 1.6.3 Third Variation Example

[0164] Figure 9 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a third variation of the first embodiment. For example... Figure 9 As shown, regarding RGB pixel 10, unit pixel 110-3 has the same... Figure 8 The unit pixel 110-2 shown has a similar pixel-sharing circuit structure, in which multiple RGB pixels 10-1 to 10-N (N is an integer greater than 2) share a floating diffusion region FD1, a reset transistor 12, an amplification transistor 13, and a selection transistor 14. Furthermore, in a similar manner with respect to IR pixel 20, unit pixel 110-3 has a similar... Figure 8The unit pixel 110-2 shown has a similar pixel-sharing circuit structure, in which multiple IR pixels 20-1 to 20-N share a floating diffusion region FD2, a reset transistor 22, an amplification transistor 23, and a selection transistor 24. The number of RGB pixels 10-1 to 10-N is not necessarily the same as the number of IR pixels 20-1 to 20-N.

[0165] This configuration allows for switching between high dynamic range (HDR) and low dynamic range (LDR) readout as needed, thereby suppressing image quality degradation in low or high light conditions. Although this description uses reference... Figure 8 The second variation described herein is a basic exemplary case, but the configuration is not limited thereto, and Figure 6 and Figure 7 The unit pixel 110 and unit pixel 110-1 shown can be used as a basis.

[0166] 1.7 Example of cross-sectional structure per unit pixel

[0167] Next, we will refer to Figure 10 An example illustrating the cross-sectional structure of the image sensor 100 according to the first embodiment. Figure 10 This is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the first embodiment. Here, the cross-sectional structure example will be described in focus on the semiconductor chip in which photoelectric conversion units PD1 and PD2 are formed in the unit pixel 110.

[0168] Additionally, the following description is an example of a cross-sectional structure for a back-illuminated device, wherein the light incident surface is located on the back side of the semiconductor substrate 50 (opposite to the element forming surface). However, the type is not limited to this, and it can also be a cross-sectional structure for a front-illuminated device, wherein the light incident surface is located on the front surface side of the semiconductor substrate 50 (the element forming surface side). Furthermore, although this description is an exemplary case of organic materials being used in the photoelectric conversion section PD1 of the RGB pixel 10, as described above, it is also permissible to use one or both of organic materials and semiconductor materials (also referred to as inorganic materials) as the photoelectric conversion material for each of the photoelectric conversion sections PD1 and PD2.

[0169] When semiconductor materials are used as photoelectric conversion materials for photoelectric conversion section PD1 and photoelectric conversion section PD2, the image sensor 100 may have a cross-sectional structure in which photoelectric conversion section PD1 and photoelectric conversion section PD2 are formed in the same semiconductor substrate 50, or a cross-sectional structure in which the semiconductor substrate in which photoelectric conversion section PD1 is formed and the semiconductor substrate in which photoelectric conversion section PD2 is formed are bonded to each other, or a cross-sectional structure in which one of photoelectric conversion section PD1 and PD2 is formed in the semiconductor substrate 50 and the other is formed in a semiconductor layer formed on the back or front surface of the semiconductor substrate 50.

[0170] like Figure 10 As shown, this embodiment has the following structure: the photoelectric conversion unit PD2 of the IR pixel 20 is formed on the semiconductor substrate 50, while the photoelectric conversion unit PD1 of the RGB pixel 10 is disposed on the back side of the semiconductor substrate 50 (opposite to the element formation surface). Figure 10 In the figure, for ease of explanation, the back surface of the semiconductor substrate 50 is located at the top, while the front surface is located at the bottom.

[0171] For example, the semiconductor substrate 50 can be formed using a semiconductor material such as silicon (Si). However, the semiconductor material is not limited to this, and various semiconductor materials, including compound semiconductors such as GaAs, InGaAs, InP, AlGaAs, InGaP, AlGaInP, and InGaAsP, are permitted to be used.

[0172] (10 RGB pixels)

[0173] The photoelectric conversion unit PD1 of the RGB pixel 10 is disposed on the back side of the semiconductor substrate 50 through an insulating layer 53. For example, the photoelectric conversion unit PD1 includes a photoelectric conversion film 34 formed of an organic material and a transparent electrode 33 and a semiconductor layer 35 disposed through the photoelectric conversion film 34. For example, the transparent electrode 33 disposed on the upper side of the photoelectric conversion film 34 in the figure (hereinafter, the upper side is described as the upper surface side and the lower side is described as the lower surface side) serves as the anode of the photoelectric conversion unit PD1, and the semiconductor layer 35 disposed on the lower surface side serves as the cathode of the photoelectric conversion unit PD1.

[0174] The semiconductor layer 35, serving as the cathode, is electrically connected to a readout electrode 36 formed in the insulating layer 53. The readout electrode 36 is connected to wirings 61, 62, 63, and 64 that penetrate the insulating layer 53 and the semiconductor substrate 50, thereby electrically extending to the front (lower) surface side of the semiconductor substrate 50. Although in Figure 10 Not shown, but wiring 64 is electrically connected to Figure 6 The floating diffusion region FD1 is shown.

[0175] A storage electrode 37, separated from the cathode by an insulating layer 53, is provided on the lower surface side of the semiconductor layer 35. Although in Figure 10 Although not shown, the storage electrode 37 is connected to the transmission control line in the pixel drive line LD1. As described above, during exposure, a voltage is applied to the storage electrode 37 to collect the charge generated in the photoelectric conversion unit PD1 into the semiconductor layer 35 near the storage electrode 37, and during readout, a voltage is applied to the storage electrode 37 to release the charge collected in the semiconductor layer 35 near the storage electrode 37 via the readout electrode 36.

[0176] Similar to the transparent electrode 33, the readout electrode 36 and the storage electrode 37 can be transparent conductive films. For example, the transparent electrode 33, the readout electrode 36, and the storage electrode 37 can be formed using transparent conductive films made of indium tin oxide (ITO) or indium zinc oxide (IZO). However, the conductive film is not limited to these, and various conductive films can be used, as long as the film is a conductive film that allows light beams in the wavelength range detected by the photoelectric conversion unit PD2 to pass through.

[0177] Furthermore, for example, semiconductor layer 35 can be a transparent semiconductor layer such as IGZO. However, the semiconductor layer is not limited to this, and various semiconductor layers can be used, as long as the layer is a semiconductor layer that allows light beams of the wavelength band detected by the photoelectric conversion unit PD2 to pass through.

[0178] Furthermore, the insulating layer 53 can be an insulating film such as a silicon oxide film (SiO2) or a silicon nitride film (SiN). However, the insulating film is not limited to this, and various insulating films can be used, as long as the film is an insulating film that allows the light beam in the band detected by the photoelectric conversion unit PD2 to pass through.

[0179] A color filter 31 is provided on the upper surface side of the transparent electrode 33, which serves as the anode, through a sealing film 32. For example, the sealing film 32 is formed of an insulating material such as silicon nitride (SiN) and may include atoms of aluminum (Al), titanium (Ti), etc., in order to prevent these atoms from diffusing from the transparent electrode 33.

[0180] For example, a color filter 31 is provided for an RGB pixel 10 that selectively transmits a light beam of a specific wavelength component, although the arrangement of the color filter 31 will be described below. However, when a monochrome pixel that acquires luminance information is provided instead of an RGB pixel 10 that acquires color information, the color filter 31 can be omitted.

[0181] (IR pixel 20)

[0182] The photoelectric conversion unit PD2 of the IR pixel 20 includes: a p-type semiconductor region 43 formed in the p-well region 42 in the semiconductor substrate 50; and an n-type semiconductor region 44 formed near the center of the p-type semiconductor region 43. For example, the n-type semiconductor region 44 serves as a charge accumulation region for accumulating the charge (electrons) generated by photoelectric conversion, while the p-type semiconductor region 43 serves as a region for forming a potential gradient for collecting the charge generated by photoelectric conversion into the n-type semiconductor region 44.

[0183] For example, an IR filter 41 that selectively transmits IR light is provided on the light incident surface side of the photoelectric conversion unit PD2. For example, the IR filter 41 can be disposed in an insulating layer 53 disposed on the back side of the semiconductor substrate 50. By arranging the IR filter 41 on the light incident surface of the photoelectric conversion unit PD2, visible light incident on the photoelectric conversion unit PD2 can be suppressed, thereby improving the signal-to-noise ratio (S / N) of IR light versus visible light. This allows for more accurate detection results of IR light.

[0184] For example, a finely textured uneven structure is provided on the light-incident surface of the semiconductor substrate 50 to suppress the reflection of incident light (IR light in this example). This uneven structure can be a structure called a moth-eye structure, or it can be an uneven structure that differs from a moth-eye structure in size or spacing.

[0185] On the front surface (lower surface in the figure) side of the semiconductor substrate 50, i.e., the element forming surface side, a vertical transistor 45 serving as a transmission transistor 21 and a floating diffusion region FD2 serving as a charge accumulation section are provided. The gate electrode of the vertical transistor 45 extends from the front surface of the semiconductor substrate 50 to the n-type semiconductor region 44 and is connected to the IR pixel driving circuit 102B via wirings 65 and 66 (part of the transmission control line of the pixel driving line LD2) formed in the interlayer insulating film 56.

[0186] The charge released by the vertical transistor 45 accumulates in the floating diffusion region FD2. The floating diffusion region FD2 is connected to the source of the reset transistor 22 and the gate of the amplification transistor 23 via wirings 67 and 68 formed in the interlayer insulating film 56. Note that the reset transistor 22, the amplification transistor 23, and the selection transistor 24 may be disposed on the device formation surface of the semiconductor substrate 50, or may be disposed on a semiconductor substrate different from the semiconductor substrate 50.

[0187] although Figure 10An exemplary case is shown in which two vertical transistors 45 (transmission transistors 21) are provided for a photoelectric conversion unit PD2, but the number is not limited to this, and one vertical transistor 45 or more vertical transistors 45 may be provided. Similarly, although the illustration shows an exemplary case in which two floating diffusion regions FD2 are provided for a photoelectric conversion unit PD2, the number is not limited to this, and one floating diffusion region FD2 or more floating diffusion regions FD2 may be provided.

[0188] (Pixel isolation structure)

[0189] The semiconductor substrate 50 has pixel isolation sections 54 that electrically isolate multiple unit pixels 110 from each other. Photoelectric conversion sections PD2 are disposed in each region divided by the pixel isolation sections 54. For example, when the image sensor 100 is viewed from the back side (upper surface in the figure) of the semiconductor substrate 50, the pixel isolation sections 54 have, for example, a grid shape inserted between the multiple unit pixels 110. Each photoelectric conversion section PD2 is formed in each region divided by the pixel isolation sections 54.

[0190] For example, the pixel isolation section 54 can be a reflective film made of a material such as tungsten (W) or aluminum (Al) that reflects light. This allows incident light entering the photoelectric conversion section PD2 to be reflected through the pixel isolation section 54, thereby extending the optical path length of the incident light in the photoelectric conversion section PD2. Furthermore, since the pixel isolation section 54 has a light-reflecting structure, light leakage to adjacent pixels can be reduced, thereby further improving image quality, distance measurement accuracy, etc. Note that the configuration of the pixel isolation section 54 having a light-reflecting structure is not limited to using a reflective film, and can be implemented, for example, by using a material having a refractive index different from that of the semiconductor substrate 50 in the pixel isolation section 54.

[0191] For example, a fixed charge film 55 is provided between the semiconductor substrate 50 and the pixel isolation portion 54. The fixed charge film 55 is formed using, for example, a high dielectric material having a negative fixed charge, to form a positive charge (hole) accumulation region at the interface with the semiconductor substrate 50, thereby suppressing the generation of dark current. Since the fixed charge film 55 is formed to have a negative fixed charge, an electric field is applied to the interface with the semiconductor substrate 50 through the negative fixed charge, thereby forming a positive charge (hole) accumulation region.

[0192] For example, the fixed charge film 55 can be formed from a hafnium oxide film (HfO2 film). Alternatively, the fixed charge film 55 can be formed to include at least one oxide of materials such as hafnium, zirconium, aluminum, tantalum, titanium, magnesium, yttrium, and lanthanides.

[0193] Figure 10An exemplary case is shown in which the pixel isolation portion 54 has a structure called full trench isolation (FTI) extending from the front surface to the back surface of the semiconductor substrate 50. However, this structure is not limited to this, and various element isolation structures, such as a structure called deep trench isolation (DTI) in which the pixel isolation portion 54 is formed from the back surface or front surface of the semiconductor substrate 50 to near the middle of the semiconductor substrate 50, can be used.

[0194] (Pupil correction)

[0195] On the upper surface of the color filter 31, a planarization film 52 formed of a silicon oxide film, a silicon nitride film, or the like is provided. For example, the upper surface of the planarization film 52 is planarized by chemical mechanical polishing (CMP). On the planarized upper surface, an on-chip lens 51 is provided for each unit pixel 110. The on-chip lens 51 of each unit pixel 100 has a curvature designed to collect incident light into photoelectric conversion units PD1 and PD2. For example, the positional relationship between the on-chip lens 51, color filter 31, IR filter 41, and photoelectric conversion unit PD2 in each unit pixel 110 can be adjusted (pupil correction) according to the distance from the center of the pixel array unit 101 (image height).

[0196] also, Figure 10 The illustrated structure may include a light-shielding film to prevent obliquely incident light from leaking into adjacent pixels. The light-shielding film may be located above the pixel isolation portion 54 disposed inside the semiconductor substrate 50 (upstream of the incident light's optical path). However, for example, when performing pupil correction, the position of the light-shielding film can be adjusted according to the distance from the center of the pixel array portion 101 (image height). For example, such a light-shielding film may be disposed within the sealing film 32 or the planarization film 52. Furthermore, for example, the material of the light-shielding film may be a light-shielding material such as aluminum (Al) or tungsten (W).

[0197] 1.8 Organic Materials

[0198] In the first embodiment, when an organic semiconductor is used as the material of the photoelectric conversion film 34, the layer structure of the photoelectric conversion film 34 can be as follows. Note that in the case of a stacked structure, the stacking order can be appropriately changed.

[0199] (1) Monolayer structure of p-type organic semiconductors

[0200] (2) Monolayer structure of n-type organic semiconductors

[0201] (3-1) Stacked structure of p-type organic semiconductor layer / n-type organic semiconductor layer

[0202] (3-2) p-type organic semiconductor layer / hybrid layer of p-type and n-type organic semiconductors (bulk heterostructure) / stacked structure of n-type organic semiconductor layer

[0203] (3-3) Stacked structure of p-type organic semiconductor layer / hybrid layer of p-type organic semiconductor and n-type organic semiconductor (bulk heterostructure)

[0204] (3-4) Stacked structure of n-type organic semiconductor layer / hybrid layer of p-type and n-type organic semiconductor (bulk heterostructure)

[0205] (4) A hybrid layer of p-type organic semiconductors and p-type organic semiconductors (bulk heterostructure)

[0206] Examples of p-type organic semiconductors include naphthalene derivatives, anthracene derivatives, phenanthrene derivatives, pyrene derivatives, perylene derivatives, tetraphenylene derivatives, pentaphenylene derivatives, quinacridone derivatives, thiophene derivatives, thiophene-thiophene derivatives, benzothiophene derivatives, benzothiophene-benzothiophene derivatives, triallylamine derivatives, carbazole derivatives, and styrene derivatives. Derivatives, fluoranthene derivatives, phthalocyanine derivatives, phthalocyanine derivatives, porphyrin derivatives, metal complexes with heterocyclic compounds as ligands, polythiophene derivatives, polybenzothiadiazole derivatives, and polyfluorene derivatives.

[0207] Examples of n-type organic semiconductors include fullerenes and fullerene derivatives (e.g., fullerenes such as C60, C70, and C74 (higher-order fullerenes, endohedral fullerenes, etc.) or fullerene derivatives (e.g., fullerene fluorides, PCBM fullerene compounds, and fullerene polymers)), organic semiconductors with larger (deeper) HOMO and LUMO than p-type organic semiconductors, and transparent inorganic metal oxides.

[0208] More specific examples of n-type organic semiconductors include: materials comprising heterocyclic compounds containing nitrogen, oxygen, or sulfur atoms, such as organic molecules having pyridine derivatives, pyrazine derivatives, pyrimidine derivatives, triazine derivatives, quinoline derivatives, quinoxaline derivatives, isoquinoline derivatives, acridine derivatives, phenazine derivatives, phenanthroline derivatives, tetrazolium derivatives, pyrazole derivatives, imidazole derivatives, thiazole derivatives, oxazole derivatives, benzimidazole derivatives, benzotriazole derivatives, benzoxazole derivatives, carbazole derivatives, benzofuran derivatives, dibenzofuran derivatives, porphyrin derivatives, polyphenylenevinylene derivatives, polybenzothiadiazole derivatives, polyfluorene derivatives, etc., as part of their molecular backbone; organometallic complexes; and phthalocyanine derivatives.

[0209] Additionally, examples of groups contained in fullerene derivatives include: halogen atoms; straight-chain, branched, or cycloalkyl or phenyl groups; groups having straight-chain or fused aromatic compounds; groups having halides; partially fluoroalkyl groups; perfluoroalkyl groups; silylalkyl group; silylalkoxy group; arylsilyl group; arylsulfanyl group; alkylsulfanyl group; arylsulfonyl group; alkylsulfonyl group; arylsulfide; alkylsulfide; amino; alkylamino; arylamino; hydroxyl; alkoxy; amide; acyloxy; carbonyl; carboxyl; carboxamide group; carboalkoxy group; acyl; sulfonyl; cyano; nitro; groups having chalcogenides; phosphine group; phosphone group; or derivatives thereof.

[0210] Examples of the film thickness of the photoelectric conversion film 34 formed from organic materials as described above may include, but are not limited to, the following values: 1 × 10⁻⁶. -8 m (meters) to 5×10 -7 m, preferably 2.5 × 10 -8 m to 3×10 -7 m, more preferably 2.5 × 10 m -8 Up to 2×10 - 7 m, and preferably 1×10 -7 m to 1.8×10 -7 Organic semiconductors are generally classified into p-type and n-type organic semiconductors. However, p-type implies easy transport of holes, and n-type implies easy transport of electrons. Therefore, organic semiconductors are not limited to the explanation that holes or electrons are the thermally excited primary charge carriers, as in inorganic semiconductors.

[0211] Examples of materials constituting the photoelectric conversion film 34 that performs photoelectric conversion of light at green wavelengths include rhodamine dye, melacyanine dye, quinacridone derivatives, and phthalocyanine dyes (phthalocyanine derivatives).

[0212] In addition, examples of materials constituting the photoelectric conversion film 34 that performs photoelectric conversion of blue light include coumaric acid dye, aluminum tris-8-hydroxyquinoline (Alq3), and cyanine dye.

[0213] Furthermore, examples of materials constituting the photoelectric conversion film 34 that performs photoelectric conversion of red light include phthalocyanine dyes and subphthalocyanine dyes (subphthalocyanine derivatives).

[0214] In addition, the photoelectric conversion film 34 can be a panchromatic photosensitive organic photoelectric conversion film that is sensitive to virtually all ranges of visible light, from the ultraviolet region to the red region.

[0215] 1.9 Example of a planar structure

[0216] Next, an example of the planar structure of the pixel array section according to this embodiment will be described. Figure 11 This is a diagram showing examples of the planar layout of each layer of the pixel array section according to the first embodiment, wherein (A) shows an example of the planar layout of the on-chip lens 51, (B) shows an example of the planar layout of the color filter 31, (C) shows an example of the planar layout of the storage electrode 37, and (D) shows an example of the planar layout of the photoelectric conversion section PD2. Figure 11 In the diagram, (A) to (D) show examples of planar layouts of surfaces parallel to the element forming surface of the semiconductor substrate 50. This description uses the following exemplary case: a 2×2 pixel Bayer array is used as the unit array, which includes pixels that selectively detect the red (R) wavelength component (hereinafter referred to as R pixel 10r), pixels that selectively detect the green (G) wavelength component (hereinafter referred to as G pixel 10g), and pixels that selectively detect the blue (B) wavelength component (hereinafter referred to as B pixel 10b).

[0217] like Figure 11 As shown in (A) to (D), this embodiment uses a configuration in which an on-chip lens 51, a color filter 31, a storage electrode 37, and a photoelectric conversion unit PD2 are provided for each unit pixel 110. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one IR pixel 20.

[0218] In this way, by arranging an RGB pixel 10 and an IR pixel 20 in a single unit pixel 110 along the direction of incident light travel, the coaxiality of the RGB pixel 10 and IR pixel 20 with respect to the incident light can be improved, thereby suppressing spatial offset between the RGB image and the IR image. This makes it possible to improve the accuracy of results obtained by processing information (RGB image and IR image) from different sensors as a whole.

[0219] 1.10 Pixel Driver Line Wiring Example

[0220] Next, we will describe a wiring example of the pixel driving line LD1 connecting RGB pixel 10 and RGB pixel driving circuit 102A, and the pixel driving line LD2 connecting IR pixel 20 and IR pixel driving circuit 102B. Figure 12 This is a plan view illustrating an example of the wiring of a pixel driving line for an RGB pixel according to a first embodiment; Figure 13 This is a plan view illustrating a wiring example for a pixel driving line for an IR pixel according to a first embodiment.

[0221] like Figure 12 and Figure 13 As shown, for example, the RGB driving line LD1 connecting the transmission gate 11, reset transistor 12, and select transistor 14 of the RGB pixel driving circuit 102A and the RGB pixel 10, and the IR driving line LD2 connecting the transmission transistor 21, reset transistor 22, select transistor 24, and discharge transistor 25 of the IR pixel driving circuit 102B and the IR pixel 20, can be routed orthogonally to each other. However, the routing is not limited to this, and the RGB driving line LD1 and the IR driving line LD2 can be routed in parallel with each other. In this case, the RGB pixel driving circuit 102A and the IR pixel driving circuit 102B can provide various control signals to the pixel array section 101 from the same side or from different sides.

[0222] 1.11 Example of a stacked structure for an image sensor

[0223] Figure 14 This is a diagram illustrating an example of a stacked structure of an image sensor according to a first embodiment. (See diagram for example.) Figure 14 As shown, the image sensor 100 has a structure in which pixel chips 140 and circuit chips 150 are vertically stacked. For example, the pixel chip 140 is a semiconductor chip including a pixel array section 101, which is an array of unit pixels 110 including RGB pixels 10 and IR pixels 20. For example, the circuit chip 150 includes... Figure 5 The semiconductor chip of the pixel circuit array shown.

[0224] For example, pixel chip 140 and circuit chip 150 can be joined to each other using direct bonding, wherein the bonding surfaces of the chips are planarized and then the chips are joined to each other by electronic force. However, the bonding method is not limited to this, and other bonding methods such as Cu-Cu bonding or bump bonding, which join copper (Cu) electrode pads formed on the bonding surfaces to each other, can also be used.

[0225] Additionally, for example, the pixel chip 140 and the circuit chip 150 are electrically connected via a connection such as a through-silicon via (TSV) that penetrates the semiconductor substrate. For example, the connection using TSVs can be implemented by employing a dual TSV method, such as a TSV provided in the pixel chip 140 and a TSV provided from the pixel chip 140 to the circuit chip 150, which are connected to each other on the outer surface of the chip, or a shared TSV method where the two chips are connected by a TSV that penetrates from the pixel chip 140 to the circuit chip 150.

[0226] Note that when the pixel chip 140 and the circuit chip 150 are joined to each other by using Cu-Cu bonding or bump bonding, the chips are electrically connected via Cu-Cu bonding or bump bonding.

[0227] 1.12 Identification Operation Example

[0228] Next, an example of the identification operation performed by the electronic device 1 according to this embodiment will be described. Here, we will use Figure 2 The recognition system shown is used to illustrate an example of recognition operation. However, as described above, the recognition operation can be implemented in the image sensor 100, by processing the image data acquired by the image sensor 100 in the processor 4, or by performing part of the processing on the image data acquired by the image sensor 100 in the image sensor 100 and the rest in the processor 4.

[0229] Figure 15 This is a flowchart illustrating an example of the identification operation according to the first embodiment. (e.g.) Figure 15 As shown, in this operation, firstly, the RGB sensor unit 71 drives the RGB pixels 10 to acquire RGB image data (step S101), and the IR sensor unit 73 drives the IR pixels 20 to acquire IR image data (step S102). The acquired RGB image data and IR image data are respectively processed in the RGB image processing unit 72 and the IR image processing unit 74 according to a predetermined process, and then input to the recognition processing unit 75. Note that in step S102, when ROI information is input from the RGB image processing unit 72 to the IR sensor unit 73 or the IR image processing unit 74, the RGB image data and / or IR image data of the area corresponding to the ROI information can be input to the recognition processing unit 75.

[0230] Next, using the input RGB image data, the recognition processing unit 75 performs object recognition processing (first recognition processing) within the field of view of the image sensor 100 (step S103). The first recognition processing can be implemented by using recognition processing such as pattern recognition and recognition processing based on artificial intelligence.

[0231] Next, using the input IR image data and the result of the first recognition process, the recognition processing unit 75 performs recognition processing (second recognition processing) to more accurately identify objects present within the field of view of the image sensor 100 (step S104). Similar to the first recognition process, the second recognition process can be implemented by using recognition processing such as pattern recognition and recognition processing based on artificial intelligence.

[0232] Next, for example, the recognition processing unit 75 outputs the result of the second recognition processing obtained in step S104 to the outside via the interface unit 76 (step S105). The recognition processing unit 75 may execute a part of the first recognition processing and output the result (intermediate data, etc.) to the outside, or it may execute a part of the second recognition processing and output the result (intermediate data, etc.).

[0233] Subsequently, the identification system determines whether the current operation has ended (step S106). If it is determined that the operation has not ended (No in step S106), the identification system returns to step S101. Conversely, if it is determined that the current operation has ended (Yes in step S106), the identification system ends the current operation.

[0234] 1.13 Functions and Effects

[0235] As described above, according to the first embodiment, since multiple sensor information can be acquired—namely, the RGB image acquired by RGB pixel 10 and the IR image acquired by IR pixel 20—the accuracy of the recognition processing is improved by using this sensor information. For example, as described above, by acquiring IR image data in addition to RGB image data, unauthorized access, such as identity theft using a photograph in face authentication, can be determined more accurately. This enables the realization of a solid-state imaging device and recognition system capable of achieving more secure authentication.

[0236] Furthermore, this implementation can further improve the accuracy of identification processing by performing multi-level identification processing using multiple sensor data. This enables the development of solid-state imaging devices and identification systems capable of more secure authentication.

[0237] 2. Second Implementation Plan

[0238] Next, the second embodiment will be described in detail with reference to the accompanying drawings. In the following description, configurations similar to those of the above embodiment will be used, thus omitting repetitive descriptions.

[0239] The first embodiment described above is an exemplary case in which one IR pixel 20 is associated with one RGB pixel 10. Conversely, the second embodiment will illustrate an exemplary case in which multiple RGB pixels 10 are associated with one IR pixel 20.

[0240] 2.1 Example of a unit pixel

[0241] First, an example of the configuration of the unit pixel 210 according to this embodiment will be described. Similar to the first embodiment, the following is an exemplary case in which the unit pixel 210 includes RGB pixels for acquiring an RGB image of the RGB three primary colors and IR pixels for acquiring an IR image of infrared (IR) light. Furthermore, for example, the RGB pixels 10 are arranged in a Bayer array.

[0242] Figure 16 This is a schematic diagram illustrating a illustrative example of the configuration of a unit pixel according to the second embodiment. For example... Figure 16 As shown, the unit pixel 210 has a structure in which one IR pixel 20 is arranged correspondingly to four RGB pixels 10 arranged in a 2x2 pattern in the light incident direction. That is, this embodiment uses the following configuration: an IR pixel 20 is positioned relative to the four RGB pixels 10 in a direction perpendicular to the arrangement direction (planar direction) of the unit pixel 210, and light transmitted through the four RGB pixels 10 located upstream of the incident light path is incident on the IR pixel 20 located downstream of the four RGB pixels 10. Therefore, in this embodiment, the optical axes of the incident light on the unit array of the Bayer array including the four RGB pixels 10 and the optical axes of the incident light on the IR pixel 20 are completely or substantially aligned with each other.

[0243] 2.2 Example of circuit configuration per unit pixel

[0244] Figure 17 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the second embodiment. Figure 17 Based on the reference in the first implementation scheme Figure 8 The second variant uses unit pixel 110-2. However, the configuration is not limited to this and can be based on any one of unit pixels 110 to 110-3.

[0245] like Figure 17 As shown, unit pixel 210 includes multiple RGB pixels 10⁻¹ to 10⁻⁴ (in... Figure 17 In this context, N is 4) and one IR pixel 20. When a unit pixel 210 includes multiple RGB pixels 10 in this manner, as in the first embodiment, referencing Figure 9In the third variation, a pixel circuit (reset transistor 12, floating diffusion region FD1, amplification transistor 13, and selection transistor 14) can be shared by multiple RGB pixels 10 (pixel sharing). Therefore, in this embodiment, multiple RGB pixels 10-1 to 10-N share a pixel circuit including reset transistor 12, floating diffusion region FD1, amplification transistor 13, and selection transistor 14. That is, in this embodiment, multiple photoelectric conversion units PD1 and transmission gates 11 are connected to a common floating diffusion region FD1.

[0246] 2.3 Example of cross-sectional structure per unit pixel

[0247] Figure 18 This is a cross-sectional view showing an example of the cross-sectional structure of an image sensor according to the second embodiment. (and) Figure 16 Similarly, this description will provide an exemplary case where each unit pixel 210 comprises four RGB pixels 10 and one IR pixel 20 arranged in a 2x2 pattern. Furthermore, with Figure 10 Similarly, an example of the cross-sectional structure of a semiconductor chip in which photoelectric conversion units PD1 and PD2 are formed in unit pixel 210 will be described below. Furthermore, in the following description, references to the first embodiment will be used. Figure 10 The cross-sectional structure of the image sensor 100 will be described similarly, without repeating the description.

[0248] like Figure 18 As shown, this implementation scheme has the same characteristics as... Figure 10 The cross-sectional structure shown is similar to the one in the diagram, where the on-chip lens 51, color filter 31, and storage electrode 37 are arranged in a 2x2 pattern, forming four sections (note that in...). Figure 18 (Two of the four are shown in the diagram), thus forming four RGB pixels 10. The four RGB pixels 10 in each unit pixel 210 can form a basic array in the pattern of a Bayer array.

[0249] 2.4 Example of a planar structure

[0250] Figure 19 This is a diagram showing examples of the planar layout of each layer of the pixel array section according to the second embodiment, wherein (A) shows an example of the planar layout of the on-chip lens 51, (B) shows an example of the planar layout of the color filter 31, (C) shows an example of the planar layout of the storage electrode 37, and (D) shows an example of the planar layout of the photoelectric conversion section PD2. Figure 19 In the diagram, (A) to (D) show examples of planar layouts of surfaces parallel to the element forming surface of the semiconductor substrate 50.

[0251] like Figure 19As shown in (A) to (D), this embodiment uses a configuration in which four on-chip lenses 51, four color filters 31, four storage electrodes 37, and one photoelectric conversion unit PD2 are provided for each unit pixel 210. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one IR pixel 20.

[0252] In this way, by arranging a basic array of a pattern comprising four RGB pixels 10 in the direction of incident light travel and an IR pixel 20 within a single pixel 110, the coaxiality of the individual RGB pixels 10 and IR pixels 20 relative to the incident light can be improved, thereby suppressing spatial offsets occurring between the RGB and IR images. This allows for improved accuracy of results obtained by processing information (RGB and IR images) from different sensors holistically.

[0253] 2.5 Examples of deformation of on-plate lenses

[0254] Although the second embodiment described above is an exemplary case of setting an on-chip lens 51 for one RGB pixel 10, the configuration is not limited thereto, and an on-chip lens can be set for multiple RGB pixels 10. Figure 20 This is a diagram illustrating an example of the planar layout of each layer of the pixel array section in a modified example of an on-chip lens according to the second embodiment. Figure 19 Similarly, (A) shows an example of a planar layout of the on-chip lens 51, (B) shows an example of a planar layout of the color filter 31, (C) shows an example of a planar layout of the storage electrode 37, and (D) shows an example of a planar layout of the photoelectric conversion unit PD2.

[0255] exist Figure 20 In the modified example of the on-chip lens shown, as shown in (A), two on-chip lenses 51 arranged in the row direction of some of the unit pixels 210 are replaced by a single on-chip lens 251 extending across two RGB pixels 10 in a 2×1 pixel configuration. Furthermore, as shown... Figure 20 As shown in (B), the two RGB pixels 10 of the shared on-chip lens 251 are configured to selectively transmit color filters 31 with the same wavelength component. Figure 20 In the example shown in (B), in the top left unit pixel 210, it is assumed that the color filter 31b, which selectively transmits the blue (B) wavelength component in the Bayer array, has been replaced by the color filter 31g, which selectively transmits the green (G) wavelength component. With this replacement, the color filters 31 of the two RGB pixels 10 sharing the on-chip lens 251 are unified into the color filter 31g.

[0256] Note that for RGB pixel 10, which replaces color filter 31 in this way, for example, pixel values ​​based on the wavelength components detected by the Bayer array can be interpolated from the pixel values ​​of surrounding pixels. Pixel interpolation can be performed using various methods such as linear interpolation.

[0257] Furthermore, a variation of the on-chip lens is represented by the case where two on-chip lenses 51 arranged in the row direction are unified. However, this configuration is not limited to this, and various variations are allowed, such as a configuration in which two on-chip lenses 51 arranged in the column direction are unified, or a configuration in which all four on-chip lenses 51 in a unit pixel 210 are replaced by a single on-chip lens. In this case, a color filter 31 that selectively transmits the same wavelength component can be used as the color filter 31 of the RGB pixel 10 sharing the on-chip lens.

[0258] Furthermore, the sharing of on-chip lens 51 between adjacent RGB pixels 10 is not limited to the second embodiment, but can also be applied to the first embodiment.

[0259] 2.6 Variations of Color Filter Arrays

[0260] Although the above embodiments and variations use a Bayer array as the filter array for color filter 31, the filter array pattern is not limited thereto. For example, various filter arrays are permitted, such as the 3×3 pixel color filter array used in X-Trans (registered trademark) CMOS sensors, the 4×4 pixel quad Bayer array (also known as a quad-array), and the 4×4 pixel color filter array (also known as a white RGB array) in which a white RGB color filter is combined with a Bayer array.

[0261] Figure 21 This is a diagram illustrating an example of the planar layout of each layer of the pixel array section in a modified example of the color filter array according to the second embodiment. Figure 19 and Figure 20 Similarly, (A) shows an example of a planar layout of the on-chip lens 51, (B) shows an example of a planar layout of the color filter 31, (C) shows an example of a planar layout of the storage electrode 37, and (D) shows an example of a planar layout of the photoelectric conversion unit PD2.

[0262] As shown in (B), Figure 21 The variant of the color filter array shown uses a square array of 4×4 pixels as the color filter array, where each color filter 31 in the 2×2 pixel Bayer array is divided into 2×2 pixels. In this square array, as... Figure 21As shown in (A), even when the on-chip lens 51 is shared by two adjacent RGB pixels 10, the same color filter 31 is arranged adjacent to each other in these RGB pixels 10 as shown in (B). This eliminates the need to change the array of color filters 31, and therefore, pixel interpolation is not required.

[0263] 2.7 Functions and Effects

[0264] As described above, according to the second embodiment, four photoelectric conversion units PD1 of four RGB pixels 10 and one photoelectric conversion unit PD2 of one IR pixel 20 are arranged in the light incident direction. Even with this configuration, similar to the first embodiment, multiple sensor information from RGB and IR images can be acquired, thereby improving the accuracy of recognition processing using this sensor information. This enables the realization of a solid-state imaging device and recognition system capable of achieving more secure authentication.

[0265] Furthermore, similar to the first embodiment, by using multiple sensor information to perform multi-level identification processing, the accuracy of the identification processing can be further improved, thereby enabling a solid-state imaging device and identification system that can achieve more secure authentication.

[0266] Since other components, operations, and effects may be similar to those in the above embodiments, their detailed descriptions will be omitted here.

[0267] 3. Third Implementation Plan

[0268] Next, the third embodiment will be described in detail with reference to the accompanying drawings. In the following description, configurations similar to those of the above-described embodiments will be used, thus omitting repetitive descriptions.

[0269] The first and second embodiments described above are examples of solid-state imaging devices and recognition systems capable of achieving more secure authentication by combining the RGB sensor unit 71 for acquiring RGB images and the IR sensor unit 73 for acquiring IR images. Conversely, the third embodiment will provide an example of a solid-state imaging device and recognition system capable of achieving more secure authentication by combining the RGB sensor unit 71 for acquiring RGB images and the distance measurement sensor unit for acquiring depth images.

[0270] In the following description, an indirect time-of-flight (ToF) sensor will be used as an example of a distance measurement sensor, which detects the two-dimensional distance distribution to an object based on the phase of light emitted from a light source as it is reflected back from the object. However, distance measurement sensors are not limited to this. For example, various types of distance measurement sensors are permitted, such as: direct ToF sensors, which detect the two-dimensional distance distribution to an object by measuring the time it takes for light emitted from a light source to be reflected back from the object; structured light distance measurement sensors, which project light in a predetermined pattern onto an object and detect the distance distribution to the object from the distortion of the pattern; stereo vision distance measurement sensors, which use two or more images captured from different locations to detect the distance distribution to an object; and other types of distance measurement sensors such as millimeter-wave radar, laser imaging detection and ranging (LIDER), and laser detection and ranging (LADAR).

[0271] 3.1 Example of the functional composition of an identification system

[0272] First, an example of the functional configuration of the identification system according to this implementation scheme will be explained. Figure 22 This is a block diagram illustrating an example of the functional configuration of an identification system according to a third embodiment. For example... Figure 22 As shown, the recognition system 370 includes two types of sensor units: an RGB sensor unit 71 and a ToF sensor unit 373. The recognition system 370 also includes an RGB image processing unit 72, a distance measurement processing unit 374, a recognition processing unit 75, and an interface (I / F) unit 76.

[0273] The RGB sensor unit 71, RGB image processing unit 72, recognition processing unit 75, and interface unit 76 may be similar to those exemplified in the first embodiment.

[0274] For example, the ToF sensor unit 373 includes a light source and a light receiving unit (corresponding to an image sensor), and detects light emitted from the light source, reflected by an object, and returned. For example, the light source may be a laser source such as a vertical cavity surface emitting laser (VCSEL), which will be described later. Furthermore, for example, the light receiving unit may be an image sensor in which the pixel circuitry of the IR pixel 20 in the IR sensor unit 73 according to the above embodiment is modified to a pixel circuitry for a ToF sensor.

[0275] For each ToF pixel, the distance measurement processing unit 374 detects the phase of the reflected light based on the emission timing of the light from the light source (hereinafter referred to as the illumination light) based on the pixel signal of each ToF pixel input from the ToF sensor unit 373, and generates a depth image indicating the two-dimensional distance distribution based on the detected phase of each ToF pixel.

[0276] Using RGB image data input from the RGB image processing unit 72 and / or depth image data input from the distance measurement processing unit 374, the recognition processing unit 75 performs recognition processing for objects, etc., present within the field of view of the RGB sensor unit 71 and / or the ToF sensor unit 373. Similar to the first embodiment, the recognition processing of the recognition processing unit 75 can be performed as recognition processing such as pattern recognition, AI recognition processing, etc. Furthermore, the recognition processing unit 75 can perform a portion of the recognition processing and output results (intermediate data, etc.).

[0277] For example, the interface unit 76 outputs the recognition results (including intermediate data, etc.) obtained by the recognition processing unit 75 and the image data obtained by the RGB sensor unit 71 and / or the ToF sensor unit 373 to the external processor 4 and / or the storage unit 3.

[0278] The RGB image processing unit 72 can perform object region determination on the RGB image data and input information such as the address of a specified region of interest (ROI) obtained as a result of the region determination to the ToF sensor unit 373 and / or the distance measurement processing unit 374. In response, the ToF sensor unit 373 can operate to acquire depth image data of the region corresponding to the ROI information input from the RGB image processing unit 72. Alternatively, the distance measurement processing unit 374 can be configured to perform distance measurement processing only on the ToF pixels in the region corresponding to the ROI information input from the RGB image processing unit 72, on the image data (hereinafter referred to as ToF image data) including the pixel signals of each ToF pixel input from the ToF sensor unit 373.

[0279] 3.2 System Configuration Example

[0280] Next, an example of the system configuration of the identification system according to this implementation scheme will be described. Figure 23 This is a schematic diagram illustrating an example of the configuration of an electronic device implementing the identification system according to the third embodiment; Figure 24 This is a block diagram illustrating an example of the configuration of an electronic device that implements the identification system according to the third embodiment.

[0281] like Figure 23 As shown, the electronic device 80 according to this embodiment includes a laser light source 81, a projection lens 83, an imaging lens 84, an image sensor 300, and a system control unit 85.

[0282] like Figure 24 As shown, for example, the laser source 81 includes a vertical-cavity surface-emitting laser (VCSEL) 382 and a light source driving unit 381 for driving the VCSEL 382. However, the type of light source is not limited to VCSEL 382, ​​and various light sources such as light-emitting diodes (LEDs) can be used. Furthermore, the laser source 81 can be any of a point source, a surface source, or a line source. For example, in the case of a surface source or a line source, the laser source 81 can have a configuration in which multiple point sources (e.g., VCSELs) are arranged in one or two dimensions.

[0283] In this embodiment, the laser light source 81 can emit light of a different wavelength than the detection band of the RGB pixels 10 in the image sensor 300, such as IR light.

[0284] The projection lens 83 is arranged on the emitting surface side of the laser source 81 and converts the light emitted from the laser source 81 into illumination light with a predetermined divergence angle.

[0285] An imaging lens 84 is arranged on the light-receiving surface side of the image sensor 300 and forms an image on the light-receiving surface of the image sensor 300 based on incident light (including reflected light from the illumination light).

[0286] like Figure 24 As shown, for example, the image sensor 300 includes a light receiving unit 384 and a sensor control unit 383 that drives the light receiving unit 384 to read RGB image data and ToF image data, and its details will be described below.

[0287] For example, the system control unit 85 includes a processor (CPU) and drives the VCSEL 382 via the light source driver unit 381. Furthermore, the system control unit 85 acquires RGB image data by controlling the image sensor 300. Simultaneously, by controlling the image sensor 300 in sync with the control of the laser light source 81, the system control unit 85 acquires ToF image data corresponding to the emission of the laser light source 81.

[0288] In this configuration, illumination light emitted from laser source 81 is projected onto subject (also referred to as measurement target or object) 901 through projection lens 83. The projected light is reflected by subject 901. Then, the light reflected by subject 901 is incident on image sensor 300 through imaging lens 84. ToF sensor unit 373 in image sensor 300 receives the reflected light, i.e., the light reflected by subject 901, and generates ToF image data. On the other hand, for example, RGB sensor unit 71 in image sensor 300 receives visible light from the incident light and generates RGB image data. The RGB image data and ToF image data generated by image sensor 300 are provided to application processor 86 of electronic device 80. Application processor 86 performs predetermined processing, such as recognition processing, on the RGB image data and ToF image data input from image sensor 300.

[0289] 3.3 Example of Image Sensor Configuration

[0290] Next, an example of the configuration of the image sensor 300 according to this embodiment will be described. Figure 25 This is a block diagram illustrating a schematic configuration example of an image sensor according to a third embodiment.

[0291] For example, the image sensor 300 according to this embodiment has a configuration similar to that of the image sensor 100 according to the first embodiment, wherein the configuration corresponding to the IR sensor unit 73 is replaced with the configuration corresponding to the ToF sensor unit 373. Specifically, for example, the unit pixel 110 is replaced with the unit pixel 310, the IR pixel driving circuit 102B is replaced with the ToF pixel driving circuit 302B, the IR signal processing circuit 103B is replaced with the ToF signal processing circuit 303B, and the IR data processing unit 108B is replaced with the ToF data processing unit 308B.

[0292] For example, a pixel array unit 101, a ToF pixel driving circuit 302B, a ToF signal processing circuit 303B, a column driving circuit 104, and a system control circuit 105 are configured. Figure 22 The ToF sensor unit 373 is included. Furthermore, for example, a ToF signal processing circuit 303B and a ToF data processing unit 308B are configured. Figure 22 The distance measurement and processing unit 374 in the middle. Figure 22 The recognition processing unit 75 can be implemented independently by the application processor 86, or it can be implemented by cooperating with the application processor 86 using the RGB data processing unit 108A and the ToF data processing unit 308B, or it can be implemented by cooperating with each other using the RGB data processing unit 108A and the ToF data processing unit 308B.

[0293] Each unit pixel 310 includes an RGB pixel 10 and a ToF pixel 320. Each ToF pixel 320 includes a photoelectric conversion unit that generates and accumulates charge based on the amount of received light, and generates a pixel signal of voltage based on the amount of incident light.

[0294] In this embodiment, one end of the pixel driving line LD2 is connected to the output terminal corresponding to each row of the ToF pixel driving circuit 302B, and sends a control signal for performing the driving when the pixel signal is read from the ToF pixel 320.

[0295] Similar to the RGB pixel driving circuit 102A, the ToF pixel driving circuit 302B includes a shift register, an address decoder, etc., and drives each pixel of the pixel array section 101 simultaneously or row-by-row for all pixels. That is, the ToF pixel driving circuit 302B, together with the system control circuit 105 that controls the ToF pixel driver circuit 302B, constitutes a driving unit that controls the operation of each ToF pixel 320 of the pixel array section 101. Furthermore, similar to the RGB pixel driving circuit 102A, the ToF pixel driving circuit 302B may include two scanning systems: a readout scanning system and a scanout scanning system.

[0296] Pixel signals output from each ToF pixel 320 in a pixel row selectively scanned by the ToF pixel driving circuit 302B are input to the ToF signal processing circuit 303B via vertical signal lines VSL2 and VSL4 for each pixel column. For example, a pixel signal based on the charge read from one readout terminal (also referred to as TapA) of the ToF pixel 320 appears in vertical signal line VSL2, and a pixel signal based on the charge read from another readout terminal (also referred to as TapB) of the ToF pixel 320 appears in vertical signal line VSL4. The ToF signal processing circuit 303B performs predetermined signal processing on the pixel signals output from each ToF pixel 320 in the selected row via vertical signal lines VSL2 or VSL4 for each pixel column of the pixel array section 101, and temporarily holds the pixel signals after signal processing.

[0297] Similar to the RGB signal processing circuit 103A, the ToF signal processing circuit 303B can perform noise removal processing on the pixel signal, such as Correlation Double Sampling (CDS) processing and Dual Data Sampling (DDS) processing. Furthermore, for example, similar to the RGB signal processing circuit 103A, the ToF signal processing circuit 303B includes an AD conversion function, converting the analog pixel signal read from the photoelectric conversion unit into a digital signal and outputting the obtained digital signal.

[0298] Based on the synchronization control signal input from the system control unit 85, the system control circuit 105 executes drive control of the RGB pixel drive circuit 102A, the ToF pixel drive circuit 302B, the RGB signal processing circuit 103A, the ToF signal processing circuit 303B, the column drive circuit 104, etc. Therefore, the drive for acquiring RGB image data and / or ToF image data is executed synchronously with the emission control of the laser light source 81.

[0299] For example, the ToF data processing unit 308B has arithmetic processing capabilities and performs various types of signal processing, such as arithmetic processing, on the image signal output from the ToF signal processing circuit 303B. The ToF data processing unit 308B can generate a depth image indicating a two-dimensional distance distribution based on the synchronization control signal input from the system control unit 85 (or the system control circuit 105) and the pixel signal read from each ToF pixel 320 at a predetermined sampling period.

[0300] For example, RGB image data or ToF image data (or depth image data) output from RGB data processing unit 108A or ToF data processing unit 308B can be pre-processed in an application processor 86 or the like in an electronic device 80 equipped with image sensor 300, or can be sent to the outside via a pre-defined network.

[0301] The image sensor 300 may include a storage unit for temporarily storing data required for signal processing in the RGB data processing unit 108A and the ToF data processing unit 308B, as well as data processed by any one or more of the RGB signal processing circuit 103A, the ToF signal processing circuit 303B, the RGB data processing unit 108A, and the ToF data processing unit 308B.

[0302] 3.3.1 Examples of image sensor variations

[0303] Figure 26 This is a block diagram illustrating a schematic configuration example of an image sensor according to a variation of the third embodiment. (The above...) Figure 25 This refers to the case where separate signal processing circuits (RGB signal processing circuit 103A and ToF signal processing circuit 303B) are set up for each of the RGB pixels 10 and ToF pixels 320. However, the arrangement of the signal processing circuits is not limited to this, and may also have other arrangements such as... Figure 26 The configuration shown includes a common signal processing circuit 303 for both the RGB pixel 10 and the ToF pixel 320. In this case, the reading of pixel signals from the RGB pixel 10 and the reading of pixel signals from the ToF pixel 320 can be performed in a time-division manner or in parallel.

[0304] 3.4 Example of a unit pixel

[0305] Here, an example of the configuration of unit pixel 310 will be described. The following is an exemplary case where unit pixel 310 includes RGB pixels for acquiring an RGB image of the RGB three primary colors and a ToF pixel 320 for acquiring a ToF image of infrared (IR) light. Figure 27 In the following description, when the color filters 31r, 31g, and 31b that transmit light through the individual color components that constitute the RGB primary colors are not distinguished, the reference numeral 31 denotes a filter.

[0306] Figure 27 This is a schematic diagram illustrating a illustrative configuration example of the pixel array section according to the third embodiment. For example... Figure 27 As shown, a reference is provided in the second embodiment. Figure 16 The configuration of the unit pixel 210 is similar, except that the IR pixel 20 is replaced by a ToF pixel 320. However, the configuration is not limited to this, and the unit pixel 310 may have features as referenced in the first embodiment. Figure 5 In a configuration similar to the unit pixel 110, the IR pixel 20 is replaced by a ToF pixel 320.

[0307] 3.5 unit pixel circuit configuration example

[0308] Figure 28 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to the third embodiment. Figure 28 Based on the first implementation plan, refer to Figure 6 The unit pixel is 110. However, the configuration is not limited to this and can be based on any one of unit pixels 110 to 110-3 or 210.

[0309] like Figure 28 As shown, unit pixel 310 includes two RGB pixels 10-1 and 10-2 and one ToF pixel 320.

[0310] (10 RGB pixels)

[0311] Similar to the RGB pixel 10 according to the first embodiment, each RGB pixel 10 includes a transmission gate 11, a floating diffusion region FD1, a reset transistor 12, an amplification transistor 13, and a selection transistor 14. However, the drain of the selection transistor 14 of one RGB pixel 10-1 is connected to the vertical signal line VSL1, while the drain of the selection transistor 14 of another RGB pixel 10-2 is connected to the vertical signal line VSL3. Furthermore, for example, the two RGB pixels 10-1 and 10-2 can be two RGB pixels 10 adjacent in the row direction in the pixel array portion 101. The number of RGB pixels 10 included in a unit pixel 310 is not limited to two, and can be three or more.

[0312] (ToF pixel 320)

[0313] For example, the ToF pixel 320 includes a photoelectric conversion unit PD2, transmission transistors 21A and 21B, floating diffusion regions FD2A and FD2B, reset transistors 22A and 22B, amplification transistors 23A and 23B, selection transistors 24A and 24B, and an exhaust transistor 25. The transmission transistor 21A, floating diffusion region FD2A, reset transistor 22A, amplification transistor 23A, and selection transistor 24A are included in a pixel circuit (hereinafter referred to as pixel circuit 320A) for reading charge from one of the two taps provided in the photoelectric conversion unit PD2, TapA. The transmission transistor 21B, floating diffusion region FD2B, reset transistor 22B, amplification transistor 23B, and selection transistor 24B are included in a pixel circuit (hereinafter referred to as pixel circuit 320B) for reading charge from the other tap, TapB, provided in the photoelectric conversion unit PD2. The drain of the selection transistor 24A of the pixel circuit 320A is connected to the vertical signal line VSL2, and the drain of the selection transistor 24B of the pixel circuit 320B is connected to the vertical signal line VSL4.

[0314] The connection relationships between the floating diffusion region FD2A or FD2B, the reset transistor 22A or 22B, and the amplification transistor 23A or 23B relative to the transmission transistor 21A or 21B can be similar to the connection relationships between the floating diffusion region FD1, the reset transistor 12, and the amplification transistor 13 relative to the transmission gate 11 in RGB pixels 10-1 or 10-2. Furthermore, the connection relationships between the amplification transistor 23A or 23B, the selection transistor 24A or 24B, and the vertical signal line VSL2 or VSL4 can be similar to the connection relationships between the amplification transistor 13 and the selection transistor 14 and the vertical signal line VSL1 or VSL3 in RGB pixels 10-1 or 10-2.

[0315] The photoelectric conversion unit PD2 includes two taps, TapA and TapB, and releases the charge generated by photoelectric conversion from the two taps TapA and TapB in a time-division manner, the details of which will be described below. Therefore, pixel circuit 320A causes a pixel signal having a voltage value corresponding to the amount of charge released from tap TapA to appear in vertical signal line VSL2, while pixel circuit 320B causes a pixel signal having a voltage value corresponding to the amount of charge released from tap TapB to appear in vertical signal line VSL4.

[0316] Similar to vertical signal lines VSL1 and VSL3, vertical signal lines VSL2 and VSL4 are connected to AD conversion circuits 103a configured for each column in the ToF signal processing circuit 303B (i.e., for each of vertical signal lines VSL2 and VSL4).

[0317] 3.6 Examples of circuit configuration variations

[0318] Next, we will use some examples to illustrate. Figure 28 The circuit configuration of the unit pixel 310 shown is a modified example.

[0319] 3.6.1 First Variation Example

[0320] Figure 29 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a first variation of the third embodiment. For example... Figure 29 As shown, with Figure 28 The unit pixel 310 shown is different; unit pixel 310-1 includes an RGB pixel 10 and a ToF pixel 320. The pixel circuitry of the RGB pixel 10 and the ToF pixel 320 can be similar to that in the reference... Figure 28 The pixel circuit described above. However, in the first variation, the pixel circuit 320B in the RGB pixel 10 and the ToF pixel 320 shares the vertical signal line VSL1, thus omitting the vertical signal lines VSL3 and VSL4. For example, the AD conversion circuit 103a connected to the vertical signal line VSL1 is used to read the pixel signal from the RGB pixel 10 and the pixel signal from the ToF pixel 320 in a time-division manner.

[0321] This configuration allows for a reduction in the number of vertical signal lines (VSLs) and the number of AD conversion circuits (103a) connected to them. This enables a reduction in circuit size, thereby achieving miniaturization and higher resolution of the image sensor (100) due to improved area efficiency.

[0322] 3.6.2 Second variation

[0323] Figure 30 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a second variation of the third embodiment. For example... Figure 30 As shown, unit pixel 310-2 has the same Figure 29The configuration is similar to that of the unit pixel 310-1 shown, where the RGB pixel 10 is connected to the vertical signal line VSL1, the pixel circuit 320A in the ToF pixel 320 is connected to the vertical data line VSL2, and the pixel circuit 320B is connected to the vertical information line VSL4. Furthermore, the vertical signal lines VSL1 and VSL2 are configured to be connected to a common AD conversion circuit 103a. Therefore, similar to the first variation of the first embodiment, this second variation includes a switching circuit 131 that switches the vertical signal line connected to the AD conversion circuit 103a to either the vertical signal line VSL1 or VSL2.

[0324] With this configuration, a portion of the RGB signal processing circuit 103A and the ToF signal processing circuit 303B can be replaced with a common signal processing circuit, which reduces the circuit size and thus enables miniaturization of the image sensor 100 and achieves higher resolution due to improved area efficiency.

[0325] 3.6.3 Third variation example

[0326] Figure 31 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a third variation of the third embodiment. For example... Figure 31 As shown, unit pixel 310-3 has the same Figure 28 The unit pixel 310 shown has a similar configuration, where vertical signal lines VSL1 and VSL2 are connected to a common AD conversion circuit 103a, and vertical signal lines VSL3 and VSL4 are also connected to a common AD conversion circuit 103a. Therefore, similar to the second variation, the third variation includes: a switching circuit 131 that switches a vertical signal line connected to one AD conversion circuit 103a to either vertical signal line VSL1 or VSL2; and a switching circuit 131 that switches a vertical signal line connected to another AD conversion circuit 103a to either vertical signal line VSL3 or VSL4.

[0327] With this configuration, the RGB signal processing circuit 103A and the ToF signal processing circuit 303B can be replaced with a common signal processing circuit, which allows for a further reduction in circuit size. This results in miniaturization of the image sensor 100 and the achievement of higher resolution due to further improved area efficiency.

[0328] 3.6.4 Fourth Variation Example

[0329] Figure 32 This is a circuit diagram illustrating a schematic configuration example of a unit pixel according to a fourth variation of the third embodiment. For example... Figure 32 As shown, unit pixel 310-4 has the same Figure 28The unit pixel 310 shown has a similar configuration, wherein each vertical signal line VSL1 and VSL2 can be connected to either of the two AD conversion circuits 103a, and each vertical signal line VSL3 and VSL4 can be connected to either of the two AD conversion circuits 103a. Therefore, similar to the third variation of the first embodiment, this fourth variation includes: switching circuits 132 and 133 that switch the vertical signal lines connected to one set of two AD conversion circuits 103a to either vertical signal line VSL1 or VSL2; and switching circuits 132 and 133 that switch the vertical signal lines connected to another set of two AD conversion circuits 103a to either vertical signal line VSL3 or VSL4.

[0330] With this configuration, for example, since the AD conversion circuit 103a for each column can be selected from multiple AD conversion circuits (two AD conversion circuits in this example), the image quality degradation caused by noise such as stripes can be suppressed.

[0331] 3.7 Example of cross-sectional structure per unit pixel

[0332] Next, we will refer to Figure 33 An example illustrating the cross-sectional structure of the image sensor 300 according to the third embodiment. Figure 33 This is a cross-sectional view showing an image sensor according to a third embodiment. Similar to the embodiments described above, an example of the cross-sectional structure of a semiconductor chip in which photoelectric conversion units PD1 and PD2 are formed in unit pixels 310 will be described below.

[0333] Additionally, similar to the embodiments described above, the following description is an example of a cross-sectional structure of a back-illuminated device, wherein the light incident surface is located on the back side of the semiconductor substrate 50 (opposite to the element forming surface). However, the type is not limited to this, and this could be a cross-sectional structure of a front-illuminated device in which the light incident surface is located on the front surface side (element forming surface side) of the semiconductor substrate 50. Furthermore, although this description is an exemplary case of organic materials being used in the photoelectric conversion section PD1 of the RGB pixel 10, similar to the embodiments described above, one or both of organic materials and semiconductor materials (also referred to as inorganic materials) can be used as the photoelectric conversion material for each of the photoelectric conversion sections PD1 and PD2. In this description, a unit pixel 310 is illustrated to include two RGB pixels 10-1 and 10-2 (corresponding to the above reference). Figure 28 The configuration is not limited to this, and a unit pixel 310 may include one RGB pixel 10 or three or more RGB pixels 10.

[0334] like Figure 33As shown, for example, the image sensor 300 according to this embodiment has the same characteristics as the reference sensor in the second embodiment. Figure 18 The image sensor 100 has a similar cross-sectional structure, wherein two taps 46 for reading out charges of different phases are provided on the element forming surface side (lower side in the figure) of the photoelectric conversion unit PD2. One tap 46 may correspond to tap TapA of the transfer transistor 21A (vertical transistor 45) connected to the pixel circuit 320A, and the other tap 46 may correspond to tap TapB of the transfer transistor 21B (vertical transistor 45) connected to the pixel circuit 320B. Furthermore, the floating diffusion region connected to the drain of the transfer transistor 21A may be used as the floating diffusion region FD2A of the pixel circuit 320A, and the floating diffusion region connected to the drain of the transfer transistor 21B may be used as the floating diffusion region FD2B of the pixel circuit 320B.

[0335] Since other components and materials may be similar to those in the above-described embodiments, detailed descriptions will be omitted here.

[0336] 3.8 Example of a planar structure

[0337] Next, an example of the planar structure of the pixel array section according to this embodiment will be described. Figure 34 This is a diagram showing an example planar layout of each layer of the pixel array section according to the third embodiment, wherein (A) shows an example planar layout of the on-chip lens 51, (B) shows an example planar layout of the color filter 31, (C) shows an example planar layout of the storage electrode 37, and (D) shows an example planar layout of the photoelectric conversion section PD2. Figure 34 Similar to the embodiments described above, (A) to (D) show examples of planar layouts of surfaces parallel to the element forming surfaces of the semiconductor substrate 50. This description uses the following exemplary case: a 2×2 pixel Bayer array is used as a unit array, which includes pixels that selectively detect the red (R) wavelength component (hereinafter referred to as R pixel 10r), pixels that selectively detect the green (G) wavelength component (hereinafter referred to as G pixel 10g), and pixels that selectively detect the blue (B) wavelength component (hereinafter referred to as B pixel 10b), and a unit pixel 310 includes four RGB pixels 10 and one ToF pixel 320 forming the unit array.

[0338] like Figure 34As shown in (A) to (D), this embodiment uses a configuration in which four on-chip lenses 51, four color filters 31, four storage electrodes 37, and one photoelectric conversion unit PD2 are provided for each unit pixel 310. In this description, one storage electrode 37 corresponds to one RGB pixel 10, and one photoelectric conversion unit PD2 corresponds to one ToF pixel 320.

[0339] In this way, by utilizing a basic array comprising a Bayer array pattern of four RGB pixels 10 arranged in the direction of incident light travel and a single pixel 310 of a ToF pixel 320, the quantum efficiency of a ToF pixel 320 can be improved. This allows for improved accuracy of results obtained by processing information acquired from different sensors (RGB images and depth images) holistically. This enables the realization of solid-state imaging devices and identification systems capable of more secure authentication.

[0340] 3.9 Example of Recognition Operation

[0341] Next, examples of identification operations performed by the identification system according to this embodiment will be described. References will be made here as appropriate. Figure 22 The identification system 370 simultaneously uses a reference Figures 23 to 24 The electronic device 80 is used to illustrate an example of an identification operation. Alternatively, as described above, the identification operation can be implemented in the image sensor 300, either by processing the image data acquired by the image sensor 300 in the application processor 86, or by performing a portion of the processing on the image data acquired by the image sensor 300 in the image sensor 300 and the remainder in the application processor 86.

[0342] Figure 35 This is a flowchart illustrating an example of the identification operation according to the third embodiment. (e.g.) Figure 35 As shown, in this operation, the system control unit 85 first drives the RGB sensor unit 71 in the image sensor 300 (see...). Figure 22 To acquire RGB image data (step S301). Furthermore, the system control unit 85 drives the laser source 81 at a predetermined sampling period so that the laser source 81 emits illumination light at the predetermined sampling period (step S302), and synchronously drives the ToF sensor unit 373 in the image sensor 300 at the predetermined sampling period (see step S302). Figure 22 ToF image data is acquired at a predetermined sampling period (step S303).

[0343] The acquisition of RGB image data can be performed in parallel with the acquisition of ToF image data, or it can be performed in a different period than the ToF image data acquisition period. In this case, the acquisition of RGB image data or the acquisition of ToF image data can be performed first. Alternatively, the acquisition of RGB image data can be performed once for every K (K is an integer greater than or equal to 1) ToF image data acquisitions.

[0344] In the RGB image data and ToF image data acquired in this manner, the RGB image data undergoes predetermined processing in the RGB image processing unit 72 and is then input to the recognition processing unit 75. In step S302 or S303, after the ROI information has been obtained from... Figure 22 When the RGB image processing unit 72 is input to the ToF sensor unit 373 or the distance measurement processing unit 374, the RGB image data and / or ToF image data of the area corresponding to the ROI information can be input to the recognition processing unit 75.

[0345] Next, using the input RGB image data, the recognition processing unit 75 performs object recognition processing (first recognition processing) within the field of view of the image sensor 300 (step S304). Similar to the first embodiment, the first recognition processing can be implemented by using recognition processing such as pattern recognition and recognition processing based on artificial intelligence.

[0346] Furthermore, regarding the ToF image data acquired in step S303, the distance measurement processing unit 374 (see...) Figure 22 Based on the phase difference of each pixel in two ToF image data acquired from each of taps TapA and TapB at a predetermined sampling period, a depth image indicating the two-dimensional distance distribution to objects present within the viewpoint is generated (step S305). The generation of the depth image data can be performed in parallel with the first recognition process, or it can be performed before the first recognition process. Furthermore, for example, the generation of the depth image data can be performed by the ToF data processing unit 308B inside the image sensor 300, or it can be performed by the application processor 86.

[0347] Next, the recognition processing unit 75 performs recognition processing (second recognition processing) to more accurately identify objects present in the viewpoint using the results of the first recognition processing and depth image data (step S306). Similar to the first recognition processing, the second recognition processing can be implemented by using recognition processing such as pattern recognition and recognition processing based on artificial intelligence.

[0348] Next, for example, the recognition processing unit 75 outputs the result of the second recognition processing obtained in step S306 to the outside via the interface unit 76 (step S307). The recognition processing unit 75 may execute a part of the first recognition processing and output the result (intermediate data, etc.) to the outside, or it may execute a part of the second recognition processing and output the result (intermediate data, etc.).

[0349] Subsequently, the identification system 370 determines whether the current operation has ended (step S308). If it is determined that the operation has not ended (No in step S308), the identification system 370 returns to step S301. Conversely, if it is determined that the current operation has ended (Yes in step S308), the identification system 370 ends the current operation.

[0350] 3.10 Functions and Effects

[0351] As described above, according to the third embodiment, since multiple sensor information can be acquired—namely, the RGB image acquired by RGB pixel 10 and the depth image based on the ToF image acquired by ToF pixel 320—the accuracy of the recognition processing is improved by using this sensor information. For example, as mentioned above, by acquiring depth image data in addition to RGB image data, unauthorized access, such as identity theft using a photograph in face authentication, can be determined more accurately. This enables the realization of a solid-state imaging device and recognition system capable of achieving more secure authentication.

[0352] Furthermore, similar to the first embodiment, in this implementation, the accuracy of the identification process can be further improved by performing multi-level identification processing using multiple sensor data. This enables the development of solid-state imaging devices and identification systems capable of achieving more secure authentication.

[0353] Since other components, operations, and effects may be similar to those in the above embodiments, their detailed descriptions will be omitted here.

[0354] 4. Examples of applications of moving bodies

[0355] The technology disclosed herein can be applied to a variety of products. For example, the technology disclosed herein can be applied to devices installed on mobile bodies such as automobiles, electric vehicles, hybrid vehicles, motorcycles, bicycles, personal motor vehicles, airplanes, drones, ships, and robots.

[0356] Figure 36 This is a block diagram illustrating a schematic configuration example of a vehicle control system, which is an example of a mobile body control system to which the technology of the embodiments according to this disclosure can be applied.

[0357] The vehicle control system 12000 includes multiple electronic control units interconnected via a communication network 12001. Figure 36 In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and a comprehensive control unit 12050. Furthermore, a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown as functional components of the comprehensive control unit 12050.

[0358] The drive system control unit 12010 controls the operation of devices related to the vehicle's drive system according to various programs. For example, the drive system control unit 12010 is used as a control device for the following devices: a drive force generating device, such as an internal combustion engine or a drive motor, for generating the driving force of the vehicle; a drive force transmission mechanism for transmitting the driving force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; and a braking device for generating the braking force of the vehicle.

[0359] The body system control unit 12020 controls the operation of various devices installed on the vehicle body according to various programs. For example, the body system control unit 12020 is used as a control device for keyless entry systems, smart key systems, power windows, or various lights such as headlights, taillights, brake lights, turn signals, or fog lights. In this case, radio waves or signals from various switches transmitted from a portable device that serves as a substitute for a key can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals and controls the vehicle's door locking devices, power windows, lights, etc.

[0360] The exterior information detection unit 12030 detects information about the exterior of the vehicle having the vehicle control system 12000. For example, the exterior information detection unit 12030 is connected to the imaging unit 12031. The exterior information detection unit 12030 causes the imaging unit 12031 to image the exterior of the vehicle and receives the captured images. Based on the received images, the exterior information detection unit 12030 can perform detection processing for objects such as people, vehicles, obstacles, markings, or symbols on the road surface, or detection processing for the distance to these objects.

[0361] Imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. Imaging unit 12031 can output an electrical signal as an image, or it can output an electrical signal as information about the measured distance. Furthermore, the light received by imaging unit 12031 can be visible light, or it can be invisible light such as infrared light.

[0362] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. For example, the in-vehicle information detection unit 12040 is connected to a driver state detection unit 12041 that detects the driver's state. The driver state detection unit 12041 includes, for example, a camera that images the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's fatigue level or concentration level, or determine whether the driver is dozing off.

[0363] The microcomputer 12051 can calculate target control values ​​for the drive force generating device, steering mechanism, or braking device based on information about the vehicle's interior or exterior (obtained by the exterior information detection unit 12030 or the interior information detection unit 12040), and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform coordinated control aimed at realizing the functions of an advanced driver assistance system (ADAS), including: collision avoidance or impact mitigation, following distance based on the vehicle's distance, vehicle speed maintenance, collision warning, or lane departure warning, etc.

[0364] Furthermore, the microcomputer 12051 can perform cooperative control designed for autonomous driving, enabling the vehicle to drive autonomously by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the inside or outside of the vehicle (which is obtained by the external information detection unit 12030 or the internal information detection unit 12040), without relying on the driver's operation.

[0365] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on information about the vehicle's exterior (obtained by the exterior information detection unit 12030). For example, the microcomputer 12051 can perform coordinated control aimed at preventing glare by controlling the headlights to switch from high beams to low beams, based on the position of the vehicle ahead or oncoming vehicle detected by the exterior information detection unit 12030.

[0366] The sound / image output unit 12052 transmits at least one output signal of sound and image to an output device capable of visually or audibly notifying passengers of the vehicle or the outside of the vehicle. Figure 36 In the example, audio speaker 12061, display unit 12062, and instrument panel 12063 are shown as output devices. For example, display unit 12062 may include at least one of an in-vehicle display and a head-up display.

[0367] Figure 37 This is a diagram showing an example of the mounting position of the imaging unit 12031.

[0368] exist Figure 37 In the imaging unit 12031, there are imaging units 12101, 12102, 12103, 12104 and 12105.

[0369] Imaging units 12101, 12102, 12103, 12104, and 12105 are, for example, located on the front nose, side mirrors, rear bumper, and rear door of vehicle 12100, and on the upper part of the interior windshield. Imaging unit 12101 on the front nose and imaging unit 12105 on the upper part of the interior windshield primarily acquire images of the front of vehicle 12100. Imaging units 12102 and 12103 on the side mirrors primarily acquire images of the sides of vehicle 12100. Imaging unit 12104 on the rear bumper or rear door primarily acquires images of the rear of vehicle 12100. Imaging unit 12105 on the upper part of the interior windshield is primarily used to detect vehicles, pedestrians, obstacles, signals, traffic signs, or lanes ahead.

[0370] Incidentally, Figure 37 An example of the imaging range of imaging units 12101 to 12104 is shown. Imaging range 12111 represents the imaging range of imaging unit 12101 installed on the front nose. Imaging ranges 12112 and 12113 represent the imaging ranges of imaging units 12102 and 12103 installed on the side mirrors, respectively. Imaging range 12114 represents the imaging range of imaging unit 12104 installed on the rear bumper or rear door. For example, by superimposing the image data captured by imaging units 12101 to 12104, a top-down image of the vehicle 12100 as viewed from above is obtained.

[0371] At least one of the imaging units 12101 to 12104 may have the function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0372] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging range 12111 to 12114 and the time change of that distance (relative speed to the vehicle 12100) based on distance information obtained from the imaging units 12101 to 12104. This allows it to extract the nearest three-dimensional object as the vehicle ahead, specifically, the three-dimensional object that exists on the vehicle 12100's travel path and travels at a predetermined speed (e.g., equal to or greater than 0 km / h) in substantially the same direction as the vehicle 12100. Furthermore, the microcomputer 12051 can pre-set a vehicle-to-vehicle distance to be maintained in front of the vehicle ahead and execute automatic braking control (including stop-and-go control) or automatic acceleration control (including start-and-go control), etc. Therefore, cooperative control designed for autonomous driving can be implemented, enabling the vehicle to drive autonomously without relying on driver operation.

[0373] For example, microcomputer 12051 can classify three-dimensional object data about three-dimensional objects into three-dimensional object data of two-wheeled vehicles, standard-sized vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data to automatically avoid obstacles. For example, microcomputer 12051 identifies obstacles around vehicle 12100 as obstacles that the driver of vehicle 12100 can visually recognize and obstacles that the driver of vehicle 12100 cannot visually recognize. Then, microcomputer 12051 determines the collision risk, which represents the risk of collision with each obstacle. If the collision risk is equal to or higher than a set value and therefore there is a possibility of collision, microcomputer 12051 outputs a warning to the driver via audio speaker 12061 or display unit 12062, and performs forced deceleration or evasive steering through drive system control unit 12010. Microcomputer 12051 can thus assist in driving to avoid collision.

[0374] At least one of the imaging units 12101 to 12104 can be an infrared camera that detects infrared light. For example, the microcomputer 12051 can identify a pedestrian by determining whether a pedestrian exists in the images captured by the imaging units 12101 to 12104. This pedestrian identification is performed, for example, by a program that extracts feature points from the images captured by the imaging units 12101 to 12104 (which are infrared cameras) and a program that determines whether a person is a pedestrian by performing pattern matching processing on a series of feature points representing the outline of an object. When the microcomputer 12051 determines that a pedestrian exists in the images captured by the imaging units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 so that a square outline for emphasis is displayed superimposed on the identified pedestrian. The sound / image output unit 12052 can also control the display unit 12062 so that an icon or the like representing a pedestrian is displayed at a desired location.

[0375] The foregoing described an example of a vehicle control system to which the technology according to this disclosure can be applied. The technology according to this disclosure can be applied to the imaging unit 12031 in the above configuration. Specifically, Figure 37 The imaging units 12101, 12102, 12103, 12104, 12105, etc. shown can be installed in the vehicle 12100. By applying the technology according to this disclosure to the imaging units 12101, 12102, 12103, 12104, 12105, etc., the accuracy of the results obtained by processing information acquired by different sensors (e.g., color images and monochrome images) as a whole can be improved.

[0376] The embodiments of this disclosure have been described above. However, the technical scope of this disclosure is not limited to the above embodiments, and various modifications can be made without departing from the protection scope of this disclosure. Furthermore, appropriate combinations of components across different embodiments and modifications are permitted.

[0377] The effects described in the various embodiments of this specification are merely examples, and therefore, other effects may exist, not limited to the exemplary effects.

[0378] Note that this technology can also have the following configuration.

[0379] (1) A solid-state imaging device, comprising:

[0380] Multiple unit pixels, arranged in a matrix; and

[0381] The signal processing circuit reads the signal from each of the unit pixels.

[0382] Each of the aforementioned unit pixels includes:

[0383] The first pixel, which is arranged on the first surface, detects light of the first wavelength band; and

[0384] The second pixel is arranged on a second surface parallel to the first surface and detects light of a second wavelength band different from the first wavelength band.

[0385] The signal processing circuit includes a first conversion circuit connected to the first pixel and the second pixel in each of the unit pixels and converting the analog signal output from each of the first pixel and the second pixel into a digital signal.

[0386] (2) The solid-state imaging device according to (1),

[0387] Wherein, at least a portion of the first pixel overlaps with the second pixel in a direction perpendicular to the first surface and the second surface.

[0388] (3) The solid-state imaging device according to (1) or (2),

[0389] The first pixel includes an organic photoelectric conversion film.

[0390] (4) The solid-state imaging apparatus according to any one of (1) to (3),

[0391] The signal processing circuit further includes a first switch that switches the connection destination of the first conversion circuit to either the first pixel or the second pixel.

[0392] (5) The solid-state imaging device according to (4),

[0393] The signal processing circuit further includes:

[0394] A second conversion circuit, connected to the first pixel and the second pixel in each of the unit pixels, converts the analog signal output from each of the first pixel and the second pixel into a digital signal; and

[0395] The second switch switches the connection destination of the second conversion circuit to either the first pixel or the second pixel.

[0396] (6) The solid-state imaging apparatus according to any one of (1) to (5),

[0397] Each of the unit pixels includes a plurality of first pixels connected to the first conversion circuit.

[0398] (7) The solid-state imaging device according to (6),

[0399] Each of the first pixels includes:

[0400] The first photoelectric conversion unit performs photoelectric conversion of the incident light; and

[0401] The first pixel circuit reads out the charge generated in the first photoelectric conversion unit, and

[0402] Multiple first pixels share at least a portion of the first pixel circuit.

[0403] (8) The solid-state imaging apparatus according to any one of (1) to (7),

[0404] Each of the unit pixels includes a plurality of second pixels connected to the first conversion circuit.

[0405] (9) The solid-state imaging device according to (8),

[0406] Each of the second pixels includes:

[0407] The second photoelectric conversion unit performs photoelectric conversion of the incident light; and

[0408] The second pixel circuit reads out the charge generated in the second photoelectric conversion unit, and

[0409] Multiple second pixels share at least a portion of the second pixel circuit.

[0410] (10) The solid-state imaging apparatus according to any one of (1) to (9),

[0411] Among them, the signal processing circuit

[0412] First image data is generated based on the signal read from the first pixel among the plurality of unit pixels, and

[0413] Second image data is generated based on the signal read from the second pixel among the plurality of unit pixels.

[0414] (11) The solid-state imaging apparatus according to any one of (1) to (10),

[0415] The second pixel includes:

[0416] The photoelectric conversion unit performs photoelectric conversion of incident light;

[0417] The first pixel circuit reads out the charge generated in the photoelectric conversion unit; and

[0418] The second pixel circuit reads out the charge generated in the photoelectric conversion unit.

[0419] The signal processing circuit further includes a second conversion circuit, which is connected to the first pixel circuit and converts the analog signal output from the first pixel circuit into a digital signal.

[0420] The first conversion circuit is connected to the first pixel and the second pixel circuit.

[0421] (12) The solid-state imaging device according to (11),

[0422] The signal processing circuit further includes a first switch that switches the connection destination of the first conversion circuit to either the first pixel circuit or the second pixel circuit.

[0423] (13) The solid-state imaging apparatus according to any one of (1) to (12),

[0424] Each of the aforementioned unit pixels includes a plurality of the first pixels.

[0425] The second pixel includes:

[0426] The photoelectric conversion unit performs photoelectric conversion of incident light;

[0427] The first pixel circuit reads out the charge generated in the photoelectric conversion unit; and

[0428] The second pixel circuit reads out the charge generated in the photoelectric conversion unit.

[0429] The signal processing circuit further includes a second conversion circuit connected to one of the plurality of first pixels and the second pixel circuit, which converts the analog signals output from the one of the plurality of first pixels and from the second pixel circuit into digital signals.

[0430] The first conversion circuit is connected to another of the plurality of first pixels and the first pixel circuit.

[0431] (14) The solid-state imaging device according to (13),

[0432] The signal processing circuit further includes:

[0433] A first switch that switches the connection destination of the first switching circuit to another of the plurality of first pixels and any one of the first pixel circuits; and

[0434] A second switch switches the connection destination of the second conversion circuit to one of the plurality of first pixels and any one of the second pixel circuits.

[0435] (15) The solid-state imaging device according to (14),

[0436] The signal processing circuit further includes:

[0437] A third conversion circuit is connected to the other of the plurality of first pixels and the first pixel circuit, and converts the analog signals output from the other of the plurality of first pixels and the first pixel circuit into digital signals, respectively.

[0438] A fourth conversion circuit is connected to one of the plurality of first pixel circuits and the second pixel circuit, and converts the analog signals output from the one of the plurality of first pixel circuits and the second pixel circuit into digital signals respectively;

[0439] A third switch switches the connection destination of the first conversion circuit to another of the plurality of first pixels and any one of the first pixel circuits; and

[0440] A fourth switch switches the connection destination of the second conversion circuit to one of the plurality of first pixels and any one of the second pixel circuits.

[0441] (16) The solid-state imaging apparatus according to any one of (1) to (15),

[0442] Wherein, the light in the first band is light that is included in the visible light band, and

[0443] The second band of light is light that is included in the infrared band.

[0444] (17) The solid-state imaging apparatus according to any one of (1) to (16), comprising:

[0445] A first driving circuit drives the first pixel among the respective unit pixels; and

[0446] A second driving circuit drives the second pixel in each of the said unit pixels.

[0447] (18) The solid-state imaging apparatus according to any one of (1) to (17),

[0448] The first pixel includes a first photoelectric conversion unit that performs photoelectric conversion of incident light.

[0449] The second pixel includes a second photoelectric conversion unit that performs photoelectric conversion of incident light, and

[0450] The first photoelectric conversion unit and the second photoelectric conversion unit are arranged along the optical axis of the incident light.

[0451] (19) An identification system, comprising:

[0452] The solid-state imaging device according to any one of (1) to (18); and

[0453] The recognition processing unit performs recognition processing based on the first detection data obtained from the first pixel and the second detection data obtained from the second pixel in the solid-state imaging device.

[0454] (20) The identification system according to (19) further includes:

[0455] A light source that emits light in the second wavelength band; and

[0456] The control unit controls the light source and the solid-state imaging device.

[0457] The control unit performs control to synchronize the emission timing of the light source and the driving timing of the second pixel in the solid-state imaging device.

[0458] (21) The identification system according to (19) or (20) further includes:

[0459] The processing unit generates depth information indicating the distance from a second pixel of the solid-state imaging device to the object, the depth information being acquired from the second pixel.

[0460] The recognition processing unit performs recognition processing based on the first detection data and the depth information.

[0461] (22) The identification system according to any one of (19) to (21),

[0462] Among them, the identification processing unit

[0463] A first identification process is performed based on one of the first detection data and the second detection data, and

[0464] A second identification process is performed based on the result of the first identification process and based on another of the first detection data and the second detection data.

[0465] List of reference numerals

[0466] 1. 80 electronic devices

[0467] 2 Imaging Lens

[0468] 3. Storage Section

[0469] 4 processors

[0470] 10, 10-1 to 10-N RGB pixels

[0471] 11. Transmission Gate

[0472] 12, 22, 22A, 22B Reset transistors

[0473] 13, 23, 23A, 23B Amplifying Transistors

[0474] 14, 24, 24A, 24B Select transistors

[0475] 20 IR pixels

[0476] 21, 21A, 21B Transmission Transistors

[0477] 25. Discharge transistors

[0478] 31, 31r, 31g, 31b color filters

[0479] 32 Sealing film

[0480] 33 Transparent Electrode

[0481] 34 Photoelectric conversion film

[0482] 35 Semiconductor layer

[0483] 36 Readout Electrodes

[0484] 37 Storage Electrode

[0485] 41 IR Filter

[0486] 42 p-well region

[0487] 43 p-type semiconductor region

[0488] 44 n-type semiconductor region

[0489] 45 Vertical Transistors

[0490] 46. ​​Taps (TapA, TapB)

[0491] 50 Semiconductor substrate

[0492] 51 On-chip lenses

[0493] 52 Planarization film

[0494] 53 Insulation layer

[0495] 54-pixel isolation section

[0496] 55 Fixed charge film

[0497] 56-layer interlayer insulation film

[0498] Wiring from 61 to 68

[0499] 70, 370 identification system

[0500] 71 RGB Sensor Section

[0501] 72 RGB Image Processing Unit

[0502] 73 IR Sensor Section

[0503] 74 IR Image Processing Unit

[0504] 75 Identification Processing Department

[0505] 76 Interface Section

[0506] 81 Laser source

[0507] 83 Projection Lens

[0508] 84 Imaging Lens

[0509] 85 System Control Department

[0510] 86 Application Processor

[0511] 100, 300 Solid-State Imaging Devices (Image Sensors)

[0512] 101 pixel array

[0513] 102A RGB pixel driving circuit

[0514] 102B IR pixel driving circuit

[0515] 103 and 303 share the same signal processing circuit.

[0516] 103a AD conversion circuit

[0517] 103A RGB signal processing circuit

[0518] 103B IR signal processing circuit

[0519] 104-column drive circuit

[0520] 105 System Control Circuit

[0521] 108A RGB Data Processing Unit

[0522] 108B IR Data Processing Department

[0523] 110, 110-1 to 110-3, 210, 310, 310-1 to 310-4 (unit pixels)

[0524] 131, 132, 133 Switching Circuits

[0525] 140 pixel chip

[0526] 150 circuit chips

[0527] 303B ToF signal processing circuit

[0528] 308B ToF Data Processing Department

[0529] 320 ToF pixels

[0530] 320A and 320B pixel circuits

[0531] 373ToF sensor section

[0532] 374 Distance Measurement and Processing Department

[0533] 381 Light Source Driver

[0534] 382VCSEL

[0535] 383 Sensor Control Unit

[0536] 384 Optical Receiver

[0537] 901 subjects

[0538] Floating diffusion regions FD1, FD2, FD2A, FD2B

[0539] LD pixel drive line

[0540] LD1 RGB driver cable

[0541] LD2 IR drive line

[0542] PD1 and PD2 photoelectric conversion units

[0543] VSL1, VSL2, VSL3, VSL4 Vertical signal lines

Claims

1. A solid-state imaging device, comprising: Multiple unit pixels, arranged in a matrix; and The signal processing circuit reads the signal from each of the unit pixels. Each of the aforementioned unit pixels includes: A plurality of first pixels are arranged on a first surface and detect light of a first wavelength band, wherein each of the first pixels includes: The first photoelectric conversion unit performs photoelectric conversion of the incident light; and The third pixel circuit reads out the charge generated in the first photoelectric conversion unit, and The plurality of first pixels share at least a portion of the third pixel circuit; and A second pixel is disposed on a second surface parallel to the first surface and detects light of a second wavelength band different from the first wavelength band, and the second pixel includes: The second photoelectric conversion unit performs photoelectric conversion of the incident light; The first pixel circuit reads out the charge generated in the second photoelectric conversion unit; and The second pixel circuit reads out the charge generated in the second photoelectric conversion unit. The signal processing circuit includes: A first conversion circuit is connected to the first pixel and the second pixel in each of the unit pixels and converts the analog signal output from each of the first pixel and the second pixel into a digital signal; A second conversion circuit is connected to one of the plurality of first pixels and the second pixel circuit, and converts the analog signal output from the one of the plurality of first pixels and the second pixel circuit into a digital signal, and the first conversion circuit is connected to the other of the plurality of first pixels and the first pixel circuit; A first switch switches the connection destination of the first conversion circuit to the other first pixel among the plurality of first pixels and any one of the first pixel circuits; The second switch switches the connection destination of the second conversion circuit to either the first pixel or the second pixel circuit among the plurality of first pixels; A third conversion circuit is connected to the other first pixel and the first pixel circuit among the plurality of first pixels, and converts the analog signals output from the other first pixel and the first pixel circuit respectively into digital signals; A fourth conversion circuit is connected to the circuits of the first pixel and the second pixel among the plurality of first pixels, and converts the analog signals output from the first pixel and the second pixel circuit respectively into digital signals; A third switch switches the connection destination of the first conversion circuit to either the other first pixel among the plurality of first pixels and any one of the first pixel circuits; and A fourth switch switches the connection destination of the second conversion circuit to either one of the first pixels or any one of the second pixel circuits.

2. The solid-state imaging device according to claim 1, in, At least a portion of the first pixel overlaps with the second pixel in a direction perpendicular to the first surface and the second surface.

3. The solid-state imaging device according to claim 1, in, The first pixel includes an organic photoelectric conversion film.

4. The solid-state imaging device according to any one of claims 1-3, in, Each of the unit pixels includes a plurality of second pixels connected to the first conversion circuit.

5. The solid-state imaging device according to claim 4, in, Multiple second pixels share at least a portion of the second pixel circuit.

6. The solid-state imaging device according to any one of claims 1-3, in, The signal processing circuit First image data is generated based on the signal read from the first pixel among the plurality of unit pixels, and Second image data is generated based on the signal read from the second pixel among the plurality of unit pixels.

7. The solid-state imaging device according to any one of claims 1-3, in, The light in the first band is light that is included in the visible light band, and The second band of light is light that is included in the infrared band.

8. The solid-state imaging apparatus according to any one of claims 1-3, comprising: A first driving circuit drives the first pixel among the various unit pixels; and A second driving circuit drives the second pixel in each of the said unit pixels.

9. The solid-state imaging device according to any one of claims 1-3, in, The first photoelectric conversion unit and the second photoelectric conversion unit are arranged along the optical axis of the incident light.

10. An identification system, comprising: Solid-state imaging apparatus according to any one of claims 1-9; and The recognition processing unit performs recognition processing based on the first detection data obtained from the first pixel and the second detection data obtained from the second pixel in the solid-state imaging device.

11. The identification system according to claim 10, further comprising: A light source that emits light in the second wavelength band; and The control unit controls the light source and the solid-state imaging device. The control unit performs control to synchronize the emission timing of the light source and the driving timing of the second pixel in the solid-state imaging device.

Images