Imaging device and imaging method

Abstract

[Problem] To provide an imaging device capable of reducing a useless area on a substrate. [Solution] The imaging device having a plurality of substrates stacked includes: a readout-dedicated circuit configured on a substrate different from a substrate on which a pixel array unit including a plurality of photoelectric conversion elements is configured, and performing a readout operation of an electric signal obtained by photoelectric conversion of the plurality of photoelectric conversion elements; and a circuit configured on a substrate different from a substrate on which the readout-dedicated circuit is configured, and performing an operation other than an operation of the readout-dedicated circuit based on the electric signal.

Description

Technical Field

[0001] This invention relates to a camera device and a camera method. Background Technology

[0002] Traditionally, imaging devices typically use synchronous solid-state image sensors that capture image data (frames) in sync with a synchronization signal such as a vertical synchronization signal. Since synchronous solid-state image sensors can only acquire image data in each cycle of the synchronization signal (e.g., 1 / 60th of a second), they are ill-suited for applications requiring higher processing speeds, such as those in transportation or robotics. Therefore, asynchronous solid-state image sensors have been proposed, wherein each pixel includes an address event detection circuit (e.g., see Patent Document 1) for real-time detection of pixel light intensity exceeding a threshold as an address event for each pixel address.

[0003] Reference List

[0004] Patent documents

[0005] Patent Document 1: Japanese Translation of PCT Application No. 2016-533140 Summary of the Invention

[0006] The technical problem to be solved by the present invention

[0007] The aforementioned asynchronous solid-state image sensor can generate and output data at a much higher speed than synchronous solid-state image sensors. Therefore, for example, in the transportation sector, image recognition processing for people or obstacles can be performed at high speed to improve safety. However, the address event detection circuit has a larger circuit size compared to synchronous pixel circuits, resulting in an increased installation area when such circuits are placed in each pixel compared to synchronous types.

[0008] In recent years, with the development of semiconductor integrated circuit technology, it has become possible to stack two substrates and transmit or receive signals at high speed between the upper and lower substrates. Therefore, it is also possible to construct an imaging device by stacking a substrate on which the above-mentioned address event detection circuit is configured and a substrate on which a pixel array unit is configured.

[0009] However, when there is a significant difference in the area of ​​the circuitry mounted on the two stacked substrates, the external dimensions of a solid-state image sensor depend on the size of the substrate with the larger circuitry area, raising concerns about the ability to reduce size. Furthermore, because the circuitry components constituting a solid-state image sensor are interconnected, and the circuitry area varies depending on each component, it is difficult to allocate the circuitry so that the mounting areas of the two stacked substrates are identical. One of the two stacked substrates can have a larger circuitry area, while the other can have a smaller circuitry area and a larger unused area. A larger unused area on the substrate means more wasted space, which increases chip cost.

[0010] Therefore, the present invention provides an imaging device and imaging method capable of reducing unwanted areas on a substrate.

[0011] Solutions to technical problems

[0012] To address the aforementioned problems, according to the present invention, a camera device comprising a plurality of stacked substrates is provided, the camera device comprising:

[0013] A dedicated readout circuit is disposed on a different substrate from the substrate on which pixel array units including multiple photoelectric conversion elements are disposed, and is configured to perform a readout operation of the electrical signal obtained by photoelectric conversion through the multiple photoelectric conversion elements; and

[0014] The circuit is disposed on a different substrate from the substrate on which the readout dedicated circuit is disposed, and is configured to perform operations other than the operation of the readout dedicated circuit based on the electrical signal.

[0015] The readout circuit may be configured to convert the electrical signal obtained by photoelectric conversion through the photoelectric conversion element into a voltage signal and perform gain adjustment.

[0016] The substrate on which the circuit is configured to perform operations other than those of the readout dedicated circuit can perform at least one of the following processes: processing to convert a voltage signal output from the readout dedicated circuit into a digital signal in units of two or more pixel groups arranged in a first direction; predetermined signal processing for the digital signal; and processing to drive the plurality of photoelectric conversion elements in units of two or more pixel groups arranged in a second direction.

[0017] In the circuit configured to perform operations other than those of the readout dedicated circuit, the circuit portion having a power supply voltage exceeding a predetermined reference voltage can be disposed on the same substrate as the plurality of photoelectric conversion elements.

[0018] The camera device may include at least a portion of an AD unit, the at least a portion of which is disposed on a substrate on which the pixel array unit is disposed, and is configured to convert pixel signals read from the readout dedicated circuit into digital signals.

[0019] The AD unit can convert the pixel signal read from the readout circuit into a digital signal in units of two or more pixel groups configured in the first direction.

[0020] The AD unit can be segmented and disposed on a substrate and other substrates on which the pixel array unit is disposed.

[0021] The camera device may include a pixel group driving unit disposed on a substrate on which the pixel array unit is disposed, and configured to drive the pixel array unit in units of two or more pixel groups disposed in a second direction.

[0022] The pixel group driving unit can be divided and disposed on a substrate and other substrates on which the pixel array unit is disposed.

[0023] The camera device may include:

[0024] The first substrate has the readout dedicated circuit disposed thereon;

[0025] A second substrate, stacked on the first substrate, and wherein the pixel array units are disposed; and

[0026] A third substrate is stacked with the same layer height as the second substrate on the first substrate, and at least a portion of the circuitry configured to perform operations other than those of the readout dedicated circuitry is disposed thereon.

[0027] The first substrate may be larger than the second substrate, and

[0028] The second substrate may be larger than the third substrate.

[0029] The camera device may include:

[0030] The first substrate has the readout dedicated circuit disposed thereon;

[0031] A second substrate, stacked on the first substrate, and wherein the pixel array units are disposed; and

[0032] A third substrate is stacked below the first substrate and is disposed thereon at least a portion of the circuit configured to perform operations other than those of the readout dedicated circuit.

[0033] The imaging device may include a fourth substrate, which is configured with the same layer height as the third substrate.

[0034] Parts of the circuit configured to perform operations other than those of the readout dedicated circuit are respectively disposed on the third substrate and the fourth substrate.

[0035] The second substrate may be larger than the third substrate and the fourth substrate.

[0036] The readout circuit can be configured on the first substrate to at least partially overlap with the pixel array unit when a planar view of the first substrate and the second substrate is viewed in the stacking direction.

[0037] A dedicated readout circuit can be provided for each of the plurality of photoelectric conversion elements.

[0038] The readout circuit can be configured to be associated with two or more of the aforementioned photoelectric conversion elements.

[0039] The pixel array unit and the readout circuit may include a change detection unit configured to output a detection signal indicating whether the change in the electrical signal of each of the plurality of photoelectric conversion elements exceeds a predetermined threshold.

[0040] The pixel array unit and the readout dedicated circuit may include a pixel AD unit, which is configured to convert the electrical signal obtained by photoelectric conversion through each of the photoelectric conversion elements into a digital signal.

[0041] The pixel array unit and the readout circuit may include a light detection unit configured to detect the incident position and incident time of light incident on the plurality of photoelectric conversion elements.

[0042] The camera device may include:

[0043] The first output unit is configured to output a first signal from the pixel array unit; and

[0044] The second output unit is configured to output a second signal from the readout dedicated circuit.

[0045] The plurality of substrates may be joined by at least one of Cu-Cu bonding, through-silicon via (TSV) and bump bonding.

[0046] The plurality of substrates may be wafers or semiconductor chips.

[0047] According to another aspect of the present invention, a camera method using a plurality of stacked substrates is provided, the camera method comprising the following steps:

[0048] A readout operation is performed in a dedicated readout circuit to read out electrical signals obtained through photoelectric conversion by multiple photoelectric conversion elements. This dedicated readout circuit is disposed on a different substrate from the substrate on which pixel array units including the multiple photoelectric conversion elements are disposed.

[0049] In a circuit disposed on a substrate different from the substrate on which the readout dedicated circuit is disposed, operations other than the operation of the readout dedicated circuit are performed based on the electrical signal. Attached Figure Description

[0050] [ Figure 1 ]

[0051] Figure 1 This is a block diagram showing a schematic construction of the camera device 1 according to the first embodiment.

[0052] [ Figure 2A ]

[0053] Figure 2A This is a diagram showing a first layout example of one of the substrates to be stacked.

[0054] [ Figure 2B ]

[0055] Figure 2B This is a diagram showing a first layout example of another substrate that will be stacked.

[0056] [ Figure 3A ]

[0057] Figure 3A This is a diagram showing a second layout example of one of the substrates to be stacked.

[0058] [ Figure 3B ]

[0059] Figure 3B This is a diagram showing a second layout example of another substrate to be stacked.

[0060] [ Figure 4A ]

[0061] Figure 4A This is a diagram showing a third layout example of one of the substrates to be stacked.

[0062] [ Figure 4B ]

[0063] Figure 4B This is a diagram showing a third layout example of another substrate to be stacked.

[0064] [ Figure 5A ]

[0065] Figure 5A This is a diagram showing a fourth layout example of one of the substrates to be stacked.

[0066] [ Figure 5B ]

[0067] Figure 5B This is a diagram showing a fourth layout example of another substrate to be stacked.

[0068] [ Figure 6 ]

[0069] Figure 6 This is a diagram showing the positional relationship between pixel array units and pixel AFE units.

[0070] [ Figure 7A ]

[0071] Figure 7A Is adopted Figure 2A and Figure 2B A schematic 3D diagram showing the layout and configuration.

[0072] [ Figure 7B ]

[0073] Figure 7B Is adopted Figure 3A and Figure 3B A schematic 3D diagram showing the layout and configuration.

[0074] [ Figure 8A ]

[0075] Figure 8A yes Figure 2A The layout diagram of the modified example.

[0076] [ Figure 8B ]

[0077] Figure 8B yes Figure 2B The layout diagram of the modified example.

[0078] [ Figure 8C ]

[0079] Figure 8C yes Figure 3A The layout diagram of the modified example.

[0080] [ Figure 8D ]

[0081] Figure 8D yes Figure 3B The layout diagram of the modified example.

[0082] [ Figure 9A ]

[0083] Figure 9A It shows Figure 7A A three-dimensional diagram of a modified example.

[0084] [ Figure 9B ]

[0085] Figure 9B It shows Figure 7B A three-dimensional diagram of a modified example.

[0086] [ Figure 10A ]

[0087] Figure 10A This diagram illustrates the connection between each pixel circuit and each sub-AFE unit.

[0088] [ Figure 10B ]

[0089] Figure 10B This diagram illustrates the connection between each pixel circuit and each sub-AFE unit.

[0090] [ Figure 10C ]

[0091] Figure 10C This diagram illustrates the connection between each pixel circuit and each sub-AFE unit.

[0092] [ Figure 10D ]

[0093] Figure 10D This diagram illustrates the connection between each pixel circuit and each sub-AFE unit.

[0094] [ Figure 11 ]

[0095] Figure 11 This is a block diagram illustrating an example of the internal structure of a column processing unit.

[0096] [ Figure 12 ]

[0097] Figure 12 This is a block diagram illustrating an example of the internal structure of a row drive unit.

[0098] [ Figure 13A ]

[0099] Figure 13A This is the layout diagram of the second substrate in the CoW method.

[0100] [ Figure 13B ]

[0101] Figure 13B This is the layout diagram of the first substrate in the CoW method.

[0102] [ Figure 14A ]

[0103] Figure 14A In the context of Figure 13A Layout diagram of a second substrate in which chips are arranged in different directions.

[0104] [ Figure 14B ]

[0105] Figure 14B In the context of Figure 13A Layout diagram of the first substrate in which chips are arranged in different directions.

[0106] [ Figure 15A ]

[0107] Figure 15A This is the first layout diagram of the third substrate of a camera device with a three-layer structure.

[0108] [ Figure 15B ]

[0109] Figure 15B This is a first layout diagram of the second substrate of a camera device with a three-layer structure.

[0110] [ Figure 15C ]

[0111] Figure 15C This is a first layout diagram of the first substrate of a camera device with a three-layer structure.

[0112] [ Figure 16A ]

[0113] Figure 16A This is a second layout diagram of the third substrate of a camera device with a three-layer structure.

[0114] [ Figure 16B ]

[0115] Figure 16B This is a second layout diagram of the second substrate of a camera device with a three-layer structure.

[0116] [ Figure 16C ]

[0117] Figure 16C This is a second layout diagram of the first substrate of a camera device with a three-layer structure.

[0118] [ Figure 17A ]

[0119] Figure 17A This is the third layout diagram of the third substrate of a camera device with a three-layer structure.

[0120] [ Figure 17B ]

[0121] Figure 17B This is the third layout diagram of the second substrate of a camera device with a three-layer structure.

[0122] [ Figure 17C ]

[0123] Figure 17C This is the third layout diagram of the first substrate of a camera device with a three-layer structure.

[0124] [ Figure 18 ]

[0125] Figure 18 This is a block diagram illustrating a schematic structure of a camera device without a column processing unit.

[0126] [ Figure 19A ]

[0127] Figure 19A This is a schematic diagram illustrating the generation of address event detection signals.

[0128] [ Figure 19B ]

[0129] Figure 19B This is a schematic diagram illustrating the generation of the SPAD signal.

[0130] [ Figure 19C ]

[0131] Figure 19C This is a schematic diagram illustrating the generation of grayscale signals.

[0132] [ Figure 20A ]

[0133] Figure 20A This is a diagram showing an example where one pixel circuit corresponds to a sub-AFE unit.

[0134] [ Figure 20B ]

[0135] Figure 20B This is a diagram illustrating an example where multiple pixel circuits correspond to a sub-AFE unit.

[0136] [ Figure 20C ]

[0137] Figure 20C This is a diagram illustrating an example where one pixel circuit corresponds to multiple sub-AFE units.

[0138] [ Figure 21A ]

[0139] Figure 21A This is a first layout diagram of the second substrate excluding the column processing unit.

[0140] [ Figure 21B ]

[0141] Figure 21B This is a first layout diagram of the first substrate without the column processing unit.

[0142] [ Figure 22A ]

[0143] Figure 22A This is a second layout diagram of the second substrate without the column processing unit.

[0144] [ Figure 22B ]

[0145] Figure 22B This is a second layout diagram of the first substrate excluding the column processing unit.

[0146] [ Figure 23A ]

[0147] Figure 23A This is a diagram showing a first example of a pixel array unit.

[0148] [ Figure 23B ]

[0149] Figure 23B This is a diagram showing a second example of a pixel array unit.

[0150] [ Figure 23C ]

[0151] Figure 23C This is a diagram showing a third example of a pixel array unit.

[0152] [ Figure 24 ]

[0153] Figure 24 This is a block diagram illustrating an example of an address event detection circuit.

[0154] [ Figure 25 ]

[0155] Figure 25 This is a circuit diagram illustrating an example of the internal structure of a current-to-voltage conversion circuit.

[0156] [ Figure 26 ]

[0157] Figure 26 This is a circuit diagram illustrating an example of the internal structure of a subtractor and a quantizer.

[0158] [ Figure 27 ]

[0159] Figure 27 This is a circuit diagram of the pixel circuit of a global shutter type camera device.

[0160] [ Figure 28 ]

[0161] Figure 28 This is a circuit diagram of the pixel circuit of a rolling shutter type camera device.

[0162] [ Figure 29 ]

[0163] Figure 29 This is a circuit diagram of the area surrounding the AD-type pixel circuit that performs A / D conversion on a pixel-area basis.

[0164] [ Figure 30 ]

[0165] Figure 30 This is a flowchart illustrating the processing procedure performed by the camera device 1 according to the first embodiment.

[0166] [ Figure 31 ]

[0167] Figure 31 This is a block diagram illustrating another construction example of an address event detection circuit.

[0168] [ Figure 32 ]

[0169] Figure 32 This is a block diagram showing the construction of a camera device according to the second construction example.

[0170] [ Figure 33 ]

[0171] Figure 33 This is a block diagram illustrating a schematic example of the construction of a vehicle control system.

[0172] [ Figure 34 ]

[0173] Figure 34 This diagram shows an example of the installation location of the camera unit and the vehicle exterior information detection unit. Detailed Implementation

[0174] The following description, with reference to the accompanying drawings, will illustrate an embodiment of the camera device. While the main components of the camera device will be described primarily, the device may also include other components or functions not shown or described. The following description does not exclude components or functions not shown or described.

[0175] (First Implementation Plan)

[0176] Figure 1 This is a block diagram showing a schematic construction of the camera device 1 according to the first embodiment. Figure 1The camera device 1 includes a pixel array unit 2, a row driving unit 3, a column processing unit 4, a pixel AFE unit 5, a column driving unit 6, a signal processing unit 7, and a system control unit 8. In some cases, Figure 1 The camera device 1 may include an additional signal processing unit 9.

[0177] The pixel array unit 2 includes a plurality of pixel circuits 2a arranged in the row and column directions. Each pixel circuit 2a includes a photoelectric conversion element and a readout circuit that reads out the electrical signal obtained by photoelectric conversion through the photoelectric conversion element. The readout circuit is a circuit that transmits the electrical signal obtained by photoelectric conversion to the pixel analog front-end (AFE) unit, which will be described below. More specifically, the readout circuit includes a transmission transistor, etc.

[0178] The row driving unit 3 sequentially drives each row in the pixel array unit 2. The row driving unit 3 drives row by row a plurality of pixel circuits 2a (hereinafter referred to as pixel groups) connected to each row in the pixel array unit 2. As described below, the row driving unit 3 includes a circuit section supplied with a high power supply voltage and a circuit section supplied with a low power supply voltage.

[0179] The column processing unit 4 sequentially reads the output signals of multiple pixel circuits 2a (hereinafter referred to as pixel groups) connected to each column of the pixel array unit 2, and performs analog-to-digital conversion. As described below, the column processing unit 4 includes a circuit section supplied with a high power supply voltage and a circuit section supplied with a low power supply voltage.

[0180] Pixel AFE unit 5 performs the operation of reading out the electrical signal obtained by photoelectric conversion through multiple photoelectric conversion elements. More specifically, pixel AFE unit 5 is a dedicated readout circuit that performs the operation of comparing and quantizing the voltage signal output from each pixel circuit 2a with a reference signal. For example, pixel AFE unit 5 includes a reset transistor, an amplification transistor, and a selection transistor. The power supply voltage supplied to pixel circuit 2a may be different from the power supply voltage supplied to pixel AFE unit 5, and for example, a power supply voltage with a lower voltage level than that of pixel circuit 2a may be supplied to pixel AFE unit 5.

[0181] Pixel AFE unit 5 is disposed in association with pixel array unit 2. As described below, in this embodiment, considering that pixel AFE unit 5 is disposed on a different substrate than the substrate on which pixel array unit 2 is disposed, pixel AFE unit 5 and pixel array unit 2 are disposed in a vertically overlapping position. This allows pixel AFE unit 5 and pixel array unit 2 to transmit or receive signals at high speed by joining them using Cu-Cu bonding, through-silicon vias (TSVs), or microbump bonding.

[0182] Similar to pixel array unit 2, pixel AFE unit 5 includes multiple sub-AFE units 5a arranged in the row and column directions. Each sub-AFE unit 5a converts the electrical signal output from the corresponding pixel circuit 2a into a voltage signal and performs gain adjustment. Each sub-AFE unit 5a can perform processing for quantizing the voltage signal.

[0183] The column driving unit 6 sequentially drives each column in the pixel AFE unit 5. The column driving unit 6 sequentially outputs voltage signals or quantized data from multiple sub-AFE units 5a connected to each column in the pixel AFE unit 5, and inputs the voltage signals or quantized data to the signal processing unit 7. As described below, the column driving unit 6 includes a circuit section supplied with a high power supply voltage and a circuit section supplied with a low power supply voltage.

[0184] Signal processing unit 7 performs various signal processing on the output signal of pixel AFE unit 5. For example, signal processing unit 7 performs correlated double sampling (CDS) processing or image recognition processing. When signal processing unit 7 cannot perform all signal processing alone, additional signal processing unit 9 can be used to perform further signal processing. Furthermore, a memory may be included for storing data indicating the results of signal processing performed by signal processing unit 7 or additional signal processing unit 9. Figure 1 (Not shown in the image).

[0185] The system control unit 8 controls each unit of the imaging device 1. For example, the system control unit 8 controls the timing of each row in the pixel array unit 2 driven by the row driving unit 3 and the timing of the output of each column of the pixel circuit 2a in the pixel array unit 2 read by the column processing unit 4. In addition, the system control unit 8 controls the timing of the pixel AFE unit 5 driven by the column driving unit 6 and the timing of the signal processing unit 7 performing signal processing.

[0186] The camera device 1 according to this embodiment is divided and arranged on multiple stacked substrates. A feature of this embodiment is that it minimizes the unused area of ​​each substrate. The substrate in this embodiment can be a wafer or a semiconductor chip (hereinafter simply referred to as a chip). In this embodiment, any of the following methods can be used: wafer-on-wax (WoW) for stacking wafers, chip-on-wax (CoW) for stacking wafers and chips, and chip-on-chip (CoC) for stacking chips.

[0187] Since the power supply voltage used by the camera device 1 varies depending on the circuits inside the camera device 1, it is conceivable, for example, to divide the substrate on which the circuits are mounted according to the power supply voltage level used by each circuit. For example, it can be divided into a substrate on which circuits with power supply voltages higher than a predetermined reference voltage level are mounted and a substrate on which circuits with power supply voltages equal to or lower than the reference voltage level are mounted.

[0188] Furthermore, the imaging device 1 includes internal circuitry for processing analog signals and circuitry for processing digital signals. Typically, circuitry for processing analog signals is susceptible to noise and other factors, making it difficult to microfabricate. On the other hand, even when circuitry for processing digital signals is microfabricated, there is little concern about degradation of its electrical characteristics. Therefore, a substrate with circuitry for processing analog signals and a substrate with circuitry for processing digital signals can be separated, and the circuitry on the substrate with circuitry for processing digital signals can be microfabricated. Microfabrication of the circuitry allows for the mounting of larger-scale circuitry on the substrate and reduces power consumption.

[0189] Figure 2A and Figure 2B This diagram shows a first layout example of two substrates 11 and 12 that will be stacked. Figure 2A The substrate 12 is stacked on Figure 2B Example on substrate 11. Figure 2B The substrate 11 can be a wafer or a chip. When Figure 2B When the substrate 11 is a chip, Figure 2A The substrate 12 can be a wafer or a chip, but when Figure 2B When the substrate 11 is a chip, Figure 2A The substrate 12 is a chip. Therefore, Figure 2B The substrate 11 has a similar Figure 2A The substrate 12 is the same or larger in size.

[0190] Pixel array unit 2 and column processing unit 4 are configured in Figure 2A On substrate 12. Figure 2A substrate 12 and Figure 2B The substrate 11 is joined by a plurality of joints 13. The joints 13 can be joined to the two substrates by adhesives or the like, or by Cu-Cu bonding, TSV, or microbump bonding, etc. Because Figure 2A The substrate 12 is stacked on Figure 2B On substrate 11, therefore, in the following text, it can be... Figure 2B The substrate 11 is referred to as the first substrate 11 and can be referred to in the following text as such. Figure 2A The substrate 12 is referred to as the second substrate 12.

[0191] exist Figure 2AThe illustration shows an example where pixel array unit 2 is disposed approximately in the central portion of the second substrate 12 and column processing unit 4 is disposed near pixel array unit 2; however, the specific placement of pixel array unit 2 and column processing unit 4 is arbitrary. However, as described below, since pixel array unit 2 performs signal transmission and reception with pixel AFE units 5 disposed on other substrates, it is preferable to place pixel array unit 2 and pixel AFE units 5 as close as possible to each other. Ideally, when stacking substrates 11 and 12, pixel array unit 2 and pixel AFE units 5 are preferably configured to overlap each other perpendicularly. Furthermore, from the perspective of reducing parasitic capacitance or resistance, it is preferable to place column processing unit 4 near pixel array unit 2.

[0192] Pixel AFE unit 5, column driving unit 6, row driving unit 3, signal processing unit 7, and additional signal processing unit 9 are configured in Figure 2B On the first substrate 11. The additional signal processing unit 9 can be omitted.

[0193] With the two substrates 11 and 12 stacked, when viewed from the normal direction of the substrate surface, the pixel AFE unit 5 is positioned perpendicularly overlapping with the pixel array unit 2. The row driving unit 3 is positioned along the first end face of the pixel AFE unit 5, the column driving unit 6 is positioned along the second end face of the pixel AFE unit 5, the signal processing unit 7 is positioned along the third end face of the pixel AFE unit 5, and an additional signal processing unit 9 is positioned along the fourth end face of the pixel AFE unit 5. Furthermore, the system control unit 8 is positioned along the corner of the pixel AFE unit 5.

[0194] like Figure 1 As shown, since the signal processing unit 7 uses the output of the column processing unit 4 to perform signal processing, therefore... Figure 2B The signal processing unit 7 in the first substrate 11 is configured with Figure 2A The column processing unit 7 and the column processing unit 4 are located at a position where they overlap vertically in the second substrate 12. The signal processing unit 7 and the column processing unit 4 are joined by Cu-Cu bonding, TSV, or microbump bonding, and perform the transmission and reception of various signals. Therefore, even when the column processing unit 4 and the signal processing unit 7 are disposed on different substrates, the signal processing unit 7 can quickly perform signal processing using the output of the column processing unit 4, without being affected by parasitic capacitance or parasitic resistance.

[0195] Therefore, in Figure 2AThe second substrate 12 is provided with not only pixel array units 2 but also column processing units 4, which reduces the unused area of ​​the second substrate 12 and allows the circuit mounting area of ​​the first substrate 11 to be close to that of the second substrate 12. Furthermore, since the circuit mounting area of ​​the second substrate 12 can be reduced, the substrate size of either the first substrate 11 or the second substrate 12 can be reduced, and miniaturization of the imaging device 1 can be achieved. Moreover, since the circuits in both substrates 11 and 12 are arranged in signal flow sequence, the signal propagation delay time can be shortened, and the circuits are less susceptible to noise.

[0196] Figure 3A and Figure 3B This diagram illustrates a second layout example of two substrates 11 and 12 to be stacked. Since the column processing unit 4 performs A / D conversion processing for each column, it requires multiple comparators, multiple counters, multiple switches, and multiple memories, etc. Therefore, the more columns there are, the larger the circuitry of the column processing unit 4 becomes. Furthermore, the power supply voltages used by the comparators and counters in the column processing unit 4 may differ. Therefore, the column processing unit 4 is divided into two parts and partially disposed on the first substrate 11 and the second substrate 12. More specifically, in the column processing unit 4, circuitry sections using high power supply voltages, such as comparators, are disposed on the second substrate 12, while circuitry sections using low power supply voltages, such as counters, are disposed on the first substrate 11. This allows for a reduction in the types of power supply voltages supplied to the first substrate 11 and the second substrate 12.

[0197] Since the signal processing unit 7 receives signals from both the pixel AFE unit 5 and the column processing unit 4, the signal processing unit 7 is configured in... Figure 3B Between the pixel AFE unit 5 and the column processing unit 4 on the first substrate 11.

[0198] Therefore, in Figure 3A and Figure 3B In the second layout example, when the circuit size of the column processing unit 4 is large or the power supply voltage level used by the column processing unit 4 varies depending on the internal circuitry of the column processing unit 4, since the column processing unit 4 is divided and disposed on the first substrate 11 and the second substrate 12, variations in the circuit mounting area of ​​the first substrate 11 and the second substrate 12 can be suppressed, and the supplied power supply voltage can be allocated to each substrate. Furthermore, the signal processing unit 7 is disposed between the pixel AFE unit 5 and the column processing unit 4, thereby improving the signal transmission and reception speed between the signal processing unit 7 and the column processing unit 4, as well as the signal transmission and reception speed between the signal processing unit 7 and the pixel AFE unit 5.

[0199] Figure 4A and Figure 4BThis is a diagram showing a third layout example of two substrates 11 and 12 to be stacked. In the third layout example, in the circuit constituting the row driving unit 3, circuit portions driven by high voltage (e.g., level shifters) and circuit portions driven by low voltage, such as level shifters, are arranged on different substrates.

[0200] Line drive unit 3 is disposed on the first substrate 11 and Figure 4A The row driving unit 3 is located at a vertically overlapping position on the second substrate 12. The row driving unit 3 in the second substrate 12 includes a circuit portion driven by a high voltage, such as a level shifter, and the row driving unit 3 in the first substrate 11 includes a circuit portion driven by a low voltage, such as a shift register. The row driving units 3 of each substrate are joined by Cu-Cu bonding, TSV, or microbump bonding, etc.

[0201] Despite Figure 4A and Figure 4B In the middle, the column processing unit 4 is disposed on the second substrate 12, but, with Figure 3A and Figure 3B Similarly, the column processing unit 4 can be divided and disposed on the first substrate 11 and the second substrate 12. Alternatively, the column processing unit 4 can be disposed on the first substrate 11 instead of the second substrate 12.

[0202] Figure 5A and Figure 5B This diagram illustrates a fourth layout example of two substrates 11 and 12 to be stacked. In this fourth layout example, the row driving unit 3 and the column processing unit 4 are disposed on the second substrate 12. This allows for a further reduction in the difference in circuit mounting area between the first substrate 11 and the second substrate 12.

[0203] Besides the first to fourth layout examples mentioned above, various other layout examples can be considered. The common feature of the first to fourth layouts is that: Figure 6 As shown, the pixel array unit 2 on the second substrate 12 and the pixel AFE unit 5 on the first substrate 11 are arranged in a vertically overlapping position. This allows for efficient readout from each pixel of the pixel array unit 2. The pixel array unit 2 and the pixel AFE unit 5 can partially overlap each other vertically, and they do not always need to be completely vertically overlapped. Furthermore, the circuit mounting areas of the pixel array unit 2 and the pixel AFE unit 5 do not necessarily have to be the same.

[0204] Figure 7A Is adopted Figure 2A and Figure 2B A schematic 3D diagram illustrating the layout and configuration. Figure 2A and Figure 2BIn this case, the column processing unit 4 is disposed on the second substrate 12, and the signal processing unit 7 is disposed on the first substrate 11. For example... Figure 7A As shown, the column processing unit 4 and the signal processing unit 7 are arranged in a vertically overlapping position. Therefore, the digital pixel data obtained by the A / D conversion of the column processing unit 4 can be transmitted to the signal processing unit 7 in the first substrate 11 over the shortest distance via Cu-Cu bonding or the like, enabling rapid signal processing without being affected by parasitic capacitance or resistance of the signal lines. Furthermore, in Figure 7A In the layout configuration, since analog signals are mainly processed in the second substrate 12 and digital signals are mainly processed in the first substrate 11, the circuits on the first substrate 11 can be formed by microfabrication technology.

[0205] Figure 7B Is adopted Figure 3A and Figure 3B A schematic 3D diagram illustrating the layout and configuration. Figure 3A and Figure 3B In this case, since the column processing unit 4 is disposed on both the first substrate 11 and the second substrate 12, for example, the first half of the processing of the column processing unit 4 using a high power supply voltage can be performed in the column processing unit 4 on the second substrate 12, and the second half of the processing of the column processing unit 4 using a low power supply voltage can be performed in the column processing unit 4 on the first substrate 11. Figure 7B As shown, since the column processing unit 4 of the first substrate 11 and the column processing unit 4 of the second substrate 12 are arranged in a vertically overlapping position, signal transmission and reception can be performed quickly between the column processing units 4 of the two substrates.

[0206] In the first to fourth layout examples described above, although the pixel array unit 2 is disposed in the approximately central portion of the second substrate 12, the placement of the pixel array unit 2 is arbitrary. Furthermore, the number or placement of the joint portions 13 is arbitrary.

[0207] Figure 8A and Figure 8B yes Figure 2A and Figure 2B A variation of the above. Figure 8C and Figure 8D yes Figure 3A and Figure 3B A variation. In Figure 8A and Figure 8C In the second substrate 12 shown, the pixel array unit 2 is arranged along one end edge of the second substrate 12, and the bonding portion 13 is arranged along the other end edge opposite to that one end edge. Figure 8B and Figure 8DIn the first substrate 11 shown, the pixel AFE unit 5 is configured to overlap vertically with the pixel array unit 2 of the second substrate 12, and the joint portion 13 is also configured to overlap vertically with each other in the first substrate 11 and the second substrate 12.

[0208] As described above, various variations of the layout configuration of each circuit in the first substrate 11 and the second substrate 12 can be considered.

[0209] Figure 9A It shows Figure 7A A three-dimensional diagram of a modified example, and Figure 9B It shows Figure 7B A three-dimensional diagram of a modified example. In Figure 9A In this design, the column processing unit 4 of the second substrate 12 is divided into two parts and arranged along two opposite end edges of the second substrate 12. Hereinafter, these two divided column processing units 4 are referred to as split column processing units 4a. Similarly, the signal processing unit 7 of the first substrate 11 is also divided into two parts and arranged along two opposite end edges of the first substrate 11. Hereinafter, these two divided signal processing units 7 are referred to as split signal processing units 7a. The split column processing unit 4a and the split signal processing unit 7a at each end edge are arranged to overlap each other perpendicularly. The split column processing unit 4a and the split signal processing unit 7a are joined by Cu-Cu bonding, TSV, or microbump bonding, etc., and perform the transmission and reception of various signals.

[0210] Figure 9B and Figure 9A Similarly, in either the first substrate 11 or the second substrate 12, two segmented column processing units 4a are arranged along two opposite end edges.

[0211] Therefore, the column processing unit 4 is divided into two and arranged along the two opposite ends of the second substrate 12, so that the distance from the pixel array unit 2 to the column processing unit 4 can be as uniform as possible. Similarly, the signal processing unit 7 is divided into two and arranged along the two opposite ends of the first substrate 11, so that the distance from the pixel AFE unit 5 to the signal processing unit 7 can be as uniform as possible.

[0212] like Figure 6 As shown, in this embodiment, pixel array unit 2 and pixel AFE unit 5 are configured at vertically overlapping positions on different substrates, enabling pixel array unit 2 and pixel AFE unit 5 to rapidly transmit and receive various signals. Pixel array unit 2 and pixel AFE unit 5 transmit and receive various signals through Cu-Cu bonding, TSV, or microbump bonding.

[0213] Figure 10A , Figure 10B , Figure 10C and Figure 10D This diagram illustrates the connection between each pixel circuit 2a in pixel array unit 2 and each sub-AFE unit 5a in pixel AFE unit 5. Figure 10A The connection method shows an example of how each pixel circuit 2a in the pixel array unit 2 and a corresponding sub-AFE unit 5a in the pixel AFE unit 5 transmit and receive various signals. Figure 10B An example is shown of how multiple pixel circuits 2a in pixel array unit 2 and a corresponding sub-AFE unit 5a in pixel AFE unit 5 transmit and receive various signals. Figure 10C An example is shown of how each pixel circuit 2a in pixel array unit 2 and the corresponding multiple sub-AFE units 5a in pixel AFE unit 5 transmit and receive various signals. Figure 10D An example is shown in which some pixel circuits 2a in pixel array unit 2 and a corresponding sub-AFE unit 5a in pixel AFE unit 5 transmit and receive various signals.

[0214] The connection method between pixel array unit 2 and pixel AFE unit 5 can be as follows: Figures 10A to 10D As shown in either of them. Figure 10D As shown, some pixel circuits 2a in pixel array unit 2 do not perform the functions of sending signals to pixel AFE unit 5 or receiving signals from pixel AFE unit 5.

[0215] Figure 11 This is a block diagram illustrating an example of the internal structure of column processing unit 4. (See diagram for example.) Figure 11 As shown, the column processing unit 4 includes a comparator 21, an up / down counter (hereinafter simply referred to as a counter) 23, and a memory 24 for each column signal line extending in the column direction. Furthermore, the column processing unit 4 is provided with a DAC 22.

[0216] DAC 22 generates a reference signal. Comparator 21 compares the voltage signal on the column signal line with the reference signal. Counter 23 counts the value over a period of time until the comparison result from comparator 21 is inverted. The count value of counter 23 is stored in memory 24. A digital signal indicating the count value of counter 23 stored in memory 24 is transmitted to signal processing unit 7.

[0217] A high power supply voltage is supplied to the column processing unit 4. Figure 11 The circuit section above the dashed line Ln1 (comparator 21, etc.) supplies a low power supply voltage to the circuit section below the dashed line Ln1 (counter 23 and memory 24, etc.). Therefore, in the above... Figure 2A and Figure 3A In this configuration, a high power supply voltage supply circuit section (comparator 21, etc.) that is part of the column processing unit 4 is disposed on the second substrate 12.

[0218] Figure 12 This is a block diagram illustrating an example of the internal structure of the row drive unit 3. (See diagram for example.) Figure 12 As shown, the row driving unit 3 includes an address selection logic unit 25, multiple decoders 26, multiple memories 27, multiple level shifters 28, and multiple drivers 29. Each driver 29 is connected to a row selection line of the pixel array unit 2.

[0219] Address selection logic unit 25 transmits the address signal sent from system control unit 8 to each decoder 26. Decoder 26 decodes the address signal. The decoded signal is temporarily stored in memory 27. Level shifter 28 shifts the voltage level of the data stored in memory 27 and supplies the resulting data to driver 29. Driver 29 drives the corresponding column select line according to the timing of the address signal. Signal transmission and reception can be performed between multiple drivers 29 and multiple memories 27.

[0220] A low power supply voltage is supplied to the address selection logic unit 25, decoder 26, and memory 27 in the row drive unit 3. Furthermore, a high power supply voltage is supplied to the level shifter 28 and driver 29. Therefore, in the above... Figure 4A In the above, a portion of the line drive unit 3 is disposed on the second substrate 12.

[0221] As described above, the substrate of this embodiment is a wafer or chip. When the substrate is composed of chips, multiple chips can be configured with the same layer height. More specifically, the first substrate 11 can be a wafer, the second substrate 12 can be multiple chips, and the multiple chips can be stacked on the wafer via chip-on-wafer (CoW).

[0222] Figure 13A and Figure 13B This is a layout diagram showing a CoW example. Figure 13A The layout of the second substrate 12 is shown, and Figure 13BThe layout of the first substrate 11 is shown. The first substrate 11 is a wafer used as a substrate. Pixel AFE units 5, row driving units 3, column driving units 6, column processing units 4, signal processing units 7, and system control units 8 are disposed on the first substrate 11. The second substrate 12 consists of two chips. Pixel array units 2 are disposed on one of these chips (hereinafter referred to as the first chip) 15. Signal processing units 7 are disposed on the other chip (hereinafter referred to as the second chip) 16. The signal processing units 7 on the second chip 16 are disposed at a position perpendicularly overlapping with the signal processing units 7 on the first substrate 11. Similarly, the pixel array units 2 on the first chip 15 are disposed at a position perpendicularly overlapping with the pixel AFE units 5 on the first substrate 11. Pixel array units 2, which process analog signals, are disposed on the first chip 15, while signal processing units 7, which process digital signals, are disposed on the second chip 16. Therefore, the signal processing units 7 can be formed in the second chip 16 using microfabrication processes, and even when the circuit size of the signal processing units 7 is large, the signal processing units 7 can be mounted in the second chip 16, which is smaller than the first chip 15. In addition, the types of power supply voltages supplied to the first chip 15 and the second chip 16 can be different.

[0223] The configuration position or size of the first chip 15 and the second chip 16 constituting the first substrate 11 is arbitrary. Figure 14A and Figure 14B This illustrates that the first chip 15 and the second chip 16 are configured in conjunction with... Figure 13A and Figure 13B Layout diagrams of examples in different orientations. First substrate 11 and second substrate 12 are... Figure 14A and Figure 14B The layout and in Figure 13A and Figure 13B The layout is the same as in the previous one.

[0224] Although an example of a camera device 1 consisting of two stacked substrates 11 and 12 has been shown above, a camera device 1 can also be constructed by stacking three or more substrates. Figure 15A , Figure 15B and Figure 15C This is a diagram showing a first layout example of a camera device 1 with a three-layer structure. Figure 15A The layout configuration of the topmost third substrate 14 is shown. Figure 15B The layout configuration of the second substrate 12 of the second stage is shown, and Figure 15C The layout configuration of the bottommost first substrate 11 is shown.

[0225] Pixel array unit 2 is disposed on third substrate 14. Pixel AFE unit 5 is disposed on second substrate 12 at a position perpendicularly overlapping with pixel array unit 2. Row driving unit 3, column driving unit 6, column processing unit 4, signal processing unit 7, and system control unit 8 are disposed on first substrate 11. In the first layout example, the free area of ​​first substrate 11 is the largest, followed by the free area of ​​second substrate 12. Therefore, the free area varies from first substrate to third substrate 14.

[0226] Figure 16A , Figure 16B and Figure 16C This diagram illustrates a second layout example of a camera device 1 with a three-layer structure. In the second layout example, as... Figure 16B As shown, in the first layout example, the row driving unit 3 and column processing unit 4, which are disposed on the first substrate 11, are disposed on the second substrate 12. This allows for the suppression of variations in the free areas of the second substrate 12 and the third substrate 14. Furthermore, since the column processing unit 4 and the column driving unit 6 use high power supply voltages, the circuitry using high power supply voltages can be integrated on the second substrate 12, while only logic circuitry can be disposed on the first substrate 11. Moreover, the first substrate 11 can be formed using microfabrication processes, thereby achieving low power consumption.

[0227] Figure 17A , Figure 17B and Figure 17C This diagram illustrates a third layout example of a camera device 1 with a three-layer structure. In this third layout example, the first substrate 11 is composed of a first chip 15 and a second chip 16. A column processing unit 4 is disposed on the first chip 15, and a signal processing unit 7 and a system control unit 8 are disposed on the second chip 16. In this third layout example, when there is no free area on the second substrate 12 for arranging the column processing unit 4 and the row driving unit 3 which use high power supply voltage, the first substrate 11 is divided into two chips. The column processing unit 4, which uses high power supply voltage, can be disposed on the first chip 15, and only the logic circuit portion can be disposed on the second chip 16. This allows the second chip 16 to be formed using microfabrication processes and reduces power consumption.

[0228] In the above description, an example of a camera device 1 having a column processing unit 4 has been given, but it is also conceivable to have a camera device 1 without a column processing unit 4. This embodiment is also applicable to such a camera device 1.

[0229] Figure 18 This is a block diagram showing a schematic structure of a camera device 1 without the column processing unit 4. Figure 18 The camera device 1 has a range from Figure 1 The construction of column processing unit 4 is omitted. Row-by-row processing is driven by row driving unit 3. Figure 18 The pixel array unit 2. The electrical signals converted by photoelectric conversion by multiple pixel circuits 2a connected to each row are sequentially transmitted to the pixel AFE unit 5. In addition, when the address event detection circuit described below is provided in the pixel circuit 2a, the event detection signal detected by the address event detection circuit is also transmitted to the pixel AFE unit 5.

[0230] The pixel array unit 2 is used not only for imaging purposes, but also for address event detection, light signal receiving position and light receiving timing detection, or A / D conversion of each pixel.

[0231] Figure 19A This diagram schematically illustrates the generation of an address event detection signal (hereinafter referred to as the DVS (Dynamic Vision Sensor) signal) when pixel array unit 2 is used for address event detection. When the pixel circuit 2b (hereinafter referred to as the pixel circuit 2b for DVS) in pixel array unit 2 detects an address event, the DVS signal is output from pixel AFE unit 5.

[0232] Figure 19B This diagram schematically illustrates the generation of a single-photon avalanche diode (SPAD) signal when pixel array unit 2 is used for optical signal reception position and timing detection. When the pixel circuit 2c for SPAD in pixel array unit 2 detects an optical signal, a digitized SPAD signal is output from pixel AFE unit 5. In pixel AFE unit 5, A / D conversion processing is performed.

[0233] Figure 19C This diagram schematically illustrates the generation of grayscale signals when each pixel of the pixel array unit 2 undergoes A / D conversion. For example, pixels 2d for phase difference detection are provided in at least a portion of the pixel array unit 2. The pixels 2d for phase difference detection are constructed such that a pixel is divided into left and right parts, and photoelectric conversion is performed on each divided pixel, detecting the difference in the electrical signals obtained by the photoelectric conversion between the two divided pixels. This difference corresponds to the defocusing amount and can be used for autofocus, etc. The aforementioned difference, or grayscale signal, is then converted by the pixel A / D unit 5 and output.

[0234] Without column processing unit 4, there are multiple correspondences between each pixel circuit 2a in pixel array unit 2 and each sub-AFE unit 5a in pixel AFE unit 5. Figure 20A An example is shown in which a pixel circuit 2a in pixel array unit 2 corresponds to a sub-AFE unit 5a in pixel AFE unit 5. Figure 20B An example is shown in which multiple pixel circuits 2a in pixel array unit 2 correspond to a sub-AFE unit 5a in pixel AFE unit 5. Figure 20C An example is shown in which one pixel circuit 2a in pixel array unit 2 corresponds to multiple sub-AFE units 5a in pixel AFE unit 5.

[0235] Figure 21A and Figure 21B This is a diagram showing a first layout example of a camera device 1 without the column processing unit 4. Figure 21A The layout configuration of the second substrate 12 is shown, and Figure 21B The layout of the first substrate 11 is shown. Pixel AFE unit 5, row driving unit 3, column driving unit 6, signal processing unit 7, and system control unit 8 are disposed on the first substrate 11. The second substrate 12 includes a first chip 15 and a second chip 16. Pixel array unit 2 is disposed on the first chip 15. Signal processing unit 7 is disposed on the second chip 16. The signal processing unit 7 in the second chip 16 is disposed at a position perpendicularly overlapping with the signal processing unit 7 on the first substrate 11. Since the second chip 16 can be formed using microfabrication processes, the size of the second chip 16 can be smaller than the size of the first chip 15.

[0236] Figure 22A and Figure 22B This is a diagram showing a second layout example of a camera device 1 without the column processing unit 4. Figure 22A and Figure 21A The difference lies in: in the configuration Figure 22A The signal processing unit 7 of the second chip 16 in the second substrate 12 shown includes a memory 27, and other structures, including the layout of the first substrate 11, are also included. Figure 21A and Figure 21B The same applies. Microfabrication technology is used on the second chip 16, allowing circuits requiring a larger mounting area, such as memory 27, to be implemented using relatively smaller chips.

[0237] In at least a portion of the pixel array unit 2, a pixel circuit 2b for DVS, a pixel circuit 2c for SPAD, or a pixel circuit 2d for pixel-level A / D conversion can be provided. That is, the pixel array unit 2 can contain at least two of the following: pixel circuit 2a for imaging, pixel circuit 2b for DVS, pixel circuit 2c for SPAD, and pixel circuit 2d for pixel A / D conversion. The internal structure of the pixel AFE unit 5 can be modified according to the type of pixel circuit in the pixel array unit 2.

[0238] Figure 23A This is a diagram showing a first example of pixel array unit 2. Figure 23AThe pixel array unit 2 shows an example in which the pixel circuit 2b for DVS is set in a portion of the pixel circuit 2a for imaging. In this case, when the pixel circuit 2b for DVS detects an address event, an address event detection signal (DVS signal) is output from the pixel AFE unit 5.

[0239] Figure 23B This is a diagram showing a second example of pixel array unit 2. Figure 23B The pixel array unit 2 shows an example in which the pixel circuit 2c for SPAD is arranged in a portion of the pixel circuit 2a for imaging. In this case, when the pixel circuit 2c for SPAD detects a light signal, the pixel AFE unit 5 outputs a SPAD signal indicating the detection position and detection timing.

[0240] Figure 23C This is a diagram showing a third example of pixel array unit 2. Figure 23C The pixel array unit 2 shows an example in which a pixel circuit 2d for performing A / D conversion on a pixel-by-pixel basis is disposed in a portion of a pixel circuit 2a for imaging. In this case, the electrical signal obtained by photoelectric conversion through the pixel circuit 2d for performing A / D conversion on a pixel-by-pixel basis is transmitted to the pixel AFE unit 5, and the digital pixel data obtained by A / D conversion is output.

[0241] When pixel array unit 2 and pixel AFE unit 5 detect an address event, an address event detection circuit is set in at least one of pixel array unit 2 and pixel AFE unit 5.

[0242] Figure 24 This is a block diagram showing an example of an address event detection circuit 300. Figure 24 The address event detection circuit 300 includes a current-to-voltage conversion circuit 310, a buffer 320, a subtractor 330, a quantizer 340, and a transmission circuit 350.

[0243] The current-to-voltage conversion circuit 310 converts the photocurrent from the corresponding photodiode 221 into a voltage signal. The current-to-voltage conversion circuit 310 supplies the voltage signal to the buffer 320.

[0244] The buffer 320 corrects the voltage signal from the current-to-voltage conversion circuit 310. The buffer 320 outputs the corrected voltage signal to the subtractor 330.

[0245] Subtractor 330 reduces the level of the voltage signal from buffer 320 based on the row drive signal from row drive circuit 251. Subtractor 330 then supplies the reduced voltage signal to quantizer 340.

[0246] The quantizer 340 quantizes the voltage signal from the subtractor 330 into a digital signal and outputs the digital signal as a detection signal to the transmission circuit 350.

[0247] The transmission circuit 350 sends the detection signal from the quantizer 340 to the signal processing circuit based on the column drive signal from the column drive circuit.

[0248] For example, Figure 24 The current-to-voltage conversion circuit 310 and the buffer 320 are mounted on the pixel array unit 2, and the subtractor 330, the quantizer 340 and the transmission circuit 350 are mounted on the pixel AFE unit 5.

[0249] Figure 25 This is a circuit diagram showing an example of the internal structure of the current-to-voltage conversion circuit 310. Figure 25 The current-to-voltage conversion circuit 310 includes N-type transistors 311, 313, and P-type transistors 312. For example, metal-oxide-semiconductor (MOS) transistors are used as these transistors.

[0250] The source of N-type transistor 311 is connected to the cathode of photodiode 221, and its drain is connected to the power supply terminal. P-type transistor 312 and N-type transistor 313 are connected in series between the power supply terminal and the ground terminal. Furthermore, the junction of P-type transistor 312 and N-type transistor 313 is connected to the gate of N-type transistor 311 and the input terminal of buffer 320. Additionally, a predetermined bias voltage Vbias1 is applied to the gate of P-type transistor 312.

[0251] The drains of N-type transistors 311 and 313 are connected to the power supply side, and this circuit is called a source follower. The photocurrent from photodiode 221 is converted into a voltage signal through the two source followers connected in a loop. In addition, P-type transistor 312 supplies a constant current to N-type transistor 313.

[0252] Furthermore, as a countermeasure against interference, the ground wires of the optical receiver chip 201 and the detection chip 202 are separated from each other.

[0253] Figure 26 This is a circuit diagram illustrating an example of the internal structure of subtractor 330 and quantizer 340. Subtractor 330 includes capacitor 331, capacitor 333, inverter 332, and switch 334.

[0254] One end of capacitor 331 is connected to the output terminal of buffer 320, and the other end is connected to the input terminal of inverter 332. Capacitor 333 is connected in parallel with inverter 332. Switch 334 opens or closes the path connecting the two ends of capacitor 333 according to the horizontal drive signal.

[0255] Inverter 332 inverts the voltage signal input through capacitor 331. Inverter 332 outputs the inverted signal to the non-inverting input terminal (+) of comparator 341.

[0256] Incidentally, in the rolling shutter type camera device 1, since the photoelectric conversion results are transmitted simultaneously with the progressive scan of the pixel array unit 2, it is possible to obtain an image in which fast-moving objects are distorted and can be visually identified. Therefore, a global shutter type camera device 1 is proposed in which the photoelectric conversion results for one frame are stored in the memory 27, and the photoelectric conversion results are read from the memory 27 to generate the captured image.

[0257] Figure 27 This is a circuit diagram of the pixel circuit 2a of the global shutter type camera device 1. Figure 27 The pixel circuit 2a includes a photoelectric conversion element 31, a transmission transistor 32, a reset transistor 33, a buffer 34, and a memory 35. The memory 35 includes a P-phase memory cell 35a and a D-phase memory cell 35b.

[0258] The P-phase storage unit 35a stores the potential at the time of reset. The D-phase storage unit 35b stores the potential corresponding to the electrical signal obtained through photoelectric conversion. Potential changes can be offset by detecting the difference between the potential stored in the P-phase storage unit 35a and the potential stored in the D-phase storage unit 35b. The potentials in the P-phase storage unit 35a and the D-phase storage unit 35b are re-stored each time a recording is performed.

[0259] In this implementation plan, Figure 27 To the left of the dashed line, the photoelectric conversion element 31, the transmission transistor 32 and the reset transistor 33 are disposed in the pixel array unit 2 of the second substrate 12, and to the right of the dashed line, the memory 35 is disposed in the pixel AFE unit 5 of the first substrate 11.

[0260] Figure 28 This is a circuit diagram of the pixel circuit 2a of the rolling shutter type camera device 1. Figure 28 The pixel circuit 2a includes a photoelectric conversion element 31, a transmission transistor 32, a reset transistor 33, an amplification transistor 36, and a selection transistor 37.

[0261] In this implementation plan, Figure 28 To the left of the dashed line, the photoelectric conversion element 31 and the transmission transistor 32 are disposed in the pixel array unit 2 of the second substrate 12, and to the right of the dashed line, the reset transistor 33, the amplification transistor 36 and the selection transistor 37 are disposed in the pixel AFE unit 5 of the first substrate 11.

[0262] Figure 29This is a circuit diagram of the area surrounding the AD-type pixel circuit 2a, which performs A / D conversion on a pixel-area basis. Figure 29 In the pixel circuit 2a, multiple pixel circuits 2a are connected to one A / D converter (hereinafter referred to as ADC). Each pixel circuit 2a includes a photoelectric conversion element 31, a transmission transistor 32, a reset transistor 33, an amplification transistor 36, and a selection transistor 37.

[0263] In this implementation plan, Figure 29 The pixel array unit 2 on the left side of the dashed line is disposed in the pixel AFE unit 5 on the first substrate 11, and the ADC 30 on the right side of the dashed line is disposed in the pixel AFE unit 5 on the first substrate 11.

[0264] Figure 30 This is a flowchart illustrating the processing procedure performed by the imaging device 1 according to the first embodiment. First, the pixel AFE unit 5 (readout dedicated circuit) disposed on a different substrate from the substrate on which the pixel array unit 2, which includes a plurality of photoelectric conversion elements 31, is disposed, performs a readout operation of the electrical signal obtained by photoelectric conversion through the plurality of photoelectric conversion elements 31 (step S1).

[0265] Then, operations other than the operation of the pixel AFE unit 5 are performed in the circuit of a substrate different from the substrate on which the pixel AFE unit 5 is disposed (step S2).

[0266] Therefore, in this embodiment, the pixel array unit 2 and the pixel AFE unit 5 are disposed on different substrates, and the circuitry in the imaging device 1 that performs operations other than those of the pixel AFE unit 5 is disposed on a different substrate from the substrate on which the pixel AFE unit 5 is disposed. This allows for suppression of variations in the circuit mounting area on each substrate, reduction of the unused area on each substrate, and lower power consumption.

[0267] For example, by dividing a substrate on which circuitry using high power supply voltage is disposed and a substrate on which circuitry using low power supply voltage is disposed, the types of power supply voltages supplied to each substrate can be reduced, the wiring patterns for supplying power supply voltages can be shortened, and power supply noise can be reduced. More specifically, for circuits such as column processing unit 4, row driving unit 3, and column driving unit 6, which include circuitry portions using high power supply voltage and circuitry portions using low power supply voltage, by dividing each circuitry portion and distributing it on multiple substrates, it is possible to reduce substrate size or power consumption.

[0268] When multiple circuit sections for transmitting and receiving signals are divided and arranged on multiple substrates, each circuit section is arranged in a vertically overlapping position, and each circuit section is joined by Cu-Cu bonding, TSV or micro-bump bonding, etc. Therefore, each circuit section is not easily affected by parasitic resistance or parasitic capacitance, and the signal propagation delay time can be shortened.

[0269] Furthermore, by separating the substrate on which the circuitry, including analog circuitry, is disposed and the substrate on which the digital circuitry is disposed, the circuitry on the substrate on which the digital circuitry is disposed can be formed using state-of-the-art microfabrication technology, thereby reducing the substrate size and power consumption.

[0270] The pixel array unit 2 can be used not only for imaging purposes, but also for address event detection, light signal receiving position and light receiving timing detection, and A / D conversion in units of pixels, etc., and the circuits constituting the imaging device 1 can be distributed to multiple substrates according to each purpose.

[0271] (Second Implementation Plan)

[0272] In the first embodiment described above, for example, it has been explained that it includes... Figure 24 The address event detection circuit 300 is a camera device 1, but the internal structure of the address event detection circuit 300 is not necessarily limited to... Figure 24 . Figure 31 This is a block diagram illustrating another construction example of the address event detection circuit 300. In addition to the current-to-voltage conversion circuit 310, buffer 320, subtractor 330, quantizer 340, and transmission circuit 350, according to... Figure 31 The address event detection circuit 300 of the example configuration also includes a storage unit 360 and a control unit 370. In the following text, it will be included... Figure 24 The camera device 1 of the address event detection circuit 300 is referred to as the first configuration example, and will include... Figure 31 The camera device 1 of the address event detection circuit 300 is referred to as the second construction example.

[0273] Storage unit 360 is disposed between quantizer 340 and transmission circuit 350, and stores the output of quantizer 340 (i.e., the comparison result of comparator 3341 in quantizer 340) based on the sampled signal supplied from control unit 370. Storage unit 360 may be a sampling circuit such as a switch or capacitor, or a digital storage circuit such as a latch or flip-flop.

[0274] Control unit 370 will set a predetermined threshold voltage V th The inverting (-) input terminal of comparator 3341 is supplied. The threshold voltage V supplied from control unit 370 to comparator 3341 is...th Different voltage values ​​can be used in a time-division manner. For example, the control unit 370 supplies different threshold voltages V corresponding to on-events when the change in the indicator photocurrent exceeds the upper limit threshold. th1 And the threshold voltage V corresponding to the off-event when the indicated change drops below the lower limit threshold. th2 This allows a single comparator 3341 to detect multiple types of address events.

[0275] For example, a threshold voltage V corresponding to the disconnection event is supplied from the control unit 370 to the inverting (-) input terminal of the comparator 3341. th2 During this period, storage unit 360 can store the threshold voltage V that has been used corresponding to the conduction event. th1 The comparison result of comparator 3341. Storage cell 360 can be inside or outside pixel 216. Furthermore, storage cell 360 is not a necessary component of address event detection circuit 300. That is, storage cell 360 can be omitted.

[0276] [According to the second construction example, the camera device 1 (scanning type)]

[0277] Including the above Figure 24 The camera device 210 of the first configuration example of the address event detection circuit 300 shown is an asynchronous camera device 1 that reads events using an asynchronous readout method. However, the event readout method is not limited to asynchronous readout and can also be synchronous readout. The camera device 1 that applies synchronous readout is the same scanning camera device 1 as a typical camera device 1 that performs imaging at a predetermined frame rate.

[0278] Figure 32 This is a block diagram showing an example of the construction of a camera device 1 (i.e., a scanning camera device 1) according to a second construction example, which is used as a camera device 210 in a camera system applying the technology according to the invention.

[0279] like Figure 32 As shown, the camera device 210 according to the second construction example of the camera device 1 of the present invention includes a pixel array unit 211, a signal processing unit 212, a driving unit 213, a readout area selection unit 214, and a signal generation unit 215.

[0280] Pixel array unit 211 includes a plurality of pixels 216. The plurality of pixels 216 output output signals in response to a selection signal from readout region selection unit 214. For example, each of the plurality of pixels 216 can be configured as follows: Figure 24 As shown, each pixel has a quantizer. Multiple pixels 216 output signals corresponding to changes in light intensity. (As shown...) Figure 32 As shown, multiple pixels 216 can be arranged in a matrix-like two-dimensional configuration.

[0281] The driving unit 213 drives each of the plurality of pixels 216 so that the pixel signal generated by each pixel 216 is output to the signal processing unit 212. The driving unit 213 and the signal processing unit 212 are circuit units used to acquire grayscale information. Therefore, when only event information is acquired, the driving unit 213 and the signal processing unit 212 can be omitted.

[0282] The readout region selection unit 214 selects some pixels from the plurality of pixels 216 included in the pixel array unit 211. For example, the readout region selection unit 214 selects any one or more rows included in the two-dimensional matrix structure corresponding to the pixel array unit 211. The readout region selection unit 214 selects one or more rows sequentially according to a preset period. In addition, the readout region selection unit 214 can determine the selection region in response to requests from each pixel 216 of the pixel array unit 211.

[0283] The signal generation unit 215 generates an event signal corresponding to an active pixel among the selected pixels where an event has been detected, based on the output signal of the pixel selected by the readout region selection unit 214. The event is a change in light intensity. An active pixel is a pixel whose change in light intensity corresponding to the output signal exceeds or falls below a preset threshold. For example, the signal generation unit 215 compares the pixel's output signal with a reference signal, detects an active pixel that outputs the output signal when the output signal is greater than or less than the reference signal, and generates an event signal corresponding to that active pixel.

[0284] For example, signal generation unit 215 may be configured to include column selection circuitry for arbitrating signals input to signal generation unit 215. Furthermore, signal generation unit 215 may be configured to output not only information about active pixels where events have been detected, but also information about non-active pixels where events have not been detected.

[0285] The signal generation unit 215 outputs the address information and timestamp information (e.g., (X,Y,T)) of the active pixels that have detected the event via the output line 218. However, the data output from the signal generation unit 215 can be not only address information and timestamp information, but also frame format information (e.g., (0,0,1,0,...)).

[0286] [Example of constructing a column processing unit]

[0287] exist Figure 11The present invention has shown an example in which an analog-to-digital converter (ADC) including a comparator 21, a counter 23, and a memory 24 is configured in the column processing unit 4 in a one-to-one correspondence with the pixel columns of the pixel array unit 2; however, the present invention is not limited to this example. For example, a configuration in which the ADC is configured in units of multiple pixel columns and used in a time-division manner among multiple pixel columns may also be adopted.

[0288] The analog-to-digital converter (ADC) converts the analog pixel signal SIG supplied via the vertical signal line VSL into a digital signal with more bits than the address event detection signal described above. For example, when the address event detection signal is 2 bits, the pixel signal is converted into a digital signal with more than 3 bits (e.g., 16 bits). The ADC supplies the digital signal generated by the analog-to-digital conversion to the signal processing unit 212.

[0289] [Noise Incident]

[0290] Incidentally, the camera device 1 according to the first construction example is an asynchronous camera device 1, wherein a detection unit (i.e., an address event detection circuit 300) can be provided in at least some pixel circuits 2a, which detects in real time that the pixel light amount exceeds a predetermined threshold as an address event for each pixel address.

[0291] In the asynchronous camera device 1 according to the first construction example, initially, when some event (i.e., a real event) occurs in the scene, data acquisition caused by the occurrence of the real event is performed. However, in the asynchronous camera device 1, even in a scene where no real event occurs, data acquisition may be unnecessarily performed due to noise events (false events) such as sensor noise. Therefore, not only are noise signals read out, but the throughput of the signal output is also reduced.

[0292] <Examples of the application of the technology according to the present invention>

[0293] The technology according to the present invention can be applied to a variety of products. More specific application examples will be described below. For example, the technology according to the present invention can be implemented as a distance measuring device installed on any type of mobile body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobile devices, airplanes, drones, ships, robots, construction machinery, and agricultural machinery (tractors).

[0294] [Moving Object]

[0295] Figure 33This is a block diagram illustrating a schematic construction example of a vehicle control system 7000, which is an example of a mobile body control system to which the technology according to the present invention can be applied. The vehicle control system 7000 includes multiple electronic control units connected via a communication network 7010. Figure 33 In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting the multiple control units can be an in-vehicle communication network conforming to any standard such as Controller Area Network (CAN), Local Interconnect Network (LIN), Local Area Network (LAN), or FlexRay (registered trademark).

[0296] Each control unit includes a microcomputer that performs calculations according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used for various calculations, and drive circuits that drive various devices that serve as control targets. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside or outside the vehicle via wired or wireless communication. Figure 33 The functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a position measurement unit 7640, a beacon receiving unit 7650, an in-vehicle equipment I / F 7660, an audio / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690. Other control units also include the microcomputer, communication I / F, and storage unit.

[0297] The drive system control unit 7100 controls the operation of devices related to the vehicle's drive system according to various programs. For example, the drive system control unit 7100 functions as a control device for devices such as an internal combustion engine or drive motor that generate driving force for the vehicle, a driving force transmission mechanism that transmits driving force to the wheels, a steering mechanism that adjusts the vehicle's steering angle, and a braking device that generates braking force for the vehicle. The drive system control unit 7100 may also have the function of a control device for anti-lock braking systems (ABS) or electronic stability control (ESC).

[0298] The vehicle condition detection unit 7110 is connected to the drive system control unit 7100. For example, the vehicle condition detection unit 7110 includes at least one of the following: a gyroscope sensor that detects the angular velocity of the vehicle's axial rotational motion; an acceleration sensor that detects the vehicle's acceleration; and sensors that detect the amount of accelerator pedal operation, brake pedal operation, steering wheel angle, engine speed, and wheel speed. The drive system control unit 7100 uses signals input from the vehicle condition detection unit 7110 to perform computational processing to control the internal combustion engine, drive motor, electric power steering system, and braking system, etc.

[0299] The body system control unit 7200 controls the operation of various devices installed on the vehicle body according to various programs. For example, the body system control unit 7200 is used as a control device for keyless entry systems, smart key systems, power windows, or various lights such as headlights, taillights, brake lights, turn signals, or fog lights. In this case, radio waves or signals from various switches transmitted from a portable device that replaces the key can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signal inputs and controls the vehicle's door locking devices, power windows, and lights, etc.

[0300] The battery control unit 7300 controls the secondary battery 7310, which serves as a power source for the drive motor, according to various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity from the battery device including the secondary battery 7310 is input to the battery control unit 7300. The battery control unit 7300 uses these signals to perform calculations and processes, and performs temperature adjustment control of the secondary battery 7310 or control of cooling devices included in the battery device.

[0301] The exterior information detection unit 7400 detects information about the exterior of the vehicle on which the vehicle control system 7000 is mounted. For example, at least one of the camera unit 7410 and the exterior information detection unit 7420 is connected to the exterior information detection unit 7400. The camera unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. For example, the exterior information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather and a surrounding information detection sensor for detecting other vehicles, obstacles, or pedestrians around the vehicle on which the vehicle control system 7000 is mounted.

[0302] Environmental sensors may be, for example, at least one of a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunlight sensor for detecting sunlight intensity, and a snow sensor for detecting snowfall. Surrounding information detection sensors may be at least one of an ultrasonic sensor, a radar device, and a light detection and ranging or laser imaging detection and ranging (LIDAR) device. The camera unit 7410 and the exterior information detection unit 7420 may be configured as independent sensors or devices, or as a device integrating multiple sensors or devices.

[0303] here, Figure 34 An example of the mounting positions of the camera unit 7410 and the vehicle exterior information detection unit 7420 is shown. Camera units 7910, 7912, 7914, 7916, and 7918 are, for example, installed at at least one of the following locations on the front nose, side mirrors, rear bumper, rear door, and the upper part of the interior windshield of the vehicle 7900. The camera unit 7910 installed at the front nose and the camera unit 7918 installed on the upper part of the interior windshield primarily acquire images of the front of the vehicle 7900. The camera units 7912 and 7914 installed at the side mirrors primarily acquire images of the sides of the vehicle 7900. The camera unit 7916 installed on the rear bumper or rear door primarily acquires images of the rear of the vehicle 7900. The camera unit 7918 installed on the upper part of the interior windshield is mainly used to detect vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes ahead.

[0304] also, Figure 34 Examples of the shooting ranges of each camera unit 7910, 7912, 7914, and 7916 are shown. Camera range a represents the shooting range of camera unit 7910 located at the front nose; camera ranges b and c represent the shooting ranges of camera units 7914 and 7912 located on the side mirrors; and camera range d represents the shooting range of camera unit 7916 located on the rear bumper or rear door. For example, a bird's-eye view image of vehicle 7900 can be obtained by overlaying the image data captured by camera units 7910, 7912, 7914, and 7916.

[0305] The exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930, located at the front, rear, sides, corners, and above the windshield inside the vehicle 7900, can be, for example, ultrasonic sensors or radar devices. For instance, the exterior information detection units 7920, 7926, and 7930 located at the front nose, rear bumper, rear door, and above the windshield inside the vehicle 7900 can be LIDAR devices. These exterior information detection units 7920 to 7930 are primarily used for detecting vehicles, pedestrians, or obstacles ahead.

[0306] Return to Figure 33 Continuing the explanation, the exterior information detection unit 7400 enables the camera unit 7410 to capture images of the exterior of the vehicle and receives the captured image data. Furthermore, the exterior information detection unit 7400 receives detection information from the connected exterior information detection section 7420. When the exterior information detection section 7420 is an ultrasonic sensor, radar device, or LIDAR device, the exterior information detection unit 7400 transmits ultrasonic waves or electromagnetic waves, and receives the received reflected wave information. Based on the received information, the exterior information detection unit 7400 can perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, or text on the road surface. Based on the received information, the exterior information detection unit 7400 can perform environmental recognition processing for identifying rain, fog, and road conditions. Based on the received information, the exterior information detection unit 7400 can calculate the distance to objects outside the vehicle.

[0307] Furthermore, the exterior information detection unit 7400 can perform image recognition processing or distance detection processing based on the received image data to identify people, vehicles, obstacles, signs, or text on the road surface. The exterior information detection unit 7400 can perform processing such as distortion correction or alignment on the received image data and synthesize image data captured by different camera units 7410 to generate a bird's-eye view or panoramic image. The exterior information detection unit 7400 can use image data captured by different camera units 7410 to perform viewpoint switching processing.

[0308] The in-vehicle information detection unit 7500 detects in-vehicle information. For example, a driver state detection unit 7510, which detects the driver's state, is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures images of the driver, a biosensor that detects the driver's biological information, and a microphone that collects sound within the vehicle. For example, the biosensor is disposed on the seat surface or steering wheel and detects the biological information of passengers sitting in the seat or drivers holding the steering wheel. The in-vehicle information detection unit 7500 can calculate the driver's fatigue level or concentration level based on the detection information input from the driver state detection unit 7510, and can determine whether the driver is asleep. The in-vehicle information detection unit 7500 can perform processing such as noise cancellation on the collected audio signals.

[0309] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is implemented by a device that can be operated by a passenger for input, such as a touch panel, button, microphone, switch, or joystick. Data obtained by recognizing voice input through the microphone can be input to the integrated control unit 7600. The input unit 7800 can be, for example, a remote control device using infrared or other radio waves, or an external connection device corresponding to the operation of the vehicle control system 7000, such as a mobile phone or personal digital assistant (PDA). The input unit 7800 can be, for example, a camera, in which case the passenger can input information via gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger can be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger or others using the input unit 7800, and outputs the input signal to the integrated control unit 7600. Passengers can input various types of data into the vehicle control system 7000, or instruct processing operations through the operation input unit 7800.

[0310] The storage unit 7690 may include a read-only memory (ROM) for storing various programs executed by a microcomputer and a random access memory (RAM) for storing various parameters, calculation results, and sensor values. Furthermore, the storage unit 7690 may be implemented using a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a photomagnetic-optical storage device.

[0311] The Universal Communication I / F 7620 is a universal communication I / F that mediates communication with various devices in the external environment 7750. In the Universal Communication I / F 7620, cellular communication protocols such as GSM (Gateway Mobile Communications System), WiMAX, LTE, or LTE-Advanced (LTE-A), or other wireless communication protocols such as Wireless LAN (also known as Wi-Fi) or Bluetooth, can be implemented. For example, the Universal Communication I / F 7620 can connect to devices (e.g., application servers or control servers) existing on external networks (e.g., the Internet, cloud networks, or dedicated business networks) via a base station or access point. Furthermore, using, for example, peer-to-peer (P2P) technology, the Universal Communication I / F 7620 can connect to terminals existing near the vehicle (e.g., terminals of drivers, pedestrians, or shops, or machine-type communication (MTC) terminals).

[0312] The Dedicated Communication I / F 7630 is a communication I / F that supports communication protocols designed for vehicles. For example, the Dedicated Communication I / F 7630 can implement wireless access (WAVE), dedicated short-range communication (DSRC), or standard protocols such as cellular communication protocols in a vehicle environment, combining the lower layer IEEE 802.11p and the upper layer IEEE 1609. The Dedicated Communication I / F 7630 typically performs V2X communication, which includes one or more of the following concepts: vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0313] For example, the position measurement unit 7640 receives GNSS signals from Global Navigation Satellite System (GNSS) satellites (e.g., GPS signals from Global Positioning System (GPS) satellites), performs position measurements, and generates position information including the vehicle's latitude, longitude, and altitude. The position measurement unit 7640 can specify its current location by exchanging signals with a wireless access point, or it can obtain position information from a terminal with position measurement capabilities, such as a mobile phone, PHS device, or smartphone.

[0314] For example, beacon receiving unit 7650 receives radio waves or electromagnetic waves transmitted from, for example, a radio station installed on a road, and acquires information such as current location, traffic congestion, road closure, or estimated time. The functionality of beacon receiving unit 7650 can be included in the aforementioned dedicated communication I / F 7630.

[0315] The in-vehicle device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 can establish a wireless connection using wireless communication protocols such as Wireless LAN, Bluetooth (registered trademark), Near Field Communication (NFC), or Wireless USB (WUSB). Furthermore, the in-vehicle device I / F 7660 can establish a wired connection such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI (registered trademark)), or Mobile High Definition Link (MHL) via connection terminals not shown (and, if necessary, cables). For example, the in-vehicle devices 7760 may include at least one of a passenger's mobile or wearable device and an information device carried in or attached to the vehicle. Additionally, the in-vehicle device 7760 may include a navigation device that performs route searches to any destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0316] The vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The vehicle network I / F 7680 sends or receives signals according to predetermined protocols supported by the communication network 7010.

[0317] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the following: general communication I / F 7620, dedicated communication I / F 7630, position measurement unit 7640, beacon receiving unit 7650, in-vehicle equipment I / F 7660, and in-vehicle network I / F 7680. For example, the microcomputer 7610 calculates control target values ​​for the drive force generating device, steering mechanism, or braking device based on the acquired in-vehicle and out-of-vehicle information, and outputs control commands to the drive system control unit 7100. For example, the microcomputer 7610 performs cooperative control aimed at realizing functions of advanced driver assistance systems (ADAS), including collision avoidance or impact mitigation, distance-based following, speed maintenance, collision warning, and lane departure warning. Furthermore, by controlling the drive force generating device, steering mechanism, or braking device based on the acquired information about the vehicle's surroundings, the microcomputer 7610 can perform coordinated control aimed at, for example, autonomous driving, which enables the vehicle to drive autonomously without relying on the driver's operation.

[0318] Based on information acquired via at least one of the following: general communication I / F 7620, dedicated communication I / F 7630, position measurement unit 7640, beacon receiving unit 7650, in-vehicle equipment I / F 7660, and vehicular network I / F 7680, the microcomputer 7610 can generate three-dimensional distance information between the vehicle and objects such as surrounding structures or people, and create local map information including surrounding information of the vehicle's current position. Furthermore, based on the acquired information, the microcomputer 7610 can predict hazards such as vehicle collisions, pedestrians approaching or entering closed roads, and generate warning signals. For example, the warning signal could be a signal used to generate a warning sound or activate warning lights.

[0319] The audio-visual output unit 7670 sends an output signal of at least one of audio and visual signals to an output device capable of visually or audibly notifying passengers or the outside of the vehicle of information. Figure 33In the example, the audio speaker 7710, display unit 7720, and dashboard 7730 are illustrated as output devices. For example, display unit 7720 may include at least one of an in-vehicle display and a head-up display. Display unit 7720 may have augmented reality (AR) display functionality. Output devices may be other than these devices, such as headphones, wearable devices (such as glasses-type displays worn by passengers), projectors, or lights. When the output device is a display device, the display device visually displays, in various formats such as text, images, tables, and charts, the results obtained through various processes performed by microcomputer 7610 or information received from other control units. When the output device is an audio output device, the audio output device converts audio signals, including regenerated audio data or acoustic data, into analog signals and audibly outputs the audio signals.

[0320] exist Figure 33 In the example shown, at least two control units connected via communication network 7010 can be integrated into a single control unit. Alternatively, each control unit can consist of multiple control units. Furthermore, the vehicle control system 7000 may include other control units (not shown). Moreover, as described above, these other control units may possess some or all of the functions of any one of the control units. That is, any control unit can perform predetermined computational processing as long as it sends and receives information via communication network 7010. Similarly, sensors or devices connected to any one control unit can be connected to other control units, and multiple control units can send or receive detection information to each other via communication network 7010.

[0321] Examples of vehicle control systems to which the technology according to the present invention can be applied have been described above. For example, the technology according to the present invention can be applied to camera units 7910, 7912, 7914, 7916 and 7918, external information detection units 7920, 7922, 7924, 7926, 7928 and 7930, or driver state detection unit 7510, etc., in the above-described configuration. Specifically, a camera system having the camera device 1 of the present invention can be applied to a camera unit or a detection unit. Since the effects of noise events such as sensor noise can be mitigated by applying the technology according to the present invention, and the occurrence of real events can be reliably and quickly sensed, safe vehicle operation can be achieved.

[0322] This technology can have the following structure.

[0323] (1) A camera device comprising a plurality of stacked substrates, the camera device comprising:

[0324] A dedicated readout circuit is disposed on a different substrate from the substrate on which pixel array units including multiple photoelectric conversion elements are disposed, and is configured to perform a readout operation of the electrical signal obtained by photoelectric conversion through the multiple photoelectric conversion elements; and

[0325] The circuit is disposed on a different substrate from the substrate on which the readout dedicated circuit is disposed, and is configured to perform operations other than the operation of the readout dedicated circuit based on the electrical signal.

[0326] (2) The camera device according to (1), wherein the readout circuit is configured to convert the electrical signal obtained by photoelectric conversion through the photoelectric conversion element into a voltage signal and perform gain adjustment, and

[0327] The substrate on which the circuit is configured to perform operations other than those of the readout dedicated circuit performs at least one of the following processes: processing to convert a voltage signal output from the readout dedicated circuit into a digital signal in units of two or more pixel groups arranged in a first direction; predetermined signal processing for the digital signal; and processing to drive the plurality of photoelectric conversion elements in units of two or more pixel groups arranged in a second direction.

[0328] (3) The camera device according to (1) or (2), wherein, in the circuit configured to perform operations other than the operation of the readout dedicated circuit, a circuit portion having a power supply voltage exceeding a predetermined reference voltage is disposed on the same substrate as the plurality of photoelectric conversion elements.

[0329] (4) The camera device according to (3) includes at least a portion of an AD unit, the at least a portion of which is disposed on a substrate on which the pixel array unit is disposed, and is configured to convert pixel signals read from the readout dedicated circuit into digital signals.

[0330] (5) The camera device according to (4), wherein the AD unit converts the pixel signal read from the readout circuit into a digital signal in units of two or more pixel groups arranged in the pixel array unit in the first direction.

[0331] (6) The camera device according to (4) or (5), wherein the AD unit is divided and disposed on a substrate and other substrates on which the pixel array unit is disposed.

[0332] (7) The camera device according to any one of (4) to (6) includes a pixel group driving unit disposed on a substrate on which the pixel array unit is disposed, and is configured to drive the pixel array unit in units of two or more pixel groups disposed in a second direction.

[0333] (8) The camera device according to (7), wherein the pixel group driving unit is divided and disposed on a substrate and other substrates on which the pixel array unit is disposed.

[0334] (9) The camera device according to any one of (1) to (8), comprising:

[0335] The first substrate has the readout dedicated circuit disposed thereon;

[0336] A second substrate, stacked on the first substrate, and wherein the pixel array units are disposed; and

[0337] A third substrate is stacked with the same layer height as the second substrate on the first substrate, and at least a portion of the circuitry configured to perform operations other than those of the readout dedicated circuitry is disposed thereon.

[0338] (10) The imaging device according to (9), wherein the first substrate is larger than the second substrate, and

[0339] The second substrate is larger than the third substrate.

[0340] (11) The camera device according to any one of (1) to (8), comprising:

[0341] The first substrate has the readout dedicated circuit disposed thereon;

[0342] A second substrate, stacked on the first substrate, and wherein the pixel array units are disposed; and

[0343] A third substrate is stacked below the first substrate and is disposed thereon at least a portion of the circuit configured to perform operations other than those of the readout dedicated circuit.

[0344] (12) The camera device according to (11) includes:

[0345] A fourth substrate, which is configured with the same layer height as the third substrate,

[0346] Parts of the circuit configured to perform operations other than those of the readout dedicated circuit are respectively disposed on the third substrate and the fourth substrate.

[0347] (13) The camera device according to (12), wherein the second substrate is larger than the third substrate and the fourth substrate.

[0348] (14) The camera device according to any one of (9) to (13), wherein the readout dedicated circuit is disposed on the first substrate to at least partially overlap with the pixel array unit when viewing a plan view of the first substrate and the second substrate in the stacking direction.

[0349] (15) The camera device according to any one of (9) to (14), wherein a readout dedicated circuit is provided for each of the plurality of photoelectric conversion elements.

[0350] (16) The camera device according to any one of (9) to (14), wherein the readout dedicated circuit is configured to be associated with two or more of the photoelectric conversion elements.

[0351] (17) The camera device according to any one of (1) to (16), wherein the pixel array unit and the readout circuitry include a change detection unit configured to output a detection signal indicating whether the change in the electrical signal of each of the plurality of photoelectric conversion elements exceeds a predetermined threshold.

[0352] (18) The camera device according to any one of (1) to (16), wherein the pixel array unit and the readout dedicated circuit include a pixel AD unit configured to convert an electrical signal obtained by photoelectric conversion through each of the photoelectric conversion elements into a digital signal.

[0353] (19) The camera device according to any one of (1) to (16), wherein the pixel array unit and the readout dedicated circuit include a light detection unit configured to detect the incident position and incident time of light incident on the plurality of photoelectric conversion elements.

[0354] (20) The camera device according to any one of (1) to (19), comprising:

[0355] The first output unit is configured to output a first signal from the pixel array unit; and

[0356] The second output unit is configured to output a second signal from the readout dedicated circuit.

[0357] (21) The imaging device according to any one of (1) to (20), wherein the plurality of substrates are joined by at least one of Cu-Cu bonding, through silicon via (TSV) and bump bonding.

[0358] (22) The camera device according to any one of (1) to (21), wherein the plurality of substrates are wafers or semiconductor chips.

[0359] (23) A camera method using a plurality of stacked substrates, the camera method comprising the following steps:

[0360] A readout operation is performed in a dedicated readout circuit to read out electrical signals obtained through photoelectric conversion by multiple photoelectric conversion elements. This dedicated readout circuit is disposed on a different substrate from the substrate on which pixel array units including the multiple photoelectric conversion elements are disposed.

[0361] In a circuit disposed on a substrate different from the substrate on which the readout dedicated circuit is disposed, operations other than the operation of the readout dedicated circuit are performed based on the electrical signal.

[0362] The present invention is not limited to the various embodiments described above, and includes various modifications that can be conceived by those skilled in the art, and the effects of the present invention are not limited to the above description. That is, various additions, changes, and partial deletions can be made without departing from the concept and spirit of the present invention as defined in the claims and their equivalents.

[0363] List of reference numerals

[0364] 1. Camera device

[0365] 2-pixel array unit

[0366] 3-line drive unit

[0367] 4-column processing unit

[0368] 5-pixel AFE unit

[0369] 6-column drive unit

[0370] 7. Signal Processing Unit

[0371] 8 System Control Unit

[0372] 9. Signal Processing Unit

[0373] 11 First substrate

[0374] 12 Second substrate

[0375] 21 Comparator

[0376] 23 Up / Down Counter

[0377] 24 Memory

[0378] 25 Address Selection Logic Unit

[0379] 26 Decoders

[0380] 27. Memory

[0381] 28 Level shifter

[0382] 29 drives

[0383] 300 Address Event Detection Circuit

[0384] 31 Photoelectric conversion element

[0385] 32 Transmission Transistors

[0386] 33 Reset transistor

[0387] 36 Amplifying Transistors

[0388] 37 Select Transistor

[0389] 210 Camera device

[0390] 211 pixel array unit

[0391] 212 Signal Processing Unit

[0392] 213 Drive Unit

[0393] 214 Readout Region Selection Unit

[0394] 215 Signal Generation Unit

[0395] 310 Current-to-Voltage Conversion Circuit

[0396] 320 buffer

[0397] 330 Subtractor

[0398] 340 quantizer

[0399] 350 Transmission Circuit

Claims

1. A camera device comprising a plurality of stacked substrates, the camera device comprising: A pixel analog front end, which is configured on the first substrate; A pixel array unit, comprising a plurality of photoelectric conversion elements, is disposed on a second substrate different from the first substrate, wherein the pixel analog front end is configured to perform a readout operation of an electrical signal obtained by photoelectric conversion through the plurality of photoelectric conversion elements; and The signal processing unit is configured to perform operations other than those of the pixel analog front end based on the electrical signal. In this configuration, the first substrate is driven by a first voltage, the second substrate is driven by a second voltage, and the first voltage is lower than the second voltage. In the signal processing unit, a circuit portion having a power supply voltage exceeding a predetermined reference voltage is disposed on the second substrate, identical to the plurality of photoelectric conversion elements, while the remaining circuit portion of the signal processing unit is disposed on the first substrate. The pixel array unit and the pixel analog front end are configured to overlap in the stacking direction.

2. The camera device according to claim 1, wherein, The pixel analog front end is a circuit configured to convert the electrical signal obtained by photoelectric conversion through the photoelectric conversion element into a voltage signal and perform gain adjustment. The first substrate performs at least one of the following processes: converting a voltage signal output from the analog front end of the pixel array unit into a digital signal in units of two or more pixel groups arranged in a first direction; performing predetermined signal processing on the digital signal; and driving the plurality of photoelectric conversion elements in units of two or more pixel groups arranged in a second direction.

3. The camera device according to claim 1, further comprising at least a portion of an AD unit, the at least a portion of which is disposed on a second substrate on which the pixel array unit is disposed, and configured to convert pixel signals read from the pixel analog front end into digital signals.

4. The camera device according to claim 3, wherein, The AD unit converts the pixel signal read from the pixel analog front end into a digital signal in units of two or more pixel groups configured in the first direction.

5. The camera device according to claim 3, wherein, The AD unit is segmented and disposed on the second substrate and other substrates on which the pixel array unit is disposed.

6. The camera device according to claim 3 further includes a pixel group driving unit, the pixel group driving unit being disposed on the second substrate on which the pixel array unit is disposed, and configured to drive the pixel array unit in units of two or more pixel groups disposed in the second direction.

7. The camera device according to claim 6, wherein, The pixel group driving unit is divided and disposed on the second substrate and other substrates on which the pixel array unit is disposed.

8. The camera device according to claim 1, wherein, The second substrate is stacked on the first substrate.

9. The camera device according to claim 8, wherein, The first substrate is larger than the second substrate.

10. The camera device according to claim 8, wherein, A pixel analog front end is provided for each of the plurality of photoelectric conversion elements.

11. The camera device according to claim 8, wherein, The pixel analog front end is configured to be associated with two or more of the aforementioned photoelectric conversion elements.

12. The camera device according to claim 1, wherein, The pixel array unit and the pixel analog front end include a change detection unit, which is configured to output a detection signal indicating whether the change in the electrical signal of each of the plurality of photoelectric conversion elements exceeds a predetermined threshold.

13. The camera device according to claim 1, wherein, The pixel array unit and the pixel analog front end include a pixel AD unit, which is configured to convert the electrical signal obtained by photoelectric conversion through each of the photoelectric conversion elements into a digital signal.

14. The camera device according to claim 1, wherein, The pixel array unit and the pixel analog front end include a light detection unit configured to detect the incident position and incident time of light incident on the plurality of photoelectric conversion elements.

15. The camera device according to claim 1, further comprising: The first output unit is configured to output a first signal from the pixel array unit; and The second output unit is configured to output a second signal from the pixel analog front end.

16. The camera device according to claim 1, wherein, The plurality of substrates are joined by at least one of Cu-Cu bonding, through-silicon vias, and bump bonding.

17. The camera device according to any one of claims 1 to 16, wherein, The multiple substrates are wafers or semiconductor chips.

18. A method for imaging using a plurality of stacked substrates, the method comprising the following steps: An electrical signal is obtained by photoelectric conversion through multiple photoelectric conversion elements included in the pixel array unit, wherein the multiple photoelectric conversion elements are disposed on the second substrate; The readout operation of the electrical signal is performed in a pixel analog front-end, the pixel analog front-end being disposed on a first substrate different from the second substrate; and In the signal processing unit, operations other than those performed by the pixel analog front end are executed based on the electrical signal. In this configuration, the first substrate is driven by a first voltage, the second substrate is driven by a second voltage, and the first voltage is lower than the second voltage. In the signal processing unit, a circuit portion having a power supply voltage exceeding a predetermined reference voltage is disposed on the second substrate, identical to the plurality of photoelectric conversion elements, while the remaining circuit portion of the signal processing unit is disposed on the first substrate. The pixel array unit and the pixel analog front end are configured to overlap in the stacking direction.

Images