camera device

Abstract

The present disclosure provides an imaging device capable of outputting a signal other than an event detection signal (for example, a pixel signal of a gray scale). The imaging device of the present disclosure has a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip. A pixel array unit in which an event pixel that detects an occurrence of an event in which a luminance change amount of a pixel exceeds a predetermined threshold as an event and outputs an event detection signal and a ranging pixel that includes a light receiving element that receives reflected light from a ranging target based on irradiation light from a light source unit and generates a signal in response to reception of a photon are mixed is provided in the first layer semiconductor chip. An analog front end unit for the event pixel that processes the event detection signal and an analog front end unit for the ranging pixel that processes the signal from the light receiving element are provided in the second layer semiconductor chip corresponding to the event pixel and the ranging pixel, respectively.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an imaging device. BACKGROUND

[0002] In contrast to a scanning type (synchronous type) imaging device that performs imaging in synchronization with a synchronization signal such as a vertical synchronization signal, there is a non-scanning type (asynchronous type) imaging device called a DVS (dynamic vision sensor) (for example, refer to Patent Literature 1). The non-scanning type imaging device detects an event occurrence in which a change amount in luminance of a pixel in which incident light is photoelectrically converted exceeds a predetermined threshold value, and outputs an event detection signal.

[0003] LIST OF CITATIONS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: WO 2019 / 087471 A1 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] It is desired that the non-scanning type (asynchronous type) imaging device called a DVS not only be able to detect a movement (moving object) of a subject by detecting an event occurrence, but also be able to output a signal other than an event detection signal, such as a pixel signal of a gradation corresponding to an amount of incident light.

[0008] An object of the present disclosure is to provide an imaging device that is able to output a signal other than an event detection signal, such as a pixel signal of a gradation corresponding to an amount of incident light, in addition to being able to output an event detection signal indicating an event occurrence.

[0009] TECHNICAL SOLUTION TO THE PROBLEM

[0010] The imaging device of the present disclosure for achieving the above object includes a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip,

[0011] wherein a pixel array unit in which an event pixel that detects an event occurrence in which a change amount in luminance of a pixel exceeds a predetermined threshold value and outputs an event detection signal and a gradation pixel that outputs a pixel signal of a gradation corresponding to an amount of incident light are mixed is provided in the first layer semiconductor chip, and

[0012] In the second layer semiconductor chip, an analog front-end unit for event pixels that processes the event detection signal and an analog front-end unit for grayscale pixels that processes the pixel signal are provided corresponding to the event pixels and the grayscale pixels, respectively.

[0013] The imaging device of the present disclosure for achieving the above object includes a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip,

[0014] wherein a pixel array unit in which event pixels and range-finding pixels are mixed is provided in the first layer semiconductor chip, the event pixels detect an occurrence of an event in which a luminance change amount of a pixel exceeds a predetermined threshold value and output an event detection signal, the range-finding pixels include a light-receiving element that receives reflected light from a range-finding target based on irradiation light from a light source unit and generates a signal in response to reception of a photon, and

[0015] In the second layer semiconductor chip, an analog front-end unit for event pixels that processes the event detection signal and an analog front-end unit for range-finding pixels that processes the signal from the light-receiving element are provided corresponding to the event pixels and the range-finding pixels, respectively. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a block diagram schematically showing a system configuration of an imaging device according to a first embodiment of the present disclosure.

[0017] Figure 2 A of is a block diagram showing an example of a configuration of a grayscale circuit, Figure 2 B of is a block diagram showing an example of a configuration of a DVS unit.

[0018] Figure 3 is a circuit diagram showing a specific example of a configuration of a grayscale pixel.

[0019] Figure 4 is a circuit diagram showing a specific example of a configuration of a DVS pixel.

[0020] Figure 5 is a block diagram showing an example of a configuration of an address event detection unit.

[0021] Figure 6 is a schematic view showing a stacked chip structure in a first specific example in the imaging device according to the first embodiment of the present disclosure.

[0022] Figure 7 A of is a graph showing a correspondence relationship 1 between grayscale pixels and DVS pixels of the first layer semiconductor chip and each analog front-end unit of the second layer semiconductor chip,Figure 7 Figure B is a diagram showing the correspondence between the grayscale pixels and DVS pixels of the first-layer semiconductor chip and the analog front-end units of the second-layer semiconductor chip.

[0023] Figure 8 It is an image diagram showing a pixel array of grayscale pixels and DVS pixels.

[0024] Figure 9 This is a schematic diagram illustrating a stacked chip structure of a second specific example in a camera device according to a first embodiment of the present disclosure.

[0025] Figure 10 Figure A is a diagram illustrating the pixel arrangement of a first configuration example in a camera device according to a first embodiment of the present disclosure. Figure 10 Figure B is a diagram illustrating the pixel arrangement of a second construction example in a camera device according to a first embodiment of the present disclosure.

[0026] Figure 11 Figure A is a diagram illustrating the pixel arrangement of a third configuration example in a camera device according to a first embodiment of the present disclosure. Figure 11 Figure B is a diagram illustrating the pixel arrangement of a fourth construction example in a camera device according to a first embodiment of the present disclosure.

[0027] Figure 12 Figure A is a diagram illustrating the arrangement of the DVS unit in a first arrangement example of a camera device according to a first embodiment of the present disclosure, and the connection relationship between the DVS unit and the DVS AFE. Figure 12 Figure B is a diagram showing the arrangement of the DVS unit in a second arrangement example of a camera device according to a first embodiment of the present disclosure, and the connection relationship between the DVS unit and the DVS AFE.

[0028] Figure 13 Figure A is a diagram illustrating the arrangement of the DVS unit in a third arrangement example of a camera device according to a first embodiment of the present disclosure, and the connection relationship between the DVS unit and the DVS AFE. Figure 13 Figure B is a diagram showing the arrangement of the DVS unit in a fourth arrangement example of a camera device according to a first embodiment of the present disclosure, and the connection relationship between the DVS unit and the DVS AFE.

[0029] Figure 14 A is a schematic diagram illustrating a stacked chip structure according to a first construction example. Figure 14 B is a diagram schematically illustrating a stacked chip structure according to the second construction example.

[0030] Figure 15 This is a schematic diagram illustrating a stacked chip structure according to the third construction example.

[0031] Figure 16 is a block diagram schematically showing a system configuration of an imaging device according to a second embodiment of the present disclosure.

[0032] Figure 17 is a circuit diagram showing an example of a configuration of a basic pixel circuit of a range-finding pixel.

[0033] Figure 18 A of FIG. 10 is a schematic view showing a stacked chip structure in a first specific example of an imaging device according to the second embodiment of the present disclosure, Figure 18 B of FIG. 10 is a schematic view showing a stacked chip structure in a second specific example of an imaging device according to the second embodiment of the present disclosure.

[0034] Figure 19 A of FIG. 11 is a view showing a pixel arrangement in a first configuration example of an imaging device according to the second embodiment of the present disclosure, Figure 19 B of FIG. 11 is a view showing a pixel arrangement in a second configuration example of an imaging device according to the second embodiment of the present disclosure.

[0035] Figure 20 A of FIG. 12 is a view showing a pixel arrangement in a third configuration example of an imaging device according to the second embodiment of the present disclosure, Figure 20 B of FIG. 12 is a view showing a pixel arrangement in a fourth configuration example of an imaging device according to the second embodiment of the present disclosure.

[0036] Figure 21 A of FIG. 13 is a view showing an arrangement of DVS units and a connection relationship of the DVS units with a DVS AFE in a first arrangement example of an imaging device according to the second embodiment of the present disclosure, Figure 21 B of FIG. 13 is a view showing an arrangement of DVS units and a connection relationship of the DVS units with a DVS AFE in a second arrangement example of an imaging device according to the second embodiment of the present disclosure.

[0037] Figure 22 A of FIG. 14 is a view showing an arrangement of DVS units and a connection relationship of the DVS units with a DVS AFE in a third arrangement example of an imaging device according to the second embodiment of the present disclosure, Figure 22 B of FIG. 14 is a view showing an arrangement of DVS units and a connection relationship of the DVS units with a DVS AFE in a fourth arrangement example of an imaging device according to the second embodiment of the present disclosure.

[0038] Figure 23 is a block diagram schematically showing a system configuration of an imaging device according to a third embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] A manner for implementing the technology of the present disclosure (hereinafter, referred to as an embodiment) will be described below with reference to the drawings. The technology of the present disclosure is not limited to these embodiments. In the following description, the same elements or elements having the same functions will be denoted by the same reference numerals, and repetitive description will be omitted. Note that the description will be made in the following order.

[0040] 1. General Description of Imaging Device of the Present Disclosure

[0041] 2. Imaging Device (Event Pixels + Gray Scale Pixels) According to First Embodiment

[0042] 2-1. System Configuration Example

[0043] 2-1-1. Configuration Example of Gray Scale Pixels

[0044] 2-1-2. Configuration Example of DVS Pixels

[0045] 2-1-3. Configuration Example of Address Event Detection Unit

[0046] 2-2. Laminated Chip Structure

[0047] 2-2-1. First Specific Example (Example of Two-Layer Structure)

[0048] 2-2-2. Second Specific Example (Example of Three-Layer Structure)

[0049] 2-3. Configuration Example of Pixel Array Unit

[0050] 2-3-1. First Configuration Example (Example in Which Units Including Gray Scale Pixels and DVS Pixels Are Arranged Side by Side According to Predetermined Rule)

[0051] 2-3-2. Second Configuration Example (Example in Which Units Including DVS Pixels Are Arranged Side by Side in Pixel Row Units)

[0052] 2-3-3. Third Configuration Example (Example in Which Units Including DVS Pixels Are Arranged Side by Side in Pixel Column Units)

[0053] 2-3-4. Fourth Configuration Example (Example in Which DVS Units Are Irregularly Arranged Side by Side)

[0054] 2-4. Arrangement Example of DVS Units

[0055] 2-4-1. First Arrangement Example (Example in Which DVS Unit Includes Four DVS Pixels Arranged Side by Side in a Square)

[0056] 2-4-2. Second Arrangement Example (Example in Which DVS Unit Includes Four DVS Pixels Arranged Side by Side in Row Direction)

[0057] 2-4-3. Third arrangement example (example in which a DVS unit includes four DVS pixels arranged side by side in a column direction)

[0058] 2-4-4. Fourth arrangement example (example in which a size of a DVS unit is larger than a size of a gray-scale pixel)

[0059] 2-5. Configuration example of stacked chip structure in case of gray-scale pixel

[0060] 2-5-1. First configuration example of stacked chip structure (configuration example of global shutter function)

[0061] 2-5-2. Second configuration example of stacked chip structure (configuration example of read function)

[0062] 2-5-3. Third configuration example of stacked chip structure (configuration example of area AD)

[0063] 2-6. Effects and advantages of first embodiment

[0064] 3. Imaging device according to second embodiment (event pixel + ranging pixel)

[0065] 3-1. System configuration example

[0066] 3-2. Stacked chip structure

[0067] 3-2-1. First specific example (example in which a SPAD element is mounted on an upper chip and a quenching circuit is mounted on a lower chip)

[0068] 3-2-2. Second specific example (example in which a plurality of SPAD elements on an upper chip are regarded as a unit and a quenching circuit and a time measurement unit are mounted on a lower chip)

[0069] 3-3. Configuration example of pixel array unit

[0070] 3-3-1. First configuration example (example in which units including a DVS pixel and a ranging pixel are arranged in a matrix shape)

[0071] 3-3-2. Second configuration example (example in which units including a DVS pixel are arranged side by side in a pixel row unit)

[0072] 3-3-3. Third configuration example (example in which units including a DVS pixel are arranged side by side in a pixel column unit)

[0073] 3-3-4. Fourth configuration example (example in which DVS units are arranged randomly)

[0074] 3-4. Effects and advantages of second embodiment

[0075] 4. The imaging device (event pixel + gray scale pixel + distance measuring pixel) according to the third embodiment

[0076] 5. Modified example

[0077] 6. Application example

[0078] 7. Configuration that the present disclosure can have

[0079] [General description of the imaging device of the present disclosure]

[0080] In the imaging device of the present disclosure, when a predetermined number of event pixels is defined as an event pixel unit, the event pixel unit can be arranged side by side in the pixel array unit according to a predetermined rule.

[0081] In the imaging device of the present disclosure including the above-described preferred configuration, the event pixel unit can be combined as a pixel unit with a predetermined number of gray scale pixels, and can be arranged in the pixel array unit. Further, in the matrix-like pixel array of the pixel array unit, the event pixel unit can be arranged side by side in units of a pixel row or in units of a pixel column.

[0082] Further, in the imaging device of the present disclosure including the above-described preferred configuration, when a predetermined number of event pixels is defined as an event pixel unit, the event pixel unit can be irregularly arranged side by side in the pixel array unit.

[0083] Further, in the imaging device of the present disclosure including the above-described preferred configuration, the event pixels can be arranged side by side in the event pixel unit according to a predetermined rule. Specifically, in the event pixel unit, the event pixels can be arranged side by side in a square shape, or arranged side by side in a row direction or a column direction of the matrix-like pixel array.

[0084] Further, in the imaging device of the present disclosure including the above-described preferred configuration, the size of the event pixels can be larger than the size of the gray scale pixels. Further, the number of the gray scale pixels can be more than the number of the event pixels.

[0085] Further, in the imaging device of the present disclosure including the above-described preferred configuration, in addition to the event pixels and the gray scale pixels, a distance measuring pixel that includes a light receiving element that receives reflected light from a distance measuring target based on irradiation light from a light source unit and generates a signal in response to reception of a photon can be mixed in the pixel array unit. Further, the second layer semiconductor chip can be provided with a quenching circuit that controls the light receiving element corresponding to the distance measuring pixel.

[0086] Further, in the imaging device of the present disclosure including the above-described preferred configuration, the light receiving element of the distance measuring pixel can include an avalanche photodiode that operates in a Geiger mode, preferably a single-photon avalanche diode.

[0087] <Camera according to the first embodiment>

[0088] A camera according to a first embodiment of the present disclosure has a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip. Further, an event pixel and a gray scale pixel are mixed in a pixel array unit provided in the first layer semiconductor chip.

[0089] Here, the event pixel is a pixel that detects a phenomenon in which a luminance change amount of the pixel exceeds a predetermined threshold as an event occurrence and outputs an event detection signal. The event pixel is used for a non-scanning (asynchronous) camera called a DVS. Therefore, hereinafter, there is a case where the event pixel is described as a DVS pixel. The gray scale pixel outputs a pixel signal of a gray scale level corresponding to an amount of incident light.

[0090] [System configuration example]

[0091] Figure 1 is a block diagram schematically showing a system configuration of a camera according to the first embodiment of the present disclosure.

[0092] As shown in Figure 1 , the camera 10A according to the first embodiment includes a pixel array unit 11, an access control unit 12, a DVS reading unit 13, a column signal processing unit 14, a DVS signal processing unit 15, a gray scale signal processing unit 16, a timing control unit 17, a time stamp generation unit 18, and output interfaces (I / F) 19A and 19B.

[0093] The pixel array unit 11 is provided with a gray scale circuit 20 and a DVS unit 30 as an event pixel unit in a mixed state. As shown in Figure 2 A of FIG. 1, the gray scale circuit 20 includes a gray scale pixel 21 and a gray scale memory 22 that stores a pixel signal output from the gray scale pixel 21. As shown in Figure 2 B of FIG. 1, the DVS unit 30 includes a predetermined number of DVS pixels 31, for example, four DVS pixels 31 as a unit. In addition to the four DVS pixels 31, the DVS unit 30 includes a selector 32 that selects event detection signals output from the four DVS pixels 31 and a DVS analog front end (AFE) 35 that processes the event detection signals selected by the selector 32.

[0094] When reading a pixel signal of a gray scale level corresponding to an amount of incident light from the gray scale pixel 21, the access control unit 12 outputs various drive signals to each gray scale pixel 21 of the pixel array unit 11. Further, when reading an event detection signal from the DVS pixel 31, the access control unit 12 outputs a row drive signal to each DVS pixel 31 of the pixel array unit 11.

[0095] The DVS reading unit 13 reads an event detection signal from each DVS pixel 31 of the pixel array unit 11 under the drive of the access control unit 12, and supplies the read event detection signal as event data to the DVS signal processing unit 15.

[0096] The column signal processing unit 14 includes, for example, an analog-digital converter provided for each pixel column and equal in number to the pixel columns of the grayscale pixels 21, and under the drive of the access control unit 12, converts an analog pixel signal (grayscale luminance signal) read from each grayscale pixel 21 of the pixel array unit 11 into a digital pixel signal, and supplies the digital pixel signal to the grayscale signal processing unit 16.

[0097] The DVS signal processing unit 15 performs signal processing such as addition of a time stamp as described later on the event detection signal read from each DVS pixel 31 of the pixel array unit 11 by the DVS reading unit 13, and then outputs the event detection signal as event data to the outside of the imaging device 10A via an output interface (I / F) 19A.

[0098] The grayscale signal processing unit 16 performs signal processing such as correlated double sampling (CDS) processing or addition of a time stamp as described later on the digital pixel signal output from the column signal processing unit 14, and then outputs the digital pixel signal as grayscale data to the outside of the imaging device 10A via an output interface (I / F) 19B.

[0099] The timing control unit 17 generates various timing signals, clock signals, and control signals, and performs drive control of the DVS signal processing unit 15, the grayscale signal processing unit 16, and the like based on the generated signals.

[0100] The time stamp generation unit 18 generates a time stamp (time information) indicating the relative time of the output timing of the event data or the pixel data, for the event data output from the DVS reading unit 13 and the pixel data output from the column signal processing unit 14. The generated time stamp is added to the event data in the DVS signal processing unit 15, and to the pixel data in the grayscale signal processing unit 16.

[0101] Next, the specific configuration of the grayscale pixel 21 and the DVS pixel 31 will be described with concrete examples.

[0102] (Configuration Example of Grayscale Pixel)

[0103] Figure 3 is a circuit diagram showing a concrete example of the configuration of the grayscale pixel 21.

[0104] The gradation pixel 21 includes, for example, a photodiode 211 as a photoelectric conversion element. The gradation pixel 21 includes, in addition to the photodiode 211, a transfer transistor 212, a reset transistor 213, an amplification transistor 214, and a selection transistor 215.

[0105] Here, the four transistors of the transfer transistor 212, the reset transistor 213, the amplification transistor 214, and the selection transistor 215 use, for example, N-type MOS field effect transistors (FETs). However, the combination of the conductive types of the four transistors 212 to 215 exemplified here is merely an example, and the combination is not limited thereto.

[0106] For the gradation pixel 21, a plurality of pixel control lines are commonly wired to each gradation pixel 21 on the same pixel row. The plurality of pixel control lines are connected to the transfer transistor 212, the reset transistor 213, the amplification transistor 214, and the selection transistor 215 of each gradation pixel 21 in units of pixel rows. Figure 1 The access control unit 12 has outputs corresponding to each pixel row as illustrated. When reading a pixel signal of a gradation level corresponding to the amount of incident light from the gradation pixel 21, the access control unit 12 appropriately outputs various drive signals, specifically, a transfer signal TRG, a reset signal RST, and a selection signal SEL, to the plurality of pixel control lines.

[0107] The photodiode 211 has an anode connected to a low-potential side power supply (for example, ground), photoelectrically converts received light into a photocharge (here, a photoelectron) of an amount of charge corresponding to the amount of light, and accumulates the photocharge. The cathode of the photodiode 211 is electrically connected to the gate electrode of the amplification transistor 214 via the transfer transistor 212. Here, the region electrically connected to the gate electrode of the amplification transistor 214 is a floating diffusion portion (floating diffusion region / impurity diffusion region) FD. The floating diffusion portion FD is a charge-voltage conversion unit that converts a charge into a voltage.

[0108] The access control unit 12 supplies a high level (for example, V DD ) that becomes effective as the transfer signal TRG to the gate electrode of the transfer transistor 212. The transfer transistor 212 becomes conductive in response to the transfer signal TRG, thereby transferring the photocharge photoelectrically converted by the photodiode 211 and accumulated in the photodiode 211 to the floating diffusion portion FD.

[0109] The reset transistor 213 is connected between the node of the high-potential side power supply voltage V DD and the floating diffusion portion FD. The access control unit 12 supplies a high level that becomes effective as the reset signal RST to the gate electrode of the reset transistor 213. The reset transistor 213 becomes conductive in response to the reset signal RST, and resets the floating diffusion portion FD by discarding the charge of the floating diffusion portion FD to the node of the voltage V DD .

[0110] The amplification transistor 214 has a gate electrode connected to the floating diffusion FD, and has a drain electrode connected to a node of the high-potential-side power supply voltage V DD The amplification transistor 214 functions as a source follower input unit for reading a signal obtained by photoelectric conversion in the photodiode 211. That is, the source electrode of the amplification transistor 214 is connected to the signal line VSL via the selection transistor 215. Further, the amplification transistor 214 and the load current source I connected to one end of the signal line VSL are a source follower configuration in which the voltage of the floating diffusion FD is converted into the potential of the signal line VSL.

[0111] The selection transistor 215 has a drain electrode connected to the source electrode of the amplification transistor 214, and has a source electrode connected to the signal line VSL. The access control unit 12 supplies the gate electrode of the selection transistor 215 with a selection signal SEL which becomes active at a high level. The selection transistor 215 becomes conductive in response to the selection signal SEL, thereby transmitting the signal output from the amplification transistor 214 to the signal line VSL in a case where the gray-scale pixel 21 is in a selected state.

[0112] Note that, in the above-described circuit example, as the circuit configuration of the gray-scale pixel 21, an example has been described in which a 4Tr configuration including the transfer transistor 212, the reset transistor 213, the amplification transistor 214, and the selection transistor 215, that is, four transistors (Tr) is exemplified, but the circuit configuration is not limited thereto. For example, the circuit configuration can be a 3Tr configuration in which the selection transistor 215 is omitted and the amplification transistor 214 has the function of the selection transistor 215, or can be a circuit configuration of 5Tr or more in which the number of transistors is increased as needed.

[0113] (DVS Pixel Configuration Example)

[0114] Figure 4 is a circuit diagram showing a specific example of the configuration of the DVS pixel 31.

[0115] The plurality of (for example, four) DVS pixels 31 constituting the DVS unit 30 each include a photoelectric conversion unit 311 and an address event detection unit 312. Note that, as one peripheral circuit of the DVS pixel 31, an arbiter unit 34 is provided. The arbiter unit 34 arbitrates requests from each of the plurality of DVS pixels 31, and transmits a response based on the arbitration result to the DVS pixel 31.

[0116] In the DVS pixel 31 having the above-described configuration, the photoelectric conversion unit 311 includes a photoelectric conversion element (light-receiving element) 3111 and a control transistor 3112. As the control transistor 3112, for example, an N-type metal oxide semiconductor (MOS) transistor can be used.

[0117] The photoelectric conversion element (light receiving element) 3111 photoelectrically converts incident light and generates a charge of an amount of charge corresponding to an amount of incident light. For example, the access control unit 12 (refer to Figure 1 ) supplies a transfer signal OFG to a gate electrode of the control transistor 3112. In response to the transfer signal OFG, the control transistor 3112 supplies the charge photoelectrically converted by the photoelectric conversion element 3111 to the address event detection unit 312.

[0118] The address event detection unit 312 detects whether an address event (hereinafter, simply described as an "event") has occurred, based on whether the amount of change in the photocurrent from the photoelectric conversion unit 311 has exceeded a predetermined threshold value. The address event includes, for example, an ON event indicating that the amount of change in the photocurrent exceeds an upper threshold value and an OFF event indicating that the amount of change in the photocurrent falls below a lower threshold value. Further, event data (event detection signal) indicating a detection result of the address event includes, for example, one bit for indicating an ON event detection result and one bit for indicating an OFF event detection result.

[0119] When the address event occurs, the address event detection unit 312 supplies a request for transmitting the event detection signal to the arbiter unit 34. Then, upon receiving a response to the request from the arbiter unit 34, the address event detection unit 312 supplies the event detection signal (event data) to the DVS reading unit 13.

[0120] (Configuration Example of Address Event Detection Unit)

[0121] Figure 5 is a block diagram showing an example of the configuration of the address event detection unit 312. As Figure 5 indicated, the address event detection unit 312 according to the present configuration example includes a current-voltage conversion unit 3121, a buffer 3122, a subtracter 3123, a quantizer 3124, and a transfer unit 3125.

[0122] The current-voltage conversion unit 3121 converts the photocurrent from the photoelectric conversion unit 311 of the DVS pixel 31 into a logarithmic voltage signal. The current-voltage conversion unit 3121 supplies the voltage signal converted by the current-voltage conversion unit 3121 to the buffer 3122. The buffer 3122 buffers the voltage signal supplied from the current-voltage conversion unit 3121 and supplies the voltage signal to the subtracter 3123.

[0123] The access control unit 12 (refer to Figure 1A row drive signal is provided to subtractor 3123. Subtractor 3123 reduces the level of the voltage signal provided from buffer 3122 according to the row drive signal. Then, subtractor 3123 provides the reduced voltage signal to quantizer 3124. Quantizer 3124 quantizes the voltage signal provided from subtractor 3123 into a digital signal and outputs the digital signal as an event detection signal (event data) to transmission unit 3125.

[0124] The transmission unit 3125 transmits the event detection signal (event data) provided by the quantizer 3124 to the arbitrator unit 34, etc. When an address event is detected, the transmission unit 3125 provides a request to the arbitrator unit 34 to send the event detection signal. Then, upon receiving a response to the request from the arbitrator unit 34, the transmission unit 3125 provides the event detection signal (event data) to the DVS reading unit 13. Event data can be read from the DVS pixel 31 by reading multiple lines.

[0125] [Layered Chip Structure]

[0126] exist Figure 1 For convenience, the system configuration of the imaging device 10A according to the first embodiment is shown as a so-called planar structure, in which the pixel array unit 11, access control unit 12, DVS readout unit 13, column signal processing unit 14, DVS signal processing unit 15, grayscale signal processing unit 16, timing control unit 17, and timestamp generation unit 18 are arranged in a planar manner. However, in this respect, the imaging device 10A according to the first embodiment has a stacked chip structure formed by stacking at least two semiconductor chips, including a first layer semiconductor chip and a second layer semiconductor chip. Specific examples of the stacked chip structure will be described below.

[0127] (First specific example)

[0128] The first specific example is a stacked chip structure with two layers. Figure 6 A schematic diagram of a stacked chip structure according to a first specific example is shown.

[0129] like Figure 6 As shown, the stacked chip structure according to the first specific example has a two-layer structure in which a first semiconductor chip 41 and a second semiconductor chip 42 are stacked. Furthermore, the first semiconductor chip 41 is provided with a pixel array unit 11, in which grayscale pixels 21 and DVS pixels 31 are mixed. That is, the first semiconductor chip 41 is a pixel chip provided with grayscale pixels 21 and DVS pixels 31.

[0130] The second-layer semiconductor chip 42 is provided with a pixel analog front-end (AFE) region 44 corresponding to the pixel array unit 11 of the first-layer semiconductor chip 41. Furthermore, in the pixel AFE region 44, corresponding to the grayscale pixel 21 and the DVS pixel 31, a grayscale memory 22 (see reference) is provided. Figure 2 The analog front-end unit for grayscale pixels 21 (A) and including DVSAFE35 (reference) Figure 2 The analog front-end unit for DVS pixel 31 (B). That is, the second-layer semiconductor chip 42 is a pixel AFE chip with pixel AFE region 44.

[0131] Furthermore, in the second-layer semiconductor chip 42, an access control unit 12, a DVS reading unit 13, a column signal processing unit 14, a DVS signal processing unit 15, a grayscale signal processing unit 16, a timing control unit 17, and a timestamp generation unit 18 are further provided in the surrounding area of ​​the pixel AFE region 44. Note that the grayscale pixel 21 and DVS pixel 31 of the first-layer semiconductor chip 41 are electrically connected to each analog front-end unit of the second-layer semiconductor chip 42 through junctions 45A and 45B formed by metal-metal bonding including Cu-Cu bonding, through-silicon vias (TSV), or microbumps.

[0132] Here, the correspondence between the grayscale pixel 21 and DVS pixel 31 of the first layer semiconductor chip 41 used as the upper chip and the analog front-end unit 25 for grayscale pixel 21 and analog front-end unit 35 for DVS pixel 31 of the second layer semiconductor chip 42 used as the lower chip will be explained.

[0133] Figure 7 Figure A is a diagram illustrating the correspondence between the grayscale pixels 21 and DVS pixels 31 of the first-layer semiconductor chip 41 and the analog front-end units 25, 35 of the second-layer semiconductor chip 42. Figure 7 In the correspondence shown in A, in the first layer semiconductor chip 41, the DVS unit 30 includes, for example, four DVS pixels 31 as a unit. Furthermore, with each DVS unit 30 as a unit, the DVS pixel 31 is associated with an analog front-end unit 35 for the DVS pixel 31. Additionally, the grayscale pixel 21 is associated one-to-one with the analog front-end unit 25 for the grayscale pixel 21.

[0134] Figure 7 Figure B is a diagram illustrating the correspondence between the grayscale pixels 21 and DVS pixels 31 of the first-layer semiconductor chip 41 and the analog front-end units 25, 35 of the second-layer semiconductor chip 42. Figure 7In correspondence 2 shown in B, in the first layer semiconductor chip 41, the DVS unit 30 includes, for example, four DVS pixels 31 as a unit. Correspondingly, for grayscale pixels 21, the grayscale pixel unit 24 includes four pixels as a unit. Furthermore, between the first layer semiconductor chip 41 and the second layer semiconductor chip 42, the grayscale pixel unit 24 and the DVS unit 30 are associated one-to-one with each analog front-end unit 25, 35 on a unit basis.

[0135] Figure 8 An image diagram showing a pixel array of grayscale pixels 21 and DVS pixels 31 is presented. Here, for the grayscale pixels 21 used as the first layer semiconductor chip 41 on the upper chip, a Bayer array of red (R) / green (G) / blue (B) is illustrated. Figure 8 In the example of the image diagram shown, DVS pixels 31 are partially arranged in a pixel array with an R / Gr / Gb / B Bayer array. From Figure 8 It is clear that the number of grayscale pixels 21 is greater than the number of DVS pixels 31, which are event pixels. Therefore, by arranging more grayscale pixels 21 than DVS pixels 31, the image quality of grayscale data can be improved.

[0136] In the second semiconductor chip 42 used as the lower chip, the analog front-end unit 35 for DVS pixels 31 is shared by four DVS pixels 31 in a 2x2 configuration, and the analog front-end unit 25 for grayscale pixels 21 is electrically connected to the grayscale pixels 21 of the upper chip on a pixel-by-pixel basis, thereby realizing a voltage domain global shutter.

[0137] (Second specific example)

[0138] The second specific example is a stacked chip structure with a three-layer structure. Figure 9 A schematic diagram of a stacked chip structure according to a second specific example is shown.

[0139] like Figure 9 As shown, the stacked chip structure according to the second specific example has a three-layer structure in which a first semiconductor chip 41, a second semiconductor chip 42, and a third semiconductor chip 43 are stacked. Furthermore, the first semiconductor chip 41 is provided with a pixel array unit 11, in which grayscale pixels 21 and DVS pixels 31 are mixed. That is, the first semiconductor chip 41 is a pixel chip provided with grayscale pixels 21 and DVS pixels 31.

[0140] The second-layer semiconductor chip 42 is provided with a pixel analog front-end (AFE) region 44 corresponding to the pixel array unit 11 of the first-layer semiconductor chip 41. Furthermore, in the pixel AFE region 44, corresponding to the grayscale pixel 21 and the DVS pixel 31, a grayscale memory 22 (see reference) is provided.Figure 2 The analog front-end unit for grayscale pixels 21 (A) and including DVSAFE35 (reference) Figure 2 The analog front-end unit for DVS pixel 31 (B). That is, the second-layer semiconductor chip 42 is a pixel AFE chip with pixel AFE region 44.

[0141] The third-layer semiconductor chip 43 includes an access control unit 12, a DVS reading unit 13, a column signal processing unit 14, a DVS signal processing unit 15, a grayscale signal processing unit 16, a timing control unit 17, and a timestamp generation unit 18. Note that the first-layer semiconductor chip 41 and the second-layer semiconductor chip 42 are electrically connected to each other via junctions 45A and 45B formed by metal-to-metal bonding (including Cu-Cu bonding), TSVs, or microbumps, and the second-layer semiconductor chip 42 and the third-layer semiconductor chip 43 are electrically connected to each other via junctions 45B and 45C formed by metal-to-metal bonding (including Cu-Cu bonding), TSVs, or microbumps.

[0142] In the specific examples above, two-layer and three-layer structures are illustrated as stacked chip structures, but stacked chip structures are not limited to two-layer and three-layer structures, and stacked chip structures with four or more layers can also be used.

[0143] [Example of pixel array unit construction]

[0144] Next, an example of the construction of the pixel array unit 11 in the camera device 10A according to the first embodiment will be described, in which grayscale pixels 21 and DVS pixels 31 are mixed.

[0145] (First Construction Example)

[0146] A first example of the construction of the pixel array unit 11 is an example in which units including grayscale pixels 21 and DVS pixels 31 are arranged side by side in the pixel array unit 11 according to a predetermined rule. Figure 10 A illustrates the pixel arrangement according to the first construction example.

[0147] The pixel arrangement according to the first construction example has the following structure: for example, pixel units X are arranged in a two-dimensional matrix, each pixel unit X including 12 grayscale pixels 21 arranged side by side in a square, each unit consisting of 4 pixels (R / Gr / Gb / B) and DVS units 30, each unit consisting of 4 DVS pixels 31. Through the pixel arrangement according to the first construction example, grayscale data can be associated with event data on a unit basis.

[0148] (Second Construction Example)

[0149] A second configuration example is an example in which units including DVS pixels 31 are arranged side by side in units of pixel rows. Figure 10 FIG. 12B of the drawings shows a pixel arrangement according to the second configuration example.

[0150] The pixel arrangement according to the second configuration example has a configuration in which, for example, pixel units X are arranged side by side in units of pixel rows, the pixel units X having 12 gradation pixels 21 arranged side by side in a square in units of 4 pixels (R / Gr / Gb / B) and DVS units 30 arranged side by side in units of 4 DVS pixels 31. With the pixel arrangement according to the second configuration example, when the gradation pixels 21 are accessed, a pixel row in which no DVS pixel 31 is present (i.e., a pixel row having no defect) can be selected.

[0151] (Third Configuration Example)

[0152] A third configuration example is an example in which units including DVS pixels 31 are arranged side by side in units of pixel columns. Figure 11 FIG. 13A of the drawings shows a pixel arrangement according to the third configuration example.

[0153] The pixel arrangement according to the third configuration example has a configuration in which, for example, pixel units X are arranged side by side in units of pixel columns, the pixel units X having 12 gradation pixels 21 arranged side by side in a square in units of 4 pixels (R / Gr / Gb / B) and DVS units 30 arranged side by side in units of 4 DVS pixels 31. With the pixel arrangement according to the third configuration example, compared with the case of the pixel arrangement according to the second configuration example, the amount of event data included in a pixel row can be uniformized, and thus, gradation data (R / G / B data) having stable image quality can be obtained. This is similar to the case of the pixel arrangement according to the first configuration example.

[0154] (Fourth Configuration Example)

[0155] A fourth configuration example is an example in which DVS units 30 are arranged side by side irregularly. Figure 11 FIG. 14B of the drawings shows a pixel arrangement according to the fourth configuration example.

[0156] The pixel arrangement according to the fourth configuration example has a configuration in which, for example, DVS units 30 in units of 4 DVS pixels 31 are arranged randomly, for example, the 4 DVS pixels 31 being arranged side by side in a square or being arranged side by side in a row direction / column direction. With the pixel arrangement according to the fourth configuration example, high-resolution data can be recovered, similarly to compressed sensing.

[0157] [Arrangement Example of DVS Unit]

[0158] Next, a configuration example of the DVS unit 30 in the imaging device 10A according to the first embodiment will be described. Here, for example, it is assumed that the DVS unit 30 includes 4 DVS pixels 31 as a unit, and the arrangement of the DVS unit 30 and the connection relationship between the DVS unit 30 and a DVS analog front end (AFE) 35 in a stacked chip structure will be described. The DVS pixels 31 are arranged side by side in the DVS unit 30 according to a predetermined rule.

[0159] (First Arrangement Example)

[0160] The first arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in a square. Figure 12 FIG. A illustrates the arrangement of the DVS unit 30 and the connection relationship between the DVS unit 30 and the DVS AFE 35 according to the first arrangement example. In the first arrangement example of the DVS unit 30, one DVS AFE 35 having a size of a rectangular region corresponding to 4 DVS pixels 31 is associated with and electrically connected to the 4 DVS pixels 31 arranged side by side in a square.

[0161] (Second Arrangement Example)

[0162] The second arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in a row direction. Figure 12 FIG. B illustrates the arrangement of the DVS unit and the connection relationship between the DVS unit and the DVS AFE 35 according to the second arrangement example. In the second arrangement example of the DVS unit 30, one DVS AFE 35 having a size of a horizontal long rectangular region corresponding to 4 DVS pixels 31 is associated with and electrically connected to the 4 DVS pixels 31 arranged side by side in a horizontal long rectangular shape in a row direction.

[0163] (Third Arrangement Example)

[0164] The third arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in a column direction. Figure 13 FIG. A illustrates the arrangement of the DVS unit and the connection relationship between the DVS unit and the DVS AFE 35 according to the third arrangement example. In the third arrangement example of the DVS unit 30, one DVS AFE 35 having a size of a vertical long rectangular region corresponding to 4 DVS pixels 31 is associated with and electrically connected to the 4 DVS pixels 31 arranged side by side in a vertical long rectangular shape in a column direction.

[0165] (Fourth Arrangement Example)

[0166] The fourth arrangement example is an example in which the size of the DVS unit 30 is larger than the size of the grayscale pixel 21. Figure 13FIG. 4B shows an arrangement of a DVS unit according to a fourth arrangement example and a connection relationship of the DVS unit with a DVS AFE 35. In the fourth arrangement example of the DVS unit 30, the size of the DVS unit 30 is larger than the size of the grayscale pixel 21. For example, the size of the DVS unit 30 is as large as the size of 4 DVS pixels 31 arranged side by side in a square shape. Further, one DVS AFE 35 having the same size as the DVS unit 30 is associated with and electrically connected to the DVS unit 30.

[0167] [Configuration example of stacked chip structure in case of grayscale pixel]

[0168] Next, a configuration example of a stacked chip structure in the case of the grayscale pixel 21 will be described.

[0169] (First configuration example of stacked chip structure)

[0170] The first configuration example of the stacked chip structure is a configuration example of a global shutter function. Figure 14 FIG. 4A schematically shows a stacked chip structure according to the first configuration example. In the stacked chip structure according to the first configuration example, in the first layer semiconductor chip 41 serving as an upper chip, the photodiode 211, the transfer transistor 212, and the buffer 216 are two-dimensionally arranged in a matrix shape for each pixel. Further, on the second layer semiconductor chip 42 serving as a lower chip, a data processing unit 217 and the like that processes a signal component (so-called D-phase data) and a reset component (so-called P-phase data) are mounted in correspondence with the pixel.

[0171] (Second configuration example of stacked chip structure)

[0172] The second configuration example of the stacked chip structure is a configuration example of a read function. Figure 14 FIG. 4B schematically shows a stacked chip structure according to the second configuration example. In the stacked chip structure according to the second configuration example, in the first layer semiconductor chip 41 serving as an upper chip, the photodiode 211 and the transfer transistor 212 are two-dimensionally arranged in a matrix shape for each pixel. Further, on the second layer semiconductor chip 42 serving as a lower chip, a reset transistor 213, an amplification transistor 214, and a selection transistor 215 and the like are mounted in correspondence with the pixel.

[0173] (Third configuration example of stacked chip structure)

[0174] The third configuration example of the stacked chip structure is a configuration example of a region AD. Figure 15 FIG. 4C schematically shows a stacked chip structure according to the third configuration example. In the stacked chip structure according to the third configuration example, for example, in the first layer semiconductor chip 41 serving as an upper chip, for each pixel, the photodiode 211, the transfer transistor 212, and the buffer 216 are two-dimensionally arranged in a matrix shape. Further, on the second layer semiconductor chip 42 serving as a lower chip, the reset transistor 213, the amplification transistor 214, and the selection transistor 215 and the like are mounted in correspondence with the pixel.Figure 3 The gray scale pixels 21 of the illustrated circuit configuration are two-dimensionally arranged in a matrix shape. Further, corresponding to the two-dimensionally arranged gray scale pixels 21, an analog-digital converter (ADC) 141 is mounted on the second layer semiconductor chip 42 serving as a lower chip. This configuration is a configuration example of a regional AD.

[0175] [Effects and advantages of the first embodiment]

[0176] As described above, the imaging device 10A according to the first embodiment has a configuration in which the gray scale pixels 21 and the DVS pixels 31 as event pixels are mixed in the pixel array unit 11. Therefore, with the imaging device 10A according to the first embodiment, not only an event detection signal (event data) indicating an event occurrence can be output, but also a pixel signal (gray scale data) of a gray scale corresponding to an amount of incident light can be output. With this arrangement, as an example, by detecting a movement of a subject by the DVS pixels 31 and correcting using the event data, imaging without motion blur can be achieved even in a dim environment, and gray scale data with excellent S / N can be obtained by using the gray scale pixels 21.

[0177] [Imaging device according to the second embodiment]

[0178] The imaging device according to the second embodiment of the present disclosure has a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip, in which an event pixel and a ranging pixel are mixed in a pixel array unit provided in the first layer semiconductor chip.

[0179] Here, the ranging pixel is a pixel including a light receiving element (photodetecting element) that receives reflected light from a ranging target based on irradiation light from a light source unit and generates a signal in response to reception of a photon. As a light receiving element that generates a signal in response to reception of a photon, for example, a single photon avalanche diode (SPAD) element can be exemplified. The SPAD element operates in a region called Geiger mode, which has no DC stable point, in which the element operates at a reverse voltage exceeding a breakdown voltage.

[0180] Note that, here, as the light receiving element of the pixel, the SPAD element is exemplified, but the light receiving element is not limited to the SPAD element. That is, as the light receiving element, in addition to the SPAD element, various elements that operate in Geiger mode, such as an avalanche photodiode (APD) and a silicon photomultiplier (SiPM), can be used.

[0181] [Example of system configuration]

[0182] Figure 16This is a block diagram schematically illustrating the system configuration of a camera device according to a second embodiment of the present disclosure.

[0183] In addition to the pixel array unit 11, access control unit 12, DVS reading unit 13, DVS signal processing unit 15, timing control unit 17, timestamp generation unit 18, and output interface (I / F) 19A, the camera device 10B according to the second embodiment also includes a time measurement unit (time-to-digital converter: TDC) 61, a ranging signal processing unit 62, and an output interface 19C.

[0184] The pixel array unit 11 is configured with a DVS unit 30 and a ranging pixel 50 in a mixed state. As the DVS unit 30, a DVS unit 30 having the configuration illustrated in the first embodiment can be used, which includes a DVS pixel 31 as an event pixel (see reference). Figure 2 (B).

[0185] Figure 17 An example of the basic pixel circuit structure of the ranging pixel 50 is shown. Here, the basic structure of a pixel is shown. The ranging pixel 50 includes, for example, a SPAD element 51 as a light receiving element, a quenching circuit 52, and a readout circuit 53.

[0186] SPAD element 51 has a cathode connected to a first power supply and an anode connected to a second power supply via a quenching circuit 52. A large voltage, i.e., a voltage higher than or equal to the breakdown voltage, is applied from the first power supply to the cathode of SPAD element 51 to cause avalanche breakdown.

[0187] The quenching circuit 52 stops avalanche breakdown by reducing the voltage between the terminals of the SPAD element 51, i.e., the voltage between the cathode and anode, below the breakdown voltage of the PN diode. This operation is called quenching.

[0188] Readout circuit 53, for example, includes a P-type MOS transistor Q. p and N-type MOS transistor Q n The CMOS inverter circuit detects the reaction edge of the SPAD element 51. The detection output of the readout circuit 53 is provided as the SPAD output (pixel output). Figure 16 The time measurement unit (TDC) 61 shown.

[0189] The camera device 10B according to the second embodiment, having the above-described structure, includes a DVS pixel 31 and a ranging pixel 50, and is capable of measuring the distance to the target (subject). The ranging pixel 50 is configured to acquire event information and includes a SPAD element 51 that generates a signal in response to the reception of photons.

[0190] As a measurement method for measuring the distance to the distance-measuring target, for example, a ToF (Time of Flight) method for measuring the time for which light emitted from a light source unit toward the distance-measuring target is reflected by the distance-measuring target and returned can be employed. In order to employ the ToF method, the light source unit 70 for emitting light toward the distance-measuring target is provided as an external device of the image pickup device 10B according to the second embodiment. The distance-measuring device (system) can be constituted by the combination of the image pickup device 10B according to the second embodiment and the light source unit 70.

[0191] The light source unit 70 includes, for example, a semiconductor laser, and emits laser light toward a measurement target (object) by being driven by a laser drive unit (not shown). The laser light emitted from the light source unit 70 is reflected by the measurement target (object), and the reflected light is incident on the distance-measuring pixel 50 including the SPAD element 51.

[0192] Returning to Figure 16 An explanation will be given. Based on the SPAD output from the distance-measuring pixel 50, a time measurement unit (TDC) 61 measures the time for which the laser light emitted toward the measurement target is reflected by the measurement target and returned. The time measurement is performed a plurality of times, and the time is measured by detecting the peak position in a ToF histogram obtained by accumulating the times of the plurality of measurements.

[0193] As a method of time measurement, a method of measuring the time from the timing at which the laser light is emitted from the light source unit 70 toward the measurement target to the timing at which the reflected light from the measurement target is received at the distance-measuring pixel 50 can be exemplified. As another method of time measurement, a method of emitting pulsed light from the light source unit 70 at a predetermined period, detecting the period at which the pulsed light is received at the distance-measuring pixel 50, and measuring the time from the phase difference between the light emission period and the light reception period can be exemplified.

[0194] The measurement result of the time measurement unit 61 is supplied to a distance-measuring signal processing unit 62. The distance-measuring signal processing unit 62 performs predetermined signal processing on the measurement result of the time measurement unit 61, and then outputs the result as distance-measuring data via the output interface 19C.

[0195] [Stacked Chip Structure]

[0196] In Figure 16In the present embodiment, for convenience, the system configuration of the imaging device 10B according to the second embodiment is shown as a planar structure in which the pixel array unit 11, the access control unit 12, the DVS reading unit 13, the time measurement unit 61, the DVS signal processing unit 15, the ranging signal processing unit 62, the timing control unit 17, and the time stamp generation unit 18, and the like are arranged in a planar manner. However, at this point, similarly to the imaging device 10A according to the first embodiment, the imaging device 10B according to the second embodiment has a stacked chip structure formed by stacking at least two semiconductor chips including the first layer semiconductor chip and the second layer semiconductor chip.

[0197] Hereinafter, a specific example of the stacked chip structure will be described. As the stacked chip structure, a two-layer structure is exemplified below, but the stacked chip structure is not limited to the two-layer structure, and similarly to the case of the imaging device 10A according to the first embodiment, a stacked chip structure having three or more layers can also be employed.

[0198] (First Specific Example)

[0199] The first specific example is an example in which the SPAD element 51 is mounted on the first layer semiconductor chip 41 serving as an upper chip and the quenching circuit 52 is mounted on the second layer semiconductor chip 42 serving as a lower chip. Figure 18 FIG. A of the same drawing illustrates a schematic view of the stacked chip structure according to the first specific example.

[0200] As illustrated in FIG. A of the same drawing, in the stacked chip structure according to the first specific example, the SPAD element 51 is arranged in an array on the first layer semiconductor chip 41 serving as an upper chip, and the quenching circuit 52 that controls the quenching operation of the SPAD element 51 and the reading circuit 53 are mounted one-to-one on the second layer semiconductor chip 42 serving as a lower chip. Figure 18

[0201] (Second Specific Example)

[0202] The second specific example is an example in which a plurality of SPAD elements 51 on the first layer semiconductor chip 41 serving as an upper chip is regarded as a unit and the quenching circuit 52 and the time measurement unit (TDC) 61 are mounted on the second layer semiconductor chip 42 serving as a lower chip. Figure 18 FIG. B of the same drawing illustrates a schematic view of the stacked chip structure according to the second specific example.

[0203] As illustrated in FIG. B of the same drawing, in the stacked chip structure according to the second specific example, a plurality of SPAD elements 51 on the first layer semiconductor chip 41 serving as an upper chip is regarded as a unit, and the quenching circuit 52 and the time measurement unit (TDC) 61 are mounted on the second layer semiconductor chip 42 serving as a lower chip. Figure 18 ​In the layered chip structure according to the second specific example, as shown in B of the same drawing, a plurality of SPAD elements 51, which are regarded as units, are arranged in an array on the first layer semiconductor chip 41 serving as an upper chip, and the quenching circuit 52, the reading circuit 53, and the time measurement unit (TDC) 61 are installed one-to-one on the second layer semiconductor chip 42 serving as a lower chip.

[0204] [Configuration example of pixel array unit]

[0205] Next, a configuration example of the pixel array unit 11 in the imaging device 10B according to the second embodiment, in which the DVS pixels 31 and the ranging pixels 50 are mixed, will be described.

[0206] (First configuration example)

[0207] The first configuration example of the pixel array unit 11 is an example in which units including the DVS pixels 31 and the SPAD elements 51 are arranged in a matrix. Figure 19 A of the same drawing shows a pixel arrangement according to the first configuration example.

[0208] The pixel arrangement according to the first configuration example has a configuration in which, for example, pixel units X are two-dimensionally arranged in a matrix, and each pixel unit X has 4 DVS pixels 31 and 12 SPAD elements 51 arranged side by side in a square. With the pixel arrangement according to the first configuration example, it is possible to associate ranging data with event data in units.

[0209] (Second configuration example)

[0210] The second configuration example is an example in which units including the DVS pixels 31 are arranged side by side in units of pixel rows. Figure 19 B of the same drawing shows a pixel arrangement according to the second configuration example.

[0211] The pixel arrangement according to the second configuration example has a configuration in which, for example, pixel units X are arranged side by side in units of pixel rows, and each pixel unit X has 4 DVS pixels 31 and 12 SPAD elements 51 arranged side by side in a square. With the pixel arrangement according to the second configuration example, when the SPAD elements 51 are accessed, it is possible to select a pixel row in which there is no DVS pixel 31 (i.e., a pixel row having no defect).

[0212] (Third configuration example)

[0213] The third configuration example is an example in which units including the DVS pixels 31 are arranged side by side in units of pixel columns. Figure 20 A of the same drawing shows a pixel arrangement according to the third configuration example.

[0214] The pixel arrangement according to the third configuration example has a configuration in which, for example, the pixel units X are arranged side by side in units of pixel columns, and each pixel unit X has 4 DVS pixels 31 and 12 SPAD elements 51 arranged side by side in a square. With the pixel arrangement according to the third configuration example, the amount of event data included in a pixel row can be uniformized compared to the case of the pixel arrangement according to the second configuration example, and thus, it is possible to obtain gray scale data (R / G / B data) with stable image quality. This is similar to the case of the pixel arrangement according to the first configuration example.

[0215] (Fourth Configuration Example)

[0216] The fourth configuration example is an example in which the DVS units 30 are arranged randomly. Figure 20 FIG. B illustrates a pixel arrangement according to the fourth configuration example.

[0217] The pixel arrangement according to the fourth configuration example has a configuration in which, for example, the DVS units 30 are arranged randomly in units of 4 DVS pixels 31, for example, arranged side by side in a square or arranged side by side in a row direction / column direction. With the pixel arrangement according to the fourth configuration example, it is possible to restore high-resolution data similar to compressed sensing.

[0218] [Arrangement Example of DVS Unit]

[0219] Next, a configuration example of the DVS unit 30 in the imaging device 10B according to the second embodiment will be described. Here, for example, it is assumed that the DVS unit 30 includes 4 DVS pixels 31 as a unit, and the arrangement of the DVS unit 30 and the connection relationship between the DVS unit 30 and the DVS analog front end (AFE) 35 in the stacked chip structure will be described.

[0220] (First Arrangement Example)

[0221] The first arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in a square. Figure 21 FIG. A illustrates the arrangement of the DVS unit 30 according to the first arrangement example and the connection relationship between the DVS unit and the DVS AFE 35. In the first arrangement example of the DVS unit 30, one DVS AFE 35 having a size of a rectangular region corresponding to 4 DVS pixels 31 is associated with and electrically connected to 4 DVS pixels 31 arranged side by side in a square.

[0222] (Second Arrangement Example)

[0223] The second arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in a row direction. Figure 21Fig. 7B shows the arrangement of the DVS unit according to the second arrangement example and the connection relationship of the DVS unit with the DVS AFE 35. In the second arrangement example of the DVS unit 30, one DVS AFE 35 having a size corresponding to a horizontal-long rectangular region of the 4 DVS pixels 31 is associated with and electrically connected to the 4 DVS pixels 31 arranged side by side in a horizontal-long rectangular shape in the row direction.

[0224] (Third arrangement example)

[0225] The third arrangement example is an example in which the DVS unit 30 includes 4 DVS pixels 31 arranged side by side in the column direction. Figure 22 Fig. 7A shows the arrangement of the DVS unit according to the third arrangement example and the connection relationship of the DVS unit with the DVS AFE 35. In the third arrangement example of the DVS unit 30, one DVS AFE 35 having a size corresponding to a vertical-long rectangular region of the 4 DVS pixels 31 is associated with and electrically connected to the 4 DVS pixels 31 arranged side by side in a vertical-long rectangular shape in the column direction.

[0226] (Fourth arrangement example)

[0227] The fourth arrangement example is an example in which the size of the DVS unit 30 is larger than the size of the distance measuring pixel 50 including the SPAD element 51. Figure 22 Fig. 7B shows the arrangement of the DVS unit according to the fourth arrangement example and the connection relationship of the DVS unit with the DVS AFE 35. In the fourth arrangement example of the DVS unit 30, the size of the DVS unit 30 is larger than the size of the distance measuring pixel 50 including the SPAD element 51. For example, the size of the DVS unit 30 is as large as the size of the 4 DVS pixels 31 arranged side by side in a square shape. Further, one DVS AFE 35 having the same size as the DVS unit 30 is associated with and electrically connected to the DVS unit 30.

[0228] [Effects and advantages of the second embodiment]

[0229] As described above, the imaging device 10B according to the second embodiment has a configuration in which the DVS pixels 31 that are event pixels and the distance measuring pixels 50 including the SPAD elements 51 are mixed. Therefore, with the imaging device 10B according to the second embodiment, not only event data indicating the occurrence of an event can be output, but also distance measuring data as distance information to a measurement target (subject) can be output. With this arrangement, for example, by detecting a moving object with the DVS pixels 31 and performing distance measurement on the detected moving object based only on the distance measuring data, low-power distance measurement can be achieved compared to a case in which all subjects are distance-measured.

[0230] [Imaging device according to the third embodiment]

[0231] The imaging device according to the third embodiment of the present disclosure has a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip, in which event pixels, grayscale pixels, and ranging pixels are mixed in a pixel array unit provided in the first layer semiconductor chip. That is, the imaging device according to the third embodiment of the present disclosure has a configuration in which the configuration of the imaging device according to the first embodiment and the configuration of the imaging device according to the second embodiment are combined.

[0232] [system configuration example]

[0233] Figure 23 is a block diagram schematically showing a system configuration of the imaging device according to the third embodiment of the present disclosure.

[0234] The imaging device 10C according to the third embodiment includes a pixel array unit 11, an access control unit 12, a DVS reading unit 13, a column signal processing unit 14, a DVS signal processing unit 15, a grayscale signal processing unit 16, a timing control unit 17, a time stamp generation unit 18, and output interfaces 19A, 19B. This is Figure 1 the configuration of the imaging device 10A according to the first embodiment. In addition, the imaging device 10C according to the third embodiment further includes a time measurement unit (TDC) 61, a ranging signal processing unit 62, and an output interface 19C. This is Figure 16 the configuration of the imaging device 10B according to the second embodiment.

[0235] [Effects of the third embodiment]

[0236] As described above, the imaging device 10C according to the third embodiment has a configuration in which the DVS pixels 31 as event pixels, the grayscale pixels 21, and the ranging pixels 50 including the SPAD elements 51 are mixed. Therefore, with the imaging device 10C according to the third embodiment, not only event data indicating occurrence of an event can be output, but also a pixel signal (grayscale data) of a grayscale level corresponding to an amount of incident light and ranging data as distance information to a measurement target (subject) can be output.

[0237] With this arrangement, the effects of the first embodiment and the effects of the second embodiment can be obtained. Specifically, as an example, by detecting a movement of a subject with the DVS pixels 31 and correcting using event data, imaging without motion blur can be achieved even in a dim environment, and grayscale data with excellent S / N can be obtained by using the grayscale pixels 21. In addition, for example, by detecting a moving object with the DVS pixels 31 and ranging the detected moving object based only on the ranging data, ranging with low power consumption can be achieved compared to the case of ranging all subjects.

[0238] <Modified example>

[0239] Although the technology according to the present disclosure has been described above based on the preferred embodiments, the technology according to the present disclosure is not limited to these embodiments. The configurations and structures of the imaging device according to the first embodiment in which event pixels and gray-scale pixels are mixed, the configurations and structures of the imaging device according to the second embodiment in which event pixels and ranging pixels are mixed, and the configurations and structures of the imaging device according to the third embodiment in which event pixels, gray-scale pixels, and ranging pixels are mixed, which are described in the above embodiments, are all examples, and can be appropriately modified.

[0240] <Application example>

[0241] The imaging device according to the first embodiment, the imaging device according to the second embodiment, or the imaging device according to the third embodiment can be used for various devices that sense light such as visible light, infrared light, ultraviolet light, and X-rays. Specific examples of the various devices are listed below.

[0242] • A device for taking an image for appreciation, such as a digital camera or a portable device with a camera function, and the like.

[0243] • A device for transportation, such as a vehicle-mounted sensor that takes an image of the front, rear, surroundings, or interior of a vehicle, or the like, for safe driving such as automatic stopping, or recognizing the state of a driver, or the like; a monitoring camera that monitors a traveling vehicle or a road; or a ranging sensor that measures the distance between vehicles, or the like.

[0244] • A device for a household electric appliance such as a television, a refrigerator, or an air conditioner, which takes an image of a user's gesture and performs a device operation according to the gesture.

[0245] • A device for medical care or health care, such as an endoscope or a device that performs angiography by receiving infrared light, or the like.

[0246] • A device for security, such as a monitoring camera for preventing crime or a camera for personal authentication, or the like.

[0247] • A device for beauty, such as a skin measurement device that takes an image of skin or a microscope that takes an image of a scalp, or the like.

[0248] • A device for sports, such as a sports camera or a wearable camera for sports applications, or the like.

[0249] • A device for agriculture, such as a camera for monitoring the state of a field or crops, or the like.

[0250] <Configuration that the present disclosure can have>

[0251] Note that the present disclosure can also have the following configuration.

[0252] <<A. Imaging device>>

[0253] [A-01] An imaging device including a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip,

[0254] wherein a pixel array unit in which an event pixel and a gray scale pixel are mixed is provided in the first layer semiconductor chip, the event pixel detects an occurrence of an event in which a change amount of luminance of a pixel exceeds a predetermined threshold value and outputs an event detection signal, and the gray scale pixel outputs a pixel signal of a gray scale level corresponding to an amount of incident light, and

[0255] an analog front end unit for event pixels that processes the event detection signal and an analog front end unit for gray scale pixels that processes the pixel signal are provided in the second layer semiconductor chip corresponding to the event pixel and the gray scale pixel, respectively.

[0256] [A-02] The imaging device according to [A-01],

[0257] wherein when a predetermined number of event pixels is defined as an event pixel unit,

[0258] the event pixel unit is arranged side by side in the pixel array unit according to a predetermined rule.

[0259] [A-03] The imaging device according to [A-02],

[0260] wherein the event pixel unit is combined with a predetermined number of gray scale pixels as a pixel unit and arranged in the pixel array unit.

[0261] [A-04] The imaging device according to [A-02],

[0262] wherein in a matrix-shaped pixel array of the pixel array unit, the event pixel unit is arranged side by side in units of pixel rows.

[0263] [A-05] The imaging device according to [A-02],

[0264] wherein in a matrix-shaped pixel array of the pixel array unit, the event pixel unit is arranged side by side in units of pixel columns.

[0265] [A-06] The imaging device according to [A-01],

[0266] wherein when a predetermined number of event pixels is defined as an event pixel unit,

[0267] The event pixel units are irregularly arranged side by side in the pixel array unit.

[0268] [A-07] The imaging device according to [A-02],

[0269] wherein the event pixels are arranged side by side in the event pixel unit according to a predetermined rule.

[0270] [A-08] The imaging device according to [A-07],

[0271] wherein the event pixels are arranged side by side in the event pixel unit in a square shape.

[0272] [A-09] The imaging device according to [A-07],

[0273] wherein the event pixels are arranged side by side in the event pixel unit in a row direction of a matrix-shaped pixel array.

[0274] [A-10] The imaging device according to [A-07],

[0275] wherein the event pixels are arranged side by side in the event pixel unit in a column direction of a matrix-shaped pixel array.

[0276] [A-11] The imaging device according to [A-07],

[0277] wherein a size of the event pixels is larger than a size of the grayscale pixels.

[0278] [A-12] The imaging device according to any one of [A-01] to [A-11],

[0279] wherein a number of the grayscale pixels is larger than a number of the event pixels.

[0280] [A-13] The imaging device according to [A-01],

[0281] wherein, in addition to the event pixels and the grayscale pixels, a ranging pixel that includes a light-receiving element that receives reflected light from a ranging target based on irradiation light from a light source unit and generates a signal in response to reception of a photon is also mixed in the pixel array unit.

[0282] [A-14] The imaging device according to [A-13],

[0283] wherein a quenching circuit that controls the light-receiving element is provided in the second layer semiconductor chip corresponding to the ranging pixel.

[0284] [A-15] The imaging device according to [A-13] or [A-14],

[0285] wherein the light-receiving element of the range-finding pixel includes an avalanche photodiode operating in a Geiger mode.

[0286] [A-16] The imaging device according to [A-15],

[0287] wherein the light-receiving element of the range-finding pixel includes a single-photon avalanche diode.

[0288] <<B. Other Imaging Devices>>

[0289] [B-01] An imaging device including a stacked chip structure formed by stacking at least two semiconductor chips including a first layer semiconductor chip and a second layer semiconductor chip,

[0290] wherein a pixel array unit in which an event pixel and a range-finding pixel are mixed is provided in the first layer semiconductor chip, the event pixel detects an occurrence of an event in which a luminance change amount of a pixel exceeds a predetermined threshold value and outputs an event detection signal, the range-finding pixel includes a light-receiving element that receives reflected light from a range-finding target based on irradiation light from a light source unit and generates a signal in response to reception of a photon, and

[0291] an analog front-end unit for the event pixel that processes the event detection signal and an analog front-end unit for the range-finding pixel that processes the signal from the light-receiving element are provided in the second layer semiconductor chip corresponding to the event pixel and the range-finding pixel, respectively.

[0292] [B-02] The imaging device according to [B-01],

[0293] wherein a quenching circuit that controls the light-receiving element is provided in the second layer semiconductor chip corresponding to the range-finding pixel.

[0294] [B-03] The imaging device according to [B-01] or [B-02],

[0295] wherein the light-receiving element of the range-finding pixel includes an avalanche photodiode operating in a Geiger mode.

[0296] [B-04] The imaging device according to [B-03],

[0297] wherein the light-receiving element of the range-finding pixel includes a single-photon avalanche diode.

[0298] List of reference signs

[0299] 10A imaging device according to the first embodiment

[0300] 10B imaging device according to the second embodiment

[0301] 10C imaging device according to the third embodiment

[0302] 11 pixel array unit

[0303] 12 access control unit

[0304] 13 DVS readout unit

[0305] 14 column signal processing unit

[0306] 15 DVS signal processing unit

[0307] 16 grayscale signal processing unit

[0308] 17 timing control unit

[0309] 18 timestamp generation unit

[0310] 19A, 19B output interface (I / F)

[0311] 20 grayscale circuit

[0312] 21 grayscale pixel

[0313] 22 grayscale memory

[0314] 24 grayscale pixel unit

[0315] 30 DVS unit (event pixel unit)

[0316] 31 DVS pixel

[0317] 32 selector

[0318] 35 DVS AFE (analog front end)

[0319] 41 first layer semiconductor chip

[0320] 42 second layer semiconductor chip

[0321] 43 third layer semiconductor chip

[0322] 50 range-finding pixel

[0323] 51 SPAD element

[0324] 52 quenching circuit

[0325] 53 readout circuit

[0326] 61 time measurement unit (TDC)

[0327] 62 distance measurement signal processing unit

[0328] 70 light source unit

Claims

1. A camera device comprising a stacked chip structure formed by stacking at least two semiconductor chips, including a first semiconductor chip layer and a second semiconductor chip layer. in, A pixel array unit is provided in the first layer of the semiconductor chip. Event pixels and grayscale pixels are mixed in the pixel array unit. The event pixel detects the phenomenon that the brightness change of a pixel exceeds a predetermined threshold as an event and outputs an event detection signal. The grayscale pixel outputs a pixel signal with a grayscale level corresponding to the incident light amount. In the second layer semiconductor chip, an analog front-end unit for the event pixel and an analog front-end unit for the grayscale pixel are respectively provided to process the event detection signal and process the pixel signal, corresponding to the event pixel and the grayscale pixel. In addition to the event pixels and the grayscale pixels, ranging pixels are also mixed in the pixel array unit. Each ranging pixel includes a light receiving element that receives reflected light from the ranging target based on illumination light from the light source unit and generates a signal in response to the reception of photons. In this process, a quenching circuit is provided in the second layer semiconductor chip corresponding to the ranging pixel, and the quenching circuit controls the light receiving element.

2. The camera device according to claim 1, in, When a predetermined number of event pixels are defined as event pixel units The event pixel units are arranged side by side in the pixel array unit according to a predetermined rule.

3. The camera device according to claim 2, in, The event pixel unit is combined with a predetermined number of grayscale pixels to form a pixel unit, and arranged in the pixel array unit.

4. The camera device according to claim 2, in, In the matrix-like pixel array of the pixel array unit, the event pixel units are arranged side by side in units of pixel rows.

5. The camera device according to claim 2, in, In the matrix-like pixel array of the pixel array unit, the event pixel units are arranged side by side in units of pixel columns.

6. The camera device according to claim 1, in, When a predetermined number of event pixels are defined as event pixel units The event pixel units are arranged irregularly side by side in the pixel array unit.

7. The camera device according to claim 2, in, The event pixels are arranged side-by-side in the event pixel unit according to a predetermined rule.

8. The camera device according to claim 7, in, The event pixels are arranged side by side in a square within the event pixel unit.

9. The camera device according to claim 7, in, The event pixels are arranged side by side in the event pixel unit along the row direction of the matrix-like pixel array.

10. The camera device according to claim 7, in, The event pixels are arranged side-by-side along the column direction of the matrix-like pixel array in the event pixel unit.

11. The camera device according to claim 7, in, The size of the event pixel is larger than the size of the grayscale pixel.

12. The camera device according to claim 1, in, The number of grayscale pixels is greater than the number of event pixels.

13. The camera device according to any one of claims 1 to 12, in, The light-receiving element of the ranging pixel includes an avalanche photodiode operating in Geiger mode.

14. The camera device according to claim 13, in, The light-receiving element of the ranging pixel includes a single-photon avalanche diode.

15. A camera device comprising a stacked chip structure formed by stacking at least two semiconductor chips, including a first semiconductor chip layer and a second semiconductor chip layer. in, A pixel array unit is provided in the first layer of the semiconductor chip. Event pixels and ranging pixels are mixed in the pixel array unit. The event pixel detects the phenomenon that the brightness change of a pixel exceeds a predetermined threshold as an event and outputs an event detection signal. The ranging pixel includes a light receiving element. The light receiving element receives reflected light from the ranging target based on the illumination light from the light source unit and generates a signal in response to the reception of photons. In the second layer of the semiconductor chip, an analog front-end unit for the event pixel is provided for processing the event detection signal, and an analog front-end unit for the ranging pixel is provided for processing the signal from the light receiving element, corresponding to the event pixel and the ranging pixel, respectively. In this process, a quenching circuit is provided in the second layer semiconductor chip corresponding to the ranging pixel, and the quenching circuit controls the light receiving element.

16. The camera device according to claim 15, in, The light-receiving element of the ranging pixel includes an avalanche photodiode operating in Geiger mode.

17. The camera device according to claim 16, in, The light-receiving element of the ranging pixel includes a single-photon avalanche diode.

Images