Imaging apparatus and imaging method

Abstract

[Problem] To output event information sensitively and at high speed using a simple configuration. [Solution] An image forming apparatus includes: a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal; a detection unit that outputs a detection signal when an absolute value of a change amount of the electric signal within a pixel exceeds a predetermined threshold value; a signal processing unit that performs predetermined signal processing based on the detection signal output from the detection unit; an AZ output unit that outputs an auto-zero signal for initializing the detection unit; a time code generator that outputs a time code that changes at a predetermined period; a first holding circuit that holds the time code output from the time code generator when the auto-zero signal is output; a second holding circuit that holds the time code output from the time code generator when the detection signal is output; and a transfer unit that transfers the time code held in the first holding circuit and the time code held in the second holding circuit to the signal processing unit in association with each other.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an imaging device and an imaging method. BACKGROUND

[0002] An imaging device that acquires data of a portion in which a luminance level changes due to a specific event only when an event occurs in an imaging scene is known (see Patent Literature 1). An imaging device of this type can be referred to as an event-based vision sensor (EVS).

[0003] Furthermore, an imaging device that outputs not only event information but also a luminance value is also known (see Patent Literature 2).

[0004] LIST OF CITATIONS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2017-535999

[0007] Patent Literature 2: Japanese Patent Application Publication No. 2016-533140 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] In a case where flickering occurs, since a plurality of pixels detect an event at almost the same time, it takes time to process the detected event. Furthermore, in a case where event information and a luminance value are output together, since the luminance value must be read at high speed, there is a possibility that photoelectric conversion performance in darkness deteriorates.

[0010] An imaging device in which an analog-digital converter is provided for each pixel so as to be able to output a luminance value at high speed is known. However, if an attempt is made to output a luminance value and event information in association with each other in an imaging device of this type, the configuration of the imaging device can be complicated.

[0011] Therefore, the present disclosure provides an imaging device and an imaging method that are able to output event information with high sensitivity and high speed with a simple configuration.

[0012] SOLUTION TO PROBLEM

[0013] In order to solve the above problem, according to the present disclosure, there is provided an imaging device including:

[0014] a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal;

[0015] a detection section that outputs a detection signal in a case where an absolute value of an amount of change in the electric signal within a pixel of the plurality of pixels exceeds a predetermined threshold value;

[0016] a signal processing section that performs predetermined signal processing based on the detection signal output from the detection section;

[0017] an AZ output section that outputs an auto-zero signal for initializing the detection section;

[0018] a time code generator that outputs a time code that varies at a predetermined cycle;

[0019] a first holding circuit that holds the time code output from the time code generator when the auto-zero signal is output;

[0020] a second holding circuit that holds the time code output from the time code generator when the detection signal is output; and

[0021] a transfer section that transfers the time code held in the first holding circuit and the time code held in the second holding circuit to the signal processing section in association with each other.

[0022] The first holding circuit can hold the time code corresponding to the auto-zero signal for each of a plurality of pixels, and

[0023] The second holding circuit can hold the time code corresponding to the detection signal for a pixel of the plurality of pixels that has output the detection signal.

[0024] The second holding circuit can hold data for the plurality of pixels by holding predetermined data for a pixel that does not output the detection signal.

[0025] The transfer section can associate the data for the plurality of pixels held by the first holding circuit and the data for the plurality of pixels held by the second holding circuit, and sequentially transfer the data to the signal processing section per pixel.

[0026] A cycle in which the time code corresponding to the auto-zero signal for the plurality of pixels is held in the first holding circuit, a cycle in which the time code corresponding to the detection signal for the plurality of pixels is held in the second holding circuit, and a cycle in which the data for the plurality of pixels held by the first holding circuit and the second holding circuit are sequentially read per pixel and transferred by the transfer section can be sequentially repeated.

[0027] In a cycle of one frame in which the data for the plurality of pixels are read, a cycle in which the time code corresponding to the auto-zero signal for the plurality of pixels is held in the first holding circuit, a cycle in which the time code corresponding to the detection signal for the plurality of pixels is held in the second holding circuit, and a cycle in which the data for the plurality of pixels held by the first holding circuit and the second holding circuit are sequentially read per pixel and transferred by the transfer section can be set.

[0028] The transfer section can include a plurality of stages of clusters connected in series, the clusters transfer the time code generated by the time code generator and sequentially transfer the data held by the first holding circuit and the data held by the second holding circuit per pixel, and

[0029] Each of the plurality of stages of clusters can transfer the time code generated by the time code generator to a cluster in a subsequent stage or the signal processing section and transfer the data held by the first holding circuit and the data held by the second holding circuit to the cluster in the subsequent stage or the signal processing section.

[0030] The cluster in the first stage can hold the time code output from the time code generator and hold the data held by the corresponding first holding circuit and the data held by the second holding circuit at the same timing, and

[0031] The clusters in the second and subsequent stages can transfer the time code sequentially transferred from the time code generator held in the cluster in the preceding stage to the cluster in the subsequent stage or the signal processing section, transfer the corresponding data held by the first holding circuit and the data held by the second holding circuit to the cluster in the subsequent stage or the signal processing section, and transfer the data held by the cluster in the preceding stage to the cluster in the subsequent stage or the signal processing section.

[0032] The transfer section can include:

[0033] a first cluster in the plurality of stages, the first cluster transferring the time code generated by the time code generator; and

[0034] a second cluster in the plurality of stages, the second cluster transferring the data held by the first holding circuit and the data held by the second holding circuit,

[0035] Each of the first clusters in the plurality of stages can transfer the data held by the corresponding first holding circuit and the data held by the second holding circuit to the corresponding second cluster, and

[0036] The signal processing section can receive the time code output from the first cluster in the final stage and receive the data output from the second cluster in the final stage.

[0037] The first cluster in the first stage can hold the time code output from the time code generator,

[0038] The second cluster in the first stage can hold the data held by the corresponding first holding circuit and the data held by the second holding circuit,

[0039] The first cluster in the second and subsequent stages can hold the time code sequentially transferred from the time code generator held in the first cluster in the preceding stage, and

[0040] The second cluster in the second stage and the subsequent stages can transfer data held by the corresponding first holding circuit and data held by the corresponding second holding circuit to the second cluster in the subsequent stage or the signal processing section, and transfer data held by the second cluster in the previous stage to the second cluster in the subsequent stage or the signal processing section.

[0041] During a first frame period in which data of the plurality of pixels is read, a period in which a time code corresponding to the auto-zero signal of the plurality of pixels is held in the first holding circuit and a period in which a time code corresponding to the detection signal of the plurality of pixels is held in the second holding circuit can be set, and

[0042] Each of the plurality of second clusters can transfer, in a second frame period after the first frame period, data of the plurality of pixels held in the first holding circuit in the first frame period and data of the plurality of pixels held in the second holding circuit in the first frame period to the second cluster of the subsequent stage or the signal processing section.

[0043] The detection processing of the detection signal of the plurality of pixels can also be started asynchronously with a frame period in which data of the plurality of pixels is read, the detection signal of the pixel is reset after a time code corresponding to the detection signal of the pixel outputting the detection signal is held in the second holding circuit, the auto-zero signal is output, data held by the second holding circuit is transferred to the corresponding second cluster, and then the detection processing of the detection signal of the pixel is resumed after a time code corresponding to the auto-zero signal is held in the first holding circuit.

[0044] Each of the plurality of pixels can include an analog-to-digital converter that generates a first digital signal obtained by digitally converting a reset level of an electric signal generated by a plurality of photoelectric conversion elements and a second digital signal obtained by digitally converting the electric signal,

[0045] The first holding circuit can hold a time code corresponding to the auto-zero signal or the first digital signal, and

[0046] The second holding circuit can hold a time code corresponding to the detection signal or the second digital signal.

[0047] The imaging device can further include:

[0048] a first selector that selects one of the auto-zero signal and the first digital signal and provides the selected one to the first holding circuit; and

[0049] a second selector that selects one of the detection signal and the second digital signal and provides the selected one to the second holding circuit,

[0050] The first selector and the second selector can cooperate with each other to perform a selection operation such that, when the first holding circuit holds a time code corresponding to the auto-zero signal, the second holding circuit holds a time code corresponding to the detection signal, and when the first holding circuit holds a time code corresponding to the first digital signal, the second holding circuit holds a time code corresponding to the second digital signal.

[0051] The time code generator can output a time code including a Gray code.

[0052] In another aspect of the present disclosure, an imaging method is provided, including:

[0053] Among a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal, in a case where an absolute value of a change amount of the electric signal exceeds a predetermined threshold value, a detection signal is output from a detection section;

[0054] A predetermined signal processing is performed based on the detection signal;

[0055] An auto-zero signal for initializing the detection section is output;

[0056] When the auto-zero signal is output, a time code output from a time code generator is held in a first holding circuit;

[0057] When the detection signal is output, the time code output from the time code generator is held in a second holding circuit; and

[0058] The time code held in the first holding circuit and the time code held in the second holding circuit are associated and transmitted to a signal processing section. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a block diagram describing a schematic configuration of an imaging system 2 including an imaging device according to a first embodiment.

[0060] Figure 2 is a block diagram depicting a schematic configuration of an imaging device according to the first embodiment.

[0061] Figure 3 is a block diagram describing a detailed configuration of each pixel in a pixel array section.

[0062] Figure 4 is a circuit diagram depicting an example of a detailed configuration of a pixel circuit, a differential input circuit in an ADC, a voltage conversion circuit, and a positive feedback circuit.

[0063] Figure 5 is a block diagram showing an internal configuration of an address event detection section in Figure 3

[0064] ​Figure 6 is a circuit diagram that describes an example of the voltage comparison section.

[0065] Figure 7 is a circuit diagram that shows a configuration example of the initialization control section.

[0066] Figure 8 is a block diagram that shows the internal configuration of a cluster in the time code transfer section according to the first embodiment.

[0067] Figure 9 is a block diagram that further shows the internal configuration of the data holding section and the repeater within each cluster of Figure 8

[0068] Figure 10 is a block diagram that describes an example of the internal configuration of the time code generator.

[0069] Figure 11 is a timing chart that indicates the timing at which the time code transfer section of the first embodiment transfers a time code or the like.

[0070] Figure 12A is a schematic perspective view that shows a first example of a laminate chip.

[0071] Figure 12B is a schematic perspective view that shows a second example of a laminate chip.

[0072] Figure 13 is a block diagram that indicates the internal configuration of a cluster of the time code transfer section of the second embodiment.

[0073] Figure 14 is a timing chart that indicates the timing at which the time code transfer section of the second embodiment transfers a time code or the like.

[0074] Figure 15 is a timing chart in the case where an asynchronous output event detection signal and an auto-zero signal are output.

[0075] Figure 16 is a block diagram that depicts a schematic configuration of an imaging system including a discrimination section.

[0076] Figure 17 is a block diagram that indicates the internal configuration of a cluster of the time code transfer section of the third embodiment and its peripheral circuit.

[0077] Figure 18 is a timing chart that indicates the timing at which the time code transfer section of the third embodiment transfers a time code or the like.

[0078] Figure 19 is a timing chart in the case where a first repeater and a second repeater are provided within each cluster.

[0079] Figure 20 ​is a block diagram of an example of a schematic configuration of a vehicle control system that is an example of a mobile body control system that describes a technology applicable according to the embodiment of the present disclosure.

[0080] Figure 21 is a diagram that describes an example of a mounting position of an imaging section. DETAILED DESCRIPTION

[0081] Hereinafter, an embodiment of an imaging device will be described with reference to the drawings. Although the following will mainly describe main components of the imaging device, the imaging device can have components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0082] (First Embodiment)

[0083] Figure 1 is a block diagram of a schematic configuration of an imaging system 2 including the imaging device 1 according to the first embodiment. In Figure 1 The imaging system 2 described in

[0084] In the imaging system 2 described in Figure 1 , the imaging lens 3 captures incident light from an object and forms an image on an imaging surface of the imaging device 1. The imaging device 1 photoelectrically converts the incident light captured by the imaging lens 3 in units of pixels to obtain imaging data. As described later, at least a part of the imaging device 1 can be implemented by a semiconductor device.

[0085] The imaging device 1 performs predetermined signal processing on the imaging image data and records data on a processing result and a detection signal of an address event, which will be described later (hereinafter, it can be described as a "detection signal" or an "event detection signal"), in the recording section 4. The method of generating the event detection signal will be described later. The control section 5 includes, for example, a microcomputer (CPU: Central Processing Unit), and controls an imaging operation in the imaging device 1. The information processing section 6 performs various types of information processing based on the event detection signal output from the imaging device 1. Note that the information processing section 6 can be integrated within the imaging device 1.

[0086] Figure 2 is a block diagram of a schematic configuration of the imaging device 1 according to the first embodiment. The imaging device 1 is formed on a semiconductor substrate 11, and includes a pixel array section 12, a time code transfer section 13, a pixel drive circuit 14, a digital analog converter (DAC) 15, a time code generator 16, a vertical drive circuit 17, a signal processing section 18, and a controller 20.

[0087] The semiconductor substrate 11 is a substrate formed using, for example, silicon (Si), and a pixel array section 12 in which a plurality of pixels 21 are arranged in a two-dimensional array is formed on the semiconductor substrate 11. Hereinafter, a group of pixels 21 arranged in a horizontal direction X of the pixel array section 12 is referred to as a "row", and a group of pixels 21 arranged in a direction Y perpendicular to the row is referred to as a "column".

[0088] Each of the plurality of pixels 21 arranged in a two-dimensional array generates a charge signal (analog pixel signal) corresponding to an amount of light received by a light-receiving element (hereinafter, it can be referred to as a photodiode) 22 in the pixel 21, converts the analog pixel signal into a digital pixel signal, and outputs the digital pixel signal. As described above, in the imaging device 1 in Figure 1

[0089] The time code generator 16 generates a time code that varies at a predetermined period. The time code generator 16 generates the time code regardless of an image capturing timing of each pixel 21 or a timing at which an event is detected. In the present embodiment, the time code is used to specify the pixel 21 that has output the event detection signal and an auto-zero signal described later. The time code generator 16 supplies the generated time code to the time code transfer section 13. In Figure 1

[0090] The time code transfer section 13 transfers the time code generated by the time code generator 16 to the signal processing section 18, and transfers the time code corresponding to the pixel 21 that has output the auto-zero signal and the time code corresponding to the pixel 21 that has output the event detection signal to the signal processing section 18. As Figure 1 indicated in FIG. 8, in a case where a plurality of time code transfer sections 13 are provided, each time code transfer section 13 transfers the time code corresponding to the auto-zero signal and the time code corresponding to the event detection signal for a plurality of surrounding pixels 21. The internal configuration of the time code transfer section 13 is described later.

[0091] ​​The pixel driving circuit 14 drives the pixel circuit provided in the pixel 21 and an ADC which will be described later. The DAC generates a reference signal (a reference voltage signal) REF, which is a ramp signal whose level (voltage) monotonously decreases with the passage of time, and supplies the reference signal REF to each pixel 21. The reference signal is used to convert an analog pixel signal into a digital pixel signal in each pixel 21.

[0092] The vertical driving circuit 17 drives any one of a plurality of pixel rows provided along the vertical direction Y of the pixel array section 12. The event detection signal and the auto-zero signal of each pixel 21 belonging to the pixel row driven by the vertical driving circuit 17 are transmitted to the corresponding time code transmission section 13.

[0093] The signal processing section 18 receives the time codes from the time code generator 16, the time codes of the event detection signal corresponding to each pixel 21, and the time codes of the auto-zero signal corresponding to each pixel 21, which are transmitted from the plurality of time code transmission sections 13. The signal processing section 18 performs predetermined signal processing based on these time codes.

[0094] The signal processing section 18 can include a storage section that stores each time code transmitted from the plurality of time code transmission sections 13.

[0095] The controller 20 controls each section in the imaging device 1. The controller 20 has a timing generation circuit 20a. The timing generation circuit 20a supplies a signal indicating the operation timing of each section in the imaging device 1 to each section.

[0096] [Configuration Example of Pixel 21]

[0097] Figure 3 is a block diagram that describes the detailed configuration of each pixel 21 in the pixel array section 12. As shown in Figure 3 each pixel 21 includes a light receiving section 32, a pixel circuit 33, an analog-digital converter (hereinafter, referred to as an ADC or a pixel ADC) 34, and an address event detection section 35.

[0098] The light receiving section 32 generates an analog pixel signal corresponding to the amount of received light. For example, the light receiving section 32 includes a photodiode 22.

[0099] The pixel circuit 33 generates an analog pixel signal of a reset level and an analog pixel signal of a luminance signal level corresponding to an exposure amount. The reset level is a voltage at the time when a floating diffusion (FD) is initialized after the start of exposure. The luminance signal level is a voltage corresponding to the exposure amount at the end of exposure. The pixel circuit 33 sequentially supplies the analog pixel signal of the reset level and the analog pixel signal of the luminance signal level to the ADC 34.

[0100] The ADC 34 converts the analog pixel signal into a digital pixel signal at a reset level and a luminance signal level. In the present specification, the digital pixel signal obtained by performing AD conversion on the analog pixel signal at the reset level is referred to as P-phase data, and the digital pixel signal obtained by performing AD conversion on the analog pixel signal at the luminance signal level is referred to as D-phase data.

[0101] The ADC 34 includes a differential input circuit 41, a voltage conversion circuit 42, a positive feedback circuit 43, and a data storage section 44. The differential input circuit 41 compares a reference signal REF generated by the DAC 15 shown in FIG. 1 with the analog pixel signal (reset level or luminance signal level) from the pixel circuit 33, and supplies a signal indicating the comparison result to the voltage conversion circuit 42. The voltage conversion circuit 42 converts the voltage level of the signal indicating the comparison result of the differential input circuit 41, and supplies the converted signal to the positive feedback circuit 43. Figure 2

[0102] The positive feedback circuit 43 generates a signal VCO corresponding to the output signal of the voltage conversion circuit 42 and performs positive feedback on the voltage conversion circuit 42.

[0103] The data storage section 44 stores a digital pixel signal corresponding to the signal VCO. The time code output from the time code generator 16 and transferred by the time code transfer section 13 is input to the data storage section 44. Further, the signal WORD input to the data storage section 44 becomes, for example, a high level at the time of reading the pixel 21. The data storage section 44 holds the time code at the time when the logic of the output signal VCO of the positive feedback circuit 43 is inverted.

[0104] Figure 4 is a circuit diagram depicting an example of the detailed configuration of the pixel circuit 33 and the differential input circuit 41, the voltage conversion circuit 42, and the positive feedback circuit 43 in the ADC 34. The pixel circuit 33 includes a discharge transistor Ql, a transfer transistor Q2, a gain control transistor Q3, a reset transistor Q4, a floating diffusion (hereinafter, referred to as FD) 36, and a capacitor Cl. The transistors Ql to Q4 in the pixel circuit 33 are, for example, NMOS transistors.

[0105] The discharge transistor Ql releases the charge accumulated in the photodiode 22 at the start of exposure based on a drive signal OFG from the pixel drive circuit 14. The transfer transistor Q2 transfers the charge from the photodiode 22 to the FD 36 at the end of exposure based on a transfer signal TX from the pixel drive circuit 14. The FD 36 accumulates the transferred charge and generates a voltage corresponding to the amount of the accumulated charge.

[0106] ​The reset transistor Q4 initializes the FD 36 based on a reset signal RST from the pixel drive circuit 14. The gain control transistor Q3 controls the analog gain of the voltage with respect to the FD 36 based on a control signal FDG from the pixel drive circuit 14. The voltage of the FD 36 is reduced by the analog gain and output, so that the saturated signal amount of the expandable pixel 21 is made. The gain control transistor Q3 and the reset transistor Q4 are connected in series, and one end of the capacitor Cl is connected to the connection node thereof.

[0107] The differential input circuit 41 outputs a signal corresponding to the voltage difference between the voltage corresponding to the electric charge photoelectrically converted by the photodiode 22 and the reference voltage REF. The differential input circuit 41 includes a pair of NMOS transistors Q5 and Q6 constituting a current mirror circuit, an NMOS transistor Q7, a pair of PMOS transistors Q8 and Q9, and a PMOS transistor Q10. The reference signal REF is input to the gate of the transistor Q5, and the voltage signal of the FD 36 is input to the gate of the transistor Q6.

[0108] The transistor Q7 is connected between the sources of the transistors Q5 and Q6 and a ground node. A bias voltage Vb is input to the gate of the transistor Q7. The pair of transistors Q8 and Q9 are connected between a power supply voltage node VDDH and the drains of the pair of transistors Q5 and Q6. The gate of the transistor Q10 is connected to the drains of the transistors Q6 and Q9.

[0109] The voltage conversion circuit 42 includes an NMOS transistor Q11. A power supply voltage VDDL is input to the gate of the transistor Q11, and the drain of the transistor Q11 is connected to the drain of the transistor Q10. The voltage conversion circuit 42 converts the output voltage of the differential input circuit 41 to a lower voltage level.

[0110] The positive feedback circuit 43 includes PMOS transistors Q12 to Q15 and NMOS transistors Q16 to Q18. The source of Q11 and the drain of Q16 are connected, and the source of Q16 is grounded. A signal INI from the pixel drive circuit 14 is input to the gate of the transistor Q16. Each of the gates of the transistors Q14 and Q17 is connected to the source of the transistor Q11. A signal TESTVCO from the pixel drive circuit 14 is input to each of the gates of the transistors Q15 and Q18. A signal VCO is output from the drain of the transistor Q17. A signal INI2 from the pixel drive circuit 14 is input to the gate of the transistor Q12.

[0111] Figure 5 is a block diagram showing the internal configuration of the address event detection section 35 in Figure 3 . As shown in Figure 5 , the address event detection section 35 includes a voltage comparison section 37 and an initialization control section 38.

[0112] The voltage comparison section 37 compares the analog differential signal corresponding to the amount of change in the incident light with a predetermined voltage (upper limit voltage or lower limit voltage) indicating the boundary of a predetermined voltage range, and outputs event detection signals COMP+ and COMP- indicating the comparison result.

[0113] Each time the comparison result COMP is output, the initialization control section 38 supplies an auto-zero signal XAZ to the voltage comparison section 37 to control the differential signal to an initial value. Note that, in the present specification, the signals AZ and XAZ whose logics are inverted from each other are collectively referred to as an auto-zero signal.

[0114] [Configuration example of voltage comparison section 37]

[0115] Figure 6 is a circuit diagram describing an example of the voltage comparison section 37. The voltage comparison section 37 includes a logarithmic response section 45, a buffer 46, a differential circuit 47, and a comparator 48.

[0116] A current-voltage conversion section 49 in the logarithmic response section 45 generates a pixel voltage Vp obtained by logarithmically converting a charge (photo current) generated by the photodiode 22. The current-voltage conversion section 49 includes NMOS transistors Q21 and Q22, a capacitor C2, and a PMOS transistor Q23.

[0117] The source of the transistor Q21 is connected to the cathode of the photodiode 22, and the drain is connected to a power supply voltage node. The transistor Q22 and the transistor Q23 are connected in series between the power supply voltage node and a node of a predetermined reference potential (ground potential or the like). Further, a connection node of the transistor Q22 and the transistor Q23 is connected to the gate of the transistor Q21 and an input node of the buffer 46. A connection node between the drain of the transistor Q21 and the cathode of the photodiode 22 is connected to the gate of the transistor Q23.

[0118] Further, a predetermined bias voltage Vblog is applied to the gate of the transistor Q22. The capacitor C2 is connected between the gate of the transistor Q21 and the gate of the transistor Q23.

[0119] As described later, it is also possible to stack a semiconductor chip in which the photodiode 22 is arranged and a semiconductor chip in which the circuit on the rear stage side of the photodiode 22 is arranged. Alternatively, it is possible to stack a semiconductor chip on which the photodiode 22, the transistors Q21 to Q23, and the capacitor C2 are arranged and a semiconductor chip on which the circuit on the rear stage side thereof is arranged. As described above, in the circuit product illustrated in FIG. 8, the circuit components provided on the same or different semiconductor chips are arbitrary. Figure 6 The circuit components provided on the same or different semiconductor chips are arbitrary in the circuit product illustrated in FIG. 8.

[0120] The buffer 46 outputs the input pixel voltage to the differentiation circuit 47. The buffer 46 can increase the driving force for driving the subsequent stage. Further, the buffer 46 can ensure noise isolation associated with the switching operation in the subsequent stage.

[0121] Further, the buffer 46 includes a transistor Q24 and a transistor Q25. As these transistors, for example, MOS transistors are used.

[0122] In the buffer 46, the transistor Q24 and the transistor Q25 are connected in series between the power supply voltage node and the terminal of the reference potential. Further, the gate of the transistor Q25 is connected to the logarithmic response section 45, and the connection node of the transistor Q24 and the transistor Q25 is connected to the differentiation circuit 47. A predetermined bias voltage Vbsf is applied to the gate of the transistor Q24.

[0123] The differentiation circuit 47 obtains the amount of change of the pixel voltage Vp by a differentiation operation. The amount of change of the pixel voltage Vp indicates the amount of change of the light quantity. The differentiation circuit 47 supplies a differentiation signal Vout indicating the amount of change of the light quantity to the comparator 48.

[0124] Further, the differentiation circuit 47 includes capacitors C3 and C4, a transistor Q26 and a transistor Q27, and a transistor Q28.

[0125] The transistor Q27 and the transistor Q28 are connected in series between the power supply voltage node and the reference potential node. A predetermined bias voltage Vbdiff is input to the gate of the transistor Q28. These transistors Q27 and Q28 function as an inverter circuit in which the gate of the transistor Q27 is set as an input node nl and the connection node of the transistor Q27 and the transistor Q28 is set as an output node n2.

[0126] The capacitor C3 is inserted between the buffer 46 and the gate of the transistor Q27. The capacitor C3 supplies a current corresponding to the time derivative (in other words, the amount of change) of the pixel voltage Vp from the buffer 46 to the gate of the transistor Q27. Further, the capacitor C4 is inserted between the gate of the transistor Q27 and the output node n2.

[0127] The transistor Q26 opens and closes the path between the input node nl and the output node n2 in accordance with an auto-zero signal XAZ from the initialization control section 38. For example, every time the count value is calculated, the initialization control section 38 sets the auto-zero signal XAZ from high level to low level and instructs initialization. Then, the transistor Q26 is switched to the on state in accordance with the auto-zero signal XAZ, and the differentiation signal Vout is set to an initial value.

[0128] The comparator 48 compares the differentiated signal Vout with a predetermined voltage (upper limit voltage or lower limit voltage) that represents a boundary of a certain voltage range. The comparator 48 includes transistors Q29 and Q31, and transistors Q30 and Q32. As these transistors, for example, MOS transistors are used.

[0129] In the comparator 48, the transistors Q29 and Q30 are connected in series between the power supply voltage node and the reference voltage node, and the transistors Q31 and Q32 are also connected in series between the power supply voltage node and the reference voltage node. Further, the gates of the transistors Q29 and Q31 are connected to the differentiated circuit 47. The upper limit voltage Vhigh is applied to the gate of the transistor Q30, and the lower limit voltage Vlow is applied to the gate of the transistor Q32.

[0130] The voltage of the connection node of the transistors Q29 and Q30 is output as a comparison result COMP+ with the upper limit voltage. The voltage of the connection node of the transistors Q31 and Q32 is output as a comparison result COMP- with the lower limit voltage. With this connection, the comparator 48 outputs a high-level comparison result COMP+ when the differentiated signal Vout is higher than the upper limit voltage Vhigh, and outputs a low-level comparison result COMP- when the differentiated signal Vout is lower than the lower limit voltage Vlow. The event detection signal COMP is a signal that includes the comparison results COMP+ and COMP-.

[0131] Note that although the comparator 48 compares both the upper limit voltage and the lower limit voltage with the differentiated signal Vout, only one of the upper limit voltage and the lower limit voltage can be compared with the differentiated signal Vout. Thus, the internal configuration of the comparator 48 can be simplified. For example, only the transistors Q29 and Q30 are needed compared with the upper limit voltage, and the transistors Q31 and Q32 can be omitted.

[0132] Figure 7 is a circuit diagram that shows a configuration example of the initialization control section 38. The initialization control section 38 includes delay sections 51 and 52, and exclusive OR (XOR) gates 53 and 54.

[0133] The delay section 51 delays the comparison result COMP+ from the comparator 48. The delay section 51 supplies a delayed signal to the XOR gate 53. The delay section 52 delays the comparison result COMP- from the comparator 48. The delay section 51 supplies a delayed signal to the XOR gate 54.

[0134] XOR gate 53 generates the XOR of the comparison result COMP+ before and after the delay. XOR gate 54 generates the XOR of the comparison result COMP- before and after the delay. XOR gates 53 and 54 generate a pulse signal. This pulse signal is output to the differentiating circuit 47 as an auto-zero signal XAZ.

[0135] [Configuration of Timecode Transmission Unit 13]

[0136] like Figure 2 As shown, for example, multiple timecode transmission units 13 are provided in the horizontal direction of the pixel array unit 12. The timecode generator 16 is connected to one end of the timecode transmission unit 13, and the signal processing unit 18 is connected to the other end. The timecode transmission unit 13, as described later, is composed of multiple clusters connected in series. Each cluster holds timecodes for predetermined columns of pixels 21 in multiple rows. Furthermore, each cluster sequentially transmits timecodes from the timecode generator 16.

[0137] As described above, each cluster of the timecode transmission unit 13 sequentially transmits the timecode from the timecode generator 16, the timecode corresponding to the pixel 21 that outputs the auto-zero signal, and the timecode corresponding to the pixel 21 that outputs the event detection signal. Furthermore, each cluster in the timecode transmission unit 13 transmits specific data from pixels 21 that have not yet output event detection signals and auto-zero signals.

[0138] Figure 8 This is a block diagram illustrating the internal configuration of cluster 61 in the timecode transmission unit 13 of the first embodiment. For example... Figure 8 As shown, each cluster 61 in the timecode transmission unit 13 includes a data holding unit 62 and a repeater 63. The data holding unit 62 holds the timecode corresponding to the pixel 21 that outputs the auto-zero signal, and also holds the timecode corresponding to the pixel 21 that outputs the event detection signal.

[0139] A data holding unit 62 can sequentially store the timecodes of event detection signals and auto-zero signals for multiple pixels for each pixel. The data holding unit 62 holds specific data (e.g., zero) for pixels 21 that have not yet output an event detection signal or auto-zero signal. As described above, the data holding unit 62 only holds timestamps for pixels 21 that have already output an event detection signal or auto-zero signal, and holds specific data for other pixels 21. Therefore, the timecode transmission unit 13 transmits the timecodes or specific data of all pixels in the order of pixel arrangement, and the signal processing unit 18 that receives data from the timecode transmission unit 13 can easily specify which pixel 21 outputs an event detection signal or auto-zero signal.

[0140] Note that the data holding section 62 can hold not only the time code corresponding to the pixel 21 for which the event detection signal has been output, but also polarity information indicating whether the luminance signal tends to increase or decrease. Further, the data holding section 62 can hold the address information of the pixel 21 for which the event detection signal has been output together.

[0141] As described above, the time code transfer section 13 includes clusters 61 connected in series in a plurality of stages. The cluster 61 of the first stage holds the time code output from the time code generator 16, and holds the time code corresponding to the auto-zero signal and the time code corresponding to the event detection signal held in the corresponding data holding section 62. The clusters 61 in the second and subsequent stages transfer the time code sequentially generated from the time code generator 16 held in the cluster 61 in the preceding stage to the cluster 61 in the subsequent stage, transfer the time code corresponding to the auto-zero signal and the time code corresponding to the event detection signal held in the corresponding data holding section 62 to the cluster 61 in the subsequent stage, and transfer the data held by the cluster 61 in the preceding stage to the cluster 61 in the subsequent stage.

[0142] Figure 9 is a block diagram of the internal configuration of the data holding section 62 and the repeater 63 in each cluster 61 further realizing Figure 8 As shown in Figure 9 , each data holding section 62 includes a first data holding section (first holding circuit) 64 holding the time code corresponding to the auto-zero signal and a second data holding section (second holding circuit) 65 holding the time code corresponding to the event detection signal. For example, each of the first data holding section 64 and the second data holding section 65 can hold the time code of 128 pixels (32 pixels in the horizontal direction x 4 pixels in the vertical direction). Note that the number of pixels to be held is arbitrary. The first data holding section 64 and the second data holding section 65 hold specific data (for example, zero) of the pixel 21 for which the event detection signal or the auto-zero signal has not been output.

[0143] As shown in Figure 9As shown, each of the relays 63 includes a bidirectional buffer 66 and a D-type flip-flop (hereinafter referred to as a D-F / F) 67. The D-F / F 67 holds data transmitted from the relay 63 in the previous stage at the rising timing of the clock signal AD-CLK. The data held by the D-F / F 67 includes the time code from the time code generator 16. The time code is input to the first data holding section 64 and the second data holding section 65 via the bidirectional buffer 66. The first data holding section 64 holds the time code synchronized with the clock signal AD-CLK when the event detection signal is input. For the pixel 21 for which the event detection signal is not input, a certain data is held synchronized with the clock signal AD-CLK. Similarly, the second data holding section 65 holds the time code synchronized with the clock signal AD-CLK when the auto-zero signal is input.

[0144] Figure 9 The data holding section 62 in the time code generator 16 holds, for example, 15-bit time code data. Similarly, the relay 63 sequentially transfers, for example, 15-bit time code data.

[0145] Figure 10 is a block diagram that describes an example of the internal configuration of the time code generator 16. Figure 10 The time code generator 16 in the time code generator 16 includes a binary counter 68 and a binary / gray scale converter 69. The binary counter 68 is a counter that counts binary numbers, and has a configuration in which a plurality of binary frequency dividers are connected in series. The binary frequency dividers are configured by feeding back the xQ output of the D-F / F 68a to the D input. The period of the output signal of each binary frequency divider differs by a power of 2.

[0146] The binary / gray scale converter 69 converts the binary count value from the binary counter 68 to a Gray code. The Gray code changes only one bit between adjacent codes, and can reduce power consumption by minimizing the number of bit changes between codes. As described above, by setting the time code output from the time code generator 16 to a Gray code, it is possible to suppress the power consumption when the time code transfer section 13 sequentially transfers the time code.

[0147] Note that the time code output from the time code generator 16 is not necessarily a Gray code. For example, Figure 10 The output value of the binary counter 68 in the time code generator 16 can be used as is as the time code. In this case, the binary / gray scale converter 69 can be omitted, but in the case of using a gray scale code, the power consumption at the time of time code transfer increases.

[0148] Figure 11 is a timing chart that shows the timing at which the time code transfer section 13 of the first embodiment relays the time code and the like. Figure 11 The two timing charts on the upper side in the figure describe the vertical synchronization signal XVS of the pixel array section 12 and the operation period of the imaging device 1 in each frame. Furthermore,Figure 11 The lower four timing charts in FIG. 6 describe the generation timing of the clock signal AD-CLK, the time code from the time code generator 16 transmitted by the repeater 63, the auto-zero signal AZ, and the event detection signal in one frame period.

[0149] As shown in the upper two timing charts in FIG. 6, for each frame, there is a period (time tl to t2) in which the time code corresponding to the pixel 21 that has output the auto-zero signal AZ is held in the first data holding section 64, a period (time t2 to t3) in which the time code corresponding to the pixel 21 that has output the event detection signal is held in the second data holding section 65, and a period (time t3 to t4) in which the time codes held in the first data holding section 64 and the second data holding section 65 are transmitted via the time code transmission section 13. Figure 11

[0150] In the example of FIG. 6, the auto-zero signal is output at time t2, and the event detection signal is output at time t21 between times t2 and t3. Figure 11 Between times tl to t3, the clusters 61 of the time code transmission section 13 sequentially transmit the time code from the time code generator 16. Further, the first data holding section 64 in the data holding section 62 holds the time code corresponding to the pixel 21 that has output the auto-zero signal at time t2. Similarly, at time t21, the second data holding section 65 holds the time code corresponding to the pixel 21 that has output the event detection signal.

[0151] From time t3 to t4, the clusters 61 in the time code transmission section 13 respectively transmit the corresponding time codes of the pixel 21 that has output the auto-zero signal of itself and the pixel 21 that has output the event detection signal, and the other pixels transmit specific data.

[0152] More specifically, in the present embodiment, for each frame, all of the pixels 21 in the pixel array section 12 are scanned to detect the pixel 21 that has output the auto-zero signal or the event detection signal, and the first data holding section 64 or the second data holding section 65 holds the time code corresponding to the pixel 21 that has output the auto-zero signal or the event detection signal.

[0153] The auto-zero signal and the event detection signal output from any of the pixels 21 in the pixel array section 12 are input to

[0154] and Figure 8 Figure 9 ​​The corresponding data holding section 62 shown in FIG. 6B is described. The first data holding section 64 in the data holding section 62 holds a time code corresponding to the pixel 21 that has output the auto-zero signal, and the second data holding section 65 holds a time code corresponding to the pixel 21 that has output the event detection signal. The time code is held in synchronization with the clock signal AD-CLK. The clock signal AD-CLK is a signal with which the time code generator 16 updates the time code. Figure 11 An example in which the time code corresponding to the auto-zero signal is held at time t2 is described. In the present embodiment, all of the pixels 21 output the auto-zero signal at the same timing for each frame, but since the time until the time code from the time data generator is transferred to the plurality of data holding sections 62 is different, each data holding section 62 holds the time code corresponding to the auto-zero signal at different timings. As described above, since the time codes are sequentially transferred among the clusters 61, each data holding section 62 holds the time code corresponding to the auto-zero signal at different timings. Therefore, the value of the time code corresponding to the auto-zero signal becomes a different value for each data holding section 62.

[0155] Figure 11 An example in which the event detection signal is output at time t21 between times t2 and t3 is described, and the first data holding section 64 holds a time code corresponding to the event detection signal in synchronization with the clock signal AD-CLK. As described above, the data holding section 62 holds the time code corresponding to the pixel 21 that has output the auto-zero signal and the time code corresponding to the pixel 21 that has output the event detection signal. These time codes are transferred to the signal processing section 18 by the time code transfer section 13. The signal processing section 18 can determine in which pixel 21 the event has occurred by taking the difference between the time code corresponding to the pixel 21 that has output the event detection signal and the time code corresponding to the pixel 21 that has output the auto-zero signal.

[0156] Note that, for the pixel 21 in which no event has occurred, the time code transfer section 13 transfers a specific data (for example, zero) in synchronization with the clock signal AD-CLK, instead of the time code. Therefore, the signal processing section 18 can more accurately detect in which pixel 21 the event has occurred by extracting the time code contained in the specific data and counting the number of specific data between the time codes.

[0157] The imaging device 1 according to the present embodiment can include, for example, a plurality of stacked semiconductor chips (hereinafter, referred to as a laminate chip) 70. Figure 12A is a schematic perspective view showing a first example of the laminate chip 70. Figure 12AIt includes a first semiconductor chip 71 and a second semiconductor chip 72 stacked below the first semiconductor chip 71. For example, a plurality of pixels 21 in the pixel array section 12 are arranged on the first semiconductor chip 71. A part that receives light and performs photoelectric conversion is provided at least on the first semiconductor chip 71. On the second semiconductor chip 72, a data holding section 62, a time code generator 16, a time code transmission section 13, a signal processing section 18, etc. are arranged.

[0158] The first semiconductor chip 71 and the second semiconductor chip 72 transmit signals to each other via connection portions such as vias (VIA), Cu-Cu junctions, bumps, etc.

[0159] Figure 12B This is a schematic perspective view showing a second instance of the laminated chip 70. Figure 12B The laminated chip 70 includes a first semiconductor chip 71, a second semiconductor chip 72 stacked below the first semiconductor chip 71, and a third semiconductor chip 73 stacked below the second semiconductor chip 72. Figure 12B In the first semiconductor chip 71, for example, a plurality of pixels 21 are provided in the pixel array section 12. In the second semiconductor chip 72, a timecode generator 16, a timecode transmission section 13, a signal processing section 18, etc. are provided. In the third semiconductor chip 73, a semiconductor memory such as a data holding section 62 is mainly arranged.

[0160] It should be noted that Figure 12A and Figure 12B This is merely a representative example of laminated chip 70, and various modifications are conceivable. Furthermore, in Figure 12A and Figure 12B The first semiconductor chip 71 and the second semiconductor chip 72 in Figure 12B The arrangement of any circuit on the third semiconductor chip 73 is arbitrary, and various variations are conceivable. Furthermore, a laminated chip 70 with four or more layers can be configured, or a laminated chip 70 with multiple layers in which at least one layer of the laminated chip 70 is divided into the same layer can be configured.

[0161] As described above, in the first embodiment, since multiple clusters 61 are used in the time code transmission unit 13 to transmit the time code corresponding to the auto-zeroing signal output from any pixel 21 in the pixel array unit 12 and the time code corresponding to the event detection signal, the signal processing unit 18 can easily and quickly detect which pixel 21 in the pixel array unit 12 has occurred.

[0162] The time code transmission section 13 is used to transmit the time code corresponding to the luminance value in the case where the AD conversion is performed for each pixel 21. According to the present embodiment, the time code transmission section 13 provided to perform the AD conversion for each pixel 21 can be diverted to transmit the time code corresponding to the auto-zero signal and the time code corresponding to the event detection signal. Thus, the event can be detected without complicating the internal configuration of the imaging device 1.

[0163] Further, in the case where the time code is transmitted to the signal processing section 18 using the time code transmission section 13, the plurality of data holding sections 62 provided along the transmission direction of the time code hold the timing of the time code differently. Thus, by holding the time code corresponding to the pixel 21 outputting the auto-zero and taking the difference from the time code corresponding to the pixel 21 outputting the event detection signal, the problem of the timing of holding the time code being offset for each data holding section 62 can be solved.

[0164] (Second Embodiment)

[0165] In the second embodiment, two columns of clusters 61 are provided in the time code transmission section 13.

[0166] Figure 13 is a block diagram showing the internal configuration of the cluster 61 in the time code transmission section 13 of the second embodiment. Figure 13 The time code transmission section 13 in is composed of a first relay 81 in the plurality of stages and a second relay 82 in the plurality of stages. The time code from the time code generator 16 is input to the first relay 81 in the first stage among the first relays 81 in the plurality of stages. The first relay 81 in the final stage and the second relay 82 in the final stage are connected to the signal processing section 18.

[0167] Each first relay 81 in the plurality of stages is associated with one of the second relays 82 in the plurality of stages. More specifically, each first relay 81 in the plurality of stages transmits the time code to the corresponding second relay 82. For example, the first relay 81 in the plurality of stages performs the process of sequentially transmitting the time code from the time code generator 16 and transmitting the time code to the signal processing section 18, and transmits the time code corresponding to the auto-zero signal received from the data holding section 62 and the time code corresponding to the event detection signal to the corresponding second relay 82. The second relay 82 in the plurality of stages sequentially transmits the time code corresponding to the auto-zero signal and the time code corresponding to the event detection signal transmitted from the first relay 81 in the plurality of stages, and transmits the time code to the signal processing section 18.

[0168] Figure 14 is a timing chart showing the timing of diverting the time code and the like by the time code transmission section 13 of the second embodiment. As shown in Figure 14As shown, all of the pixels 21 in the pixel array section 12 output the auto-zero signal and hold the time codes in the first data holding section 64 (time t2 to t3), after which event detection is performed on all of the pixels 21, and the time codes of the pixels 21 that have output the event detection signal are held in the second data holding section 65 (aging t3 to t4). The above-described operation is performed in one frame period from time t1 to time t4. In the next frame period, the time codes held in the first data holding section 64 and the time codes held in the second data holding section 65 are transferred from the first repeater 81 to the second repeater 82 between times t4 and t5, and are transferred among the plurality of second repeaters 82 and read by the signal processing section 18 between times t5 and t7. Further, from time t5 to time t6, all of the pixels 21 in the pixel array section 12 again output the auto-zero signal and hold the time codes in the first data holding section 64, after which the event detection process is performed, and the time codes corresponding to the pixels 21 that have output the event detection signal are held in the second data holding section 65 (time t6 to t8).

[0169] By comparing Figure 14 and Figure 11 it can be seen that, in Figure 14 , the output of the auto-zero signal and the event detection can be performed on one frame, and the transfer process to the signal processing section 18 is performed in the next frame, so that each process can be performed using spare time. That is, in the second embodiment, in addition to the plurality of first repeaters 81 that transfer the time codes from the time code generator 16, the plurality of second repeaters 82 that transfer the time codes corresponding to the auto-zero signal and the event detection signal are provided, so that the dead time in which the event detection cannot be performed can be shortened.

[0170] Note that each of the pixels 21 in the pixel array section 12 does not necessarily need to output the auto-zero signal at the same timing, and for the pixels 21 in which an event has occurred, the auto-zero signal can be output when the event detection signal is held in the data holding section 62, and the event detection can be started when the auto-zero signal is held in the data holding section 62. In this case, each of the pixels 21 in the pixel array section 12 outputs the event detection signal and the auto-zero signal asynchronously.

[0171] Figure 15 is a timing chart in the case where the event detection signal and the auto-zero signal are output asynchronously. In the case of Figure 15 , the auto-zero signal is output when the time code corresponding to the event detection signal is held in the data holding section 62, and the event detection is started when the time code corresponding to the auto-zero signal is held in the data holding section 62. Therefore, the length of the event detection period varies depending on the event generation timing.

[0172] In Figure 15 the case, although the effect of detecting an event asynchronously with the frame period can be expected, a dead zone occurs in which an event cannot be detected after the time code is held in the data holding section 62 until the time code corresponding to the event detection signal is transmitted from the first repeater 81 to the second repeater 82.

[0173] Since a plurality of events can occur simultaneously in the plurality of pixels 21, a discrimination section 23 can be provided as shown in Figure 16 The discrimination section 23 discriminates a request for an event occurrence from each of the plurality of pixels 21 and transmits a response based on the discrimination result to the pixel 21. The pixel 21 that has received the response from the discrimination section 23 transmits an event detection signal to the corresponding data holding section 62.

[0174] As described above, in the second embodiment, the first repeater 81 in the plurality of stages and the second repeater 82 in the plurality of stages are provided in the time code transmitting section 13, the time code from the time code generator 16 is transmitted in the first repeater 81 in the plurality of stages, and the time code corresponding to the auto-zero signal and the event detection signal is transmitted in the second repeater 82 in the plurality of stages. Therefore, the time code and the event detection corresponding to the auto-zero signal and the event detection signal can be simultaneously and in parallel executed, and the time of the dead zone in which the event detection cannot be executed can be shortened.

[0175] (Third Embodiment)

[0176] In the third embodiment, the processing of transmitting the time code corresponding to the digital pixel 21 data subjected to the AD conversion for each pixel 21 to the signal processing section 18 by the time code transmitting section 13 and the processing of transmitting the time code corresponding to the auto-zero signal and the event detection signal to the signal processing section 18 by the time code transmitting section 13 can be switched and executed.

[0177] Figure 17 is a block diagram showing the internal configuration of the cluster 61 and its peripheral circuit in the time code transmitting section 13 of the third embodiment. In Figure 17 the pixel 21 in outputs the P-phase data and the D-phase data AD-converted by the ADC 34 in the pixel 21. The P-phase data is a digital pixel signal obtained by performing AD conversion on a reset level of an analog pixel signal. The D-phase data is a digital pixel signal obtained by performing AD conversion on a luminance signal level optoelectronically converted by the photodiode 22. In this case, the P-phase data and the D-phase data are Figure 4 the output signal VCO of the positive feedback circuit 43 in Figure 3 In the present embodiment, the data holding section 62 is provided instead of the data storage section 44 in

[0178] AsFigure 17 As shown in FIG. 6, a first selector (MUX) 83 and a second selector (MUX) 84 are connected to each cluster 61. The first selector 83 selects either the P-phase data or the auto-zero signal, and supplies the selected data to the first data holding section 64 in the data holding section 62. The second selector 84 selects one of the D-phase data and the event detection signal, and supplies the selected data to the second data holding section 65 in the data holding section 62.

[0179] In a case where the luminance signal of each pixel 21 in the pixel array section 12 is transferred to the signal processing section 18, the first selector 83 and the second selector 84 select the P-phase data and the D-phase data, respectively, and supply the P-phase data and the D-phase data to the first data holding section 64 and the second data holding section 65, respectively. The first data holding section 64 holds the time code corresponding to the P-phase data, and the second data holding section 65 holds the time code corresponding to the D-phase data.

[0180] Further, in a case where the event detection result is transferred to the signal processing section 18, the first selector 83 and the second selector 84 select either the auto-zero signal or the event detection signal, and supply the auto-zero signal or the event detection signal to the first data holding section 64 or the second data holding section 65. The first data holding section 64 holds the time code corresponding to the auto-zero signal, and the second data holding section 65 holds the time code corresponding to the event detection signal.

[0181] Figure 18 is a timing chart that describes the timing at which the time code transfer section 13 according to the third embodiment transfers the time codes and the like. In Figure 18 In FIG. 6, the time tl to the time t6 is one frame period. The holding and transfer of the time codes corresponding to the auto-zero signal and the event detection signal and the holding and transfer of the time codes corresponding to the luminance signal of each pixel 21 are performed within one frame period. More specifically, from the time tl to the time t2, the data holding section 62 holds the time code corresponding to the auto-zero signal. From the time t2 to the time t3, the data holding section 62 holds the time code corresponding to the event detection signal. From the time t3 to the time t4, the time code transfer section 13 transfers the time codes corresponding to the auto-zero signal and the event detection signal held in the data holding section 62, and exposes each pixel 21 in the pixel array section 12. From the time t4 to the time t5, the data holding section 62 holds the time codes corresponding to the P-phase data and the D-phase data of each pixel 21 in the pixel array section 12. From the time t5 to the time t6, the time code transfer section 13 transfers the time codes corresponding to the luminance signal of each pixel 21 held in the data holding section 62.

[0182] In Figure 17In the time code transfer section 13, one repeater 63 is provided in each cluster 61, but a first repeater 81 and a second repeater 82 can be similarly provided in each cluster 61. Figure 15 In this case, when the first repeater 81 in the multistage transfers the time code from the time code generator 16, the second repeater 82 in the multistage transfers the time code corresponding to the auto-zero signal, the time code corresponding to the event detection signal, and the time code corresponding to the luminance signal (P-phase data and D-phase data) held in each data holding section 62.

[0183] Figure 19 A timing chart for the case where the first repeater 81 and the second repeater 82 are provided in each cluster 61. The time t1 to t6 and the time t6 to t12 are each one frame period. From the time t1 to the time t2, the data holding section 62 holds the time code corresponding to the auto-zero signal. From the time t2 to the time t4, the data holding section 62 holds the time code corresponding to the event detection signal. Each pixel 21 in the pixel array section 12 is exposed from the time t3 to the time t4 during this processing. From the time t4 to the time t5, the time code transfer section 13 transfers the time code corresponding to the auto-zero signal and the event detection signal held in the data holding section 62. During this processing from the time t4 to the time t6, the data holding section 62 holds the time code corresponding to the P-phase data and the D-phase data of each pixel 21 in the pixel array section 12. From the time t6 to the time t11, the time code corresponding to the auto-zero signal and the event detection signal is held and transferred similarly to from the time t1 to the time t5. During this processing, from the time t6 to the time t8, the time code transfer section 13 transfers the time code corresponding to the P-phase data and the D-phase data held by the data holding section 62.

[0184] From Figure 18 With Figure 19 As can be seen from the comparison between

[0185] As described above, in the third embodiment, not only the event detection result but also the time code corresponding to the luminance signal of each pixel 21 can be relayed in the time code transfer section 13 using the plurality of clusters 61. Therefore, the time code transfer section 13 can be effectively utilized. In addition, by providing the plurality of clusters 61 in the time code transfer section 13, the relay efficiency can be improved.

[0186] <Applications of the Technology According to the Present Disclosure>

[0187] The technology according to the present disclosure can be applied to various products. In the following, more specific application examples will be described. For example, the technology according to the present disclosure can be implemented as a distance measurement device mounted on any type of mobile body such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, a construction machine, an agricultural machine (tractor), and the like.

[0188] [mobile body]

[0189] Figure 20 is a block diagram depicting an example of a schematic configuration of a vehicle control system 7000 that is an example of a mobile body control system to which the technology according to the embodiments of the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected to each other via a communication network 7010. In the example shown in the figure, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an on-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 that connects the plurality of control units can be, for example, an in-vehicle communication network conforming to any standard such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), or FlexRay (registered trademark), and the like. Figure 20

[0190] Each control unit includes a microcomputer that performs arithmetic operation processing according to various programs, a storage section that stores programs executed by the microcomputer, parameters and the like used for various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via the communication network 7010, and a communication I / F for communicating with devices, sensors, and the like inside and outside the vehicle by wired or wireless communication. In the example shown in the figure, as a functional configuration of the integrated control unit 7600, a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning section 7640, a beacon reception section 7650, a vehicle-mounted device I / F 7660, a sound / image output section 7670, a vehicle-mounted network I / F 7680, and a storage section 7690 are described. Other control units similarly include a microcomputer, a communication I / F, a storage section, and the like. Figure 20

[0191] ​​The drive system control unit 7100 controls the operation of devices related to the drive system of the vehicle in accordance with various programs. For example, the drive system control unit 7100 functions as a control device for a drive force generation device such as an internal combustion engine, a drive motor, or the like that generates a drive force of the vehicle, a drive force transmission mechanism that transmits the drive force to the wheels, a steering mechanism that adjusts the steering angle of the vehicle, a brake device that generates a braking force of the vehicle, and the like. The drive system control unit 7100 can have a function as a control device such as an anti-lock brake system (ABS), an electronic stability control (ESC), or the like.

[0192] The vehicle state detection portion 7110 is connected to the drive system control unit 7100. The vehicle state detection portion 7110 includes, for example, at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor that detects the operation amount of the accelerator pedal, the operation amount of the brake pedal, the steering angle of the steering wheel, the engine speed, the wheel speed, and the like. The drive system control unit 7100 performs arithmetic processing using signals input from the vehicle state detection portion 7110 and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, and the like.

[0193] The body system control unit 7200 controls the operation of various devices provided to the vehicle body in accordance with various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, and the like. In this case, radio waves emitted from a mobile device can be input to the body system control unit 7200 as a substitute for the signals of the buttons or various switches. The body system control unit 7200 receives these input radio waves or signals and controls the door lock device, the power window device, the lamps, and the like of the vehicle.

[0194] The battery control unit 7300 controls the secondary battery 7310 that is a power supply source of the drive motor in accordance with various programs. For example, information such as the battery temperature, the battery output voltage, the remaining capacity of the battery, and the like is input from a battery device including the secondary battery 7310 to the battery control unit 7300 and the like. The battery control unit 7300 performs arithmetic processing using these signals and performs temperature adjustment control of the secondary battery 7310 or control of a cooling device and the like included in the battery device.

[0195] The outside-vehicle information detecting unit 7400 detects information outside a vehicle including the vehicle control system 7000. For example, at least one of an imaging section 7410 or an outside-vehicle information detecting section 7420 is connected to the outside-vehicle information detecting unit 7400. The imaging section 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, or another camera. For example, the outside-vehicle information detecting section 7420 includes at least one of an environmental sensor for detecting the current weather or climate or a surrounding information detecting sensor for detecting another vehicle, an obstacle, a pedestrian, or the like around the vehicle in which the vehicle control system 7000 is installed.

[0196] The environmental sensor can be at least one of a raindrop sensor that detects rainy weather, a fog sensor that detects fog, a sunlight sensor that detects the degree of sunlight, or a snow sensor that detects snowfall. The surrounding information detecting sensor can be at least one of an ultrasonic sensor, a radar device, or a light detection and ranging, laser imaging detection and ranging (LIDAR) device. The imaging section 7410 and the outside-vehicle information detecting section 7420 can be provided as independent sensors or devices or can be provided as a device in which a plurality of sensors or devices are integrated.

[0197] Here, Figure 21 Examples of the installation positions of the imaging section 7410 and the outside-vehicle information detecting section 7420 are described. The imaging sections 7910, 7912, 7914, 7916, and 7918 are arranged, for example, at at least one position on the front nose, the side mirror, the rear bumper, and the rear door of the vehicle 7900, and at a position on the upper portion of the vehicle interior windshield. The imaging section 7910 provided at the front nose portion of the vehicle interior and the imaging section 7918 provided at the upper portion of the windshield mainly obtain images of the front of the vehicle 7900. The imaging sections 7912 and 7914 provided at the side mirrors mainly obtain images of the sides of the vehicle 7900. The imaging section 7916 provided at the rear bumper or the rear door mainly obtains images of the rear of the vehicle 7900. The imaging section 7918 provided at the upper portion of the windshield within the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.

[0198] Note that, Figure 21 One example of the imaging ranges of the respective imaging sections 7910, 7912, 7914, and 7916 is described. The imaging range a indicates the imaging range of the imaging section 7910 provided to the front nose. The imaging ranges b and c indicate the imaging ranges of the imaging sections 7912 and 7914 provided to the side mirrors, respectively. The imaging range d indicates the imaging range of the imaging section 7916 provided to the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 7900 viewed from above is obtained by superimposing the image data imaged by the imaging sections 7910, 7912, 7914, and 7916.

[0199] The outside information detecting sections 7920, 7922, 7924, 7926, 7928, 7930 provided at the front, rear, side, corner, and upper portion of the windshield in the cabin of the vehicle 7900 can also be ultrasonic sensors or radar devices, for example. The outside information detecting sections 7920, 7926, 7930 provided at the front nose, rear bumper, rear door, and upper portion of the windshield in the vehicle interior of the vehicle 7900 can also be LIDAR devices, for example. These outside information detecting sections 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, and the like.

[0200] Returning to Figure 20 will be continued. The outside information detecting unit 7400 causes the imaging sections 7410 to take images of the outside of the vehicle, and receives the taken image data. In addition, the outside information detecting unit 7400 receives detection information from the connected outside information detecting sections 7420. In the case where the outside information detecting sections 7420 are ultrasonic sensors, radar devices, or LIDAR devices, the outside information detecting unit 7400 transmits ultrasonic waves, electromagnetic waves, or the like, and receives information of the received reflected waves. The outside information detecting unit 7400 can perform processing of detecting objects such as humans, vehicles, obstacles, signs, and characters on the road surface, or processing of detecting distances to the objects, based on the received information. In addition, the outside information detecting unit 7400 can also perform environmental recognition processing of recognizing rain, fog, road surface conditions, and the like, based on the received information. The outside information detecting unit 7400 can calculate distances to objects outside the vehicle based on the received information.

[0201] In addition, the outside information detecting unit 7400 can perform image recognition processing of recognizing humans, vehicles, obstacles, signs, characters on the road surface, or the like, or processing of detecting distances thereto, based on the received image data. The outside information detecting unit 7400 can perform processing such as distortion correction, alignment, and the like, on the received image data, and combine image data imaged by different imaging sections 7410 to generate an aerial view image or a panoramic image. The outside information detecting unit 7400 can perform viewpoint conversion processing using image data imaged by different imaging sections 7410.

[0202] The in-vehicle information detecting unit 7500 detects information on the inside of the vehicle. For example, the in-vehicle information detecting unit 7500 is connected with a driver state detecting portion 7510 that detects the state of the driver. The driver state detecting portion 7510 can include a camera that images the driver, a biometric sensor that detects biometric information of the driver, a microphone that collects sound in the inside of the vehicle, and the like. For example, the biometric sensor is provided on the surface of a seat, a steering wheel, or the like, and detects biometric information of a passenger seated on the seat or a driver holding the steering wheel. Based on the detection information input from the driver state detecting portion 7510, the in-vehicle information detecting unit 7500 can calculate the degree of fatigue of the driver or the degree of concentration of the driver, or can determine whether the driver is dozing off. The in-vehicle information detecting unit 7500 can perform processing such as noise canceling processing on a collected sound signal or the like.

[0203] The integrated control unit 7600 controls the overall operation in the vehicle control system 7000 in accordance with various programs. An input portion 7800 is connected with the integrated control unit 7600. The input portion 7800 is realized by, for example, a device capable of being operated and input by an occupant, such as a touch panel, a button, a microphone, a switch, a lever, or the like. Data obtained by performing voice recognition on a sound input by the microphone can be input to the integrated control unit 7600. The input portion 7800 can be, for example, a remote control device using infrared rays or other radio waves, or an external connection device corresponding to the operation of the vehicle control system 7000, such as a mobile phone, a personal digital assistant (PDA), or the like. The input portion 7800 can be, for example, a camera, and in this case, the occupant can input information by a gesture. Alternatively, data obtained by detecting the motion of a wearable device worn by the occupant can be input. Further, the input portion 7800 can include, for example, an input control circuit or the like that generates an input signal based on information input by the occupant or the like using the above-described input portion 7800, and outputs the input signal to the integrated control unit 7600. By operating the input portion 7800, the occupant or the like inputs various data to the vehicle control system 7000, or instructs processing operation.

[0204] The storage portion 7690 can include a read only memory (ROM) that stores various programs executed by the microcomputer, and a random access memory (RAM) that stores various parameters, calculation results, sensor values, and the like. Further, the storage portion 7690 can be realized by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0205] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices existing in the external environment 7750. The general-purpose communication I / F 7620 can implement a cellular communication protocol such as Global System for Mobile Communications (GSM) (registered trademark), WiMAX, Long Term Evolution (LTE), LTE-Advanced (LTE-A), or another wireless communication protocol such as wireless LAN (also referred to as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication I / F 7620 can be connected to a device (for example, an application server or a control server) existing on an external network (for example, the Internet, a cloud network, or a company-specific network) via, for example, a base station or an access point. Furthermore, the general-purpose communication I / F 7620 can be connected to a terminal (for example, a driver, a pedestrian, or a terminal of a store, or a machine type communication (MTC) terminal) existing in the vicinity of the vehicle using, for example, a peer-to-peer (P2P) technique.

[0206] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol prepared for use in a vehicle. For example, the dedicated communication I / F 7630 can implement a standard protocol such as Wireless Access in Vehicular Environments (WAVE) (which is a combination of IEEE 802.11p as a lower layer and IEEE 1609 as an upper layer), Dedicated Short-Range Communication (DSRC), or a cellular communication protocol. The dedicated communication I / F 7630 generally performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0207] The positioning section 7640 receives a Global Navigation Satellite System (GNSS) signal (for example, a Global Positioning System (GPS) signal from a GPS satellite) from a GNSS satellite, for example, performs positioning, and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning section 7640 can specify the current position by exchanging signals with a wireless access point or can acquire position information from a terminal such as a mobile phone, a PHS, or a smart phone having a positioning function.

[0208] The beacon receiving section 7650 receives a radio wave or an electromagnetic wave transmitted from a wireless station installed on a road or the like, for example, and acquires information such as the current position, traffic congestion, a closed road, a required time, and the like. Note that the function of the beacon receiving section 7650 can be included in the above-described dedicated communication I / F 7630.

[0209] The in-vehicle device I / F 7660 is a communication interface that mediates connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 can establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless USB (WUSB). Furthermore, the in-vehicle device I / F 7660 can establish a wired connection such as universal serial bus (USB), high-definition multimedia interface (HDMI) (registered trademark), mobile high-definition link (MHL), or the like via an unillustrated connection terminal (and, if necessary, a cable). The in-vehicle devices 7760 can include, for example, at least one of a mobile device or a wearable device possessed by an occupant, or an information device carried in or attached to the vehicle. Furthermore, the in-vehicle devices 7760 can include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0210] The in-vehicle network I / F 7680 is an interface that coordinates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like in accordance with a predetermined protocol supported by the communication network 7010.

[0211] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various programs on the basis of information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning section 7640, the beacon receiving section 7650, the in-vehicle device I / F 7660, or the in-vehicle network I / F 7680. For example, the microcomputer 7610 can calculate control target values of a driving force generation device, a steering mechanism, or a braking device on the basis of acquired information about the inside and outside of the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control intended to realize a function of an advanced driver assistance system (ADAS) including collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, maintenance of a vehicle speed of driving, warning of a vehicle collision, warning of deviation of the vehicle from a lane, and the like. In addition, the microcomputer 7610 can perform cooperative control intended for automatic driving that causes the vehicle to automatically travel without depending on an operation of a driver and the like by controlling a driving force generation device, a steering mechanism, a braking device, and the like on the basis of information about the outside or the inside of the vehicle on which information is obtained.

[0212] The microcomputer 7610 can generate three-dimensional distance information between the vehicle and an object such as a surrounding structure, a person, or the like, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning section 7640, the beacon receiving section 7650, the in-vehicle device I / F 7660, or the in-vehicle network I / F 7680, and create local map information including surrounding information of the current position of the vehicle. Further, the microcomputer 7610 can predict a danger such as a collision of the vehicle, an approach or entry of a pedestrian, or the like, to a closed road, based on the acquired information, and generate a warning signal. The warning signal can be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0213] The sound / image output section 7670 transmits an output signal of at least one of a sound and an image to an output device capable of visually or aurally notifying information to an occupant of the vehicle or outside of the vehicle. In Figure 20 In the example described above, the audio speaker 7710, the display section 7720, and the instrument panel 7730 are described as output devices. The display section 7720 can include at least one of an on-board display and a head-up display, for example. The display section 7720 can have an augmented reality (AR) display function. The output device can be another device different from these devices, such as a wearable device (such as a headphone), a glasses-type display worn by a passenger, or the like, a projector, a lamp, or the like. In the case where the output device is a display device, the display device visually displays a result obtained by various processing performed by the microcomputer 7610 or information received from another control unit in various formats such as text, an image, a table, a graph, or the like. Further, in the case where the output device is a sound output device, the sound output device converts an audio signal including reproduced sound data, sound data, or the like, into an analog signal, and outputs the analog signal in an audible manner.

[0214] Note that, in the example described in Figure 20 the above, at least two control units connected via the communication network 7010 can be integrated into one control unit. Alternatively, each control unit can include a plurality of control units. Further, the vehicle control system 7000 can include another control unit (not depicted). Further, in the above description, some or all of the functions performed by any control unit can be provided to another control unit. That is, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing can be performed by any control unit. Similarly, a sensor or a device connected to an arbitrary control unit can be connected to another control unit, and a plurality of control units can transmit and receive detection information to and from each other via the communication network 7010.

[0215] The above has described an example of a vehicle control system to which the technology according to the present disclosure can be applied. The technology according to the present disclosure can be applied to, for example, among the configurations described above, the imaging sections 7910, 7912, 7914, 7916, and 7918, the outside-vehicle information detection sections 7920, 7922, 7924, 7926, 7928, and 7930, the driver state detection section 7510, and the like. Specifically, the imaging system 10 in the Figure 1

[0216] Note that the present technology can have the following configurations.

[0217] (1) An imaging device comprising:

[0218] a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal;

[0219] a detection section that outputs a detection signal in a case where an absolute value of an amount of change in the electric signal within a pixel of the plurality of pixels exceeds a predetermined threshold value;

[0220] a signal processing section that performs predetermined signal processing based on the detection signal output from the detection section;

[0221] an AZ output section that outputs an auto-zero signal for initializing the detection section;

[0222] a time code generator that outputs a time code that changes at a predetermined period;

[0223] a first holding circuit that holds the time code output from the time code generator when the auto-zero signal is output;

[0224] a second holding circuit that holds the time code output from the time code generator when the detection signal is output; and

[0225] a transfer section that transfers the time code held in the first holding circuit and the time code held in the second holding circuit to the signal processing section in association.

[0226] (2) The imaging device according to (1),

[0227] wherein the first holding circuit holds, for each of the plurality of pixels, the time code corresponding to the auto-zero signal, and

[0228] ​The second holding circuit holds the time code corresponding to the detection signal for the pixel of the plurality of pixels that has output the detection signal.

[0229] (3) The imaging device according to (1) or (2),

[0230] wherein the second holding circuit holds data of the plurality of pixels by holding predetermined data of the pixel that does not output the detection signal.

[0231] (4) The imaging device according to (3),

[0232] wherein the transfer section associates the data of the plurality of pixels held by the first holding circuit and the data of the plurality of pixels held by the second holding circuit, and sequentially transfers the data to the signal processing section per pixel.

[0233] (5) The imaging device according to (3) or (4),

[0234] wherein a period in which a time code corresponding to an auto-zero signal of the plurality of pixels is held in the first holding circuit, a period in which a time code corresponding to a detection signal of the plurality of pixels is held in the second holding circuit, and a period in which data of the plurality of pixels held by the first holding circuit and the second holding circuit is sequentially read per pixel and transferred by the transfer section are sequentially repeated.

[0235] (6) The imaging device according to (5),

[0236] wherein during a period of one frame in which data of the plurality of pixels is read, a period in which a time code corresponding to an auto-zero signal of the plurality of pixels is held in the first holding circuit, a period in which a time code corresponding to a detection signal of the plurality of pixels is held in the second holding circuit, and a period in which data of the plurality of pixels held by the first holding circuit and the second holding circuit is sequentially read per pixel and transferred by the transfer section are provided.

[0237] (7) The imaging device according to any one of (1) to (6),

[0238] wherein the transfer section includes a plurality of stages of clusters connected in series, the clusters transfer the time code generated by the time code generator, and sequentially transfer data held by the first holding circuit and data held by the second holding circuit per pixel, and

[0239] Each of the clusters in the multiple stages transmits the time code generated by the time code generator to the cluster in the subsequent stage or the signal processing section, and transmits the data held by the first holding circuit and the data held by the second holding circuit to the cluster in the subsequent stage or the signal processing section.

[0240] (8) The imaging device according to (7),

[0241] wherein the cluster in the first stage holds the time code output from the time code generator, and holds the data held by the corresponding first holding circuit and the data held by the second holding circuit at the same timing, and

[0242] The clusters in the second and subsequent stages transmit the time code sequentially transferred from the time code generator held in the cluster of the preceding stage to the cluster of the subsequent stage or the signal processing section, transmit the corresponding data held by the first holding circuit and the data held by the second holding circuit to the cluster of the subsequent stage or the signal processing section, and transmit the data held by the cluster of the preceding stage to the cluster of the subsequent stage or the signal processing section.

[0243] (9) The imaging device according to any one of (1) to (6),

[0244] wherein the transmission section includes:

[0245] a first cluster in the multiple stages that transmits the time code generated by the time code generator; and

[0246] a second cluster in the multiple stages that transmits the data held by the first holding circuit and the data held by the second holding circuit,

[0247] Each of the first clusters in the multiple stages transmits the data held by the corresponding first holding circuit and the data held by the second holding circuit to the corresponding second cluster, and

[0248] The signal processing section receives the time code output from the first cluster in the final stage, and receives the data output from the second cluster in the final stage.

[0249] (10) The imaging device according to (9),

[0250] wherein a first cluster in the first stage holds the time code output from the time code generator,

[0251] a second cluster in the first stage holds the data held by the corresponding first holding circuit and the data held by the second holding circuit,

[0252] the first cluster in the second stage and subsequent stages holds the time codes sequentially transferred from the time code generator held in the first cluster in the preceding stage, and

[0253] the second cluster in the second stage and subsequent stages transfers data held by the corresponding first holding circuit and data held by the corresponding second holding circuit to the second cluster in the subsequent stage or the signal processing section, and transfers data held by the second cluster in the preceding stage to the second cluster in the subsequent stage or the signal processing section.

[0254] (11) The imaging device according to (9) or (10),

[0255] wherein, during a first frame period in which data of the plurality of pixels is read, a period in which a time code corresponding to the auto-zero signal of the plurality of pixels is held in the first holding circuit and a period in which a time code corresponding to the detection signal of the plurality of pixels is held in the second holding circuit are set, and

[0256] each of the plurality of second clusters transfers, in a second frame period after the first frame period, the data of the plurality of pixels held in the first holding circuit and the data of the plurality of pixels held in the second holding circuit in the first frame period to the second cluster of the subsequent stage or the signal processing section.

[0257] (12) The imaging device according to (9) or (10),

[0258] wherein, detection processing of the detection signal of the plurality of pixels is started asynchronously with a frame period in which data of the plurality of pixels is read, after a time code corresponding to the detection signal of the pixel outputting the auto-zero signal is held in the second holding circuit, the detection signal of the pixel is reset, data held by the second holding circuit is transferred to the corresponding second cluster, and thereafter, after a time code corresponding to the auto-zero signal is held in the first holding circuit, the detection processing of the detection signal of the pixel is resumed.

[0259] (13) The imaging device according to any one of (1) to (12),

[0260] wherein each of the plurality of pixels includes an analog-digital converter that generates a first digital signal obtained by digitally converting a reset level of the electric signal generated by the plurality of photoelectric conversion elements and a second digital signal obtained by digitally converting the electric signal,

[0261] the first holding circuit holds the time code corresponding to the auto-zero signal or the first digital signal, and

[0262] The second holding circuit holds the time code corresponding to the detection signal or the second digital signal.

[0263] (14) The imaging device according to (13), further comprising:

[0264] a first selector that selects one of the auto-zero signal and the first digital signal and supplies the selected one to the first holding circuit; and

[0265] a second selector that selects one of the detection signal and the second digital signal and supplies the selected one to the second holding circuit,

[0266] wherein the first selector and the second selector cooperate with each other to perform the selection operation such that when the first holding circuit holds the time code corresponding to the auto-zero signal, the second holding circuit holds the time code corresponding to the detection signal, and when the first holding circuit holds the time code corresponding to the first digital signal, the second holding circuit holds the time code corresponding to the second digital signal.

[0267] (15) The imaging device according to any one of (1) to (14),

[0268] wherein the time code generator outputs the time code including a Gray code.

[0269] (16) An imaging method comprising:

[0270] among a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal, in a case where an absolute value of a change amount of the electric signal exceeds a predetermined threshold value, outputting a detection signal from a detection section;

[0271] performing predetermined signal processing based on the detection signal;

[0272] outputting an auto-zero signal for initializing the detection section;

[0273] when the auto-zero signal is outputted, holding a time code outputted from a time code generator in a first holding circuit;

[0274] when the detection signal is outputted, holding the time code outputted from the time code generator in a second holding circuit; and

[0275] transferring the time code held in the first holding circuit and the time code held in the second holding circuit in association to a signal processing section.

[0276] Aspects of the present disclosure are not limited to the respective embodiments described above, but include various modifications that can be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above. That is, various additions, modifications, and partial deletions can be made without departing from the conceptual idea and spirit of the present disclosure obtained from the content defined in the claims and the equivalents thereof.

[0277] List of reference numerals

[0278] 1 imaging device

[0279] 2 imaging system

[0280] 3 imaging lens

[0281] 4 recording section

[0282] 5 control section

[0283] 6 information processing section

[0284] 11 semiconductor substrate

[0285] 12 pixel array section

[0286] 13 time code transmission section

[0287] 14 pixel drive circuit

[0288] 15 DAC

[0289] 16 time code generator

[0290] 17 vertical drive circuit

[0291] 18 signal processing section

[0292] 20 controller

[0293] 21 pixel

[0294] 32 light receiving section

[0295] 33 pixel circuit

[0296] 34 ADC

[0297] 35 address event detection section

[0298] 36 floating diffusion (FD)

[0299] 37 voltage comparison section

[0300] 38 initialization control section

[0301] 41 differential input circuit

[0302] 42 voltage conversion circuit

[0303] 43 positive feedback circuit

[0304] 44 data storage section

[0305] 45 logarithmic response section

[0306] 46 buffer

[0307] 47 differentiation circuit

[0308] 48 comparator

[0309] 49 current-voltage conversion section

[0310] 51, 52 delay sections

[0311] 53, 54 XOR gates

[0312] 61 cluster

[0313] 62 data holding section

[0314] 63 repeater

[0315] 64 first data holding section

[0316] 65 second data holding section

[0317] 66 bidirectional buffer

[0318] 67 D-F / F

[0319] 68 binary counter

[0320] 69 binary / gray scale converter

[0321] 71 first semiconductor chip

[0322] 72 second semiconductor chip

[0323] 73 third semiconductor chip

Claims

1. An imaging device comprising: a plurality of pixels each having a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal; a detection section that outputs a detection signal in a case where an absolute value of an amount of change in the electric signal within a pixel of the plurality of pixels exceeds a predetermined threshold value; a signal processing section that performs predetermined signal processing based on the detection signal output from the detection section; an AZ output section that outputs an auto-zero signal for initializing the detection section; a time code generator that outputs a time code that varies at a predetermined period; a first holding circuit that holds the time code output from the time code generator when the auto-zero signal is output; a second holding circuit that holds the time code output from the time code generator when the detection signal is output; and a transfer section that transfers the time code held in the first holding circuit and the time code held in the second holding circuit to the signal processing section in association.

2. The imaging device according to claim 1, the first holding circuit holds the time code corresponding to the auto-zero signal for each of the plurality of pixels, and wherein the second holding circuit holds the time code corresponding to the detection signal for a pixel of the plurality of pixels that has output the detection signal.

3. The imaging device according to claim 1, the second holding circuit holds data of the plurality of pixels by holding predetermined data for a pixel that does not output the detection signal. wherein 4. The imaging device according to claim 3, the transfer section associates data of the plurality of pixels held by the first holding circuit and data of the plurality of pixels held by the second holding circuit, and sequentially transfers the data to the signal processing section per pixel. wherein, 5. The imaging device according to claim 3, a period in which the time code corresponding to the auto-zero signal for the plurality of pixels is held in the first holding circuit, a period in which the time code corresponding to the detection signal for the plurality of pixels is held in the second holding circuit, and a period in which data of the plurality of pixels held by the first holding circuit and the second holding circuit are sequentially read per pixel and transferred by the transfer section are sequentially repeated. wherein 6. The imaging device according to claim 5, during a period of one frame in which data of the plurality of pixels are read, a period in which the time code corresponding to the auto-zero signal for the plurality of pixels is held in the first holding circuit, a period in which the time code corresponding to the detection signal for the plurality of pixels is held in the second holding circuit, and a period in which data of the plurality of pixels held by the first holding circuit and the second holding circuit are sequentially read per pixel and transferred by the transfer section are provided. wherein 7. The imaging device according to claim 1, the transfer section includes a plurality of stages of clusters connected in series, the clusters transfer the time code generated by the time code generator, and sequentially transfer data held by the first holding circuit and data held by the second holding circuit per pixel, and wherein ​ Each of the clusters in the multiple stages transmits the time code generated by the time code generator to the cluster in the subsequent stage or the signal processing section, and transmits the data held by the first holding circuit and the data held by the second holding circuit to the cluster in the subsequent stage or the signal processing section.

8. The imaging apparatus according to claim 7, wherein, The cluster in the first stage holds the time code output from the time code generator, and holds the data held by the corresponding first holding circuit and the data held by the second holding circuit at the same timing, and The clusters in the second and subsequent stages transmit the time code sequentially transmitted from the time code generator held in the cluster of the preceding stage to the cluster of the subsequent stage or the signal processing section, transmit the corresponding data held by the first holding circuit and the data held by the second holding circuit to the cluster of the subsequent stage or the signal processing section, and transmit the data held by the cluster of the preceding stage to the cluster of the subsequent stage or the signal processing section.

9. The imaging apparatus according to claim 1, wherein The transmission section includes: a first cluster in the multiple stages that transmits the time code generated by the time code generator; and a second cluster in the multiple stages that transmits the data held by the first holding circuit and the data held by the second holding circuit, Each of the first clusters in the multiple stages transmits the data held by the corresponding first holding circuit and the data held by the second holding circuit to the corresponding second cluster, and The signal processing section receives the time code output from the first cluster in the final stage, and receives the data output from the second cluster in the final stage.

10. The imaging apparatus according to claim 9, wherein The first cluster in the first stage holds the time code output from the time code generator, The second cluster in the first stage holds the data held by the corresponding first holding circuit and the data held by the second holding circuit, The first clusters in the second and subsequent stages hold the time code sequentially transmitted from the time code generator held in the first cluster in the preceding stage, and The second clusters in the second and subsequent stages transmit the data held by the corresponding first holding circuit and the data held by the corresponding second holding circuit to the second cluster in the subsequent stage or the signal processing section, and transmit the data held by the second cluster in the preceding stage to the second cluster in the subsequent stage or the signal processing section.

11. The imaging apparatus according to claim 9, wherein During a first frame period in which the data of the plurality of pixels is read, the period in which the time code corresponding to the auto-zero signal of the plurality of pixels is held in the first holding circuit, and the period in which the time code corresponding to the detection signal of the plurality of pixels is held in the second holding circuit are set, and During a first frame period in which the data of the plurality of pixels is read, the period in which the time code corresponding to the auto-zero signal of the plurality of pixels is held in the first holding circuit, and the period in which the time code corresponding to the detection signal of the plurality of pixels is held in the second holding circuit are set, and Each of the plurality of second clusters transfers, in a second frame period after the first frame period, data of the plurality of pixels held in the first holding circuit and data of the plurality of pixels held in the second holding circuit to a subsequent stage second cluster or the signal processing section.

12. The imaging device according to claim 9, wherein the detection processing of the detection signal of the plurality of pixels is started asynchronously with a frame period in which data of the plurality of pixels is read, after a time code corresponding to the detection signal of the pixel of which the detection signal is output is held in the second holding circuit, the detection signal of the pixel is reset, an auto-zero signal is output, data held by the second holding circuit is transferred to the corresponding second cluster, and thereafter, after a time code corresponding to the auto-zero signal is held in the first holding circuit, the detection processing of the detection signal of the pixel is resumed.

13. The imaging device according to claim 1, wherein each of the plurality of pixels includes an analog-digital converter that generates a first digital signal obtained by digitally converting a reset level of the electric signal generated by the plurality of photoelectric conversion elements and a second digital signal obtained by digitally converting the electric signal, the first holding circuit holds the time code corresponding to the auto-zero signal or the first digital signal, and the second holding circuit holds the time code corresponding to the detection signal or the second digital signal.

14. The imaging device according to claim 13, further comprising: a first selector that selects one of the auto-zero signal and the first digital signal and supplies the selected one to the first holding circuit; and a second selector that selects one of the detection signal and the second digital signal and supplies the selected one to the second holding circuit, wherein the first selector and the second selector cooperate with each other to perform the selection operation such that when the first holding circuit holds the time code corresponding to the auto-zero signal, the second holding circuit holds the time code corresponding to the detection signal, and when the first holding circuit holds the time code corresponding to the first digital signal, the second holding circuit holds the time code corresponding to the second digital signal.

15. The imaging device according to any one of claims 1 to 14, the time code generator outputs the time code including a Gray code. wherein, 16. An imaging method comprising: among a plurality of pixels each of which has a plurality of photoelectric conversion elements that photoelectrically convert incident light to generate an electric signal, outputting a detection signal from a detection section in a case where an absolute value of a variation amount of the electric signal exceeds a predetermined threshold; performing predetermined signal processing based on the detection signal; outputting an auto-zero signal for initializing the detection section; when the auto-zero signal is output, holding a time code output from a time code generator in a first holding circuit; when the detection signal is output, holding a time code output from a time code generator in a second holding circuit; and ​ The time codes held in the first holding circuit and the time codes held in the second holding circuit are associated and transmitted to a signal processing section.

Images