Light detection device

Abstract

[Problem] To provide a light detection device capable of reducing power consumption and data quantity. [Solution] This light detection device includes a plurality of first pixels arranged in a one-dimensional or two-dimensional direction. Each of the plurality of first pixels is provided with a first photoelectric conversion element for accumulating charges corresponding to the light quantity of incident light, and an event detection circuit for detecting an event resulting from a change in luminance on the basis of the charges. A pixel block including two or more first pixels among the plurality of first pixels has a switch for switching whether internal nodes of the two or more first pixels are connected to each other.

Description

Photodetector

[0001] The present disclosure relates to a light detection device.

[0002] An event-based vision sensor (EVS) has been proposed that detects events, such as changes in brightness, in an image capture scene. Compared to frame-type vision sensors, such as complementary metal oxide semiconductor (CMOS) image sensors, which scan all pixels at predetermined intervals to acquire gradation information for each pixel, the EVS is characterized by its ability to detect events at high speed and with low power consumption (see Patent Document 1).

[0003] International Publication No. 2019 / 087471

[0004] Patent Document 1 discloses a technique for reducing the circuit scale of each chip by forming an EVS into a stacked structure of multiple chips.

[0005] However, conventional EVS systems have a fixed resolution for event detection, and perform event detection across the entire image even when an event occurs only in a small portion of the image, resulting in problems such as high power consumption. Furthermore, event detection is also performed on the background portion of the image, resulting in unnecessary data volume and a high possibility of noise being included in the event.

[0006] Therefore, the present disclosure provides a photodetector that consumes less power and can reduce the amount of data.

[0007] In order to solve the above problems, according to the present disclosure, there is provided a photodetection device comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels comprising: a first photoelectric conversion element that accumulates an electric charge according to the amount of incident light; and an event detection circuit that detects an event due to a change in brightness based on the electric charge, and a pixel block including two or more first pixels among the plurality of first pixels has a switch that switches whether or not internal nodes of the two or more first pixels are connected to each other.

[0008] Each of the plurality of first pixels may have a logarithmic response unit that generates a voltage signal based on the charge, the event detection circuit may detect the event based on the voltage signal, and the internal node may be a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit.

[0009] The two or more first pixels may detect the event by turning on the switch and connecting internal nodes of the two or more first pixels together, thereby integrating the two or more first pixels.

[0010] The switch may be turned on to connect internal nodes of the two or more first pixels to each other, thereby reducing the resolution at which the event is detected.

[0011] A plurality of pixel blocks may be provided, each having the switch, and with a plurality of the switches corresponding to the plurality of pixel blocks turned off, switching control of the corresponding switches may be performed for each of the plurality of pixel blocks based on the results of detecting the events for the plurality of first pixels.

[0012] The image sensor may further include an output circuit that outputs spatiotemporal differential signals between two or more of the first pixels as the event.

[0013] According to the present disclosure, there is provided a photodetection device comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels having a first photoelectric conversion element that accumulates electric charge according to the amount of incident light, and an event detection circuit that detects an event due to a luminance change based on the electric charge, wherein the event detection circuit has an output circuit that outputs spatiotemporal differential signals between two or more of the plurality of first pixels as the event.

[0014] The spatiotemporal differential signal may be generated by temporally differentiating a spatial differential signal generated by spatially differentiating two of the first pixels.

[0015] The event detection circuit may include a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge, a differentiator that time-differentiates the output voltage of the buffer circuit, and a comparator that compares the output signal of the differentiator with a reference signal to detect the event, and the spatially differentiated signal may be generated by detecting a difference between output signals of the two buffer circuits between the two first pixels.

[0016] The event detection circuit may include a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge, a differentiator that time-differentiates the output voltage of the buffer circuit, and a comparator that compares the output signal of the differentiator with a reference signal to detect the event, and the spatially differentiated signal may be generated by detecting a difference between the output signals of the two differentiators between the two first pixels.

[0017] The event detection circuit may include: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that time-differentiates the output voltage of the buffer circuit; a comparator that detects the event by comparing the output signal of the differentiator with a reference signal; and an operator that generates the spatiotemporal differential signal by calculating the exclusive OR of the output signals of the two comparators.

[0018] The plurality of first pixels may be divided into a plurality of pixel blocks each including two or more of the first pixels, and a control line for controlling at least one of resolution, sensitivity, power gating, or responsiveness may be connected to each of the plurality of pixel blocks.

[0019] Each of the plurality of control signals transmitted through the plurality of control lines connected to the plurality of pixel blocks may control at least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block.

[0020] The control signal may control a threshold for detecting the event at each of the two or more first pixels in the corresponding pixel block.

[0021] The control signal may perform switching control to determine whether or not internal nodes of the two or more first pixels included in the corresponding pixel block are connected to each other.

[0022] The image sensor may include a plurality of second pixels arranged in one or two dimensions, each of the plurality of second pixels having a second photoelectric conversion element that accumulates a charge corresponding to the amount of incident light, and a pixel circuit that generates a pixel signal corresponding to the amount of incident light based on the charge.

[0023] The plurality of first pixels may be disposed in a dispersed manner between the plurality of second pixels.

[0024] One or more of the first pixels may be arranged in each pixel block including a plurality of the second pixels adjacent to each other in a first direction and a second direction that intersect with each other.

[0025] The pixel block may include a first pixel block including 2x2=4 second pixels adjacent to each other in a first direction and a second direction that intersect with each other, and two second pixel blocks arranged adjacent to each other in the first direction and the second direction, and of the 2x2=4 first pixel blocks included in the second pixel block, one of two first pixel blocks arranged diagonally may include four green second pixels, and the other may include three green second pixels and one first pixel, and one of the remaining two first pixel blocks may include three blue second pixels and one first pixel, and the other may include four red second pixels.

[0026] A first pixel block, each including 2×2=4 second pixels adjacent to each other in a first direction and a second direction intersecting each other, may include second pixel blocks arranged two by two in the first direction and the second direction, and of the 2×2=4 first pixel blocks included in the second pixel block, two first pixel blocks arranged diagonally may each include three green second pixels and one first pixel, one of the remaining two first pixel blocks may include three blue second pixels and one first pixel, and the other may include three red second pixels and one first pixel, and the four first pixels included one by one in each of the four first pixel blocks may be arranged adjacent to each other in the first direction and the second direction.

[0027] 1 is a block diagram of an electronic device according to a first embodiment of the present disclosure. FIG. 1 is a block diagram showing a schematic configuration of a photodetector according to the first embodiment. FIG. 2 is a schematic perspective view showing an example of a photodetector according to the present disclosure having a two-layer stacked structure. FIG. 3 is a diagram showing an example of an arrangement of gradation pixels and EVS pixels in a pixel array section of a first substrate. FIG. 4 is a diagram showing an example of integrating multiple AFE circuits by providing an AFE circuit for each first pixel block. FIG. 5 is a circuit diagram showing the circuit configuration of gradation pixels. FIG. 6 is a circuit diagram showing the internal configuration of EVS pixels and an event detection circuit. FIG. 7 is a diagram showing an example of a photodetector having a stacked structure of a first substrate, a second substrate, and a third substrate. FIG. 8 is a circuit diagram showing an example of arranging a logarithmic response unit and a portion of a buffer on the second substrate. FIG. 9 is a circuit diagram of a first example of switching the resolution of EVS pixels. FIG. 10 is a circuit diagram of a second example of switching the resolution of EVS pixels. FIG. 11 is a circuit diagram of a third example of switching the resolution of EVS pixels. FIG. 12 is a diagram showing an example of an object on which event detection should be performed. FIG. 13 is a diagram showing an event image obtained by performing event detection on the entire area of ​​the object at high resolution. FIG. 14 is a diagram showing an example of performing event detection on only a moving object portion of the object at high resolution. 14A and 14B are diagrams illustrating a first method for determining the resolution of an object; a diagram illustrating a second method for determining the resolution of an object; a block diagram illustrating a schematic configuration of a photodetector according to a second embodiment; a block diagram illustrating a schematic configuration of a photodetector according to a first modified example of the second embodiment; a block diagram illustrating a schematic configuration of a photodetector according to a second modified example of the second embodiment; a block diagram illustrating a schematic configuration of a photodetector according to a third modified example of the second embodiment; a diagram illustrating a first example of generating a spatiotemporal differential signal using an ExOR circuit; a diagram illustrating a second example of generating a spatiotemporal differential signal using an ExOR circuit; a diagram illustrating a third example of generating a spatiotemporal differential signal using an ExOR circuit; a diagram illustrating a fourth example of generating a spatiotemporal differential signal using an ExOR circuit; a diagram illustrating a fifth example of generating a spatiotemporal differential signal using an ExOR circuit; a diagram for explaining a third embodiment; a circuit diagram of a front-stage portion of an EVS pixel applicable to the third embodiment; a diagram illustrating an example in which the resolution is individually controlled by a control signal for each pixel block of FIG. 14A; a circuit diagram illustrating a first specific example of the third embodiment; a circuit diagram illustrating a second specific example of the third embodiment. 1 is a plan view of a pixel block according to a first example; 2 is a plan view of a pixel block according to a second example; 3 is a plan view of a pixel block according to a third example; 4 is a plan view of a pixel block according to a fourth example; 5 is a plan view of a pixel block according to a fifth example; 6 is a plan view showing a hybrid pixel according to the first example;6A and 6B are plan views showing hybrid pixels according to a second example; plan views showing hybrid pixels according to a third example; plan views showing hybrid pixels according to a fourth example; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the first modified example of FIG. 6; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the second modified example of FIG. 6; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the third modified example of FIG. 6; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the fourth modified example of FIG. 6; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the fifth modified example of FIG. 6; circuit diagrams showing the locations of the EVS pixels and the event detection circuits according to the sixth modified example of FIG. 6; a diagram showing a first example of a layout arrangement; a diagram showing a second example of a layout arrangement; a block diagram showing an example of a schematic configuration of a vehicle control system; an explanatory diagram showing an example of the installation positions of an outside vehicle information detection unit and an imaging unit.

[0028] Hereinafter, embodiments of a light detection device and electronic equipment will be described with reference to the drawings. The following description will focus on the main components of the light detection device and electronic equipment, but the light detection device may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0029] 1 is a block diagram of an electronic device 30 according to a first embodiment of the present disclosure. The electronic device 30 has a function of generating an image according to the luminance of incident light. The electronic device 30 in FIG. 1 includes a light detection device 1, an imaging lens 31, an image processing unit 32, a recording unit 33, and a control unit 34. The electronic device 30 can be applied to, for example, a surveillance camera or a camera mounted on an industrial robot, or a camera for general use, but the specific use and configuration of the electronic device 30 are arbitrary.

[0030] The imaging lens 31 collects incident light and guides it to the photodetector 1. The photodetector 1 captures an image of the incident light. The photodetector 1 has a function of generating a captured image by photoelectrically converting light in a predetermined wavelength range, such as visible light or infrared light. The captured image generated by the photodetector 1 is sent to an image processor 32 and a recording unit 33.

[0031] The image processing unit 32 performs predetermined image processing on the captured image, such as color or brightness adjustment, image compression, image recognition, tracking, or analysis. The image processed by the image processing unit 32 is output to the recording unit 33, for example.

[0032] The recording unit 33 records the image output from the light detection device 1 or the image processing unit 32. The recording unit 33 may be disposed in a server connected via a network. In the electronic device 30 according to this embodiment, at least one of the image processing unit 32 and the recording unit 33 in FIG. 1 can be omitted.

[0033] The control unit 34 controls the operation of the light detection device 1. Furthermore, although not explicitly shown in FIG.

[0034] 2 is a block diagram showing a schematic configuration of a photodetector 1 according to a first embodiment. The photodetector 1 in FIG. 2 includes a pixel array unit 2, a gradation ADC (Analog-Digital Converter) 3, an event detection circuit 4, an EVS digital circuit 5, a control circuit 6, and a bias circuit 7.

[0035] The pixel array unit 2 has a plurality of gradation pixels 8 and a plurality of EVS pixels 9. It is assumed that the number of gradation pixels 8 in the pixel array unit 2 is greater than the number of EVS pixels 9, but the ratio between the number of gradation pixels 8 and the number of EVS pixels 9 is arbitrary.

[0036] A plurality of gradation pixels 8 and a plurality of EVS pixels 9 are arranged in the row direction x and the column direction y of the pixel array unit 2. Fig. 2 shows an example in which the pixel array unit 2 is divided into a plurality of pixel blocks BL. Each pixel block BL has two or more gradation pixels 8 and one EVS pixel 9. There is no particular limit to the number of pixel blocks BL in the pixel array unit 2.

[0037] Both the gradation pixel 8 and the EVS pixel 9 have a photoelectric conversion element PD (photodiode) and a pixel circuit. The pixel circuit for the gradation pixel 8 converts the charge photoelectrically converted by the photoelectric conversion element PD of the gradation pixel 8 into a voltage to generate a pixel signal. The pixel signal includes gradation information. The pixel circuit for EVS detects an event based on a voltage signal corresponding to the charge photoelectrically converted by the photoelectric conversion element PD of the EVS pixel 9. The event is information indicating that the amount of change in luminance has exceeded a predetermined threshold. The specific circuit configurations of the gradation pixel 8 and the EVS pixel 9 will be described later.

[0038] In this specification, the photoelectric conversion element PDi for the gradation pixel 8, or a portion of the pixel circuit arranged on the same substrate as the photoelectric conversion element PDi for the gradation pixel 8, may be referred to as the gradation pixel 8, or the entire photoelectric conversion element PDi and pixel circuit for the gradation pixel 8 may be referred to as the gradation pixel 8. Similarly, the photoelectric conversion element PDe for the EVS pixel 9, or a portion of the pixel circuit (more specifically, a logarithmic response unit and an event detection circuit, as described later) arranged on the same substrate as the photoelectric conversion element PDe for the EVS pixel 9 may be referred to as the EVS pixel 9, or the entire photoelectric conversion element PDe, logarithmic response unit, and event detection circuit for the EVS pixel 9 may be referred to as the EVS pixel 9. In this way, in this specification, the term gradation pixel 8 or EVS pixel 9 may refer to at least a portion of the pixel circuit.

[0039] The pixel signals photoelectrically converted by the gradation pixels 8 are transmitted to the gradation ADC 3 via signal lines extending in the column direction y. A voltage signal corresponding to the charge photoelectrically converted by the EVS pixels 9 is transmitted to the event detection circuit 4.

[0040] The grayscale ADC 3 performs AD conversion on pixel signals on a plurality of signal lines arranged at predetermined intervals along the row direction x to generate digital pixel signals.

[0041] Although FIG. 2 illustrates one event detection circuit 4, in reality, an event detection circuit 4 is provided for each EVS pixel 9, and detects whether an event has occurred by comparing a voltage signal corresponding to the charge photoelectrically converted by the corresponding EVS pixel 9 with a threshold voltage. More specifically, the event detection circuit 4 outputs an event signal indicating whether an event has occurred. The event signal includes, for example, an on-event signal and an off-event signal. The on-event signal is a signal that detects an on-event indicating that the amount of change in luminance of the EVS pixel 9 has exceeded a predetermined threshold. For example, when the on-event signal becomes logically high, it indicates that an on-event has occurred. The off-event signal is a signal that detects an off-event indicating that the amount of change in luminance of the EVS pixel 9 has fallen below a predetermined threshold. For example, when the off-event signal becomes logically high, it indicates that an off-event has occurred.

[0042] In this embodiment, ON event signals detected by all EVS pixels 9 in the pixel array unit 2 are output simultaneously, and at a different timing, OFF event signals detected by all EVS pixels 9 in the pixel array unit 2 are output simultaneously. In this specification, ON event signals and OFF event signals are collectively referred to as event signals.

[0043] The event detection circuit 4 may simultaneously detect an on-event and an off-event and output an on-event signal and an off-event signal, in which case the event signal will be two bits, one bit representing the on-event signal and the other bit representing the off-event signal.

[0044] The EVS digital circuit 5 generates digital signals corresponding to a plurality of event signals detected by a plurality of EVS pixels 9 in the pixel array unit 2 .

[0045] The control circuit 6 generates and supplies various control signals to be used by the various components in the photodetector 1 shown in Fig. 2. The bias circuit 7 generates various bias signals to be used by the various components in the photodetector 1 shown in Fig. 2.

[0046] (Two-Layer Stacking) The photodetector 1 according to the present disclosure can be realized as a stacked chip. FIG. 3 is a schematic perspective view showing an example of a photodetector 1 according to the present disclosure having a two-layer stacked structure. The photodetector 1 in FIG. 3 includes a first substrate SB1 disposed on the light incident surface side and a second substrate SB2 stacked on the first substrate SB1. The first substrate SB1 is provided with, for example, photoelectric conversion elements PDi and PDe of the gradation pixels 8 and the EVS pixels 9. The second substrate SB2 is provided with, for example, a gradation ADC 3, an event detection circuit 4, an EVS digital circuit 5, a control circuit 6, and a bias circuit 7. The pixel circuits for the gradation pixels 8 may be disposed on the first substrate SB1, or may be distributed across the first substrate SB1 and the second substrate SB2. The first substrate SB1 and the second substrate SB2 are bonded and transmit signals via, for example, a copper-copper connection (CCC). Alternatively, the first substrate SB1 and the second substrate SB2 may be joined by vias or bumps other than CCC. In this specification, the first substrate SB1 may be referred to as the upper chip SB1, and the second substrate SB2 may be referred to as the lower chip SB2.

[0047] 1 and 2 show an example in which the pixel array section 2 has the gradation pixels 8 and the EVS pixels 9, the pixel array section 2 may have the EVS pixels 9 without the gradation pixels 8.

[0048] (Resolution Conversion of EVS Pixels 9) The photodetector 1 according to the present disclosure is characterized in that the resolution of the EVS pixels 9 can be changed. FIG. 4A is a diagram showing an example of the arrangement of the gradation pixels 8 and the EVS pixels 9 in the pixel array section 2 of the first substrate SB1. In the example of FIG. 4A , a first pixel block BL1 is provided, which includes four pixels in total, two in the first direction x and two in the second direction y, and a second pixel block BL2, two in each of the first direction x and the second direction y. A plurality of second pixel blocks BL2 are arranged in the pixel array section 2 in the first direction x and the second direction y. Each first pixel block BL1 is basically a pixel block of gradation pixels 8 of the same color. The second pixel block BL2 includes two green first pixel blocks BL1, one red first pixel block BL1, and one blue first pixel block BL1. One pixel in one green first pixel block BL1 and one pixel in one blue first pixel block BL1 are EVS pixels 9 rather than gradation pixels 8. In this way, one second pixel block BL2 has 14 gradation pixels 8 and two EVS pixels 9.

[0049] 4B , for example, an analog front-end circuit (hereinafter referred to as an AFE circuit) 12 is arranged on the second substrate SB2 for each first pixel block BL1. As will be described later, the AFE circuit 12 includes an event detection circuit 4 for the EVS pixels 9. If the second pixel block BL2 has four first pixel blocks BL1, four AFE circuits 12 are provided on the second substrate SB2.

[0050] The photodetector 1 according to the present disclosure can, as needed, integrate multiple AFE circuits 12 to perform event detection using a single large AFE circuit 12L. For example, when event detection is performed using multiple EVS pixels 9 included in a second pixel block BL2 as shown in FIG. 4B , in this embodiment, an event detection mode using four AFE circuits 12 and an event detection mode using a single large AFE circuit 12L in which the four AFE circuits 12 are integrated as shown in FIG. 4C can be selected. Although the large AFE circuit 12L includes four AFE circuits 12, only one of them is actually used for event detection, which reduces power consumption compared to performing event detection using the four AFE circuits 12 individually.

[0051] (Circuit Configuration of Gradation Pixel 8) FIG. 5 is a circuit diagram showing the circuit configuration of the gradation pixel 8. As shown in FIG. 5, the gradation pixel 8 has a photoelectric conversion element PDi and a pixel circuit 13. The pixel circuit 13 of the gradation pixel 8 has a transfer transistor Q1, a reset transistor Q2, an amplification transistor Q3, and a selection transistor Q4. All of the transistors Q1 to Q4 in the pixel circuit 13 are NMOS transistors. In this specification, the transistors in the pixel circuit 13 may be collectively referred to as pixel transistors.

[0052] The anode of the photoelectric conversion element PDi is set to a reference voltage (e.g., ground voltage) node, and the cathode is connected to the source of the transfer transistor Q1. The drain of the transfer transistor Q1 is connected to the floating diffusion FD, the source of the reset transistor Q2, and the gate of the amplifier transistor Q3. The drain of the amplifier transistor Q3 is connected to a power supply voltage node, and the source of the amplifier transistor Q3 is connected to the drain of the select transistor Q4. The source of the select transistor Q4 is connected to a signal line.

[0053] Note that FIG. 5 is an example of the circuit configuration of the pixel circuit 13 of the gradation pixel 8, and various circuit configurations different from that of FIG. 5 can also be used.

[0054] 6 is a circuit diagram showing the internal configuration of the EVS pixel 9 and the event detection circuit 4. The EVS pixel 9 has a logarithmic response unit 21 and a buffer 22. The logarithmic response unit 21 generates a voltage signal by logarithmically converting the charge photoelectrically converted by the photoelectric conversion element PDe. By performing the logarithmic conversion, the dynamic range during photoelectric conversion of the EVS pixel 9 can be widened, thereby improving sensitivity.

[0055] The logarithmic response unit 21 has NMOS transistors Q21 to Q23 and a PMOS transistor Q24. The gate and drain of transistor Q22 are connected together, and it functions as a diode. The drain of transistor Q21 is connected to the power supply voltage node, and the source of transistor Q21 is connected to the gate and drain of transistor Q22. The gate of transistor Q21 is connected to the drain of transistor Q23 and the drain of transistor Q24, and this connection node is the output node of logarithmic response unit 21. The source of transistor Q24 is connected to the power supply voltage node, and a predetermined bias voltage Vlog is applied to the gate of transistor Q24.

[0056] The buffer 22 has NMOS transistors Q25 and Q26 cascode-connected between the power supply voltage node and the ground voltage node. The output node of the logarithmic response unit 21 is connected to the gate of the transistor Q25. A reference voltage SF is applied to the gate of the transistor Q26. The source of the transistor Q25 and the drain of the transistor Q26 form the output node of the buffer 22. The transistor Q26 functions as a source follower circuit, and a voltage signal having a waveform corresponding to the source voltage of the transistor Q24 is output from the source of the transistor Q25.

[0057] 6 shows an example in which the logarithmic response unit 21 in the EVS pixel 9 and the transistor Q25 in the buffer 22 are arranged on the upper chip SB1, and the transistor Q26 in the buffer 22 is arranged on the lower chip SB2. This is just one example, and as will be described later, for example, the transistor Q26 in the buffer 22 may be arranged on the upper chip SB1, or the transistors Q25 and Q26 that constitute the buffer 22 may be arranged on the lower chip SB2.

[0058] The event detection circuit 4 in Fig. 6 has a synchronous configuration and outputs an event signal indicating whether or not each of the multiple EVS pixels 9 has detected an event, in synchronization with a predetermined frame period. The event detection circuit 4 in Fig. 6 includes a differentiator 23, a comparator 24, and an output circuit 25. The differentiator 23 includes a capacitor C1, NMOS transistors Q27 and Q28, and a PMOS transistor Q29.

[0059] Capacitor C1 is connected between the output node of buffer 22 and the gate of transistor Q29. Capacitor C1 supplies a current to the gate of transistor Q29 that corresponds to the amount of change in voltage signal Vp, which is the time derivative of voltage signal Vp output from buffer 22. Transistors Q29 and Q28 are cascode-connected between a power supply voltage node and a ground voltage node. A switch 26 that selects one of three bias voltages is connected to the gate of transistor Q28. Based on a switching control signal BIAS_SW output from control circuit 6, switch 26 selects one of the following: a bias voltage AZ for auto-zero, a bias voltage POS for positive event detection, and a bias voltage NEG for negative event detection. The gate of transistor Q28 is set to an optimal bias voltage through switching control by switch 26.

[0060] The differentiator 23 outputs a differentiated signal from the drain of the transistor Q28 and the source of the transistor Q29. This differentiated signal is input to the comparator 24.

[0061] The comparator 24 has a PMOS transistor Q31 and an NMOS transistor Q30 cascode-connected between a power supply voltage node and a ground voltage node. A threshold signal 2NDCOMP is applied to the gate of the transistor Q30. The gate of the transistor Q31 is connected to the source of the transistor Q29, and the differentiated signal output from the differentiator 23 is input to the gate of the transistor Q31. If the voltage level of the differentiated signal is higher than the voltage level of the threshold signal 2NDCOMP, the comparator 24 outputs a high-level event signal indicating that an event has been detected.

[0062] The comparator 24 in FIG. 6 can switch between an ON event signal and an OFF event signal and output them from the source of the transistor Q31 by switching the gate voltage of the transistor Q28 using the switch 26.

[0063] The output circuit 25 has a logic circuit 27 composed of logic gates, flip-flops, etc. The logic circuit 27 receives control signals LATCH_EN and POLARITY output from the control circuit 6. The control signal LATCH_EN is a signal that determines the timing for latching the event signal. The control signal POLARITY is a signal that specifies the polarity of the event signal. The logic circuit 27 outputs the event signal in synchronization with the frame period. The event signal output from the logic circuit 27 is input to the EVS digital circuit 5.

[0064] The EVS digital circuit 5 generates a digital signal based on the event signals detected by all the EVS pixels 9 in the pixel array unit 2 .

[0065] (Three-Layer Stacking) The photodetector 1 according to the present disclosure can be configured by stacking three or more substrates. Fig. 7 is a schematic perspective view showing an example of a photodetector 1 according to the present disclosure having a three-layer structure, and Fig. 8 is a circuit diagram of the EVS pixel 9 and event detection circuit 4 corresponding to Fig. 7 .

[0066] FIG. 7 shows an example of a photodetector 1 having a stacked structure of a first substrate SB1, a second substrate SB2, and a third substrate SB3. The gradation pixels 8 and the EVS pixels 9 in the pixel array section 2 are arranged on the first substrate SB1. Also, as shown in FIG. 8 , a portion of the logarithmic response unit 21 connected to the EVS pixels 9 is arranged on the first substrate SB1. Also, as shown in FIG. 8 , the logarithmic response unit 21 connected to the EVS pixels 9 and a portion of the buffer 22 are arranged on the second substrate SB2. Also, the pixel circuits 13 for the gradation pixels 8 are arranged on either the first substrate SB1 or the second substrate SB2, or are divided between the first substrate SB1 and the second substrate SB2. As shown in FIG. 8 , a portion of the buffer 22 for the EVS pixels 9, a differentiator 23, a comparator 24, and an output circuit 25 are arranged on the third substrate SB3. Additionally, an AD converter and a signal processing circuit for the gradation pixels 8 are arranged on the third substrate SB3. The first substrate SB1 and the second substrate SB2 are bonded and signal transmitted by, for example, a TSV (Through Silicon Via), and the second substrate SB2 and the third substrate SB3 are bonded and signal transmitted by, for example, a CCC. There are many possible variations as to which circuit parts of the photodetector 1 should be arranged on the first to third substrates SB1 to SB3, and Figure 8 is just one example.

[0067] (Method for Switching the Resolution of the EVS Pixels 9) As described above, the photodetector 1 according to the first embodiment is capable of switching the resolution of the EVS pixels 9 as needed. A number of circuit configurations are possible for switching the resolution of the EVS pixels 9. In this embodiment, a pixel block BL including two or more EVS pixels 9 includes a switch that switches whether or not the internal nodes of the two or more EVS pixels 9 are connected to each other. By controlling the switching of this switch, at least one of the resolution, sensitivity, power gating, and responsiveness of the two or more EVS pixels 9 is switched. The internal node is, for example, a node connected to the first photoelectric conversion element PDe, a node connected to the logarithmic response unit 21, or a node connected to the event detection circuit 4, as shown in FIG. 9A (described later).

[0068] 9A is a circuit diagram of a first example of switching the resolution of the EVS pixel 9, Fig. 9B is a circuit diagram of a second example of switching the resolution of the EVS pixel 9, and Fig. 9C is a circuit diagram of a third example of switching the resolution of the EVS pixel 9. In all of the first to third examples, either high resolution or low resolution can be selected.

[0069] 9A , the resolution is switched by connecting a plurality of switches SW1 to SW4 to the cathodes of two photoelectric conversion elements PDe for two EVS pixels 9. More specifically, in the first example, as shown in FIG. 9A , a first switch SW1 and a second switch SW2 are connected in series between the cathodes of two photoelectric conversion elements PDe (hereinafter referred to as the first photoelectric conversion element PD1 and the second photoelectric conversion element PD2) for two EVS pixels 9 (hereinafter referred to as the first EVS pixel 9 and the second EVS pixel 9), a third switch SW3 is connected between the cathode of the first photoelectric conversion element PD1 and the first logarithmic response unit 21 a, and a fourth switch SW4 is connected between the cathode of the second photoelectric conversion element PD2 and the second logarithmic response unit 21 b.

[0070] In the first example, when high resolution is selected, the first switch SW1 and the second switch SW2 are turned off and the third switch SW3 and the fourth switch SW4 are turned on, so that the first EVS pixel 9 and the second EVS pixel 9 perform event detection individually.

[0071] In the first example, when a low resolution is selected, for example, the fourth switch SW4 is turned off and the first switch SW1 to the third switch SW3 are turned on. As a result, event detection based on the charges photoelectrically converted by the first photoelectric conversion element PD1 and the second photoelectric conversion element PD2 is performed by the buffer 22, differentiator 23, event detection circuit 4, and comparator 24 connected to the first logarithmic response unit 21a, and event detection is performed by the single event detection circuit 4 by integrating the two EVS pixels 9. In this case, the buffer 22, differentiator 23, event detection circuit 4, and comparator 24 connected to the second logarithmic response unit 21b are not used, thereby reducing power consumption.

[0072] In a second example shown in FIG. 9B , a switch SW is connected to two differentiators 23 (hereinafter referred to as a first differentiator 23a and a second differentiator 23b) for two EVS pixels 9 to switch the resolution. More specifically, in the second example, as shown in FIG. 9B , a switch SW is provided that is connected between the gate of the transistor Q29 in the first differentiator 23a and the gate of the transistor Q29 in the second differentiator 23b. When this switch SW is turned on, the gate of the transistor Q29 in the first differentiator 23a is connected to the gate of the transistor Q29 in the second differentiator 23b. Event detection can be stopped for either the first differentiator 23a or the second differentiator 23b by controlling the bias voltage input to the gate of the transistor Q28.

[0073] As a result, when the switch SW is on, event detection can be performed using only one of the first differentiator 23 a or the second differentiator 23 b. On the other hand, when the switch SW is off, the first differentiator 23 a and the second differentiator 23 b can perform event detection independently of each other.

[0074] 9C , four EVS pixels 9 are integrated to enable event detection. In the third example, as shown in FIG. 9C , first to fourth switches SW1 to SW4 are connected between the cathodes of four photoelectric conversion elements PDe (hereinafter, first to fourth photoelectric conversion elements PD1 to PD4), a fifth switch SW5 is connected between the connection node of the first switch SW1 and the second switch SW2 and the connection node of the third switch SW3 and the fourth switch SW4, and a third switch SW6 is connected between the cathode of the first photoelectric conversion element PD1 and the gate of the transistor Q23 of the first logarithmic response unit 21 a. a sixth switch SW6 connected between the cathode of the second photoelectric conversion element PD2 and the gate of the transistor Q23 of the second logarithmic response unit 21 b; a seventh switch SW7 connected between the cathode of the third photoelectric conversion element PD3 and the gate of the transistor Q23 of the third logarithmic response unit 21 c; and a ninth switch SW9 connected between the cathode of the fourth photoelectric conversion element PD4 and the gate of the transistor Q23 of the fourth logarithmic response unit 21 d.

[0075] By turning on the first to sixth switches SW1 to SW6 and turning off the seventh to ninth switches SW7 to SW9, the four EVS pixels 9 are integrated, and an event can be detected by the first event detection circuit 4 subsequent to the first logarithmic response unit 21a connected to the first photoelectric conversion element PD1. In this case, the operation of the first to third event detection circuits 4 subsequent to the second to fourth logarithmic response units 21b, 21c, and 21d connected to the second to fourth photoelectric conversion elements PD2 to PD4 can be stopped, thereby reducing power consumption.

[0076] 9A to 9C, switches SW1 to SW9 for switching the resolution are provided at nodes connected to the photoelectric conversion elements PD or at nodes connected to the event detection circuits 4. Alternatively, the switches may be connected to nodes connected to the logarithmic response units 21. By connecting the switches to nodes connected to the logarithmic response units 21, the responsiveness of the EVS pixels 9 can be adjusted.

[0077] (Switching the Resolution of EVS Pixels 9) In this embodiment, the resolution can be switched for any region of an event image generated by performing event detection on all EVS pixels 9 in the pixel array unit 2. FIG. 10A is a diagram illustrating an example of a subject for which event detection is to be performed. The subject represents an event detection target included in the angle of view at which event detection can be performed by the photodetector 1 according to the first embodiment. FIG. 10A illustrates an example in which the subject includes a moving object Obj1 and a background object Obj2. The moving object Obj1 included in the subject is a target for which event detection is to be performed, whereas the background object Obj2 is an object for which event detection is not required. However, an event for the background object Obj2 (e.g., a tree) may be detected due to the background object Obj2 (e.g., a tree) swaying in the wind.

[0078] 10B shows an event image obtained by performing event detection on the entire subject at high resolution. The event image in FIG. 10B includes not only a moving object Obj1 but also a background object Obj2. Because the event image in FIG. 10B includes the results of event detection performed by all EVS pixels 9 in the pixel array section 2, the amount of data is large, and the power consumption required to generate the event image also increases.

[0079] 10C shows an example in which event detection is performed at high resolution only on the moving object Obj1 in the subject, and at low resolution on the background object Obj2 to generate an event image. The event image generated by Fig. 10C can be generated with low power consumption and a small data volume.

[0080] In this way, in the first embodiment, event detection can be performed at high resolution for only a portion of the subject, and at low resolution for the remainder, so that the amount of data can be reduced without degrading the image quality of the event image, and the event image can be generated with low power consumption.

[0081] The part of the subject and the resolution at which the event is to be detected may be determined using machine learning, or may be determined by the user, or may be determined by other methods.

[0082] 11A is a diagram illustrating a first method for determining the resolution of a subject. The first method in FIG. 11A includes a machine-learned inference device 15 connected to the light detection device 1 according to the first embodiment.

[0083] 11A performs event detection at high resolution for the entire area of ​​the subject and generates an event image. The light detection device 1 inputs the generated event image to the inference device 15. The inference device 15 uses machine learning techniques to infer objects for which event detection should originally be performed from the event image input from the light detection device 1, and returns information about the inferred objects to the light detection device 1. The light detection device 1 performs event detection at high resolution for the area of ​​the object inferred by the inference device 15, and performs event detection at low resolution for other areas.

[0084] Each time the angle of view for event detection is changed, the photodetector 1 performs high-resolution event detection for the entire subject the first time, and thereafter performs high-resolution event detection for only a portion of the subject based on the inference results of the inference device 15, thereby reducing the amount of data in the event image and enabling event detection with low power consumption.

[0085] FIG. 11B is a diagram illustrating a second method for determining the resolution of a subject. In the second method of FIG. 11B , each time the angle of view of the photodetector 1 is changed, an event image is generated by performing event detection at high resolution for the entire subject the first time. The generated event image is displayed, for example, on a display device. The user looks at the event image displayed on the display device and visually specifies an area where event detection should be performed at high resolution. The specified area is notified to the photodetector 1. In accordance with the user's specification, the photodetector 1 performs event detection at high resolution for only a portion of the subject and performs event detection at low resolution for the remaining portion to generate an event image.

[0086] In this way, in the first embodiment, since it is possible to perform event detection by switching the resolution of any pixel region of the subject, it is possible to significantly reduce the amount of data without degrading the image quality of the generated event image, and it is possible to generate the event image with low power consumption. Furthermore, according to the first embodiment, it is possible to switch the resolution of any pixel region of the subject as needed, so even if the location of the event changes over time, it is possible to generate the event image with low power consumption without increasing the amount of data or degrading the image quality.

[0087] Second Embodiment A photodetector 1 according to a second embodiment is characterized in that it detects an event signal including spatiotemporal differential information of a plurality of EVS pixels 9 .

[0088] The photodetector 1 according to the second embodiment has the same block configuration as that shown in Figures 1 and 2. However, the pixel array section 2 does not need to have a hybrid pixel configuration in which gradation pixels 8 and EVS pixels 9 are mixed, and the pixel array section 2 may have EVS pixels 9 without gradation pixels 8. Note that, although an example in which spatiotemporal differential signals of two EVS pixels 9 are generated will be described below, the photodetector 1 according to the second embodiment can also be applied to the case in which spatiotemporal differential signals of three or more EVS pixels 9 are generated.

[0089] (Block configuration for generating spatiotemporal derivatives) Fig. 12A is a block diagram showing a schematic configuration of a photodetector 1 according to the second embodiment. The block diagram of Fig. 12A shows a block configuration related to the processing of the EVS pixels 9, and the processing of the gradation pixels 8 is the same as in the first embodiment, so description thereof will be omitted. The photodetector 1 according to the second embodiment has two logarithmic response units 21a, 21b connected to the two EVS pixels 9, a differentiator 16, a buffer 22, a differentiator 23, a comparator 24, and an output circuit 25.

[0090] The differentiator 16 detects a difference signal between the voltage signals output from the two logarithmic response units 21 a and 21 b. This difference signal corresponds to a spatially differentiated signal between the event images. This spatially differentiated signal is input to the differentiator via a buffer 22.

[0091] The differentiator 23 performs time differentiation on the spatially differentiated signal. As a result, the differentiator 23 generates and outputs a spatiotemporal differentiated signal. The comparator 24 generates an event signal by comparing the spatiotemporal differentiated signal with a predetermined reference signal. In this way, the event signals generated by the photodetector 1 according to the second embodiment are spatiotemporal differentiated signals of the multiple EVS pixels 9. By using the spatiotemporal differentiated signal as the event signal, it is possible to detect a change in luminance of the contour (edge) portion of an object as an event, and to generate an event image that includes only event information of the contour portion and excludes other unnecessary event information. This reduces the data volume of the event image.

[0092] 12A , performing spatiotemporal differentiation requires at least two EVS pixels 9. To perform spatiotemporal differentiation of the event image over the entire area of ​​the pixel array unit 2, for example, a process may be performed in which, for all EVS pixels 9 in the pixel array unit 2, two adjacent EVS pixels 9 generate spatiotemporal differential signals.

[0093] 12B is a block diagram showing a schematic configuration of a photodetector 1 according to a first modified example of the second embodiment. The photodetector 1 according to the first modified example of the second embodiment differs from the first embodiment in the order of processing for performing spatiotemporal differentiation. The photodetector 1 according to the first modified example of the second embodiment includes two logarithmic response units 21 a and 21 b connected to two EVS pixels 9, two buffers 22 a and 22 b, a differentiator 23, a comparator 24, and an output circuit 25.

[0094] In the photodetector 1 according to the first modification of the second embodiment, two voltage signals output from two buffers 22 a and 22 b are input to a differentiator 16 to generate a spatially differentiated signal. The generated spatially differentiated signal is input to a differentiator 23 and time-differentiated. As a result, a spatiotemporal differentiated signal is output from the differentiator 23. The subsequent processing is the same as that shown in FIG. 12A .

[0095] 12C is a block diagram showing a schematic configuration of a photodetector 1 according to a second modification of the second embodiment. The photodetector 1 according to the second modification of the second embodiment includes two logarithmic response units 21 a, 21 b connected to two EVS pixels 9, two buffers 22 a, 22 b, two differentiators 23 a, 23 b, two comparators 24 a, 24 b, two output circuits 25 a, 25 b, and an ExOR circuit 17. The processing order from the two logarithmic response units 21 a, 21 b to the two output circuits 25 a, 25 b is the same as that shown in FIG. 6 .

[0096] Two event signals are output from the two output circuits 25a and 25b. These event signals are, for example, binary digital signals.

[0097] The ExOR circuit 17 calculates the exclusive OR of the two event signals to generate a spatiotemporal differential signal, which is the final event signal.

[0098] In this way, the event signal of the EVS pixel 9 is a digital signal, and to detect the spatial differentiation of a plurality of event signals corresponding to a plurality of EVS pixels 9, the plurality of event signals can simply be input to the ExOR circuit 17. The ExOR circuit 17 outputs 1 when the logic of the plurality of input signals differs, and therefore can easily calculate and output the difference between a plurality of digital signals.

[0099] 12D is a block diagram showing a schematic configuration of a photodetector 1 according to a third modified example of the second embodiment. The photodetector 1 according to the third modified example of the second embodiment includes two logarithmic response units 21 a and 21 b connected to two EVS pixels 9, two buffers 22 a and 22 b, two differentiators 23 a and 23 b, two comparators 24 a and 24 b, a difference calculator 16, and an output circuit 25.

[0100] The difference calculator 16 detects the difference between the two event signals output from the two comparators 24 a and 24 b, thereby generating a spatiotemporal differential signal of the event signal, which is then output from the output circuit 25 as the final event signal.

[0101] 12A to 12D show an example in which spatial differentiation is performed by taking the difference between two EVS pixels 9, but as described above, spatial differentiation may also be performed by taking the difference between three or more EVS pixels 9. The multiple EVS pixels 9 that are the targets of spatial differentiation do not necessarily have to be adjacent to each other. Because the event signal output from the event detection circuit 4 is a binary digital signal, a spatiotemporal differential signal can be easily generated by inputting two event signals corresponding to two EVS pixels 9 to the ExOR circuit 17.

[0102] 13A to 13E are diagrams showing variations in the locations of two EVS pixels 9 corresponding to two event signals input to the ExOR circuit 17. The blocks of EVS pixels 9 shown in FIGS. 13A to 13E include event detection circuits 4, and each block outputs an event signal.

[0103] 13A is a diagram showing a first example of generating a spatiotemporal differential signal using the ExOR circuit 17. In the first example, two event signals output from two EVS pixels 9 are input to the ExOR circuit 17 to generate a spatiotemporal differential signal.

[0104] 13B is a diagram showing a second example of generating a spatiotemporal differential signal using the ExOR circuit 17. In the second example, four event signals output from four EVS pixels 9 are input to the ExOR circuit 17 to generate a spatiotemporal differential signal.

[0105] 13C is a diagram showing a third example of generating a spatiotemporal differential signal using the ExOR circuit 17. In the third example, two event signals output from two non-adjacent EVS pixels 9 in the same pixel row are input to the ExOR circuit 17 to generate a spatiotemporal differential signal.

[0106] 13D is a diagram showing a fourth example of generating a spatiotemporal differential signal using the ExOR circuit 17. In the fourth example, two event signals output from two EVS pixels 9 located in different columns of different pixel rows are input to the ExOR circuit 17 to generate a spatiotemporal differential signal.

[0107] 13E is a diagram showing a fifth example of generating spatiotemporal differential signals using the ExOR circuit 17. In the fifth example, a spatiotemporal differential signal is generated between two adjacent EVS pixels 9 among four adjacent EVS pixels 9 in the same pixel row. This makes it possible to generate three spatiotemporal differential signals.

[0108] As described above, in the second embodiment, an event signal composed of a spatiotemporal differential signal is generated using the EVS pixel 9, so that an event can be detected based on, for example, a change in brightness of the contour (edge) portion of an object. Therefore, an event image can be generated that includes only event information of the contour portion and excludes other unnecessary event information, thereby reducing the data volume of the event image. Furthermore, since a spatiotemporal differential signal can be generated simply by adding a differentiator 16 to an existing event detection circuit 4, fewer design changes to the existing event detection circuit 4 are required. Furthermore, since the differentiator 16 can be placed anywhere within the event detection circuit 4, layout design is simplified. In particular, a spatiotemporal differential signal can be generated simply by inputting multiple event signals to the ExOR circuit 17.

[0109] Third Embodiment In a third embodiment, at least one of resolution, sensitivity, power gating, and responsiveness is switched and controlled for each pixel block BL of an arbitrary size.

[0110] 14 is a diagram illustrating a third embodiment. The photodetector 1 according to the third embodiment includes a plurality of control lines L connected to a plurality of pixel blocks BL, each of which includes a plurality of EVS pixels 9. Each control line L supplies a control signal to the corresponding pixel block BL. The control signal controls at least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block BL. Power gating refers to stopping the operation of some of the event detection circuits 4 to reduce power consumption.

[0111] For example, if each pixel block BL has four EVS pixels 9 and first to ninth switches SW1 to SW9 as shown in Figure 9C, the resolution can be adjusted by controlling the switching of the first to ninth switches SW1 to SW9 using a control signal.

[0112] 15 is a circuit diagram of the front-stage portion of the EVS pixel 9 applicable to the third embodiment. 15 shows the circuit configuration of the logarithmic response unit 21, buffer 22, and differentiator 23 of the EVS pixel 9.

[0113] The control signal transmitted by the control line L in Fig. 14 may be, for example, a bias signal input to the gate of the transistor Q28 in the differentiator 23 in Fig. 15. This allows the sensitivity of the EVS pixel 9 to be adjusted.

[0114] Alternatively, a control signal may be input to the gate of, for example, the transistor Q24 of the logarithmic response unit 21. This allows the response of the EVS pixel 9 to be adjusted.

[0115] Figure 16 is a diagram showing an example in which the resolution is individually controlled by a control signal for each pixel block BL in Figure 14. Figure 16 shows an example in which, of the four pixel blocks BL1 to BL4 in Figure 14, pixel blocks BL1 and BL4 perform event detection at high resolution, while pixel blocks BL2 and BL3 perform event detection at low resolution. In pixel blocks BL2 and BL3, the four EVS pixels 9 in each pixel block BL are integrated and event detection is performed using one of the event detection circuits 4, which reduces resolution but reduces power consumption. In pixel blocks BL1 and BL4, the four EVS pixels 9 in each pixel block BL perform event detection individually, which increases resolution but increases power consumption.

[0116] Note that the control signals transmitted through the control lines L connected to each pixel block BL can be used not only to switch the resolution but also to control the sensitivity, power gating, or responsiveness, so that at least one of the resolution, sensitivity, power gating, and responsiveness can be made different for each pixel block BL.

[0117] FIG. 17A is a circuit diagram showing a first specific example of the third embodiment, and FIG. 17B is a circuit diagram showing a second specific example of the third embodiment.

[0118] The first specific example shown in Fig. 17A has the same circuit configuration as Fig. 9C. The first to ninth switches SW1 to SW9 in Fig. 17A are switched by a control signal transmitted over a control line L. By controlling the switching of the first to ninth switches SW1 to SW9, it is possible to select a mode in which an event is detected individually by four EVS pixels 9, or a mode in which an event is detected by integrating two to four EVS pixels 9.

[0119] 17B has a first switch SW1 to a fifth switch SW5 connected between the gates of the transistors Q29 of the four differentiators 23. By individually controlling and switching these first to fifth switches SW1 to SW5, it is possible to select a mode in which four differentiated signals are generated and a mode in which two to four EVS pixels 9 are integrated to output one differentiated signal.

[0120] When controlling the switching of each switch in Figures 17A and 17B, first, with the switches connecting multiple EVS pixels 9 turned off, event detection is performed on each EVS pixel 9 to generate an event image, and based on the event image, a pixel area where event detection should be performed at high resolution or low resolution is identified, and the switch is switched for each pixel block BL corresponding to the identified pixel area to adjust the resolution, sensitivity, power gating, or responsiveness.

[0121] (Variations of Pixel Block BL) The size and shape of the pixel block BL are arbitrary, and various variations are possible. Representative variations of the pixel block BL will be described below in order.

[0122] 18A is a plan view of a pixel block BL according to Example 1. As shown in Fig. 18A , the pixel block BL according to Example 1 is composed of a plurality of EVS pixels 9 arranged adjacent to each other in a first direction x (row direction), and the number of EVS pixels 9 included in the pixel block BL is arbitrary. Separate pixel blocks BL are arranged in the second direction y (column direction).

[0123] 18B is a plan view of a pixel block BL according to Example 2. As shown in FIG. 18B , the pixel block BL according to Example 2 is composed of a plurality of EVS pixels 9 arranged adjacent to each other in the second direction y (column direction), and the number of EVS pixels 9 included in the pixel block BL is arbitrary. Separate pixel blocks BL are arranged in the first direction x (row direction).

[0124] 18C is a plan view of a pixel block BL according to a third example. As shown in FIG. 18C , the pixel block BL according to the third example is composed of a plurality of EVS pixels 9, each of which is arranged adjacent to the other in a first direction x (row direction) and a second direction y (column direction). In the example of FIG. 18C , the pixel block BL includes 2 × 2 = four EVS pixels 9, but the number of EVS pixels 9 in the first direction x and the second direction y included in the pixel block BL may be any number. Furthermore, the size and shape of the pixel block BL may be set to match the size and shape of a ROI (Region of Interest) included in the event image.

[0125] 18D is a plan view of a pixel block BL according to a fourth example. The EVS pixels 9 in the pixel block BL do not necessarily need to be adjacent to each other and may be dispersed. Each of the pixel blocks BL1 and BL2 according to the fourth example includes EVS pixels 9 dispersed in a first direction x (row direction) and a second direction y (column direction). In the fourth example, the pixel blocks BL1 and BL2 include a plurality of EVS pixels 9 disposed in a staggered pattern in the first direction x and the second direction y, but the pixel positions and the number of EVS pixels 9 included in the pixel blocks BL1 and BL2 are arbitrary.

[0126] 18E is a plan view of a pixel block BL according to the fifth example. The pixel block BL according to the fifth example includes a plurality of gradation pixels 8 and one EVS pixel 9. The pixel array unit 2 may also include a plurality of types of pixel blocks BL with different properties.

[0127] 18E shows an example in which the pixel array unit 2 includes a plurality of first pixel blocks BL1 and a plurality of second pixel blocks BL2. The first pixel block BL1 includes, for example, three gradation pixels 8 and one EVS pixel 9, and the three gradation pixels 8 are the same color. The threshold for event detection for the EVS pixel 9 is set according to the colors of the other three gradation pixels 8. The plurality of first pixel blocks BL1 include one including three red gradation pixels 8 and one EVS pixel 9, one including three green gradation pixels 8 and one EVS pixel 9, and one including three blue gradation pixels 8 and one EVS pixel 9. The size and shape of the plurality of second pixel blocks BL2 are variably adjusted depending on, for example, the size and shape of the ROI. A plurality of switches are connected to a plurality of EVS pixels 9 in the second pixel block BL2, for example, as shown in any of Figures 9A to 9C, and by controlling the switching of these switches, the resolution of the EVS pixels 9 can be changed to match the size and shape of the ROI.

[0128] (Variations of Hybrid Pixels) When a pixel block BL has hybrid pixels that are a mixture of gradation pixels 8 and EVS pixels 9, various variations can be considered for the arrangement of the gradation pixels 8 and EVS pixels 9 that constitute the hybrid pixel. Representative variations of hybrid pixels will be described below in order.

[0129] 19A is a plan view showing a hybrid pixel according to Example 1. The pixel block BL according to Example 1 has a pixel size of 2×2=4 pixels, three of which are gradation pixels 8 and the remaining one is an EVS pixel 9.

[0130] 19B is a plan view showing a hybrid pixel according to the second example. The pixel block BL according to the second example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 includes 2 × 2 = four small pixel blocks BL (hereinafter referred to as the first pixel blocks BL1). Three of the four first pixel blocks BL1 include four grayscale pixels 8 of different colors. The remaining first pixel block BL1 includes four EVS pixels 9.

[0131] FIG. 19C is a plan view showing a hybrid pixel according to a third example. The pixel block BL according to the third example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 includes 2 × 2 = four small pixel blocks BL (hereinafter referred to as the first pixel blocks BL1). Each of the four first pixel blocks BL1 includes three or four gradation pixels 8, and the gradation pixels 8 within one first pixel block BL1 are the same color. Specifically, the second pixel block BL2 includes a first pixel block BL1 including four red gradation pixels 8, a first pixel block BL1 including four green gradation pixels 8, a first pixel block BL1 including three green gradation pixels 8 and one EVS pixel 9, and a first pixel block BL1 including three blue gradation pixels 8 and one EVS pixel 9.

[0132] 19D is a plan view showing a hybrid pixel according to a fourth example. The pixel block BL according to the fourth example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 includes 2 × 2 = 4 small pixel blocks BL (hereinafter referred to as the first pixel block BL1). Each of the four first pixel blocks BL1 includes three gradation pixels 8 and one EVS pixel 9. The second pixel block BL2 includes two first pixel blocks BL1 each including three green gradation pixels 8 and one EVS pixel 9, a first pixel block BL1 including three red gradation pixels 8 and one EVS pixel 9, and a first pixel block BL1 including three blue gradation pixels 8 and one EVS pixel 9.

[0133] As described above, in the third embodiment, a control line L is connected to each pixel block BL, and at least one of the resolution, sensitivity, power gating, and responsiveness of each pixel block BL can be controlled by a control signal supplied to each pixel block BL via the control line L. This allows, for example, at least one of the resolution, sensitivity, power gating, and responsiveness to be individually adjusted for each pixel block BL. The location, size, and shape of the pixel block BL are arbitrary. For example, a pixel block BL set to match the location, size, and shape of an ROI included in an event image can be set to a different resolution, sensitivity, power gating, or responsiveness from other pixel blocks BL. Furthermore, the pixel block BL may include hybrid pixels. In this case, the resolution, sensitivity, power gating, or responsiveness of the EVS pixels 9 in the pixel block BL may be set according to the color of the gradation block within the pixel block BL.

[0134] As described above, the photodetector 1 according to the present disclosure can have a stacked structure in which a plurality of chips (substrates) are stacked. In this case, there are many variations in which circuit parts of the photodetector 1 are arranged on which chips.

[0135] 6 described above shows an example in which the EVS pixels 9 and the event detection circuit 4 are arranged separately on the first substrate SB1 and the second substrate SB2. Below, examples (first to sixth modified examples) in which the EVS pixels 9 and the event detection circuit 4 are arranged separately on the first substrate SB1 and the second substrate SB2 in a manner different from that shown in FIG. 6 will be described in order. Note that in the first to sixth modified examples described below, when not only the EVS pixels 9 but also the gradation pixels 8 are arranged in the pixel array section 2, the allocation of the gradation pixels 8 to the first substrate SB1 and the second substrate SB2 is arbitrary.

[0136] 20A is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the first modified example of Fig. 6. In the first modified example shown in Fig. 20A, the photoelectric conversion element PDe for the EVS pixel 9 is arranged on the first substrate SB1, and the subsequent circuits including the logarithmic response unit 21 are arranged on the second substrate SB2. This allows the size of each photoelectric conversion element PDe on the first substrate SB1 to be increased, thereby improving sensitivity.

[0137] 20B is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the second modified example of FIG. 6. In the second modified example shown in FIG. 20B, the photoelectric conversion element PDe for the EVS pixel 9 and a portion of the circuitry of the logarithmic response unit 21 are arranged on the first substrate SB1. For example, the transistors Q21 to Q23 of the logarithmic response unit 21 are arranged on the first substrate SB1, and the transistor Q24 of the logarithmic response unit 21 is arranged on the second substrate SB2. In addition, the downstream circuits from the buffer 22 onwards are also arranged on the second substrate SB2.

[0138] 20C is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the third modified example of Fig. 6. In the third modified example shown in Fig. 20C, the photoelectric conversion element PDe for the EVS pixel 9, the logarithmic response unit 21, and the buffer 22 are arranged on the first substrate SB1, and the subsequent circuits from the differentiator 23 onwards are arranged on the second substrate SB2.

[0139] Fig. 20D is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the fourth modification of Fig. 6. In the fourth modification shown in Fig. 20D, the photoelectric conversion element PDe for the EVS pixel 9, the logarithmic response unit 21, the buffer 22, and the capacitor C1 in the differentiator 23 are arranged on the first substrate SB1, and the transistors Q27 to Q29 and the switch 26 in the differentiator 23, and the subsequent circuits from the comparator 24 onwards are arranged on the second substrate SB2.

[0140] Fig. 20E is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the fifth modification of Fig. 6. In the fifth modification shown in Fig. 20E, the photoelectric conversion element PDe for the EVS pixel 9, the logarithmic response unit 21, the buffer 22, and the differentiator 23 are arranged on the first substrate SB1, and the subsequent circuits from the comparator 24 onwards are arranged on the second substrate SB2.

[0141] Fig. 20F is a circuit diagram showing the locations of the EVS pixel 9 and the event detection circuit 4 according to the sixth modified example of Fig. 6. In the fifth modified example shown in Fig. 20E, the photoelectric conversion element PDe for the EVS pixel 9, the logarithmic response unit 21, the buffer 22, the differentiator 23, and the comparator 24 are arranged on the first substrate SB1, and the downstream circuits from the output circuit 25 onwards are arranged on the second substrate SB2.

[0142] Various variations are possible for the layout arrangement of the first substrate SB1 and the second substrate SB2 when the pixel array unit 2 includes gradation pixels 8 and EVS pixels 9. Fig. 21A is a diagram showing a first example of the layout arrangement. In the first example shown in Fig. 21A, a plurality of photoelectric conversion elements PDi and pixel circuits 13 for a plurality of gradation pixels 8, and a plurality of photoelectric conversion elements PDe and a plurality of logarithmic response units 21 for a plurality of EVS pixels 9 are arranged on the first substrate SB1. In the example of Fig. 21A, the gradation pixels 8 and the EVS pixels 9 are arranged in a two-dimensional direction at a ratio of one EVS pixel 9 to three gradation pixels 8.

[0143] On the second substrate SB2, an event detection circuit 4, an ADC for the gradation pixels 8, a readout circuit, a digital circuit 5 for EVS, a control circuit 6, a bias circuit 7, and the like are arranged.

[0144] 21B is a diagram showing a second example of the layout arrangement. On the first substrate SB1 according to the second example, a plurality of gradation pixels 8 and a plurality of EVS pixels 9 are arranged at a ratio of one EVS pixel 9 for every 15 gradation pixels 8. On the second substrate SB2, an ADC for the gradation pixels 8, a readout circuit, an EVS digital circuit 5, a control circuit 6, a bias circuit 7, and the like are arranged. In FIG. 21B, the ratio of EVS pixels 9 is smaller than in FIG. 21A, so the size of the event detection circuit 4 on the second substrate SB2 can be reduced.

[0145] In this way, when the photodetector 1 has a two-layer stacked structure, any circuit portion of the photodetector 1 according to the first to third embodiments can be allocated to the first substrate SB1 and the second substrate SB2. Furthermore, when the photodetector 1 has a three-layer stacked structure, any circuit portion of the photodetector 1 according to the first to third embodiments can be allocated to the first to third substrates SB1 to SB3. The bonding and signal transmission between the first substrate SB1 and the second substrate SB2, and the bonding and signal transmission between the second substrate SB2 and the third substrate SB3 can be performed using CCC, vias, bumps, etc.

[0146] (Application Examples) The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, construction machinery, or agricultural machinery (tractor).

[0147] 22 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 22 , the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard, such as a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), or FlexRay (registered trademark).

[0148] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a memory unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various controlled devices. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle via wired or wireless communication. Figure 22 illustrates the functional configuration of the integrated control unit 7600, including a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a memory unit 7690. The other control units also include a microcomputer, a communication I / F, a memory unit, and the like.

[0149] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating drive force for the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating braking force for the vehicle. The drivetrain control unit 7100 may also function as a control device for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.

[0150] A vehicle state detection unit 7110 is connected to the drivetrain control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor that detects the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine rotation speed, the rotation speed of the wheels, etc. The drivetrain control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0151] The body system control unit 7200 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0152] The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, in accordance with various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

[0153] The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0154] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

[0155] 23 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0156] 23 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 7910, 7912, 7914, and 7916, a bird's-eye view image of the vehicle 7900 viewed from above can be obtained.

[0157] The outside vehicle information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The outside vehicle information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside vehicle information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

[0158] Returning to FIG. 22 , the explanation will be continued. The outside vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside vehicle information detection unit 7400 also receives detection information from the connected outside vehicle information detection unit 7420. If the outside vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside vehicle information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. Based on the received information, the outside vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, text on the road, etc. Based on the received information, the outside vehicle information detection unit 7400 may also perform environment recognition processing for recognizing rainfall, fog, road conditions, etc. Based on the received information, the outside vehicle information detection unit 7400 may also calculate the distance to an object outside the vehicle.

[0159] The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, vehicles, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by combining image data captured by different image capturing units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different image capturing units 7410.

[0160] The interior information detection unit 7500 detects information inside the vehicle. A driver state detection unit 7510 that detects the driver's state is connected to the interior information detection unit 7500, for example. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound from within the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or the driver gripping the steering wheel. The interior information detection unit 7500 may calculate the driver's level of fatigue or concentration based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The interior information detection unit 7500 may perform processing such as noise canceling on the collected audio signal.

[0161] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 may be implemented by a device that can be operated by a passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input through a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a personal digital assistant (PDA) that can operate the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information using gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and the like operate this input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0162] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0163] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (Global System of Mobile communications), WiMAX (registered trademark), LTE (Long Term Evolution), or LTE-Advanced (LTE-A), or other wireless communication protocols such as a wireless LAN (also referred to as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I / F 7620 may connect to a terminal located near the vehicle (for example, a terminal of a driver, pedestrian, or store, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.

[0164] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in vehicles. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE 802.11p and an upper layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0165] The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites), and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

[0166] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closures, required travel time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0167] The in-vehicle device I / F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish wireless connections using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). Furthermore, the in-vehicle device I / F 7660 may establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0168] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0169] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various programs based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output control commands to the drivetrain control unit 7100. For example, the microcomputer 7610 may perform cooperative control aimed at realizing functions of an Advanced Driver Assistance System (ADAS), including vehicle collision avoidance or impact mitigation, following driving based on the following distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc. based on information acquired about the vehicle's surroundings.

[0170] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including information about the vicinity of the vehicle's current location, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. Furthermore, the microcomputer 7610 may predict dangers, such as a vehicle collision, the approach of a pedestrian, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0171] The audio / image output unit 7670 transmits at least one audio and / or image output signal to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information. In the example of FIG. 22 , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may also be other devices, such as headphones, a wearable device such as an eyeglass-type display worn by the occupant, a projector, or a lamp. When the output device is a display device, the display device visually displays results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats, such as text, images, tables, and graphs. When the output device is an audio output device, the audio output device converts audio signals, such as reproduced audio data or acoustic data, into analog signals and audibly outputs the analog signals.

[0172] In the example shown in FIG. 22 , at least two control units connected via the communication network 7010 may be integrated into a single control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In addition, in the above description, some or all of the functions performed by one of the control units may be assigned to another control unit. In other words, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by one of the control units. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

[0173] The present technology may be configured as follows: (1) A photodetector comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels comprising: a first photoelectric conversion element that accumulates a charge according to the amount of incident light; and an event detection circuit that detects an event due to a luminance change based on the charge, and a pixel block including two or more first pixels among the plurality of first pixels has a switch that switches whether or not internal nodes of the two or more first pixels are connected to each other. (2) The photodetector according to (1), each of the plurality of first pixels comprising a logarithmic response unit that generates a voltage signal based on the charge, the event detection circuit detects the event based on the voltage signal, and the internal node is a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit. (3) The photodetector according to (1) or (2), wherein the two or more first pixels detect the event by integrating the two or more first pixels by turning on the switch and connecting internal nodes of the two or more first pixels. (4) The photodetector according to (3), wherein the resolution for detecting the event is reduced by turning on the switch and connecting internal nodes of the two or more first pixels. (5) The photodetector according to (4), wherein a plurality of pixel blocks each having the switch are provided, and wherein switching control of the corresponding switch is performed for each of the plurality of pixel blocks based on a result of detecting the event for the plurality of first pixels with a plurality of the switches corresponding to the plurality of pixel blocks turned off. (6) The photodetector according to any one of (1) to (5), further comprising an output circuit that outputs spatiotemporal differential signals of two or more of the first pixels among the plurality of first pixels as the event.(7) A photodetector comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels comprising: a first photoelectric conversion element that accumulates a charge according to the amount of incident light; and an event detection circuit that detects an event due to a change in luminance based on the charge, wherein the event detection circuit comprises an output circuit that outputs spatiotemporal differential signals between two or more of the plurality of first pixels as the event. (8) The photodetector according to (6) or (7), wherein the spatiotemporal differential signals are generated by temporally differentiating a spatial differential signal generated by spatially differentiating two of the first pixels. (9) The photodetector according to (8), wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that temporally differentiates an output voltage of the buffer circuit; and a comparator that detects the event by comparing an output signal of the differentiator with a reference signal, wherein the spatial differential signal is generated by detecting a difference between output signals of the two buffer circuits between two of the first pixels. (10) The photodetector according to (8), wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that time-differentiates the output voltage of the buffer circuit; and a comparator that compares the output signal of the differentiator with a reference signal to detect the event, and the spatially differentiated signal is generated by detecting a difference between output signals of the two differentiators for two of the first pixels. (11) The photodetector according to (6) or (7), wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that time-differentiates the output voltage of the buffer circuit; a comparator that compares the output signal of the differentiator with a reference signal to detect the event, and a calculator that generates the spatiotemporal differentiated signal by calculating an exclusive OR of the output signals of the two comparators.(12) The photodetector according to any one of (1) to (11), wherein the plurality of first pixels are divided into a plurality of pixel blocks each including two or more of the first pixels, and a control line for controlling at least one of resolution, sensitivity, power gating, or responsiveness is connected to each of the plurality of pixel blocks. (13) The photodetector according to (12), wherein each of a plurality of control signals transmitted through a plurality of the control lines connected to the plurality of pixel blocks controls at least one of resolution, sensitivity, power gating, or responsiveness of the corresponding pixel block. (14) The photodetector according to (13), wherein the control signal controls a threshold for detecting the event in each of the two or more first pixels in the corresponding pixel block. (15) The photodetector according to (13), wherein the control signal switches between whether or not internal nodes of the two or more first pixels included in the corresponding pixel block are connected to each other. (16) The photodetector according to any one of (1) to (15), comprising a plurality of second pixels arranged in a one-dimensional or two-dimensional direction, each of the plurality of second pixels having: a second photoelectric conversion element that accumulates a charge according to the amount of incident light; and a pixel circuit that generates a pixel signal according to the amount of incident light based on the charge. (17) The photodetector according to (16), wherein the plurality of first pixels are disposed in a dispersed manner between the plurality of second pixels. (18) The photodetector according to (17), wherein one or more first pixels are disposed for each pixel block including a plurality of the second pixels adjacent to each other in a first direction and a second direction that intersect with each other.(19) A photodetection device as described in (17) or (18), comprising: first pixel blocks each including 2 x 2 = four second pixels adjacent to each other in a first direction and a second direction that intersect with each other; and second pixel blocks arranged two by two in the first direction and the second direction; and of the 2 x 2 = four first pixel blocks included in the second pixel block, one of two first pixel blocks arranged diagonally includes four green second pixels, and the other includes three green second pixels and one first pixel; and one of the remaining two first pixel blocks includes three blue second pixels and one first pixel, and the other includes four red second pixels. (20) The photodetector device according to (17) or (18), further comprising: first pixel blocks each including 2 x 2 = four second pixels adjacent in a first direction and a second direction intersecting each other, and second pixel blocks arranged two adjacent to each other in the first direction and the second direction; of the 2 x 2 = four first pixel blocks included in the second pixel block, two first pixel blocks arranged diagonally each include three green second pixels and one first pixel, one of the remaining two first pixel blocks includes three blue second pixels and one first pixel, and the other includes three red second pixels and one first pixel; and the four first pixels included one by one in each of the four first pixel blocks are arranged adjacent to each other in the first direction and the second direction.

[0174] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0175] REFERENCE SIGNS LIST 1 Photodetector, 2 Pixel array unit, 4 Event detection circuit, 5 EVS digital circuit, 6 Control circuit, 7 Bias circuit, 8 Gradation pixel, 9 EVS pixel, 12 AFE circuit, 12L AFE circuit, 13 Pixel circuit, 15 Inference device, 16 Differentiator, 17 ExOR circuit, 21 Logarithmic response unit, 21a First logarithmic response unit, 21b Second logarithmic response unit, 21c Third logarithmic response unit, 21d Fourth logarithmic response unit, 22 Buffer, 23 Differentiator, 24 Comparator, 25 Output circuit, 26 Switch, 27 Logic circuit, 30 Electronic device, 31 Imaging lens, 32 Image processing unit, 33 Recording unit, 34 Control unit

Claims

1. A photodetection device comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels comprising: a first photoelectric conversion element that accumulates an electric charge according to the amount of incident light; and an event detection circuit that detects an event due to a luminance change based on the electric charge, and a pixel block including two or more first pixels among the plurality of first pixels has a switch that switches whether or not internal nodes of the two or more first pixels are connected to each other.

2. The photodetection device of claim 1, wherein each of the plurality of first pixels has a logarithmic response unit that generates a voltage signal based on the charge, the event detection circuit detects the event based on the voltage signal, and the internal node is a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit.

3. The photodetection device according to claim 1, wherein the two or more first pixels detect the event by turning on the switch and connecting internal nodes of the two or more first pixels to each other, thereby integrating the two or more first pixels.

4. The photodetection device according to claim 3, wherein the switch is turned on to connect internal nodes of each of the two or more first pixels to each other, thereby reducing a resolution for detecting the event.

5. The photodetection device according to claim 4, further comprising: a plurality of pixel blocks each having the switch; and, with a plurality of switches corresponding to the plurality of pixel blocks turned off, switching control of the corresponding switch is performed for each of the plurality of pixel blocks based on the result of detecting the event for the plurality of first pixels.

6. The photodetection device according to claim 1, further comprising an output circuit that outputs spatiotemporal differential signals between two or more of the plurality of first pixels as the event.

7. A photodetection device comprising a plurality of first pixels arranged in one or two dimensions, each of the plurality of first pixels having a first photoelectric conversion element that accumulates an electric charge according to the amount of incident light, and an event detection circuit that detects an event due to a luminance change based on the electric charge, the event detection circuit having an output circuit that outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event.

8. The light detection device according to claim 6, wherein the spatiotemporal differential signal is generated by performing temporal differentiation on a spatial differential signal generated by performing spatial differentiation between the two first pixels.

9. The photodetection device according to claim 8, wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that performs time differentiation on the output voltage of the buffer circuit; and a comparator that detects the event by comparing the output signal of the differentiator with a reference signal; and wherein the spatial differential signal is generated by detecting a difference between output signals of the two buffer circuits between two of the first pixels.

10. The photodetection device according to claim 8, wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that performs time differentiation on the output voltage of the buffer circuit; and a comparator that detects the event by comparing the output signal of the differentiator with a reference signal; and the spatial differential signal is generated by detecting a difference between output signals of the two differentiators between two of the first pixels.

11. The photodetection device of claim 6, wherein the event detection circuit comprises: a buffer circuit that buffers a voltage signal obtained by logarithmically converting the charge; a differentiator that performs time differentiation on the output voltage of the buffer circuit; a comparator that detects the event by comparing the output signal of the differentiator with a reference signal; and an operator that generates the time-space differential signal by calculating the exclusive OR of the output signals of the two comparators.

12. The photodetection device according to claim 1, wherein the plurality of first pixels are divided into a plurality of pixel blocks, each of which includes two or more of the first pixels, and each of the plurality of pixel blocks is connected to a control line that controls at least one of resolution, sensitivity, power gating, or responsiveness.

13. The photodetection device according to claim 12, wherein each of a plurality of control signals transmitted through a plurality of control lines connected to a plurality of pixel blocks controls at least one of the resolution, sensitivity, power gating, or responsiveness of the corresponding pixel block.

14. The optical detection device of claim 13, wherein the control signal controls a threshold for detecting the event at each of the two or more first pixels in the corresponding pixel block.

15. The photodetection device according to claim 13, wherein the control signal performs switching control to determine whether or not internal nodes of the two or more first pixels included in the corresponding pixel block are connected to each other.

16. The photodetection device according to claim 1, comprising a plurality of second pixels arranged in one or two dimensions, each of the plurality of second pixels having: a second photoelectric conversion element that accumulates an electric charge corresponding to the amount of incident light; and a pixel circuit that generates a pixel signal corresponding to the amount of incident light based on the electric charge.

17. The light detection device according to claim 16, wherein the plurality of first pixels are disposed in a dispersed manner between the plurality of second pixels.

18. The light detection device according to claim 17, wherein one or more of the first pixels are arranged for each pixel block including a plurality of the second pixels adjacent to each other in a first direction and a second direction that intersect with each other.

19. The photodetection device of claim 17, comprising second pixel blocks arranged two by two in the first and second directions, each first pixel block including 2 x 2 = four of the second pixels adjacent to each other in a first direction and a second direction that intersect with each other, and wherein of the 2 x 2 = four first pixel blocks included in the second pixel block, one of two first pixel blocks arranged in a diagonal direction includes four green second pixels and the other includes three green second pixels and one first pixel, and one of the remaining two first pixel blocks includes three blue second pixels and one first pixel, and the other includes four red second pixels.

20. The photodetection device of claim 17, comprising second pixel blocks arranged two by two in the first direction and the second direction, each first pixel block including 2 x 2 = four of the second pixels adjacent to each other in a first direction and a second direction that intersect with each other, wherein of the 2 x 2 = four of the first pixel blocks included in the second pixel block, two first pixel blocks arranged in a diagonal direction each include three green second pixels and one first pixel, one of the remaining two first pixel blocks includes three blue second pixels and one first pixel, and the other includes three red second pixels and one first pixel, and the four first pixels included in each of the four first pixel blocks are arranged adjacent to each other in the first direction and the second direction.

Images