Light detection device

Abstract

[Problem] To provide a light detection device capable of reducing power consumption and the amount of data. [Solution] This light detection device is provided with a plurality of first pixels arranged in a one-dimensional direction or a two-dimensional direction. Each of the plurality of first pixels is provided with a first photoelectric conversion element for accumulating charges corresponding to the amount of incident light; and an event detection circuit for detecting an event caused by a brightness change based on the charge. A pixel block including two or more first pixels among the plurality of first pixels has a switch for switching whether or not internal nodes of the two or more first pixels are connected to each other.

Description

Technical Field

[0001] This disclosure relates to optical detection devices. Background Technology

[0002] An event-based vision sensor (EVS) has been proposed that detects events such as changes in brightness in a camera scene. Compared to frame-type vision sensors (e.g., complementary metal-oxide-semiconductor (CMOS) image sensors) that scan all pixels at predetermined time intervals to obtain grayscale information of each pixel, the EVS is characterized by its ability to detect events with low power and high speed (see Patent Document 1). List of existing technologies Patent documents

[0003] Patent Document 1: WO 2019 / 087471 A Summary of the Invention The problem the invention aims to solve

[0004] Patent document 1 discloses a technique for reducing the circuit size of each chip by forming the EVS into a stacked structure of multiple chips.

[0005] However, a problem exists: conventional EVS has a fixed resolution for event detection, and even when an event occurs in only a small portion of the image, it performs event detection across the entire image area, resulting in high power consumption. Furthermore, since event detection is also performed on the background of the image, the amount of data unnecessarily increases, and the possibility of noise in the events increases.

[0006] Therefore, this disclosure provides a light detection device that can reduce power consumption and data volume. Solution to the problem

[0007] To address the aforementioned problems, this disclosure provides a light detection device, comprising: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and A pixel block that includes two or more first pixels among the plurality of first pixels includes: A switch that toggles whether the internal nodes of two or more first pixels are connected to each other.

[0008] Each of the plurality of first pixels may include a logarithmic response unit, which generates a voltage signal based on the charge. The event detection circuit can detect the event based on the voltage signal, and Each of the internal nodes may be a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit.

[0009] The two or more first pixels can turn on the switch and connect the internal nodes of the two or more first pixels to integrate the two or more first pixels and detect the event.

[0010] The switch can be turned on to connect the internal nodes of the two or more first pixels to each other and reduce the resolution of the event detection.

[0011] The optical detection device may further include: Multiple pixel blocks, each pixel block containing the switch, wherein, While the multiple switches corresponding to the multiple pixel blocks are truncated, the corresponding switches are switched for each of the multiple pixel blocks based on the detection results of the events for the multiple first pixels.

[0012] The optical detection device may further include an output circuit that outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event.

[0013] This disclosure also provides a light detection device, which includes: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and The event detection circuit includes an output circuit, which outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event.

[0014] The spatiotemporal differential signal can be generated by temporally differentiating the spatial differential signal generated by performing spatial differentiation between the two first pixels.

[0015] The event detection circuit may include: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal can be generated by detecting the difference between the output signals of the two buffer circuits of the two first pixels.

[0016] The event detection circuit may include: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal can be generated by detecting the difference between the output signals of the two differentiators of the two first pixels.

[0017] The event detection circuit may include: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; A comparator that compares the output signal of the differentiator with a reference signal and detects the event; and A calculator that generates the spatiotemporal differential signal by calculating the XOR of the output signals of the two comparators.

[0018] The plurality of first pixels can be divided into a plurality of pixel blocks, and each pixel block includes two or more first pixels from the plurality of first pixels. Control lines controlling at least one of resolution, sensitivity, power gating, and responsiveness can be connected to each pixel block among the plurality of pixel blocks.

[0019] Each of the multiple control signals transmitted via the multiple control lines connected to the multiple pixel blocks can control at least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block.

[0020] The control signal can control the threshold for detecting the event in each of the two or more first pixels in the corresponding pixel block.

[0021] The control signal can switch whether to connect the internal nodes of two or more first pixels contained in the corresponding pixel block.

[0022] The optical detection device may further include: Multiple second pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of second pixels may include: The second photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and A pixel circuit that generates a pixel signal corresponding to the amount of incident light based on the charge.

[0023] The plurality of first pixels can be arranged in a dispersed manner among the plurality of second pixels.

[0024] For each pixel block comprising a plurality of second pixels that are adjacent to each other in a first direction and a second direction that intersect each other, one or more of the plurality of first pixels may be arranged as first pixels.

[0025] The optical detection device may further include: The second pixel block comprises two adjacent first pixel blocks arranged in a first direction and a second direction that intersect each other. Each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction. In the second pixel block containing 2 × 2 = 4 first pixel blocks, one of the two first pixel blocks arranged diagonally may contain four green second pixels and the other may contain three green second pixels and one first pixel. In the remaining two first pixel blocks, one may contain three blue second pixels and one first pixel, and the other may contain four red second pixels.

[0026] The optical detection device may further include: The second pixel block comprises two adjacent first pixel blocks arranged in a first direction and a second direction that intersect each other. Each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction. In the second pixel block comprising 2 × 2 = 4 first pixel blocks, each of the two first pixel blocks arranged diagonally may contain three green second pixels and one first pixel; one of the remaining two first pixel blocks may contain three blue second pixels and one first pixel, and the other may contain three red second pixels and one first pixel. The four first pixels, each comprising one of the four first pixel blocks, can be arranged adjacent to each other in the first direction and the second direction. Attached Figure Description

[0027] Figure 1This is a block diagram of an electronic device according to a first embodiment of the present disclosure. Figure 2 This is a block diagram illustrating a schematic construction of a light detection device according to a first embodiment. Figure 3 This is a schematic perspective view showing an example of a light detection device according to the present disclosure having a double-layer stacked structure. Figure 4A This diagram shows an example of the arrangement of grayscale pixels (gradation pixels) and EVS pixels in the pixel array section of the first substrate. Figure 4B This is a diagram illustrating an example of setting up AFE circuits for each first pixel block and integrating multiple AFE circuits into one unit. Figure 5 This is a circuit diagram showing the circuit construction of grayscale pixels. Figure 6 This is a circuit diagram showing the internal structure of the EVS pixel and event detection circuit. Figure 7 This is a diagram illustrating an example of a photodetector having a stacked structure of a first substrate, a second substrate, and a third substrate. Figure 8 This is a circuit diagram showing an example of a logarithmic response section and a portion of a buffer arranged on a second substrate. Figure 9A This is a circuit diagram of a first example of switching the resolution of EVS pixels. Figure 9B This is a circuit diagram of a second example for switching the resolution of EVS pixels. Figure 9C This is a circuit diagram of the third example of switching the resolution of EVS pixels. Figure 10A This is a diagram showing an example of a subject to which event detection is to be performed. Figure 10B This is a diagram showing an event image obtained by detecting events in the entire area of ​​the subject at high resolution. Figure 10C This is a diagram illustrating an example of performing event detection at high resolution only on the portion corresponding to the moving body included in the subject. Figure 11A This is a diagram used to illustrate a first method for determining the resolution of a subject. Figure 11B This is a diagram used to illustrate a second method for determining the resolution of a subject. Figure 12A This is a block diagram illustrating a schematic construction of a light detection device according to a second embodiment. Figure 12BThis is a block diagram illustrating a schematic construction of a light detection device according to a first variation of the second embodiment. Figure 12C This is a block diagram illustrating a schematic construction of a light detection device according to a second variation of the second embodiment. Figure 12D This is a block diagram illustrating a schematic structure of a light detection device according to a third variation of the second embodiment. Figure 13A This is a diagram illustrating a first example of generating a spatiotemporal differential signal using an ExOR circuit. Figure 13B This is a diagram illustrating a second example of generating a spatiotemporal differential signal using an ExOR circuit. Figure 13C This is a diagram illustrating a third example of generating a spatiotemporal differential signal using an ExOR circuit. Figure 13D This is a diagram illustrating a fourth example of generating a spatiotemporal differential signal using an ExOR circuit. Figure 13E This is a diagram illustrating the fifth example of generating a spacetime differential signal using an ExOR circuit. Figure 14 This is a diagram illustrating the third implementation scheme. Figure 15 This is a circuit diagram of the front-end section of the EVS pixel applicable to the third implementation scheme. Figure 16 It shows the target Figure 14 An example diagram showing how the resolution of each pixel block is individually controlled by a control signal. Figure 17A This is a circuit diagram illustrating a first specific example of the third implementation scheme. Figure 17B This is a circuit diagram illustrating a second specific example of the third implementation scheme. Figure 18A It is a planar diagram based on the pixel blocks of the first example. Figure 18B It is a planar diagram of the pixel blocks based on the second example. Figure 18C It is a planar diagram based on the pixel blocks of the third example. Figure 18D It is a planar diagram of the pixel blocks based on the fourth example. Figure 18E It is a planar diagram of the pixel blocks based on the fifth example. Figure 19A This is a planar diagram showing a hybrid pixel based on the first example. Figure 19B This is a planar diagram showing the mixed pixels according to the second example. Figure 19C This is a planar diagram showing the mixed pixels according to the third example. Figure 19D This is a planar diagram showing the mixed pixels according to the fourth example. Figure 20A It shows according to Figure 6 The circuit diagram showing the arrangement of the EVS pixel and event detection circuit in the first variant. Figure 20B It shows according to Figure 6 The circuit diagram shows the arrangement of the EVS pixel and event detection circuit in the second variation. Figure 20C It shows according to Figure 6 The circuit diagram shows the arrangement of the EVS pixel and event detection circuit in the third variation. Figure 20D It shows according to Figure 6 The circuit diagram shows the arrangement of the EVS pixel and event detection circuit in the fourth variation. Figure 20E It shows according to Figure 6 The circuit diagram shows the arrangement of the EVS pixel and event detection circuit in the fifth variation. Figure 20F It shows according to Figure 6 The circuit diagram showing the arrangement of the EVS pixel and event detection circuit in the sixth variation. Figure 20F This is a diagram showing the first example of a layout arrangement. Figure 6 This is a diagram showing a second example of the layout arrangement. Figure 21A This is a block diagram illustrating an example of a schematic construction of a vehicle control system. Figure 21B This is an explanatory diagram showing an example of the installation location of the vehicle exterior information detection unit and the camera unit. Detailed Implementation

[0028] In the following description, embodiments of the light detection device and electronic device will be illustrated with reference to the accompanying drawings. The main components of the light detection device and electronic device will be described primarily below; however, the light detection device may include components and functions not shown or described. The following description is not intended to exclude components and functions not shown or described.

[0029] Figure 22 This is a block diagram of an electronic device 30 according to a first embodiment of the present disclosure. The electronic device 30 has the function of generating an image based on the brightness of the incident light. Figure 23The electronic device 30 includes a light detection device 1, a camera lens 31, an image processing unit 32, a storage unit 33, and a control unit 34. The electronic device 30 can be used as, for example, a surveillance camera, a camera mounted on an industrial robot, a general-purpose camera, etc., but it can also be used in any specific application and configuration of the electronic device 30.

[0030] The camera lens 31 converges the incident light and guides it to the light detection device 1. The light detection device 1 captures an image of the incident light. The light detection device 1 has the function of photoelectric conversion of light (such as visible light or infrared light) within a predetermined wavelength range to generate a captured image. The captured image generated by the light detection device 1 is sent to the image processing unit 32 and the storage unit 33.

[0031] The image processing unit 32 performs predetermined image processing on the captured image, such as color or brightness adjustment, image compression, image recognition, tracking, or analysis. The image processed by the image processing unit 32 is output to the storage unit 33, for example.

[0032] The storage unit 33 stores images output from the light detection device 1 or the image processing unit 32. The storage unit 33 can be located in a server or the like connected via a network. In the electronic device 30 according to this embodiment, this storage unit can be omitted. Figure 1 At least one of the image processing unit 32 and the storage unit 33.

[0033] The control unit 34 controls the operation of the light detection device 1. Furthermore, although... Figure 1 Although not clearly shown, the control unit 34 can control the image processing unit 32 and the storage unit 33.

[0034] (First Implementation Plan) Figure 1 This is a block diagram showing a schematic construction of the light detection device 1 according to the first embodiment. Figure 1 The light detection device 1 includes a pixel array 2, a grayscale analog-to-digital converter (ADC) 3, an event detection circuit 4, an EVS digital circuit 5, a control circuit 6, and a bias circuit 7.

[0035] The pixel array section 2 has a plurality of grayscale pixels 8 and a plurality of EVS pixels 9. It is assumed that the number of grayscale pixels 8 in the pixel array section 2 is greater than the number of EVS pixels 9, but any ratio between the number of grayscale pixels 8 and the number of EVS pixels 9 can be used.

[0036] Multiple grayscale pixels 8 and multiple EVS pixels 9 are arranged in the row direction x and column direction y of the pixel array section 2. Figure 2An example is shown where the pixel array 2 is divided into multiple pixel blocks BL. Each pixel block BL includes two or more grayscale pixels 8 and one EVS pixel 9. Note that there is no particular limitation on the number of pixel blocks BL in the pixel array 2.

[0037] Each of the grayscale pixel 8 and the EVS pixel 9 includes a photoelectric conversion element PD (photodiode) and a pixel circuit. The pixel circuit for each grayscale pixel 8 converts the charge obtained as a result of the photoelectric conversion of the photoelectric conversion element PD of the grayscale pixel 8 into a voltage to generate a pixel signal. The pixel signal contains grayscale information. The pixel circuit for the EVS detects events based on a voltage signal corresponding to the charge obtained as a result of the photoelectric conversion of the photoelectric conversion element PD of the corresponding EVS pixel 9. An event is information indicating that the amount of brightness change exceeds a predetermined threshold. The specific circuit construction of the grayscale pixel 8 and the EVS pixel 9 will be described later.

[0038] Note that in this specification, the photoelectric conversion element PDi for each grayscale pixel 8, or a portion of the pixel circuit arranged on the same substrate as the photoelectric conversion element PDi for the grayscale pixel 8, can be referred to as grayscale pixel 8, and the entirety of the photoelectric conversion element PDi for each grayscale pixel 8 and the pixel circuit can be referred to as grayscale pixel 8. Similarly, the photoelectric conversion element PDe for each EVS pixel 9, or a portion of the pixel circuit (more specifically, the logarithmic response unit and event detection circuit described later) arranged on the same substrate as the photoelectric conversion element PDe for the EVS pixel 9, can be referred to as EVS pixel 9, and the entirety of the photoelectric conversion element PDe, the logarithmic response unit, and the event detection circuit for the EVS pixel 9 can be referred to as EVS pixel 9. As described above, in this specification, grayscale pixel 8 or EVS pixel 9 can refer to at least a portion of the pixel circuit.

[0039] The pixel signal obtained as a result of photoelectric conversion of grayscale pixel 8 is transmitted to grayscale ADC 3 via a signal line extending in the column direction y. The voltage signal corresponding to the charge obtained as a result of photoelectric conversion of EVS pixel 9 is transmitted to the corresponding event detection circuit 4.

[0040] The grayscale ADC 3 performs AD conversion on pixel signals in multiple signal lines arranged at predetermined intervals along the row direction x to generate digital pixel signals.

[0041] although Figure 2An event detection circuit 4 is shown, but in practice, an event detection circuit 4 is provided for each EVS pixel 9, and an event is detected by comparing a voltage signal corresponding to the charge obtained as a result of the photoelectric conversion of the corresponding EVS pixel 9 with a threshold voltage. More specifically, the event detection circuit 4 outputs an event signal indicating whether an event has occurred. Event signals include, for example, on-event signals and off-event signals. An on-event signal is used to detect an on-event indicating that the brightness change of the EVS pixel 9 has exceeded a predetermined threshold. For example, when the on-event signal goes high, it indicates that an on-event has occurred. An off-event signal is used to detect an off-event indicating that the brightness change of the EVS pixel 9 has fallen below a predetermined threshold. For example, when the off-event signal goes high, it indicates that an off-event has occurred.

[0042] In this embodiment, the on event signals detected by all EVS pixels 9 in the pixel array unit 2 are output simultaneously, and the off event signals detected by all EVS pixels 9 in the pixel array unit 2 are output simultaneously at another time. In this specification, the on event signals and the off event signals are collectively referred to as event signals.

[0043] Event detection circuit 4 can simultaneously detect both on and off events, and simultaneously output both on and off event signals. In this case, each event signal has two bits, one bit representing the on event signal and the other bit representing the off event signal.

[0044] EVS digital circuit 5 generates digital signals corresponding to multiple event signals detected by multiple EVS pixels 9 in pixel array 2.

[0045] Control circuit 6 generation Figure 2 The components in the optical detection device 1 will use various control signals, and supply these control signals to the components. The bias circuit 7 generates... Figure 2 The components of the optical detection device 1 will use the bias signal.

[0046] (Double layer stacked) The optical detection device 1 according to this disclosure can be implemented by a chip having a stacked structure. Figure 2 This is a schematic perspective view showing an example of a light detection device 1 according to the present disclosure having a double-layer stacked structure. Figure 2The light detection device 1 includes a first substrate SB1 disposed on the light incident surface side and a second substrate SB2 stacked on the first substrate SB1. For example, photoelectric conversion elements PDi and PDe, representing grayscale pixels 8 and EVS pixels 9, are disposed on the first substrate SB1. On the second substrate SB2, a grayscale ADC 3, an event detection circuit 4, an EVS digital circuit 5, a control circuit 6, and a bias circuit 7 are disposed. The pixel circuit for the grayscale pixel 8 can be disposed on the first substrate SB1, or it can be disposed in a distributed manner on the first substrate SB1 and the second substrate SB2. The first substrate SB1 and the second substrate SB2 are joined and transmit signals via, for example, a copper-copper connection (CCC). Alternatively, in addition to CCC, the first substrate SB1 and the second substrate SB2 can be joined via vias or bumps. In this specification, the first substrate SB1 can be referred to as the upper chip SB1, and the second substrate SB2 can be referred to as the lower chip SB2.

[0047] exist Figure 3 and Figure 3 The image shows an example of a pixel array section 2 containing grayscale pixels 8 and EVS pixels 9, but a pixel array section 2 containing EVS pixels 9 but not grayscale pixels 8 can be configured.

[0048] (Resolution conversion of EVS pixel 9) The light detection device 1 according to this disclosure is characterized in that the resolution of the EVS pixel 9 can be varied. Figure 1 This diagram illustrates an example arrangement of grayscale pixels 8 and EVS pixels 9 in the pixel array section 2 of the first substrate SB1. Figure 2 In the example, a second pixel block BL2 is provided. Within each second pixel block BL2, two first pixel blocks BL1 are arranged in each of the first x and second y directions. Each first pixel block BL1 includes two pixels in each of the first x and second y directions, totaling four pixels. In the pixel array section 2, multiple second pixel blocks BL2 are arranged in the first x and second y directions. A first pixel block BL1 is essentially a block of grayscale pixels 8 of the same color. Each second pixel block BL2 includes two green first pixel blocks BL1, one red first pixel block BL1, and one blue first pixel block BL1. One pixel in a green first pixel block BL1 and one pixel in a blue first pixel block BL1 are not grayscale pixels 8, but EVS pixels 9. Therefore, a second pixel block BL2 includes 14 grayscale pixels 8 and two EVS pixels 9.

[0049] like Figure 4AAs shown, analog front-end circuitry (hereinafter referred to as AFE circuitry) 12 is arranged on the second substrate SB2, for example, for each first pixel block BL1. As will be explained later, the AFE circuitry 12 includes event detection circuitry 4 for the EVS pixel 9, etc. In the case where each second pixel block BL2 includes four first pixel blocks BL1, four AFE circuits 12 are provided on the second substrate SB2.

[0050] According to the present disclosure, the optical detection device 1 can integrate multiple AFE circuits 12 as needed, and use a single large AFE circuit 12L for event detection. For example, in... Figure 4A In the case of event detection using multiple EVS pixels 9 included in the second pixel block BL2, as shown in this embodiment, it is possible to select a mode that uses four AFE circuits 12 to detect events and a mode that uses a large AFE circuit 12L that integrates the four AFE circuits 12 as shown in FIG. 4C to detect events. The large AFE circuit 12L includes four AFE circuits 12, but in reality only one of them is used for event detection, and power consumption can be reduced compared to the case where event detection is performed by four AFE circuits 12 individually.

[0051] (Circuit structure for 8 grayscale pixels) Figure 4B This is a circuit diagram showing the circuit construction of each grayscale pixel 8. (Example) Figure 4B As shown, grayscale pixel 8 includes a photoelectric conversion element PDi and a pixel circuit 13. The pixel circuit 13 of grayscale pixel 8 includes a transmission transistor Q1, a reset transistor Q2, an amplification transistor Q3, and a selection transistor Q4. All transistors Q1 to Q4 in pixel circuit 13 are NMOS transistors. In this specification, the transistors in pixel circuit 13 can be collectively referred to as pixel transistors.

[0052] The anode of the photoelectric conversion element PDi is set to a reference voltage (e.g., ground) node, and the cathode is connected to the source of the transfer transistor Q1. The drain of the transfer transistor Q1 is connected to the floating diffuser FD, the source of the reset transistor Q2, and the gate of the amplification transistor Q3. The drain of the amplification transistor Q3 is connected to the power supply voltage node, and the source of the amplification transistor Q3 is connected to the drain of the select transistor Q4. The source of the select transistor Q4 is connected to the signal line.

[0053] Notice, Figure 5 This is an example of the circuit construction of pixel circuit 13 for grayscale pixel 8, and it can be adopted with... Figure 5 Different circuit structures.

[0054] (Circuit structure of EVS pixel 9) Figure 5This is a circuit diagram showing the internal structure of each EVS pixel 9 and each event detection circuit 4. The EVS pixel 9 includes a logarithmic response unit 21 and a buffer 22. The logarithmic response unit 21 generates a voltage signal, which is obtained by logarithmically converting the charge obtained as a result of photoelectric conversion in the photoelectric conversion element Pde. Through logarithmic conversion, the dynamic range of the EVS pixel 9 during photoelectric conversion can be expanded, and the sensitivity can be improved.

[0055] The logarithmic response unit 21 includes NMOS transistors Q21 to Q23 and a PMOS transistor Q24. Transistor Q22 has a gate and drain connected to each other and functions as a diode. The drain of transistor Q21 is connected to the power supply voltage node, and the source of transistor Q21 is connected to the gate and drain of transistor Q22. The gate of transistor Q21 is connected to the drain of transistors Q23 and Q24, and this connection node is the output node of the logarithmic response unit 21. The source of transistor Q24 is connected to the power supply voltage node, and a predetermined bias voltage Vlog is applied to the gate of transistor Q24.

[0056] Buffer 22 includes NMOS transistors Q25 and Q26 cascaded between the power supply voltage node and the ground voltage node. The output node of the logarithmic response unit 21 is connected to the gate of transistor Q25. The gate of transistor Q26 is supplied with a reference voltage SF. The source of transistor Q25 and the drain of transistor Q26 are the output nodes of buffer 22. Transistor Q26 acts as a source follower circuit and outputs a voltage signal with a waveform corresponding to the source voltage of transistor Q24 from the source of transistor Q25.

[0057] Figure 5 An example is shown where the logarithmic response unit 21 of EVS pixel 9 and the transistor Q25 of buffer 22 are arranged in the upper chip SB1, and the transistor Q26 of buffer 22 is arranged in the lower chip SB2. This is an example, and as will be explained later, for example, the transistor Q26 of buffer 22 may be arranged in the upper chip SB1, or both transistors Q25 and Q26 constituting buffer 22 may be arranged in the lower chip SB2.

[0058] Figure 6 The event detection circuit 4 has a synchronization structure and outputs an event signal indicating whether each of the plurality of EVS pixels 9 has detected an event synchronously with a predetermined frame period. Figure 6 The event detection circuit 4 includes a differentiator 23, a comparator 24, and an output circuit 25. The differentiator 23 includes a capacitor C1, NMOS transistors Q27 and Q28, and a PMOS transistor Q29.

[0059] Capacitor C1 is connected between the output node of buffer 22 and the gate of transistor Q29. Capacitor C1 supplies a current to the gate of transistor Q29 corresponding to the change in voltage signal Vp, which is the time derivative of the voltage signal Vp output from buffer 22. Transistors Q29 and Q28 are cascaded between the power supply voltage node and the ground voltage node. A switch 26, selecting one of three bias voltages, is connected to the gate of transistor Q28. Based on the switching control signal BIAS_SW output from control circuit 6, switch 26 selects one of the bias voltages AZ for automatic zeroing, POS for positive event detection, and NEG for negative event detection. By switching switch 26, the gate of transistor Q28 is set to the optimal bias voltage.

[0060] Differentiator 23 outputs a differentiated signal from the drain of transistor Q28 and the source of transistor Q29. This differentiated signal is then input to comparator 24.

[0061] Comparator 24 includes a PMOS transistor Q31 and an NMOS transistor Q30 cascaded between the power supply voltage node and the ground voltage node. A threshold signal 2NDCOMP is applied to the gate of transistor Q30. The gate of transistor Q31 is connected to the source of transistor Q29, and the differentiated signal output from differentiator 23 is input to the gate of transistor Q31. When the voltage level of this differentiated signal is higher than the voltage level of the threshold signal 2NDCOMP, comparator 24 outputs a high-level event signal indicating that an event has been detected.

[0062] By using switch 26 to switch the gate voltage of transistor Q28, Figure 6 The comparator 24 can switch from the source of transistor Q31 and output an enable event signal or an disable event signal.

[0063] Output circuit 25 includes logic circuit 27 with logic gates, flip-flops, etc. Control signals LATCH_EN and POLARITY, output from control circuit 6, are input to logic circuit 27. The control signal LATCH_EN defines the timing for latching event signals. The control signal POLARITY specifies the polarity of the event signal. The event signal output from logic circuit 27 is input to EVS digital circuit 5.

[0064] The EVS digital circuit 5 generates digital signals based on event signals detected in all EVS pixels 9 of the pixel array section 2.

[0065] (Three layers stacked) The optical detection device 1 according to this disclosure can be constructed by stacking three or more substrates. Figure 6This is a schematic perspective view showing an example of a three-layer structure for a light detection device 1 according to the present disclosure, and Figure 6 Is with Figure 7 The circuit diagram of the corresponding EVS pixel 9 and event detection circuit 4.

[0066] Figure 8 An example of a stacked structure of a light detection device 1 having a first substrate SB1, a second substrate SB2, and a third substrate SB3 is shown. Grayscale pixels 8 and EVS pixels 9 in the pixel array section 2 are arranged on the first substrate SB1. Furthermore, as... Figure 7 As shown, a portion of the logarithmic response section 21 connected to the EVS pixel 9 is disposed on the first substrate SB1. Figure 7 As shown, a portion of the logarithmic response section 21 and the buffer 22 connected to the EVS pixel 9 are arranged on the second substrate SB2. Furthermore, the pixel circuit 13 for the grayscale pixel 8 is arranged on one of the first substrate SB1 and the second substrate SB2, or on both the first substrate SB1 and the first substrate SB2. Figure 8 As shown, a portion of the buffer 22 for the EVS pixel 9, the differentiator 23, the comparator 24, and the output circuit 25 are arranged on the third substrate SB3. Additionally, an AD converter for the grayscale pixel 8, signal processing circuitry, etc., are arranged on the third substrate SB3. The first substrate SB1 and the second substrate SB2 are joined and transmit signals via, for example, through-silicon vias (TSVs), and the second substrate SB2 and the third substrate SB3 are joined and transmit signals via, for example, CCCs. Various modifications can be conceived regarding which circuit portions of the light detection device 1 are arranged on the first to third substrates SB1 to SB3, and... Figure 8 Only examples of them are shown.

[0067] (How to switch the resolution of EVS pixel 9) As described above, the light detection device 1 according to the first embodiment can switch the resolution of the EVS pixel 9 as needed. Various circuit configurations can be considered as methods for switching the resolution of the EVS pixel 9. In this embodiment, each pixel block BL comprising two or more EVS pixels 9 includes a switch for switching whether to connect internal nodes of the two or more EVS pixels 9. Switching this switch switches at least one of the resolution, sensitivity, power gating, or responsiveness of the two or more EVS pixels 9. The aforementioned internal nodes include, for example, connections to... (described later) Figure 8 The nodes of the first photoelectric conversion element PDe, the node connected to the logarithmic response unit 21, and the node connected to the event detection circuit 4 are shown.

[0068] Figure 8 This is a circuit diagram of a first example of switching the resolution of EVS pixel 9.Figure 9A This is a circuit diagram for a second example of switching the resolution of EVS pixel 9, and Figure 9A This is a circuit diagram for a third example of switching the resolution of EVS pixel 9. In any of the first to third examples, either high resolution or low resolution can be selected.

[0069] exist Figure 9B In the first example shown, multiple switches SW1 to SW4 are connected to the cathodes of the two photoelectric conversion elements PDe for the two EVS pixels 9 to switch the resolution. More specifically, in the first example, as... Figure 9C As shown, a first switch SW1 and a second switch SW2 are provided, which are connected in series between the cathodes of two photoelectric conversion elements PDe (hereinafter referred to as the first photoelectric conversion element PD1 and the second photoelectric conversion element PD2) for two EVS pixels 9 (hereinafter referred to as the first EVS pixel 9 and the second EVS pixel 9), a third switch SW3 is connected between the cathode of the first photoelectric conversion element PD1 and the first logarithmic response unit 21a, and a fourth switch SW4 is connected between the cathode of the second photoelectric conversion element PD2 and the second logarithmic response unit 21b.

[0070] In the first example, when high resolution is selected, the first switch SW1 and the second switch SW2 are turned off, and the third switch SW3 and the fourth switch SW4 are turned on. Therefore, the first EVS pixel 9 and the second EVS pixel 9 perform event detection respectively.

[0071] In the first example, when a low resolution is selected, for example, the fourth switch SW4 is turned off, and the first switch SW1 to the third switch SW3 are turned on. Therefore, event detection of the charge obtained based on the photoelectric conversion results in the first photoelectric conversion element PD1 and the second photoelectric conversion element PD2 is performed by the buffer 22, differentiator 23, event detection circuit 4, and comparator 24 connected to the first logarithmic response unit 21a, and integration is performed over the two EVS pixels 9, with event detection performed by one event detection circuit 4. In this case, since the buffer 22, differentiator 23, event detection circuit 4, and comparator 24 connected to the second logarithmic response unit 21b are not used, power consumption can be reduced.

[0072] exist Figure 9A In the second example shown, switch SW is connected to two differentiators 23 (hereinafter referred to as first differentiator 23a and second differentiator 23b) for the two EVS pixels 9 to switch the resolution. More specifically, in the second example, as... Figure 9AAs shown, a switch SW is provided between the gate of transistor Q29 in the first differentiator 23a and the gate of transistor Q29 in the second differentiator 23b. When switch SW is turned on, the gates of transistor Q29 in the first differentiator 23a and the second differentiator 23b are connected to each other. One of the first differentiators 23a and the second differentiator 23b can stop event detection by controlling the bias voltage input to the gate of transistor Q28.

[0073] Therefore, when the switch SW is turned on, event detection can be performed using only one of the first differentiator 23a and the second differentiator 23b. On the other hand, when the switch SW is turned off, the first differentiator 23a and the second differentiator 23b can perform event detection independently of each other.

[0074] exist Figure 9B In the third example shown, four EVS pixels 9 are merged to achieve event detection. In the third example, as... Figure 9B As shown, there are first to fourth switches SW1 to SW4 connected between the cathodes of the four photoelectric conversion elements PDe (hereinafter referred to as the first to fourth photoelectric conversion elements PD1 to PD4), a fifth switch SW5 connected between the connection node of the first switch SW1 and the second switch SW2 and the connection node of the third switch SW3 and the fourth switch SW4, a sixth switch SW6 connected between the cathode of the first photoelectric conversion element PD1 and the gate of the transistor Q23 of the first logarithmic response unit 21a, a seventh switch SW7 connected between the cathode of the second photoelectric conversion element PD2 and the gate of the transistor Q23 of the second logarithmic response unit 21b, an eighth switch SW8 connected between the cathode of the third photoelectric conversion element PD3 and the gate of the transistor Q23 of the third logarithmic response unit 21c, and a ninth switch SW9 connected between the cathode of the fourth photoelectric conversion element PD4 and the gate of the transistor Q23 of the fourth logarithmic response unit 21d.

[0075] By turning on the first to sixth switches SW1 to SW6 and turning off the seventh to ninth switches SW7 to SW9, the four EVS pixels 9 can be integrated, and events can be detected by the first event detection circuit 4, which is connected to the first logarithmic response unit 21a of the first photoelectric conversion element PD1. In this case, the operation of the first to third event detection circuits 4, which are connected to the second to fourth logarithmic response units 21b, 21c, and 21d of the second to fourth photoelectric conversion elements PD2 to PD4, can be stopped, thereby reducing power consumption.

[0076] like Figure 9CAs shown, switches SW1 to SW9 for switching resolution are located at nodes connected to the photoelectric conversion element PD or at nodes connected to the event detection circuit 4. Alternatively, the switches can be connected to nodes connected to the logarithmic response unit 21. By connecting the switches to nodes connected to the logarithmic response unit 21, the responsiveness of the EVS pixel 9 can be adjusted.

[0077] (Switching the resolution of EVS Pixel 9) In this embodiment, the resolution can be switched for any region of the event image generated by performing event detection in all EVS pixels 9 of the pixel array section 2. Figure 9C This is a diagram illustrating an example of a subject to which event detection is to be performed. The subject is an event detection target included in the field of view where the light detection device 1 according to the first embodiment is capable of performing event detection. Figures 9A-9C An example is shown where the subject includes a moving object Obj1 and a background object Obj2. The moving object Obj1 included in the subject is the object for which event detection is initially performed, while the background object Obj2 is the object for which event detection is not required. However, when the background object Obj2 (e.g., a tree) is shaken by the wind, events of the background object Obj2 may be detected.

[0078] Figure 10A An event image is shown, obtained by detecting events in the entire area of ​​the subject at high resolution. Figure 10A The event image includes not only the moving object Obj1, but also the background object Obj2. Because... Figure 10B The event image in the image includes the event detection results of all EVS pixels 9 in the pixel array section 2, thus increasing the amount of data and the power consumption required to generate the event image.

[0079] Figure 10B An example is shown where event detection is performed only at high resolution on the portion corresponding to the moving object Obj1 included in the subject, and at low resolution on the portion corresponding to the background object Obj2, to generate an event image. This can be achieved with low power consumption. Figure 10B The event images generated were reduced in size and amount of data.

[0080] As described above, in the first embodiment, since only a portion of the subject is detected at high resolution and the rest is detected at low resolution, the amount of data can be reduced without degrading the image quality of the event image, and the event image can be generated with low power consumption.

[0081] The resolution at which part of the subject is detected as an event can be determined using machine learning, by the user, or by other methods.

[0082] Figure 10C This is a schematic diagram illustrating a first method for determining the resolution of a subject. Figure 10C The first method includes an inference device 15 for performing machine learning connected to a light detection device 1 according to a first embodiment.

[0083] Figure 11A The light detection device 1 performs high-resolution event detection on the entire area of ​​the subject and generates an event image. The light detection device 1 inputs the generated event image to the inference device 15. The inference device 15 uses a machine learning method to infer the object initially intended for event detection based on the event image input from the light detection device 1, and returns information about the inferred object to the light detection device 1. The light detection device 1 performs high-resolution event detection on the area of ​​the object inferred by the inference device 15, and low-resolution event detection on other areas.

[0084] Each time the viewing angle of event detection changes, the light detection device 1 performs high-resolution event detection on the entire area of ​​the subject at the first moment. Subsequently, based on the inference result of the inference device 15, only a part of the subject is subjected to high-resolution event detection, thereby enabling event detection with low power consumption while reducing the amount of event image data.

[0085] Figure 11A This is a diagram illustrating a second method for determining the resolution of a subject. Figure 11A In the second method, each time the viewing angle of the light detection device 1 changes, high-resolution event detection is initially performed on the entire area of ​​the subject to generate an event image. The generated event image is displayed on, for example, a display device. The user views the event image displayed on the display device and visually specifies the area to be subjected to high-resolution event detection. The specified area is notified to the light detection device 1. Based on the user's specification, the light detection device 1 generates the event image by performing high-resolution event detection only on a portion of the subject and low-resolution event detection on the other parts of the subject.

[0086] As described above, in the first embodiment, since event detection can be performed by switching the resolution of any pixel region of the subject, the amount of data can be greatly reduced without degrading the image quality of the event image to be generated, and the event image can be generated with low power consumption. Furthermore, according to the first embodiment, since the resolution of any pixel region of the subject can be switched as needed, the event image can be generated with low power consumption even when the location of the event changes over time, without increasing the amount of data or degrading the image quality.

[0087] (Second Implementation Plan) According to the second embodiment, the light detection device 1 detects an event signal including spatiotemporal differential information of multiple EVS pixels 9.

[0088] According to the second embodiment, the optical detection device 1 has a similar Figure 11B and Figure 11B A similar block structure. However, the pixel array section 2 does not need to have a mixed pixel structure that combines grayscale pixels 8 and EVS pixels 9, and a pixel array section 2 that includes EVS pixels 9 but does not include grayscale pixels 8 can be provided instead. Note that although an example of generating a spatiotemporal differential signal of two EVS pixels 9 will be described below, the light detection device 1 according to the second embodiment can also be used when generating spatiotemporal differential signals of three or more EVS pixels 9.

[0089] (Block construction for generating spacetime differentials) Figure 1 This is a block diagram showing a schematic construction of the light detection device 1 according to the second embodiment. Figure 2 The block diagram illustrates the block structure related to the processing of EVS pixels 9, and since the processing of grayscale pixels 8 is similar to that of the first embodiment, its description is omitted. The light detection device 1 according to the second embodiment includes two logarithmic response units 21a and 21b connected to the two EVS pixels 9, a differential 16, a buffer 22, a differentiator 23, a comparator 24, and an output circuit 25.

[0090] Differential converter 16 detects the differential signal between the voltage signals output from the two logarithmic response units 21a and 21b. This differential signal corresponds to the spatial differential signal between event images. This spatial differential signal is input to the differentiator via buffer 22.

[0091] Differentiator 23 performs temporal differentiation on the spatially differentiated signal. Therefore, differentiator 23 generates and outputs a spatiotemporal differentiated signal. Comparator 24 generates an event signal by comparing the spatiotemporal differentiated signal with a predetermined reference signal. Therefore, the event signal generated by the light detection device 1 according to the second embodiment is a spatiotemporal differentiated signal of multiple EVS pixels 9. By using the spatiotemporal differentiated signal as the event signal, it is possible to detect brightness changes in the contour (edge) portion of an object as events, and generate an event image that includes only the contour portion and excludes other unnecessary event information. Therefore, it is possible to reduce the amount of data in the event image.

[0092] like Figure 12A As shown in the block diagram, performing spatiotemporal differentiation requires at least two EVS pixels 9. For example, in order to perform spatiotemporal differentiation of an event image over the entire area of ​​the pixel array 2, it is only necessary to perform the process of generating a spatiotemporal differentiation signal between two adjacent EVS pixels 9 for all EVS pixels 9 of the pixel array 2.

[0093] Figure 12A This is a block diagram illustrating a schematic construction of a light detection device 1 according to a first variant of the second embodiment. The light detection device 1 according to the first variant of the second embodiment differs from the first embodiment in the processing order of performing spatiotemporal differentiation. The light detection device 1 according to the first variant of the second embodiment includes two logarithmic response units 21a and 21b connected to two EVS pixels 9, two buffers 22a and 22b, a differentiator 23, a comparator 24, and an output circuit 25.

[0094] In the optical detection device 1 of the first variant according to the second embodiment, two voltage signals output from two buffers 22a and 22b are input to a differential 16 to generate a spatial differential signal. The generated spatial differential signal is input to a differentiator 23 and undergoes temporal differentiation. Therefore, a spatiotemporal differential signal is output from the differentiator 23. Subsequent processing and... Figure 12A The same.

[0095] Figure 12B This is a block diagram illustrating a schematic construction of a light detection device 1 according to a second modification of the second embodiment. The light detection device 1 according to the second modification of the second embodiment includes two logarithmic response units 21a and 21b connected to two EVS pixels 9, two buffers 22a and 22b, two differentiators 23a and 23b, two comparators 24a and 24b, two output circuits 25a and 25b, and an ExOR circuit 17. The processing sequence from the two logarithmic response units 21a and 21b to the two output circuits 25a and 25b is... Figure 12A Similarity in.

[0096] Two event signals are output from two output circuits 25a and 25b. For example, these event signals are binary digital signals.

[0097] ExOR circuit 17 calculates the XOR of two event signals to generate a spatiotemporal differential signal. The spatiotemporal differential signal output from ExOR circuit 17 is the final event signal.

[0098] Therefore, the event signal of EVS pixel 9 is a digital signal, and in order to detect the spatial differentiation of multiple event signals corresponding to multiple EVS pixels 9, it is sufficient to simply input multiple event signals to ExOR circuit 17. Since ExOR circuit 17 outputs 1 when the logic of multiple input signals is different, the difference between multiple digital signals can be easily calculated and output.

[0099] Figure 12CThis is a block diagram illustrating a schematic construction of a light detection device 1 according to a third variation of the second embodiment. The light detection device 1 according to the third variation of the second embodiment includes two logarithmic response units 21a and 21b connected to two EVS pixels 9, two buffers 22a and 22b, two differentiators 23a and 23b, two comparators 24a and 24b, a differential circuit 16, and an output circuit 25.

[0100] Differential converter 16 detects the difference between two event signals output from comparators 24a and 24b to generate a spatiotemporal differential signal of the event signal. The spatiotemporal differential signal is output from output circuit 25 as the final event signal.

[0101] although Figure 6 An example of performing spatial differentiation by taking the difference between two EVS pixels 9 is shown, but as mentioned above, spatial differentiation can also be performed by taking the difference between three or more EVS pixels 9. The multiple EVS pixels 9 to be spatially differentiated do not necessarily need to be adjacent to each other. Since the event signal output from the event detection circuit 4 is a binary digital signal, the spatiotemporal differentiation signal can be easily generated by inputting the two event signals corresponding to the two EVS pixels 9 to the ExOR circuit 17.

[0102] (The type of EVS pixel 9 that performs spatiotemporal differentiation) Figure 12D This is a diagram showing the change in the arrangement position of the two EVS pixels 9 corresponding to the two event signals input to the ExOR circuit 17. Figures 12A-12D The block of EVS pixel 9 shown contains event detection circuit 4, and outputs event signals from each block.

[0103] Figures 13A-13E This diagram illustrates a first example of generating a spatiotemporal differential signal using the ExOR circuit 17. In this first example, two event signals output from two EVS pixels 9 are input to the ExOR circuit 17 to generate the spatiotemporal differential signal.

[0104] Figures 13A-13E This is a diagram illustrating a second example of generating a spatiotemporal differential signal using the ExOR circuit 17. In this second example, four event signals output from four EVS pixels 9 are input to the ExOR circuit 17 to generate the spatiotemporal differential signal.

[0105] Figure 13A A third example of generating a spatiotemporal differential signal using the ExOR circuit 17 is shown. In this third example, two event signals output from two non-adjacent EVS pixels 9 in the same pixel row are input to the ExOR circuit 17 to generate the spatiotemporal differential signal.

[0106] Figure 13BThis diagram illustrates a fourth example of generating a spatiotemporal differential signal using the ExOR circuit 17. In this fourth example, two event signals output from two EVS pixels 9 located in different columns of multiple different pixel rows are input to the ExOR circuit 17 to generate the spatiotemporal differential signal.

[0107] Figure 13C This diagram illustrates a fifth example of generating a spatiotemporal differential signal using the ExOR circuit 17. In this fifth example, a spatiotemporal differential signal is generated between two adjacent EVS pixels 9 out of four adjacent EVS pixels 9 in the same pixel row. Therefore, it is possible to generate three spatiotemporal differential signals.

[0108] As described above, in the second embodiment, since an event signal containing a spatiotemporal differential signal is generated using EVS pixels 9, events can be detected, for example, based on brightness changes in the contour (edge) portion of an object. Therefore, an event image can be generated that includes only event information of the contour portion without excluding other unnecessary event information, thus reducing the amount of data in the event image. Furthermore, since the spatiotemporal differential signal can be generated simply by adding the differential 16 to the existing event detection circuit 4, design changes to the existing event detection circuit 4 can be reduced. Moreover, since the differential 16 can be arranged at any location in the event detection circuit 4, layout design is straightforward. In particular, the spatiotemporal differential signal can be generated simply by inputting multiple event signals to the ExOR circuit 17.

[0109] (Third Implementation Plan) In the third implementation, at least one of resolution, sensitivity, power gating, and responsiveness is switched for each pixel block BL having an arbitrary size.

[0110] Figure 13D The diagram illustrates a third embodiment. According to the third embodiment, the light detection device 1 includes multiple control lines L connected to multiple pixel blocks BL, each pixel block BL containing multiple EVS pixels 9. Each control line L supplies a control signal to the corresponding pixel block BL. At least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block BL is controlled by the control signal. Power gating refers to stopping the operation of a portion of the event detection circuit 4 to reduce power consumption.

[0111] For example, such as Figure 13E As shown, when each pixel block BL includes four EVS pixels 9 and the first to ninth switches SW1 to SW9, the resolution can be adjusted by controlling the switching of the first to ninth switches SW1 to SW9 through the control signal.

[0112] Figure 14 This is a circuit diagram of the front-end section of the EVS pixel 9 applicable to the third implementation scheme. Figure 9CThe circuit structure of the logarithmic response unit 21, buffer 22 and differentiator 23 of the EVS pixel 9 is shown.

[0113] For example, by Figure 15 The control signal transmitted by the control line L can be input to... Figure 15 The bias signal of the gate of transistor Q28 in the differentiator 23 is used. Therefore, the sensitivity of EVS pixel 9 can be adjusted.

[0114] Alternatively, for example, a control signal can be input to the gate of transistor Q24 in the logarithmic response unit 21. Therefore, the responsiveness of the EVS pixel 9 can be adjusted.

[0115] Figure 14 It shows the results of targeting Figure 15 A diagram illustrating an example of how the resolution is individually controlled by the control signals of each pixel block (BL). Figure 16 It shows in Figure 14 In the example of four pixel blocks BL1 to BL4, pixel blocks BL1 and BL4 perform high-resolution event detection, while pixel blocks BL2 and BL3 perform low-resolution event detection. In pixel blocks BL2 and BL3, since the four EVS pixels 9 in each pixel block BL are integrated and event detection is performed using one of the event detection circuits 4, power consumption is reduced, but resolution is lowered. In pixel blocks BL1 and BL4, since the four EVS pixels 9 in each pixel block BL perform event detection individually, resolution is improved, but power consumption is increased.

[0116] Note that the control signals transmitted via the control lines L connected to each pixel block BL can be used not only to switch resolutions, but also to control sensitivity, power gating, or responsiveness. Therefore, for each pixel block BL, at least one of the resolution, sensitivity, power gating, or responsiveness can be made different.

[0117] Figure 16 A circuit diagram of a first specific example of the third embodiment is shown, and Figure 14 A circuit diagram of a second specific example of the third implementation scheme is shown.

[0118] Figure 17A The first concrete example shown has the same as Figure 17B The same circuit construction. Figure 17A The first to ninth switches SW1 to SW9 are switched by a control signal transmitted via control line L. By switching the first to ninth switches SW1 to SW9, a mode in which four EVS pixels 9 detect events individually can be selected, or a mode in which two to four EVS pixels 9 are integrated and detect events together can be selected.

[0119] Figure 9CThe second specific example shown includes a first switch SW1 to a fifth switch SW5 connected between the gates of the transistors Q29 of the four differentiators 23. By switching the first to fifth switches SW1 to SW5 respectively, a mode that generates four differential signals and a mode that integrates two to four EVS pixels 9 and outputs a single differential signal can be selected.

[0120] Switching Figure 17A and Figure 17B In the case of each switch, firstly, with the switches connecting multiple EVS pixels 9 disconnected, an event image is generated by performing event detection for each EVS pixel 9. Based on the event image, the pixel region for which high-resolution or low-resolution event detection is to be performed is specified, and the switch of each pixel block BL corresponding to the specified pixel region is switched to adjust the resolution, sensitivity, power gating, or responsiveness.

[0121] (A variation of pixel block BL) Pixel blocks (BLs) of any size and shape can be used, and various variations are possible. Representative variations of pixel blocks (BLs) will be explained below.

[0122] Figure 17A It is a planar diagram based on the pixel block BL of the first example. For example... Figure 17B As shown, each pixel block BL according to the first example includes a plurality of EVS pixels 9 arranged adjacent to each other in the first direction x (row direction), and each pixel block BL may include any number of EVS pixels 9. Different pixel blocks BL are arranged in the second direction y (column direction).

[0123] Figure 18A It is a planar diagram based on the pixel block BL of the second example. For example... Figure 18A As shown, each pixel block BL according to the second example includes a plurality of EVS pixels 9 arranged adjacent to each other in the second direction y (column direction), and each pixel block BL may include any number of EVS pixels 9. Different pixel blocks BL are arranged in the first direction x (row direction).

[0124] Figure 18B It is a planar diagram based on the pixel block BL of the third example. For example... Figure 18B As shown, according to the third example, each pixel block BL comprises multiple EVS pixels 9 arranged adjacent to each other in the first direction x (row direction) and the second direction y (column direction). Figure 18CThe example shown illustrates that each pixel block BL comprises 2 × 2 = 4 EVS pixels 9, but in the first direction x and the second direction y, each pixel block BL can comprise any number of EVS pixels 9. Furthermore, the size and shape of each pixel block BL can be determined based on the size and shape of the region of interest (ROI) contained in the event image.

[0125] Figure 18C This is a plan view of pixel block BL according to the fourth example. The EVS pixels 9 in each pixel block BL do not necessarily need to be arranged adjacent to each other and can be arranged in a dispersed manner. Each of the pixel blocks BL1 and BL2 according to the fourth example includes EVS pixels 9 arranged in a dispersed manner in the first direction x (row direction) and the second direction y (column direction). In the fourth example, pixel blocks BL1 and BL2 include multiple EVS pixels 9 arranged in an interleaved manner in the first direction x and the second direction y, but the pixel position and number of EVS pixels 9 contained in pixel blocks BL1 and BL2 can be arbitrary.

[0126] Figure 18C This is a plan view of the pixel block BL according to the fifth example. Each pixel block BL according to the fifth example includes multiple grayscale pixels 8 and one EVS pixel 9. Furthermore, the pixel array section 2 may include various types of pixel blocks BL with different characteristics.

[0127] exist Figure 18D The example shows a pixel array 2 comprising multiple first pixel blocks BL1 and multiple second pixel blocks BL2. Each first pixel block BL1 includes, for example, three grayscale pixels 8 and one EVS pixel 9, and the three grayscale pixels 8 are of the same color. The event detection threshold of the EVS pixel 9 is set according to the colors of the other three grayscale pixels 8. The multiple first pixel blocks BL1 include a first pixel block BL1 with three red grayscale pixels 8 and one EVS pixel 9, a first pixel block BL1 with three green grayscale pixels 8 and one EVS pixel 9, and a first pixel block BL1 with three blue grayscale pixels 8 and one EVS pixel 9. For example, the size and shape of each of the multiple second pixel blocks BL2 can be variably adjusted according to the size and shape of the ROI. For example, as Figure 18E As shown in any one of them, multiple switches are connected to multiple EVS pixels 9 in each second pixel block BL2, and by switching these switches, the resolution of the EVS pixels 9 can be switched according to the size and shape of the ROI.

[0128] (Changes in the blended pixels) When a pixel block BL contains a mixed pixel that combines grayscale pixel 8 and EVS pixel 9, various variations of the arrangement of the grayscale pixel 8 and EVS pixel 9 constituting the mixed pixel can be conceived. Representative variations of the mixed pixel will be described one by one below.

[0129] Figure 18E This is a planar diagram showing the mixed pixels according to the first example. The pixel block BL according to the first example has a size of 2 × 2 = 4 pixels (three of which are grayscale pixels 8, and the remaining one is an EVS pixel 9).

[0130] Figures 9A-9C This diagram shows a planar view of the mixed pixels according to the second example. The pixel block BL according to the second example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 comprises 2 × 2 = 4 smaller pixel blocks BL (hereinafter referred to as the first pixel block BL1). Three of the four first pixel blocks BL1 each comprise four grayscale pixels 8 of different colors. The remaining first pixel block BL1 comprises four EVS pixels 9.

[0131] Figure 19A This illustrates a planar view of the mixed pixels according to the third example. The pixel block BL according to the third example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 comprises 2 × 2 = 4 smaller pixel blocks BL (hereinafter referred to as first pixel blocks BL1). Each of the four first pixel blocks BL1 comprises three or four grayscale pixels 8, and the grayscale pixels 8 in a first pixel block BL1 are of the same color. Specifically, the second pixel block BL2 includes a first pixel block BL1 with four red grayscale pixels 8, another first pixel block BL1 with four green grayscale pixels 8, another first pixel block BL1 with three green grayscale pixels 8 and one EVS pixel 9, and another first pixel block PL1 containing three blue grayscale pixels 8 and one EVS pixel 9.

[0132] Figure 19BThis is a planar view of the mixed pixels according to the fourth example. The pixel block BL according to the fourth example (hereinafter referred to as the second pixel block BL2) has a size of 4 × 4 = 16 pixels. The second pixel block BL2 includes 2 × 2 = 4 smaller pixel blocks BL (hereinafter referred to as the first pixel blocks BL1). Each of the four first pixel blocks BL1 includes three grayscale pixels 8 and one EVS pixel 9. The second pixel block BL2 includes two first pixel blocks BL1 each including three green grayscale pixels 8 and one EVS pixel 9, another first pixel block BL1 including three red grayscale pixels 8 and one EVS pixel 9, and another first pixel block BL1 including three blue grayscale pixels 8 and one EVS pixel 9.

[0133] As described above, in the third embodiment, a control line L is connected to each pixel block BL, and at least one of the resolution, sensitivity, power gating, and responsiveness of each pixel block BL can be controlled by a control signal supplied to the pixel block BL via the control line L. Therefore, for example, at least one of the resolution, sensitivity, power gating, and responsiveness can be adjusted individually for each pixel block BL. Any position, size, and shape of the pixel block BL can be used. For example, for a pixel block BL set according to the position, size, and shape of the ROI contained in the event image, a different resolution, sensitivity, power gating, or responsiveness can be set compared to other pixel blocks BL. Furthermore, each pixel block BL can contain mixed pixels. In this case, the resolution, sensitivity, power gating, or responsiveness of the EVS pixel 9 in each pixel block BL can be set according to the color of the grayscale block in the pixel block BL.

[0134] (Fourth Implementation Plan) As described above, the photodetector 1 according to this disclosure can have a stacked structure in which multiple chips (substrates) are stacked on top of each other. In this case, various modifications exist depending on which circuit portion of the photodetector 1 is arranged on which chip.

[0135] The above Figure 19C An example is shown where the EVS pixel 9 and the event detection circuit 4 are respectively arranged on the first substrate SB1 and the second substrate SB2. In the following description, the EVS pixel 9 and the event detection circuit 4 will be described one by one in relation to... Figure 19D Examples of different arrangements on the first substrate SB1 and the second substrate SB2 (first to sixth variations). Note that in the first to sixth variations described below, when not only EVS pixels 9 but also grayscale pixels 8 are arranged in the pixel array section 2, the grayscale pixels 8 can be distributed to the first substrate SB1 and the second substrate SB2 in any way.

[0136] Figure 6 It shows according toFigure 6 The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the first modified example. Figure 20A In the first modified example shown, the photoelectric conversion element PDe of the EVS pixel 9 is arranged on the first substrate SB1, and the subsequent circuit after the logarithmic response unit 21 is arranged on the second substrate SB2. Therefore, the size of each photoelectric conversion element PDe on the first substrate SB1 can be increased, and the sensitivity can be improved.

[0137] Figure 6 It shows according to Figure 20A The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the second variation. Figure 20B In the second variation shown, the photoelectric conversion element PDe of the EVS pixel 9 and the circuit portion as part of the logarithmic response unit 21 are arranged on the first substrate SB1. For example, transistors Q21 to Q23 of the logarithmic response unit 21 are arranged on the first substrate SB1, and transistor Q24 of the logarithmic response unit 21 is arranged on the second substrate SB2. Furthermore, subsequent circuitry after the buffer 22 is also arranged on the second substrate SB2.

[0138] Figure 6 It shows according to Figure 20B The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the third variation. Figure 20C In the third variant shown, the photoelectric conversion element PDe, the logarithmic response unit 21 and the buffer 22 of the EVS pixel 9 are arranged on the first substrate SB1, and the subsequent circuit after the differentiator 23 is arranged on the second substrate SB2.

[0139] Figure 6 It shows according to Figure 20C The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the fourth variation. Figure 20D In the fourth variant shown, the photoelectric conversion element PDe of the EVS pixel 9, the logarithmic response unit 21, the buffer 22, and the capacitor C1 of the differentiator 23 are arranged on the first substrate SB1, and the transistors Q27 to Q29 of the differentiator 23, the switch 26, and the subsequent circuit after the comparator 24 are arranged on the second substrate SB2.

[0140] Figure 6 It shows according to Figure 20D The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the fifth variation. Figure 20E In the fifth variant shown, the photoelectric conversion element PDe, the logarithmic response unit 21, the buffer 22 and the differentiator 23 of the EVS pixel 9 are arranged on the first substrate SB1, and the subsequent circuit after the comparator 24 is arranged on the second substrate SB2.

[0141] Figure 6 It shows according to Figure 20E The circuit diagram shows the arrangement of the EVS pixel 9 and the event detection circuit 4 in the sixth variation. Figure 20F In the fifth variant shown, the photoelectric conversion element PDe, logarithmic response unit 21, buffer 22, differentiator 23 and comparator 24 of EVS pixel 9 are arranged on the first substrate SB1, and the subsequent circuit after the output circuit 25 is arranged on the second substrate SB2.

[0142] When the pixel array section 2 includes grayscale pixels 8 and EVS pixels 9, various variations of the layout arrangement of the first substrate SB1 and the second substrate SB2 can be conceived. Figure 6 This is a diagram showing a first example of a layout arrangement. Figure 20E In the first example shown, multiple photoelectric conversion elements PDi and pixel circuits 13 of multiple grayscale pixels 8, and multiple photoelectric conversion elements PDe and logarithmic response units 21 of multiple EVS pixels 9 are arranged on the first substrate SB1. Figure 21A In the example, grayscale pixel 8 and EVS pixel 9 are arranged in a two-dimensional direction at a ratio of one EVS pixel to three grayscale pixels 8.

[0143] Event detection circuit 4, ADC for grayscale pixel 8, readout circuit, EVS digital circuit 5, control circuit 6, bias circuit 7, etc. are arranged on the second substrate SB2.

[0144] Figure 21A This diagram illustrates a second example of the layout arrangement. On a first substrate SB1 according to the second example, a plurality of grayscale pixels 8 and a plurality of EVS pixels 9 are arranged at a ratio of 1 EVS pixel 9 to 15 grayscale pixels 8. An ADC, readout circuit, EVS digital circuit 5, control circuit 6, bias circuit 7, etc., for the grayscale pixels 8 are arranged on a second substrate SB2. Figure 21A In the middle, because the ratio of EVS pixel 9 is less than Figure 21B The ratio in the middle can reduce the size of the event detection circuit 4 on the second substrate SB2.

[0145] As described above, when the photodetector 1 has a two-layer stacked structure, any circuit portion of the photodetector 1 according to any of the first to third embodiments can be assigned to the first substrate SB1 and the second substrate SB2. Furthermore, when the photodetector 1 has a three-layer stacked structure, any circuit portion of the photodetector 1 according to any of the first to third embodiments can be assigned to the first to third substrates SB1 to SB3. The bonding and signal transmission between the first substrate SB1 and the second substrate SB2, and between the second substrate SB2 and the third substrate SB3, can be performed using CCCs, through-holes, or bumps, etc.

[0146] (Application example) The technology disclosed herein can be applied to a variety of products. For example, the technology disclosed herein can also be implemented as a device installed on any type of mobile body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, robots, construction machinery, and agricultural machinery (tractors).

[0147] Figure 21B This is a block diagram illustrating a schematic construction example of a vehicle control system 7000, which is an example of a mobile body control system to which the technology of this disclosure can be applied. The vehicle control system 7000 includes multiple electronic control units interconnected via a communication network 7010. Figure 21A In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and a comprehensive control unit 7600. The communication network 7010 connecting the multiple control units can be, for example, an in-vehicle communication network conforming to any standard such as Controller Area Network (CAN), Local Interconnect Network (LIN), Local Area Network (LAN), or FlexRay (registered trademark).

[0148] Each control unit includes: a microcomputer that performs arithmetic processing according to various types of programs; a storage unit that stores programs executed by the microcomputer, parameters for various types of operations, etc.; and drive circuits that drive various types of control target devices. Each control unit also includes: a network interface (I / F) for communicating with other control units via the communication network 7010; and a communication I / F for communicating with devices, sensors, etc. inside and outside the vehicle via wired or radio communication. Figure 22The integrated control unit 7600 shown is configured with the following functionalities: a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiver 7650, an in-vehicle device I / F 7660, an audio / visual output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690. Other control units similarly include a microcomputer, communication I / Fs, and a storage unit.

[0149] The drive system control unit 7100 controls the operation of devices related to the vehicle's drive system according to various types of programs. For example, the drive system control unit 7100 functions as a control device for devices such as an internal combustion engine and a drive motor that generate driving force for the vehicle; a driving force transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the vehicle's steering angle; and a braking device that generates braking force for the vehicle. The drive system control unit 7100 may also function as a control device for systems such as anti-lock braking system (ABS) or electronic stability control (ESC).

[0150] The drive system control unit 7100 is connected to the vehicle condition detection unit 7110. For example, the vehicle condition detection unit 7110 includes at least one of the following: a gyroscope sensor for detecting the angular velocity of the vehicle's axial rotational motion, an acceleration sensor for detecting the vehicle's acceleration, and sensors for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, engine speed, or wheel rotation speed. The drive system control unit 7100 performs arithmetic processing using signals input from the vehicle condition detection unit 7110 and controls the internal combustion engine, drive motor, electric power steering, and braking system.

[0151] The vehicle body system control unit 7200 controls the operation of various types of devices installed on the vehicle body according to various types of programs. For example, the vehicle body system control unit 7200 is used as a control device for keyless entry systems, smart key systems, power windows, or various types of lights such as headlights, reversing lights, brake lights, turn signals, and fog lights. In this case, radio waves or various types of switch signals sent from a moving device that serves as a substitute for a key can be input to the vehicle body system control unit 7200. The vehicle body system control unit 7200 receives these input radio waves or signals and controls the vehicle's door locking devices, power windows, lights, etc.

[0152] The battery control unit 7300 controls the battery 7310, which serves as a power source for driving the motor, according to various types of programs. For example, information such as battery temperature, battery output voltage, and remaining battery charge is provided to the battery control unit 7300 from the battery assembly containing the battery 7310. The battery control unit 7300 uses these signals to perform arithmetic processing and to control the temperature of the battery 7310 or to control the cooling device supplied to the battery assembly.

[0153] The exterior information detection unit 7400 detects information about the exterior of the vehicle, including the vehicle control system 7000. For example, the exterior information detection unit 7400 is connected to at least one of a camera unit 7410 and an exterior information detection unit 7420. The camera unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a single-lens reflex camera, an infrared camera, and other cameras. The exterior information detection unit 7420 includes, for example, at least one of an environmental sensor for detecting current atmospheric or weather conditions and a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., in the vicinity of the vehicle including the vehicle control system 7000.

[0154] For example, the environmental sensor can be at least one of a raindrop sensor for detecting rainfall, a fog sensor for detecting fog, a sunlight sensor for detecting sunlight intensity, and a snow sensor for detecting snowfall. The surrounding information detection sensor can be at least one of an ultrasonic sensor, a radar device, and a LIDAR device (light detection and ranging device, or laser imaging detection and ranging device). Each of the camera unit 7410 and the external information detection unit 7420 can be configured as an independent sensor or device, or can be configured as a device that integrates multiple sensors or devices.

[0155] here, Figure 22 Examples of the mounting positions of the camera unit 7410 and the exterior information detection unit 7420 are shown. Camera units 7910, 7912, 7914, 7916, and 7918 are, for example, installed at at least one of the following locations: the front nose, side mirrors, rear bumper, and rear door of the vehicle 7900, and the upper part of the windshield inside the vehicle. The camera unit 7910 installed at the front nose and the camera unit 7918 installed at the upper part of the windshield inside the vehicle primarily obtain images of the front of the vehicle 7900. The cameras 7912 and 7914 installed at the side mirrors primarily obtain images of the sides of the vehicle 7900. The camera unit 7916 installed at the rear bumper or rear door primarily obtains images of the rear of the vehicle 7900. The camera unit 7918 installed at the upper part of the windshield inside the vehicle is mainly used to detect vehicles, pedestrians, obstacles, signals, traffic signs, lanes, etc., ahead.

[0156] Notice, Figure 22Examples of the camera ranges of camera units 7910, 7912, 7914, and 7916 are shown. Camera range a represents the camera range of camera unit 7910, which is installed at the front nose. Camera ranges b and c represent the camera ranges of camera units 7912 and 7914, which are installed at the side mirrors, respectively. Camera range d represents the camera range of camera unit 7916, which is installed at the rear bumper or rear door. For example, a bird's-eye view of the vehicle 7900 viewed from above can be obtained by overlaying image data captured by camera units 7910, 7912, 7914, and 7916.

[0157] The exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930, located at the front, rear, sides, corners, and above the windshield inside the vehicle 7900, can be, for example, ultrasonic sensors or radar devices. For instance, the exterior information detection units 7920, 7926, and 7930 located at the front nose, rear bumper, rear door, and above the windshield inside the vehicle 7900 can be LIDAR devices. These exterior information detection units 7920 to 7930 are primarily used to detect vehicles, pedestrians, or obstacles ahead.

[0158] Will return Figure 23 Continuing the explanation, the exterior information detection unit 7400 causes the camera unit 7410 to capture images of the exterior of the vehicle and receives data from the captured images. Additionally, the exterior information detection unit 7400 receives detection information from the exterior information detection section 7420 connected to it. If the exterior information detection section 7420 is an ultrasonic sensor, radar device, or LIDAR device, the exterior information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information from the received reflected waves. Based on the received information, the exterior information detection unit 7400 can perform processing such as detecting objects like people, vehicles, obstacles, signs, and characters on the road surface, or processing the distance to these objects. The exterior information detection unit 7400 can perform environmental recognition processing based on the received information, such as identifying rain, fog, or road conditions. The exterior information detection unit 7400 can calculate the distance to objects outside the vehicle based on the received information.

[0159] Furthermore, based on the received image data, the vehicle exterior information detection unit 7400 can perform image recognition processing to identify people, vehicles, obstacles, signs, or characters on the road surface, or to detect their distances. The vehicle exterior information detection unit 7400 can perform processing on the received image data, such as distortion correction and alignment, and combine image data captured by multiple different cameras 7410 to generate a bird's-eye view or panoramic image. The vehicle exterior information detection unit 7400 can use image data captured by different cameras 7410 to perform viewpoint conversion processing.

[0160] The in-vehicle information detection unit 7500 detects information about the interior of the vehicle. The in-vehicle information detection unit 7500 is connected, for example, to a driver state detection unit 7510 that detects the driver's state. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biological information, a microphone that collects sounds from inside the vehicle, etc. The biosensor is disposed, for example, on the seat surface or steering wheel, and detects the biological information of passengers sitting in the seat or the driver holding the steering wheel. Based on the detection information input from the driver state detection unit 7510, the in-vehicle information detection unit 7500 can calculate the driver's fatigue level or concentration level, or determine whether the driver is dozing off. The in-vehicle information detection unit 7500 can perform processing such as noise cancellation on the audio signals obtained by collecting sound.

[0161] The integrated control unit 7600 controls the overall operation within the vehicle control system 7000 according to various types of programs. The integrated control unit 7600 is connected to the input unit 7800. For example, the input unit 7800 may be implemented by a device such as a touch panel, button, microphone, switch, or joystick that allows input by a passenger. Data obtained through voice recognition of voice input via a microphone can be supplied to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an external connection device supporting the operation of the vehicle control system 7000, such as a mobile phone or personal digital assistant (PDA). The input unit 7800 may be, for example, a camera. In this case, the passenger can input information through gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger can be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the aforementioned input unit 7800, and outputs the generated input signal to the integrated control unit 7600. Passengers can input various types of data or give instructions for processing operations to the vehicle control system 7000 through the operation input unit 7800.

[0162] The storage unit 7690 may include a read-only memory (ROM) for storing various types of programs executed by a microcomputer and a random access memory (RAM) for storing various parameters, operation results, or sensor values. Furthermore, the storage unit 7690 may be implemented using a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device.

[0163] The General Communication I / F 7620 is a widely used communication I / F that regulates communication with various devices present in the external environment 7750. The General Communication I / F 7620 can implement cellular communication protocols such as GSM (Global System for Mobile Communications), WiMAX (Global Microwave Interconnection Access), LTE (Long Term Evolution), or LTE-A Advanced (LTE-A), or other wireless communication protocols such as Wireless LAN (also known as Wi-Fi), Bluetooth, etc. The General Communication I / F 7620 can, for example, connect to devices (e.g., application servers or control servers) existing on external networks (e.g., the Internet, cloud networks, or corporate private networks) via base stations or access points. Additionally, for example, the General Communication I / F 7620 can use peer-to-peer (P2P) technology to connect to terminals appearing near the vehicle (e.g., terminals belonging to drivers, pedestrians, or shops, or machine-type communication (MTC) terminals).

[0164] The Dedicated Communication I / F 7630 is a communication I / F that supports communication protocols developed for use in vehicles. For example, the Dedicated Communication I / F 7630 can implement standard protocols such as Wireless Access in a Vehicle Environment (WAVE) (a combination of IEEE 802.11p as the lower layer and IEEE 1609 as the upper layer), Dedicated Short Range Communication (DSRC), or cellular communication protocols. The Dedicated Communication I / F 7630 typically performs V2X communication, which includes one or more of the following concepts: vehicle-to-vehicle communication, road-to-vehicle communication, vehicle-to-home communication, and pedestrian-to-vehicle communication.

[0165] The positioning unit 7640 performs positioning, for example, by receiving GNSS signals from Global Navigation Satellite System (GNSS) satellites (e.g., GPS signals from Global Positioning System (GPS) satellites) and generates location information including the vehicle's latitude, longitude, and altitude. Incidentally, the positioning unit 7640 can identify its current location by exchanging signals with a wireless access point, or by obtaining location information from a terminal such as a mobile phone, a Personal Handheld Phone System (PHS), or a smartphone with positioning capabilities.

[0166] The beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from a radio station installed on a road, thereby obtaining information about current location, congestion, road closures, or estimated time. Incidentally, the functionality of the beacon receiver 7650 can be included in the aforementioned dedicated communication I / F 7630.

[0167] The vehicle-mounted device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various vehicle-mounted devices 7760 present in the vehicle. The vehicle-mounted device I / F 7660 can establish a wireless connection using wireless communication protocols such as Wireless LAN, Bluetooth (registered trademark), Near Field Communication (NFC), or Wireless Universal Serial Bus (WUSB). Alternatively, the vehicle-mounted device I / F 7660 can establish a wired connection via a connection terminal not shown in the figure (and, if necessary, via a cable) through Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI (registered trademark)), Mobile High-Definition Link (MHL), etc. For example, the vehicle-mounted device 7760 may include at least one of passenger-owned mobile devices and wearable devices, as well as information devices carried to or attached to the vehicle. The vehicle-mounted device 7760 may also include a navigation device for searching routes to any destination. The vehicle-mounted device I / F 7660 exchanges control signals or data signals with these vehicle-mounted devices 7760.

[0168] The vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The vehicle network I / F 7680 sends and receives signals in accordance with the predetermined protocols supported by the communication network 7010.

[0169] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various types of programs based on information obtained via at least one of a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an on-board device I / F 7660, and an on-board network I / F 7680. For example, the microcomputer 7610 can calculate control target values ​​for the drive force generating device, steering mechanism, or braking device based on the obtained information about the vehicle's interior and exterior, and output control commands to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control aimed at realizing functions of an advanced driver assistance system (ADAS), including vehicle collision avoidance or mitigation, following distance-based driving, vehicle speed maintenance, vehicle collision warning, and vehicle lane departure warning. In addition, the microcomputer 7610 can control the drive force generating device, steering mechanism, braking device, etc. based on the information obtained about the vehicle's surroundings, and perform cooperative control intended for autonomous driving, which allows the vehicle to drive autonomously without relying on the driver's operation.

[0170] The microcomputer 7610 can generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information obtained via at least one of a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, a vehicle-mounted device I / F 7660, and a vehicle-mounted network I / F 7680, and also generate local map information including surrounding information about the vehicle's current position. Furthermore, the microcomputer 7610 can predict dangers such as vehicle collisions or pedestrians approaching or entering closed roads based on the obtained information, and generate warning signals. For example, the warning signal may be a signal used to generate an alarm sound or illuminate a warning light.

[0171] The audio-visual output unit 7670 sends an output signal of at least one of audio and visual information to an output device capable of visually or audibly notifying passengers of the vehicle or the outside of the vehicle. Figure 23In the example, an audio speaker 7710, a display unit 7720, and a dashboard 7730 are shown as output devices. For example, the display unit 7720 may include at least one of an in-vehicle display and a head-up display. The display unit 7720 may have augmented reality (AR) display functionality. The output device may be other than these, such as headphones, wearable devices such as glasses-type displays worn by passengers, projectors, or lamps. When the output device is a display device, the display device visually displays the results obtained from various types of processing performed by the microcomputer 7610 or information received from other control units in various forms such as text, images, tables, and charts. In addition, when the output device is an audio output device, the audio output device converts an audio signal composed of regenerated audio data or sound data into an analog signal and audibly outputs the analog signal.

[0172] Note that in Figure 22 Figure 22 Figure 22 In the example shown, at least two control units connected to each other via communication network 7010 can be integrated into one control unit. Alternatively, each individual control unit may include multiple control units. Furthermore, the vehicle control system 7000 may include other control units not shown in the figures. Additionally, some or all of the functions performed by one control unit as described above can be assigned to another control unit. That is, any control unit can perform predetermined arithmetic processing as long as information is sent and received via communication network 7010. Similarly, sensors or devices connected to one of the control units can be connected to another control unit, and multiple control units can send and receive detection information to each other via communication network 7010.

[0173] Note that this technology can have the following configuration. (1) A light detection device, comprising: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and A pixel block that includes two or more first pixels among the plurality of first pixels includes: A switch that toggles whether the internal nodes of two or more first pixels are connected to each other. (2) The optical detection device according to (1), wherein, Each of the plurality of first pixels includes a logarithmic response unit, which generates a voltage signal based on the charge. The event detection circuit detects the event based on the voltage signal, and Each of the internal nodes is a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit. (3) The optical detection device according to (1) or (2), wherein, The two or more first pixels turn on the switch and connect the internal nodes of the two or more first pixels to integrate the two or more first pixels and detect the event. (4) The optical detection device according to (3), wherein, The switch is turned on to connect the internal nodes of the two or more first pixels to each other, and the resolution of the event detection is reduced. (5) The optical detection device according to (4) further includes: Multiple pixel blocks, each pixel block containing the switch, wherein, While the multiple switches corresponding to the multiple pixel blocks are truncated, the corresponding switches are switched for each of the multiple pixel blocks based on the detection results of the events for the multiple first pixels. (6) The light detection apparatus according to any one of (1) to (5) further comprises: An output circuit outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event. (7) A light detection device, comprising: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and The event detection circuit includes an output circuit, which outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event. (8) The optical detection device according to (6) or (7), wherein, The spatiotemporal differential signal is generated by temporally differentiating a spatial differential signal generated by performing spatial differentiation between two first pixels. (9) The optical detection device according to (8), wherein, The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal is generated by detecting the difference between the output signals of the two buffer circuits of the two first pixels. (10) The optical detection device according to (8), wherein The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal is generated by detecting the difference between the output signals of the two differentiators of the two first pixels. (11) The optical detection device according to (6) or (7), wherein, The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; A comparator that compares the output signal of the differentiator with a reference signal and detects the event; and A calculator that generates the spatiotemporal differential signal by calculating the XOR of the output signals of the two comparators. (12) The light detection apparatus according to any one of (1) to (11), wherein, The plurality of first pixels are divided into a plurality of pixel blocks, and each pixel block includes two or more of the plurality of first pixels. Control lines controlling at least one of resolution, sensitivity, power gating, and responsiveness are connected to each pixel block among the plurality of pixel blocks. (13) The optical detection device according to (12), wherein, At least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block is controlled by a plurality of control signals transmitted through a plurality of control lines connected to the plurality of pixel blocks. (14) The optical detection device according to (13), wherein, The control signal controls the threshold for detecting the event in each of the two or more first pixels in the corresponding pixel block. (15) The optical detection device according to (13), wherein, The control signal controls whether to connect the internal nodes of two or more first pixels contained in the corresponding pixel block. (16) The light detection apparatus according to any one of (1) to (15), further comprising: Multiple second pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of second pixels includes: The second photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and A pixel circuit that generates a pixel signal corresponding to the amount of incident light based on the charge. (17) The optical detection device according to (16), wherein, The plurality of first pixels are arranged in a dispersed manner among the plurality of second pixels. (18) The optical detection device according to (17), wherein, For each pixel block comprising a plurality of second pixels that are adjacent to each other in a first direction and a second direction that intersect each other, one or more of the plurality of first pixels are arranged as first pixels. (19) The light detection device according to (17) or (18) further includes: The second pixel block comprises two adjacent first pixel blocks arranged in a first direction and a second direction that intersect each other. Each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction. In the second pixel block, which contains 2 × 2 = 4 first pixel blocks, one of the two first pixel blocks arranged diagonally contains four green second pixels and the other contains three green second pixels and one first pixel. In the remaining two first pixel blocks, one contains three blue second pixels and one first pixel, and the other contains four red second pixels. (20) The light detection device according to (17) or (18) further includes: The second pixel block comprises two adjacent first pixel blocks arranged in a first direction and a second direction that intersect each other. Each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction. In the second pixel block comprising 2 × 2 = 4 first pixel blocks, each of the two first pixel blocks arranged diagonally contains three green second pixels and one first pixel; one of the remaining two first pixel blocks contains three blue second pixels and one first pixel, and the other contains three red second pixels and one first pixel. The four first pixels, each comprising one of the four first pixel blocks, are arranged adjacent to each other in the first direction and the second direction.

[0174] This disclosure is not limited to the various embodiments described above, but includes various modifications that can be conceived by those skilled in the art, and the effects of this disclosure are not limited to the foregoing. In other words, various additions, modifications, and partial deletions can be made without departing from the conceptual idea and spirit of this disclosure derived from the matters defined in the claims and their equivalents. List of reference numerals 1. Optical detection device 2-pixel array 4. Event Detection Circuit 5 EVS Digital Circuits 6. Control Circuit 7 Bias Circuit 8 grayscale pixels 9 EVS pixels 12 AFE circuit 12L AFE circuit 13-pixel circuit 15. Reasoning device 16 Differential Units 17 ExOR Circuit 21 Logarithmic Response Unit 21a First Logarithmic Response Unit 21b Second Logarithmic Response Unit 21c Third Logarithmic Response Unit 21d Fourth Logarithmic Response Unit 22 Buffer 23 Differentiator 24 comparators 25 Output Circuit 26 Switch 27 Logic Circuits 30 Electronic devices 31 Camera lens 32 Graphics Processing Department 33 Storage Department 34 Control Department

Claims

1. A light detection device, comprising: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and A pixel block that includes two or more first pixels among the plurality of first pixels includes: A switch that toggles whether the internal nodes of two or more first pixels are connected to each other.

2. The optical detection device according to claim 1, wherein, Each of the plurality of first pixels includes a logarithmic response unit, which generates a voltage signal based on the charge. The event detection circuit detects the event based on the voltage signal, and Each of the internal nodes is a node connected to the first photoelectric conversion element, a node connected to the logarithmic response unit, or a node connected to the event detection circuit.

3. The optical detection device according to claim 1, wherein, The two or more first pixels turn on the switch and connect the internal nodes of the two or more first pixels to integrate the two or more first pixels and detect the event.

4. The optical detection device according to claim 3, wherein, The switch is turned on to connect the internal nodes of the two or more first pixels to each other, and the resolution of the event detection is reduced.

5. The optical detection device according to claim 4, further comprising: Multiple pixel blocks, each pixel block containing the switch, wherein, While the multiple switches corresponding to the multiple pixel blocks are truncated, the corresponding switches are switched for each of the multiple pixel blocks based on the detection results of the events for the multiple first pixels.

6. The optical detection device according to claim 1, further comprising: An output circuit outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event.

7. A light detection device, comprising: Multiple first pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of first pixels includes: The first photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and An event detection circuit, which detects events caused by changes in brightness based on the charge detection, and The event detection circuit includes an output circuit, which outputs a spatiotemporal differential signal between two or more of the plurality of first pixels as the event.

8. The optical detection device according to claim 6, wherein, The spatiotemporal differential signal is generated by temporally differentiating a spatial differential signal generated by performing spatial differentiation between two first pixels.

9. The optical detection device according to claim 8, wherein, The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal is generated by detecting the difference between the output signals of the two buffer circuits of the two first pixels.

10. The optical detection device according to claim 8, wherein... The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; and A comparator compares the output signal of the differentiator with a reference signal and detects the event. The spatial differential signal is generated by detecting the difference between the output signals of the two differentiators of the two first pixels.

11. The optical detection device according to claim 6, wherein, The event detection circuit includes: A buffer circuit that buffers the voltage signal obtained by logarithmically converting the charge; A differentiator that performs time differentiation on the output voltage of the buffer circuit; A comparator that compares the output signal of the differentiator with a reference signal and detects the event; and A calculator that generates the spatiotemporal differential signal by calculating the XOR of the output signals of the two comparators.

12. The optical detection device according to claim 1, wherein, The plurality of first pixels are divided into a plurality of pixel blocks, and each pixel block includes two or more of the plurality of first pixels. Control lines controlling at least one of resolution, sensitivity, power gating, and responsiveness are connected to each pixel block among the plurality of pixel blocks.

13. The optical detection device according to claim 12, wherein, At least one of the resolution, sensitivity, power gating, and responsiveness of the corresponding pixel block is controlled by a plurality of control signals transmitted through a plurality of control lines connected to the plurality of pixel blocks.

14. The optical detection device according to claim 13, wherein, The control signal controls the threshold for detecting the event in each of the two or more first pixels in the corresponding pixel block.

15. The optical detection device according to claim 13, wherein, The control signal controls whether to connect the internal nodes of two or more first pixels contained in the corresponding pixel block.

16. The optical detection device according to claim 1, further comprising: Multiple second pixels, arranged in a one-dimensional or two-dimensional direction, wherein, Each of the plurality of second pixels includes: The second photoelectric conversion element accumulates a charge corresponding to the amount of incident light; and A pixel circuit that generates a pixel signal corresponding to the amount of incident light based on the charge.

17. The optical detection device according to claim 16, wherein, The plurality of first pixels are arranged in a dispersed manner among the plurality of second pixels.

18. The optical detection device according to claim 17, wherein, For each pixel block comprising a plurality of second pixels that are adjacent to each other in a first direction and a second direction that intersect each other, one or more of the plurality of first pixels are arranged as first pixels.

19. The light detection device according to claim 17, further comprising: The second pixel block, wherein two first pixel blocks are arranged adjacent to each other in a first direction and a second direction that intersect each other, and each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction, wherein, In the second pixel block, which contains 2 × 2 = 4 first pixel blocks, one of the two first pixel blocks arranged diagonally contains four green second pixels and the other contains three green second pixels and one first pixel. In the remaining two first pixel blocks, one contains three blue second pixels and one first pixel, and the other contains four red second pixels.

20. The light detection device according to claim 17, further comprising: The second pixel block, wherein two first pixel blocks are arranged adjacent to each other in a first direction and a second direction that intersect each other, and each first pixel block contains 2 × 2 = 4 second pixels that are adjacent to each other in the first direction and the second direction, wherein, In the second pixel block comprising 2 × 2 = 4 first pixel blocks, each of the two first pixel blocks arranged diagonally contains three green second pixels and one first pixel; one of the remaining two first pixel blocks contains three blue second pixels and one first pixel, and the other contains three red second pixels and one first pixel. The four first pixels, each comprising one of the four first pixel blocks, are arranged adjacent to each other in the first direction and the second direction.

Images