Light detection device and light detection system

Abstract

[Problem] To enable reduced size, reduced cost, and reduced energy consumption, and to reduce an amount of output data including event information. [Solution] A light detection device comprising: a pixel array unit having a plurality of pixels which detect an event if a change in the luminance of incident light exceeds a prescribed threshold; a first control unit that switches a frequency with which the event is detected in a first region which is part of a pixel region including the plurality of pixels, and a second region that is at least partially different from the first region; and a second control unit that switches between access control for the first region and access control for the second region.

Description

Optical detection device and optical detection system

[0001] The present disclosure relates to optical detection devices and optical detection systems.

[0002] An event-based sensor (EVS) outputs an event signal when a change in the luminance of incident light exceeds a threshold. Compared to conventional image sensors, it can output event signals at lower power consumption and higher speeds, making it effective for tracking fast-moving objects. Conventional EVSs may output multiple event signals in response to swaying leaves in the wind around the moving object of interest. Therefore, a technology has been proposed in which, in addition to the EVS pixels that output event signals, grayscale pixels capable of acquiring grayscale information are provided to perform pattern recognition, and the probability of detecting an event is controlled in units of one or more pixel circuit regions according to the pattern recognition results (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-136958

[0004] In Patent Document 1, grayscale pixels are provided in addition to EVS pixels, and pattern recognition is performed within the sensor using the grayscale pixels, which complicates the internal configuration of the sensor. Furthermore, Patent Document 1 also provides an arbiter that arbitrates multiple event signals output from multiple EVS pixels, further complicating the internal configuration of the sensor. Therefore, the technology of Patent Document 1 increases the size of the sensor, increasing costs and power consumption. Furthermore, Patent Document 1 does not take into consideration reducing the amount of output data, including event information.

[0005] Therefore, the present disclosure provides a photodetector and a photodetection system that can be made smaller, less expensive, and consume less power, and that can reduce the amount of output data including event information.

[0006] In order to solve the above problems, according to the present disclosure, there is provided a light detection device comprising: a pixel array section having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control section that switches the frequency of detecting the event between a first region that is a part of the pixel region including the plurality of pixels and a second region that is at least partially different from the first region; and a second control section that switches between access control for the first region and access control for the second region.

[0007] The first region may be a region of interest having a size smaller than that of the second region, and the first control unit may detect the event in the first region more frequently than in the second region.

[0008] The first control unit may switch the resolution at which the event is detected between the first area and the second area.

[0009] The first control unit may detect the event for each of all the pixels in the first region, and may detect the event for each of two or more of the pixels in the second region.

[0010] The first control unit may detect the event for each of all the pixels in the second region, and may detect the event for each of two or more of the pixels in the first region.

[0011] The first region or the second region may include a current summing circuit that sums currents generated by photoelectric conversion in two or more of the pixels.

[0012] The pixel may further include a switch that switches whether or not the currents generated by photoelectric conversion in two or more of the pixels adjacent to each other in a first direction or a second direction intersecting the first direction are to be added in the current adding circuit.

[0013] Each of the plurality of pixels may have a photoelectric conversion element that generates an electric charge according to the amount of incident light, and a source follower circuit that outputs a voltage according to the electric charge, and the first region or the second region may have a voltage averaging circuit that averages the voltages output from the source follower circuits of two or more of the pixels.

[0014] The pixel may further include a switch that switches whether or not to average the voltages output from the source follower circuits of two or more pixels adjacent in a first direction or a second direction intersecting the first direction.

[0015] The plurality of pixels in the pixel array unit may detect the event a plurality of times in the first region and may detect the event once in the second region during one frame period.

[0016] The image sensor may further include a signal processing circuit that generates event data including a detection signal of the event output from the plurality of pixels and additional information related to the detection signal, wherein the additional information may include at least one of a pixel position of the event, information specifying the first region or the second region in which the event was detected, and a detection time of the event.

[0017] The event detection signal output device may further include a signal processing circuit that performs at least one of a filtering process that thins out the event detection signals output from the plurality of pixels, and a compression process that compresses the event detection signals output from the plurality of pixels.

[0018] Each of the plurality of pixels may have an event detection circuit that detects the event, and a threshold value serving as a reference for detecting the event in the event detection circuit may differ between the first region and the second region.

[0019] The event detection circuit may have a current-voltage conversion circuit, a buffer, a differentiation circuit, a comparator, and an output circuit, and the frequency of operation of at least one of the buffer, the differentiation circuit, the comparator, and the output circuit may differ between the first area and the second area.

[0020] The plurality of pixels may include a first pixel that outputs a pixel signal including gradation information according to the amount of incident light, and a second pixel that detects the event.

[0021] The plurality of pixels may each include two or more pixels arranged along a first direction, and may have a plurality of pixel groups arranged in a second direction intersecting the first direction, and the pixel array unit may sequentially output the event detection signal for each pixel group in which the event occurred, excluding a pixel group in which the event did not occur, among the plurality of pixel groups.

[0022] According to the present disclosure, there is provided an optical detection system comprising: an optical detection device that outputs an event signal; and a signal processing device having a signal processing circuit that performs signal processing based on the event signal, wherein the optical detection device comprises: a pixel array unit having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control unit that switches the frequency of detecting the event between a first region that is a part of the pixel region of the pixel array unit and a second region other than the first region; and a second control unit that switches between access control for the first region and access control for the second region.

[0023] The signal processing device may receive a detection signal of the event detected in the first region and first additional information added to the detection signal from the photodetector via a first virtual channel, and may also receive a detection signal of the event detected in the second region and second additional information added to the detection signal from the photodetector via a second virtual channel.

[0024] The signal processing device may identify a frame in which the event occurred based on the first additional information and the second additional information.

[0025] The signal processing device may perform Debra processing based on the event detection signal transmitted from the light detection device.

[0026] 1. A block diagram of a photodetection system according to an embodiment of the present disclosure. 2. A schematic perspective view showing an example of a photodetection device according to the present disclosure having a two-layer stacked structure. 3. A schematic perspective view showing an example of a photodetection device according to the present disclosure having a three-layer structure. 4. A block diagram of a photodetection device and a photodetection system according to an embodiment of the present disclosure. 5. A circuit diagram of an EVS pixel. 6. A circuit diagram of an EVS pixel that simplifies the circuit configuration of FIG. 4A. 7. A circuit diagram of an EVS pixel according to a modification of FIG. 4. 8. A circuit diagram of an EVS pixel that simplifies the circuit configuration of FIG. 5A. 9. A diagram explaining first and second regions set in any pixel region of a pixel array unit. 10. A block diagram showing the internal configuration of a timing control unit and an access control unit. 11. A timing diagram of a photodetection device according to an embodiment. 12. A diagram showing the format of event data output by a signal processing unit. 13. A diagram explaining the first and second regions. 14. A timing diagram corresponding to FIG. 10. 15. A diagram showing the format of event data output by a signal processing unit. 16. A diagram explaining the first and second regions according to a modification of FIG. 10. 17. A timing diagram corresponding to FIG. 13. 18. A diagram showing the format of event data corresponding to FIG. 13. 31. A diagram showing an example in which a third region is arranged to include the first and second regions. A timing chart corresponding to FIG. 16. A diagram showing an example in which four regions of interest are set. A timing chart corresponding to FIG. 18. A diagram showing an example in which three regions of interest are in an inclusive relationship. A block diagram of the periphery of a pixel array unit when thinning processing is performed by an EVS readout unit. A block diagram of the periphery of a pixel array unit when thinning processing is performed by a signal processing unit. A timing diagram for when pixel rows in which no events have occurred are skipped. A diagram for explaining a first example of binning processing. A diagram for explaining a second example of binning processing. A diagram for explaining a third example of binning processing. A diagram for explaining a fourth example of binning processing. A diagram showing a connection between a photodetector and an AP. A block diagram showing a first example of the internal configuration of the signal processing unit. A flowchart showing the processing operation of the signal processing unit of FIG. 29. A block diagram showing a second example of the internal configuration of the signal processing unit. A flowchart showing the processing operation of the signal processing unit of FIG. 31. A diagram for explaining the threshold of a first region and the threshold of a second region that includes the first region therein. A block diagram of an event detection circuit provided in each pixel in the pixel array unit. Fig. 36 is a block diagram of an event detection circuit that controls output of an event signal. Fig. 37 is a block diagram of an event detection circuit according to a first modified example of Fig. 35. Fig. 38 is a block diagram of an event detection circuit according to a first modified example of Fig. 35. Fig. 39 is a plan view showing a pixel block according to a first example that can be applied to a pixel array unit according to the present embodiment.41. A plan view showing a pixel block according to a second example that can be applied to the pixel array unit according to the present embodiment. A plan view showing a pixel block according to a third example that can be applied to the pixel array unit according to the present embodiment. A plan view showing a pixel block according to a fourth example that can be applied to the pixel array unit according to the present embodiment. A circuit diagram of a gradation pixel. A block diagram showing the internal configurations of a signal processing unit, an output I / F, and an AP. A block diagram showing a schematic configuration of a light detection system including a light detection device having a region-of-interest determination unit. A block diagram showing a schematic configuration of a light detection system according to a first modified example of FIG. 41. A block diagram showing a schematic configuration of a light detection system according to a second modified example of FIG. 41. A block diagram showing a schematic configuration of a light detection system according to a third modified example of FIG. 41. A block diagram showing an example of a schematic configuration of a vehicle control system. An explanatory diagram showing an example of the installation positions of an outside vehicle information detection unit and an imaging unit.

[0027] Hereinafter, an embodiment of a photodetector will be described with reference to the drawings. The following description will focus on the main components of the photodetector, but the photodetector may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0028] FIG. 1 is a block diagram of a light detection system 30 according to an embodiment of the present disclosure. This light detection system 30 has a function of generating an image according to the brightness of incident light. The light detection system 30 in FIG. 1 includes a light detection device 1, an imaging lens 31, an image processing unit 32, a recording unit 33, and a control unit 34. The light detection system 30 is applicable to various electronic devices, such as a surveillance camera or a camera mounted on an industrial robot, or a general-purpose camera, but the specific use and configuration of the light detection system 30 are arbitrary.

[0029] The imaging lens 31 collects incident light and guides it to the photodetector 1. The photodetector 1 captures an image of the incident light. The photodetector 1 causes light within a predetermined wavelength range, such as visible light or infrared light, to be incident on multiple pixels, and outputs an event signal from a pixel whose luminance change exceeds a predetermined threshold. The event signal output from the photodetector 1 is sent to an image processor 32 and a recorder 33. In this specification, each of the multiple pixels included in the photodetector 1 may be referred to as an EVS (Event-based Vision Sensor) pixel. The photodetector 1 may include a pixel that outputs a pixel signal containing gradation information corresponding to the amount of incident light. In this specification, a pixel that outputs a pixel signal containing gradation information may be referred to as a gradation pixel.

[0030] The image processing unit 32 performs predetermined image processing on the captured image, such as color or brightness adjustment, image compression, image recognition, tracking, or analysis. The image processed by the image processing unit 32 is recorded in the recording unit 33, for example.

[0031] The recording unit 33 records the image output from the light detection device 1 or the image processing unit 32. The recording unit 33 may be disposed in a server connected via a network. In the light detection system 30 according to this embodiment, at least one of the image processing unit 32 and the recording unit 33 in FIG. 1 can be omitted.

[0032] The control unit 34 controls the operation of the light detection device 1. Although not specifically shown in FIG.

[0033] (Two-Layer Stacking) The photodetector 1 according to the present disclosure can be realized as a stacked chip. FIG. 2A is a schematic perspective view illustrating an example of a two-layer stacked photodetector 1 according to the present disclosure. The photodetector 1 in FIG. 2A includes a first substrate SB1 disposed on the light incident surface side and a second substrate SB2 stacked on the first substrate SB1. For example, a photoelectric conversion element for each pixel is disposed on the first substrate SB1. This specification mainly describes an example in which the photoelectric conversion element is a photodiode. Peripheral circuits (e.g., transfer transistors, etc.) for the photodiode may also be disposed on the first substrate SB1. Multiple transistors for generating event signals are disposed on the second substrate SB2. The first substrate SB1 and the second substrate SB2 are bonded and transmit signals, for example, via a copper-copper connection (CCC). Alternatively, the first substrate SB1 and the second substrate SB2 may be bonded using vias or bumps other than CCC. In this specification, the first substrate SB1 may be referred to as a light-receiving chip SB1, and the second substrate SB2 may be referred to as a detection chip SB2.

[0034] (Three-Layer Stacking) The photodetector 1 according to the present disclosure can be configured by stacking three or more substrates. FIG. 2B is a schematic perspective view showing an example of a three-layer structure of the photodetector 1 according to the present disclosure. FIG. 2B shows an example of a stacked structure of the photodetector 1 including a first substrate SB1, a second substrate SB2, and a third substrate SB3. The first substrate SB1 is provided with a photoelectric conversion element for each pixel. The second substrate SB2 is provided with, for example, a current-voltage conversion circuit and an event detection circuit. The third substrate SB2 is provided with, for example, a signal processing circuit.

[0035] The circuit elements and the like arranged on the first substrate SB1, the second substrate SB2, and the third substrate SB3 are arbitrary and may be combined in any desired manner. Below, an example in which the photodetector 1 according to the present disclosure has the two-layer stacked structure shown in FIG. 2A will be mainly described. Hereinafter, the first substrate SB1 may be referred to as the light-receiving chip, and the second substrate SB2 may be referred to as the detection chip.

[0036] 3 is a block diagram of the light detection device 1 and a light detection system 30 according to an embodiment of the present disclosure. The light detection system 30 according to the embodiment includes the light detection device 1 and an application processor (hereinafter, referred to as AP) 2.

[0037] As shown in FIG. 3 , the photodetector 1 according to one embodiment includes a pixel array unit 3, an EVS readout unit 4, a timing control unit (first control unit) 5, an access control unit (second control unit) 6, a signal processing unit 7, a timestamp generation unit 8, and an output interface unit (hereinafter, output I / F) 9.

[0038] The pixel array unit 3 has a plurality of pixels 10 arranged in a first direction (e.g., horizontal direction) X and a second direction (e.g., vertical direction) Y that intersect with each other. Each pixel 10 is an EVS pixel 10 that detects an event and outputs an event signal when a change in luminance of incident light exceeds a predetermined threshold. Each EVS pixel 10 has a photoelectric conversion element 11 and an event detection circuit 12. The photoelectric conversion element 11 is, for example, a photodiode. The specific circuit configuration of the event detection circuit 12 will be described later. Hereinafter, each pixel 10 in the pixel array unit 3 will sometimes be simply referred to as a pixel 10 and sometimes as an EVS pixel 10.

[0039] The EVS readout unit 4 controls, for example, reading out an event signal for each pixel row of the pixel array unit 3 and transmitting the event signal to the signal processing unit 7 .

[0040] The signal processing unit 7 generates event data including the event signal transmitted from the EVS readout unit 4. As will be described later, the event data includes additional information such as the pixel position where the event was detected and the time when the event was detected, in addition to the event signal. The signal processing unit 7 can also perform at least one of a filtering process that thins out the event signals output from the plurality of pixels 10 and a compression process that compresses the event signals output from the plurality of pixels 10.

[0041] The timing control unit 5 switches the frequency of event detection between a first region, which is a portion of the pixel region of the pixel array unit 3, and a second region, which is at least partially different from the first region. The first region is a region of interest (ROI). The second region may be a region of interest, or the entire pixel region of the pixel array unit 3. When both the first region and the second region are regions of interest and the first region is included in part of the second region, the first region will be referred to as ROI_1 and the second region will be referred to as ROI_2 in this specification. The internal configuration of the timing control unit 5 will be described later. It is also possible to set three or more regions of interest in the pixel array unit 3. In this case, the timing control unit 5 controls the frequency of event detection for each region of interest.

[0042] The access control unit 6 switches between access control for the first region ROI_1 and access control for the second region ROI_2. The internal configuration of the access control unit 6 will be described later. The access control includes control of reading out event signals in the first region ROI_1 or the second region ROI_2.

[0043] The timestamp generator 8 generates a timestamp indicating the time at which the event signal is output. The signal processor 7 includes the timestamp generated by the timestamp generator 8 in the event data as detection time information of the event.

[0044] The output I / F 9 controls the transmission of event data generated by the signal processing unit 7 to the AP 2. The AP 2 receives the event data transmitted from the image sensor and performs various signal processing. For example, the AP 2 generates an event image. The event image is, for example, a binary image in which the contours of moving objects are emphasized.

[0045] The image sensor and AP2 are typically configured on different semiconductor chips, but in some cases, the image sensor and AP2 may be integrated into a single stacked chip as shown in Figure 2A or 2B.

[0046] 4A and 4B are circuit diagrams of each EVS pixel 10. As shown in FIGS. 4A and 4B , the EVS pixel 10 according to this embodiment includes a photoelectric conversion element 11 and an event detection circuit 12. The event detection circuit 12 includes a current-voltage conversion circuit 13, a buffer 14, a differentiation circuit 15, a quantizer 16, and an output circuit 17. The current-voltage conversion circuit 13 and the photoelectric conversion element 11 form a logarithmic response unit.

[0047] The current-voltage conversion circuit 13 includes NMOS transistors Q1 to Q3 and a PMOS transistor Q4. The gate and drain of transistor Q2 are connected, and it functions as a diode. Transistor Q2 is not essential and can be omitted as shown in FIG. 4B. FIG. 4B has the same circuit configuration as FIG. 4A, except that transistor Q2 of FIG. 4A is omitted. The drain of transistor Q1 is connected to the power supply voltage node, and the source of transistor Q1 is connected to the gate and drain of transistor Q2. The gate of transistor Q1 is connected to the drain of transistor Q3 and the drain of transistor Q4, and this connection node is the output node of the current-voltage conversion circuit 13. The source of transistor Q4 is connected to the power supply voltage node, and a predetermined bias voltage Vlog is applied to the gate of transistor Q4.

[0048] The buffer 14 has NMOS transistors Q5 and Q6 cascode-connected between a power supply voltage node and a ground voltage node. The output node of the current-voltage conversion circuit 13 is connected to the gate of the transistor Q5. A reference voltage SF is applied to the gate of the transistor Q6. The source and drain of the transistor Q5 form the output node of the buffer 14. The transistor Q5 functions as a source follower circuit, and a voltage signal having a waveform corresponding to the source voltage of the transistor Q4 is output from the source of the transistor Q5.

[0049] 4A and 4B show an example in which the current-voltage conversion circuit 13 in the EVS pixel 10 and the transistor Q5 in the buffer 14 are arranged on the light-receiving chip SB1, and the transistor Q6 in the buffer 14 is arranged on the detection chip SB2. This is just one example, and as will be described later, for example, the transistor Q6 in the buffer 14 may be arranged on the light-receiving chip SB1, or the transistors Q5 and Q6 that constitute the buffer 14 may be arranged on the detection chip SB2.

[0050] 4A and 4B has a synchronous configuration and outputs an event signal indicating whether or not each of the EVS pixels 10 has detected an event, for example, on a pixel row basis, in synchronization with a predetermined frame period. In this way, the event signal is a binary signal indicating whether or not an event has been detected.

[0051] 4A and 4B includes a differentiating circuit 15, a quantizer 16, and an output circuit 17. The differentiating circuit 15 includes a capacitor C1, NMOS transistors Q7 and Q8, and a PMOS transistor Q9.

[0052] Capacitor C1 is connected between the output node of buffer 14 and the gate of transistor Q9. Capacitor C1 supplies a current to the gate of transistor Q9 that corresponds to the amount of change in voltage signal Vp, which is the time derivative of voltage signal Vp output from buffer 14. Transistors Q9 and Q8 are cascode-connected between a power supply voltage node and a ground voltage node. A switch 18 that selects between two bias voltages is connected to the gate of transistor Q8. Switch 18 selects either the auto-zero bias voltage AZ or the event signal bias voltage EVT based on a switching control signal BIAS_SW output from access control unit 6. Through switching control by switch 18, the gate of transistor Q8 is set to an optimal bias voltage.

[0053] The differentiating circuit 15 outputs a differentiated signal from the drain of the transistor Q8 and the source of the transistor Q9. This differentiated signal is input to the quantizer 16.

[0054] The quantizer 16 has a PMOS transistor Q11 and an NMOS transistor Q10 cascode-connected between a power supply voltage node and a ground voltage node. A threshold signal Vth is applied to the gate of the transistor Q10. The gate of the transistor Q11 is connected to the drains of the transistors Q8 and Q9, and receives the differentiated signal output from the differentiating circuit 15. If the voltage level of the differentiated signal is higher than the voltage level of the threshold signal Vth, the quantizer 16 outputs a high-level event signal indicating that an event has been detected.

[0055] The output circuit 17 has a logic circuit 17a configured with logic gates, flip-flops, etc. A control signal LATCH_EN output from the access control unit 6 is input to the logic circuit 17a. The control signal LATCH_EN is a signal that defines the timing for latching an event signal. The logic circuit 17a outputs the event signal in synchronization with the frame period.

[0056] 4A and 4B are merely examples, and various circuit configurations are applicable. For example, the event detection circuit 12 in FIG. 4A and 4B generates one type of event signal COMP, but may generate two or more types of event signals COMP+ and COMP-.

[0057] 5A and 5B are circuit diagrams of an EVS pixel 10 according to a modification of FIGS. 4A and 4B . FIG. 5B has the same circuit configuration as FIG. 5A , except that the transistor Q2 in FIG. 5A is omitted. In FIGS. 5A and 5B , components common to FIGS. 4A and 4B are designated by the same reference numerals, and the following description will focus on the differences. The configuration of the quantizer 16 in the event detection circuit 12 in FIGS. 5A and 5B is partially different from that of the quantizer 16 in FIGS. 4A and 4B . The quantizer 16 in FIGS. 5A and 5B includes an inverter 19 and transistors Q12 and Q13 in addition to the configuration of the quantizer 16 in FIGS. 4A and 4B . Furthermore, the switch 18 selects one of the auto-zero bias voltage AZ, the bias voltage POS for positive-polarity event detection, and the bias voltage NEG for negative-polarity event detection based on a switching control signal BIAS_SW output from the access control unit 6. The gate of transistor Q8 is set to an optimum bias voltage by switching control of switch 18. Control signals LATCH_EN and POLARITY output from access control unit 6 are input to logic circuit 17a. The control signal LATCH_EN is a signal that defines the timing for latching the event signal. The control signal POLARITY is a signal that specifies the polarity of the event signal. The logic circuit 17a outputs the event signal in synchronization with the frame period.

[0058] Transistors Q12 and Q13 are cascode-connected between a power supply voltage node and a reference voltage (e.g., ground) node. The output signal Vout of the differentiating circuit 15 is applied to the gate of transistor Q13. A threshold voltage Vlow is applied to the gate of transistor Q12. A threshold voltage Vhigh is applied to the gate of transistor Q10. Transistors Q12 and Q13 compare the output signal Vout with the threshold voltage Vlow. Specifically, when the output signal Vout of the differentiating circuit 15 is higher than the threshold voltage Vlow, transistor Q13 turns off, and the event signal COMP- output from the drain of transistor Q13 goes low. In this specification, the event signal COMP+ may be referred to as an on-event signal, and the event signal COMP- may be referred to as an off-event signal.

[0059] The quantizer 16 in FIGS. 5A and 5B can switch between an on-event signal and an off-event signal and output them from the sources of the transistors Q11 and Q13 by switching the gate voltage of the transistor Q8 using the switch 18.

[0060] 6 is a diagram illustrating a first region ROI_1 and a second region ROI_2 set in an arbitrary pixel region of the pixel array unit 3. In the example of FIG. 6, the first region ROI_1 is included in part of the second region ROI_2. As will be described later, the first region ROI_1 and the second region ROI_2 may be arranged so as not to overlap each other. The second region ROI_2 is part or all of the pixel region of the pixel array unit 3. FIG. 6 shows an example in which the second region ROI_2 is a part of the pixel array unit 3.

[0061] In this way, the first region ROI_1 is a region of interest that is smaller in size than the second region ROI_2. In the example of Fig. 6, the first region ROI_1 detects events that occur due to a moving object (e.g., a vehicle) included in a part of the pixel region of the pixel array unit 3. The second region ROI_2 detects events that occur due to not only a vehicle but also a moving object of less importance, such as a tree swaying in the wind, that is located around the moving object.

[0062] The timing control unit 5 detects events in the first region ROI_1 more frequently than in the second region ROI_2. That is, the timing control unit 5 sets the frame rate of the first region ROI_1 higher than the frame rate of the second region ROI_2. This allows events in the first region ROI_1 to be detected multiple times during the detection period of events in the second region ROI_2, thereby enabling accurate detection of the movement of moving objects (vehicles) in the first region ROI_1.

[0063] 7 is a block diagram showing the internal configuration of the timing control unit 5 and the access control unit 6. One timing control unit 5 is provided for the pixel array unit 3, while one access control unit 6 is provided for each row or for multiple rows. Note that it is also possible to integrate multiple access control units 6 into one, and in FIG. 3 and other figures, one access control unit 6 is provided for the pixel array unit 3.

[0064] The timing control unit 5 generates a frame synchronization signal, a line synchronization signal, an event detection control signal, an AZ control signal, a low power consumption control signal, and an event transfer signal, and sends these signals to at least one of the access control unit 6 and the signal processing unit 7. The timing control unit 5 also counts frames. A frame may refer to the entire pixel array unit 3, or the entire first region ROI_1 or the entire second region ROI_2. In this specification, the entire first region ROI_1 or the entire second region ROI_2 may be called a frame or a subframe.

[0065] The frame synchronization signal is a signal synchronized with a frame or subframe. The frame count refers to the number of times that an event signal is output over the entire area of ​​the pixel array unit 3. Alternatively, the frame count refers to the number of times that an event signal is output over the entire area of ​​the first region ROI_1 or the second region ROI_2.

[0066] The timing control unit 5 has an ROI_1 setting unit 5a, an ROI_2 setting unit 5b, a multiplexer 5c, and a region control unit 5d.

[0067] The ROI_1 setting unit 5a outputs information specifying the first region ROI_1, and the ROI_2 setting unit 5b outputs information specifying the second region ROI_2. The specifying information includes information such as pixel positions and sizes of the first region ROI_1 or the second region ROI_2.

[0068] The multiplexer 5c selects the output of either the ROI_1 setting unit 5a or the ROI_2 setting unit 5b based on the frame count. Information specifying the first region ROI_1 or the second region ROI_2 selected by the multiplexer 5c is sent to the access control unit 6 and the signal processing unit 7.

[0069] The region control unit 5d outputs pixel position information of the first region ROI_1 or the second region ROI_2 and a data enable signal based on the information specifying the first region ROI_1 or the second region ROI_2 selected by the multiplexer 5c. The timing control unit 5 sends a frame count and a data enable signal to the signal processing unit 7.

[0070] The access control unit 6 has a plurality of logic gates 6 a, 6 b, 6 c, and 6 d, an ROI determination unit 6 e, and a decoder 6 f. The plurality of logic gates perform various logical operations based on the frame synchronization signal, line synchronization signal, event detection control signal, AZ control signal, low power consumption control signal, and event transfer signal sent from the timing control unit 5, to generate the event detection signal, AZ control signal, low power consumption control signal, and event transfer signal.

[0071] As described above, the timing control unit 5 can switch the frequency of event detection between the first region ROI_1 and the second region ROI_2. This means that the temporal resolution can be different between the first region ROI_1 and the second region ROI_2. Furthermore, the timing control unit 5 and the access control unit 6 may be able to switch the resolution of event detection between the first region ROI_1 and the second region ROI_2. This means that the spatial resolution can be different between the first region ROI_1 and the second region ROI_2.

[0072] 7 shows an example in which the plurality of logic gates includes three AND gates 6a, 6b, and 6d and one NAND gate 6c, but the types and number of logic gates are arbitrary. Fig. 7 shows an example in which the AND gate 6a outputs an event detection signal, the AND gate 6b outputs an AZ control signal, the NAND gate 6c outputs a low power consumption control signal, and the AND gate 6d outputs an event transfer signal.

[0073] The ROI discrimination unit 6e discriminates between the first region ROI_1 and the second region ROI_2 based on the output signal of the multiplexer 5c in the timing control unit 5. The AND gate 6c generates a low power consumption control signal for the first region ROI_1 or the second region ROI_2 based on the discrimination information of the ROI discrimination unit 6e and the low power consumption control signal.

[0074] The decoder 6f outputs a signal to select the first region ROI_1 or the second region ROI_2 based on the output signal of the region control unit 5d. The AND gate 6d transmits the event transfer signal to the first region ROI_1 or the second region ROI_2 based on the output signal of the decoder 6f and the event transfer signal.

[0075] The event detection signal, AZ control signal, low power consumption control signal, and event transfer signal generated by the access control unit 6 are sent to the pixel array unit 3. The event signal output from the pixel array unit 3 is sent to the signal processing unit 7 via the EVS readout unit 4 shown in FIG.

[0076] 8 is a timing diagram of the photodetector 1 according to an embodiment. During the period from time t1 to t2, an event is detected within the second region ROI_2. During the period from time t2 to t3, an AZ signal is input to the differential circuit 15 within the event detection circuit 12 in the second region ROI_2, and an event reset operation is performed. During the period from time t3 to t4, the EVS readout unit 4 within the second region ROI_2 reads out an event signal, and the signal processing unit 7 processes the event signal.

[0077] Thereafter, during the period from time t5 to t6, event detection is performed within the first region ROI_1. During the period from time t6 to t7, a reset operation of the differentiation circuit 15 within the first region ROI_1 is performed. During the period from time t7 to t8, the EVS readout unit 4 within the first region ROI_1 reads out an event signal, and the signal processing unit 7 processes the event signal.

[0078] Thereafter, the event detection process for the first region ROI_1 similar to that from time t5 to t8 is performed a total of four times (times t9 to t12, t13 to t16, and t17 to t20).

[0079] Thereafter, the event detection process for the second region ROI_2, which is similar to the process from time t1 to time t4, is performed once (time t21 to time t24).

[0080] As shown in Fig. 8 , after an event detection process is performed once for the second region ROI_2, the event detection process is performed four times consecutively for the first region ROI_1. In this embodiment, the time period for this series of processes is referred to as a frame period, and the period for each event detection process is referred to as a subframe period. In the example of Fig. 8 , one frame period includes one subframe period for the second region ROI_2 and four subframe periods for the first region ROI_1.

[0081] In FIG. 8, the event detection process for the first region ROI_1 is performed four times in succession, but the specific number of times is arbitrary, and any number of times equal to or greater than two can be set.

[0082] 9 is a diagram showing the format of event data output by the signal processing unit 7. As shown in FIG. 9, event data is generated for each subframe. The event data generated by the event detection process for the first region ROI_1 and the event data generated by the event detection process for the second region ROI_2 have the same data format, but since the second region ROI_2 has a larger number of pixels, the number of event data may be larger.

[0083] As shown in FIG. 9 , the event data has a data structure in which embedded data (EBD) and line data for each row are arranged between a frame start signal (FS) and a frame end signal (FE). The EBD is arranged between a line header (LH) and a line footer (LF). The EBD includes event control information, such as information regarding resolution (spatial resolution), information identifying the region, or timestamp information. The information included in the EBD is arbitrary. The line data for each row includes a line header (LH), binary data indicating the presence or absence of an event for each pixel 10, and a line footer (LF). The number of line data provided is equal to the number of pixel rows in the first region ROI_1 or the second region ROI_2. FIG. 9 shows an example in which the event data includes line data for N rows (N is an integer equal to or greater than 2). As described below, line data for rows in which no event occurred may be omitted.

[0084] FIG. 10 is a diagram illustrating a first region ROI_1 and a second region ROI_2. As shown in FIG. 10, the first region ROI_1 is provided within the second region ROI_2, for example. The second region ROI_2 may be the entire pixel array section 3 or a portion of the pixel array section 3. Because the first region ROI_1 is a more important region of interest than the second region ROI_2, event detection is performed on all pixels 10 within the first region ROI_1. Because the second region ROI_2 is not as important as the first region ROI_1, event detection is performed, for example, by thinning or by binning. When thinning event detection is performed, event detection is performed on each of multiple pixels 10 within the second region ROI_2, and event detection is omitted for the other pixels 10, or event detection results are not output. When binning event detection is performed, event detection is performed on a pixel block including any number of pixels 10 equal to or greater than two. For example, an event may be detected by the sum of the currents flowing through a plurality of pixels 10 in a pixel block, or the summed average value of the voltages in a plurality of pixels 10. Alternatively, if at least one pixel 10 in a pixel block consisting of four pixels 10 detects an event, an event signal may be output from that pixel block.

[0085] In the second region ROI_2, event detection is performed in units of two or more predetermined number of pixels 10. In the example of Fig. 10, event detection is performed in units of 2 x 2 = 4 pixels in the second region ROI_2. Meanwhile, event detection is performed for each pixel 10 in the first region ROI_1.

[0086] FIG. 11 is a timing diagram corresponding to FIG. 10. In the example of FIG. 11, the event detection process for the first region ROI_1 is performed twice within the period in which the event detection process for the second region ROI_2 is performed. During the period from time t1 to t4 in FIG. 11, the event detection process, reset process, and event signal readout process for the second region ROI_2 are performed consecutively. Thereafter, during the period from time t5 to t8 and the period from time t9 to t12, the event detection process, reset process, and event signal readout process for the first region ROI_1 are performed consecutively.

[0087] Fig. 12 is a diagram showing the format of event data output by the signal processing unit 7. The format of the event data is the same as that shown in Fig. 9. One frame of event data includes one subframe of event data corresponding to one event detection process in the second region ROI_2, and two subframes of event data corresponding to two event detection processes in the first region ROI_1.

[0088] 10 to 12 , in the first region ROI_1, events are detected for each pixel 10, and in the second region ROI_2, for example, binning processing is performed to detect events for each set of pixels 10. This allows event detection to be performed at high resolution in the region of interest, while event detection in the second region ROI_2 can be performed at low resolution and at high speed. Performing event detection in the second region ROI_2 at low resolution reduces the amount of event data sent from the photodetector 1 to the AP 2, speeding up transmission of the event data and reducing the processing load on the signal processing unit 7 in the photodetector 1 and the AP 2, thereby reducing power consumption.

[0089] FIG. 13 is a diagram illustrating a first region ROI_1 and a second region ROI_2 according to a modified example of FIG. 10 . In FIG. 13 , similarly to FIG. 10 , the first region ROI_1 is provided inside the second region ROI_2, but binning processing is performed in the first region ROI_1, and events are detected in units of multiple pixels 10. On the other hand, in the second region ROI_2, event detection is performed for each pixel 10. That is, the event detection processing for the first region ROI_1 in FIG. 13 is the same as the event detection processing for the second region ROI_2 in FIG. 10 , and the event detection processing for the second region ROI_2 in FIG. 13 is the same as the event detection processing for the first region ROI_1 in FIG. 10 .

[0090] 14 is a timing diagram corresponding to FIG. 13. One frame period includes one subframe for event detection in the second region ROI_2 and two subframes for event detection in the first region ROI_1. Event detection in the first region ROI_1 can be performed faster than in FIG. 11.

[0091] Fig. 15 is a diagram showing the format of the event data corresponding to Fig. 13. The format of the event data is basically the same as that of Fig. 12, but the event data of the event detection process for the second region ROI_2 includes event signals for all pixels 10 in the second region ROI_2. Also, the event data of the event detection process for the first region ROI_1 includes event signals for each of multiple pixels 10 in the first region ROI_1.

[0092] As shown in Figures 13 to 15, in the second region ROI_2, events are detected for each pixel 10, and in the first region ROI_1, events are detected for each set of pixels 10, for example, by performing a binning process, thereby enabling faster event detection in the region of interest and preventing missed event detection outside the region of interest.

[0093] In the above explanation, an example was described in which the first region ROI_1, which is the region of interest, is located inside the second region ROI_2, but the pixel positions and sizes of the first region ROI_1 and the second region ROI_2 are arbitrary, and the first region ROI_1 and the second region ROI_2 may be arranged so as not to overlap, or the second region ROI_2 may be approximately the same size as the first region ROI_1 or smaller than the first region ROI_1.

[0094] 16 is a diagram showing an example in which a first region ROI_1 and a second region ROI_2 are arranged so as not to overlap each other, and a third region ROI_3 is arranged so as to include the first region ROI_1 and the second region ROI_2. In the example of FIG. 16, the first region ROI_1 and the second region ROI_2 are approximately the same size. Note that the sizes of the first region ROI_1 and the second region ROI_2 are arbitrary, and the first region ROI_1 and the second region ROI_2 may partially overlap each other.

[0095] FIG. 17 is a timing diagram corresponding to FIG. 16 . In FIG. 17 , the size of the second region ROI_2 is changed within one frame period, and event detection processing is performed. From time t1 to t4, the second region ROI_2 is set to a size similar to that of the first region ROI_1, and event detection processing is performed for each pixel 10, for each group of pixels 10, or by binning processing. Thereafter, from time t5 to t8, t9 to t12, t13 to t16, and t17 to t20, event detection is performed for each pixel 10, for each group of pixels 10, or by binning processing for the first region ROI_1. From time t21 to t24, event detection is performed for each pixel 10, for each group of pixels 10, or by binning processing for the third region ROI_3.

[0096] In the example of Figure 16, two regions of interest (a first region ROI_1 and a second region ROI_2) are set at different locations within the pixel array unit 3, but three or more regions of interest may also be set within the pixel array unit 3.

[0097] 18 is a diagram showing an example in which four regions of interest (first region ROI_1, second region ROI_2, third region ROI_3, and fourth region ROI_4) are set within the pixel array unit 3. In the example of FIG. 18, the first to third regions ROI_1 to ROI_3 are each approximately the same size, and the first to third regions ROI_1 to ROI_3 are provided within the fourth region ROI_4. Note that the pixel positions and sizes of the first to fourth regions ROI_1 to ROI_4 are arbitrary.

[0098] FIG. 19 is a timing diagram corresponding to FIG. 18 . From time t1 to t4, event detection processing, reset processing, and event signal read processing are performed for the fourth region ROI_4. From time t5 to t6, event detection processing is performed in parallel for the first to third regions ROI_1 to ROI_3. From time t6 to t7, reset processing is performed in parallel for the first to third regions ROI_1 to ROI_3. From time t7 to t8, event signal read processing is performed sequentially for the first to third regions ROI_1 to ROI_3. The event detection processing for the first to third regions ROI_1 to ROI_3 performed between times t5 and t8 is also performed between times t9 to t12, t13 to t16, and t17 to t20. From time t21 to t24, event detection processing for the fourth region ROI_4 is performed, similar to times t1 to t4. The time from t1 to t20 is one frame period.

[0099] As described above, the number of regions of interest set in the pixel array unit 3, pixel positions, and sizes are arbitrary, and various modifications are possible.

[0100] FIG. 20 is a diagram showing an example in which three regions of interest (first to third regions ROI_1 to ROI_3) are in an inclusive relationship. In the example of FIG. 20, the first region ROI_1 is provided within the second region ROI_2, which is provided within the third region ROI_3. The first to third regions ROI_1 to ROI_3 are provided within the fourth region ROI_4. The fourth region ROI_4 may be the entire pixel region of the pixel array unit 3, or may be a partial region. For each of the first region ROI_1 to third region ROI_3, event detection is performed for each pixel 10, for each set of multiple pixels 10, or by binning processing. In a typical example, in the first region ROI_1, event detection is performed for each pixel 10, in the second region ROI_2, event detection is performed for every two or more pixels 10, and in the third region ROI_3, event detection is performed for every pixel 10 that is thinner than that in the second region ROI_2.

[0101] The process of thinning out the pixels 10 (hereinafter referred to as thinning process) may be performed by the pixel array unit 3, the EVS readout unit 4, or the signal processing unit 7.

[0102] 21 is a block diagram of the periphery of the pixel array unit 3 when thinning processing is performed by the EVS readout unit 4. In the example of FIG. 21 , all pixels 10 in the pixel array unit 3 output event signals, and the output event signals are input to the EVS readout unit 4. The EVS readout unit 4 thins out some of the multiple event signals output from the pixel array unit 3 and outputs them. The event signals output from the EVS readout unit 4 are input to the signal processing unit 7.

[0103] As shown in FIG. 21 , if the EVS readout unit 4 performs thinning processing of the event signal, each pixel 10 in the pixel array unit 3 simply needs to output an event signal, thereby simplifying the circuit configuration of the pixel array unit 3.

[0104] 22 is a block diagram of the periphery of the pixel array unit 3 when thinning processing is performed by the signal processing unit 7. All pixels 10 in the pixel array unit 3 output event signals. The event signals output from the pixel array unit 3 are input to the signal processing unit 7 via the EVS readout unit 4. The signal processing unit 7 performs thinning processing on the event signals from the EVS readout unit 4 and then performs signal processing on the event signals.

[0105] In the example of FIG. 22, the EVS readout unit 4 only relays the event signal from the pixel array unit 3, and the internal configuration of the EVS readout unit 4 can be simplified.

[0106] In each of the above examples, when event information is read out for each pixel row, pixel rows in which no events occurred may be skipped and only pixel rows in which events occurred may be read out sequentially and continuously. This allows the event information in the pixel array unit 3 to be read out in a short time, reducing the amount of event data and increasing the frame rate.

[0107] 23 is a timing diagram for skipping pixel rows in which no events have occurred. As shown in Fig. 23, at the timing when a pixel row in which no events have occurred is read, event information for a pixel row in which an event has occurred, which is located after that pixel row, is read. This makes it possible to continuously read only pixel rows in which events have occurred at high speed, and to read all event information in the pixel array unit 3 in a short time.

[0108] Various methods can be considered for the above-described binning process. FIG. 24 is a diagram illustrating a first example of the binning process. In the first example of the binning process, the currents flowing through two or more adjacent pixels 10 in a row that are the target of the binning process are added together, and event detection is performed using the added current. The example in FIG. 24 illustrates an example in which binning is performed on multiple adjacent pixels 10 in a row. A switch 21 is provided to switch whether or not to short-circuit the cathodes of two photoelectric conversion elements 11 in two adjacent pixels 10. The switch 21 is formed, for example, by an NMOS transistor and functions as a current summing circuit. A binning instruction signal is input to the gate of the NMOS transistor. When the binning instruction signal is, for example, high level, the switch 21 is turned on, and the cathodes of the photoelectric conversion elements 11 of two adjacent pixels 10 in the row are short-circuited. For example, when binning is performed on four pixels 10, three of the pixels 10 are reset by the AZ signal, and the remaining pixel 10 performs event detection based on the sum of the currents flowing through the four pixels 10.

[0109] FIG. 25 is a diagram illustrating a second example of binning processing. In the second example of binning processing, it is possible to individually select whether to perform binning processing on two or more pixels 10 adjacent in the row direction and whether to perform binning processing on two or more pixels 10 adjacent in the column direction. A first switch 21a, for example, made of an NMOS transistor, is provided between two adjacent pixels 10 in the row direction. Similarly, a second switch 21b, for example, made of an NMOS transistor, is provided between two adjacent pixels 10 in the column direction. The second switch 21b functions as a voltage averaging circuit. A first control signal controlling the first switch 21a goes high when performing binning processing on two or more pixels 10 adjacent in the row direction, and a second control signal controlling the second switch 21b goes high when performing binning processing on two or more pixels 10 adjacent in the column direction. When the first control signal is low, row binning is not performed, and an event signal is detected for each adjacent pixel 10 in the row direction. When the second control signal is at a low level, the binning process in the column direction is not performed, and an event signal is detected for each pixel 10 adjacent in the column direction.

[0110] 26 is a diagram illustrating a third example of binning processing. In the third example of binning processing, output nodes of source follower circuits 22 provided at output portions of buffers 14 in two or more pixels 10 adjacent to each other in the row direction that are the subject of binning processing are connected to each other to generate an arithmetic average value of voltages, and event detection is performed using the generated arithmetic average value of voltages.

[0111] 26 , a switch 23 is provided which is connected between one ends of capacitors C1 in differentiating circuits 15 which are connected downstream of buffers 14 in two pixels 10 to be binned. This switch 23 is formed, for example, by an NMOS transistor 23, and a binning instruction signal is input to the gate of the NMOS transistor 23. When the binning instruction signal is at a high level, for example, the switch 23 is turned on, and the nodes at one ends of the two capacitors C1 in the two differentiating circuits 15 are shorted together, and this node becomes the average value of the voltages before the shorting.

[0112] FIG. 27 is a diagram illustrating a fourth example of binning processing. In the fourth example of binning processing, similar to FIG. 25 , it is possible to individually select whether to perform binning processing on two or more pixels 10 adjacent in the row direction and whether to perform binning processing on two or more pixels 10 adjacent in the column direction. A first switch 23 a, e.g., an NMOS transistor, is provided between two adjacent pixels 10 in the row direction. Similarly, a second switch 23 b, e.g., an NMOS transistor, is provided between two adjacent pixels 10 in the column direction. As in FIG. 26 , the first switch 23 a switches whether to short-circuit the outputs of the source follower circuits 22 in two adjacent pixels 10 in the row direction. Similarly, the second switch 23 b switches whether to short-circuit the outputs of the source follower circuits 22 in two adjacent pixels 10 in the column direction. The first switch 23 a is switched by a first control signal, and the second switch 23 b is switched by a second control signal. Since the first switch 23a and the second switch 23b are individually controlled to be switched, it is possible to select whether or not to perform event detection by binning processing or for each pixel 10 in the row direction or column direction separately.

[0113] The photodetector 1 according to this embodiment can transmit event data including an event signal to the AP 2. FIG. 28 illustrates a connection between the photodetector 1 and the AP 2. A virtual channel 24 can be established between the photodetector 1 and the AP 2 as needed. The virtual channel switches the channel for transmitting event data based on information contained in the event data from the photodetector 1. For example, the AP 2 receives event data detected in a first region ROI_1 via a first virtual channel 24a and event data detected in a second region ROI_2 via a second virtual channel 24b. The AP 2 can perform different signal processing when receiving event data for the first region ROI_1 and when receiving event data for the second region ROI_2. In the example illustrated in FIG. 28, the AP 2 includes a data receiver 2a, a first processing unit 2b, and a second processing unit 2c. The data receiving unit 2a receives event data for a first region ROI_1 via a first virtual channel 24a and event data for a second region ROI_2 via a second virtual channel 24b from the photodetector 1. The first processing unit 2b performs signal processing on the received event data for the first region ROI_1. The second processing unit 2c performs signal processing on the received event data for the second region ROI_2.

[0114] The photodetector 1 may transmit the event data to the AP 2 without necessarily using a virtual channel. In this case, the AP 2 can check the EBD included in the event data, determine which region the event data belongs to, and perform signal processing according to the region.

[0115] 29 is a block diagram showing a first example of the internal configuration of the signal processing unit 7. The first example of the signal processing unit 7 has an event filter unit 25. The event filter unit 25 performs event drop processing to filter event signals from the pixel array unit 3, thereby reducing the number of event signals.

[0116] 30 is a flowchart showing the processing operation of the signal processing unit 7 of FIG. 29. When the signal processing unit 7 receives an event signal from the pixel array unit 3 (step S1), it determines whether the event signal is an event signal for the first region ROI_1 (step S2). Since the first region ROI_1 is a region of interest, if step S1 is YES, the event drop setting is set to low frequency (step S3). If step S1 is NO, the event drop setting is set to high frequency (step S4). After the processing of step S2 or S3 is completed, the event signal is reduced (dropped) based on the event drop setting set in step S2 or S3 (step S5), and event data including the reduced event signal is output (step S6).

[0117] 31 is a block diagram showing a second example of the internal configuration of the signal processing unit 7. The second example of the signal processing unit 7 has an event filter unit 25 and a data compression unit 26. The data compression unit 26 compresses the event signal. Note that the signal processing unit 7 may have only one of the event filter unit 25 and the data compression unit 26.

[0118] Fig. 32 is a flowchart showing the processing operation of the signal processing unit 7 in Fig. 31. Steps S11 to S15 are the same as steps S1 to S5 in Fig. 30. After the event signal is reduced (dropped) in step S5, the event signal is compressed using the data compression unit 26 (step S16), and event data including the compressed event signal is output (step S17).

[0119] As shown in FIGS. 31 and 32, by compressing the event signal, the size of the event data output from the signal processing unit 7 can be reduced, and the amount of data transmitted to the AP 2 can be reduced.

[0120] The threshold value, which is the criterion for detecting an event, may be different between the first region ROI_1 and the other regions (e.g., the second region ROI_2). Here, the threshold value is the threshold value Vth input to the quantizer 16 in Figures 4A and 4B.

[0121] 33 is a diagram illustrating the thresholds of a first region ROI_1 and a second region ROI_2 that includes the first region ROI_1. Fig. 33 shows an example in which an event signal is output in the first region ROI_1 when a change in the illuminance (brightness) of incident light exceeds a first threshold, and an event signal is output in the second region ROI_2 when a change in the illuminance of incident light exceeds a second threshold. The first threshold is set to a value smaller than the second threshold.

[0122] 33, each pixel 10 in the first region ROI_1 outputs an ON event signal or an OFF event signal each time a change in illuminance (luminance) exceeds a first threshold. An ON event signal is output when the rate at which illuminance (luminance) increases exceeds the first threshold, and an OFF event signal is output when the rate at which illuminance (luminance) decreases exceeds the first threshold.

[0123] As shown on the right side of Fig. 33 , each pixel 10 in the second region ROI_2 outputs an ON event signal or an OFF event signal each time a change in illuminance (luminance) exceeds a second threshold. An ON event signal is output when the rate at which illuminance (luminance) increases exceeds the second threshold, and an OFF event signal is output when the rate at which illuminance (luminance) decreases exceeds the second threshold. In the example of Fig. 33 , because the second threshold is greater than the first threshold, the number of ON event signals and OFF event signals output is greater in the first region ROI_1 than in the second region ROI_2.

[0124] In this way, by adjusting the threshold value for event detection for each region set in the pixel array unit 3, the number of events detected can be optimized for each region.

[0125] The threshold for event detection can be switched on a row-by-row or column-by-column basis within the pixel array unit 3. FIG. 34 is a block diagram of the event detection circuit 12 included in each pixel 10 within the pixel array unit 3. The threshold to be input to the quantizer 16 is generated by a logic circuit 27. A row designation signal and a column designation signal are input to the logic circuit 27. The logic circuit 27 switches the threshold on a row-by-column or column-by-column basis based on the row designation signal and the column designation signal. The quantizer 16 outputs an event signal (e.g., an on-event signal and an off-event signal) based on the threshold output from the logic circuit 27.

[0126] The photodetector 1 according to this embodiment can switch the frequency of outputting an event signal for each of a plurality of regions (e.g., a first region ROI_1 and a second region ROI_2) set in the pixel array unit 3. Switching the frequency of outputting an event signal requires control to switch whether or not to output an event signal. There are several possible methods for switching whether or not to output an event signal.

[0127] 35 is a block diagram of the event detection circuit 12 that controls the output of an event signal. The buffer 14 in the event detection circuit 12 in FIG. 35 has an enable terminal that switches whether the buffer 14 is operated, and a logic circuit 27 is connected to the enable terminal. A row designation signal and a column designation signal are input to the logic circuit 27. Based on the row designation signal and the column designation signal, the logic circuit 27 outputs an enable signal that switches whether the buffer 14 is operated on a row or column basis. The buffer 14 of each pixel 10 switches whether to output a voltage signal corresponding to the amount of incident light, based on the enable signal output from the corresponding logic circuit 27.

[0128] 35 , it is possible to control the timing of outputting an event signal in units of any row or column within the pixel array unit 3. Therefore, by using the event detection circuit 12 of Fig. 35 , it is possible to arbitrarily set the frequency of outputting an event signal for each region set in units of rows or columns within the pixel array unit 3.

[0129] FIG. 36 is a block diagram of the event detection circuit 12 according to the first modification of FIG. 35 . As shown in FIG. 36 , the quantizer 16 in the event detection circuit 12 according to the first modification has an enable terminal that switches whether or not the quantizer 16 is operated, and a logic circuit 27 is connected to the enable terminal. A row designation signal and a column designation signal are input to the logic circuit 27. Based on the row designation signal and the column designation signal, the logic circuit 27 outputs an enable signal that switches whether or not the quantizer 16 is operated, with at least one of a row or a column as a unit. The quantizer 16 of each pixel 10 switches whether or not to output an enable signal based on the enable signal output from the corresponding logic circuit 27.

[0130] 37 is a block diagram of the event detection circuit 12 according to the first modification of FIG. 35 . As shown in FIG. 37 , the output circuit 17 in the event detection circuit 12 according to the first modification has a switch 17b that switches whether or not to supply a power supply voltage to the output circuit 17, and the switch 17b is switch-controlled by a logic circuit 27. A row designation signal and a column designation signal are input to the logic circuit 27. The logic circuit 27 controls the switch 17b on a row-by-row or column-by-column basis based on the row designation signal and the column designation signal. As a result, the output circuit 17 of each pixel 10 switches whether or not to output an enable signal on a row-by-row or column-by-column basis based on the output signal of the corresponding logic circuit 27.

[0131] In the photodetector 1 of Figures 35 to 37, the frequency at which at least one of the buffer, differential circuit, comparator, or output circuit in the event detection circuit 12 operates differs for each area set in the pixel array section 3, and therefore, by controlling the operation of the event detection circuit 12 in accordance with the frequency, unnecessary power consumption in the event detection circuit 12 can be reduced.

[0132] In the above description, it is assumed that all the pixels 10 provided in the pixel array unit 3 are EVS pixels 10. However, the pixel array unit 3 may include a mixture of EVS pixels 10 and gradation pixels 10Z. The gradation pixels 10Z are pixels 10 that output pixel signals containing luminance information corresponding to the amount of incident light, and by combining them with color filters, gradation information can be obtained for each pixel 10. In this specification, a configuration in which EVS pixels 10 and gradation pixels 10Z are mixed is referred to as a hybrid configuration. The hybrid configuration includes a pixel configuration in which pixel blocks each including a plurality of gradation pixels 10Z and one or more EVS pixels 10 are arranged in a first direction X (horizontal direction) and a second direction Y (vertical direction).

[0133] 38A is a plan view showing a pixel block according to a first example that can be applied to the pixel array unit 3 according to this embodiment. As shown in FIG. 38A , the pixel block according to the first example is a Bayer array consisting of 2×2=4 pixels, of which three pixels 10 are RGB gradation pixels 10Z and the remaining pixel 10 is an EVS pixel 10. The EVS pixel 10 has, for example, the event detection circuit 12 shown in FIG. 4A , FIG. 4B , FIG. 5A , or FIG. 5B .

[0134] 38B is a plan view showing a pixel block according to a second example that can be applied to the pixel array unit 3 according to this embodiment. As shown in FIG. 38B , the pixel block according to the second example is a quad array consisting of 4×4=16 pixels 10. In the quad array, three 2×2=4 gradation pixels 10Z of the same color and two 2×2=4 EVS pixels 10 are arranged in the first direction X and two in the second direction Y.

[0135] Figure 38C is a plan view showing a pixel block according to a third example that can be applied to the pixel array unit 3 according to this embodiment. The pixel block according to the third example is a partial modification of the quad arrangement of Figure 38B, and has one set of four red gradation pixels 10Z, one set of three blue gradation pixels 10Z, four green gradation pixels 10Z, three green gradation pixels 10Z, and two EVS pixels 10. The two EVS pixels 10 are arranged along one side of the pixel block.

[0136] Figure 38D is a plan view showing a pixel block according to a fourth example that can be applied to the pixel array unit 3 according to this embodiment. The pixel block according to the fourth example is a partial modification of the quad arrangement of Figure 38B, and has one set of three red gradation pixels 10Z, one set of three blue gradation pixels 10Z, two sets of three green gradation pixels 10Z, and four EVS pixels 10. The four EVS pixels 10 are arranged in the center of the pixel block.

[0137] Fig. 39 is a circuit diagram of the gradation pixel 10Z. As shown in Fig. 39, the gradation pixel 10Z has a photoelectric conversion element 11, a transfer transistor Q21, a floating diffusion FD, an amplification transistor Q22, a selection transistor Q23, and a reset transistor Q24.

[0138] The photoelectric conversion element 11 is, for example, a photodiode, and its anode is grounded. The transfer transistor Q21 is connected between the floating diffusion FD and the cathode of the photoelectric conversion element 11. The amplifier transistor Q22 and the selection transistor Q23 form a source follower circuit 22. The gate of the amplifier transistor Q22 is connected to the floating diffusion FD. The drain of the amplifier transistor Q22 is connected to a power supply voltage node, the source of the amplifier transistor Q22 is connected to the drain of the selection transistor Q23, and the source of the selection transistor Q23 is connected to the vertical signal line VSL. The drain of the reset transistor Q24 is connected to the power supply voltage node, and the source of the reset transistor Q24 is connected to the floating diffusion FD.

[0139] Although omitted in FIG. 39, the gradation pixel 10Z may include a conversion efficiency switching transistor or a discharge transistor.

[0140] The signal processing unit 7 in the photodetector 1 according to this embodiment generates event data by adding additional information to the event signal output from the pixel array unit 3, and sends the event data to the AP 2. The signal processing unit 7 has an additional information generation unit that generates the additional information.

[0141] Fig. 40 is a block diagram showing the internal configuration of the signal processing unit 7, output I / F 9, and AP 2 in the photodetector 1 according to this embodiment. Fig. 40 shows a block configuration related to the generation of additional information among the processes performed by the signal processing unit 7.

[0142] As shown in FIG. 40 , the signal processing unit 7 includes an additional information generation unit 41. The additional information generation unit 41 includes an event access unit 42, an event count unit 43, an event number analysis unit 44, an event frequency analysis unit 45, an attention calculation unit 46, an optical flow analysis unit 47, an exposure row detection unit 48, and a data processing unit 49. The event access unit 42 sends a timestamp, a frame number, and region information to a data transmission unit 50 of the output I / F 9. The event number analysis unit 44 sends an event detection threshold and the number of events to the data transmission unit 50. The event frequency analysis unit 45 sends flicker presence / absence information, a flicker occurrence position, a flicker intensity, and a flicker frequency to the data transmission unit 50. The attention calculation unit 46 sends the attention level of each pixel 10 to the data transmission unit 50. The optical flow analysis unit 47 sends the optical flow value of each pixel 10 to the data transmission unit 50. The data processing unit 49 transmits the classification value and the amount of change in luminance of each pixel 10 to the data transmission unit 50. The event detection circuit 12 transmits the event raw data, information on the presence or absence of an event, the polarity of the event, and the pixel position (coordinates) of the event to the data transmission unit 50.

[0143] 40, the AP 2 has a data receiving unit 2a and an event-related data processing unit 2d. The data receiving unit 2a receives region information, a timestamp, the number of frames, an event detection threshold, ROI information, the presence or absence of flicker, the position where flicker occurs, flicker intensity, flicker frequency, the presence or absence of EVS movement, the EVS movement direction, the frame data amount, event raw data, and exposure frame information.

[0144] The event-related data processing unit 2d has an ROI calculation unit 53, a recognition unit 54, an AE / AF unit 55, a VLC unit 56, a SLAM unit 57, an OIS / EIS unit 58, a motion detection unit 59, a Gesture unit 60, a 3DNR unit 61, a Deblur unit 62, and a VFI (Video Frame Interpolation) unit 63. The ROI calculation unit 53 outputs coordinate information of the area to be acquired. The recognition unit 54 outputs the recognition result of the target area. The AE / AF unit 55 outputs distance information to the object. The VLC unit 56 outputs distance information to the object. The SLAM unit 57 outputs information on the amount of movement per unit time. The OIS / EIS unit 58 outputs information on the amount of movement per unit time. The motion detection unit 59 outputs the presence or absence of a subject within the screen. The gesture unit 60 outputs the detection result of a specific action (waving, raising a hand, etc.). The 3DNR unit 61 outputs coordinate information of a moving subject. The deblur unit 62 outputs information on the amount of movement of the subject per unit time based on an event signal included in the event data transmitted from the light detection device 1. The VFI unit 63 outputs information on the amount of movement of the subject per unit time.

[0145] As described above, the photodetection device 1 according to this embodiment switches the frequency of event detection between a region of interest (e.g., the first region ROI_1) and other regions (e.g., the second region ROI_2). The region of interest can be determined based on an image captured by the photodetection device 1. In this case, a region of interest determination unit needs to be provided in the photodetection device 1 or AP2.

[0146] FIG. 41 is a block diagram showing a schematic configuration of a photodetection system 30 including a photodetector 1 having a region-of-interest determiner 64. In FIG. 41 , the region-of-interest determiner 64 is provided in the signal processing unit 7. The region-of-interest determiner 64 determines a region of interest based on an event signal output from the EVS readout unit 4. For example, the region-of-interest determiner 64 determines a region where a greater number of event signals are generated as the region of interest. The region of interest can increase the event detection frequency and the event detection resolution compared to other regions. In other words, the region of interest can increase the spatial resolution and temporal resolution compared to other regions. Information such as the pixel positions of the region of interest determined by the region-of-interest determiner 64 is sent to the timing control unit 5.

[0147] Fig. 42 is a block diagram showing a schematic configuration of a photodetection system 30 according to a first modified example of Fig. 41. The photodetection device 1 included in the photodetection system 30 of Fig. 42 has a hybrid-configuration pixel array unit 3. The pixel array unit 3 is provided with a plurality of gradation pixels 10Z and a plurality of EVS pixels 10.

[0148] 42 includes an AD (Analog-Digital) converter 65, a first signal processing unit 7a, a second signal processing unit 7b, a first output I / F 9a, and a second output I / F 9b in addition to the components of the photodetector 1 in FIG. 41. The first signal processing unit 7a processes event signals, while the second signal processing unit 7b processes pixel signals output from the gradation pixels 10Z. The first output I / F 9a outputs event data generated by the first signal processing unit 7a.

[0149] The AD conversion unit 65 AD converts pixel signals output from each gradation pixel 10Z of the pixel array unit 3. The second signal processing unit 7b has a region of interest determination unit 64. The region of interest determination unit 64 determines a region of interest based on digital pixel signals obtained by AD converting the pixel signals output from the gradation pixels 10Z. The region of interest determination unit 64 determines a region of interest taking into consideration not only changes in luminance but also changes in gradation, including color.

[0150] Fig. 43 is a block diagram showing a schematic configuration of the light detection system 30 according to the second modified example of Fig. 41. The light detection system 30 according to the second modified example is characterized in that an attention area determination unit 64 is provided inside the AP 2. The internal configuration of the light detection device 1 has a configuration in which the attention area determination unit 64 is omitted from Fig. 41 or 42, for example.

[0151] The attention area determination unit 64 according to the second modification determines an attention area based on the result of image recognition performed by the AP 2, and also determines the spatial resolution and the temporal resolution of the attention area.

[0152] Fig. 44 is a block diagram showing a schematic configuration of a light detection system 30 according to the third modified example of Fig. 41. In the light detection system 30 according to the third modified example, a user 66 determines a region of interest and provides information specifying the region of interest to the light detection device 1. The light detection device 1 may display, on a display device (not shown), text or an image prompting the user 66 to determine the region of interest, to attract the user's attention.

[0153] In this way, in the photodetector 1 according to this embodiment, the frequency of event detection is switched between the first region ROI_1 and the second region ROI_2 within the pixel array unit 3, so the temporal resolution can be made different between the first region ROI_1 and the second region ROI_2. Furthermore, the spatial resolution can also be made different between the first region ROI_1 and the second region ROI_2. This improves the quality of the event image in the first region ROI_1, which is the region of interest, while intentionally lowering the quality of the event image in the second region ROI_2, which is less important. This reduces the amount of event data sent from the photodetector 1 to the AP 2 and also reduces the processing load on the signal processing unit 7 and the AP 2, thereby reducing power consumption.

[0154] <Application to a Mobile Body> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0155] FIG. 45 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0156] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 45, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0157] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0158] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0159] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0160] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0161] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0162] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0163] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0164] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0165] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 45, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0166] FIG. 46 is a diagram showing an example of the installation position of the imaging unit 12031.

[0167] In FIG. 46, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0168] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0169] 46 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0170] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0171] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0172] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0173] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0174] The above describes an example of a vehicle control system to which the technology according to the present disclosure can be applied. The technology according to the present disclosure can be applied to, for example, the image capturing unit 12031 among the components described above. Specifically, by applying the technology according to the present disclosure to the image capturing unit 12031, it is possible to obtain a captured image that is easier to see, thereby reducing driver fatigue.

[0175] The present technology may be configured as follows: (1) A photodetector including: a pixel array unit having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control unit that switches a frequency of detecting the event between a first region that is a part of the pixel region including the plurality of pixels and a second region that is at least partially different from the first region; and a second control unit that switches access control between the first region and the second region. (2) The photodetector according to (1), wherein the first region is a region of interest that is smaller in size than the second region, and the first control unit detects the event in the first region more frequently than in the second region. (3) The photodetector according to (1) or (2), wherein the first control unit switches the resolution at which the event is detected between the first region and the second region. (4) The photodetector according to (3), wherein the first control unit detects the event at each of all the pixels in the first region, and detects the event for two or more of the pixels in the second region. (5) The photodetector according to (3), wherein the first control unit detects the event at each of all the pixels in the second region and detects the event for each of two or more of the pixels in the first region. (6) The photodetector according to any one of (1) to (5), wherein the first region or the second region has a current adder circuit that adds currents generated by photoelectric conversion at each of two or more of the pixels. (7) The photodetector according to (6), further comprising a switch that switches whether or not currents generated by photoelectric conversion at each of two or more of the pixels adjacent to each other in a first direction or a second direction intersecting the first direction are added in the current adder circuit. (8) The photodetector device according to any one of (1) to (5), wherein each of the plurality of pixels has a photoelectric conversion element that generates an electric charge according to the amount of incident light, and a source follower circuit that outputs a voltage according to the electric charge, and the first region or the second region has a voltage averaging circuit that averages the voltages output from the source follower circuits of two or more of the pixels.(9) The photodetector according to (8), further comprising a switch that switches whether to average voltages output from the source follower circuits of two or more of the pixels adjacent to each other in a first direction or a second direction intersecting the first direction. (10) The photodetector according to any one of (1) to (9), wherein the plurality of pixels in the pixel array unit detect the event a plurality of times in the first region and detect the event once in the second region during one frame period. (11) The photodetector according to any one of (1) to (10), further comprising a signal processing circuit that generates event data including detection signals of the event output from the plurality of pixels and additional information related to the detection signals, wherein the additional information includes at least one of a pixel position of the event, information specifying the first region or the second region in which the event was detected, and a detection time of the event. (12) The photodetector according to any one of (1) to (10), further comprising a signal processing circuit that performs at least one of a filter process that thins out the event detection signals output from the plurality of pixels and a compression process that compresses the event detection signals output from the plurality of pixels. (13) The photodetector according to any one of (1) to (12), wherein each of the plurality of pixels has an event detection circuit that detects the event, and a threshold value serving as a reference for detecting the event in the event detection circuit is different between the first region and the second region. (14) The photodetector according to (13), wherein the event detection circuit has a current-voltage conversion circuit, a buffer, a differentiation circuit, a comparator, and an output circuit, and a frequency at which at least one of the buffer, the differentiation circuit, the comparator, or the output circuit is operated is different between the first region and the second region. (15) The photodetection device according to any one of (1) to (14), wherein the plurality of pixels include a first pixel that outputs a pixel signal including gradation information according to the amount of incident light, and a second pixel that detects the event.(16) The photodetector according to any one of (1) to (15), wherein the plurality of pixels each include two or more pixels arranged along a first direction and have a plurality of pixel groups arranged in a second direction intersecting the first direction, and the pixel array unit sequentially outputs the event detection signal for each pixel group in which the event has occurred, excluding a pixel group in which the event has not occurred among the plurality of pixel groups. (17) A photodetector system comprising: a photodetector that outputs an event signal; and a signal processing device having a signal processing circuit that performs signal processing based on the event signal, wherein the photodetector comprises: a pixel array unit having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control unit that switches the frequency of event detection between a first region that is a part of a pixel region of the pixel array unit and a second region other than the first region; and a second control unit that switches access control between the first region and the second region. (18) The optical detection system according to (17), wherein the signal processing device receives a detection signal of the event detected in the first region and first additional information added to the detection signal from the optical detection device via a first virtual channel, and receives a detection signal of the event detected in the second region and second additional information added to the detection signal from the optical detection device via a second virtual channel. (19) The optical detection system according to (18), wherein the signal processing device identifies a frame in which the event occurred based on the first additional information and the second additional information. (20) The optical detection system according to any one of (17) to (19), wherein the signal processing device performs Debra processing based on the event detection signal transmitted from the optical detection device.

[0176] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0177] 1 Photodetector, 2 AP, 2a Data receiver, 2b First processing unit, 2c Second processing unit, 2d Event-related data processing unit, 3 Pixel array unit, 4 Readout unit, 5 Timing control unit, 5a ROI_1 setting unit, 5b ROI_2 setting unit, 5c Multiplexer, 5d Area control unit, 6 Access control unit, 6a, 6b, 6c, 6d Logic gate, 6e ROI discrimination unit, 6f Decoder, 7 Signal processing unit, 7a First signal processing unit, 7b Second signal processing unit, 8 Time stamp generation unit, 9 Output interface unit (output I / F), 10 EVS pixel, 10Z Gradation pixel, 11 Photoelectric conversion element, 12 Event detection circuit, 13 Current-voltage conversion circuit, 14 Buffer, 15 Differentiation circuit, 16 Quantizer, 17 Output circuit, 17a Logic circuit, 17b Switch, 18 Switch, 19 Inverter, 21 Switch, 21a First switch, 21b Second switch, 22 Source follower circuit, 23 NMOS transistor, 23 Switch, 23a First switch, 23b Second switch, 24 Virtual channel, 24a First virtual channel, 24b Second virtual channel, 25 Event filter unit, 26 Data compression unit, 27 Logic circuit, 30 Light detection system, 31 Imaging lens, 32 Image processing unit, 33 Recording unit, 34 Control unit, 41 Additional information generation unit, 42 Event access unit, 43 Event count unit, 44 Event number analysis unit, 45 Event frequency analysis unit, 46 Attention calculation unit, 47 Optical flow analysis unit, 48 Exposure row detection unit, 49 Data processing unit, 50 Data transmission unit, 53 ROI calculation unit, 54 Recognition unit, 55 AE / AF unit, 56 VLC unit, 57 SLAM unit, 58 OIS / EIS unit, 59 detection unit, 60 Gesture unit, 61 DNR unit, 62 Deblur unit, 63 VFI unit, 64 attention area determination unit, 65 AD conversion unit, 65 conversion unit, 66 user

Claims

1. A photodetection device comprising: a pixel array section having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control section that switches the frequency of detecting the event between a first region that is a part of the pixel region including the plurality of pixels and a second region that is at least partially different from the first region; and a second control section that switches between access control for the first region and access control for the second region.

2. The optical detection device of claim 1, wherein the first region is a region of interest smaller in size than the second region, and the first control unit detects the events in the first region more frequently than in the second region.

3. The optical detection device according to claim 1, wherein the first control unit switches the resolution at which the event is detected between the first area and the second area.

4. The optical detection device according to claim 3, wherein the first control unit detects the event at each of all of the pixels in the first region, and detects the event for each of two or more of the pixels in the second region.

5. The optical detection device according to claim 3, wherein the first control unit detects the event at each of all of the pixels in the second region, and detects the event for each of two or more of the pixels in the first region.

6. The photodetection device according to claim 1, wherein the first region or the second region has a current summing circuit that sums currents generated by photoelectric conversion in two or more of the pixels.

7. The photodetection device according to claim 6, further comprising a switch that switches whether or not currents generated by photoelectric conversion in two or more of the pixels adjacent to each other in a first direction or a second direction intersecting the first direction are to be added in the current adding circuit.

8. The photodetection device according to claim 1, wherein each of the plurality of pixels has a photoelectric conversion element that generates an electric charge according to the amount of incident light, and a source follower circuit that outputs a voltage according to the electric charge, and the first region or the second region has a voltage averaging circuit that averages the voltages output from the source follower circuits possessed by each of two or more of the pixels.

9. The photodetection device according to claim 8, further comprising a switch that switches whether or not to average the voltages output from the source follower circuits of two or more adjacent pixels in a first direction or a second direction intersecting the first direction.

10. The photodetection device according to claim 1, wherein the plurality of pixels in the pixel array detect the event a plurality of times in the first region and detect the event once in the second region during one frame period.

11. The optical detection device according to claim 1, further comprising a signal processing circuit that generates event data including detection signals of the event output from the plurality of pixels and additional information related to the detection signals, the additional information including at least one of the pixel position of the event, information identifying the first region or the second region in which the event was detected, and a detection time of the event.

12. The photodetection device according to claim 1, further comprising a signal processing circuit that performs at least one of a filtering process for thinning out the event detection signals output from the plurality of pixels, and a compression process for compressing the event detection signals output from the plurality of pixels.

13. The photodetection device according to claim 1, wherein each of the plurality of pixels has an event detection circuit that detects the event, and a threshold value serving as a reference for detecting the event in the event detection circuit is different between the first region and the second region.

14. The photodetection device according to claim 13, wherein the event detection circuit has a current-voltage conversion circuit, a buffer, a differentiation circuit, a comparator, and an output circuit, and the frequency of operation of at least one of the buffer, the differentiation circuit, the comparator, and the output circuit differs between the first region and the second region.

15. The light detection device according to claim 1, wherein the plurality of pixels include a first pixel that outputs a pixel signal including gradation information according to the amount of incident light, and a second pixel that detects the event.

16. The photodetection device of claim 1, wherein the plurality of pixels each include two or more pixels arranged along a first direction, and the photodetection device has a plurality of pixel groups arranged in a second direction intersecting the first direction, and the pixel array section sequentially outputs the event detection signal for each pixel group in which the event has occurred, excluding a pixel group in which the event has not occurred, among the plurality of pixel groups.

17. An optical detection system comprising: an optical detection device that outputs an event signal; and a signal processing device having a signal processing circuit that performs signal processing based on the event signal, wherein the optical detection device has: a pixel array unit having a plurality of pixels that detects an event when a change in luminance of incident light exceeds a predetermined threshold; a first control unit that switches the frequency of detecting the event between a first region that is a part of the pixel region of the pixel array unit and a second region other than the first region; and a second control unit that switches between access control for the first region and access control for the second region.

18. The optical detection system of claim 17, wherein the signal processing device receives a detection signal of the event detected in the first region and first additional information added to the detection signal from the optical detection device via a first virtual channel, and receives a detection signal of the event detected in the second region and second additional information added to the detection signal from the optical detection device via a second virtual channel.

19. The optical detection system according to claim 18, wherein the signal processing device identifies a frame in which the event occurred based on the first additional information and the second additional information.

20. The optical detection system according to claim 17, wherein the signal processing device performs Debra processing based on the event detection signal transmitted from the optical detection device.

Images