Photon counting photodetector with motion detection

Abstract

There is provided a light detection device including: a pixel including a light receiving element; and a pixel circuit including a counter circuit and a control circuit. The light receiving element is configured to receive light. The counter circuit is configured to receive a first signal based on an output of the light receiving element. The counter circuit is configured to output a second signal based on a difference between a number of the first signals in the first time period and a number of the first signals in the second time period. The control circuit is used for controlling the counter circuit.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Japanese Priority Patent Application JP2022-096651 filed on Jun. 15, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a photodetector. Background Art

[0004] PTL 1 proposes a device that includes a plurality of pixels and detects light. Each of the plurality of pixels includes a single photon avalanche diode (SPAD) element and a counter circuit.

[0005] [Citation List]

[0006] [Patent Document]

[0007] PTL 1:JP 2019-161551 Summary of the invention

[0008] [Technical issues]

[0009] There is a demand for light detection devices with improved detection performance.

[0010] It is desirable to provide a light detection device having good detection performance.

[0011] [Solution to the problem]

[0012] According to an embodiment of the present disclosure, a light detection device includes: a pixel including a light receiving element; and a pixel circuit including a counter circuit and a control circuit. The light receiving element is configured to receive light. The counter circuit is configured to receive a first signal based on an output of the light receiving element. The counter circuit is configured to output a second signal based on a difference between the number of first signals in a first time period and the number of first signals in a second time period. The control circuit is configured to control the counter circuit.

[0013] A light detection device according to an embodiment of the present disclosure includes: a pixel including a light receiving element; and a pixel circuit including a counter circuit and a control circuit. The light detection device is configured to detect an intensity signal and a motion signal.

[0014] An electronic device according to an embodiment of the present disclosure includes a signal processor and a light detection device. The light detection device includes: a pixel including a light receiving element; and a pixel circuit including a counter circuit and a control circuit. The light receiving element is configured to receive light. The counter circuit is configured to receive a first signal based on an output of the light receiving element and output a second signal based on the difference between the number of first signals in a first time period and the number of first signals in a second time period. The control circuit is configured to control the counter circuit. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a diagram showing an example of a schematic configuration of a photodetector according to an embodiment of the present disclosure.

[0016] Figure 2 is a diagram showing a configuration example of a photodetector according to an embodiment of the present disclosure.

[0017] Figure 3 is a block diagram showing a configuration example of a photodetector according to an embodiment of the present disclosure.

[0018] Figure 4 is a timing chart showing an operation example of the photodetector according to an embodiment of the present disclosure.

[0019] Figure 5 is a flowchart illustrating an operation example of a photodetector according to an embodiment of the present disclosure.

[0020] Figure 6 is a schematic diagram illustrating an example of a cross-sectional configuration of a photodetector according to an embodiment of the present disclosure.

[0021] Figure 7 is a diagram showing a configuration example of a photodetector according to Modification 1 of the present disclosure.

[0022] Figure 8 : is a diagram showing a configuration example of a photodetector according to Modification 2 of the present disclosure.

[0023] Fig. 9 is a diagram illustrating another configuration example of a photodetector according to Modification 2 of the present disclosure.

[0024] Fig.10 is a timing chart showing an operation example of the photodetector according to Modification 3 of the present disclosure.

[0025] Fig.11 : is a diagram showing a configuration example of a photodetector according to Modification 4 of the present disclosure.

[0026] Fig.12 is a block diagram showing an example of a schematic configuration of a practical vehicle control system.

[0027] Fig.13 1 is a diagram to assist in explaining an example of the installation positions of the vehicle exterior information detection portion and the imaging portion.

[0028] Fig.14 is a diagram showing an example of a schematic configuration of an endoscopic surgery system.

[0029] Fig.15 is a block diagram depicting an example of the functional configuration of a camera head and a camera control unit (CCU). DETAILED DESCRIPTION

[0030] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be noted that the description is made in the following order.

[0031] 1. Implementation Method

[0032] 2. Modifications

[0033] 3. Use Cases

[0034] 4. Practical application examples

[0035] <1. Implementation Method>

[0036] Figure 1 is a diagram showing an example of a schematic configuration of a photodetector according to an embodiment of the present disclosure. The photodetector 1 is a device (i.e., a light detecting device) configured to detect incident light. The photodetector 1 includes a plurality of pixels P. Each of the plurality of pixels P includes a light receiving element. The photodetector 1 is configured to photoelectrically convert incident light and generate a signal. The photodetector 1 is a sensor configured to detect an event. The photodetector 1 can be applied as, for example, an event-driven sensor (referred to as an event vision sensor (EVS), an event-driven sensor (EDS), a dynamic vision sensor (DVS), etc.).

[0037] exist Figure 1 In the example shown, the photodetector 1 includes a region (pixel unit 100) in which a plurality of pixels P are two-dimensionally arranged in a matrix. The light receiving element (light receiving portion) of each pixel P is, for example, a SPAD element. The photodetector 1 acquires incident light (image light) from a measurement target through an optical system (not shown) including an optical lens. The light receiving element can receive light, generate charge through photoelectric conversion, and generate photocurrent.

[0038] The photodetector 1 includes a processor 110 configured to perform signal processing. The processor 110 is a signal processing circuit. The processor 110 performs signal processing (information processing). The processor 110 performs various signal processing on the signal of each pixel. The processor 110 outputs the signal of the pixel subjected to signal processing. Although described below, the processor 110 may generate and output a signal related to the motion of the measurement target and a signal related to the intensity value. The intensity value (e.g., the intensity value associated with the pixel, such as an R pixel, a G pixel, or a B pixel) may also be referred to as a gray value in this article.

[0039] The processor 110 is also a controller. The processor 110 is configured to control each unit of the photodetector 1. The processor 110 includes a plurality of circuits, including, for example, a timing generator, a shift register, an address decoder, etc. The timing generator generates various timing signals. The processor 110 can provide a signal for driving the pixel P for each pixel P and control the operation of the pixel P.

[0040] Figure 2 : is a diagram showing a configuration example of a photodetector according to an embodiment. Figure 2 As shown, the photodetector 1 includes a first substrate 101, a second substrate 102, and a third substrate 103. The first substrate 101, the second substrate 102, and the third substrate 103 are stacked on top of each other. The first substrate 101 is stacked on the upper part of the second substrate 102. The second substrate 102 is stacked on the upper part of the third substrate 103.

[0041] The photodetector 1 has a structure (stacked structure) in which a first substrate 101, a second substrate 102, and a third substrate 103 are stacked in the Z-axis direction. Figure 2 As shown in the figure, the incident direction of the light from the object to be measured is defined as the Z-axis direction, the left and right direction of the figure perpendicular to the Z-axis direction is defined as the X-axis direction, and the direction perpendicular to the Z-axis direction and the X-axis direction is defined as the Y-axis direction. Figure 2 The direction of the arrow in indicates the direction.

[0042] A pixel unit 100 is provided on the first substrate 101. In the pixel unit 100, a plurality of pixels P are arranged in a horizontal direction (row direction) and a vertical direction (column direction), the horizontal direction being a first direction, and the vertical direction being a second direction orthogonal to the first direction. Each pixel P in the pixel unit 100 includes an optical filter 15. The optical filter 15 is configured to selectively transmit light within a specific wavelength range in a plurality of incident lights. The optical filter 15 is an RGB color filter, an optical filter transmitting infrared light, etc. Each pixel P is a pixel including a light receiving element configured to receive visible light and output a photocurrent, a pixel including a light receiving element configured to receive invisible light (also referred to as invisible light) and output a photocurrent, etc.

[0043] In one example, the plurality of pixels P in the pixel unit 100 include: a pixel (R pixel) including a filter 15 transmitting light in a red wavelength range; a pixel (G pixel) including a filter 15 transmitting light in a green wavelength range; and a pixel (B pixel) including a filter 15 transmitting light in a blue wavelength range. The R pixel, the G pixel, and the B pixel are arranged, for example, according to a so-called Bayer arrangement. The R pixel, the G pixel, and the B pixel are configured to generate a signal of an R component, a signal of a G component, and a signal of a B component, respectively. RGB pixel signals can be obtained based on charges generated by photoelectric conversion of the R pixel, the G pixel, and the B pixel.

[0044] It should be noted that the filter provided in the pixel P is not limited to the color filter for the primary colors (RGB). For example, the pixel P may include a color filter for complementary colors such as cyan (Cy), magenta (Mg), and yellow (Ye). In addition, it is also possible to avoid providing the filter 15 in part or all of the pixels P of the photodetector 1.

[0045] A processor 110 is disposed on the second substrate 102 and the third substrate 103. Figure 2 As shown, the processor 110 includes a pixel circuit unit 120 and a signal processing unit 130. The pixel circuit unit 120 and the signal processing unit 130 are respectively disposed in the second substrate 102 and the third substrate 103. Figure 2 In the example shown, the pixel circuit unit 120 is disposed in the second substrate 102 . The signal processing unit 130 includes a logic circuit, a memory, etc. The signal processing unit 130 is disposed in the third substrate 103 .

[0046] Figure 3 1 is a block diagram showing a configuration example of a photodetector according to an embodiment. The photodetector 1 includes a pixel P and a processor 110. The pixel P includes a light receiving element 10. The processor 110 includes a pixel circuit unit 120 and a signal processing unit 130. Figure 3 In the example shown, the pixel circuit unit 120 includes a generating section (also referred to as a generating circuit) 20, a supply section (also referred to as a supply circuit) 25, a counter section (also referred to as a counter circuit) 30, a control section (also referred to as a control circuit) 35, and a determining section (also referred to as a determining circuit) 40. The pixel circuit unit 120 is provided for each pixel P.

[0047] The light receiving element 10 is configured to receive light and generate a signal. The light receiving element 10 is a SPAD element. The light receiving element 10 can convert incident photons into electric charges and output a signal S1, which is an electrical signal corresponding to the incident photons. It should be noted that the light receiving element 10 is also referred to as a photoelectric conversion element (photoelectric conversion unit) configured to photoelectrically convert light.

[0048] The light receiving element 10 is electrically coupled to, for example, a power supply line, an electrode, or the like that allows a predetermined voltage to be supplied. Figure 3 In the example shown, the cathode as one of the electrodes of the light receiving element 10 is electrically coupled to the power line through the supply section 25. The power supply voltage is supplied through the power line. The anode as the other electrode of the light receiving element 10 is electrically coupled to the ground line side or a large negative voltage source.

[0049] The voltage supplied by the supply unit 25 can be applied between the cathode and anode of the light receiving element 10 so that a voltage serving as a potential difference greater than the breakdown voltage of the light receiving element 10 is applied. In other words, the potential difference between the two ends of the light receiving element 10 can be set to a potential difference greater than the breakdown voltage. In the case where a reverse bias voltage greater than the breakdown voltage is applied to the light receiving element 10, the light receiving element 10 can operate in Geiger mode. In the light receiving element 10 in Geiger mode, an avalanche multiplication phenomenon may occur in response to the incidence of photons and a pulse current may be generated. In the pixel P, a signal S1 corresponding to the photocurrent flowing through the light receiving element 10 in response to the incidence of photons is output to the generating unit 20.

[0050] The generating section 20 is configured to generate a signal S2 based on the signal S1 generated by the light receiving element 10. Figure 3 In the example shown, the generating section 20 includes an inverter. The generating section 20 includes a PMOS transistor and an NMOS transistor coupled in series. The input section of the generating section 20 is electrically connected to the cathode of the light receiving element 10 and the supply section 25. The output section of the generating section 20 is electrically coupled to the input section 31 of the counter section 30.

[0051] The generating section 20 receives a signal S1 from the light receiving element 10. The signal level of the signal S1 changes according to the current flowing through the light receiving element 10. In other words, the voltage (potential) of the signal S1 changes according to the current flowing through the light receiving element 10. For example, when the voltage of the signal S1 is higher than the threshold value, the generating section 20 outputs a low-level signal S2. In addition, when the voltage of the signal S1 is less than the threshold value, the generating section 20 outputs a high-level signal S2. The generating section 20 can output the signal S2 used as a pulse signal to the counter section 30 based on the voltage of the signal S1.

[0052] exist Figure 3 In the example shown, when the light receiving element 10 receives a photon and this causes the signal S1 to have a voltage less than the threshold voltage of the inverter used as the generating section 20, the inverter causes the voltage of the signal S2 to shift from a low level to a high level. It should be noted that the generating section 20 may include a buffer circuit, an AND circuit, or the like.

[0053] The supply section 25 is configured to supply voltage and current to the light receiving element 10. The supply section 25 is electrically coupled to a power line that provides a power supply voltage. The supply section 25 can supply voltage and current to the light receiving element 10. Figure 3 In the example shown in FIG. 2 , the supply section 25 includes a PMOS transistor. It should be noted that the supply section 25 may include a resistor.

[0054] In the case where avalanche multiplication occurs so that the potential difference between the electrodes of the light receiving element 10 is less than the breakdown voltage, the supply section 25 can supply current to the light receiving element 10. The supply section 25 recharges the light receiving element 10 to allow the light receiving element 10 to operate in the Geiger mode again. The supply section 25 is a recharging section. In other words, the supply section 25 recharges the light receiving element 10 with electric charge and recharges the voltage of the light receiving element 10.

[0055] The counter section 30 is configured to perform counting according to the received signal. The counter section 30 has an input section 31 (input terminal). The counter section 30 counts the pulses of the signal received by the input section 31. Figure 3 In the example shown, the input section 31 of the counter section 30 receives the signal S2 as a pulse signal. The counter section 30 is configured to count the pulses of the signal S2 and output a signal based on the difference in the number of pulses of the signal S2 between two time periods (also referred to as the number of the signal S2). The counter section 30 is provided for each pixel P. The counter section 30 includes, for example, an up-down counter.

[0056] The control section 35 is configured to control the counter section 30. The control section 35 includes a timing generator. The control section 35 is provided for each pixel P. The control section 35 is a timing control section. The control section 35 generates a timing signal based on a clock signal, a synchronization signal, etc. received from the outside, for example, and controls the operation of the counter section 30.

[0057] The control section 35 outputs a signal (start signal) for instructing (requesting) to start counting to the counter section 30, and controls the timing at which the counter section 30 starts counting. In addition, the control section 35 outputs a signal (stop signal) for instructing to end counting to the counter section 30, and controls the timing at which the counter section 30 ends counting. The control section 35 may also control the supply section 25. It should be noted that the supply section 25 may be controlled by a control circuit different from the control section 35.

[0058] exist Figure 3In the example shown, the control section 35 receives an overflow signal from the counter section 30 as a signal (detection signal) indicating that the count value of the counter section 30 has reached the reference value. The overflow signal is a signal indicating an overflow of the count value. In other words, the overflow signal is a signal indicating an overflow of the amount of light received by the light receiving element 10. The control section 35 provides a stop signal generated based on the overflow signal to the counter section 30 to control the counter section 30.

[0059] In one example, when a start signal is input from the control unit 35 to the counter unit 30, the counter unit 30 starts counting in the first time period Ta. The counter unit 30 counts up the number of pulses of the signal S2 in the first time period Ta. When the count value of the counter unit 30 reaches the reference value, the counter unit 30 outputs an overflow signal indicating that the count value reaches the reference value to the control unit 35, and starts counting in the second time period Tb. The counter unit 30 uses the reference value as an initial value and counts down the number of pulses of the signal S2 in the second time period Tb. The overflow signal as a detection signal is a signal indicating the end of the first time period Ta and the beginning of the second time period Tb.

[0060] The control section 35 generates a stop signal according to the overflow signal received from the counter section 30. The control section 35 outputs the stop signal to the counter section 30. The control section 35 outputs the stop signal indicating the end of the second time period Tb to the counter section 30 so that the respective lengths of the first time period Ta and the second time period Tb are equal. When the stop signal is input from the control section 35 to the counter section 30, the counter section 30 ends counting in the second time period Tb.

[0061] The counter section 30 generates and outputs a signal (differential signal S3) based on the difference between the number of pulses of the signal S2 in the first time period Ta and the number of pulses of the signal S2 in the second time period Tb. The counter section 30 may output a signal indicating a count value corresponding to the difference between the number of pulses of the signal S2 in the first time period Ta and the number of pulses of the signal S2 in the second time period Tb as the differential signal S3. For example, the signal value or count value of the differential signal S3 is a value obtained by subtracting the count value of the down count from the count value of the up count.

[0062] The determination unit 40 is configured to determine the magnitude of the difference in the number of pulses based on the differential signal S3. For example, the determination unit 40 is configured to determine whether the signal value of the differential signal S3 is greater than a predetermined threshold value. Figure 3 In the example shown, the determination section 40 includes a first threshold determination section 41 and a second threshold determination section 42 .

[0063] The first threshold determination section 41 compares the value of the differential signal S3 with the first threshold. The second threshold determination section 42 compares the value of the differential signal S3 with the second threshold. For example, the first threshold determination section 41 determines whether the signal value of the differential signal S3 or the value obtained by subtracting the count value in the second time period Tb from the count value in the first time period Ta is less than the first threshold. The second threshold determination section 42 determines whether the signal value of the differential signal S3 is greater than the second threshold.

[0064] In the case where the signal value of the differential signal S3 drops below the first threshold value, the first threshold value determination section 41 determines that the motion of the object used as the measurement target causes the positive event to occur. In the case where the signal value of the differential signal S3 is not lower than the first threshold value, the first threshold value determination section 41 determines that the positive event does not occur. In the case where the signal value of the differential signal S3 exceeds the second threshold value, the second threshold value determination section 42 determines that the motion of the measurement target causes the negative event to occur. In the case where the signal value of the differential signal S3 does not exceed the second threshold value, the second threshold value determination section 42 determines that the negative event does not occur.

[0065] In this way, the determination section 40 can detect the presence or absence of the occurrence of an event based on the differential signal S3. When the signal value of the differential signal S3 is less than the first threshold value or greater than the second threshold value, the determination section 40 determines that the event has occurred. In other words, when the amount of received light that changes due to the movement of the measurement object causes the amount of change in the count value to exceed the upper or lower threshold value, the determination section 40 determines that the event is "present".

[0066] The determination section 40 generates and outputs a signal (motion signal) related to the motion of the measurement target based on the determination results made by the first threshold determination section 41 and the second threshold determination section 42. For example, the determination section 40 outputs the signal value (count value) indicating the difference signal S3 and the presence or absence of the occurrence of an event as the motion signal S11 to the signal processing unit 130. The motion signal S11 may include a signal indicating the presence or absence of the occurrence of a positive event and a negative event.

[0067] The signal processing unit 130 is configured to acquire the differential signal S3 and the motion signal S11 of each pixel P and perform signal processing. The signal processing unit 130 is a signal processing circuit. The signal processing unit 130 can perform various signal processing by using the differential signal S3 and the motion signal S11. Figure 3 In the example shown, the signal processing unit 130 includes a bit inversion section 60 , an addition section 70 , and a memory section 75 .

[0068] The bit inversion unit 60 is configured to invert the bit value of the received signal. Figure 3In the example shown, the bit inversion section 60 receives the differential signal S3 from the counter section 30. The bit inversion section 60 inverts the bit value of the differential signal S3. The bit inversion section 60 generates a signal (grayscale signal) based on the sum of the number of pulses of the signal S2 in the first time period Ta and the number of pulses of the signal S2 in the second time period Tb.

[0069] The grayscale signal S12 has a signal value obtained by, for example, adding the count value of the second time period Tb to the count value of the first time period Ta. The grayscale signal S12 is a signal indicating grayscale. In other words, the bit inversion unit 60 restores the grayscale signal S12 indicating the grayscale value of the pixel by using the differential signal S3. The bit inversion unit 60 performs inversion processing on the differential signal S3 of each pixel P and generates the grayscale signal S12 of the pixel P.

[0070] The adding section 70 and the memory section 75 receive the grayscale signal S12 of each pixel P from the bit inversion section 60. In addition, the adding section 70 and the memory section 75 receive the motion signal S11 of each pixel P from the determination section 40. The memory section 75 is configured to hold the signal of each pixel. The memory section 75 is a frame memory. The memory section 75 can store (record) the grayscale signal S12 and the motion signal S11 of each pixel in units of frames.

[0071] The adding section 70 is configured to perform a process of adding signals of pixels. The adding section 70 estimates the moving direction of the moving object by, for example, using the motion signal S11 of each pixel P. The adding section 70 aligns (adjusts) the grayscale signal S12 of each pixel P with reference to the grayscale signal S12 held in the memory section 75 based on the result of the estimation. The adding section 70 performs a process of adding and averaging a plurality of grayscale signals S12.

[0072] The adding section 70 is an averaging section. The adding section 70 averages the plurality of grayscale signals S12 as described above to generate an average grayscale signal S12. By integrating the plurality of grayscale signals S12, the S / N ratio of the grayscale signal S12 can be improved. In this way, the signal processing unit 130 can acquire the motion signal S11, the grayscale signal S12, and the average grayscale signal S12 and output the motion signal S11, the grayscale signal S12, and the average grayscale signal S12 to the outside of the photodetector 1.

[0073] Figure 4 is a timing chart showing an operation example of the photodetector according to the embodiment. Figure 4The synchronization signal, the count value of the counter section 30, the overflow signal, the start / stop signal, and the motion signal S11 are schematically shown on the same time axis. For example, the synchronization signal is generated based on the imaging frame rate. The synchronization signal indicates the time interval of the subframe in the main frame. The control section 35 generates a start signal in synchronization with the synchronization signal. The control section 35 outputs the start signal to the counter section 30.

[0074] Figure 4 An example is shown in which the counter section 30 is an 8-bit up-down counter circuit. The time period from time t1 to time t2, the time period from time t6 to time t7, and the time period from time t11 to time t12 are the first time period Ta ( Figure 4 In each of these time periods, the counter section 30 counts up the number of pulses of the signal S2.

[0075] The time period from time t2 to time t3, the time period from time t7 to time t8, and the time period from time t12 to time t13 are the second time period Tb ( Figure 4 In each of the time periods Tb1 to Tbn, the counter section 30 counts down the number of pulses of the signal S2. Figure 4 In the embodiment, the first time period Ta1 and the second time period Tb1 have the same time interval. In addition, the first time period Ta2 and the second time period Tb2 have the same time interval. The first time period Tan and the second time period Tbn have the same time interval.

[0076] At time t1 of the first subframe, the counter section 30 starts counting the pulses of the signal S2 according to the start signal received from the control section 35. At time t2, when the count value reaches "255" as a reference value, the counter section 30 outputs an overflow signal to the control section 35. In addition, "255" is set as an initial value for down counting. The overflow signal serves as a signal indicating the timing of the end of the first period Ta1 and the start of the second period Tb1.

[0077] At time t2, the counter section 30 starts to count down the pulses of the signal S2. The control section 35 learns the length of the first time period Ta1 based on the start signal and the overflow signal. The control section 35 generates a stop signal and provides the stop signal to the counter section 30 to make the length of the second time period Tb1 equal to the length of the first time period Ta1.

[0078] At time t3, the counter section 30 ends the countdown of the pulses of the signal S2 according to the stop signal received from the control section 35. The counter section 30 outputs a differential signal S3 indicating a count value "|100" which is a difference between a count value obtained by counting up the pulses in the first time period Ta1 and a count value obtained by counting down the pulses in the second time period Tb1.

[0079] In the time period from time t4 to time t5, the determination section 40 compares "|100" and "}40". "|100" is the value of the differential signal S3. "}40" is a threshold value. The first threshold determination section 41 determines that the signal value "|100" of the differential signal S3 is lower than the first threshold value "|40", and the detection of the movement of the measurement object causes a positive event to occur. It is to be noted that the signal value "8|100" of the differential signal S3 does not exceed the second threshold value "+40". ​​Therefore, the second threshold determination section 42 determines that a negative event has not occurred.

[0080] In the time period from time t5 to time t6, the determination section 40 generates a motion signal S11 indicating the signal value of the differential signal S3 and the occurrence of a positive event according to the determination results made by the first threshold determination section 41 and the second threshold determination section 42. The bit inversion section 60 of the signal processing unit 130 performs an inversion process on the bit value of the differential signal S3 and generates a grayscale signal S12. The signal processing unit 130 outputs the motion signal S11 of each pixel P generated in the first subframe to the outside. In addition, the signal processing unit 130 may output the grayscale signal S12 of each pixel P generated in the first subframe to the outside. The signal processing unit 130 causes the memory section 75 to maintain the grayscale signal S12 and the motion signal S11 of each pixel P in the first subframe.

[0081] At time t6 in the second subframe, the count value of the counter section 30 is reset. The counter section 30 starts counting the pulses of the signal S2 according to the start signal. When the count value reaches the reference value "255" at time t7, the counter section 30 outputs an overflow signal to the control section 35. "255" is set as the initial value for down counting.

[0082] At time t7, the counter section 30 starts counting down the pulses of the signal S2. The control section 35 learns the length of the first time period Ta2 by using the overflow signal. The control section 35 generates a stop signal and provides the stop signal to the counter section 30 to make the length of the second time period Tb2 equal to the length of the first time period Ta2.

[0083] At time t8, the counter section 30 ends counting down the pulses of the signal S2 according to the stop signal. The counter section 30 outputs a differential signal S3 indicating a difference "+20" between the count value in the first period Ta2 and the count value in the second period Tb2.

[0084] In the time period from time t8 to time t9, the determination section 40 compares the value “+20” of the differential signal S3 with the threshold value “+40”. The first threshold determination section 41 and the second threshold determination section 42 respectively determine that the value of the differential signal S3 falls within the threshold value, and determine that a positive event has not occurred or a negative event has not occurred.

[0085] During the time period from time t9 to time t10, the determination section 40 generates a motion signal S11 indicating the signal value of the differential signal S3 and indicating that no event has occurred. The bit inversion section 60 performs an inversion process on the bit value of the differential signal S3 and generates a grayscale signal S12. The signal processing unit 130 outputs the motion signal S11 of each pixel P generated in the second subframe to the outside. The signal processing unit 130 may also output the grayscale signal S12 of each pixel P generated in the second subframe to the outside. In addition, the signal processing unit 130 causes the memory section 75 to retain the grayscale signal S12 and the motion signal S11 of each pixel P in the second subframe.

[0086] exist Figure 4 At time t11 in the Nth subframe shown, the count value of the counter section 30 is reset. Thereafter, the counter section 30 starts counting the pulses of the signal S2 according to the start signal. When the count value reaches the reference value "255" at time t12, the counter section 30 outputs an overflow signal to the control section 35. "255" is set as the initial value of the count down.

[0087] At time t12, the counter section 30 starts counting down the pulses of the signal S2. The control section 35 learns the length of the first period Tan by using the overflow signal. The control section 35 provides a stop signal to the counter section 30 to make the length of the second period Tbn equal to the length of the first period Tan.

[0088] At time t13, the counter section 30 ends counting down the pulses of the signal S2 according to the stop signal. The counter section 30 outputs a differential signal S3 indicating a difference "+100" between the count value in the first period Tan and the count value in the second period Tbn.

[0089] During the time period from time t14 to time t15, the determination section 40 compares the value "+100" of the differential signal S3 with the threshold value "|40". The signal value "+100" of the differential signal S3 is not lower than the first threshold value "|40". Therefore, the first threshold determination section 41 determines that the positive event has not occurred. The second threshold determination section 42 determines that the signal value "+100" of the differential signal S3 is higher than the second threshold value "+40", and detects that the motion of the measurement target causes the negative event to occur.

[0090] During the time period from time t15 to time t16, the determination section 40 generates a motion signal S11 indicating the signal value of the differential signal S3 and the occurrence of a negative event according to the determination results made by the first threshold determination section 41 and the second threshold determination section 42. The bit inversion section 60 performs an inversion process on the bit value of the differential signal S3 and generates a grayscale signal S12. The signal processing unit 130 outputs the motion signal S11 of each pixel P generated in the Nth subframe to the outside. The signal processing unit 130 may also output the grayscale signal S12 of each pixel P generated in the Nth subframe to the outside. In addition, the signal processing unit 130 causes the memory section 75 to maintain the grayscale signal S12 and the motion signal S11 of each pixel P in the Nth subframe.

[0091] In the time period from time t16 to time t17, the adding section 70 of the signal processing unit 130 calculates the moving direction of the moving object by using the motion signal S11 in each subframe held in the memory section 75. The adding section 70 aligns the grayscale signal S12 of each pixel P in a plurality of subframes according to the movement of the object, and adds and averages the grayscale signal S12 in each subframe. The signal processing unit 130 can output the grayscale signal S12 averaged by the adding section 70 to the outside.

[0092] In this way, the photodetector 1 according to the present embodiment includes a counter section 30 and a control section 35, each of which is configured to acquire a differential signal S3. The counter section 30 and the control section 35 are provided for each pixel P. This makes it possible to detect the differential signal S3 of each pixel P and calculate the motion signal S11 and the grayscale signal S12 of the pixel P. The motion signal and the grayscale signal of the same pixel P can be obtained simultaneously. A photodetector with high detection performance can be realized.

[0093] In addition, in the present embodiment, a SPAD element is used as the light receiving element 10. This makes it possible to suppress the reduction in detection accuracy in the case of measurement with low illumination and low contrast. The movement of the measurement target can be accurately detected and the grayscale value can be output. In addition, by averaging the grayscale signal, the S / N ratio can be improved, and a grayscale signal with less noise can be obtained.

[0094] Figure 5FIG. 1 is a flowchart showing an operation example of the photodetector according to this embodiment. Figure 5 This flowchart describes an operation example of the photodetector 1.

[0095] exist Figure 5 In step S100 shown, the count value of the counter section 30 is reset according to the start signal from the control section 35. The counter section 30 starts counting the pulses of the signal S2 generated in response to the reception of the photon. When the count value reaches the reference value (for example, the full code value "255"), the counter section 30 outputs an overflow signal to the control section 35. In step S110, the counter section 30 starts counting down the pulses of the signal S2.

[0096] In step S120, the counter section 30 stops down counting of pulses according to the stop signal from the control section 35. The counter section 30 outputs a differential signal S3 indicating the count result to the determination section 40 and the signal processing unit 130.

[0097] In step S130, the determination section 40 compares the signal value of the differential signal S3 with the threshold value to determine whether an event has occurred. In step S140, the determination section 40 outputs the motion signal S11 indicating the signal value of the differential signal S3 and whether an event has occurred to the signal processing unit 130 according to the determination result. In addition, the determination section 40 outputs the motion signal S11 to the outside of the photodetector 1.

[0098] In step S150, the bit inversion section 60 performs bit inversion processing on the differential signal S3 to generate a grayscale signal S12. In step S160, the signal processing unit 130 learns whether to perform processing of averaging the grayscale signal S12. In the case where the averaging process is not performed ("No" in step S160), the process proceeds to step S170. In step S170, the signal processing unit 130 outputs the grayscale signal S12 of each pixel P to the outside of the photodetector 1. The grayscale signal S12 is generated by the bit inversion section 60.

[0099] In the case where the averaging process is performed (YES in step S160), the process proceeds to step S180. It should be noted that the presence or absence of the averaging process may be automatically set by the photodetector 1 or may be specified by the user.

[0100] In step S180, the adding section 70 shifts and adds and averages the grayscale signal S12 of each pixel P in the corresponding frame stored in the memory section 75 according to the moving direction of the target obtained from the motion signal S11. In step S190, the signal processing unit 130 may output the grayscale signal S12 averaged by the adding section 70 to the outside. After step S190, the photodetector 1 completes Figure 5The process is shown in the flowchart.

[0101] Figure 6 1 is a schematic diagram showing an example of a cross-sectional configuration of a photodetector according to this embodiment. The photodetector 1 has a configuration in which a first substrate 101, a second substrate 102, and a third substrate 103 are stacked in the Z-axis direction. The first substrate 101, the second substrate 102, and the third substrate 103 each include a semiconductor substrate (e.g., a silicon substrate).

[0102] like Figure 6 As shown, the first substrate 101, the second substrate 102 and the third substrate 103 have first surfaces 11S1, 12S1 and 13S1 and second surfaces 11S2, 12S2 and 13S2, respectively. The first surfaces 11S1, 12S1 and 13S1 are provided with transistors. Each of the first surfaces 11S1, 12S1 and 13S1 is an element formation surface on which elements such as transistors are formed. Each of the first surfaces 11S1, 12S1 and 13S1 is provided with a gate electrode, a gate oxide film, etc.

[0103] The first substrate 101 is provided with a plurality of pixels P each including a light receiving element 10. Figure 6 As schematically shown in FIG. 1 , each pixel P includes a multiplication section 17 (multiplication region) configured to perform avalanche multiplication. On the second surface 11S2 side of the first substrate 101 , a lens section 16 , an optical filter 15 , and the like are provided for each pixel P. The lens section 16 condenses light.

[0104] like Figure 6 As shown, the first surface 11S1 of the first substrate 101 is provided with a wiring layer 111. In addition, the first surface 12S1 of the second substrate 102 is provided with a wiring layer 121. The second surface 12S2 of the second substrate 102 is provided with a wiring layer 122. The first surface 13S1 of the third substrate 103 is provided with a wiring layer 131. The wiring layers 111, 121, 122, 131 are composed of, for example, a conductor film and an insulating film. Each of the wiring layers 111, 121, 122 and 131 includes a plurality of wirings, vias, etc. Each of the wiring layers 111, 121, 122 and 131 includes, for example, two or more layers of wiring.

[0105] Each of the wiring layers 111, 121, 122, and 131 has a configuration in which a plurality of wirings are stacked, for example, with an interlayer insulating layer (interlayer insulating film) interposed therebetween. Each wiring layer is formed using aluminum (Al), copper (Cu), tungsten (W), polysilicon (Poly-Si), or the like. In one example, each of the interlayer insulating layers is formed by using a single-layer film containing one of silicon oxide (SiO), silicon nitride (SiN), silicon oxynitride (SiON), or the like, or a stacked film containing two or more of them.

[0106] Note that the first substrate 101 and the wiring layer 111 are sometimes collectively referred to as the first substrate 101 (or the first circuit layer). In addition, the second substrate 102 and the wiring layers 121 and 122 are sometimes collectively referred to as the second substrate 102 (or the second circuit layer). The third substrate 103 and the wiring layer 131 are sometimes collectively referred to as the third substrate 103 (or the third circuit layer).

[0107] exist Figure 6 In the example shown, the first substrate 101 and the second substrate 102 are stacked by bonding electrodes, wherein the first surface 11S1 and the first surface 12S1 are opposite to each other. Elements such as transistors are formed on the first face 11S1 and the first face 12S1. In other words, the first substrate 101 and the second substrate 102 are bonded with their respective front surfaces opposite to each other. The second substrate 102 and the third substrate 103 are stacked by bonding electrodes having the second surface 12S2 and the first surface 13S1 opposite to each other. Elements such as transistors are formed on the first surface 13S1. In other words, the second substrate 102 and the third substrate 103 are bonded when the rear surface of the second substrate 102 and the front surface of the third substrate 103 are opposite to each other.

[0108] In one example, the first substrate 101 and the second substrate 102 are bonded together by bonding metal electrodes each including copper (Cu). In other words, the first substrate 101 and the second substrate 102 are bonded together by forming a Cu-Cu junction. In addition, the second substrate 102 and the third substrate 103 are also bonded together by forming, for example, a Cu-Cu junction. It should be noted that the electrodes for bonding may include, for example, a metal material such as nickel (Ni), cobalt (Co), or tin (Sn) other than copper (Cu). Alternatively, these electrodes may include another material.

[0109] The second substrate 102 and the wiring layers 121 and 122 include a plurality of through electrodes 80. Each through electrode 80 is an electrode that penetrates the second substrate 102. The through electrode 80 is formed to extend in the Z-axis direction and reach the inside of the wiring layer 122 of the second substrate 102. The through electrode 80 includes, for example, tungsten (W), aluminum (Al), cobalt (Co), molybdenum (Mo), ruthenium (Ru), etc. The through electrode 80 electrically connects the circuit provided on the first surface 12S1 side of the second substrate 102 and the circuit provided on the first surface 13S1 side of the third substrate 103.

[0110] The first substrate 101 is provided with a pad (PAD). The pad is an electrode formed by using, for example, aluminum (Al). It should be noted that the pad may include another metal material. A plurality of pads are provided in the photodetector 1. For example, each pad may supply a power supply voltage input from the outside to each circuit of the first substrate 101 to the third substrate 103.

[0111] <Work and Effect>

[0112] The photodetector (photodetector 1) according to the present embodiment includes a light receiving element (light receiving element 10) and an input section (input section 31). The light receiving element photoelectrically converts light. An input section receives a first signal (signal S2) based on the charge generated by the light receiving element. The photodetector includes a counter section (counter 30) and a control section (control section 35). The counter section is configured to output a second signal (differential signal S3) based on the difference between the number of pulses of the first signal in the first time period and the number of pulses of the first signal in the second time period. The control section is configured to control the counter section. The counter section and the control section are provided for each pixel. Each pixel includes a light receiving element.

[0113] The photodetector 1 according to the present embodiment includes a counter section 30 configured to acquire a differential signal S3. The counter section 30 and the control section 35 are provided for each pixel P. This makes it possible to detect the differential signal of each pixel P and obtain a motion signal and a grayscale signal of the pixel P. A photodetector with high detection performance can be realized.

[0114] Next, a modification of the present disclosure will be described. Hereinafter, the same reference numerals are given to the same components as those in the above-described embodiment, and the description thereof will be appropriately omitted.

[0115] (1-1. Modification 1)

[0116] In the above-mentioned embodiment, the configuration example of the filter 15 is described, but the configuration of the filter 15 is not limited thereto. Figure 7 In the example shown in , each pixel P does not necessarily include the filter 15. The photodetector 1 may have a configuration in which the filter 15 such as an RGB color filter is not provided. It should be noted that each of the pixels P is not limited to a pixel including the light receiving element 10 configured to receive visible light and output a photocurrent as described above. The pixel P may be a pixel including the light receiving element 10 configured to receive invisible light (e.g., infrared light) and output a photocurrent.

[0117] (1-2. Modification 2)

[0118] In the above-described embodiment and modification examples, the configuration example of the photodetector 1 has been described, but the photodetector 1 may have a configuration in which two substrates are stacked. Figure 8 1 is a diagram showing a configuration example of a photodetector according to Modification 2. The photodetector 1 has a configuration (two-layer configuration) in which a first substrate 101 and a second substrate 102 are stacked in the Z-axis direction. In this case, for example, a pixel circuit unit 120 and a signal processing unit 130 may be provided in the second substrate 102. It should be noted that Fig. 9 As shown in , the filter 15 (such as an RGB color filter) is not necessarily provided in each pixel P of the first substrate 101.

[0119] (1-3. Modification 3)

[0120] In the above-mentioned embodiment, an example of starting counting in synchronization with a synchronization signal has been described. However, the photodetector 1 can start counting by using a start signal asynchronous with the synchronization signal. Compared with the case where the photodetector 1 controls the timing of the counting operation in synchronization with the synchronization signal, the photodetector 1 makes it possible to perform high-speed operation.

[0121] Fig.10 is a timing chart showing an operation example of the photodetector according to Modification 3. Fig.10 In the illustrated embodiment, the control section 35 of the photodetector 1 may supply the start signal to the counter section 30 without synchronizing with the synchronization signal. Figure 4 This makes it possible to reduce the time from the end timing of the count operation to the start timing of the down count operation (such as Fig.10 ). Thus, the motion signal and the grayscale signal can be detected at high speed.

[0122] (1-4. Modification 4)

[0123] Fig.11 is a diagram showing an example of a configuration of a photodetector according to Modification 4. The counter section 30 may include a first counter 32a and a second counter 32b, each of which is configured to count the signal S2 received by the input section 31. For example, the first counter 32a is configured to count the number of pulses of the signal S2 in the first time period Ta. In addition, the second counter 32b is configured to count the number of pulses of the signal S2 in the second time period Tb. The counter section 30 can generate and output a differential signal S3 based on the difference between the count value of the first counter 32a and the count value of the second counter 32b. In this modification, an effect similar to that according to the above-described embodiment can also be obtained.

[0124] <3. Example of use>

[0125] For example, the above-described photodetector 1 may be used in various cases of sensing light such as visible light, infrared light, ultraviolet light, or X-rays as follows.

[0126] - A device that captures images for viewing, such as a digital camera or a mobile device each having a camera function;

[0127] - Equipment used in traffic, such as on-board sensors that capture images of the front, rear, surroundings, interior, etc. of a car for safe driving (such as automatic stop) and for identifying the driver's status; monitoring cameras that monitor traveling vehicles and roads; and ranging sensors that measure the distance between vehicles;

[0128] - Devices used in home appliances (such as televisions, refrigerators, or air conditioners) to capture images of user gestures and operate the appliance based on the gestures

[0129] - Equipment for medical care and health care purposes, such as endoscopes or equipment for photographing blood vessels by receiving infrared light

[0130] - Equipment used for security purposes, such as surveillance cameras for crime prevention or cameras for personal authentication

[0131] -Equipment for beauty purposes, such as skin measurement equipment that takes images of the skin and microscopes that take images of the scalp

[0132] -Devices for sports use, such as action cameras or wearable cameras for sports applications

[0133] -Agricultural equipment, such as cameras used to monitor the conditions of fields and crops.

[0134] <4. Practical Application Examples>

[0135] (Useful example of mobile object)

[0136] The technology according to the embodiments of the present disclosure (the present technology) can be applied to various products. For example, the technology according to the embodiments of the present disclosure can be implemented as a device installed on any type of mobile body (such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot).

[0137] Fig.12 : is a block diagram showing an example of a schematic configuration of a vehicle control system as an example of a moving body control system to which the technology according to the embodiment of the present disclosure can be applied.

[0138] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. Fig.12In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are shown as the functional configuration of the integrated control unit 12050.

[0139] The drive system control unit 12010 controls the operation of the devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 is used as a control device to control: a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine, a drive motor, etc., a drive force transmitting mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating the braking force of the vehicle, etc.

[0140] The body system control unit 12020 controls the operation of various types of equipment configured for the body according to various programs. For example, the body system control unit 12020 is used as a control device to control the following items: a keyless entry system, a smart key system, a power window device, or various lights such as headlights, reverse lights, brake lights, turn signals, fog lights, etc. In this case, the body system control unit 12020 can receive as input radio waves transmitted from a mobile device that replaces the key or signals from various switches. The body system control unit 12020 receives these input radio waves or signals to control the door lock device, power window device, lights, etc. of the vehicle.

[0141] The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to an imaging unit 12031. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to image an image of the vehicle exterior, and receives the imaged image. Based on the received image, the vehicle exterior information detection unit 12030 may perform a process of detecting an object (such as a person, vehicle, obstacle, sign, symbol, etc. on the road), or perform a process of detecting the distance to the object.

[0142] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or can output the electrical signal as information about the measured distance. In addition, the light received by the imaging unit 12031 may be visible light, or may be invisible light such as infrared rays.

[0143] The in-vehicle information detection unit 12040 detects information about the interior of the vehicle. The in-vehicle information detection unit 12040 may be connected to a driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that photographs the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 may calculate the driver's fatigue level or the driver's concentration level, or may distinguish whether the driver is dozing off.

[0144] The microcomputer 12051 can calculate a control target value for a driving force generation device, a steering mechanism, or a braking device based on information about the interior or exterior of the vehicle obtained by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing functions of an advanced driver assistance system (ADAS), including collision avoidance or impact buffering for the vehicle, following driving based on vehicle-to-vehicle distance, vehicle speed maintenance driving, an alarm for vehicle collision, an alarm for vehicle lane deviation, and the like.

[0145] In addition, the microcomputer 12051 can control the driving force generating device, the steering mechanism, and the braking device based on the information about the outside or inside of the vehicle obtained by the outside information detection unit 12030 or the inside information detection unit 12040, thereby performing collaborative control intended for automatic driving, etc. that is independent of the driver's operation.

[0146] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the exterior of the vehicle obtained by the exterior information detection unit 12030. For example, the microcomputer 12051 can control the headlights to change from high beam to low beam based on the position of the preceding vehicle or oncoming vehicle detected by the exterior information detection unit 12030, thereby performing cooperative control aimed at preventing glare by controlling the headlights.

[0147] The sound / image output unit 12052 transmits an output signal of at least one of sound and image to an output device, which can notify information to the passengers of the vehicle or the outside of the vehicle in a visual or auditory manner. Fig.12 In the example of FIG. 1206 , an audio speaker 12061, a display portion 12062, and an instrument panel 12063 are shown as output devices. The display portion 12062 may include, for example, at least one of an in-vehicle display and a head-up display.

[0148] Fig.13 12031 is a diagram showing an example of the installation position of the imaging unit 12031.

[0149] exist Fig.13 In the figure, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104 and 12105.

[0150] The imaging units 12101, 12102, 12103, 12104 and 12105 may be arranged at the positions of the front nose, side mirrors, rear bumper, rear door and the upper portion of the windshield inside the vehicle 12100. The imaging unit 12101 arranged at the front nose and the imaging unit 12105 arranged at the upper portion of the windshield inside the vehicle mainly obtain images in front of the vehicle 12100. The imaging units 12102 and 12103 arranged at the side mirrors mainly obtain images of the sides of the vehicle 12100. The imaging unit 12104 arranged at the rear bumper or rear door mainly obtains images of the rear of the vehicle 12100. The imaging unit 12105 arranged at the upper portion of the windshield inside the vehicle is mainly used to detect the front vehicle, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0151] By the way, Fig.13 An example of the shooting range of the imaging units 12101 to 12104 is shown. Imaging range 12111 represents the imaging range of the imaging unit 12101 arranged at the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging units 12102 and 12103 arranged at the side mirrors. Imaging range 12114 represents the imaging range of the imaging unit 12104 arranged at the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 12100 observed from above can be obtained by superimposing the image data imaged by the imaging units 12101 to 12104.

[0152] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0153] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the time change of the distance (relative speed relative to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, and thereby extract the nearest three-dimensional object as the leading vehicle, which specifically exists on the driving path of the vehicle 12100 and travels at a predetermined speed (e.g., equal to or greater than 0 km / h) in the substantially same direction as the vehicle 12100. In addition, the microcomputer 12051 can pre-set the following distance to be maintained from the leading vehicle, and perform automatic braking control (including following parking control), automatic acceleration control (including following starting control), etc. Therefore, cooperative control intended for automatic driving, etc. that does not depend on the operation of the driver, can be performed.

[0154] For example, the microcomputer 12051 can classify the three-dimensional object data about the three-dimensional object into three-dimensional object data of two-wheeled vehicles, standard-sized vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, and extract the classified three-dimensional object data for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes whether the obstacles around the vehicle 12100 are obstacles that can be visually identified by the driver of the vehicle 12100, or obstacles that are difficult for the driver of the vehicle 12100 to visually identify. Then, the microcomputer 12051 determines the collision risk, which indicates the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 outputs an alarm to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or avoidance steering via the drive system control unit 12010. Thus, the microcomputer 12051 can assist driving to avoid collisions.

[0155] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 is capable of identifying pedestrians by determining whether there are pedestrians in the imaging images of the imaging units 12101 to 12104. This pedestrian recognition is performed, for example, by the following programs: a program for extracting characteristic points in the imaging images of the imaging units 12101 to 12104 as infrared cameras, and a program for determining whether it is a pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there are pedestrians in the imaging images of the imaging units 12101 to 12104 and thus identifies the pedestrians, the sound / image output unit 12052 controls the display unit 12062 to display a square outline superimposed on the identified pedestrian for emphasizing the identified pedestrian. The sound / image output unit 12052 can also control the display unit 12062 to display an icon representing a pedestrian at a desired position.

[0156] An embodiment of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied has been described above. The technology according to an embodiment of the present disclosure can be applied to, for example, the imaging section 12031 among the above-mentioned components. Specifically, for example, the photodetector 1 can be applied to the imaging section 12031. Application of the technology according to an embodiment of the present disclosure to the imaging section 12031 makes it possible to obtain a high-definition captured image and to perform highly accurate control using the captured image in the mobile body control system.

[0157] (Example of actual application of endoscopic surgery system)

[0158] The technology according to the embodiment of the present disclosure (the present technology) can be applied to various products. For example, the technology according to the embodiment of the present disclosure can be applied to an endoscopic surgery system.

[0159] Fig.14 It is a diagram of one example of a schematic configuration of an endoscopic surgery system to which the technology (the present technology) according to one embodiment of the present disclosure can be applied.

[0160] exist Fig.14 , a state is shown in which a surgeon (medical doctor) 11131 is using an endoscopic surgery system 11000 to perform surgery on a patient 11132 on a bed 11133. As depicted, the endoscopic surgery system 11000 includes an endoscope 11100, other surgical tools 11110 such as a pneumoperitoneum tube 11111 and an energy treatment tool 11112, a support arm device 11120 supporting the endoscope 11100, and a cart 11200 equipped with various devices for endoscopic surgery.

[0161] The endoscope 11100 includes a lens barrel 11101 of which a predetermined length region at the distal end is inserted into a body cavity of a patient 11132 and a camera 11102 connected to the proximal end of the lens barrel 11101. In the depicted example, the endoscope 11100 is depicted as including a hard scope having a hard lens barrel 11101. However, the endoscope 11100 may also include a soft scope having a soft lens barrel 11101.

[0162] The lens barrel 11101 has an opening at its distal end for mounting an objective lens. The light source device 11203 is connected to the endoscope 11100 so that light generated by the light source device 11203 is introduced into the distal end of the lens barrel 11101 through a light guide extending in the lens barrel 11101 and irradiated through the objective lens toward an observation target of the body cavity of the patient 11132. It should be noted that the endoscope 11100 may be a straight-view mirror or may be an oblique mirror or a side-view mirror.

[0163] The camera 11102 is provided with an optical system and an imaging element so that the reflected light (observation light) from the observation target is collected on the imaging element through the optical system. The observation light is photoelectrically converted by the imaging element to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. The image signal is transmitted to the CCU 11201 as RAW data.

[0164] The CCU 11201 includes a central processing unit (CPU), a graphics processing unit (GPU), etc. to integrally control the operation of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera 11102 and performs various image processing on the image signal for displaying an image based on the image signal, such as development processing (demosaic processing).

[0165] The display device 11202 displays thereon an image based on an image signal on which image processing has been performed by the CCU 11201 under the control of the CCU 11201 .

[0166] The light source device 11203 includes a light source such as a light emitting diode (LED) or the like, and supplies irradiation light when imaging the surgical area onto the endoscope 11100 .

[0167] The input device 11204 is an input interface for the endoscopic surgery system 11000. The user can input various information or instructions into the endoscopic surgery system 11000 through the input device 11204. For example, the user inputs an instruction to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) of the endoscope 11100.

[0168] The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 for burning or cutting of tissues, sealing of blood vessels, etc. The pneumoperitoneum device 11206 feeds gas into the body cavity of the patient 11132 through the pneumoperitoneum tube 11111 to inflate the body cavity to ensure the field of view of the endoscope 11100 and the working space of the surgeon. The recorder 11207 is a device capable of recording various information related to the operation. The printer 11208 is a device capable of printing information related to the operation in various forms such as text, images or charts.

[0169] It should be noted that the light source device 11203 that supplies irradiation light when imaging the surgical area to the endoscope 11100 may include a white light source, including, for example, an LED, a laser source, or a combination thereof. When the white light source includes a combination of red, green, and blue (RGB) laser sources, since the output intensity and output time of each color (each wavelength) can be controlled with high precision, the white balance of the captured image can be adjusted by the light source device 11203. Further, in this case, if the laser beams of the respective RGB laser sources are irradiated on the observation target in time, the driving of the imaging element of the camera 11102 is controlled synchronously with the irradiation time, and images corresponding to the R, G, and B colors individually can be captured in time. According to this method, a color image can be obtained even if a color filter is not provided to the imaging element.

[0170] Furthermore, the light source device 11203 can be controlled so that the output light intensity changes for each predetermined time. By controlling the driving of the imaging element of the camera 11102 in synchronization with the time of the change in light intensity to acquire images differentiated according to time, and synthesizing the images, a high dynamic range image without underexposed thick shadows and overexposed highlight areas can be generated.

[0171] Further, the light source device 11203 can be configured to supply light of a predetermined wavelength band, ready for specific light observation. In specific light observation, for example, by utilizing the wavelength dependence of the absorption of light by body tissues, using a narrow-band irradiation light compared to the irradiation light (i.e., white light) during ordinary observation, a predetermined tissue such as blood vessels on the surface of a mucosa is imaged with high contrast, which is narrow-band observation (narrow-band imaging). Alternatively, in specific light observation, fluorescence observation can be performed to obtain an image from the fluorescence generated by irradiating excitation light. In fluorescence observation, fluorescence observation (autofluorescence observation) of body tissues can be performed by irradiating excitation light on body tissues, or a fluorescent image can be obtained by locally injecting a reagent such as indocyanine green (ICG) into body tissues and irradiating excitation light corresponding to the fluorescence wavelength of the reagent onto the body tissues. The light source device 11203 can be configured to supply such narrow-band light and / or excitation light suitable for the above-mentioned specific light observation.

[0172] Fig.15 It is depicted in Fig.14 A block diagram of an example of the functional configuration of camera 11102 and CCU 11201 is depicted in FIG.

[0173] The camera 11102 includes a lens unit 11401, an imaging unit 11402, a driving unit 11403, a communication unit 11404, and a camera control unit 11405. The CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera 11102 and the CCU 11201 are connected via a transmission cable 11400 for communication with each other.

[0174] The lens unit 11401 is an optical system provided at a connection position with the lens barrel 11101. Observation light collected by the distal end of the lens barrel 11101 is guided to the camera 11102 and introduced into the lens unit 11401. The lens unit 11401 includes a combination of a plurality of lenses, including a zoom lens and a focusing lens.

[0175] The number of imaging elements included in the imaging unit 11402 can be one (single-board type) or multiple (multi-board type). When the imaging unit 11402 is configured as a multi-board type, for example, image signals corresponding to each R, G and B are generated by each imaging element, and the image signals can be synthesized to obtain a color image. The imaging unit 11402 can also be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to a three-dimensional (3D) display, respectively. If a 3D display is performed, the surgeon 11131 can more accurately understand the depth of the living tissue in the surgical area. It should be noted that when the imaging unit 11402 is configured as a multi-board type, the lens unit 11401 also provides multiple systems corresponding to each imaging element.

[0176] Furthermore, the camera unit 11402 may not necessarily be disposed on the camera head 11102. For example, the camera unit 11402 may be disposed immediately after the objective lens in the lens barrel 11101.

[0177] The driving unit 11403 includes an actuator and moves the zoom lens and the focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera control unit 11405. Thus, the magnification and focus of the image captured by the camera unit 11402 can be appropriately adjusted.

[0178] The communication unit 11404 includes a communication device for transmitting and receiving various information to and from the CCU 11201. The communication unit 11404 transmits the image signal acquired by the imaging unit 11402 to the CCU 11201 through the transmission cable 11400 as RAW data.

[0179] In addition, the communication unit 11404 receives a control signal for controlling the driving of the camera 11102 from the CCU 11201 and supplies the control signal to the camera control unit 11405. The control signal includes information related to imaging conditions, such as information specifying a frame rate for capturing an image, information specifying an exposure value when capturing an image, and / or information specifying a magnification and a focus for capturing an image.

[0180] It should be noted that imaging conditions such as frame rate, exposure value, magnification or focus may be appropriately specified by the user, or may be automatically set based on the acquired image signal by the control unit 11413 of the CCU 11201. In the latter case, an automatic exposure (AE) function, an automatic focus (AF) function, and an automatic white balance (AWB) function are mounted in the endoscope 11100.

[0181] The camera control unit 11405 controls the driving of the camera 11102 based on the control signal received from the CCU 11201 through the communication unit 11404 .

[0182] The communication unit 11411 includes a communication device for transmitting and receiving various information to and from the camera 11102. The communication unit 11411 receives an image signal transmitted thereto from the camera 11102 through the transmission cable 11400.

[0183] Further, the communication unit 11411 transmits a control signal for controlling the driving of the camera 11102 to the camera 11102. The image signal and the control signal may be transmitted through electrical communication, optical communication, or the like.

[0184] The image processing unit 11412 performs various image processing on the image signal in the form of RAW data transmitted thereto from the camera 11102 .

[0185] The control unit 11413 performs various controls related to imaging of the operation area, etc., through the endoscope 11100 , and displaying the captured image obtained by imaging the operation area, etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102 .

[0186] Further, the control unit 11413 controls the display device 11202 to display the captured image based on the image signal that has been image-processed by the image processing unit 11412, in which the surgical area, etc. are imaged. At this time, the control unit 11413 can use various image recognition technologies to identify various objects in the captured image. For example, the control unit 11413 can identify surgical tools such as forceps, etc., specific living areas, bleeding, fog when using the energy treatment tool 11112, etc. by detecting the shape, color, etc. of the edge of the object contained in the captured image. When it controls the display device 11202 to display the captured image, the control unit 11413 can use the recognition result to display various surgical support information in a manner overlapping with the image of the surgical area. When the surgical support information is displayed and presented to the surgeon 11131 in an overlapping manner, the burden of the surgeon 11131 can be reduced and the surgeon 11131 can continue the operation with certainty.

[0187] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 to each other is an electric signal cable for communication of electric signals, an optical fiber for optical communication, or a composite cable for both electric communication and optical communication.

[0188] Here, although wired communication is performed by utilizing the transmission cable 11400 in the described example, communication between the camera 11102 and the CCU 11201 may also be wireless communication.

[0189] An example of an endoscopic surgical system to which the technology according to an embodiment of the present disclosure can be applied has been described above. The technology according to an embodiment of the present disclosure can be advantageously applied to, for example, the imaging unit 11402 of the camera head 11102 provided to the endoscope 11100 among the above-mentioned components. Applying the technology according to an embodiment of the present disclosure to the imaging unit 11402 makes it possible to increase the sensitivity of the imaging unit 11402 and to provide a high-definition endoscope 11100.

[0190] Although the present disclosure has been described with reference to embodiments, variations, use cases, and practical application examples, the content of the present technology is not limited to the above-mentioned embodiments, etc. Various modifications are possible. For example, the above-mentioned modifications have been described as modifications of the above-mentioned embodiments, but the components according to the various modifications can be appropriately combined.

[0191] A photodetector according to an embodiment of the present disclosure includes: a light receiving element; a counter unit; and a control unit. The light receiving element is configured to receive light and output a photocurrent. The counter unit includes an input unit for receiving a first signal based on the photocurrent. The counter unit is configured to output a second signal based on the difference between the number of pulses of the first signal in a first time period and the number of pulses of the first signal in a second time period. The control unit is configured to control the counter unit. The counter unit and the control unit are provided for each pixel. Each pixel includes a light receiving element. This makes it possible to obtain a motion signal and a grayscale signal of the pixel. A photodetector with high detection performance can be realized.

[0192] It should be noted that the effects described herein are merely examples, but are not limited to the description. Other effects may exist. In addition, the embodiments of the present disclosure may also have the following configurations. (1)

[0194] A light detection device, comprising:

[0195] a pixel including a light receiving element configured to receive light; and

[0196] A pixel circuit, comprising:

[0197] a counter circuit configured to receive a first signal based on an output of the light receiving element and output a second signal based on a difference between the number of the first signal in the first time period and the number of the first signal in the second time period; and

[0198] The control circuit is configured to control the counter circuit. (2)

[0200] The light detection device according to (1), wherein the light receiving element includes a single photon avalanche diode. (3)

[0202] The light detection device according to (1) or (2), wherein the counter circuit is configured to output the detection signal when the number of the first signals in the first time period reaches a reference value. (4)

[0204] The light detection device according to (3), wherein the detection signal indicates an end of the first time period. (5)

[0206] The light detection device according to any one of (3) to (4), wherein the counter circuit includes an up-and-down counter. (6)

[0208] The light detection device according to any one of (3) to (5), wherein the control circuit is configured to control the counter circuit based on the detection signal. (7)

[0210] The light detection device according to any one of (3) to (6), wherein the control circuit is configured to output a stop signal indicating an end of the second time period to the counter circuit based on the detection signal. (8)

[0212] The light detection device according to (7), wherein the control circuit is configured to output a stop signal indicating an end of the second time period to the counter circuit to substantially equalize the respective lengths of the first time period and the second time period. (9)

[0214] The light detection device according to any one of (1) to (8), wherein the control circuit includes a timing generator. (10)

[0216] The light detection device according to any one of (1) to (9), further includes a signal processing circuit configured to generate a third signal based on a sum of the number of first signals in the first time period and the number of first signals in the second time period. (11)

[0218] The light detection device according to (10), wherein the signal processing circuit includes a bit inversion circuit configured to invert a bit value of the second signal to generate the third signal. (12)

[0220] The light detection device according to (10) or (11), wherein the signal processing circuit includes a memory configured to hold the third signal. (13)

[0222] An optical detection device according to any one of (10) to (12), wherein the signal processing circuit is configured to average multiple third signals based on the difference between the number of first signals in the first time period and the number of first signals in the second time period, and output the averaged third signal. (14)

[0224] The light detection device according to any one of (1) to (13), further includes: a plurality of pixels, wherein the plurality of pixels include at least one of pixels including a light receiving element configured to receive visible light and pixels including a light receiving element configured to receive invisible light. (15)

[0226] The light detection device according to any one of (1) to (14), wherein:

[0227] The counter circuit includes a first counter and a second counter each configured to count a first signal,

[0228] The first counter is configured to count the number of first signals within a first time period, and

[0229] The second counter is used to count the number of first signals in the second time period. (16)

[0231] The light detection device according to any one of (1) to (15), comprising:

[0232] A first substrate including a plurality of pixels; and

[0233] The second substrate includes a plurality of pixel circuits, and the first substrate is stacked on the second substrate. (17)

[0235] The light detection device according to any one of (1) to (16), wherein a pixel circuit among the plurality of pixel circuits is located directly below a pixel of the pixel array. (18)

[0237] A light detection device, comprising:

[0238] a pixel including a light receiving element; and

[0239] A pixel circuit, including a counter circuit and a control circuit,

[0240] Wherein, the light detection device is configured to detect an intensity signal and a motion signal. (19)

[0242] An electronic device, comprising:

[0243] a signal processor; and

[0244] A light detection device comprising:

[0245] a pixel including a light receiving element configured to receive light; and

[0246] A pixel circuit, comprising:

[0247] a counter circuit configured to receive a first signal based on an output of the light receiving element and output a second signal based on a difference between the number of the first signal in the first time period and the number of the first signal in the second time period; and

[0248] The control circuit is configured to control the counter circuit. (20)

[0250] A photodetector comprising:

[0251] a light receiving element configured to receive light and output a photocurrent;

[0252] a counter section including an input section receiving a first signal based on the photocurrent, the counter section being configured to output a second signal based on a difference between a number of pulses of the first signal in a first time period and a number of pulses of the first signal in a second time period; and

[0253] A control unit is configured to control the counter unit, wherein:

[0254] The counter section and the control section are provided for each pixel, and each pixel includes a light receiving element. (twenty one)

[0256] The photodetector according to (20), wherein the light receiving element includes a single photon avalanche diode. (twenty two)

[0258] The photodetector according to (20) or (21), wherein the counter section is configured to output a detection signal indicating that the number of pulses of the first signal in the first period has reached a reference value. (twenty three)

[0260] The photodetector of (22), wherein the detection signal comprises a signal indicating an end of the first time period. (twenty four)

[0262] The photodetector according to (22) or (23), wherein the counter section includes an up-and-down counter. (25)

[0264] The photodetector according to any one of (22) to (24), wherein the control section is configured to control the counter section based on the detection signal. (26)

[0266] The photodetector according to any one of (22) to (25), wherein the control section is configured to output a signal indicating an end of the second time period to the counter section based on the detection signal. (27)

[0268] The photodetector according to any one of (22) to (26), wherein the control section is configured to output a signal indicating the end of the second time period to the counter section to equalize the respective lengths of the first time period and the second time period. (28)

[0270] The photodetector according to any one of (22) to (27), wherein the control section includes a timing generator. (29)

[0272] The photodetector described in any one of (20) to (28) comprises a signal processing unit configured to generate a third signal based on a second signal output from a counter, wherein the third signal is based on the sum of the number of pulses of the first signal in a first time period and the number of pulses of the first signal in a second time period. (30)

[0274] The photodetector according to (29), wherein the signal processing unit includes a bit inversion section configured to invert a bit value of the second signal. (31)

[0276] The photodetector according to (29) or (30), wherein the signal processing unit includes a memory section configured to hold the third signal. (32)

[0278] A photodetector according to any one of (29) to (31), wherein the signal processing unit is configured to average a plurality of third signals based on a difference between the number of pulses of the first signal in a first time period and the number of pulses of the first signal in a second time period, and output an averaged third signal. (33)

[0280] The photodetector according to any one of (20) to (32), comprising a plurality of pixels each including a light receiving element, wherein

[0281] The plurality of pixels include at least one of a pixel including a light receiving element configured to receive visible light and output a photocurrent or a pixel including a light receiving element configured to receive invisible light and output a photocurrent. (34)

[0283] The photodetector according to any one of (20) to (33), wherein:

[0284] The counter section includes a first counter and a second counter, each of the first counter and the second counter being configured to count a first signal received by the input section,

[0285] The first counter is configured to count the number of pulses of the first signal in the first time period, and

[0286] The second counter is used to count the number of pulses of the first signal in the second time period. (35)

[0288] The photodetector according to any one of (20) to (34), comprising a generating section configured to generate a first signal based on the photocurrent. (36)

[0290] The photodetector according to (35), wherein the generating section is coupled to the input section and is configured to output the first signal to the input section. (37)

[0292] The photodetector according to (35) or (36), wherein the generating unit includes an inverter. (38)

[0294] The photodetector according to any one of (20) to (37), comprising:

[0295] a first substrate including a plurality of light receiving elements; and

[0296] The second substrate includes a facing portion and a control portion, and the second substrate is stacked on the first substrate.

[0297] It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.

[0298] [Reference Numbers List]

[0299] 1 Photodetector (light detection device)

[0300] 10Light receiving element

[0301] 20 generation unit (generation circuit)

[0302] 25 Supply unit (supply circuit)

[0303] 30 counter unit (counter circuit)

[0304] 35. Control unit (control circuit)

[0305] 40 determination unit (determination circuit)

[0306] 60-bit inversion unit (bit inversion circuit)

[0307] 70 Addition unit (addition circuit)

[0308] 75 Memory unit

[0309] 100 pixel unit

[0310] 110 Processor

[0311] 120 pixel circuit unit

[0312] 130 signal processing unit.

Claims

1. A light detection device, comprising: a pixel including a light receiving element configured to receive light; as well as A pixel circuit, comprising: a counter circuit configured to receive a first signal based on an output of the light receiving element and output a second signal based on a difference between the number of first signals in a first time period and the number of first signals in a second time period; as well as The control circuit is configured to control the counter circuit.

2. The light detection device according to claim 1, wherein: The light receiving element includes a single photon avalanche diode.

3. The light detection device according to claim 1, wherein: The counter circuit is configured to output a detection signal when the number of first signals in the first period reaches a reference value.

4. The light detection device according to claim 3, wherein: The detection signal indicates an end of the first time period.

5. The light detection device according to claim 3, wherein: The counter circuit includes an up-down counter.

6. The light detection device according to claim 3, wherein: The control circuit is configured to control the counter circuit based on the detection signal.

7. The light detection device according to claim 3, wherein: The control circuit is configured to output a stop signal indicating an end of the second time period to the counter circuit based on the detection signal.

8. The light detection device according to claim 7, wherein: The control circuit is configured to output the stop signal to the counter circuit indicating an end of the second time period to substantially equalize respective lengths of the first time period and the second time period.

9. The light detection device according to claim 6, wherein: The control circuit includes a timing generator.

10. The light detection device according to claim 1, further comprising: The signal processing circuit is configured to generate a third signal based on a sum of the number of first signals in the first time period and the number of first signals in the second time period.

11. The light detection device according to claim 10, wherein: The signal processing circuit includes a bit inversion circuit configured to invert a bit value of the second signal to generate the third signal.

12. The light detection device according to claim 10, wherein: The signal processing circuit includes a memory configured to hold the third signal.

13. The light detection device according to claim 10, wherein: The signal processing circuit is configured to average the plurality of third signals according to a difference between the number of first signals in the first time period and the number of first signals in the second time period, and output the averaged third signal.

14. The light detection device according to claim 1, further comprising: A plurality of pixels, wherein the plurality of pixels include at least one of a pixel including the light receiving element configured to receive visible light and a pixel including the light receiving element configured to receive invisible light.

15. The light detection device according to claim 1, wherein The counter circuit includes a first counter and a second counter each configured to count the first signal, The first counter is configured to count the number of first signals within the first time period, and The second counter is configured to count the number of first signals within the second time period.

16. The light detection device according to claim 1, comprising: A first substrate comprising a plurality of pixels; as well as The second substrate includes a plurality of the pixel circuits, and the first substrate is stacked on the second substrate.

17. The light detection device according to claim 1, wherein The pixel circuit of the plurality of pixel circuits is located directly below a pixel of the pixel array.

18. A light detection device, comprising: A pixel including a light receiving element; as well as A pixel circuit, including a counter circuit and a control circuit, Wherein, the light detection device is configured to detect intensity signals and motion signals.

19. An electronic device comprising: Signal processor; as well as A light detection device comprising: a pixel including a light receiving element configured to receive light; and A pixel circuit, comprising: a counter circuit configured to receive a first signal based on an output of the light receiving element and output a second signal based on a difference between the number of first signals in a first time period and the number of first signals in a second time period; and The control circuit is configured to control the counter circuit.

Images