Light detector and light detection system

Abstract

There is provided a light detection device including: a first pixel; a first circuit; a second pixel; and a second circuit. The first pixel includes a first light receiving element configured to receive light. A first circuit is coupled with the first light receiving element, and the first circuit is configured to output a first signal related to motion of the target and a second signal related to the intensity value. The second pixel includes a second light receiving element configured to receive light. And a second circuit coupled with the second light receiving element, and configured to output a third signal related to a distance from the target.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from Japanese Priority Patent Application JP 2022-096652 filed on Jun. 15, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a light detector and a light detection system. Background Art

[0004] Patent Document 1 proposes a device that includes a plurality of pixels and detects light. Each of the plurality of pixels includes a single photon avalanche diode (SPAD) element and a counter circuit.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: JP 2019-161551 Summary of the invention

[0008] Technical issues

[0009] There is a need for light detection devices with improved detection performance.

[0010] It is desirable to provide a light detection device having good detection performance.

[0011] Solution to the problem

[0012] According to an embodiment of the present disclosure, a light detection device includes: a first pixel; a first circuit; a second pixel; and a second circuit. The first pixel includes a first light receiving element configured to receive light. The first circuit is coupled to the first light receiving element, and the first circuit is configured to output a first signal related to the motion of the target and a second signal related to the intensity value. The second pixel includes a second light receiving element configured to receive light. The second circuit is coupled to the second light receiving element, and the second circuit is configured to output a third signal related to the distance from the target.

[0013] According to an embodiment of the present disclosure, a light detection system includes: a light source; a light detection device; and a light source controller. The light source controller is configured to control the light source based on a signal output from the light detection device. The light detection device includes a first pixel, a first circuit, a second pixel, and a second circuit. The first pixel includes a first light receiving element configured to receive light. The first circuit is coupled to the first light receiving element, and the first circuit is configured to output a first signal related to the movement of the target and a second signal related to the intensity value. The second pixel includes a second light receiving element configured to receive light. The second circuit is coupled to the second light receiving element, and the second circuit is configured to output a third signal related to the distance from the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram showing an example of a schematic configuration of a light detection system according to an embodiment of the present invention.

[0015] Figure 2 is a diagram for describing a configuration example of a photodetector according to an embodiment of the present disclosure.

[0016] Figure 3A is a diagram showing a configuration example of a photodetector according to an embodiment of the present disclosure.

[0017] Figure 3B is a diagram showing a configuration example of a photodetector according to an embodiment of the present disclosure.

[0018] Figure 4A This is a diagram for explaining an example of control of a light source by the light detection system according to the embodiment of the present invention.

[0019] Figure 4B This is a diagram for explaining an example of control of a light source by the light detection system according to the embodiment of the present invention.

[0020] Figure 5 It is a timing chart showing an operation example of the light detection system according to the embodiment of the present invention.

[0021] Fig. 6A This is a flowchart showing an operation example of the light detection system according to the embodiment of the present invention.

[0022] Figure 6B is a flow chart illustrating an example of the operation of a light detection system according to an embodiment of the present disclosure.

[0023] Figure 7 is a schematic diagram showing an example of a cross-sectional configuration of a photodetector according to an embodiment of the present disclosure.

[0024] Figure 8 : is a diagram showing a configuration example of a photodetector according to Modification 1 of the present disclosure.

[0025] Fig.9A : is a diagram showing a configuration example of a photodetector according to Modification 2 of the present disclosure.

[0026] Fig. 9B : is a diagram showing another configuration example of a photodetector according to Modification 2 of the present disclosure.

[0027] Fig.10 : is a diagram showing an example of arrangement of pixels of a photodetector according to Modification 2 of the present disclosure.

[0028] Fig.11 : is a diagram showing another arrangement example of pixels of a photodetector according to Modification 2 of the present disclosure.

[0029] Fig.12 : is a diagram showing another arrangement example of pixels of a photodetector according to Modification 2 of the present disclosure.

[0030] Fig.13 : is a diagram showing another arrangement example of pixels of a photodetector according to Modification 2 of the present disclosure.

[0031] Fig.14 : is a diagram showing another arrangement example of pixels of a photodetector according to Modification 2 of the present disclosure.

[0032] Fig.15 It is a timing chart showing an operation example of the light detection system according to the third modification of the present invention.

[0033] Fig.16 : is a diagram showing a configuration example of a photodetector according to Modification 4 of the present disclosure.

[0034] Fig.17 This is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0035] Fig.18 It is a diagram for assisting in explaining an example of the installation positions of the vehicle exterior information detection unit and the imaging unit.

[0036] Fig.19 This is a diagram showing an example of a schematic configuration of an endoscopic surgery system.

[0037] Fig. 20 is a block diagram depicting an embodiment of a functional configuration of a camera head and a camera control unit (CCU). DETAILED DESCRIPTION

[0038] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be noted that the description is made in the following order.

[0039] 1. Implementation Method

[0040] 2. Modifications

[0041] 3. Use Example

[0042] 4. Practical application examples

[0043] <1. Implementation Method>

[0044] Figure 1 2 is a diagram showing an example of a schematic structure of a light detection system according to an embodiment of the present invention. The light detection system 200 includes a light detector 1, a light source controller 210, and a light source 220. The light detector 1 is a device (i.e., a light detection device) configured to detect incident light. The light detector 1 includes a plurality of pixels P. Each of the plurality of pixels P includes a light receiving element. The light detector 1 is configured to photoelectrically convert incident light and generate a signal.

[0045] The light detector 1 according to the present embodiment is a sensor configured to detect an event. The light detector 1 can be applied as an event-driven sensor (referred to as an event vision sensor (EVS), an event-driven sensor (EDS), a dynamic vision sensor (DVS), etc.). In addition, the light detector 1 is also a distance measuring sensor configured to measure a distance in a time-of-flight (TOF) method.

[0046] exist Figure 1 In the illustrated embodiment, the photodetector 1 includes a region (pixel unit 100) in which a plurality of pixels P are two-dimensionally arranged in a matrix. The light receiving element (light receiving portion) of each pixel P is, for example, a SPAD element. The photodetector 1 acquires incident light (image light) from a measurement target through an optical system (not shown) including an optical lens. The light receiving element can receive light, generate charge through photoelectric conversion, and generate photocurrent.

[0047] The photodetector 1 includes a processor 110 configured to perform signal processing. The processor 110 is a signal processing circuit. The processor 110 performs signal processing (information processing). The processor 110 performs various signal processing on the signal of each pixel. The processor 110 outputs the signal of the pixel subjected to signal processing. Although described below, the processor 110 can generate and output a signal related to the motion of the measurement target, a signal related to the intensity value, and a signal related to the distance to the measurement target. The intensity value (e.g., the intensity value associated with the pixel, such as an R pixel, a G pixel, or a B pixel) may also be referred to as a gray value in this article.

[0048] The processor 110 is also a controller. The processor 110 is configured to control each unit of the light detector 1. The processor 110 includes a plurality of circuits, including, for example, a timing generator, a shift register, an address decoder, etc. The timing generator generates various timing signals. The processor 110 may provide a signal for driving the pixel P for each pixel P and control the operation of the pixel P. In addition, the processor 110 may provide a signal for controlling the light source controller 210 to control the operation of the light source controller 210.

[0049] The light source 220 includes a plurality of light emitting elements. The light emitting elements are, for example, light emitting diodes (LEDs) or laser diodes. In the light source 220, the plurality of light emitting elements are two-dimensionally arranged in a matrix. For example, the light source 220 may generate laser light and emit the laser light to the outside. The light source 220 may include a vertical cavity surface emitting laser (VCSEL).

[0050] The light source controller 210 is used to drive the light source 220. The light source controller 210 is a driver (driving circuit). The light source controller 210 can control the operation of the light source 220. The light source controller 210 is configured to control, for example, the current and voltage of the light emitting element in the light source 220. The light source controller 210 includes a plurality of circuits, including, for example, a digital-to-analog converter (DAC), an amplifier circuit, and the like.

[0051] The light source controller 210 may provide the light source 220 with current and voltage for driving each light emitting element in the light source 220 to control light emission of the light source 220. The light source controller 210 is a driver configured to control driving of the light emitting elements in the light source 220. The light source controller 210 may be referred to as a driver IC (driver circuit).

[0052] Figure 2 1 is a diagram for describing an example of a configuration of a light detector according to the present embodiment. Figure 2 As shown, the optical detector 1 includes a first substrate 101, a second substrate 102, and a third substrate 103. The first substrate 101, the second substrate 102, and the third substrate 103 are stacked on top of each other. The first substrate 101 is stacked on top of the second substrate 102. The second substrate 102 is stacked on top of the third substrate 103.

[0053] The photodetector 1 has a structure (stacked structure) in which a first substrate 101, a second substrate 102, and a third substrate 103 are stacked in the Z-axis direction. Figure 2 As shown in the figure, the incident direction of the light from the object to be measured is defined as the Z-axis direction, the left-right direction perpendicular to the Z-axis direction is defined as the X-axis direction, and the direction perpendicular to the Z-axis direction and the X-axis direction is defined as the Y-axis direction. Figure 2 The direction of the arrow in indicates the direction.

[0054] A pixel unit 100 is provided on the first substrate 101. In the pixel unit 100, a plurality of pixels P are arranged in a horizontal direction (row direction) and a vertical direction (column direction), the horizontal direction being a first direction, and the vertical direction being a second direction orthogonal to the first direction. Each pixel P in the pixel unit 100 includes a filter 15. The filter 15 is configured to selectively transmit light within a specific wavelength range in a plurality of incident lights. The filter 15 is an RGB color filter, a filter that transmits infrared light, or the like.

[0055] The plurality of pixels P in the pixel unit 100 include a plurality of pixels P1 and a plurality of pixels P2. The pixel P1 is a pixel having a light receiving element that receives visible light and outputs a photocurrent. Each of the pixels P2 is a pixel that includes a light receiving element configured to receive invisible light (also referred to as invisible light) and output a photocurrent. In one example, the plurality of pixels P1 include: a pixel (R pixel) including a filter 15 that transmits light in a red wavelength range; a pixel (G pixel) including a filter 15 that transmits light in a green wavelength range; and a pixel (B pixel) including a filter 15 that transmits light in a blue wavelength range. The R pixel, the G pixel, and the B pixel are arranged, for example, according to a so-called Bayer arrangement.

[0056] The R pixel, the G pixel, and the B pixel are configured to respectively generate a signal of an R component, a signal of a G component, and a signal of a B component. RGB pixel signals can be obtained based on charges generated by photoelectric conversion of the R pixel, the G pixel, and the B pixel.

[0057] Pixel P2 is, for example, a pixel (IR pixel) including a filter 15 that transmits infrared light. Pixel P2 is configured to generate a signal of an IR component. Pixel P2 can be used to measure distance in the TOF method. In the light detection system 200, the light source 220 irradiates light (e.g., infrared light) to a subject as a measurement object. The light receiving element of the pixel P2 of the light detector 1 receives light reflected from the object. The signal of the pixel P2 is a signal corresponding to the distance to the object as the measurement object. The light detector 1 is configured to obtain a distance signal of each pixel P2.

[0058] It should be noted that the filter provided in the pixel P1 in the pixel unit 100 is not limited to the filter for the primary colors (RGB). The pixel P1 may include, for example, a color filter for complementary colors such as cyan (Cy), magenta (Mg), and yellow (Ye). In addition, it is also possible to avoid providing the filter 15 in a part or all of the pixels of the light detector 1.

[0059] A processor 110 is disposed on the second substrate 102 and the third substrate 103. The processor 110 is separately disposed in the second substrate 102 and the third substrate 103. The processor 110 includes a first circuit unit 140a and a second circuit unit 140b. The first circuit unit 140a includes a pixel circuit unit 120a and a signal processing unit 130a. The second circuit unit 140b includes a pixel circuit unit 120b and a signal processing unit 130b. Figure 2 In the example shown in FIG. 1 , the pixel circuit section 120a and the pixel circuit section 120b are provided in the second substrate 102. Figure 2 As shown, the pixel circuit section 120a is provided for the pixel P1. The pixel circuit section 120b is provided for the pixel P2. The signal processing section 130a and the signal processing section 130b each include a logic circuit, a memory, etc. The signal processing section 130a and the signal processing section 130b are arranged in the third substrate 103.

[0060] Figure 3A and Figure 3B are diagrams each showing a configuration example of a photodetector according to an embodiment. Figure 3A A configuration example of the pixel P1 and the first circuit unit 140 a is shown. Figure 3B FIG. 4 shows an embodiment of the configuration of the pixel P2 and the second circuit unit 140 b. Figure 3A As shown, the photodetector 1 includes a pixel P1 and a first circuit unit 140a. The pixel P1 includes a light receiving element 10a. The first circuit unit 140a includes a pixel circuit section 120a and a signal processing section 130a. Figure 3A In the illustrated embodiment, the pixel circuit section 120a includes a generating section (also referred to as a generating circuit) 20a, a supply section (also referred to as a supply circuit) 25a, a counter section (also referred to as a counting circuit) 30a, a control section (also referred to as a control circuit) 35, and a determining section (also referred to as a determining circuit) 40. The pixel circuit section 120a is provided for each pixel P1.

[0061] The light receiving element 10a is configured to receive light and generate a signal. The light receiving element 10a is a SPAD element. The light receiving element 10a can convert incident photons into electric charges and output a signal S1a, which is an electrical signal corresponding to the incident photons. It should be noted that the light receiving element 10a is also referred to as a photoelectric conversion element (photoelectric conversion unit) configured to photoelectrically convert light.

[0062] The light receiving element 10a is electrically coupled to, for example, a power supply line, an electrode, or the like that allows a predetermined voltage to be supplied. Figure 3A In the embodiment shown, the cathode as one of the electrodes of the light receiving element 10a is electrically coupled to the power line through the supply portion 25a. The power supply voltage is provided through the power line. The anode as the other electrode of the light receiving element 10a is electrically coupled to the ground line side or a large negative voltage source.

[0063] The voltage supplied by the supply section 25a may cause a voltage serving as a potential difference greater than the breakdown voltage of the light receiving element 10a to be applied between the cathode and anode of the light receiving element 10a. In other words, the potential difference between the two ends of the light receiving element 10a may be set to a potential difference greater than the breakdown voltage. In the case of applying a reverse bias greater than the breakdown voltage to the light receiving element 10a, the light receiving element 10a may operate in the Geiger mode. In the light receiving element 10a in the Geiger mode, an avalanche multiplication phenomenon may occur in response to the incidence of photons and a pulse current may be generated. In the pixel P1, a signal S1a corresponding to the photocurrent flowing through the light receiving element 10a in response to the incidence of photons is output to the generating section 20a.

[0064] The generating section 20a is configured to generate a signal (voltage signal) S2a based on the signal S1a generated by the light receiving element 10a. Figure 3A In the embodiment shown, the generating section 20a includes an inverter. The generating section 20a includes a PMOS transistor and an NMOS transistor coupled in series. The input section of the generating section 20a is electrically coupled to the cathode of the light receiving element 10a and the supply section 25a. The output section of the generating section 20a is electrically coupled to the input section 31 of the relative component 30a.

[0065] The generating unit 20a receives the signal S1a from the light receiving element 10a. The signal level of the signal S1a changes according to the current flowing through the light receiving element 10a. In other words, the voltage (potential) of the signal S1a changes according to the current flowing through the light receiving element 10a. For example, when the voltage of the signal S1a is higher than the threshold value, the generating unit 20a outputs a low-level signal S2a. In addition, when the voltage of the signal S1a is less than the threshold value, the generating unit 20a outputs a high-level signal S2a. The generating unit 20a can output the signal S2a used as a pulse signal to the counter unit 30a based on the voltage of the signal S1a.

[0066] exist Figure 3A In the embodiment shown in , when the light receiving element 10a receives a photon and this causes the signal S1a to have a voltage less than the threshold voltage of the inverter used as the generating section 20a, the inverter causes the voltage of the signal S2a to change from a low level to a high level. It should be noted that the generating section 20a may include a buffer circuit, an AND circuit, or the like.

[0067] The supply section 25a is configured to supply voltage and current to the light receiving element 10a. The supply section 25a is electrically coupled to a power line through which a power supply voltage is supplied. The supply section 25a can supply voltage and current to the light receiving element 10a. Figure 3AIn the example shown, the supply section 25a includes a PMOS transistor. It should be noted that the supply section 25a may include a resistor.

[0068] In the case where avalanche multiplication occurs so that the potential difference between the electrodes of the light receiving element 10a is less than the breakdown voltage, the supply unit 25a can supply current to the light receiving element 10a. The supply unit 25a recharges the light receiving element 10a to allow the light receiving element 10a to operate in the Geiger mode again. The supply unit 25a is a recharging unit. In other words, the supply unit 25a recharges the light receiving element 10a with the charge and recharges the voltage of the light receiving element 10a.

[0069] The counter unit 30a is configured to perform counting based on the received signal. The counter unit 30a has an input unit 31 (input terminal). The counter unit 30a counts the pulses of the signal received by the input unit 31. Figure 3A In the embodiment shown, the input section 31 of the counter section 30a receives the signal S2a as a pulse signal. The counter section 30a is configured to count the pulses of the signal S2a and output a signal based on the difference in the number of pulses of the signal S2a between two cycles (also referred to as the number of the signal S2a). The counter section 30a is provided to each pixel P1. The counter section 30a has, for example, an up-down counter.

[0070] The control unit 35 controls the counter unit 30a. The control unit 35 has a timing generator. The control unit 35 is provided for each pixel P1. The control unit 35 is a timing control unit. The control unit 35 generates a timing signal based on a clock signal or a synchronization signal received from the outside, for example, and controls the operation of the counter unit 30a.

[0071] The control section 35 outputs a signal (start signal) for indicating (requesting) to start counting to the counter section 30a, and controls the timing at which the counter section 30a starts counting. In addition, the control section 35 outputs a signal (stop signal) for indicating the end of counting to the counter section 30a, and controls the timing at which the counter section 30a ends counting. The control section 35 can also control the supply section 25a. It should be noted that the supply section 25a can be controlled by a control circuit different from the control section 35.

[0072] exist Figure 3A In the illustrated embodiment, the control unit 35 receives an overflow signal from the counter unit 30a as a signal (detection signal) indicating that the count value of the counter unit 30a has reached a reference value. The overflow signal is a signal indicating an overflow of the count value. In other words, the overflow signal is a signal indicating an overflow of the amount of light received by the light receiving element 10a. The control unit 35 provides a stop signal generated based on the overflow signal to the counter unit 30a to control the counter unit 30a.

[0073] In one embodiment, when a start signal is input from the control section 35 to the counter section 30a, the counter section 30a starts counting in the first period Ta. The counter section 30a counts up the number of pulses of the signal S2a in the first period Ta. When the count value of the counter section 30a reaches the reference value, the counter section 30a outputs an overflow signal indicating that the count value reaches the reference value to the control section 35, and starts counting in the second period Tb. The counter section 30a counts down the number of pulses of the signal S2a in the second period Tb using the reference value as an initial value. The overflow signal as a detection signal is a signal indicating the end of the first period Ta and the beginning of the second period Tb.

[0074] The control unit 35 generates a stop signal according to the overflow signal received from the counter unit 30a. The control unit 35 outputs the stop signal to the counter unit 30a. The control unit 35 outputs the stop signal indicating the end of the second period Tb to the counter unit 30a so that the respective lengths of the first period Ta and the second period Tb are equal. When the stop signal is input to the counter unit 30a from the control unit 35, the counter unit 30a ends counting in the second period Tb.

[0075] The counter section 30a generates and outputs a signal (difference signal S3) based on the difference between the number of pulses of the signal S2a in the first period Ta and the number of pulses of the signal S2a in the second period Tb. The counter section 30a may output a signal indicating a count value corresponding to the difference between the number of pulses of the signal S2a in the first period Ta and the number of pulses of the signal S2a in the second period Tb as the difference signal S3. For example, the signal value or count value of the difference signal S3 is a value obtained by subtracting the count value of down counting from the count value of up counting.

[0076] The determination section 40 is configured to determine the magnitude of the difference in the number of pulses based on the difference signal S3. For example, the determination section 40 is configured to determine whether the signal value of the difference signal S3 is greater than a predetermined threshold value. Figure 3A In the illustrated embodiment, the determination section 40 includes a first threshold determination section 41 and a second threshold determination section 42 .

[0077] The first threshold determination section 41 compares the value of the difference signal S3 with the first threshold. The second threshold determination section 42 compares the value of the difference signal S3 with the second threshold. For example, the first threshold determination section 41 is configured to determine whether the signal value of the difference signal S3 or the value obtained by subtracting the count value in the second period Tb from the count value in the first period Ta is less than the first threshold. The second threshold determination section 42 determines whether the signal value of the differential signal S3 is greater than the second threshold.

[0078] In the case where the signal value of the difference signal S3 drops below the first threshold value, the first threshold value determination section 41 determines that the motion of the object used as the measurement target causes a positive event to occur. In the case where the signal value of the difference signal S3 is not lower than the first threshold value, the first threshold value determination section 41 determines that a positive event has not occurred. In the case where the signal value of the difference signal S3 exceeds the second threshold value, the second threshold value determination section 42 determines that a negative event has occurred due to the motion of the measurement target. In the case where the signal value of the difference signal S3 does not exceed the second threshold value, the second threshold value determination section 42 determines that a negative event has not occurred.

[0079] In this way, the determination section 40 can detect the presence or absence of the occurrence of an event based on the difference signal S3. When the signal value of the difference signal S3 is less than the first threshold value or greater than the second threshold value, the determination section 40 determines that the event has occurred. In other words, when the amount of received light changed due to the movement of the measurement object causes the amount of change in the count value to exceed the upper limit or lower limit threshold value, the determination section 40 determines that the event is "present".

[0080] The determination section 40 generates a signal (motion signal) related to the motion of the object to be measured, and outputs the signal, based on the determination results of the first threshold determination section 41 and the second threshold determination section 42. The determination section 40 outputs a signal value (count value) indicating the difference signal S3 and a signal indicating the presence or absence of an event occurrence, for example, as a motion signal S11 to the signal processing section 130a. The motion signal S11 may include a signal indicating the presence or absence of the occurrence of a positive event and a negative event.

[0081] The signal processing section 130a is configured to acquire the difference signal S3 and the motion signal S11 of each pixel P1 and perform signal processing. The signal processing section 130a is a signal processing circuit. The signal processing section 130a can perform various signal processing by using the difference signal S3 and the motion signal S11. Figure 3A In the illustrated embodiment, the signal processing section 130 a includes a bit inversion section 60 , an addition section 70 a , and a storage section 75 a .

[0082] The bit inversion section 60 is configured to invert the bit value of the received signal. Figure 3A In the example shown, the bit inversion section 60 receives the difference signal S3 from the counter section 30a. The bit inversion section 60 inverts the bit value of the difference signal S3. The bit inversion section 60 generates a signal (grayscale signal) based on the sum of the number of pulses of the signal S2a in the first period Ta and the number of pulses of the signal S2a in the second period Tb.

[0083] The grayscale signal S12 has a signal value obtained by, for example, adding the count value of the second period Tb to the count value of the first period Ta. The grayscale signal S12 is a signal indicating grayscale. In other words, the bit inversion unit 60 restores the grayscale signal S12 indicating the grayscale value of the pixel by using the difference signal S3. The bit inversion unit 60 performs an inversion process on the difference signal S3 of each pixel P1 and generates a grayscale signal S12 of the pixel P1.

[0084] The adding section 70a and the memory section 75a receive the grayscale signal S12 of each pixel P1 from the bit inversion section 60. In addition, the adding section 70a and the memory section 75a receive the motion signal S11 of each pixel P1 from the determination section 40. The storage section 75a is used to hold the signal of each pixel. The storage section 75a is a frame memory. The memory section 75a can store (record) the grayscale signal S12 and the motion signal S11 of each pixel in units of frames.

[0085] The adding section 70a is configured to perform a process of adding signals of pixels. The adding section 70a estimates the moving direction of the moving object by using, for example, the motion signal S11 of each pixel P1. The adding section 70a aligns the grayscale signal S12 of each pixel P1 with reference to the grayscale signal S12 held in the storage section 75a based on the result of the estimation. The adding section 70a performs a process of adding and averaging a plurality of grayscale signals S12.

[0086] The adding section 70a is an averaging section. The adding section 70a averages the plurality of grayscale signals S12 as described above and generates an average grayscale signal S12. By integrating the plurality of grayscale signals S12, the S / N ratio of the grayscale signal S12 can be improved. In this way, the signal processing section 130a can obtain the motion signal S11, the grayscale signal S12, and the average grayscale signal S12 and output the motion signal S11, the grayscale signal S12, and the average grayscale signal S12 to the outside of the light detector 1.

[0087] Figure 3B Pixel P2 and second circuit unit 140b are shown. Figure 3B As shown, the photodetector 1 includes a pixel P2 and a second circuit unit 140b. The pixel P2 includes a light receiving element 10b. The second circuit unit 140b includes a pixel circuit section 120b and a signal processing section 130b. Figure 3B In the illustrated embodiment, the pixel circuit section 120b includes a generating section 20b, a supply section 25b, a selecting section 27 and a counter section 30b. For example, each pixel P2 or a plurality of pixels P2 is provided with a plurality of selecting sections 27 and a plurality of counter sections 30b.

[0088] The light receiving element 10b, the generating unit 20b and the supply unit 25b have a configuration similar to the light receiving element 10b, the generating unit 20a and the supply unit 25a of the above-mentioned pixel P1. The light receiving element 10b is configured to receive light and generate a signal. The light receiving element 10b is a SPAD element. The light receiving element 10b can convert incident photons into electric charges and output a signal S1b, which is an electrical signal corresponding to the incident photons. It should be noted that the light receiving element 10b is also referred to as a photoelectric conversion element (photoelectric conversion unit) configured to photoelectrically convert light.

[0089] The generating section 20b is configured to generate a signal S2b based on the signal S1b generated by the light receiving element 10b. Figure 3B In the embodiment shown, the generating unit 20b includes an inverter. The generating unit 20b can output the signal S2b to the selecting unit 27. The signal S2b is a pulse signal based on the voltage of the signal S1b generated by the light receiving element 10b. It should be noted that the generating unit 20b may include a buffer circuit, an AND circuit, etc. The supply unit 25b is a recharging component. The supply unit 25b is configured to supply voltage and current to the light receiving element 10b.

[0090] The selection section 27 is configured to output the signal S2b from the generation section 20b to the counter section 30b selected from the plurality of counter sections 30b. The selection section 27 is a selection circuit (selector). The selection section 27 is configured to switch the output destination of the signal S2b. In other words, the selection section 27 distributes the signal S2b received from the plurality of generation sections 20b to the corresponding counter section 30b. The selection section 27 can change the output destination of the signal S2b in a time-division manner (e.g., for each subframe).

[0091] The counter section 30b is configured to perform counting according to the received signal. The counter section 30b receives the signal S2b from the generating section 20b through the selecting section 27. The signal S2b is a pulse signal. The counter section 30b counts the pulses of the signal S2b and outputs a signal based on the number of pulses of the signal S2b. For example, the counter section 30b counts the number of pulses of the signal S2b upward as a count value and outputs a signal S4 representing the count value to the signal processing section 130b.

[0092] The signal processing unit 130b is configured to acquire the signal S4 of each pixel P2 and perform signal processing. The signal processing unit 130b is a signal processing circuit. The signal processing unit 130b can perform various signal processing by using the signal S4. Figure 3B In the illustrated example, the signal processing unit 130 b includes a calculation unit 65 , an adding unit 70 b , and a storage unit 75 b .

[0093] The calculation unit 65 is configured to perform signal processing on the signal S4. The calculation unit 65 generates a signal (distance signal) related to the distance to the measurement object according to the signal S4. In one embodiment, when the light source 220 irradiates light (e.g., laser) to the measurement object, each pixel P2 of the light detector 1 receives light reflected from the measurement object. The signal S4 generated for each pixel P2 is a signal corresponding to the distance to the measurement object.

[0094] The calculation unit 65 estimates the phase difference between the irradiated light and the reflected light or the round trip time of the light, for example, based on the signal S4 of the pixel P2 generated. The calculation unit 65 calculates the distance between the light detector 1 and the object to be measured. The operation unit 65 calculates the distance to the object to be measured based on the time when the light emitted from the light source 220 is reflected by the object to be measured and reaches the light detector 1. The calculation unit 65 calculates the distance to the target for each pixel P2. The calculation unit 65 generates a distance signal S13 related to the distance to the target. In this case, the calculation unit 65 can perform noise reduction processing (for example, processing such as removing the signal component of the background light) and generate the distance signal S13 for each pixel P2.

[0095] The adding section 70b and the storage section 75b receive the distance signal S13 of each pixel P2 from the calculating section 65. In addition, the adding section 70b and the storage section 75b receive the distance signal S13 of each pixel P2 from the signal processing section 130a (see FIG. 140a ) of the first circuit unit 140a. Figure 3A ) receives the motion signal S11 of each pixel P1. The storage unit 75b is configured to hold the signal of each pixel. The storage unit 75b is a frame memory. The storage unit 75b can store the distance signal S13 of each pixel in units of frames, for example.

[0096] The adding unit 70b performs a signal addition process of the pixels. The adding unit 70b estimates the moving direction of the moving object using, for example, the motion signal S11 of each pixel P1. The adding unit 70b makes the distance signal S13 of each pixel P2 consistent with each other by referring to the distance signal S13 stored in the storage unit 75b based on the estimation result. The adding unit 70b performs an addition and average process of a plurality of distance signals S13.

[0097] The adding section 70b is an averaging section. The adding section 70b averages the plurality of distance signals S13 as described above to generate the averaged distance signal S13. By integrating the plurality of distance signals S13, the S / N ratio of the distance signal S13 can be improved. In this way, the signal processing section 130b can obtain the distance signal S13 and the average distance signal S13 and output the distance signal S13 and the average distance signal S13 to the outside of the photodetector 1.

[0098] In addition, in this embodiment, if Figure 3AAs shown, the first circuit unit 140a of the photodetector 1 includes a region detection unit 81, a distance detection unit 82, and a signal control unit 90. The region detection unit 81 detects a partial region of the region of the measurement object as the region of interest. For example, the region detection unit 81 detects a region including a moving target as the region of interest based on the motion signal S11 of each pixel P1. The region detection unit 81 generates a signal (region signal S15) related to the region of interest. The region detection unit 81 can output the generated region signal S15 to the signal processing unit 130b of the second circuit unit 140b (see Figure 3B ). The region signal S15 is a signal indicating a region of interest (RoI).

[0099] The signal control unit 90 is a signal control circuit. The signal control unit 90 is configured to perform signal processing. The signal control unit 90 is configured to control the operation of the light source controller 210 (see Figure 1 The signal control section 90 generates a signal related to, for example, the light emission position, light density, output power, etc. of the light source 220 based on the area signal S15 output from the area detection section 81 , and outputs the generated signal to the light source controller 210 .

[0100] The signal control section 90 is configured to control the light emission of the light source 220 by providing a control signal corresponding to the area signal S15 to the light source controller 210, and to make the light source 220 emit light for the area of ​​interest. The signal control section 90 can control the light emission position, light density, output power, etc. of the light source 220 according to the area signal S15. This makes it possible to perform accurate measurement on the area of ​​interest.

[0101] For example, in the case where the determination section 40 detects the occurrence of an event, the region detection section 81 determines the region including the region where the event occurs as the region of interest by using the motion signal S11. The region detection section 81 generates a region signal S15 indicating the region of interest. The signal control section 90 outputs a control signal to the light source controller 210 according to the region signal S15. The signal control section 90 adjusts the light emission position, light density, output power, etc. of the light source 220 to guide the amount of light for measurement to the region of interest. The light detector 1 detects light and can acquire a grayscale signal S12, a motion signal S11, and a distance signal S13 of the region of interest.

[0102] After the light source 220 emits light according to the area signal S15, the calculation section 65 of the signal processing section 130b generates a distance signal S13 for each pixel P2 by using the signal S4 of the pixel P2 according to the area signal S15 received from the signal processing section 130a. In this case, the calculation section 65 may generate a distance signal S13 for each pixel P2 that receives the reflected light from the area of ​​interest indicated by the area signal S15. The calculation section 65 may output the generated distance signal S13 to the distance detection section 82 of the signal processing section 130a. The distance detection section 82 receives the distance signal S13 for each of the plurality of pixels P2 corresponding to the area of ​​interest from the calculation section 65 of the signal processing section 130b.

[0103] The distance detection section 82 is configured to detect the distance to the region of interest based on the distance signal S13 of each pixel P2. The distance detection section 82 extracts the distance to the region of interest by using the distance signal S13 of each pixel P2 generated by the calculation section 65. The distance detection section 82 generates a signal (distance signal S16) related to the extracted distance to the region of interest. The distance detection section 82 may output the generated distance signal S16 to the signal control section 90 or the like.

[0104] The signal control section 90 generates a control signal related to the light emission position, light density, output power, etc. of the light source 220 based on the distance signal S16 output from the distance detection section 82, and outputs the generated control signal to the light source controller 210. The signal control section 90 may supply the control signal corresponding to the distance signal S16 to the light source controller 210. The light source controller 210 may change the light emission position, light density, etc. of the light source 220 so that the light source 220 performs light emission suitable for measuring the region of interest. This allows the light detector 1 to accurately perform measurement of the region of interest.

[0105] The signal control unit 90 may change the control of the light source 220 through the light source controller 210 according to the distance to the area of ​​interest indicated by the distance signal S16. Figure 4A As shown in , when the distance from the light source 220 (or the light detector 1) to the area of ​​interest is short, the signal control unit 90 can control the light source controller 210 and the light source 220 to achieve a wide field of view (FOV), low light density and low output power as a short distance setting.

[0106] In addition, if Figure 4A As shown in , in the case where the distance from the light source 220 to the region of interest is long, the signal control unit 90 can control the light source controller 210 and the light source 220 to achieve a narrow FOV, a high light density, and a high output power as a long distance setting. This makes it possible to accurately measure the distance to the region of interest even in the case of a long distance. It should be noted that in the case where the signal control unit 90 turns on the short distance setting or the long distance setting, as Figure 4BAs schematically shown in FIG. 1 , the signal control unit 90 can gradually change the output power (laser output intensity). In addition, the signal control unit 90 can gradually change the light density, FOV, etc. Thus, the occurrence of erroneous detection of the distance can be effectively suppressed.

[0107] Figure 5 1 is a timing chart showing an operation example of the light detection system according to the present embodiment. Figure 5 The synchronization signal, the count value of the counter section 30a, the overflow signal, the start / stop signal, the motion signal S11, the area signal S15, etc. on the same time axis are schematically shown. For example, the synchronization signal is generated based on the imaging frame rate. The synchronization signal indicates the time interval of the subframe in the main frame. The control section 35 of the first circuit unit 140a generates a start signal in synchronization with the synchronization signal. The control section 35 outputs the start signal to the counter section 30a.

[0108] Figure 5 The example of the case where the counter section 30a is an 8-bit up-down counter circuit is shown. The time period from time t1 to time t2, the time period from time t7 to time t8, and the time period from time t13 to time t14 are the first time period Ta ( Figure 5 In each of these periods, the counter section 30a counts up the number of pulses of the signal S2a.

[0109] The time period from time t2 to time t3, the time period from time t8 to time t9, and the time period from time t14 to time t15 are the above-mentioned second time period Tb ( Figure 5 In each of these periods, the counter section 30a counts down the number of pulses of the count signal S2a. Figure 5 In the embodiment, the first period Ta1 and the second period Tb1 have the same time interval. In addition, the first period Ta2 and the second period Tb2 have the same time interval. The first period Tan and the second period Tbn also have the same time interval.

[0110] At time t1 in the first subframe, the counter section 30a starts counting the pulses of the signal S2a according to the start signal received from the control section 35. In the case where the count value reaches "255" (which is a reference value) at time t2, the counter section 30a outputs an overflow signal to the control section 35. In addition, "255" is set as the initial value of the count down. The overflow signal is used as a signal indicating the timing of the end of the first period Ta1 and the start of the second period Tb1.

[0111] At time t2, the counter section 30a starts counting down the pulses of the signal S2a. The control section 35 learns the length of the first period Ta1 based on the start signal and the overflow signal. The control section 35 generates a stop signal and provides the stop signal to the counter section 30a to make the length of the second period Tb1 equal to the length of the first period Ta1.

[0112] At time t3, the counter section 30a ends the countdown of the pulses of the signal S2a according to the stop signal received from the control section 35. The counter section 30a outputs a difference signal S3 indicating a count value "|100" which is a difference between a count value obtained by counting up the pulses in the first period Ta1 and a count value obtained by counting down the pulses in the second period Tb1.

[0113] In the period from time t4 to time t5, the determination section 40 compares "|100" and "}40". "·|100" is the value of the difference signal S3. "·}40" is a threshold value. The first threshold determination section 41 determines that the signal value "|100" of the difference signal S3 is lower than the first threshold value "|40", and the detection of the movement of the measurement object causes a positive event to occur. It is to be noted that the signal value "|100" of the difference signal S3 does not exceed the second threshold value "+40". ​​Therefore, the second threshold determination section 42 determines that a negative event has not occurred.

[0114] In the time period from time t5 to time t6, the determination section 40 generates a motion signal S11 indicating the signal value of the difference signal S3 and the occurrence of a positive event according to the determination results made by the first threshold determination section 41 and the second threshold determination section 42. The bit inversion section 60 of the signal processing section 130a performs an inversion process on the bit value of the difference signal S3 and generates a grayscale signal S12. The signal processing section 130a outputs the motion signal S11 of each pixel P1 generated in the first subframe to the outside. In addition, the signal processing section 130a may output the grayscale signal S12 of each pixel P1 generated in the first subframe to the outside. The signal processing section 130a causes the storage section 75a to hold the grayscale signal S12 and the motion signal S11 of each pixel P1 in the first subframe.

[0115] In the time period from time t6 to time t7, the region detection unit 81 determines a region of interest (RoI) or a focus region based on the motion signal S11 of each pixel P1. The region detection unit 81 generates a region signal S15 indicating the focus region. The signal control unit 90 provides a control signal according to the region signal S15 to the light source controller 210, and controls the light emission position, light density, output power, etc. of the light source 220.

[0116] When the light source 220 emits light of the region of interest, the calculation section 65 of the signal processing section 130b performs a process of calculating the distance to the region of interest (depth calculation process) by using the signal S4 obtained from each of the pixels P2. The calculation section 65 may generate a distance signal S13 for each pixel P2 and output the generated distance signal S13.

[0117] At time t7 of the second subframe, the count value of the counter section 30a is reset. The counter section 30a starts counting the pulses of the signal S2a according to the start signal. When the count value reaches the reference value "255" at time t8, the counter section 30a outputs an overflow signal to the control section 35. "255" is set as the initial value of the count down.

[0118] At time t8, the counter section 30a starts to count down the pulses of the signal S2a. The control section 35 learns the length of the first period Ta2 by using the overflow signal. The control section 35 generates a stop signal and provides the stop signal to the counter section 30a to make the length of the second period Tb2 equal to the length of the first period Ta2.

[0119] At time t9, the counter section 30a ends the pulse of the count-down signal S2a according to the stop signal. The counter section 30a outputs a difference signal S3 indicating the difference "+20" between the count value in the first period Ta2 and the count value in the second period Tb2.

[0120] During the period from time t9 to time t10, the determination section 40 compares the value "+20" of the difference signal S3 with the threshold value "+40". ​​The first threshold determination section 41 and the second threshold determination section 42 respectively determine that the value of the differential signal S3 falls within the threshold value, and determine that a positive event has not occurred or a negative event has not occurred.

[0121] During the time period from time t10 to time t12, the determination section 40 generates a motion signal S11 indicating a signal value of the difference signal S3 and indicating that no event has occurred. The bit inversion section 60 performs an inversion process on the bit value of the difference signal S3 and generates a grayscale signal S12. The signal processing section 130a outputs the motion signal S11 of each pixel P1 generated in the second subframe to the outside. The signal processing section 130a may also output the grayscale signal S12 of each pixel P1 generated in the second subframe to the outside. In addition, the signal processing section 130a causes the memory section 75a to hold the grayscale signal S12 and the motion signal S11 of each pixel P1 in the second subframe.

[0122] exist Figure 5At time t13 of the Nth subframe shown, the count value of the counter section 30a is reset. Thereafter, the counter section 30a starts counting the pulses of the signal S2a according to the start signal. When the count value reaches the reference value "255" at time t14, the counter section 30a outputs an overflow signal to the control section 35. "255" is set as the initial value of the count down.

[0123] At time t14, the counter section 30a starts to count down the pulses of the signal S2a. The control section 35 learns the length of the first period Tan by using the overflow signal. The control section 35 provides a stop signal to the counter section 30a to make the length of the second period Tbn equal to the length of the first period Tan.

[0124] At time t15, the counter part 30a ends the pulse of the down-counting signal S2a according to the stop signal. The counter part 30a outputs a difference signal S3 indicating the difference "|20" between the count value in the first period Tan and the count value in the second period Tbn.

[0125] During the period from time t16 to time t17, the determination section 40 compares the value "|20" of the difference signal S3 with the threshold value "}40". The first threshold determination section 41 and the second threshold determination section 42 respectively determine that the value of the differential signal S3 falls within the threshold value, and determine that a positive event or a negative event has not occurred.

[0126] During the time period from time t17 to time t18, the determination section 40 generates a motion signal S11 indicating a signal value of the difference signal S3 and indicating that no event has occurred. The bit inversion section 60 performs an inversion process on the bit value of the difference signal S3 and generates a grayscale signal S12. The signal processing section 130a outputs the motion signal S11 of each pixel P1 generated in the Nth subframe to the outside. The signal processing section 130a may also output the grayscale signal S12 of each pixel P1 generated in the Nth subframe to the outside. In addition, the signal processing section 130a causes the storage section 75a to hold the grayscale signal S12 and the motion signal S11 of each pixel P1 in the Nth subframe.

[0127] In the time period from time t19 to time t20, the adding section 70a of the signal processing section 130a calculates the moving direction of the moving target by using the motion signal S11 in each subframe maintained in the storage section 75a. The adding section 70a aligns the grayscale signal S12 of each pixel P1 in a plurality of subframes according to the movement of the target, and adds and averages the grayscale signal S12 in each subframe. The signal processing section 130a can output the grayscale signal S12 averaged by the adding section 70a to the outside.

[0128] In this way, the optical detector 1 of the present embodiment has a first circuit unit 140a and a second circuit unit 140b. The first circuit unit 140a is used to output a motion signal S11 and a grayscale signal S12. The second circuit unit 140b is configured to output a distance signal S13. The motion signal and the grayscale signal as well as the distance signal can be obtained, and an optical detector with high detection performance is realized.

[0129] In addition, in this embodiment, a SPAD element is used as a light receiving element. This makes it possible to suppress the reduction in detection accuracy in the case of measurement with low illumination and low contrast. The movement of the measurement target can be accurately detected, and the grayscale value and the measured distance can be output. In addition, by averaging the signal, the S / N ratio can be improved, and a grayscale signal and a distance signal with less noise can be obtained.

[0130] Fig. 6A and 6B FIG. 1 is a flowchart showing an operation example of the light detection system of this embodiment. Fig. 6A and 6B An operation example of the light detection system 200 is described with reference to any one of the flowcharts.

[0131] exist Fig. 6A In step S100 shown, the count value of the counter section 30a is reset according to the start signal from the control section 35. The counter section 30a starts counting the pulses of the signal S2a generated in response to the reception of the photon. When the count value reaches the reference value (for example, the full code value "255"), the counter section 30a outputs an overflow signal to the control section 35. In step S110, the counter section 30a starts counting down the pulses of the signal S2a.

[0132] In step S120, the counter section 30a stops the countdown of the pulses according to the stop signal from the control section 35. The counter section 30a outputs a difference signal S3 indicating the count result to the determination section 40 and the signal processing section 130a.

[0133] In step S130, the determination section 40 compares the signal value of the difference signal S3 with the threshold value to determine the presence or absence of the occurrence of the event. In step S140, the determination section 40 outputs the motion signal S11, which indicates the signal value of the difference signal S3 and whether the event has occurred according to the result of the determination, to the signal processing section 130a. In addition, the determination section 40 outputs the motion signal S11 to the outside of the photodetector 1.

[0134] In step S150, the bit inversion section 60 performs bit inversion processing on the difference signal S3 to generate a grayscale signal S12. In step S160, the signal processing section 130a learns whether to perform processing of averaging the grayscale signal S12. In the case where the averaging processing is not performed ("No" in step S160), the processing proceeds to step S170. In step S170, the signal processing section 130a outputs the grayscale signal S12 of each pixel P1 to the outside of the photodetector 1. The grayscale signal S12 is generated by the bit inversion section 60.

[0135] In the case where the averaging process is performed (YES in step S160), the process proceeds to step S180. It should be noted that the presence or absence of the averaging process may be automatically set by the photodetector 1 or may be specified by the user.

[0136] In step S180, the adding section 70a moves the grayscale signal S12 of each pixel P1 in each frame stored in the storage section 75a according to the moving direction of the target obtained from the motion signal S11, adds and averages the grayscale signal S12. In step S190, the signal processing section 130a may output the grayscale signal S12 averaged by the adding section 70a to the outside.

[0137] After the above step S140, the process also proceeds to Figure 6B In step S200, the region detection unit 81 of the signal processing unit 130a generates a region signal S15 indicating the region of interest based on the motion signal S11 of each pixel P1. In step S210, the signal control unit 90 outputs a control signal to the light source controller 210 according to the region signal S15 and controls the light emitting position of the light source 220, etc.

[0138] In step S220, the calculation section 65 of the signal processing section 130b generates a distance signal S13 indicating the distance from the region of interest based on the signal S4 of each pixel P2. In step S230, the signal processing section 130a learns whether to control the light source 220 by using the distance signal S13. In the case of controlling the light source 220 ("Yes" in step S230), the process proceeds to step S240. It should be noted that the presence or absence of performing control on the light source by using the distance signal S13 may be automatically set or may be specified by the user.

[0139] In step S240, the distance detection unit 82 extracts the region of interest by using the distance signal S13 of each pixel P2. The distance detection unit 82 generates a distance signal S16 indicating the distance from the region of interest. The signal control unit 90 outputs a control signal to the light source controller 210 according to the distance signal S16 and controls the light emitting position of the light source 220, etc. After step S240, the processing of steps S200 to S220 is performed again.

[0140] In the case of "No" in step S230, the process proceeds to step S250. In other words, the process proceeds to step S250 without controlling the light source 220. In step S250, the signal processing unit 130b may output the distance signal S13 of each pixel P2 to the outside of the light detector 1. After step S250, the light detection system 200 ends the process shown in the flowchart.

[0141] Figure 7 1 is a schematic diagram showing an example of a cross-sectional configuration of a light detector according to an embodiment. The light detector 1 has a configuration in which a first substrate 101, a second substrate 102, and a third substrate 103 are stacked in the Z-axis direction. The first substrate 101, the second substrate 102, and the third substrate 103 each include a semiconductor substrate (e.g., a silicon substrate).

[0142] like Figure 7 As shown, the first substrate 101, the second substrate 102 and the third substrate 103 have first surfaces 11S1, 12S1 and 13S1 and second surfaces 11S2, 12S2 and 13S2, respectively. The first surfaces 11S1, 12S1 and 13S1 are provided with transistors. Each of the first surfaces 11S1, 12S1 and 13S1 is an element formation surface on which elements such as transistors are formed. Each of the first surfaces 11S1, 12S1 and 13S1 is provided with a gate electrode, a gate oxide film, etc.

[0143] The first substrate 101 is provided with a plurality of pixels P1 and P2 including light receiving elements 10a and 10b, respectively. Figure 7 As schematically shown in FIG. 1 , each pixel P1 and pixel P2 includes a multiplication section 17 (multiplication region) configured to perform avalanche multiplication. On the second surface 11S2 side of the first substrate 101 , a lens section 16 , an optical filter 15 , etc. are provided for each pixel P. The lens section 16 condenses light. Figure 7 Pixel P1 and pixel P2 are shown. Pixel P1 is a G pixel including a green color filter and a filter that blocks infrared light (IR-Cut filter). Pixel P2 is an IR pixel including a red color filter and a blue color filter.

[0144] like Figure 7As shown, the first surface 11S1 of the first substrate 101 is provided with a wiring layer 111. In addition, the first surface 12S1 of the second substrate 102 is provided with a wiring layer 121. The second surface 12S2 of the second substrate 102 is provided with a wiring layer 122. The first surface 13S1 of the third substrate 103 is provided with a wiring layer 131. The wiring layers 111, 121, 122, 131 are composed of, for example, a conductor film and an insulating film. Each of the wiring layers 111, 121, 122 and 131 includes a plurality of wirings, vias, etc. Each of the wiring layers 111, 121, 122 and 131 includes, for example, two or more layers of wiring.

[0145] Each of the wiring layers 111, 121, 122, and 131 has a configuration in which a plurality of wirings are stacked, for example, with an interlayer insulating layer (interlayer insulating film) interposed therebetween. Each wiring layer is formed using aluminum (Al), copper (Cu), tungsten (W), polysilicon (Poly-Si), or the like. In one example, each of the interlayer insulating layers is formed by using a single-layer film containing one of silicon oxide (SiO), silicon nitride (SiN), silicon oxynitride (SiON), or the like, or a stacked film containing two or more of them.

[0146] Note that the first substrate 101 and the wiring layer 111 are sometimes collectively referred to as the first substrate 101 (or the first circuit layer). In addition, the second substrate 102 and the wiring layers 121 and 122 are sometimes collectively referred to as the second substrate 102 (or the second circuit layer). The third substrate 103 and the wiring layer 131 are sometimes collectively referred to as the third substrate 103 (or the third circuit layer).

[0147] exist Figure 7 In the illustrated embodiment, the first substrate 101 and the second substrate 102 are stacked by bonding electrodes, wherein the first surface 11S1 and the first surface 12S1 are opposite to each other. Elements such as transistors are formed on the first face 11S1 and the first face 12S1. In other words, the first substrate 101 and the second substrate 102 are bonded with their respective front surfaces opposite to each other. The second substrate 102 and the third substrate 103 are stacked by bonding electrodes having the second surface 12S2 and the first surface 13S1 opposite to each other. Elements such as transistors are formed on the first surface 13S1. In other words, the second substrate 102 and the third substrate 103 are bonded when the rear surface of the second substrate 102 and the front surface of the third substrate 103 are opposite to each other.

[0148] In one embodiment, the first substrate 101 and the second substrate 102 are bonded together by bonding metal electrodes each including copper (Cu). In other words, the first substrate 101 and the second substrate 102 are bonded together by forming a Cu-Cu junction. In addition, the second substrate 102 and the third substrate 103 are also bonded together by forming, for example, a Cu-Cu junction. It should be noted that the electrodes for bonding may include, for example, a metal material such as nickel (Ni), cobalt (Co) or tin (Sn) other than copper (Cu). Alternatively, these electrodes may include another material.

[0149] The second substrate 102 and the wiring layers 121 and 122 include a plurality of through electrodes 80. The through electrodes 80 are electrodes that penetrate the second substrate 102. The through electrodes 80 are formed to extend in the Z-axis direction and reach the inside of the wiring layer 122 of the second substrate 102. The through electrodes 80 include, for example, tungsten (W), aluminum (Al), cobalt (Co), molybdenum (Mo), ruthenium (Ru), etc. The through electrodes 80 electrically connect the circuit provided on the first surface 12S1 side of the second substrate 102 and the circuit provided on the first surface 13S1 side of the third substrate 103.

[0150] The first substrate 101 is provided with a pad (PAD). The pad is an electrode formed by using, for example, aluminum (Al). It should be noted that the pad may include another metal material. A plurality of pads are provided in the photodetector 1. For example, each pad may supply a power supply voltage input from the outside to each circuit of the first substrate 101 to the third substrate 103.

[0151] <Work and Effect>

[0152] The photodetector (photodetector 1) according to the present embodiment includes: a plurality of first pixels (pixel P1); a first circuit unit (first circuit unit 140a); a plurality of second pixels (pixel P2); and a second circuit unit (second circuit unit 140b). Each of the plurality of first pixels includes a first light receiving element configured to receive light and output a first photocurrent. The first circuit unit is configured to acquire a signal based on the first photocurrent, and output a first signal (motion signal S11) related to the movement of the target and a second signal (grayscale signal S12) related to the grayscale value. Each of the plurality of second pixels includes a second light receiving element configured to receive light and output a second photocurrent. The second circuit unit is used to acquire a signal based on the second photocurrent, and output a third signal (distance signal S13) related to the distance from the target.

[0153] The optical detector 1 of the present embodiment has a first circuit unit 140a and a second circuit unit 140b. The first circuit unit 140a is used to output a motion signal S11 and a grayscale signal S12. The second circuit unit 140b is configured to output a distance signal S13. This makes it possible to obtain a motion signal, a grayscale signal, and a distance signal for each pixel P. A photodetector with high detection performance can be realized.

[0154] Next, a modification of the present disclosure will be described. Hereinafter, the same reference numerals are given to the same components as those in the above-described embodiment, and the description thereof will be appropriately omitted.

[0155] (1-1. Modification 1)

[0156] In the above-described embodiment, the configuration example of the photodetector 1 has been described, but the photodetector 1 may have a configuration in which two substrates are stacked. Figure 8 is a diagram showing a configuration example of a photodetector according to Modification 1. The photodetector 1 has a configuration (two-layer configuration) in which a first substrate 101 and a second substrate 102 are stacked in the Z-axis direction. In this case, for example, pixel circuit sections 120a and 120b and signal processing sections 130a and 130b may be provided in the second substrate 102.

[0157] (1-2. Modification 2)

[0158] In the above-mentioned embodiments and modified examples, the arrangement examples of the pixels P1 and the pixels P2 have been described, but the arrangement of the pixels is not limited thereto. Fig.9A Or the embodiment shown in 9B. Fig.9A In the embodiment shown in FIG. 9B , pixels P1 and pixels P2 are alternately arranged in the pixel unit 100. Pixel P2 is an IR pixel.

[0159] The size of the pixel P2, the number of the pixels P2, etc. are not limited to the illustrated embodiment. The size of the pixel P2, the number of the pixels P2, etc. may be appropriately changed. Figures 10 to 14 1 and 2 are diagrams each showing an example of arrangement of pixels of a light detector according to Modification 2. Fig.10 As shown, the pixel P1 and the pixel P2 may have the same size. In the pixel unit 100, pixel blocks each including a plurality of pixels P2 may be repeatedly arranged. Fig.11 As shown in FIG. 1 , a pixel block including 2 to 2 pixels P2 (ie, a total of four pixels P2) may be provided. It should be noted that the pixel P2 may have the same Fig.12 The size of the four pixels P1 shown in FIG. Fig.13As shown in FIG. 1 , a pixel block including 4 to 4 pixels P2 (ie, a total of 16 pixels P2) may be provided in the pixel unit 100. It should be noted that Fig.14 As shown, the pixel P2 may have a size corresponding to 16 pixels P1.

[0160] (1-3. Modification 3)

[0161] In the above embodiment, an example of starting counting in synchronization with a synchronization signal has been described. However, the optical detector 1 can start counting by using a start signal asynchronous with the synchronization signal. Compared with the case where the optical detector 1 controls the timing of the counting operation in synchronization with the synchronization signal, the optical detector 1 enables high-speed operation.

[0162] Fig.15 1 is a timing chart showing an operation example of the light detection system of Modification Example 3. Fig.15 In the embodiment shown, the control section 35 of the optical detector 1 can supply a start signal to the counter section 30a that is not synchronized with the synchronization signal. Figure 5 Compared with the synchronous operation in the above example, the time from the end timing of the count operation to the start timing of the next count operation can be reduced (such as Fig.15 ). This makes it possible to detect motion signals, grayscale signals, and distance signals at high speed.

[0163] (1-4. Modification 4)

[0164] Fig.16 : is a diagram showing a configuration example of a photodetector according to Modification 4. Fig.16 1 and 2. The configuration embodiment of the pixel P1 and the first circuit unit 140a is shown. The counter section 30a may include a first counter 32 and a second counter 33, each of which is configured to count the signal S2a received by the input section 31. For example, the first counter 32 is configured to count the number of pulses of the signal S2a in the first period Ta. In addition, the second counter 33 is configured to count the number of pulses of the signal S2a in the second period Tb. The counter section 30a can generate and output a difference signal S3 based on the difference between the count value of the first counter 32 and the count value of the second counter 33. In this modified example, an effect similar to that according to the above-described embodiment can also be obtained.

[0165] <3. Example of use>

[0166] For example, the above-described photodetector 1 may be used in various cases of sensing light such as visible light, infrared light, ultraviolet light, or X-rays as follows.

[0167] - A device that captures images for viewing, such as a digital camera or a mobile device each having a camera function;

[0168] - Equipment for traffic use, such as on-board sensors that capture images of the front, rear, surroundings, interior, etc. of a car for safe driving (such as automatic stop) and for recognizing the driver's status; monitoring cameras that monitor moving vehicles and roads; and ranging sensors that measure the distance between vehicles;

[0169] - Devices used in home appliances (such as televisions, refrigerators, or air conditioners) to capture images of user gestures and operate the appliance based on the gestures

[0170] - Equipment for medical care and health care purposes, such as endoscopes or equipment for photographing blood vessels by receiving infrared light

[0171] -Devices used for security purposes, such as surveillance cameras for crime prevention or cameras for personal authentication

[0172] -Equipment for beauty purposes, such as skin measurement equipment that takes images of the skin and microscopes that take images of the scalp

[0173] -Devices for sports use, such as action cameras or wearable cameras for sports applications

[0174] -Agricultural equipment, such as cameras used to monitor the conditions of fields and crops

[0175] <4. Practical Application Examples>

[0176] (Useful example of mobile object)

[0177] The technology according to the embodiments of the present disclosure (the present technology) can be applied to various products. For example, the technology according to the embodiments of the present disclosure can be implemented as a device installed on any type of mobile body (such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot).

[0178] Fig.17 : is a block diagram depicting an example of a schematic configuration of a vehicle control system as an example of a moving body control system to which the technology according to the embodiment of the present disclosure can be applied.

[0179] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. Fig.17In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. In addition, as a functional structure of the integrated control unit 12050, a microcomputer 12051, a sound / image output unit 12052, and an in-vehicle network interface (I / F) 12053 are exemplified.

[0180] The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 serves as a control device for a drive force generating device (such as an internal combustion engine, a drive motor, etc.) for generating a drive force of the vehicle, a drive force transmitting mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, and the like.

[0181] The body system control unit 12020 controls the operation of various devices provided on the vehicle body according to various programs. For example, the body system control unit 12020 is used as a control device for a keyless entry system, a smart key system, a power window device, or various lights such as a headlight, a backup light, a brake light, a turn signal, a fog light, etc. In this case, a radio wave transmitted from a mobile device as a substitute for a key or a signal of various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals, and controls the door lock device, the power window device, the lights, etc. of the vehicle.

[0182] The vehicle exterior information detection unit 12030 detects information outside the vehicle including the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to an imaging unit 12031. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receive the captured image. In addition, the vehicle exterior information detection unit 12030 may also detect objects such as people, vehicles, obstacles, signs, and text on the road surface, or detect their distances, based on the received image.

[0183] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or can output the electrical signal as information related to the measured distance. In addition, the light received by the imaging unit 12031 can be visible light, or can be invisible light such as infrared rays.

[0184] The in-vehicle information detection unit 12040 detects information related to the interior of the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that photographs the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's fatigue or the driver's concentration, or can determine whether the driver is dozing off.

[0185] The microcomputer 12051 can calculate the control target value of the driving force generation device, the steering mechanism, or the braking device based on the information related to the inside or outside of the vehicle obtained by the vehicle outside information detection unit 12030 or the vehicle inside information detection unit 12040, and output the control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of the advanced driver assistance system (ADAS), which includes collision prevention or shock absorption for the vehicle, follow-up driving based on the following distance, maintaining the vehicle speed for driving, warning of vehicle collision, warning of vehicle deviation from the lane, etc.

[0186] In addition, the microcomputer 12051 can perform collaborative control for automatic driving by controlling the driving force generating device, steering mechanism, braking device, etc. based on information related to the outside or inside of the vehicle obtained by the outside information detection unit 12030 or the inside information detection unit 12040, thereby enabling the vehicle to drive automatically without relying on the driver's operation, etc.

[0187] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information related to the outside of the vehicle obtained by the outside vehicle information detection unit 12030. For example, the microcomputer 12051 can perform cooperative control aimed at preventing glare by controlling the headlights to change from high beam to low beam according to the position of the front vehicle or the oncoming vehicle detected by the outside vehicle information detection unit 12030.

[0188] The sound / image output unit 12052 sends an output signal of at least one of sound and image to an output device, which can visually or auditorily notify the occupants of the vehicle or the outside of the vehicle of information. Fig.17 In the embodiment of FIG. 1 , an audio speaker 12061, a display portion 12062, and a dashboard 12063 are shown as output devices. For example, the display portion 12062 may include at least one of an on-board display and a head-up display.

[0189] Fig.18 This is a diagram of an embodiment of the installation position of the imaging unit 12031.

[0190] exist Fig.18 In the figure, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104 and 12105.

[0191] The imaging units 12101, 12102, 12103, 12104 and 12105 are, for example, disposed at positions on the front nose, side mirrors, rear bumper and rear door of the vehicle 12100 and at positions on the upper portion of the windshield inside the vehicle. The imaging unit 12101 disposed at the front nose inside the vehicle and the imaging unit 12105 disposed at the upper portion of the windshield mainly obtain images in front of the vehicle 12100. The imaging units 12102 and 12103 disposed to the side mirrors mainly obtain images on the sides of the vehicle 12100. The imaging unit 12104 disposed to the rear bumper or rear door mainly obtains images of the rear of the vehicle 12100. The imaging unit 12105 disposed at the upper portion of the windshield inside the vehicle is mainly used to detect vehicles ahead, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0192] By the way, Fig.18 An embodiment of the shooting range of the imaging units 12101 to 12104 is shown. The imaging range 12111 represents the imaging range of the imaging unit 12101 set to the front nose. The imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging units 12102 and 12103 set to the side mirrors. The imaging range 12114 represents the imaging range of the imaging unit 12104 set to the rear bumper or the rear door. For example, a bird's-eye view image of the vehicle 12100 viewed from above is obtained by overlapping the image data imaged by the imaging units 12101 to 12104.

[0193] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0194] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging range 12111 to 12114 and the time change of the distance (relative speed relative to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting objects that exist on the driving path of the vehicle 12100 and are traveling in the same direction as the vehicle 12100 at a prescribed speed (e.g., equal to or greater than 0 km / hour). In addition, the microcomputer 12051 can pre-set the following distance to keep ahead of the leading vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. As a result, coordinated control for automatic driving that allows the vehicle to travel automatically without relying on the driver's operation, etc., can be performed.

[0195] For example, the microcomputer 12051 can classify the three-dimensional object data related to the three-dimensional object into three-dimensional object data of a two-wheeled vehicle, a standard vehicle, a large vehicle, a pedestrian, a utility pole, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that can be visually recognized by the driver of the vehicle 12100 and obstacles that are difficult for the driver of the vehicle 12100 to visually recognize. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than the set value and therefore there is a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or evasive steering via the driving system control unit 12010. The microcomputer 12051 can thereby assist driving to avoid collisions.

[0196] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 may identify a pedestrian, for example, by determining whether a pedestrian exists in the captured images of the imaging units 12101 to 12104. This identification of a pedestrian is performed, for example, by a process of extracting feature points in the imaged images of the imaging units 12101 to 12104 as infrared cameras and by performing pattern matching processing on a series of feature points representing the outline of an object to determine whether it is a pedestrian. When the microcomputer 12051 determines that a pedestrian exists in the imaged images of the imaging units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 so that a square outline for emphasis is displayed as superimposed on the identified pedestrian. The sound / image output unit 12052 may also control the display unit 12062 so that an icon representing a pedestrian or the like is displayed at a desired position.

[0197] An embodiment of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied has been described above. The technology according to an embodiment of the present disclosure can be applied to, for example, the imaging section 12031 among the above-mentioned components. Specifically, for example, the light detector 1 can be applied to the imaging section 12031. Application of the technology according to an embodiment of the present disclosure to the imaging section 12031 makes it possible to obtain a high-definition captured image and to perform highly accurate control using the captured image in the mobile body control system.

[0198] (Example of actual application of endoscopic surgery system)

[0199] The technology according to the embodiment of the present disclosure (the present technology) can be applied to various products. For example, the technology according to the embodiment of the present disclosure can be applied to an endoscopic surgery system.

[0200] Fig.19 is a diagram describing an example of a schematic configuration of an endoscopic surgery system to which the technology according to an embodiment of the present disclosure (the present technology) can be applied.

[0201] exist Fig.19 , a surgeon (doctor) 11131 is shown to be performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 includes an endoscope 11100, other surgical tools 11110 such as a pneumoperitoneum tube 11111 and an energy device 11112, a support arm device 11120 on which the endoscope 11100 is supported, and a cart 11200 on which various endoscopic surgery devices are installed.

[0202] The endoscope 11100 includes a lens barrel 11101 and a camera head 11102 connected to the proximal end of the lens barrel 11101, and the lens barrel 11101 has a region of a predetermined length from its distal end to be inserted into a body cavity of a patient 11132. In the illustrated example, the endoscope 11100 is shown as a rigid endoscope having a rigid lens barrel 11101. However, the endoscope 11100 may be additionally included as a flexible endoscope having a flexible lens barrel 11101.

[0203] The lens barrel 11101 has an opening at its distal end, and the objective lens is mounted in the opening. The light source device 11203 is connected to the endoscope 11100, so that the light generated by the light source device 11203 is introduced into the front end of the lens barrel 11101 by the light guide extending inside the lens barrel 11101, and irradiated toward the observation object in the body cavity of the patient 11132 via the objective lens. In addition, the endoscope 11100 can be either a direct-view endoscope, an oblique-view endoscope, or a side-view endoscope.

[0204] The optical system and the image pickup element are arranged inside the camera head 11102, so that the reflected light (observation light) from the observation target is focused on the image pickup element through the optical system. The observation light is photoelectrically converted by the imaging element to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. The image signal is transmitted to the CCU 11201 as RAW data.

[0205] The CCU 11201 includes a central processing unit (CPU), a graphics processing unit (GPU), etc., and integrally controls the operations of the endoscope 11100 and the display device 11202. In addition, the CCU 11201 receives an image signal from the camera 11102 and performs various image processing for displaying an image based on the image signal, such as, for example, development processing (demosaic processing), on the image signal.

[0206] The display device 11202 displays an image thereon based on an image signal on which image processing has been performed under the control of the CCU 11201 .

[0207] The light source device 11203 is composed of a light source such as a light emitting diode (LED), and supplies irradiation light for imaging the surgical area to the endoscope 11100 .

[0208] The input device 11204 is an input interface of the endoscopic surgery system 11000. The user can input various information or instructions to the endoscopic surgery system 11000 through the input device 11204. For example, the user inputs instructions to change the imaging conditions of the endoscope 11100 (type of irradiation light, magnification, focal length, etc.).

[0209] The treatment tool control device 11205 controls the drive of the energy device 11112 for cauterizing or cutting tissues, sealing blood vessels, etc. In order to ensure the field of view of the endoscope 11100 and the operating space of the surgeon, the pneumoperitoneum device 11206 supplies gas to the body cavity of the patient 11132 through the pneumoperitoneum tube 11111 to expand the body cavity. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various forms (such as text, images, or graphics).

[0210] In addition, the light source device 11203 that supplies irradiation light when the surgical area is photographed to the endoscope 11100 may also include a white light source, which is composed of, for example, an LED, a laser light source, or a combination thereof. In the case where the white light source includes a combination of red, green, and blue (RGB) laser light sources, since the output intensity and output timing can be controlled with high precision for each color (each wavelength), the white balance of the captured image can be adjusted by the light source device 11203. In addition, in this case, if the laser beams from the respective RGB laser light sources are irradiated on the observation target in a time-sharing manner and the drive of the image pickup element of the camera 11102 is controlled synchronously with the irradiation timing. Then, images corresponding to the R, G, and B colors, respectively, can also be captured in a time-sharing manner. According to this method, a color image can be obtained even if a color filter is not provided for the imaging element.

[0211] In addition, the light source device 11203 can be controlled so that the intensity of the light to be output is changed every predetermined time. By controlling the drive of the image pickup element of the camera head 11102 in synchronization with the timing of the light intensity change to acquire images in time division and synthesize the images, a high dynamic range image without underexposed shadows and overexposed highlights can be generated.

[0212] In addition, the light source device 11203 can be configured to provide light of a predetermined wavelength band prepared for special light observation. In special light observation, by utilizing the wavelength dependence of light absorption of biological tissue to irradiate light of a narrower band than the irradiation light (i.e., white light) during normal observation, narrow-band light observation (narrow-band light observation) is performed to image a predetermined tissue such as blood vessels in the surface layer of the mucosa with high contrast. Alternatively, in special light observation, fluorescence observation can also be performed to obtain an image from fluorescence generated by irradiation with excitation light. In fluorescence observation, it is possible to observe fluorescence from biological tissue by irradiating excitation light to biological tissue (autofluorescence observation), or to obtain a fluorescence image by locally injecting a reagent such as indocyanine green (ICG) into the biological tissue and irradiating the biological tissue with excitation light corresponding to the fluorescence wavelength of the reagent. The light source device 11203 can be configured to provide narrow-band light and / or excitation light suitable for special light observation as described above.

[0213] Fig. 20 It is a depiction Fig.19 A block diagram of an example of a functional configuration of camera 11102 and CCU 11201 is depicted in FIG.

[0214] The camera head 11102 includes a lens unit 11401, an imaging unit 11402, a driving unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected via a transmission cable 11400 for communication with each other.

[0215] The lens unit 11401 is an optical system, and is provided at a connection position with the lens barrel 11101. Observation light acquired from the distal end of the lens barrel 11101 is guided to the camera head 11102 and introduced into the lens unit 11401. The lens unit 11401 includes a combination of a plurality of lenses including a zoom lens and a focus lens.

[0216] The number of image pickup elements included in the camera unit 11402 may be one (single-board type) or multiple (multi-board type). In the case where the camera unit 11402 is configured as a multi-board type camera unit, for example, image signals corresponding to each R, G, and B are generated by the image pickup element, and the image signals can be synthesized to obtain a color image. The camera unit 11402 may also be configured to have a pair of image pickup elements for acquiring corresponding image signals of the right eye and the left eye prepared for three-dimensional (3D) display. In the case of 3D display, the surgical operator 11131 can more accurately grasp the depth of the biological tissue in the surgical area. It should be noted that in the case where the camera unit 11402 is configured as a stereoscopic type camera unit, a plurality of systems of lens units 11401 are set corresponding to a single image pickup element.

[0217] Furthermore, the image pickup unit 11402 is not necessarily provided on the camera head 11102. For example, the image pickup unit 11402 may be provided inside the lens barrel 11101 just behind the objective lens.

[0218] The driving unit 11403 includes an actuator, and moves the zoom lens and the focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera head control unit 11405. As a result, the magnification and focus of the image picked up by the imaging unit 11402 can be appropriately adjusted.

[0219] The communication unit 11404 includes a communication device for sending and receiving various information to and from the CCU 11201. The communication unit 11404 transmits the image signal acquired from the imaging unit 11402 to the CCU 11201 as RAW data via the transmission cable 11400.

[0220] Furthermore, the communication unit 11404 receives a control signal for controlling the driving of the camera 11102 from the CCU 11201, and supplies the control signal to the camera control unit 11405. The control signal includes information related to image pickup conditions, such as information specifying a frame rate of a picked-up image, information specifying an exposure value at the time of image pickup, and / or information specifying a magnification and a focus of a picked-up image.

[0221] It should be noted that image pickup conditions such as frame rate, exposure value, magnification, or focus may be specified by a user or may be automatically set based on an acquired image signal by the control unit 11413 of the CCU 11201. In the latter case, an automatic exposure (AE) function, an automatic focus (AF) function, and an automatic white balance (AWB) function are incorporated in the endoscope 11100.

[0222] The camera control unit 11405 controls the driving of the camera 11102 based on the control signal from the CCU 111201 received through the communication unit 11404 .

[0223] The communication unit 11411 includes a communication device for sending and receiving various information to and from the camera head 11102. The communication unit 11411 receives an image signal transmitted thereto from the camera head 11102 through the transmission cable 11400.

[0224] Furthermore, the communication unit 11411 transmits a control signal for controlling the driving of the camera head 11102 to the camera head 11102. The image signal and the control signal may be transmitted through electrical communication, optical communication, or the like.

[0225] The image processing unit 11412 performs various image processing on the image signal in the form of RAW data sent thereto from the camera head 11102 .

[0226] The control unit 11413 performs various controls related to the imaging of the operation area, etc. by the endoscope 11100 and the display of the captured image obtained by imaging the operation area, etc. For example, the control unit 11413 creates a control signal for controlling the driving of the camera head 11102 .

[0227] In addition, the control unit 11413 controls the display device 11202 to display a picked-up image of the surgical area, etc., based on the image signal for which the image processing unit 11412 has performed image processing. Thus, the control unit 11413 can use various image recognition technologies to recognize various objects in the picked-up image. For example, the control unit 11413 can recognize surgical tools such as forceps, a specific living area, bleeding, fog when the energy device 11112 is used, etc., by detecting the shape, color, etc. of the edge of the object included in the picked-up image. When the control unit 11413 controls the display device 11202 to display the captured image, the control unit 11413 can use the recognition result to display various surgical support information in a manner overlapping with the image of the surgical area. In the case where the surgical support information is displayed in an overlapping manner and prompted to the surgical operator 11131, the burden of the surgical operator 11131 can be reduced, and the surgical operator 11131 can reliably perform the operation.

[0228] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 to each other is an electric signal cable prepared for communication of electric signals, an optical fiber prepared for optical communication, or a composite cable prepared for both electric communication and optical communication.

[0229] Here, although in the depicted embodiment, communication is performed by wired communication using the transmission cable 11400, communication between the camera head 11102 and the CCU 11201 may be performed by wireless communication.

[0230] An embodiment of an endoscopic surgical system to which the technology according to an embodiment of the present disclosure can be applied has been described above. The technology according to an embodiment of the present disclosure can be advantageously applied to, for example, the imaging unit 11402 provided to the camera head 11102 of the endoscope 11100 among the above-mentioned components. Applying the technology according to an embodiment of the present disclosure to the imaging unit 11402 makes it possible to increase the sensitivity of the imaging unit 11402 and to provide a high-definition endoscope 11100.

[0231] Although the present disclosure has been described with reference to embodiments, modifications, use cases, and practical application examples, the content of the present technology is not limited to the above-mentioned embodiments, etc. Various modifications are possible. For example, the above-mentioned modifications have been described as modifications of the above-mentioned embodiments, but the components according to the various modifications can be appropriately combined.

[0232] According to an embodiment of the present disclosure, a photodetector includes: a plurality of first pixels; a first circuit unit; a plurality of second pixels; and a second circuit unit. Each of the plurality of first pixels includes a first light receiving element configured to receive light and output a first photocurrent. The first circuit unit is configured to acquire a signal based on the first photocurrent, and output a first signal related to the motion of a target and a second signal related to a grayscale value. Each of the plurality of second pixels includes a second light receiving element configured to receive light and output a second photocurrent. The second circuit unit is configured to acquire a signal based on the second photocurrent, and output a third signal related to the distance from the target. This makes it possible to obtain a motion signal, a grayscale signal, and a distance signal, and realize a photodetector with high detection performance.

[0233] It should be noted that the effects described herein are merely examples, but are not limited to the description. Other effects may exist. In addition, the embodiments of the present disclosure may also have the following configurations.

[0234] (1) A light detection device comprising:

[0235] a first pixel including a first light receiving element configured to receive light;

[0236] a first circuit coupled to the first light receiving element, and the first circuit is configured to output a first signal related to the motion of the object and a second signal related to the intensity value;

[0237] a second pixel including a second light receiving element configured to receive light; and

[0238] The second circuit is coupled to the second light receiving element and is configured to output a third signal related to the distance to the target.

[0239] (2) The photodetection device according to (1), wherein the first light receiving element and / or the second light receiving element includes a single photon avalanche diode.

[0240] (3) The light detection device according to (1) or (2), wherein:

[0241] The first light receiving element is configured to receive visible light, and

[0242] The second light receiving element is configured to receive invisible light.

[0243] (4) According to any one of (1) to (3) of the light detection device, the first circuit is configured to output a first signal based on a difference between the number of voltage signals within a first time period and the number of voltage signals within a second time period, the voltage signal being based on an output of the first light receiving element.

[0244] (5) The light detection device according to (4), wherein the first circuit is configured to output the second signal based on a sum of the number of voltage signals within the first period and the number of voltage signals within the second period.

[0245] (6) The light detection device according to (4) or (5), wherein:

[0246] The first circuit includes a first counter circuit configured to output a difference signal based on a difference between the number of voltage signals within the first period and the number of voltage signals within the second period, and

[0247] The first counter circuit is configured to output a detection signal indicating that the number of voltage signals within the first period has reached a reference value.

[0248] (7) The light detection device according to (6), wherein the first counter circuit includes an up-down counter.

[0249] (8) The light detection device according to (6) or (7), wherein:

[0250] The first circuit includes a control circuit configured to control a first counting circuit, and

[0251] The control circuit is configured to output a stop signal indicating an end of the second period to the first counting circuit based on the detection signal indicating an end of the first period.

[0252] (9) The light detection device according to (8), wherein the control circuit is configured to output a stop signal indicating an end of the second period to the first counter circuit to substantially equalize respective lengths of the first period and the second period.

[0253] (10) The light detection device according to (8) or (9), wherein the control circuit includes a timing generator.

[0254] (11) The light detection device according to any one of (1) to (10), comprising:

[0255] a first generating circuit configured to generate a voltage signal based on an output of the first light receiving element; and

[0256] The second generating circuit is configured to generate a voltage signal based on the output of the second light receiving element.

[0257] (12) The light detection device according to (11), wherein the first generation circuit and / or the second generation circuit includes an inverter.

[0258] (13) The light detection device according to any one of (1) to (12), comprising:

[0259] a first substrate including a plurality of first light receiving elements and a plurality of second light receiving elements; and

[0260] The second substrate at least includes the first circuit and a part of the second circuit, and the first substrate is stacked on the second substrate.

[0261] (14) The light detection device according to any one of (1) to (13), further including: a light source controller configured to control the light source.

[0262] (15) The light detection device according to (14), wherein the first circuit is configured to generate a first area signal associated with an area of ​​the target based on the first signal.

[0263] (16) The light detection device according to (15), wherein the light source controller is configured to control at least one of a light emission position, a light density, and an output power of the light source based on the first area signal.

[0264] (17) The light detection device according to (15) or (16), wherein:

[0265] The second circuit includes at least one second counter circuit configured to count the number of voltage signals based on the output of the second light receiving element, and

[0266] The second circuit is configured to output a third signal based on the amount of the voltage signal.

[0267] (18) The light detection device according to any one of (15) to (17), wherein the second circuit is configured to output a third signal related to the distance to the target based on the first area signal.

[0268] (19) The light detection device according to (17) or (18), wherein the light source controller is configured to control at least one of a light emitting position, a light density, and an output power of the light source based on the third signal.

[0269] (20) The light detection device according to any one of (1) to (19), wherein the light detection device includes a pixel array, and the pixel array includes a first pixel and a second pixel.

[0270] (21) A light detection system comprising:

[0271] light source;

[0272] a light detection device; and

[0273] a light source controller configured to control the light source based on a signal output from the light detection device,

[0274] Wherein, the light detection device comprises:

[0275] a first pixel including a first light receiving element configured to receive light,

[0276] a first circuit coupled to the first light receiving element, and the first circuit is configured to output a first signal related to the motion of the target and a second signal related to the intensity value,

[0277] a second pixel including a second light receiving element configured to receive light, and

[0278] The second circuit is coupled to the second light receiving element and is configured to output a third signal related to the distance to the target.

[0279] (22) A light detector comprising:

[0280] a plurality of first pixels, each of the first pixels including a first light receiving element configured to receive light and output a first photocurrent;

[0281] a first circuit unit configured to acquire a signal based on the first photocurrent and output a first signal related to the motion of the target and a second signal related to the grayscale value;

[0282] a plurality of second pixels, each second pixel including a second light receiving element configured to receive light and output a second photocurrent; and

[0283] The second circuit unit is configured to obtain a signal based on the second photocurrent and output a third signal related to the distance to the target.

[0284] (23) The photodetector according to (22), wherein the first light receiving element and the second light receiving element each include a single photon avalanche diode.

[0285] (24) The photodetector according to (22) or (23), wherein:

[0286] Each of the first pixels includes a pixel including a first light receiving element configured to receive visible light and output a first photocurrent, and

[0287] Each of the second pixels includes a pixel including a second light receiving element configured to receive invisible light and output a second photocurrent.

[0288] (25) A photodetector according to any one of (22) to (24), wherein the first circuit unit is configured to output a signal based on the difference between the number of pulses of the voltage signal in the first time period and the number of pulses of the voltage signal in the second time period as a first signal, and the voltage signal is based on the first photocurrent.

[0289] (26) An optical detector according to any one of (22) to (25), wherein the first circuit unit is configured to output a signal based on the sum of the number of pulses of the voltage signal within the first time period and the number of pulses of the voltage signal within the second time period as the second signal.

[0290] (27) The photodetector according to any one of (22) to (26), wherein:

[0291] The first circuit unit includes a first counter section configured to output a signal based on a difference between the number of pulses of the voltage signal within the first period and the number of pulses of the voltage signal within the second period, and

[0292] The first counter section is configured to output a detection signal indicating that the number of pulses of the voltage signal within the first period reaches a reference value.

[0293] (28) The photodetector according to (27), wherein the first counter section includes an up-and-down counter.

[0294] (29) The photodetector according to (27) or (28), wherein:

[0295] The first circuit unit includes a control section configured to control the first counter section, and

[0296] The control section is configured to output a signal indicating an end of the second period to the first counter section based on the detection signal indicating an end of the first period.

[0297] (30) The photodetector according to (29), wherein the control section is configured to output a signal indicating an end of the second period to the first counter section to equalize respective lengths of the first period and the second period.

[0298] (31) The photodetector according to (29) or (30), wherein the control unit includes a timing generator.

[0299] (32) The photodetector according to any one of (22) to (31), comprising:

[0300] a first generating unit that generates a voltage signal based on the first photocurrent; and

[0301] The second generating unit generates a voltage signal based on the second photocurrent.

[0302] (33) The photodetector according to (32), wherein the first generating unit and the second generating unit each include an inverter.

[0303] (34) The photodetector according to any one of (22) to (33), comprising:

[0304] a first substrate including a plurality of first light receiving elements and a plurality of second light receiving elements; and

[0305] The second substrate at least includes the first circuit unit and a part of the second circuit unit, and the second substrate is stacked on the first substrate.

[0306] (35) The light detector according to any one of (22) to (34), comprising a light source controller configured to control the light source.

[0307] (36) The photodetector according to (35), wherein the first circuit unit is configured to generate a first area signal associated with an area of ​​the moving object based on the first signal.

[0308] (37) The light detector according to (36), wherein the light source controller is configured to control at least one of a light emitting position, a light density, or an output power of the light source based on the first area signal.

[0309] (38) The photodetector according to (36) or (37), wherein:

[0310] The second circuit unit includes a plurality of second counter sections, each of which is configured to count the number of pulses of the voltage signal based on the second photocurrent, and

[0311] The second circuit unit is configured to output a third signal based on the number of pulses of the voltage signal according to light emission of the light source.

[0312] (39) The photodetector according to any one of (36) to (38), wherein the second circuit unit is configured to output a third signal related to the distance to the moving target based on the first area signal. (40)

[0314] The light detector according to (39), wherein the light source controller is configured to control at least one of the light emitting position, light density or output power of the light source based on the third signal.

[0315] (41) A light detection system comprising:

[0316] light source;

[0317] a light detector; and

[0318] a light source controller configured to control the light source based on a signal output from the light detector, wherein

[0319] The light detectors include:

[0320] a plurality of first pixels, each comprising a first light receiving element configured to receive light and output a first photocurrent,

[0321] a first circuit unit configured to acquire a signal based on the first photocurrent and output a first signal related to the motion of the target and a second signal related to the grayscale value,

[0322] a plurality of second pixels, each of the second pixels including a second light receiving element configured to receive light and output a second photocurrent, and

[0323] The second circuit unit is configured to obtain a signal based on the second photocurrent and output a third signal related to the distance to the target.

[0324] It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.

[0325] [Reference Numbers List]

[0326] 1 Photodetector (light detection device)

[0327] 10a, 10b light receiving element

[0328] 20a, 20b Generating unit (generating circuit)

[0329] 25a, 25b supply unit (supply circuit)

[0330] 30a, 30b counter unit (counter circuit)

[0331] 35. Control unit (control circuit)

[0332] 40 determination unit (determination circuit)

[0333] 60-bit inversion unit (bit inversion circuit)

[0334] 70a, 70b Adding unit (adding circuit)

[0335] 75a, 75b storage unit

[0336] 100 pixel unit

[0337] 110 Processor

[0338] 120a, 120b pixel circuit part

[0339] 130a, 130b signal processing unit

[0340] 140a First circuit unit

[0341] 140b second circuit unit.

Claims

1. A light detection device, comprising: a first pixel including a first light receiving element configured to receive light; a first circuit coupled to the first light receiving element, and the first circuit is configured to output a first signal related to the motion of the target and a second signal related to the intensity value; a second pixel including a second light receiving element configured to receive light; as well as A second circuit is coupled to the second light receiving element, and the second circuit is configured to output a third signal related to the distance to the target.

2. The light detection device according to claim 1, wherein: The first light receiving element and / or the second light receiving element includes a single photon avalanche diode.

3. The light detection device according to claim 1, wherein: The first light receiving element is configured to receive visible light, and The second light receiving element is configured to receive invisible light.

4. The light detection device according to claim 1, wherein: The first circuit is configured to output the first signal based on a difference between the number of voltage signals within a first period and the number of voltage signals within a second period, the voltage signal being based on an output of the first light receiving element.

5. The light detection device according to claim 4, wherein: The first circuit is configured to output the second signal based on a sum of the number of voltage signals within the first period and the number of voltage signals within the second period.

6. The light detection device according to claim 4, wherein: The first circuit includes a first counter circuit configured to output a difference signal based on a difference between the number of voltage signals within the first period and the number of voltage signals within the second period, and The first counter circuit is configured to output a detection signal indicating that the number of voltage signals within the first period has reached a reference value.

7. The light detection device according to claim 6, wherein: The first counter circuit includes an up-down counter.

8. The light detection device according to claim 6, wherein: The first circuit includes a control circuit configured to control the first counting circuit, and The control circuit is configured to output a stop signal indicating an end of the second period to the first counting circuit based on the detection signal indicating an end of the first period.

9. The light detection device according to claim 8, wherein: The control circuit is configured to output the stop signal indicating an end of the second period to the first counter circuit to substantially equalize respective lengths of the first period and the second period.

10. The light detection device according to claim 8, wherein: The control circuit includes a timing generator.

11. The light detection device according to claim 1, comprising: a first generating circuit configured to generate a voltage signal based on an output of the first light receiving element; as well as The second generating circuit is configured to generate a voltage signal based on the output of the second light receiving element.

12. The light detection device according to claim 11, wherein The first generating circuit and / or the second generating circuit includes an inverter.

13. The light detection device according to claim 1, comprising: a first substrate including a plurality of the first light receiving elements and a plurality of the second light receiving elements; as well as The second substrate includes at least the first circuit and a part of the second circuit, and the first substrate is stacked on the second substrate.

14. The light detection device according to claim 1, further comprising: A light source controller is configured to control the light source.

15. The light detection device according to claim 14, wherein: The first circuit is configured to generate a first region signal associated with a region of the target based on the first signal.

16. The light detection device according to claim 15, wherein: The light source controller is configured to control at least one of a light emitting position, a light density, and an output power of the light source based on the first area signal.

17. The light detection device according to claim 15, wherein: The second circuit includes at least one second counter circuit configured to count the number of voltage signals based on the output of the second light receiving element, and The second circuit is configured to output the third signal based on the magnitude of the voltage signal.

18. The light detection device according to claim 15, wherein: The second circuit is configured to output the third signal related to the distance to the target based on the first area signal.

19. The light detection device according to claim 17, wherein: The light source controller is configured to control at least one of a light emitting position, a light density, and an output power of the light source based on the third signal.

20. The light detection device according to claim 1, wherein The light detection device includes a pixel array including the first pixel and the second pixel.

21. A light detection system comprising: light source; Light detection device; as well as a light source controller configured to control the light source based on a signal output from the light detection device, Wherein, the light detection device comprises: a first pixel including a first light receiving element configured to receive light, a first circuit coupled to the first light receiving element, and configured to output a first signal related to the motion of the target and a second signal related to the intensity value, a second pixel including a second light receiving element configured to receive light, and A second circuit is coupled to the second light receiving element, and the second circuit is configured to output a third signal related to the distance to the target.

Images