Optical detection device and ranging system

Abstract

[Problem] To achieve an increase in frame rate while expanding the distance measurement range and also improving the distance measurement accuracy. [Solution] An optical detection device comprising: a photoelectric conversion element that detects incident photons; a first measurement unit that measures a point in time at which the photoelectric conversion element detects photons; and a second measurement unit that, in parallel with the first measurement unit, measures the point in time with a lower time resolution than the first measurement unit.

Description

Optical detection device and ranging system

[0001] The present disclosure relates to a light detection device and a ranging system.

[0002] A distance measurement method known as the ToF (Time of Flight) method is known, in which light is emitted toward an object and the distance to the object is measured based on the light reflected from the object. The ToF method includes, for example, the dToF (direct ToF) method, which measures the distance to an object based on the time difference between the timing of light emission and the timing of light reception (i.e., the time of flight of light).

[0003] In dToF, for example, the distance to an object is measured by generating a histogram in which the time of flight of light is classified into bins for each unit time. Increasing the unit time per bin increases the distance measurement range but reduces the distance measurement accuracy. On the other hand, decreasing the unit time per bin improves the distance measurement accuracy but shortens the distance measurement range.

[0004] Therefore, a ranging system has been proposed that achieves both an expansion of the ranging range and improvement of ranging accuracy by performing ranging with a long unit time and ranging with a short unit time in a time-division manner (see Non-Patent Document 1).

[0005] Compact Real-Time Inter-Frame Histogram Builder for15-bit High-Speed ​​ToF-Imagers Based on Single-Photon Detection Ion Vornicu;Angela Darie;Ricardo Carmona-Galan;Angel Rodriguez-Vazquez, 2019

[0006] However, in the method of Non-Patent Document 1, distance measurement must be performed twice in a time-division operation, which deteriorates the frame rate.

[0007] Therefore, the present disclosure provides a light detection device and a ranging system that can improve the frame rate while simultaneously expanding the ranging range and improving the ranging accuracy.

[0008] In order to solve the above problems, the present disclosure provides a photodetection device including: a photoelectric conversion element that detects incident photons; a first measurement unit that measures the timing at which the photoelectric conversion element detects the photon; and a second measurement unit that measures the timing in parallel with the first measurement unit with a time resolution lower than that of the first measurement unit.

[0009] The first measurement section and the second measurement section may measure the timing in parallel in each of a plurality of frame periods.

[0010] The first measurement unit may measure the number of times the photon is detected for each of a plurality of first periods obtained by time-dividing the timing by a first time interval, and the second measurement unit may measure the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing by a second time interval longer than the first time interval.

[0011] The first measurement unit may have a plurality of first counters that measure the number of times the photoelectric conversion element detects a photon for each of the plurality of first periods, and the second measurement unit may have a plurality of second counters that measure the number of times the photoelectric conversion element detects a photon for each of the plurality of second periods.

[0012] The first measurement unit may include a first shift register having a plurality of first register circuits that shift a first pulse signal in synchronization with a first clock signal, the second measurement unit may include a second shift register having a plurality of second register circuits that shift a second pulse signal in synchronization with a second clock signal, the plurality of first counters may update the number of times in accordance with the timing at which the first pulse signal is output from the corresponding first register circuit, and the plurality of second counters may update the number of times in accordance with the timing at which the second pulse signal is output from the corresponding second register circuit.

[0013] The period of the first clock signal may be the period of a bin of a first histogram generated based on the measurement values ​​of the plurality of first counters, and the period of the second clock signal may be the period of a bin of a second histogram generated based on the measurement values ​​of the plurality of second counters.

[0014] The second clock signal may be the first pulse signal output from the first register circuit in the first stage of the first shift register.

[0015] The second clock signal may be a signal obtained by dividing the first clock signal by a division ratio corresponding to the number of the plurality of first counters, and synchronizing the third clock signal with the first pulse signal output from the first register circuit of the first stage of the first shift register.

[0016] The apparatus may include: a macro pixel including one or more pixels each having one or more of the photoelectric conversion elements; and a photon detection signal generation unit that outputs a photon detection signal indicating that at least one of the photoelectric conversion elements in the macro pixel has detected a photon, wherein the plurality of first counters and the plurality of second counters may update the number of times in accordance with the timing at which the photon detection signal is output.

[0017] The plurality of first counters may update the number of times when the first pulse signal is output from the corresponding first register circuit and the photon detection signal is output, and the plurality of second counters may update the number of times when the second pulse signal is output from the corresponding second register circuit and the photon detection signal is output.

[0018] The photon detection signal generation unit may include an adder that adds up the numbers of photons detected by one or more photoelectric conversion elements in the macro pixel, and a bit selection unit that selects a bit according to the value added by the adder.

[0019] The plurality of first counters and the plurality of second counters may update the number of times in accordance with the bit selected by the bit selection section.

[0020] The image sensor may include a plurality of macro pixels, each of which serves as a distance measurement point.

[0021] The apparatus may further include a third measurement unit that operates in parallel with the first measurement unit and the second measurement unit and measures the timing with a higher time resolution than the first measurement unit.

[0022] The third measurement unit may measure the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing into third time intervals that are shorter than the first time intervals.

[0023] The photoelectric conversion element may further include a first quench circuit having a passive element connected between one end of the photoelectric conversion element and a reference voltage node, and the first quench circuit may restore the voltage at the one end of the photoelectric conversion element to an initial voltage by a current flowing from the reference voltage node through the passive element to the one end of the photoelectric conversion element after the photoelectric conversion element detects a photon.

[0024] The photoelectric conversion element may further include a second quench circuit having an active element connected between one end of the photoelectric conversion element and a reference voltage node, and the second quench circuit may restore the voltage at the one end of the photoelectric conversion element to an initial voltage by controlling the active element after the photoelectric conversion element detects a photon.

[0025] The image sensor may include a pixel array unit including one or more first pixels that acquire gradation information and one or more second pixels that are used for distance measurement, and each of the first pixels and the second pixels may have one or more of the photoelectric conversion elements.

[0026] The liquid crystal display device may include a color filter arranged to overlap the pixel array portion in a planar view, the color filter including: a first color filter layer that transmits visible light and is provided in correspondence with one or more of the first pixels that acquire gradation information; and a second color filter layer that transmits near-infrared light or infrared light and is provided in correspondence with the second pixels.

[0027] The present disclosure also provides a ranging system comprising: a light detection device that receives a reflected light pulse signal that is a light pulse signal reflected by an object; a light emitting device that emits the light pulse signal; and a ranging unit that measures the distance to the object based on the reflected light pulse signal.

[0028] 15. A block diagram showing a ranging system according to a first embodiment of the present disclosure. A block diagram showing a configuration of a photodetector according to a first embodiment of the present disclosure. A circuit diagram showing a configuration of a pixel according to the first embodiment of the present disclosure. A timing chart showing the operation of the photodetector of FIG. 2. A diagram showing the relationship between two selection signals and the time of flight of light. A circuit diagram showing a detailed configuration of a high-resolution measurement unit. A circuit diagram showing a detailed configuration of a low-resolution measurement unit. A circuit diagram showing a configuration of a low-resolution measurement unit according to a modified example. A diagram showing a stacked structure of a ranging system according to an embodiment of the present disclosure. A diagram showing a stacked structure of a ranging system according to a modified example. A block diagram showing an example layout of a pixel array unit according to the first embodiment of the present disclosure. A block diagram showing an example layout of a counter. A circuit diagram showing a configuration of a pixel according to a modified example. A block diagram showing the configuration of a photodetector using the pixel of FIG. 11. A timing chart showing the operation of the photodetector of FIG. 12. A circuit diagram showing the configuration of a measurement unit applied to the photodetector of FIG. 12. A block diagram showing the configuration of a photodetector according to a modified example of FIG. 12. A timing chart showing the operation of the photodetector of FIG. 15. A block diagram showing the basic configuration of a pixel array unit. A block diagram of a pixel array unit according to a first modified example. 21. A block diagram of a pixel array unit according to a second modified example. A block diagram of a pixel array unit according to a third modified example. A cross-sectional view of a ranging system according to a first embodiment of the present disclosure. A cross-sectional view of a ranging system according to a first modified example. A cross-sectional view of a ranging system according to a second modified example. A cross-sectional view of a ranging system according to a third modified example. A diagram for explaining a ranging method according to a first comparative example. A diagram for explaining a ranging method according to the first embodiment of the present disclosure. A timing chart for explaining a ranging method according to a second comparative example. A timing chart for explaining a ranging method according to the first embodiment of the present disclosure. A block diagram showing a photodetector according to a second embodiment of the present disclosure. A timing chart showing the operation of the photodetector of FIG. 21. A block diagram showing an example of a schematic configuration of a vehicle control system. An explanatory diagram showing an example of installation positions of an outside vehicle information detection unit and an imaging unit.

[0029] Hereinafter, embodiments of a light detection device and a distance measurement system will be described with reference to the drawings. The following description will focus on the main components of the light detection device and the distance measurement system, but the light detection device and the distance measurement system may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0030] First Embodiment Fig. 1 is a block diagram showing a ranging system 1 including a light detection device 10 according to a first embodiment of the present disclosure. The ranging system 1 of Fig. 1 measures the distance to a measurement target object using the dToF method and can be mounted on, for example, an in-vehicle LiDAR. The ranging system 1 may also be used for focusing telephoto photography in a photography camera, as a sensor for an autonomous vehicle, or as a geological sensor from an aircraft. The ranging system 1 includes a light detection device 10, a light emitting device 2, a control unit 3, and a ranging unit 4.

[0031] The light detection device 10 receives a reflected light pulse (RX: Received exchange) signal that is a light pulse (TX: Transmitter exchange) signal emitted by the light emitting device 2 and reflected by the object 5 to be measured. The light detection device 10 detects the timing at which the reflected light pulse signal is received.

[0032] Furthermore, the photodetector 10 acquires information on the light emission timing from the light-emitting device 2. The photodetector 10 generates a histogram having information on the time-of-flight of light based on the light emission timing of the light-emitting device 2 and the light reception timing of the photodetector 10. More specifically, the photodetector 10 receives, multiple times, reflected light pulse signals based on light pulse signals emitted multiple times by the light-emitting device 2. The photodetector 10 generates a histogram by dividing the time-of-flight of light based on the repeatedly received reflected light pulse signals into bins for each unit time.

[0033] The light emitting device 2 intermittently emits a light pulse signal toward an object. The control unit 3 performs, for example, synchronization control to synchronize the light emission timing of the light emitting device 2 with the light reception timing of the light detecting device 10.

[0034] The distance measurement unit 4 measures the distance between the object 5 and the distance measurement system 1 based on the histogram generated by the light detection device 10. The distance measurement unit 4 extracts peaks from the histogram and performs a centroid calculation to calculate a centroid value of the time of flight of light. The distance measurement unit 4 can calculate the distance between the object 5 and the distance measurement system 1 from the centroid value.

[0035] 2 is a block diagram showing the configuration of the photodetector 10 according to the first embodiment of the present disclosure. The photodetector 10 includes a pixel array unit 11 and a pixel circuit 12. A plurality of pixels 20 are arranged in the pixel array unit 11.

[0036] 3 is a circuit diagram showing the configuration of the pixel 20 according to the first embodiment of the present disclosure. The pixel 20 includes a photoelectric conversion element 30, a pulse generation circuit 31, and a quench circuit (first quench circuit) 32.

[0037] The photoelectric conversion element 30 receives the reflected pulse signal and detects the incident photons. The photoelectric conversion element 30 is, for example, a single photon avalanche diode (SPAD). The SPAD amplifies the charge generated based on one photon by avalanche multiplication to generate a large current. This causes the cathode voltage VK of the SPAD to drop rapidly. The voltage fluctuation allows the photoelectric conversion element 30 to detect photons one by one.

[0038] The pulse generation circuit 31 generates a pulse signal PIX_PLS based on a decrease in the cathode voltage VK of the photoelectric conversion element 30. The pulse generation circuit 31 includes an inverter 33, a latch circuit 34, and a delay circuit 35. The inverter 33 inverts the cathode voltage VK and inputs it to the latch circuit 34. The latch circuit 34 holds the voltage input to the inverter 33 and outputs a signal PIX_PLS of a predetermined level (e.g., a high level) based on the held voltage. The delay circuit 35 delays the timing at which the latch circuit 34 outputs a high-level signal, and sets the output signal PIX_PLS of the latch circuit 34 to a low level. This allows the pulse generation circuit 31 to output the pulse signal PIX_PLS.

[0039] The quench circuit 32 resets the photoelectric conversion element 30. The quench circuit 32 includes a passive element 36. The passive element 36 is, for example, a resistor element. The passive element 36 is connected between the cathode of the photoelectric conversion element 30 and a reference voltage node. After the photoelectric conversion element 30 detects a photon, the quench circuit 32 supplies a current from the reference voltage node to the cathode of the photoelectric conversion element 30 via the passive element 36, thereby restoring the cathode voltage VK of the photoelectric conversion element 30 to the initial voltage.

[0040] In this specification, the pixel 20 in which the photoelectric conversion element 30 is passively reset by the passive element 36 is also referred to as a passive pixel.

[0041] As described above, when the pixel 20 detects a photon, it can transmit the detection of the photon to the pixel circuit 12 in the subsequent stage by outputting a high-level pulse signal PIX_PLS.

[0042] 2 , the pixel circuit 12 detects the timing at which one or more photoelectric conversion elements 30 detect a photon. The pixel circuit 12 includes a high-resolution measurement unit (Fine TDC or first measurement unit) 13 and a low-resolution measurement unit (Coarse TDC or second measurement unit) 14.

[0043] A photon detection signal PIX_OUT based on the pulse signal PIX_PLS of the pixel 20 is input to the high-resolution measurement unit 13 and the low-resolution measurement unit 14. As a result, the high-resolution measurement unit 13 and the low-resolution measurement unit 14 measure the timing at which the photoelectric conversion element 30 detects a photon in parallel. The high-resolution measurement unit 13 and the low-resolution measurement unit 14 each include a TDC (Time to Digital Converter), for example. The high-resolution measurement unit 13 measures the detection timing of the photon with a higher time resolution than the low-resolution measurement unit 14.

[0044] The high-resolution measurement unit 13 has a plurality of counters (first counters) 21 and a shift register (first shift register) 22. Each of the plurality of counters 21 counts the number of times a photon is detected by the photoelectric conversion element 30. When a photon detection signal PIX_OUT (described later) is input, the shift register 22 outputs a selection signal (first pulse signal) SELBIN_FINE that selects one of the plurality of counters 21 that will count the number of times.

[0045] In this specification, an example will be described in which the shift register 22 outputs a 12-bit selection signal SELBIN_FINE[11:0] corresponding to 12 counters 21 .

[0046] The shift register 22 shifts the selection signal SELBIN_FINE in synchronization with a clock signal (first clock signal) BIN_CLK.

[0047] The low-resolution measurement unit 14 has a plurality of counters (second counters) 23 and a shift register (second shift register) 24. Each of the plurality of counters 23 counts the number of times a photon is detected by the photoelectric conversion element 30. When a photon detection signal PIX_OUT is input, the shift register 24 outputs a selection signal (second pulse signal) SELBIN_COARSE that selects a counter 23 from the plurality of counters 23 that is to count the number of times.

[0048] In this specification, an example will be described in which the shift register 24 outputs a 4-bit selection signal SELBIN_COARSE[3:0] corresponding to four counters 23 .

[0049] A predetermined bit (second clock signal) of the selection signal SELBIN_FINE is input as a clock signal to the shift register 24. For example, SELBIN_FINE[0], which is the least significant bit of the selection signal SELBIN_FINE, is input to the shift register 24. The shift register 24 shifts the selection signal SELBIN_COARSE in synchronization with the clock signal SELBIN_FINE[0].

[0050] The high-resolution measurement unit 13 and the low-resolution measurement unit 14 may measure the number of times photons are detected for each macro pixel 15 made up of multiple pixels 20. The pixel array unit 11 can be arranged with multiple macro pixels 15, each of which serves as a single ranging point.

[0051] The pixel circuit 12 may be provided with an aggregation unit (photon detection signal generation unit) 16 that aggregates the pulse signals PIX_PLS output by the multiple pixels 20 for each macro pixel 15. The aggregation unit 16 in Fig. 2 has a logic circuit 25. The logic circuit 25 is, for example, an OR circuit.

[0052] The logic circuit 25 outputs, for example, a high-level photon detection signal PIX_OUT when at least one of the multiple pixels 20 in the macro pixel 15 outputs a high-level pulse signal PIX_PLS. This allows the high-resolution measurement unit 13 and the low-resolution measurement unit 14 to count the number of detections by determining that a photon has been detected in the macro pixel 15 when at least one of the multiple pixels 20 in the macro pixel 15 detects a photon.

[0053] The aggregation unit 16 enables the photodetector 10 to measure the number of photon detections by thinning out a plurality of pixels 20 in the macro pixel 15. This eliminates the need to provide a high-resolution measurement unit 13 and a low-resolution measurement unit 14 for each pixel 20, thereby reducing the circuit area of ​​the photodetector 10.

[0054] The pulse signal PIX_PLS of the pixel 20 may be input as a photon detection signal to the high-resolution measurement unit 13 and the low-resolution measurement unit 14. In this case, the high-resolution measurement unit 13 and the low-resolution measurement unit 14 can measure the number of times photons are detected for each pixel 20.

[0055] The operation of the high-resolution measurement unit 13 and the low-resolution measurement unit 14 will be described using Figures 4A and 4B. Figure 4A is a timing chart showing the operation of the photodetector 10 of Figure 2. Figure 4A illustrates the light pulse signal LASER output by the light-emitting device 2, the cathode voltage VK of the photoelectric conversion element 30, the photon detection signal PIX_OUT, the clock signal BIN_CLK that drives the high-resolution measurement unit 13, the selection signal SELBIN_FINE[11:0], the clock signal SELBIN_FINE[0] that drives the low-resolution measurement unit 14, and the selection signal SELBIN_COARSE[3:0]. Figure 4A also illustrates the time of flight of light that can be measured by the high-resolution measurement unit 13 and the low-resolution measurement unit 14. The horizontal axis of Figure 4A is the time axis.

[0056] 4A, the light emitting device 2 outputs a light pulse signal LASER during a predetermined light emission period TL. As a result, in the example of FIG. 4A, the photoelectric conversion element 30 detects photons at times t1 and t2. At times t1 and t2, the cathode voltage VK of the photoelectric conversion element 30 decreases, and a high-level photon detection signal PIX_OUT is output.

[0057] The clock signal BIN_CLK is, for example, a signal that goes high at regular intervals. Each time the clock signal BIN_CLK goes high, the shift register 22 shifts the bits of the selection signal SELBIN_FINE[11:0] that go high. As a result, as shown in FIG. 4A , the selection signal SELBIN_FINE goes high sequentially from bit 0 to bit 11 at regular intervals. After the 11th bit of the selection signal SELBIN_FINE goes high, the shift register 22 again shifts the bits that go high sequentially, starting from bit 0.

[0058] 4A illustrates an operation in which each bit of the selection signal SELBIN_FINE goes high in turn. Also, in FIG. 4A, each bit of the selection signal SELBIN_FINE[11:0] (i.e., counting from 0) is represented as 1 to 12 (i.e., counting from 1) in correspondence with the configuration of the high-resolution measurement unit 13 (see FIG. 5), which will be described later.

[0059] That is, the shift register 22 can select the counters 21 corresponding to the respective bits of the selection signal SELBIN_FINE in turn.

[0060] At the timing when a high-level photon detection signal PIX_OUT is output, the high-resolution measurement unit 13 causes one of the counters 21 selected by the shift register 22 to count up the number of times a photon has been detected.

[0061] As described above, the high-resolution measurement unit 13 can count the number of times photons are detected for each of a plurality of periods (first periods) Ta obtained by dividing the time by intervals (first time intervals or unit times) Δt at which the clock signal BIN_CLK goes high, using the corresponding counter 21. Note that the period Ta is the period of one cycle from when the least significant bit of the selection signal SELBIN_FINE goes high once until the least significant bit goes high again.

[0062] The length of the period Ta can be expressed as the number of bits Na of SELBIN_FINE × Δt. In the example of Fig. 4A, since SELBIN_FINE has 12 bits (Na = 12), Ta = 12Δt.

[0063] Furthermore, the clock signal SELBIN_FINE[0] goes high every time the clock signal SELBIN_FINE[0] goes high, the shift register 24 shifts the bit of the selection signal SELBIN_COARSE[3:0] that goes high.

[0064] That is, the shift register 24 can select the counters 23 corresponding to each bit of the selection signal SELBIN_COARSE in turn, similar to the shift register 22. Furthermore, the low-resolution measurement unit 14 causes the counter 23 selected by the shift register 24 to count up the number of photon detections at the timing when a high-level photon detection signal PIX_OUT is output.

[0065] In FIG. 4A, the 0th to 3rd bits of the selection signal SELBIN_COARSE[3:0] are represented as 1' to 4' in correspondence with the configuration of the low-resolution measurement unit 14 (see FIG. 6) described below.

[0066] As described above, the low-resolution measurement unit 14 can count the number of times a photon is detected for each of a plurality of periods (second periods) Tb obtained by dividing the period Ta (second time interval), which is the interval at which the clock signal SELBIN_FINE[0] goes high. The period Tb is the period of one cycle from when the least significant bit of the selection signal SELBIN_COARSE goes high once until the least significant bit goes high again.

[0067] The length of the period Tb can be expressed as the number of bits Nb×Ta of the selection signal SELBIN_COARSE. In the example of Fig. 4A, the selection signal SELBIN_COARSE is 4 bits (Nb=4), so Tb=4×Ta=48Δt.

[0068] In this specification, the period Tb is also referred to as one frame period. During one frame period, the photodetector 10 can measure the photon detection timing for one emission of the optical pulse signal.

[0069] 4A, the 0th bit of the selection signal SELBIN_FINE and the 0th bit of the selection signal SELBIN_COARSE are at high level, so that the counter 21 corresponding to the 0th bit and the counter 23 corresponding to the 0th bit each count up the number of times a photon has been detected.

[0070] Similarly, at time t2, the counter 21 corresponding to the 0th bit and the counter 23 corresponding to the 1st bit each count up the number of times a photon has been detected.

[0071] In this specification, the number of photon detections counted by each of the multiple counters 21 and the number of photon detections counted by each of the multiple counters 23 are also referred to as bin data. The counter 21 or counter 23 corresponding to the x-th bit can measure the x-th bin data.

[0072] The number of bits Na of the selection signal SELBIN_FINE and the number of bits Nb of the selection signal SELBIN_COARSE can be rephrased as the numbers of bins Na and Nb of the high-resolution measurement unit 13 and the low-resolution measurement unit 14, respectively.

[0073] The photon detection timing can be measured by determining which bin data the number of photon detections is counted in. In the photodetector 10, the high-resolution measurement unit 13 and the low-resolution measurement unit 14 each measure the photon detection timing.

[0074] The ranging system 1 emits an optical pulse signal multiple times, measures the number of photon detections in each of multiple frame periods, and generates a histogram having the above-mentioned bin data. The time-of-flight of light can be measured from the bin data in the histogram that has the greatest number of photon detections (i.e., the peak). The time-of-flight of light is the time from when the light-emitting device 2 emits an optical pulse signal to when the photodetector 10 receives the reflected optical pulse signal.

[0075] The high-resolution measurement unit 13 and the low-resolution measurement unit 14 measure photon detection timing in parallel as described above during each of a plurality of frame periods, and each generate measurement data in parallel. The photodetector 10 generates a high-resolution histogram (first histogram) based on the measurement data from the high-resolution measurement unit 13, and a low-resolution histogram (second histogram) based on the measurement data from the low-resolution measurement unit 14. The bin period of the high-resolution histogram is the period of the clock signal BIN_CLK. The bin period of the low-resolution histogram is the period of the clock signal SELBIN_FINE[0].

[0076] The light detection device 10 and the distance measurement unit 4 can measure the time of flight of light with the same time resolution as the high-resolution measurement unit 13 using the high-resolution histogram and the low-resolution histogram.

[0077] Specifically, the distance measurement unit 4 can roughly calculate the time of flight of light by extracting the peak of the low-resolution histogram. If the xth bin data of the low-resolution histogram is the peak, the time of flight of light can be calculated to be greater than or equal to x × Ta and less than (x + 1) × Ta. Using the bin number Na of the high-resolution measurement unit 13, the time of flight of light can be expressed as greater than or equal to x × Na × Δt and less than (x + 1) × Na × Δt.

[0078] By extracting the peak of the high-resolution histogram, the distance measurement unit 4 can calculate a more precise time-of-flight of light from the range of the time-of-flight of light calculated as described above. If the y-th bin data of the high-resolution histogram is the peak, the time-of-flight of light can be calculated to be greater than or equal to x×Na×Δt+y×Δt and less than x×Na×Δt+(y+1)×Δt.

[0079] 4B is a diagram showing the relationship between the selection signals SELBIN_FINE and SELBIN_COARSE and the time of flight of light that can be measured by the distance measurement unit 4. For example, if a peak is detected in the bin corresponding to the first bit of the selection signal SELBIN_FINE and a peak is detected in the bin corresponding to the third bit of the selection signal SELBIN_COARSE, the time of flight of light can be calculated from the above formula to be equal to or greater than 37Δt and less than 38Δt.

[0080] 4A can expand the distance (ranging range) that can be measured by the ranging unit 4. In ranging using a single measurement unit (TDC) with N bins, typically, ranging can only be performed up to a distance where the flight time of light is N×Δt.

[0081] To expand the ranging range, it is necessary to increase the number of bins N or the unit time Δt. Increasing the number of bins N requires an increase in the number of counters, which increases the circuit area of ​​the photodetector. Increasing the unit time Δt reduces the time resolution of the ranging section, resulting in a deterioration in ranging accuracy. In particular, the ranging accuracy deteriorates when the object to be measured and the ranging system 1 are close to each other.

[0082] In contrast, the technique of Fig. 4A uses a high-resolution measurement unit 13 with a bin number Na and a low-resolution measurement unit 14 with a bin number Nb, making it possible to measure distances up to a distance where the time of flight of light is Na × Nb × Δt. In the example of Fig. 4A, it is possible to measure distances up to a distance where the time of flight of light is 48Δt.

[0083] 4A , the circuit area of ​​the photodetector 10 is sufficient if it corresponds to the number of bins (Na+Nb) of the high-resolution measurement unit 13 and the low-resolution measurement unit 14. In some cases, the photodetector 10 can ensure a larger ranging range than the above-mentioned ranging range corresponding to N×Δt with a number of bins smaller than the above-mentioned number of bins N. Furthermore, there is no need to increase the unit time Δt compared to ranging using a single measurement unit.

[0084] That is, the technique of FIG. 4A can expand the distance measurement range without significantly increasing the circuit area or reducing the time resolution of distance measurement.

[0085] The distance measuring unit 4 can calculate the time of flight of light with a time resolution higher than the unit time Δt by calculating the center of gravity.

[0086] 5 is a circuit diagram showing a detailed configuration of the high-resolution measurement unit 13. In FIG. 5, a shift register 22 and a plurality of counters 21 are shown.

[0087] Each of the counters 21 has a plurality of latch circuits 41. Each of the latch circuits 41 can hold one bit of information. The latch circuits 41 can hold the number of photon detections counted by the counters 21 as multi-bit data.

[0088] The shift register 22 includes a plurality of register circuits (first register circuits) 42. The plurality of register circuits 42 output signals corresponding to the respective bits of the selection signal SELBIN_FINE. The plurality of register circuits 42 are configured to shift the selection signal SELBIN_FINE in synchronization with the clock signal BIN_CLK.

[0089] 5, the plurality of register circuits 42 are denoted by 1 to 12 in order from the first stage side. The register circuits 1 to 12 correspond to the 0th to 11th bits of the selection signal SELBIN_FINE[11:0], respectively.

[0090] Specifically, a clock signal BIN_CLK is input to each of the plurality of register circuits 42. Among the plurality of register circuits 42, the register circuit 42 corresponding to the least significant bit of the selection signal SELBIN_FINE is connected to a selection circuit 43. The selection circuit 43 selects and outputs the initialization signal INIT or the output signal from the register circuit 42 corresponding to the most significant bit of the selection signal SELBIN_FINE. Among the plurality of register circuits 42, the register circuit 42 other than the register circuit 42 corresponding to the least significant bit receives the output signal of the register circuit 42 corresponding to the immediately preceding bit.

[0091] As a result, for example, when a high-level initialization signal INIT is input, a high-level selection signal SELBIN_FINE[0] is output from the register circuit 42 corresponding to the least significant bit. Subsequently, when a high-level clock signal BIN_CLK is output, the selection signal SELBIN_FINE[0] goes low, and a high-level selection signal SELBIN_FINE[1] is output from the register circuit 42 corresponding to the bit immediately above the least significant bit. Thereafter, each bit of the selection signal SELBIN_FINE goes high in turn each time a high-level clock signal BIN_CLK is output.

[0092] The number of register circuits 42 arranged corresponds to the number of bits of the selection signal SELBIN_FINE. Also, a plurality of counters 21 are arranged for each register circuit 42. Fig. 5 shows an example in which 12 register circuits 42 and 12 counters 21 are arranged according to the 12-bit selection signal SELBIN_FINE.

[0093] A logic circuit 44 is arranged between each of the pairs of register circuits 42 and the counter 21. That is, in the example of FIG. 5, 12 logic circuits 44 are arranged. The logic circuits 44 are, for example, AND circuits. A signal of a corresponding bit of the selection signal SELBIN_FINE and a photon detection signal PIX_OUT are input to each of the plurality of logic circuits 44. When a high-level photon detection signal PIX_OUT and a high-level signal of a corresponding bit of the selection signal SELBIN_FINE are input to the logic circuit 44, the logic circuit 44 outputs a high-level signal and instructs the counter 21 to update the number of photon detections (i.e., count up).

[0094] That is, the counters 21 update the number of photon detections when a high-level photon detection signal PIX_OUT is output and when a signal of a corresponding bit of a high-level selection signal SELBIN_FINE is output from the corresponding register circuit 42.

[0095] As described above, the first-stage register circuit 42 among the plurality of register circuits 42 outputs the clock signal SELBIN_FINE[0] that drives the low-resolution measurement unit 14 .

[0096] Fig. 6 is a circuit diagram showing a detailed configuration of the low-resolution measurement unit 14. Fig. 6 illustrates a shift register 24 and a plurality of counters 23. Similar to the counter 21 in Fig. 5, the counter 23 has a plurality of latch circuits 51 that hold the number of times that a photon has been detected as multi-bit data.

[0097] The shift register 24 has a plurality of register circuits (second register circuits) 52, similar to the shift register 22 in FIG. 5 . The plurality of register circuits 52 output signals for each bit of the corresponding selection signal SELBIN_COARSE. The selection signal SELBIN_FINE[0] is input to each of the plurality of register circuits 52 as a clock signal. A selection circuit 53 is connected to the register circuit 52 corresponding to the least significant bit. The selection circuit 53 receives the output signal of the register circuit 52 corresponding to the most significant bit and the initialization signal INIT.

[0098] A logic circuit 54 is arranged between each of the pairs of register circuits 52 and counters 23. Fig. 6 shows an example in which four register circuits 52, four logic circuits 54, and four counters 23 are arranged in accordance with a 4-bit selection signal SELBIN_COARSE. In Fig. 6, the multiple register circuits 52 are denoted by 1' to 4' in order from the first stage.

[0099] The plurality of counters 23 update the number of photon detections in synchronization with the timing at which the corresponding register circuit 52 outputs a signal of a corresponding bit of the selection signal SELBIN_COARSE at a high level.

[0100] Fig. 7 is a circuit diagram showing the configuration of a low-resolution measurement unit 14a according to one modification. The low-resolution measurement unit 14a in Fig. 7 includes a frequency divider circuit 61 and a latch circuit 62. The low-resolution measurement unit 14a in Fig. 7 differs from the low-resolution measurement unit 14 in Fig. 6 in that a clock signal (second clock signal) COARSE_CLK is input to the shift register 22 via the latch circuit 62.

[0101] The frequency divider circuit 61 divides the clock signal BIN_CLK to match the drive cycle of the low-resolution measurement unit 14a, and outputs a clock signal (third clock) DIV_CLK. For example, in the example of FIG. 4A, the selection signal SELBIN_COARSE is shifted every 12 times the clock signal BIN_CLK goes high. In this example, the frequency divider circuit 61 divides the clock signal BIN_CLK by 1 / 12.

[0102] In other words, the clock signal DIV_CLK is divided by a division ratio according to the number of counters 21 (or the number of register circuits 42) in the high-resolution measurement unit 13.

[0103] The latch circuit 62 latches the clock signal DIV_CLK every time the selection signal SELBIN_FINE[0] is input, and outputs the clock signal COARSE_CLK, which is a signal obtained by synchronizing the latched clock signal DIV_CLK with the selection signal SELBIN_FINE[0].

[0104] By the low-resolution measurement unit 14a latching the clock signal DIV_CLK, the low-resolution measurement unit 14a can be driven separately from the high-resolution measurement unit 13. For example, the high-resolution measurement unit 13 can stop outputting the selection signal SELBIN_FINE[0] at any timing, regardless of whether the low-resolution measurement unit 14a is driven or not.

[0105] The ranging system 1 can be configured, for example, by a laminated substrate in which multiple substrates (chips) are stacked. FIG. 8A is a diagram showing the laminated structure of the ranging system 1 according to an embodiment of the present disclosure. The ranging system 1 in FIG. 8A has a two-layer structure in which a first substrate (pixel chip) 71 and a second substrate (logic chip) 72 are bonded together in this order. These substrates are bonded together by vias or the like. Note that the first substrate 71 and the second substrate 72 may be bonded together by Cu-Cu bonding or bumps, in addition to vias.

[0106] For example, a plurality of pixels 20 are arranged on the first substrate 71. For example, pixel circuits 12, a control unit 3, a distance measurement unit 4, etc. are arranged on the second substrate 72. Some of the components of the pixels 20 (for example, the pulse generation circuit 31) may be arranged on the second substrate 72.

[0107] The light emitting device 2 may be disposed on the first substrate 71. The light emitting device 2 may be disposed on a substrate different from the first substrate 71 and the second substrate 72. Furthermore, the substrate on which the light emitting device 2 is disposed does not have to be stacked on the first substrate 71 and the second substrate 72.

[0108] Fig. 8B is a diagram showing the layered structure of a ranging system 1a according to one modified example. The ranging system 1a in Fig. 8B has a three-layer structure formed by bonding together a first substrate (pixel chip) 71, a second substrate (first logic chip) 72, and a third substrate (second logic chip) 73 in this order. For example, a plurality of pixels 20 are arranged on the first substrate 71. For example, a pixel circuit 12 is arranged on the second substrate 72. For example, a control unit 3 and a ranging unit 4 are arranged on the third substrate 73.

[0109] In the ranging system 1a, the area of ​​the pixel circuit 12 can be increased by arranging the control unit 3, the ranging unit 4, etc. on the third substrate 73. This allows the number of counters 21 in the high-resolution measurement unit 13 and the number of counters 23 in the low-resolution measurement unit 14 to be increased. Increasing the number of counters 21 and 23 improves the time resolution of the high-resolution measurement unit 13 and the low-resolution measurement unit 14, enabling more accurate ranging. Alternatively, a larger ranging range can be measured. Furthermore, increasing the number of latch circuits 41 and 51 in the counters 21 and 23 increases the number of photon detections that the counters 21 and 23 can count, enabling more accurate ranging even when the light intensity received by the pixel 20 is high. Furthermore, the areas of the first substrate 71, the second substrate 72, and the third substrate 73 may be reduced to miniaturize the ranging system 1a.

[0110] The components arranged on each board are not limited to those described above. The distance measurement system 1 may be configured with four or more stacked boards, or may be configured with a single flat board.

[0111] 9 is a block diagram showing an example of the layout of the pixel array unit 11 according to the first embodiment of the present disclosure. The pixel array unit 11 has a plurality of macro pixels 15. The macro pixels 15 are, for example, arranged in a two-dimensional direction. Each macro pixel 15 has a plurality of pixels 20 arranged in the two-dimensional direction. While FIG. 9 shows an example in which the macro pixel 15 is composed of a plurality of pixels 20 adjacent to each other, the present invention is not limited to this, and the macro pixel 15 may also include pixels 20 arranged at a distance.

[0112] Fig. 10 is a block diagram showing an example of the layout of the counters 21 and 23. Fig. 10 illustrates a counter area 80 having multiple counters 21 and 23. The counter area 80 is used to measure the number of photon detections for one macro pixel 15. Multiple counter areas 80 corresponding to the macro pixels 15 are arranged in the pixel circuit 12. Note that the counter area 80 includes register circuits 42 and 52 corresponding to the multiple counters 21 and 23, respectively. Hereinafter, the counters 21 and the register circuits 42 may be simply referred to as counters 21. Furthermore, the counters 23 and the register circuits 52 may be simply referred to as counters 23.

[0113] The macro pixel 15 in Fig. 9 is disposed, for example, on the first substrate 71 in Fig. 8A or 8B. The counter region 80 in Fig. 10 is disposed, for example, on the second substrate 72. The multiple pixels 20 in the macro pixel 15 and the multiple counters 21 and 23 in the counter region 80 may be disposed in correspondence with each other. For example, one counter 21 or one counter 23 may be disposed directly below one pixel 20.

[0114] 9, the macro pixel 15 has 16 pixels 20. The counter area 80 in FIG. 10 has a total of 16 counters 21 and 23 associated with the macro pixel 15 in FIG.

[0115] The number and arrangement of the counters 21 and 23 within the counter area 80 are arbitrary.

[0116] 10 , the counter region 80 does not necessarily have to have the total number of counters 21 and 23 equal to the number of pixels 20 in the macro pixel 15. In addition, the arrangement of the counters 21 and 23 does not have to correspond to the pixels 20 in the macro pixel 15, and the counters 21 and 23 do not have to be arranged in the same direction as the pixels 20 in the macro pixel 15.

[0117] The high-resolution measurement unit 13 and the low-resolution measurement unit 14 according to an embodiment of the present disclosure can also be applied to pixels having a configuration different from the passive pixel of Fig. 3. Fig. 11 is a circuit diagram showing the configuration of a pixel 20a according to one modification. The pixel 20a has a photoelectric conversion element 30, a pulse generation circuit 31a, and a quench circuit (second quench circuit) 32a.

[0118] The quench circuit 32a includes an active element 81. The active element 81 is, for example, a transistor. The active element 81 is connected between the cathode of the photoelectric conversion element 30 and a reference voltage node. The active element 81 is turned on, for example, when a low-level signal XRCG is supplied to the gate. When the active element 81 is on, it supplies the voltage of the reference voltage node to the cathode of the photoelectric conversion element 30, thereby restoring the cathode voltage VK to the initial voltage after the photoelectric conversion element 30 detects a photon.

[0119] 3, the pulse generation circuit 31a includes an inverter 33 that inverts the cathode voltage of the photoelectric conversion element 30 and a latch circuit 34 that holds the output voltage of the inverter 33. The pulse generation circuit 31a also includes a logic circuit (e.g., an AND circuit) 82. A signal BINEN for generating a pulse signal is input to the latch circuit 34 and the logic circuit 82. The latch circuit 34 holds the output voltage of the inverter 33 when, for example, a high-level signal BINEN is input. The logic circuit 82 outputs the output voltage of the latch circuit 34 while, for example, the high-level signal BINEN is continuously input. This allows the pulse generation circuit 31a to output a pulse signal PIX_PLS that is high while the high-level signal BINEN is input.

[0120] 11 can actively reset the photoelectric conversion element 30 and output the pulse signal PIX_PLS by using the signals XRCG and BINEN. In this specification, the pixel 20a in FIG. 11 is also referred to as an active pixel. Furthermore, the signals XRCG and BINEN can synchronously drive multiple pixels 20a.

[0121] Fig. 12 is a block diagram showing the configuration of a photodetector 10a using the pixel 20a of Fig. 11. A plurality of pixels 20a are arranged in the pixel array section 11 of Fig. 12. Also, a plurality of macro pixels 15a, each having a plurality of pixels 20a, are arranged in the pixel array section 11. In this specification, an example will be described in which one macro pixel 15a is composed of 16 pixels 20a, similar to the macro pixel 15 of Fig. 9.

[0122] The summing unit (photon detection signal generating unit) 16a in Fig. 12 has an adder 85 and a bit selector 86. The adder 85 adds pulse signals PIX_PLS output from multiple pixels 20a in the macro pixel 15a. The bit selector 86 converts the pulse signal PIX_PLS added by the adder 85 into a photon detection signal BIT_SEL. The photon detection signal BIT_SEL is a multi-bit signal used to add the number of photon detections to multiple latch circuits 41 or 51 in the high-resolution measurement unit 13a and the low-resolution measurement unit 14b.

[0123] Fig. 13 is a timing chart showing the operation of the photodetector 10a of Fig. 12. Fig. 13 illustrates the signal XRCG input to the quench circuit 32a, the signal BINEN input to the pulse generating circuit 31a, the output value of the adder 85, and the photon detection signal BIT_SEL. Similar to Fig. 4A, Fig. 13 also illustrates the optical pulse signal LASER, the cathode voltage VK of the photoelectric conversion element 30, the clock signal BIN_CLK, the selection signal SELBIN_FINE[11:0], the clock signal SELBIN_FINE[0], the selection signal SELBIN_COARSE[3:0], and the time of flight of light.

[0124] When the photoelectric conversion element 30 detects a photon, the cathode voltage VK drops. In the pixel 20a, which is an active pixel, the dropped cathode voltage VK does not recover immediately and remains in the dropped state.

[0125] The signal XRCG becomes low level in synchronization with the clock signal BIN_CLK, for example. When the low-level signal XRCG is output, the cathode voltages VK of the multiple pixels 20a that detected the photon simultaneously rise and can be restored to the initial voltage. This allows the multiple pixels 20a to be simultaneously reset. Figure 15 shows an example in which the low-level signal XRCG is input at times t3 and t4.

[0126] The signal BINEN goes high at a predetermined timing before the signal XRCG is input, for example. In the example of Fig. 15, the signal BINEN goes high before time t3 (and time t4) and goes low at time t3 (and time t4).

[0127] When the signal BINEN becomes high level, the pixel 20a that detected the photon causes the latch circuit 34 to latch the dropped cathode voltage VK. Furthermore, while the signal BINEN is high level, the pixel 20a that detected the photon supplies a pulse signal PIX_PLS to the adder 85. When multiple pixels 20a within the macro pixel 15a detect a photon, multiple pulse signals PIX_PLS are simultaneously supplied to the adder 85.

[0128] The adder 85 performs an addition operation based on the timing when the signal BINEN becomes high level. In the example of Fig. 13, Ca pulse signals PIX_PLS are supplied to the adder 85 before time t3. Also, Cb pulse signals PIX_PLS are supplied to the adder 85 before time t4.

[0129] The bit selection unit 86 outputs a photon detection signal BIT_SEL having a value of Ca at time t3 based on the output value of the adder 85. Also, at time t4, it outputs a photon detection signal BIT_SEL having a value of Cb.

[0130] As described above, the multiple pixels 20a in the macro pixel 15a are driven synchronously by the signals XRCG and BINEN.

[0131] When the macro pixel 15a is composed of 16 pixels 20a, a maximum of 16 pulse signals PIX_PLS are simultaneously input to the adder 85. In this case, the high-resolution measurement unit 13a and the low-resolution measurement unit 14b add up a maximum of 16 photon detection counts. Fig. 12 shows an example in which the photon detection signal BIT_SEL has, for example, 16 bits of information according to the maximum photon detection count.

[0132] Fig. 14 is a circuit diagram showing the configuration of a high-resolution measurement unit 13a applied to the photodetector 10a of Fig. 12. The high-resolution measurement unit 13a has a shift register 22 and a plurality of counters 21a, similar to the high-resolution measurement unit 13 of Fig. 5. The configuration of the shift register 22 is the same as that of Fig. 5.

[0133] The plurality of logic circuits 44 each receive a signal of a corresponding bit of the selection signal SELBIN_FINE and BIT_SEL[0], which is the least significant bit of the photon detection signal BIT_SEL.

[0134] The counter 21a has a plurality of latch circuits 41, each holding one bit of information. A selection circuit 87 is disposed between two latch circuits 41. The selection circuit 87 receives the clock signal BIN_CLK and the output signal of the preceding latch circuit 41. The selection circuit 87 also receives all bits of the photon detection signal BIT_SEL except for the least significant bit. The selection circuit 87 switches between inputting the clock signal BIN_CLK or the output signal of the preceding latch circuit 41 to the succeeding latch circuit 41, depending on the value of the corresponding bit of the photon detection signal BIT_SEL.

[0135] This allows the counter 21a to add up the number of photon detections corresponding to the number of pixels 20a that have detected photons.

[0136] The counter 23 in the low-resolution measurement unit 14b in Fig. 12 has the same configuration as the counter 21a in Fig. 14. The configuration of the shift register 24 in the low-resolution measurement unit 14b is the same as that in Fig. 6. The low-resolution measurement unit 14b may also include the frequency divider circuit 61 and latch circuit 62 in Fig. 7.

[0137] 2, when a photon is detected within a macro pixel 15, the high-resolution measurement unit 13 and the low-resolution measurement unit 14 add up one photon detection count regardless of the number of pixels 20 that detected the photon. In contrast, the high-resolution measurement unit 13a and the low-resolution measurement unit 14b in Fig. 12 can add up to a maximum number of photon detection counts corresponding to the number of pixels 20a within the macro pixel 15a each time a plurality of pixels 20a simultaneously output a pulse signal PIX_PLS.

[0138] The pixel 20a may be applied to the photodetector 10 in Fig. 2. That is, when a photon is detected in the macro pixel 15a, one photon detection count may be added up regardless of the number of pixels 20a that detected the photon.

[0139] Fig. 15 is a block diagram showing the configuration of a photodetector 10b according to a modified example, which includes a pixel array unit 11 in which a plurality of pixels 20a are arranged, a consolidation unit 16 (logic circuit 25) similar to that shown in Fig. 2, a high-resolution measurement unit 13, and a low-resolution measurement unit 14.

[0140] Fig. 16 is a timing chart showing the operation of the photodetector 10b in Fig. 15. Fig. 16 illustrates the light pulse signal LASER, the cathode voltage VK of the photoelectric conversion element 30, the clock signal BIN_CLK, the signal XRCG, the signal BINEN, the photon detection signal PIX_OUT, the selection signal SELBIN_FINE[11:0], the clock signal SELBIN_FINE[0], the selection signal SELBIN_COARSE[3:0], and the time of flight of light.

[0141] 16, similarly to Fig. 13, the plurality of pixels 20a return the cathode voltage VK to the initial voltage at the timing when a low-level signal XRCG is input, and the plurality of pixels 20a simultaneously output a pulse signal PIX_PLS while a high-level signal BINEN is input.

[0142] 15 outputs the photon detection signal PIX_OUT while the high-level signal BINEN is being input, if any one high-level pulse signal PIX_PLS is input. Through the above operation, the high-resolution measurement unit 13 and the low-resolution measurement unit 14 can count the number of photon detections, as in FIG. 4A.

[0143] 17A to 17D, several configuration ideas for the pixel array unit 11 according to the first embodiment of the present disclosure will be described. Fig. 17A is a block diagram showing the basic configuration of the pixel array unit 11. In the pixel array unit 11 of Fig. 17A, a plurality of pixels (Z) 20 according to the first embodiment of the present disclosure are arranged over the entire surface.

[0144] 17B is a diagram showing the configuration of a pixel array unit 11a according to a first modified example, in which pixels (Z) 20 used for distance measurement are mixed with gradation pixels (R) 20b, gradation pixels (G) 20c, and gradation pixels (B) 20d used for obtaining visible light gradation values.

[0145] The pixel (second pixel) 20 and the gradation pixels (first pixels) 20b to 20d receive light of different wavelengths. The pixel 20 receives, for example, infrared rays (IR) or near-infrared light. The gradation pixels 20b, 20c, and 20d receive red visible light, green visible light, and blue visible light, respectively. The pixel 20 and the gradation pixels 20b to 20d each have a color filter, infrared filter, or near-infrared filter (hereinafter collectively referred to as a color filter) corresponding to the wavelength of light they receive. The color filters are arranged to overlap the pixel array section 11a in a planar view.

[0146] In the pixel array section 11a of FIG. 17B, color image data can be generated using the gradation pixels 20b to 20d, and depth image data can be generated in parallel using the pixel 20.

[0147] Fig. 17C is a block diagram showing the configuration of a pixel array unit 11b according to a second modification. The pixel array unit 11b in Fig. 17C is divided into a plurality of pixels 20 and a plurality of gradation pixels 20b to 20d, which is finer than the pixel array unit 11a in Fig. 17B.

[0148] 17C includes a plurality of macro pixels 15 b, 15 c, and 15 d. The macro pixel 15 b includes a plurality of gradation pixels 20 b, the macro pixel 15 c includes a plurality of gradation pixels 20 c, and the macro pixel 15 d includes a plurality of gradation pixels 20 d. In the pixel array unit 11 b, a pixel 20 is arranged for each of the macro pixels 15 b to 15 d.

[0149] 17D is a block diagram showing the configuration of a pixel array unit 11c according to a third modified example. The pixel array unit 11c has more pixels 20 arranged therein than the pixel array unit 11b. The pixel array unit 11c can improve the spatial resolution of the ranging system 1 more than the pixel array unit 11b.

[0150] Furthermore, when the pixels 20 are arranged over the entire surface as in FIG. 17A, the spatial resolution can be improved more than that of the pixel array section 11c.

[0151] Fig. 18A is a cross-sectional view of a ranging system 1 according to a first embodiment of the present disclosure. Similar to Fig. 8A, the ranging system 1 in Fig. 18A has a first substrate 71 and a second substrate 72. The first substrate 71 has a semiconductor layer 91 in which the photoelectric conversion elements 30 are arranged. The first substrate 71 and the second substrate 72 have a wiring layer 92. The wiring layer 92 is connected to, for example, a PAD 93.

[0152] 17B to 17D, the distance measuring system 1 in Fig. 18A has a configuration in which a pixel 20 and gradation pixels 20b to 20d are mixed together in the pixel array section 11. Fig. 18A illustrates one pixel 20 and one gradation pixel 20c arranged adjacent to the pixel 20. In Fig. 18A, the gradation pixels 20b and 20d are not illustrated.

[0153] On the first substrate 71, a photoelectric conversion element 30, a lens portion 94 that condenses light incident on the photoelectric conversion element 30, and a color filter are arranged for each pixel 20 and gradation pixels 20b to 20d.

[0154] Infrared (near-infrared) filters 95a and 95b (second color filter layers) are disposed in the pixel 20. The infrared filters 95a and 95b are configured, for example, with a red filter and a blue filter. The infrared filters 95a and 95b transmit near-infrared light or infrared light.

[0155] Color filters 95c and 95d (first color filter layer) are arranged in the gradation pixel 20c. The color filter 95c in FIG. 18A transmits green visible light to the gradation pixel 20c. The color filter 95d is, for example, a SIR (Selective Infra Red filter), and is used to prevent infrared light and the like from entering the gradation pixel 20c by absorbing infrared or near-infrared light that would otherwise be transmitted by the RGB color filter (color filter 95c in FIG. 18A). The color filter 95d and a color filter 95c that transmits red or blue visible light are arranged in the gradation pixels 20b and 20d (not shown).

[0156] The photoelectric conversion elements 30 in FIG. 18A are SPADs, each having a charge multiplier 96 .

[0157] Fig. 18B is a cross-sectional view of a ranging system 1a according to a first modified example. The ranging system 1a in Fig. 18B has a three-layer structure similar to that in Fig. 8B, specifically, a first substrate 71, a second substrate 72, and a third substrate 73. The second substrate 72 and the third substrate 73 are connected via a via portion 97.

[0158] Fig. 18C is a cross-sectional view of a ranging system 1b according to a second modified example. Similar to the ranging system 1 of Fig. 18A, the ranging system 1b of Fig. 18C has a two-layer structure of a first substrate 71 and a second substrate 72. The ranging system 1b of Fig. 18C also differs from the ranging system 1 of Fig. 18A in that pixel transistors 98 are arranged on the first substrate 71.

[0159] Fig. 18D is a cross-sectional view of a distance measuring system 1c according to a third modified example. The distance measuring system 1c in Fig. 18D has a three-layer structure similar to that of the distance measuring system 1a in Fig. 18B. In addition, in the distance measuring system 1c in Fig. 18D, pixel transistors 98 are arranged on a first substrate 71.

[0160] Below, several comparative examples will be described for the photodetector 10 according to the first embodiment of the present disclosure. The photodetector 10 measures the time of flight of light using two measurement units, a high-resolution measurement unit 13 and a low-resolution measurement unit 14. As a first comparative example, a photodetector that measures the time of flight of light using one measurement unit can be considered.

[0161] In this specification, an example is described in which the high-resolution measurement unit 13 has 12 counters 21 and the low-resolution measurement unit 14 has four counters 23. In a first comparative example, an example is described in which 16 counters are arranged in one measurement unit so that the circuit area is the same as that of the photodetector 10 in this example.

[0162] 19A is a diagram illustrating a distance measurement method according to a first comparative example. In the first comparative example, only one counter can measure the time of flight of light for one bin (i.e., unit time Δt). In other words, in the first comparative example, only distances up to 16 bins (BIN[0:15])×Δt can be measured.

[0163] Furthermore, in order to reduce errors in the center of gravity calculation when the pulse width of the reflected light pulse signal received by the light detection device 10 is shifted forward or backward from the bin width along the time axis, it is necessary to make the light emission pulse width (i.e., light emission time) of the light pulse signal LASER a constant multiple (e.g., double) of the unit time of one bin. That is, as described above, when the unit time of one bin is increased, it is necessary to increase the light emission time of the light pulse signal, which results in an increase in the light emission power of the light pulse signal.

[0164] 19B is a diagram illustrating a distance measurement method according to the first embodiment of the present disclosure. As described with reference to FIG. 4A etc., the distance measurement unit 4 is capable of measuring distances with a light flight time of up to 48Δt using 12 bins (BIN_FINE[11:0]) of the high-resolution measurement unit 13 and 4 bins (BIN_COARSE[3:0]) of the low-resolution measurement unit 14.

[0165] 19B, when measuring the same distance as in FIG. 19A, the unit time Δt can be made shorter than in FIG. 19A. This improves the time resolution of the distance measurement unit 4, and particularly improves the distance measurement accuracy for objects at close range. Furthermore, the emission time of the optical pulse signal can be reduced, and the emission power of the optical pulse signal can be reduced.

[0166] FIG. 20A is a timing chart illustrating a ranging technique according to a second comparative example. In the second comparative example, low-time-resolution ranging and high-time-resolution ranging are performed in a time-division manner. Specifically, an optical pulse signal (Tx) is emitted at a coarse modulation frequency, and low-time-resolution ranging (Coarse Measurement) is performed based on a reflected optical pulse signal (Rx) at the coarse modulation frequency. Subsequently, the low-time-resolution ranging results are read out (first readout, Readout_1). In the readout, the number of photon detections is read from a TDC or the like. Subsequently, an optical pulse signal is emitted at a fine modulation frequency, and high-time-resolution ranging (Fine Measurement) is performed based on a reflected optical pulse signal at the fine modulation frequency. Subsequently, the high-time-resolution ranging results are read out (second readout, Readout_2).

[0167] In the technique of Fig. 20A, a low time resolution ranging result and a high time resolution ranging result are obtained in one frame period (Frame) including a low time resolution ranging, a first readout, a high time resolution ranging, and a second readout. The technique of Fig. 20A also enables ranging of more bins with fewer counters, as shown in Fig. 19B.

[0168] However, the technique shown in Fig. 20A requires two ranging periods and two readout periods within one frame period. This lengthens one frame period, deteriorating the time efficiency of ranging (i.e., frame rate). Furthermore, blurring may occur due to fluctuations in the distance between the object and the ranging system between the first ranging period and the second ranging period. Furthermore, the technique shown in Fig. 20A requires that optical pulse signals with two different modulation frequencies be emitted in a time-division manner within one frame period, increasing the power consumption of the light-emitting device.

[0169] 20B is a timing chart illustrating a ranging method according to the first embodiment of the present disclosure. In the ranging method of FIG. 20B, ranging with high time resolution (Fine Measurement) and ranging with low time resolution (Coarse Measurement) are performed in parallel using the high-resolution measurement unit 13 and the low-resolution measurement unit 14. One frame period in FIG. 20B includes one ranging period and one readout period (Readout).

[0170] The method of Fig. 20B can shorten one frame period compared to the method of Fig. 20A, thereby improving the frame rate. Furthermore, there is no risk of blurring occurring due to time-division of the ranging period. Furthermore, the method of Fig. 20B only requires that a light pulse signal of one modulation frequency be emitted within one frame period, eliminating the need for time-division of the light-emitting period. This reduces the power consumption of the light-emitting device 2.

[0171] As described above, the photodetector 10 according to the first embodiment of the present disclosure includes the high-resolution measurement unit 13 and the low-resolution measurement unit 14. The photodetector 10 measures the time-of-flight of light over a rough period based on the measurement results of the low-resolution measurement unit 14. Furthermore, based on the measurement results of the high-resolution measurement unit 13, the photodetector 10 measures the time-of-flight of light over smaller periods obtained by time-dividing the measured rough period.

[0172] The photodetector 10 according to the first embodiment of the present disclosure is capable of measuring distances up to a distance where the time of flight of light is Na×Nb×Δt, with a circuit area corresponding to the number of bins (Na+Nb) of the high-resolution measurement unit 13 and the low-resolution measurement unit 14. This method does not require increasing the unit time Δt, and therefore can expand the ranging range while maintaining high time resolution. In other words, it is possible to achieve both an expanded ranging range and highly accurate ranging with high circuit area efficiency.

[0173] Furthermore, the photodetector 10 can perform high-time resolution ranging and low-time resolution ranging in parallel using the high-resolution measuring unit 13 and the low-resolution measuring unit 14. This allows for an improved frame rate compared to a method of performing ranging in time division.

[0174] Second Embodiment The photodetector 10 according to the first embodiment of the present disclosure has two measurement units (i.e., a high-resolution measurement unit 13 and a low-resolution measurement unit 14). The photodetector 10 may have a configuration including three or more measurement units.

[0175] 21 is a block diagram showing a photodetector 10c according to a second embodiment of the present disclosure. The pixel circuit 12 in FIG. 21 includes a medium-resolution measurement unit (Middle TDC or third measurement unit) 100. The medium-resolution measurement unit 100 measures photon detection timing based on the photon detection signal PIX_OUT in parallel with the high-resolution measurement unit 13 and the low-resolution measurement unit 14. The medium-resolution measurement unit 100 has a time resolution lower than that of the high-resolution measurement unit 13 but higher than that of the low-resolution measurement unit 14.

[0176] The medium resolution measurement unit 100 has a plurality of counters (third counters) 101 and a shift register (third shift register) 102. The circuit configurations of the counters 101 and the shift register 102 are similar to those of the counter 23 and the shift register 24 in Fig. 6. The shift register 102 outputs a selection signal SELBIN_MIDDLE to the counter 101.

[0177] 21 shows an example in which the shift registers 22, 102, and 24 output an 8-bit selection signal SELBIN_FINE[7:0], a 6-bit selection signal SELBIN_MIDDLE[5:0], and a 2-bit selection signal SELBIN_COARSE[1:0], respectively. Accordingly, eight counters 21, six counters 101, and two counters 23 are provided in the high-resolution measurement unit 13, the medium-resolution measurement unit 100, and the low-resolution measurement unit 14, respectively. The number of bits of the selection signals SELBIN_FINE, SELBIN_MIDDLE, and SELBIN_COARSE, and the number of counters 21, 101, and 23 are not limited to the above example, and may be any number.

[0178] The shift register 102 receives SELBIN_FINE[0], which is the least significant bit of the selection signal SELBIN_FINE, as a clock signal from the shift register 22 in the high-resolution measurement unit 13 .

[0179] The shift register 102 also generates a clock signal (fourth clock signal) MID_CLK and inputs it to the shift register 24 in the low-resolution measurement unit 14. The clock signal MID_CLK is a signal that defines the timing of bit shifting of the selection signal SELBIN_COARSE. In the example of Fig. 21, the clock signal MID_CLK is set to go high when either the least significant bit (SELBIN_MIDDLE[0]) or the third lowest bit (SELBIN_MIDDLE[3]) of the selection signal SELBIN_MIDDLE goes high.

[0180] The clock signal MID_CLK can be generated, for example, as an output signal of a logic circuit (for example, an OR circuit) 103 to which SELBIN_MIDDLE[0] and SELBIN_MIDDLE[3] are input.

[0181] Fig. 22 is a timing chart showing the operation of the photodetector 10c of Fig. 21. Fig. 22 illustrates the light pulse signal LASER, the cathode voltage VK of the photoelectric conversion element 30, the photon detection signal PIX_OUT, the clock signal BIN_CLK that drives the high-resolution measurement unit 13, the selection signal SELBIN_FINE[8:0], the clock signal SELBIN_FINE[0] that drives the medium-resolution measurement unit 100, the selection signal SELBIN_MIDDLE[5:0], the clock signal MID_CLK that drives the low-resolution measurement unit 14, the selection signal SELBIN_COARSE[1:0], and the time of flight of light.

[0182] As described above, the select signal SELBIN_MIDDLE is shifted to a high level bit in turn every time the clock signal SELBIN_FINE[0] goes high. Also, the select signal SELBIN_COARSE is shifted to a high level bit in turn every time the clock signal MID_CLK goes high (i.e., every time SELBIN_MIDDLE[0] or SELBIN_MIDDLE[3] goes high).

[0183] 22, the low-resolution measurement unit 14 counts the number of times a photon is detected for each of a plurality of time periods (second time periods) Tb divided by a time period Tc (second time interval) in which the clock signal MID_CLK goes high. In contrast, the medium-resolution measurement unit 100 counts the number of times a photon is detected for each of a plurality of time periods (second time periods) Tb divided by a time period Ta (third time interval) in which the clock signal SELBIN_FINE[0] goes high.

[0184] 4A , the distance measuring unit 4 roughly calculates the time-of-flight of light by extracting peaks of a histogram based on the measurement data from the low-resolution measurement unit 14 and the medium-resolution measurement unit 100. The distance measuring unit 4 can identify which bin of the medium-resolution measurement unit 100 the time-of-flight of light corresponds to. Furthermore, the distance measuring unit 4 can calculate more precise time-of-flight of light from the time range of the extracted bin of the medium-resolution measurement unit 100 by extracting peaks of a histogram based on the measurement data from the high-resolution measurement unit 13.

[0185] In this specification, the distance measurement performed by the low-resolution measurement unit 14 and the medium-resolution measurement unit 100 is also referred to as low-time resolution distance measurement.

[0186] The distance measurement unit 4 can measure distances up to a distance where the time of flight of light is Na×Nc×Δt, for example, by using the number of bins Nc of the medium-resolution measurement unit 100 and the number of bins Na of the high-resolution measurement unit 13. In the example of Fig. 21, distances up to a distance where the time of flight of light is 48Δt can be measured. In other words, the photodetector 10c of Fig. 21 can expand the distance measurement range to the same extent as the photodetector 10 of Fig. 2.

[0187] Furthermore, in the photodetector 10c according to the second embodiment of the present disclosure, by using both the low-resolution measurement unit 14 and the medium-resolution measurement unit 100, the possibility of errors occurring in distance measurement with low time resolution can be reduced.

[0188] In the photodetector 10 according to the first embodiment of the present disclosure, if an error occurs in the measurement results of the low-resolution measurement unit 14, a large distance measurement error may occur. For example, a peak may occur in the first bin of the measurement results of the low-resolution measurement unit 14, instead of the second bin, due to reception of ambient light, etc. In the example of FIG. 4A , the low-resolution measurement unit 14 measures the time of flight of light in rough units of 12Δt per bin (period Ta in FIG. 4A ). Therefore, if an error of one bin occurs in the low-resolution measurement unit 14, an error of as much as 12Δt occurs in the time of flight of light.

[0189] The medium-resolution measurement unit 100 according to the second embodiment of the present disclosure measures the time of flight of light in units of 8 Δt (period Ta in FIG. 22 ), which is finer than the low-resolution measurement unit 14 in FIG. 4A . Therefore, the medium-resolution measurement unit 100 can reduce the error of one bin to 8 Δt.

[0190] Furthermore, the measurement results of the low-resolution measurement unit 14 as well as the medium-resolution measurement unit 100 may be referred to prevent distance measurement errors from occurring.

[0191] The medium-resolution measurement unit 100 may be used not only to prevent distance measurement errors but also to expand the distance measurement range. For example, distance measurement may be performed up to a distance where the time of flight of light is Na×Nb×Nc×Δt (96Δt in the example of FIG. 22 ) using the bin numbers Na, Nb, and Nc of the high-resolution measurement unit 13, the low-resolution measurement unit 14, and the medium-resolution measurement unit 100, respectively. The above distance measurement method can be realized, for example, by inputting the least significant bit (SELBIN_MIDDLE[0]) of the selection signal SELBIN_MIDDLE as a clock signal to the shift register 24 in the low-resolution measurement unit 14.

[0192] The photodetector 10c may have four or more measurement units. The medium resolution measurement unit 100 may be applied to the photodetector 10a or 10b shown in FIG.

[0193] As described above, the photodetector 10c according to the second embodiment of the present disclosure includes the medium-resolution measurement unit 100. The medium-resolution measurement unit 100 can measure the time of flight of light in finer units than the low-resolution measurement unit 14. This can reduce distance measurement errors.

[0194] (Application Examples) The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, construction machinery, or agricultural machinery (tractor).

[0195] 23 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 23 , the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard, such as a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), or FlexRay (registered trademark).

[0196] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a memory unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various controlled devices. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle via wired or wireless communication. Figure 23 illustrates the functional configuration of the integrated control unit 7600, including a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a memory unit 7690. The other control units also include a microcomputer, a communication I / F, a memory unit, and the like.

[0197] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating drive force for the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating braking force for the vehicle. The drivetrain control unit 7100 may also function as a control device for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.

[0198] A vehicle state detection unit 7110 is connected to the drivetrain control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor that detects the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine rotation speed, the rotation speed of the wheels, etc. The drivetrain control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0199] The body system control unit 7200 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0200] The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, in accordance with various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

[0201] The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0202] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

[0203] 24 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0204] 24 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 7910, 7912, 7914, and 7916, a bird's-eye view image of the vehicle 7900 viewed from above can be obtained.

[0205] The outside vehicle information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The outside vehicle information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside vehicle information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

[0206] Returning to FIG. 23 , the explanation will be continued. The outside vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside vehicle information detection unit 7400 also receives detection information from the connected outside vehicle information detection unit 7420. If the outside vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside vehicle information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. Based on the received information, the outside vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, text on the road, etc. Based on the received information, the outside vehicle information detection unit 7400 may also perform environment recognition processing for recognizing rainfall, fog, road conditions, etc. Based on the received information, the outside vehicle information detection unit 7400 may also calculate the distance to an object outside the vehicle.

[0207] The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, vehicles, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by combining image data captured by different image capturing units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different image capturing units 7410.

[0208] The interior information detection unit 7500 detects information inside the vehicle. A driver state detection unit 7510 that detects the driver's state is connected to the interior information detection unit 7500, for example. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound from within the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or the driver gripping the steering wheel. The interior information detection unit 7500 may calculate the driver's level of fatigue or concentration based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The interior information detection unit 7500 may perform processing such as noise canceling on the collected audio signal.

[0209] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 may be implemented by a device that can be operated by a passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input through a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a personal digital assistant (PDA) that can operate the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information using gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and the like operate this input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0210] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0211] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (Global System of Mobile communications), WiMAX (registered trademark), LTE (Long Term Evolution), or LTE-Advanced (LTE-A), or other wireless communication protocols such as a wireless LAN (also referred to as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I / F 7620 may connect to a terminal located near the vehicle (e.g., a terminal of a driver, pedestrian, or store, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.

[0212] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in vehicles. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE 802.11p and an upper layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0213] The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites), and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

[0214] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closures, required travel time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0215] The in-vehicle device I / F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish wireless connections using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). Furthermore, the in-vehicle device I / F 7660 may establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0216] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0217] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various programs based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output control commands to the drivetrain control unit 7100. For example, the microcomputer 7610 may perform cooperative control aimed at realizing functions of an Advanced Driver Assistance System (ADAS), including vehicle collision avoidance or impact mitigation, following driving based on the following distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc. based on information acquired about the vehicle's surroundings.

[0218] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including information about the vicinity of the vehicle's current location, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. Furthermore, the microcomputer 7610 may predict dangers, such as a vehicle collision, the approach of a pedestrian, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0219] The audio / image output unit 7670 transmits at least one audio and / or image output signal to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information. In the example of FIG. 23 , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may also be other devices, such as headphones, a wearable device such as an eyeglass-type display worn by the occupant, a projector, or a lamp. When the output device is a display device, the display device visually displays results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats, such as text, images, tables, and graphs. When the output device is an audio output device, the audio output device converts audio signals, such as reproduced audio data or acoustic data, into analog signals and audibly outputs the analog signals.

[0220] In the example shown in FIG. 23 , at least two control units connected via the communication network 7010 may be integrated into a single control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by one of the control units may be performed by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by one of the control units. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to and from each other via the communication network 7010.

[0221] In the vehicle control system 7000 described above, the ranging system 1 according to this embodiment described with reference to Fig. 1 can be applied to the image capturing unit 7410 of the application example shown in Fig. 23. This allows the small-sized ranging system 1 with a high frame rate to achieve an expanded ranging range and highly accurate ranging.

[0222] Furthermore, at least some of the components of the ranging system 1 described using Fig. 1 may be realized in a module (for example, an integrated circuit module configured on a single die) for the integrated control unit 7600 shown in Fig. 23. Alternatively, the ranging system 1 described using Fig. 1 may be realized by multiple control units of the vehicle control system 7000 shown in Fig. 23.

[0223] The present technology may be configured as follows: (1) A photodetector including: a photoelectric conversion element that detects incident photons; a first measurement unit that measures timing at which the photoelectric conversion element detects a photon; and a second measurement unit that measures the timing in parallel with the first measurement unit and with a time resolution lower than that of the first measurement unit. (2) The photodetector described in (1), wherein the first measurement unit and the second measurement unit measure the timing in parallel in each of a plurality of frame periods. (3) The photodetector described in (1) or (2), wherein the first measurement unit measures the number of times the photon is detected for each of a plurality of first periods obtained by time-dividing the timing by a first time interval, and the second measurement unit measures the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing by a second time interval longer than the first time interval. (4) The photodetector according to (3), wherein the first measurement unit includes a plurality of first counters that measure the number of times the photoelectric conversion element detects a photon for each of the plurality of first periods, and the second measurement unit includes a plurality of second counters that measure the number of times the photoelectric conversion element detects a photon for each of the plurality of second periods. (5) The photodetector according to (4), wherein the first measurement unit includes a first shift register having a plurality of first register circuits that shift a first pulse signal in synchronization with a first clock signal, the second measurement unit includes a second shift register having a plurality of second register circuits that shift a second pulse signal in synchronization with a second clock signal, the plurality of first counters update the number of times in accordance with the timing at which the first pulse signal is output from the corresponding first register circuit, and the plurality of second counters update the number of times in accordance with the timing at which the second pulse signal is output from the corresponding second register circuit. (6) The photodetector according to (5), wherein the period of the first clock signal is the period of a bin of a first histogram generated based on the measurement values ​​of the plurality of first counters, and the period of the second clock signal is the period of a bin of a second histogram generated based on the measurement values ​​of the plurality of second counters.(7) The photodetector according to (5) or (6), wherein the second clock signal is the first pulse signal output from the first register circuit in a first stage of the first shift register. (8) The photodetector according to (5) or (6), wherein the second clock signal is a third clock signal obtained by dividing the first clock signal by a division ratio corresponding to the number of the plurality of first counters, and synchronized with the first pulse signal output from the first register circuit in a first stage of the first shift register. (9) The photodetector according to any one of (5) to (8), comprising: a macropixel including one or more pixels each having one or more of the photoelectric conversion elements; and a photon detection signal generator that outputs a photon detection signal indicating that at least one of the photoelectric conversion elements in the macropixel has detected a photon, wherein the plurality of first counters and the plurality of second counters update the number of times in accordance with the timing at which the photon detection signal is output. (10) The photodetector according to (9), wherein the plurality of first counters update the number of times when the corresponding first register circuits output the first pulse signal and the photon detection signal, and the plurality of second counters update the number of times when the corresponding second register circuits output the second pulse signal and the photon detection signal. (11) The photodetector according to (9) or (10), wherein the photon detection signal generator includes: an adder that adds up the numbers of photons detected by one or more photoelectric conversion elements in the macropixel; and a bit selector that selects a bit according to the value added by the adder. (12) The photodetector according to (11), wherein the plurality of first counters and the plurality of second counters update the number of times according to the bit selected by the bit selector. (13) The photodetector according to any one of (9) to (12), wherein the photodetector includes a plurality of macropixels, each of which serves as a ranging point. (14) The light detection device according to any one of (3) to (13), further comprising a third measurement unit that measures the timing with a higher time resolution than the first measurement unit in parallel with the first measurement unit and the second measurement unit.(15) The photodetector according to (14), wherein the third measurement unit measures the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing by third time intervals shorter than the first time interval. (16) The photodetector according to any one of (1) to (15), further comprising a first quench circuit having a passive element connected between one end of the photoelectric conversion element and a reference voltage node, wherein the first quench circuit restores a voltage at one end of the photoelectric conversion element to an initial voltage by a current flowing from the reference voltage node to the one end of the photoelectric conversion element via the passive element after the photoelectric conversion element detects a photon. (17) The photodetector according to any one of (1) to (15), further comprising: a second quench circuit having an active element connected between one end of the photoelectric conversion element and a reference voltage node, wherein the second quench circuit controls the active element to restore a voltage at the one end of the photoelectric conversion element to an initial voltage after the photoelectric conversion element detects a photon. (18) The photodetector according to any one of (1) to (17), further comprising: a pixel array unit including one or more first pixels that acquire grayscale information and one or more second pixels that are used for distance measurement, wherein each of the first pixels and the second pixels has one or more of the photoelectric conversion elements. (19) The photodetector according to (18), further comprising: a color filter disposed so as to overlap with the pixel array unit in a planar view, the color filter having: a first color filter layer that transmits visible light and is provided in correspondence with one or more of the first pixels for acquiring gradation information; and a second color filter layer that transmits near-infrared light or infrared light and is provided in correspondence with the second pixels. (20) A ranging system comprising: the photodetector according to any one of (1) to (19), which receives a reflected light pulse signal obtained by reflecting a light pulse signal from an object, a light emitting device that emits the light pulse signal, and a ranging unit that measures a distance to the object based on the reflected light pulse signal.

[0224] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0225] 1, 1a, 1b, 1c Distance measuring system, 2 Light emitting device, 3 Control unit, 4 Distance measuring unit, 5 Object, 10, 10a, 10b, 10c Photodetector, 11, 11a, 11b, 11c Pixel array unit, 12 Pixel circuit, 13, 13a High-resolution measurement unit, 14, 14a, 14b Low-resolution measurement unit, 15, 15a, 15b, 15c, 15d Macro pixel, 16, 16a Aggregation unit, 20, 20a Pixel, 20b, 20c, 20d Gradation pixel, 21, 21a, 23, 101 Counter, 22, 24, 102 Shift register, 25, 44, 54, 82, 103 Logic circuit, 30 Photoelectric conversion element, 31, 31a Pulse generation circuit, 32, 32a Quench circuit, 33 Inverter, 34, 41, 51, 62 Latch circuit, 35 Delay circuit, 36 Passive elements, 42, 52 Register circuit, 43, 53, 87 Selection circuit, 61 Divider circuit, 71 First substrate, 72 Second substrate, 73 Third substrate, 80 Counter area, 81 Active elements, 85 Adder, 86 Bit selection unit, 91 Semiconductor layer, 92 Wiring layer, 93 PAD, 94 Lens unit, 95a, 95b Infrared filter, 95c, 95d Color filter, 96 Charge multiplication unit, 97 Via unit, 98 Pixel transistor, 100 Medium resolution measurement unit

Claims

1. A photodetector comprising: a photoelectric conversion element that detects incident photons; a first measurement unit that measures the timing at which the photoelectric conversion element detects the photon; and a second measurement unit that measures the timing in parallel with the first measurement unit with a lower time resolution than the first measurement unit.

2. The photodetector according to claim 1, wherein the first measurement unit and the second measurement unit measure the timing in parallel during each of a plurality of frame periods.

3. The photodetector according to claim 1, wherein the first measurement unit measures the number of times the photon is detected for each of a plurality of first periods obtained by time-dividing the timing by a first time interval, and the second measurement unit measures the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing by a second time interval longer than the first time interval.

4. The photodetector according to claim 3, wherein the first measurement unit has a plurality of first counters that measure the number of times that the photoelectric conversion element detects a photon for each of the plurality of first periods, and the second measurement unit has a plurality of second counters that measure the number of times that the photoelectric conversion element detects a photon for each of the plurality of second periods.

5. The photodetector according to claim 4, wherein the first measurement unit has a first shift register having a plurality of first register circuits that shift a first pulse signal in synchronization with a first clock signal, the second measurement unit has a second shift register having a plurality of second register circuits that shift a second pulse signal in synchronization with a second clock signal, the plurality of first counters update the number of times in accordance with the timing at which the first pulse signal is output from the corresponding first register circuit, and the plurality of second counters update the number of times in accordance with the timing at which the second pulse signal is output from the corresponding second register circuit.

6. The photodetector according to claim 5, wherein the period of the first clock signal is the period of a bin of a first histogram generated based on the measurement values ​​of the plurality of first counters, and the period of the second clock signal is the period of a bin of a second histogram generated based on the measurement values ​​of the plurality of second counters.

7. The photodetector according to claim 5, wherein the second clock signal is the first pulse signal output from the first register circuit in the first stage of the first shift register.

8. The photodetector according to claim 5, wherein the second clock signal is a third clock signal obtained by dividing the first clock signal by a division ratio corresponding to the number of the plurality of first counters, and is synchronized with the first pulse signal output from the first register circuit in the first stage of the first shift register.

9. A photodetection device as described in claim 5, comprising: a macropixel including one or more pixels each having one or more of the photoelectric conversion elements; and a photon detection signal generation unit that outputs a photon detection signal indicating that at least one of the photoelectric conversion elements in the macropixel has detected a photon, wherein the plurality of first counters and the plurality of second counters update the number of times in accordance with the timing at which the photon detection signal is output.

10. The photodetector device of claim 9, wherein the plurality of first counters update the number of times when the corresponding first register circuit outputs the first pulse signal and the photon detection signal, and the plurality of second counters update the number of times when the corresponding second register circuit outputs the second pulse signal and the photon detection signal.

11. The photodetection device according to claim 9, wherein the photon detection signal generation unit comprises: an adder that adds up the number of photons detected by one or more photoelectric conversion elements in the macropixel; and a bit selection unit that selects a bit according to the value added by the adder.

12. The photodetector according to claim 11, wherein the plurality of first counters and the plurality of second counters update the number of times in accordance with the bit selected by the bit selection section.

13. The photodetection device according to claim 9, comprising a plurality of said macro-pixels, each of which serves as a distance measurement point.

14. The light detection device according to claim 3, further comprising a third measurement unit that measures the timing with a higher time resolution than the first measurement unit in parallel with the first and second measurement units.

15. The photodetector according to claim 14, wherein the third measurement unit measures the number of times the photon is detected for each of a plurality of second periods obtained by time-dividing the timing into third time intervals that are shorter than the first time intervals.

16. The photodetector according to claim 1, further comprising a first quench circuit having a passive element connected between one end of the photoelectric conversion element and a reference voltage node, wherein the first quench circuit restores the voltage at one end of the photoelectric conversion element to an initial voltage by a current flowing from the reference voltage node through the passive element to the one end of the photoelectric conversion element after the photoelectric conversion element detects a photon.

17. The photodetector according to claim 1, further comprising a second quench circuit having an active element connected between one end of the photoelectric conversion element and a reference voltage node, wherein the second quench circuit controls the active element to restore the voltage at the one end of the photoelectric conversion element to an initial voltage after the photoelectric conversion element detects a photon.

18. The photodetector according to claim 1, comprising a pixel array unit including one or more first pixels for acquiring gradation information and one or more second pixels for use in distance measurement, each of the first pixels and the second pixels having one or more of the photoelectric conversion elements.

19. The photodetector device of claim 18, further comprising a color filter arranged to overlap the pixel array portion in a planar view, the color filter comprising: a first color filter layer that transmits visible light and is provided in correspondence with one or more of the first pixels that acquire gradation information; and a second color filter layer that transmits near-infrared light or infrared light and is provided in correspondence with the second pixels.

20. A ranging system comprising: a light detection device according to claim 1 that receives a reflected light pulse signal that is a light pulse signal reflected by an object; a light emitting device that emits the light pulse signal; and a ranging unit that measures the distance to the object based on the reflected light pulse signal.

Images