Light detection device and image processing system

Abstract

[Problem] To provide a light detection device and an image processing system that leverage the characteristics of a SPAD. [Solution] This light detection device comprises: a plurality of pairs of photoelectric conversion elements that respectively detect incident photons; and a pixel circuit that, for each of the plurality of pairs of photoelectric conversion elements, outputs the number of photons detected by two photoelectric conversion elements that constitute a pair and are assessed to be in an in-focus state through comparison of the number of photons detected by the two photoelectric conversion elements constituting the pair, and does not output the number of photons detected by a pair of photoelectric conversion elements that are not assessed to be in the in-focus state.

Description

Photodetector and image processing system

[0001] The present disclosure relates to a light detection device and an image processing system.

[0002] Photodetectors using a single photon avalanche diode (SPAD) that amplifies electrons obtained by photoelectrically converting a single incident photon are known. The SPAD can detect even weak light and can detect the number of photons at a higher speed than conventional CMOS (Complementary Metal-Oxide-Semiconductor) image sensors. As an application example of a photodetector using a SPAD, for example, a photoelectric conversion device has been proposed that controls the frequency of a pulse signal that specifies the period during which the SPAD is operated so that the number of photons detected by the SPAD does not reach saturation (see Patent Document 1).

[0003] JP 2023-168794 A

[0004] A photodetector using a SPAD can detect the number of photons at high speed and can detect even weak light, thereby widening the dynamic range.

[0005] However, Patent Document 1 does not disclose any innovations for utilizing the high-speed light detection and wide dynamic range that are the characteristics of SPAD.

[0006] Therefore, the present disclosure provides a photodetector and an image processing system that take advantage of the features of SPAD.

[0007] In order to solve the above problems, according to the present disclosure, there is provided a photodetection device including: a plurality of pairs of photoelectric conversion elements that each detects incident photons; and a plurality of pixel circuits that, for each pair of the plurality of pairs of photoelectric conversion elements, output the number of photons detected by two paired photoelectric conversion elements that are determined to be in an in-focus state by comparing the numbers of photons detected by the two paired photoelectric conversion elements, and do not output the number of photons detected by a paired photoelectric conversion element that is not determined to be in an in-focus state.

[0008] The optical system may further comprise a plurality of signal generating circuits provided for each pair, for generating a plurality of output control signals that assume a predetermined logic level when it is determined that the optical system is in focus.

[0009] Each of the plurality of pixel circuits may determine whether each of the plurality of pairs of photoelectric conversion elements is in focus during a first period within an exposure period of the plurality of pairs of photoelectric conversion elements, and may output the number of photons detected by the two photoelectric conversion elements of the pair determined to be in focus during a second period within the exposure period after the first period.

[0010] For each pair, the device may have a plurality of counters including a first counter and a second counter that count the number of photons detected by the two photoelectric conversion elements of the pair during the first period, and the signal generation circuit may generate the output control signal by comparing at least some bits of a plurality of bit strings representing the number of photons counted by the first counter and the second counter.

[0011] Each pair may include a first light-shielding member that shields one of the pair of two photoelectric conversion elements from light, and a second light-shielding member that shields the other of the pair from light.

[0012] The plurality of pairs of photoelectric conversion elements having the first light-shielding member and the second light-shielding member may be arranged between a plurality of photoelectric conversion elements that detect incident photons without determining whether or not the element is in focus.

[0013] An optical member may be provided for each of the plurality of pairs of photoelectric conversion elements, for guiding incident light to each of the two photoelectric conversion elements in the pair.

[0014] The plurality of pairs of photoelectric conversion elements may determine whether or not they are in focus and detect the number of incident photons, and the plurality of pixel circuits may determine whether or not each pair of the plurality of pairs of photoelectric conversion elements is in focus during a first period that is part of an exposure period of the plurality of pairs of photoelectric conversion elements, and may output the number of photons detected by the two photoelectric conversion elements of the pair that have been determined to be in focus during a second period within the exposure period that follows the first period.

[0015] The signal generating circuit may include one counter for each of the plurality of pairs of photoelectric conversion elements that sequentially counts the number of photons detected by the two photoelectric conversion elements in the pair during the first period, and two holders that separately hold the two numbers of photons counted in sequence by the one counter, and the signal generating circuit may generate the output control signal by comparing at least some bits of the plurality of bit strings representing the two numbers of photons held by the two holders.

[0016] The signal generating circuit may compare two or more bit strings among a plurality of bit strings representing the number of photons counted by the counter to generate the two or more bit output control signal, and each bit of the output control signal may be the result of comparing different bits of the bit strings being compared.

[0017] The signal generating circuit may stop the counting operation of the counter during the second period when at least some of the bits in the plurality of bit strings representing the number of photons do not match.

[0018] When at least some of the bits in the plurality of bit strings representing the number of photons do not match, the signal generating circuit may stop the counting operation of the counter and stop the photoelectric conversion operation of the plurality of photoelectric conversion elements during the second period.

[0019] Furthermore, according to the present disclosure, there is provided a photodetection device including: a plurality of photoelectric conversion elements that respectively detect incident photons; and a plurality of pixel circuits that thin out some of the photoelectric conversion elements having approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements during a first period within an exposure period, and output the number of photons detected by the photoelectric conversion elements that have not been thinned out during a second period within the exposure period after the first period.

[0020] When the amounts of incident light on two or more pixels that are arranged identically or closely together among the plurality of photoelectric conversion elements are approximately equal, the plurality of pixel circuits may thin out some of the photoelectric conversion elements included in the two or more pixels, and output the number of photons detected by the remaining photoelectric conversion elements.

[0021] When thinning out some of the photoelectric conversion elements having approximately the same amount of incident light, a signal generating circuit may be provided that generates an output control signal having a predetermined logic value.

[0022] The pixel may include a first counter and a second counter that count the number of photons output from a first photoelectric conversion element and a second photoelectric conversion element that are included in neighboring pixels among the plurality of photoelectric conversion elements, and the signal generation circuit may generate the output control signal by comparing at least some bits of a plurality of bit strings that represent the number of photons counted by the first counter and the second counter.

[0023] The signal generating circuit may stop the counting operation of at least one of the first counter and the second counter during the second period when the output control signal reaches the predetermined logic level.

[0024] When the output control signal becomes the predetermined logic, the signal generating circuit may stop the counting operation of at least one of the first counter or the second counter during the second period, and may also stop the photoelectric conversion operation of at least some of the photoelectric conversion elements of the plurality of photoelectric conversion elements.

[0025] The photoelectric conversion element may include a SPAD (Single Photon Avalanche Diode).

[0026] Furthermore, according to the present disclosure, there is provided an image processing system comprising: a photodetection device having a plurality of photoelectric conversion elements that respectively detect incident photons; and a plurality of pixel circuits that, for each pair of photoelectric conversion elements among the plurality of photoelectric conversion elements, output the number of photons detected by two paired photoelectric conversion elements that are determined to be in an in-focus state by comparing the numbers of photons detected by the two paired photoelectric conversion elements, and do not output the number of photons detected by paired photoelectric conversion elements that are not determined to be in an in-focus state, or thin out some photoelectric conversion elements that have approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements in a first period within an exposure period, and output the number of photons detected by photoelectric conversion elements that are not thinned out in a second period within the exposure period that follows the first period; and an image generation unit that generates image data based on the number of photons detected by the plurality of pixel circuits.

[0027] 1 is a block diagram of a photodetector and an image processing system according to a first embodiment. FIG. 1 is a diagram showing the configuration of a portion of an imaging unit according to the first embodiment. FIG. 2 is a diagram explaining image plane phase detection autofocus. FIG. 2 is a diagram showing a first example of a stacked structure of an image processing system. FIG. 3 is a diagram showing a second example of a stacked structure of an image processing system. FIG. 4 is a block diagram showing the circuit configuration of a pixel according to the first embodiment. FIG. 5 is a cross-sectional view of a paired first pixel. FIG. 6 is a cross-sectional view of a paired second pixel. FIG. 7 is a block diagram showing the circuit configuration of a pixel circuit and an illuminance determination unit according to the first embodiment. FIG. 8 is a timing chart of an in-focus state of the illuminance determination unit according to the first embodiment. FIG. 9 is a timing chart of an out-of-focus state of the illuminance determination unit according to the first embodiment. FIG. 10 is a block diagram showing the configuration of a holder. FIG. 11 is a timing chart explaining the operation of a photodetector. FIG. 12 is a timing chart showing a first example of an operation of a pixel that is not in a focused state. FIG. 13 is a timing chart showing a second example of an operation of a pixel that is not in a focused state. FIG. 14 is a block diagram showing the circuit configuration of a pixel according to a first modified example. FIG. 15 is a cross-sectional view of a pixel according to the first modified example. FIG. 16 is a diagram showing the configuration of a portion of an imaging unit according to the first modified example. FIG. 17 is a block diagram showing the circuit configuration of an illuminance determination unit according to the second modified example. 1 is a timing chart illustrating the operation of the illuminance determination unit when the amounts of incident light match; a timing chart illustrating the operation of the illuminance determination unit when the amounts of incident light mismatch; a block diagram illustrating an example of the schematic configuration of a vehicle control system; and an explanatory diagram illustrating an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.

[0028] Hereinafter, embodiments of a light detection device and an image processing system will be described with reference to the drawings. The following description will focus on the main components of the light detection device and the image processing system, but the light detection device and the image processing system may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0029] 1 is a block diagram showing a configuration of an image processing system 1 including a photodetector 10 according to a first embodiment of the present disclosure. The photodetector 10 is a device (e.g., an image sensor) that detects incident light. The image processing system 1 is a device that generates image data based on the incident light detected by the photodetector 10.

[0030] The image processing system 1 includes a photodetector 10, a logic processor (image generator) 2, and an image output unit 3. The photodetector 10 includes an imaging unit 11, an illuminance determination unit 12, a holder 13, and a signal transfer unit 14.

[0031] The imaging unit 11 has a plurality of pixels 20 arranged in a two-dimensional direction. Fig. 2 is a diagram showing a configuration of a portion of the imaging unit 11 according to the first embodiment of the present disclosure. In this specification, the left-right (horizontal) direction in Fig. 2 is referred to as a first direction X, and the up-down (vertical) direction in Fig. 2 is referred to as a second direction Y. In this specification, the plurality of pixels 20 arranged in a two-dimensional direction shown in Fig. 2 is also referred to as a pixel array unit.

[0032] The pixel 20 has a photoelectric conversion element that detects incident photons, and detects the photons during a predetermined measurement period.

[0033] The plurality of pixels 20 can selectively receive light having specific color information by providing a color filter, etc. Fig. 2 shows an example in which pixels 20 that receive red (R) light, pixels 20 that receive blue (B) light, and pixels 20 that receive green (G) light are arranged.

[0034] The imaging unit 11 also includes phase difference detection pixels (hereinafter also simply referred to as pixels) 20 a and 20 b that determine whether a plurality of pixels 20 within a predetermined range are in focus. The pixels 20 a and 20 b are a pair of pixels.

[0035] For example, when the phases of light incident on the pixels 20a and 20b match, it can be determined that the pixels are in focus. Furthermore, when the phases of the incident light are not in phase, the phase difference can be adjusted to bring the pixels 20 into focus. In this specification, the focusing method using a pair of pixels 20a and 20b is also referred to as on-chip phase difference autofocus (ZAF).

[0036] 3 is a diagram illustrating image plane phase difference autofocus. For example, the photodetection device 10 has a plurality of pixels 20 and phase difference detection pixels 20a and 20b corresponding to an imaging range 30. Among these, the phase difference detection pixels 20a and 20b are aligned with an object 31 to set an angle of view for focus adjustment, and the phase difference (defocus amount) detected by the phase difference detection pixels 20a and 20b can be made as small as possible to achieve a focused state. A focused range 32 in FIG. 3 is set around the focused phase difference detection pixels 20a and 20b and indicates the range in focus.

[0037] The pixels 20 included in the focus range 32 can generate clear image data of the subject 31 without any out-of-focus blur. In addition, by compressing or discarding image data based on photons detected by other pixels 20, efficient data compression that allows the subject 31 to be appropriately extracted becomes possible.

[0038] The imaging unit 11 has a plurality of pairs of pixels 20 a and 20 b. The plurality of pairs of pixels 20 a and 20 b are arranged between the other pixels 20 .

[0039] The multiple pairs of pixels 20a and 20b include, for example, pixels 20a and 20b that detect a focus state in a first direction X and pixels 20a and 20b that detect a focus state in a second direction Y. The multiple pairs of pixels 20a and 20b that detect a focus state in the first direction X are, for example, arranged at regular intervals in the first direction X within the imaging unit 11. The multiple pairs of pixels 20a and 20b that detect a focus state in the second direction Y are, for example, arranged at regular intervals in the second direction Y within the imaging unit 11.

[0040] The imaging unit 11 in Fig. 1 has a pixel circuit that generates a pixel signal based on photons detected by the plurality of pixels 20. The pixel circuit is not shown in Fig. 1.

[0041] The illuminance determination unit 12 determines the illuminance of multiple pairs of pixels 20 a and 20 b in the imaging unit 11 and determines whether they are in focus. The holder 13 has, for example, a latch circuit or the like and latches the pixel signals output from the imaging unit 11. The signal transfer unit 14 transfers the pixel signals to the logic processing unit 2 or the like. The logic processing unit 2 performs predetermined signal processing on the transferred pixel signals to generate image data. The image output unit 3 outputs the image data generated by the logic processing unit 2 to a downstream information processing device or the like.

[0042] The image processing system 1 can be configured, for example, as a stacked chip in which multiple chips are stacked. Fig. 4A is a diagram showing a first example of the stacked structure of the image processing system 1. The photodetector 10 in Fig. 4A has a two-layer structure in which a pixel chip 41 and a logic chip 42 are bonded together in this order. These chips are bonded together by vias or the like. Note that the pixel chip 41 and the logic chip 42 may be bonded together by Cu-Cu bonding or bumps instead of by vias.

[0043] The pixel chip 41 has, for example, a photodetector 10 disposed thereon. The logic chip 42 has, for example, a logic processing unit 2 and an image output unit 3 disposed thereon.

[0044] Fig. 4B is a diagram showing a second example of the stacked structure of the image processing system 1. The image processing system 1a in Fig. 4B has a three-layer structure formed by bonding a first pixel chip 43, a second pixel chip 44, and a logic chip 42 together in this order. Some of the components of the photodetector 10 (e.g., the imaging unit 11) are arranged on the first pixel chip 43. The remaining components of the photodetector 10 are arranged on the second pixel chip 44. By arranging some of the components of the photodetector 10 on the second pixel chip 44, the image processing system 1a can increase the proportion of the area of ​​the photoelectric conversion elements in the chip area, thereby improving sensitivity and enabling chip miniaturization.

[0045] The components arranged on each chip are not limited to those described above. The image processing system 1 may be configured with four or more stacked chips, or may be configured with a single flat chip.

[0046] 5 is a block diagram showing the circuit configuration of the pixel 20 according to the first embodiment of the present disclosure. The pixel 20 includes a photoelectric conversion element 21, a transistor 22, a pulse generation circuit 23, and a counter (CN) 24. In this specification, the pulse generation circuit 23 and the counter 24 are also referred to as a pixel circuit 25.

[0047] The photoelectric conversion element 21 is, for example, a single photon avalanche diode (SPAD). Fig. 5 shows an example in which one photoelectric conversion element 21 is arranged in one pixel 20. However, this is not limiting, and multiple photoelectric conversion elements 21 may be arranged in one pixel 20.

[0048] The transistor 22 acts as a current source and supplies a predetermined current I LOAD is supplied to the photoelectric conversion element 21. As a result, the voltage V SPAD is the power supply voltage level V DD When a photon is incident on the photoelectric conversion element 21, an electric charge (for example, an electron) is generated based on the photon.

[0049] In SPAD, electrons generated by a high electric field are accelerated and collide with semiconductor atoms, generating new electrons through ionization. These electrons then collide with other semiconductor atoms, generating a large current through a chain reaction of impact ionization, known as avalanche multiplication.

[0050] Due to avalanche multiplication, the voltage V of the photoelectric conversion element 21 SPAD The voltage V drops sharply. SPAD is a predetermined threshold V th When the value of the pulse generating circuit 23 is less than 1, the pulse current I SPAD The pulse generating circuit 23 generates a pulse current I SPAD The counter 24 counts the number of times the pulse signal Trig is generated by the pulse generating circuit 23 in synchronization with the clock signal clk. The count value of the counter 24 is transferred to the latch 13 as the output signal (i.e., pixel signal) Dout of the pixel circuit 25.

[0051] As described above, the photoelectric conversion element 21 can multiply one electron obtained by photoelectrically converting one photon through avalanche multiplication, and thus can detect photons one by one. Furthermore, each time the photoelectric conversion element 21 detects a photon, the pulse generation circuit 23 outputs one pulse signal Trig. The counter 24 counts the number of pulse signals Trig. This allows the imaging unit 11 to detect the number of photons incident on the pixel 20.

[0052] The pixel circuit 25 converts the voltage V of the photoelectric conversion element 21, which is an analog signal, into SPAD into a digital output signal Dout.

[0053] FIG. 6A is a cross-sectional view of pixel 20a. FIG. 6B is a cross-sectional view of pixel 20b. The pixels 20a and 20b have an on-chip lens (OCL: On Chip Micro Lens) 51. The on-chip lens 51 is disposed on the light incident surface (upper side in FIGS. 6A and 6B ) of the pixels 20a and 20b and focuses incident light. The pixels 20a and 20b may also have a color filter 52. Note that the transistor 22 and pixel circuit 25 are not shown in FIGS. 6A and 6B .

[0054] The pixel 20a has a light-shielding member (first light-shielding member) 53a that shields the photoelectric conversion element 21a from light. The pixel 20b has a light-shielding member (second light-shielding member) 53b that shields the photoelectric conversion element 21b from light. The light-shielding member 53a and the light-shielding member 53b are provided in symmetrical positions. The light-shielding members 53a and 53b are arranged, for example, symmetrically in the first direction X or symmetrically in the second direction Y with respect to the centers of the pixels 20a and 20b. In the examples of FIGS. 6A and 6B , the light-shielding members 53a and 53b are arranged symmetrically in the first direction X.

[0055] The on-chip lens 51 can focus light passing through a camera lens (not shown) to a point within the pixels 20a and 20b that corresponds to the position through which the camera lens passes. The light-shielding members 53a and 53b limit the incident position of light from the on-chip lens 51. This allows the light-shielding members 53a and 53b to limit the light entering the pixels 20a and 20b to only light that has passed through a predetermined position on the camera lens. For example, the light-shielding members 53a and 53b can be symmetrically arranged so that light that has passed through the upper and lower ends of the camera lens enters the pixels 20a and 20b, respectively. This makes it possible to simultaneously focus the upper and lower ends of the camera lens using the pixels 20a and 20b.

[0056] The pixels 20a and 20b have photoelectric conversion elements 21a and 21b, respectively. The photoelectric conversion elements 21a and 21b are a pair of photoelectric conversion elements 21.

[0057] The photoelectric conversion elements 21a and 21b each have a light absorbing portion 54 and a charge multiplying portion 55. The light absorbing portion 54 absorbs photons incident via the on-chip lens 51 and generates charges. The charge multiplying portion 55 multiplies the charges by avalanche multiplication.

[0058] Fig. 7 is a block diagram showing the circuit configuration of the pixel circuit 25 and the illuminance determination unit 12 according to the first embodiment of the present disclosure. Fig. 7 illustrates pixels 20a and 20b, and pixel circuits 25a and 25b connected to the pixels 20a and 20b, respectively.

[0059] 5, the pixel 20a and the pixel circuit 25a have a photoelectric conversion element 21a, a transistor 22a, a pulse generation circuit 23a, and a counter (first counter, CN1) 24a. The pixel 20b and the pixel circuit 25b have a photoelectric conversion element 21b, a transistor 22b, a pulse generation circuit 23b, and a counter (second counter, CN2) 24b.

[0060] The illuminance determining unit 12 includes changeover switches 61 a and 61 b ​​, a holder (first holder) 62 a , a holder (second holder) 62 b , and a signal generating circuit 63 .

[0061] The changeover switch 61a turns on at a predetermined timing for determining illuminance, and latches at least a portion of the output signal Dout_a of the counter 24a in a holder 62a. Similarly, the changeover switch 61b latches at least a portion of the output signal Dout_b of the counter 24b in a holder 62b in accordance with the timing for determining illuminance.

[0062] The holders 62a and 62b hold at least some bits of a plurality of bit strings representing the number of photons counted by the counters 24a and 24b, respectively. The bit strings held in the holders 62a and 62b include at least one or more lower-order bits of the plurality of bit strings representing the number of photons.

[0063] 7, the counters 24a and 24b measure 0 to 15 photons during a predetermined exposure period. The counters 24a and 24b output output signals Dout_a<0:3> and Dout_b<0:3>, each having a 4-bit bit string. The holders 62a and 62b hold the third bit (Dout_a<3> and Dout_b<3>), which is the least significant bit, and the second bit (Dout_a<2> and Dout_b<2>), which is the least significant bit, of the bit string.

[0064] The retainer 13 in FIG. 1 may be used for the retainers 62a and 62b.

[0065] A signal generation circuit 63 is provided for each of the multiple pairs of photoelectric conversion elements 21 a and 21 b. The signal generation circuit 63 determines whether the corresponding photoelectric conversion element 21 a or 21 b is in focus based on the bits held in the holders 62 a or 62 b. Furthermore, when the signal generation circuit 63 determines that the corresponding photoelectric conversion element 21 a or 21 b is in focus, it generates an output control signal FLAG that becomes a predetermined logic (for example, high level). The output control signal FLAG is generated for each of the multiple pairs of photoelectric conversion elements 21 a and 21 b.

[0066] The signal generating circuit 63 includes a logic circuit that determines whether the bits held in the holders 62a and 62b match. There are several possible configurations for the signal generating circuit 63, but in the example of FIG. 7, the signal generating circuit 63 includes logic circuits 64, 65, and 66.

[0067] The logic circuit 64 determines whether the signal levels of Dout_a<3> and Dout_b<3> match. The logic circuit 65 determines whether the signal levels of Dout_a<2> and Dout_b<2> match. In FIG. 7 , the logic circuits 64 and 65 are NAND circuits. While FIG. 7 shows an example in which a match is detected when Dout_a<3> and Dout_b<3> are both 1 and Dout_a<2> and Dout_b<2> are both 1, the logic circuits 64 and 65 may be configured with NAND circuits instead of NAND circuits to detect a match when Dout_a<3> and Dout_b<3> are both 1 or 0 and Dout_a<2> and Dout_b<2> are both 1 or 0.

[0068] The logic circuit 66 generates an output control signal FLAG based on the determination results of the logic circuits 64 and 65. The logic circuit 66 is, for example, a NOR circuit.

[0069] The signal generating circuit 63 in Figure 7 determines that the photoelectric conversion elements 21a and 21b are in focus when the signal levels of Dout_a<3> and Dout_b<3> match and the signal levels of Dout_a<2> and Dout_b<2> match, and outputs a high-level output control signal FLAG.

[0070] 8A and 8B are timing charts showing the operation of the illuminance determination unit 12 according to the first embodiment of the present disclosure. Fig. 8A shows the timing chart when the photoelectric conversion elements 21 a and 21 b are in focus. In this case, the photoelectric conversion elements 21 a and 21 b detect photons at approximately the same frequency.

[0071] 8A illustrates the lower bits Dout_a<1> to Dout_a<3> and Dout_b<1> to Dout_b<3> of the output signals Dout_a and Dout_b of the counters 24a and 24b, respectively. The most significant bits Dout_a<0> and Dout_b<0> of the output signals Dout_a and Dout_b are not illustrated in FIG.

[0072] At time t0, the exposure period starts, and the photoelectric conversion elements 21a and 21b start receiving light. The counters 24a and 24b count the number of photons detected by the photoelectric conversion elements 21a and 21b.

[0073] The exposure period includes an illuminance determination period (first period) and a normal exposure period (second period) that follows the illuminance determination period. During the illuminance determination period, the imaging unit 11 determines whether each pair of photoelectric conversion elements 21 a and 21 b is in focus. The illuminance determination period starts simultaneously with the exposure period.

[0074] At time t1, the illuminance determination period ends. At time t1, the changeover switches 61a and 61b in FIG. 7 are turned on. As a result, the holders 62a and 62b hold the lower bits of the output signals Dout_a and Dout_b at the end of the illuminance determination period. The signal generation circuit 63 determines whether the held bits of the output signals Dout_a and Dout_b match.

[0075] 8A, the photoelectric conversion elements 21a and 21b detect photons at approximately the same frequency, so that the signal levels of each bit of the output signals Dout_a and Dout_b are approximately the same throughout the exposure period, and the timing at which the signal levels change is also approximately the same. Therefore, at time t1, the signal levels of Dout_a<3> and Dout_b<3> are the same, and the signal levels of Dout_a<2> and Dout_b<2> are also the same.

[0076] As a result, the signal generation circuit 63 can determine that the photoelectric conversion elements 21 a and 21 b are in focus. After the illuminance determination period ends, the signal generation circuit 63 outputs a high-level output control signal FLAG. Thereafter, at time t2, the exposure period ends.

[0077] 8B shows a timing chart when the photoelectric conversion elements 21a and 21b are not in focus. In this case, the photoelectric conversion elements 21a and 21b detect photons at different frequencies. Therefore, in FIG. 8B, the signal levels of the output signals Dout_a and Dout_b and the timing at which the signal levels are switched are different.

[0078] At time t1 in Fig. 8B, the illuminance determination period ends. In the example of Fig. 8B, the signal levels of Dout_a<2> and Dout_b<2> are different. Therefore, the signal generation circuit 63 determines that the photoelectric conversion elements 21a and 21b are not in focus, and keeps the output control signal FLAG at a low level.

[0079] As described above, when the photoelectric conversion elements 21 a and 21 b are in focus, the signal levels of each bit of the output signals Dout_a and Dout_b are approximately the same. The signal generation circuit 63 according to the first embodiment of the present disclosure can determine whether the photoelectric conversion elements 21 a and 21 b are in focus by determining whether the signal levels of at least the lower bits of the output signals Dout_a and Dout_b are the same at the end of the illuminance determination period.

[0080] The illuminance determining unit 12 may be configured to omit the changeover switches 61a and 61b and include a holder that holds the output control signal FLAG at the end of the illuminance determination period.

[0081] 9 is a block diagram showing the configuration of the holder 13. The holder 13 has a plurality of D flip-flop circuits (DFF) 71 that hold a plurality of output signals Dout output by the imaging unit 11 and shift-transfer the signals to a downstream SRAM or the like, a repeater 72 that inputs a clock signal MCLK to the plurality of D flip-flop circuits 71 to synchronize them, and a GC generation circuit (GC Gen) 73 that generates a GC signal that resets the data held in the holder 13.

[0082] The output control signal FLAG generated by the signal generation circuit 63 may be held in a holder (FLAG Latch) 74 in the holder 13. This allows the determination results of the focus states of the photoelectric conversion elements 21 a and 21 b to be shared by multiple pixels 20.

[0083] In this specification, a plurality of pixels 20, the in-focus state of which is determined based on whether a pair of photoelectric conversion elements 21 a and 21 b is in-focus, are also referred to as one cluster. The output control signal FLAG held in the holder 74 is shared by clusters.

[0084] Fig. 10 is a timing chart illustrating the operation of the photodetector device 10. Fig. 10 illustrates a vertical synchronization signal XVS that synchronizes the operations of the multiple pixels 20, pulse currents PX1 and PX2 output from a pair of photoelectric conversion elements 21a and 21b, pulse signals CN1 and CN2 measured by the counters 24a and 24b in Fig. 7, and an output control signal FLAG.

[0085] At time t0, an exposure period and an illuminance determination period begin. At time t1, the illuminance determination period ends, and the signal generation circuit 63 determines whether each of the multiple pairs of photoelectric conversion elements 21a and 21b is in focus. At time t1, a normal exposure period begins. At time t2, the normal exposure period ends, and the exposure period also ends. At time t2, a blank period begins, during which the light receiving operation of the multiple pixels 20 is stopped.

[0086] At time t3, the blank period ends and the exposure period and illuminance determination period start again. Similarly, at time t4, the illuminance determination period ends, and at time t5, the normal exposure period and exposure period end and the blank period starts. At time t6, the blank period ends.

[0087] The exposure periods (times t0 to t2 and t3 to t5) are also periods during which the imaging unit 11 writes the output signal Dout to the holder 13 (Data Write). The blank periods (times t2 to t3 and t5 to t6) are also periods during which the output signal Dout is read from the holder 13 (Data Read).

[0088] The output signal Dout read out during the blank period is transferred to the logic processing unit 2 by the signal transfer unit 14. The logic processing unit 2 can generate image data for each exposure period based on the transferred output signal Dout.

[0089] During the normal exposure period, the pixel circuit 25 in the imaging unit 11 outputs the number of photons detected by the pair of photoelectric conversion elements 21 a and 21 b determined to be in focus during the illuminance determination period and the plurality of photoelectric conversion elements 21 in the corresponding cluster to the holder 13. Furthermore, the pixel circuit 25 does not output the number of photons detected by the pair of photoelectric conversion elements 21 a and 21 b not determined to be in focus and the plurality of photoelectric conversion elements 21 in the corresponding cluster to the holder 13. This enables the photodetector 10 to reduce the amount of data written to the holder 13 and to reduce the amount of data transferred by the signal transfer unit 14.

[0090] The logic processing unit 2 can generate image data of the in-focus range based on the number of photons detected by the pair of two photoelectric conversion elements 21a and 21b that have been determined to be in-focus.

[0091] 10 has the feature that the focus state of a plurality of pixels 20 is determined during an exposure period, and the determination result can be seamlessly reflected in the output signal Dout that is output during the exposure period. Therefore, the photodetector 10 does not need to perform an operation such as holding the output signal Dout until the focus state can be determined, and memory, etc. can be reduced.

[0092] 5, the pixel 20 and the pixel circuit 25 change the voltage V of the photoelectric conversion element 21 in parallel with the exposure period. SPAD The illuminance determination unit 12 has the advantage of being able to perform AD conversion on the output signal Dout, which is a digital signal. The illuminance determination unit 12 determines the focus state of the plurality of pixels 20 based on the output signal Dout, which is a digital signal. Therefore, the focus determination of the plurality of pixels 20 can be realized by a simple logical operation. The illuminance determination unit 12 can also be configured by a combination of simple logical circuits, etc.

[0093] As described above, the photodetector 10 has the feature that it can determine focus within the photodetector 10 without providing a large-scale circuit.

[0094] Another feature of the SPAD is its ability to detect even weak light. The SPAD detects incident light in units of one photon, and each detected photon is reflected in the output signal Dout as one count. This allows the photodetector 10 to determine the focusing state with high accuracy, in units of one photon.

[0095] The above-described method for determining the focus state can be applied to both low-light and high-light environments, and can determine the focus state over a wide range of illuminance.

[0096] 11A is a timing chart showing a first example of the operation of the pixel 20 in an out-of-focus state. In FIG. 11A, the pulse currents PX1 and PX2 rise at different frequencies, and the pulse signals CN1 and CN2 are output at different frequencies. Therefore, the output control signal FLAG does not go high at time t1.

[0097] 11A, the counting operations of the counters 24a and 24b are stopped based on the low-level output control signal FLAG. Therefore, high-level pulse signals CN1 and CN2 are not output during the normal exposure period. This reduces the power consumption of the photodetector 10 during the normal exposure period.

[0098] 5. The switch turns off when the output control signal FLAG is at a low level at the end of the illuminance determination period, thereby disconnecting the pulse generation circuit 23 from the counter 24. A similar switch may be disposed between the photoelectric conversion element 21 and the pulse generation circuit 23.

[0099] 11B is a timing chart showing a second example of the operation of the pixel 20 that is not in focus. During the normal exposure period in FIG. 11B, the counting operations of the counters 24a and 24b are stopped and the photoelectric conversion operations of the multiple photoelectric conversion elements 21 are stopped based on the low-level output control signal FLAG. Therefore, during the normal exposure period, no rising edges of the pulse currents PX1 and PX2 occur. This allows the power consumption of the photodetector 10 during the normal exposure period to be further reduced than in FIG. 11A.

[0100] The operation of FIG. 11B can be realized by providing a switch or the like between the photoelectric conversion element 21 and the transistor 22 of FIG. 5, which switches the connection in response to the output control signal FLAG.

[0101] Fig. 12 is a block diagram showing a circuit configuration of a pixel 20c according to a first modified example of the first embodiment of the present disclosure. The pixel 20c shown in Fig. 12 differs from the pixel 20 in Fig. 5 in that it has two photoelectric conversion elements 21a and 21b. The pixel 20c can determine whether or not focus is achieved using the pair of photoelectric conversion elements 21a and 21b.

[0102] The pixel 20c has a changeover switch 81a that switches whether the counter 24 and the photoelectric conversion element 21a are connected, and a changeover switch 81b that switches whether the counter 24 and the photoelectric conversion element 21b are connected. The changeover switches 81a and 81b are turned on in sequence. This allows the single counter 24 to count the number of photons detected by the photoelectric conversion elements 21a and 21b in sequence.

[0103] During the illuminance determination period, the holders 62a and 62b separately hold two numbers of photons counted in sequence by the counter 24. Specifically, the holder 62a holds the number of photons detected by the photoelectric conversion element 21a, and the holder 62b holds the number of photons detected by the photoelectric conversion element 21b.

[0104] As shown in Figures 8A and 8B, the signal generation circuit 63 compares at least some of the bits (lower bits) of the multiple bit strings representing the two photon counts held in the holders 62a and 62b, and generates an output control signal FLAG.

[0105] This allows the illuminance determining unit 12 to determine whether the photoelectric conversion elements 21a and 21b are in focus. The subsequent operations are the same as those shown in Fig. 10, Fig. 11A, or Fig. 11B.

[0106] Figure 13 is a cross-sectional view of a pixel 20c. The pixel 20c has an on-chip lens (optical member) 51a. The on-chip lens 51a is disposed for each pixel 20c, and guides incident light to each of the photoelectric conversion elements 21a and 21b for each of the multiple pairs of photoelectric conversion elements 21a and 21b. Note that the light absorption unit 54 and the charge multiplication unit 55 are not shown in Figure 13.

[0107] 14 is a diagram showing a partial configuration of an image capturing unit 11a according to a first modified example of the first embodiment of the present disclosure. The image capturing unit 11a has a plurality of pixels 20c arranged over the entire surface.

[0108] The image capture unit 11 in FIG. 2 uses a pair of pixels 20a and 20b to perform focus determination for each cluster made up of multiple pixels 20. In contrast, the image capture unit 11a in FIG. 14 is characterized in that focus determination can be performed for each pixel 20c using a pair of photoelectric conversion elements 21a and 21b that each pixel 20c has. This improves the accuracy of focus determination. In this specification, the above-described focusing method is also referred to as Dual Pixel AF (Auto Focus) or 2PD (Photodiode) AF.

[0109] Fig. 15 is a block diagram showing the circuit configuration of an illuminance determination unit 12a according to a second modification of the first embodiment of the present disclosure. The illuminance determination unit 12 in Fig. 7 outputs a 1-bit output control signal FLAG, whereas the illuminance determination unit 12a in Fig. 15 outputs a multi-bit output control signal FLAG. In the example of Fig. 15, the output control signal FLAG has 2 bits, but may have 3 or more bits.

[0110] The signal generating circuit 63a in FIG. 15 includes logic circuits 66a and 66b that output the bits FLAG<0> and FLAG<1> of the output control signal FLAG, respectively.

[0111] The logic circuit 66a outputs a FLAG<0> of a predetermined logic (for example, a high level) when the signal levels of Dout_a<3> and Dout_b<3> match. The logic circuit 66a is, for example, a NOT circuit that inverts the output signal of the logic circuit 64.

[0112] The logic circuit 66b outputs a FLAG<1> of a predetermined logic (for example, a high level) when the signal levels of Dout_a<3> and Dout_b<3> match and the signal levels of Dout_a<2> and Dout_b<2> match. The logic circuit 66b is, for example, a NOR circuit that performs a NOR operation on the output signals of the logic circuits 64 and 65.

[0113] In FLAG<0>, the in-focus state is determined based on the least significant bit, whereas in FLAG<1>, the in-focus state is determined based on two bits, the least significant bit and the bit immediately above it. In other words, FLAG<1> can determine the in-focus state with higher accuracy.

[0114] The photodetector 10 can switch between determining that the photoelectric conversion elements 21 a and 21 b are in focus when FLAG<0> is at a predetermined logic level and determining that the photoelectric conversion elements 21 a and 21 b are in focus when FLAG<1> is at a predetermined logic level, thereby enabling the precision of focus determination to be arbitrarily switched.

[0115] 15 shows an example in which the illuminance determination unit 12a outputs a two-bit output control signal FLAG. The output control signal FLAG is not limited to this example and may be composed of a bit string of three or more bits. The third bit of the output control signal FLAG may be used, for example, to determine whether the signal levels of Dout_a<1> and Dout_b<1> match.

[0116] Alternatively, FLAG<1> may be set to a predetermined logic level when the signal levels of Dout_a<2> and Dout_b<2> match, without determining Dout_a<3> and Dout_b<3>. In this case, FLAG<0> can determine the in-focus state with higher accuracy than FLAG<1>. The photodetector 10 may switch the accuracy of the in-focus determination between three levels: determining the in-focus state when FLAG<1> is set to a predetermined logic level, determining the in-focus state when FLAG<0> is set to a predetermined logic level, or determining the in-focus state when both FLAG<0> and FLAG<1> are set to a predetermined logic level.

[0117] 15 compares two or more bit strings of the output signals Dout_a and Dout_b. This allows the signal generation circuit 63a to generate an output control signal FLAG of two or more bits, each bit of which contains information on the comparison result of a different bit in the bit strings being compared. The photodetector 10 can change the precision of focus determination by switching which bit in the output control signal FLAG is used for focus determination.

[0118] FIG. 15 shows an example in which the signal generating circuit 63a is applied to the pixels 20a and 20b, but the present invention is not limited to this and may be applied to the pixel 20c in FIG.

[0119] In this way, the photodetector 10 according to the first embodiment of the present disclosure determines whether or not multiple pairs of photoelectric conversion elements 21 a and 21 b are in focus during an illuminance determination period, which is a portion of the exposure period from the beginning. Then, only if it determines that the pixel is in focus, it outputs the number of photons of the pixel determined to be in focus during the exposure period following the illuminance determination period. Therefore, it is possible to output only in-focus, valid pixel information, thereby reducing the burden on downstream signal processing. Furthermore, in determining whether or not the pixel is in focus, the photodetector 10 determines whether or not the pixel is in focus based on whether the signal levels of the lower-order bits of the multiple bit strings representing the number of photons are consistent. Therefore, it is possible to quickly determine whether or not the pixel is in focus.

[0120] As a result, the photodetector 10 utilizes the high-speed photodetection characteristic of SPAD to rapidly determine the in-focus state within the exposure period and output only the number of photons of in-focus pixels. Furthermore, this in-focus state determination method is highly accurate, enabling determination with an accuracy of one photon. Furthermore, this in-focus state determination method is applicable to both low-light and high-light environments, effectively utilizing the wide dynamic range characteristic of SPAD. In other words, the photodetector 10 according to the first embodiment of the present disclosure is capable of determining in-focus state by utilizing the characteristics of SPAD.

[0121] Second Embodiment In the first embodiment of the present disclosure, the illuminance determination unit 12 is used for focus determination. However, the present disclosure is not limited to this, and the illuminance determination unit 12 can be used in various other applications. For example, the illuminance determination unit 12 may be used to determine whether the amounts of light incident on multiple photoelectric conversion elements 21 are approximately the same. In this case, multiple pixel signals with the same number of photons are generated, and therefore, these multiple pixel signals may be thinned out and output.

[0122] Fig. 16 is a block diagram showing an image processing system 1b according to a second embodiment of the present disclosure. The illuminance determination unit 12b in Fig. 16 differs from the illuminance determination unit 12 in Fig. 1 in that it is used to thin out a plurality of photoelectric conversion elements 21 that detect a plurality of pixel signals or photons.

[0123] The image processing system 1b may have a configuration including an illuminance complementing unit 91. The illuminance complementing unit 91 instructs the logic processing unit 2 to complement the thinned pixel signals based on the determination result of the illuminance determining unit 12b.

[0124] The configuration of the illuminance determining unit 12b is the same as that of the illuminance determining unit 12 shown in Fig. 7 or 12. That is, the illuminance determining unit 12b determines whether the amounts of incident light on the plurality of photoelectric conversion elements 21 match.

[0125] 17A and 17B are timing charts illustrating the operation of the illuminance determination unit 12b according to the second embodiment of the present disclosure. FIG. 17A illustrates an example in which the incident light amounts match between multiple (e.g., two) photoelectric conversion elements 21 being compared. In this case, the rising frequencies of the pulse currents PX1 and PX2 are approximately the same between the two photoelectric conversion elements 21, and the output frequencies of the pulse signals CN1 and CN2 are also approximately the same. The illuminance determination unit 12b compares the lower-order bits of the multiple output signals Dout from the counter 24, similar to FIGS. 8A and 8B . When the lower-order bits of the multiple output signals Dout match, the illuminance determination unit 12b outputs an output control signal FLAG of a predetermined logic level (e.g., high level) at the end of the illuminance determination period (time t1).

[0126] The operation in FIG. 17A differs from the operation in FIG. 10 in that when the output control signal FLAG goes high, driving of one of the photoelectric conversion elements 21 is stopped during the normal exposure period.

[0127] 11B, the illuminance determination unit 12b stops the counting operation of the counter 24 based on the high-level output control signal FLAG, and also stops the photoelectric conversion operation of one of the photoelectric conversion elements 21. Therefore, in FIG. 17A, the pulse current PX2 does not rise, and the pulse signal CN2 is not output.

[0128] 11A, the illuminance determination unit 12b may continue the photoelectric conversion operation of the photoelectric conversion element 21 and stop the counting operation of the counter 24. Alternatively, the illuminance determination unit 12b may control the holder 13 not to transfer the output signal Dout to the signal transfer unit 14 without stopping the counter 24 and the photoelectric conversion element 21.

[0129] In this specification, stopping at least one of the operation of the photoelectric conversion element 21, the operation of the counter 24, and the transfer operation of the holder 13 so that the number of photons detected by the photoelectric conversion element 21 is not output from the holder 13 is also referred to as thinning out the photoelectric conversion element 21.

[0130] The holder 13 outputs the number of photons of the remaining photoelectric conversion elements 21 that have not been thinned out of the plurality of photoelectric conversion elements 21 to be compared to the signal transfer unit 14 .

[0131] 17A shows an example in which the illuminance determination unit 12b compares the amounts of light incident on two photoelectric conversion elements 21 and thins out one photoelectric conversion element 21. However, the illuminance determination unit 12b is not limited to this, and may compare the amounts of light incident on three or more photoelectric conversion elements 21 and thin out two or more photoelectric conversion elements 21.

[0132] 17B shows an example in which the amounts of incident light do not match among a plurality of (e.g., two) photoelectric conversion elements 21 being compared. In this case, the illuminance determination unit 12b outputs a low-level output control signal FLAG at the end of the illuminance determination period (time t1), similarly to FIG. 11A or FIG. 11B. FIG. 17B differs from FIG. 11A or FIG. 11B in that the photoelectric conversion elements 21 are not thinned out during the normal exposure period.

[0133] When the least significant bits of the plurality of output signals Dout match, the illuminance determination unit 12b determines that the amounts of incident light match, and thins out the photoelectric conversion elements 21. When the least significant bits of the plurality of output signals Dout do not match, the illuminance determination unit 12b determines that the amounts of incident light do not match, and does not thin out the photoelectric conversion elements 21. In other words, the illuminance determination unit 12b performs control that is opposite to the operation of the illuminance determination unit 12 according to the first embodiment of the present disclosure.

[0134] The multiple photoelectric conversion elements 21 to be determined may be included in one pixel 20. For example, as in FIG. 12 , one counter 24 may sequentially count the number of photons detected by the multiple photoelectric conversion elements 21 and input the count to the illuminance determination unit 12b. In this case, the illuminance determination unit 12b can determine whether the amount of incident light matches on a pixel-by-pixel basis. The illuminance determination unit 12b can also thin out some of the multiple photoelectric conversion elements 21 in the pixel 20.

[0135] The plurality of photoelectric conversion elements 21 to be determined may be included in a plurality of mutually adjacent pixels 20. For example, as in FIG. 7 , a plurality of counters 24 (e.g., counters 24a and 24b) may count the number of photons detected by the plurality of photoelectric conversion elements 21, respectively, and input the counted number of photons to the illuminance determination unit 12b. In this case, the illuminance determination unit 12b may thin out some of the plurality of pixels 20. The illuminance determination unit 12b may thin out not only pixels 20 for which it has confirmed that the amount of incident light is approximately the same, but also pixels 20 nearby, by considering the amount of incident light to be the same.

[0136] As described above, by thinning out the photoelectric conversion elements 21, the photodetector 10 can reduce the amount of data written to the holder 13 and can also reduce the amount of data transferred by the signal transfer unit 14. Furthermore, the logic processing unit 2 can compensate for the number of thinned photons and generate image data for the entire image capturing unit 11.

[0137] In this way, the illuminance determination unit 12b according to the second embodiment of the present disclosure determines whether the amounts of incident light match among the multiple photoelectric conversion elements 21 during the illuminance determination period within the exposure period. By thinning out some of the multiple photoelectric conversion elements 21 for which the amounts of incident light match, it is possible to reduce the amount of data transferred by the signal transfer unit 14, etc. Also, in the second embodiment of the present disclosure, as in the first embodiment, the photodetector 10 can determine whether the amounts of incident light match by taking advantage of the high-speed and high-precision photon detection that is a feature of SPAD.

[0138] (Application Examples) The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, construction machinery, or agricultural machinery (tractor).

[0139] 18 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 18 , the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside-vehicle information detection unit 7400, an inside-vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be an in-vehicle communication network conforming to any standard, such as a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), or FlexRay (registered trademark).

[0140] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a memory unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various controlled devices. Each control unit includes a network I / F for communicating with other control units via a communication network 7010, and a communication I / F for communicating with devices or sensors inside and outside the vehicle via wired or wireless communication. Figure 18 illustrates the functional configuration of the integrated control unit 7600, including a microcomputer 7610, a general-purpose communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I / F 7660, an audio / video output unit 7670, an in-vehicle network I / F 7680, and a memory unit 7690. The other control units also include a microcomputer, a communication I / F, a memory unit, and the like.

[0141] The drivetrain control unit 7100 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 7100 functions as a control device for a drive force generating device for generating drive force for the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating braking force for the vehicle. The drivetrain control unit 7100 may also function as a control device for an ABS (Antilock Brake System) or an ESC (Electronic Stability Control), etc.

[0142] A vehicle state detection unit 7110 is connected to the drivetrain control unit 7100. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor that detects the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine rotation speed, the rotation speed of the wheels, etc. The drivetrain control unit 7100 performs arithmetic processing using signals input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0143] The body system control unit 7200 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0144] The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, in accordance with various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

[0145] The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of an imaging unit 7410 and an outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0146] The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

[0147] 19 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0148] 19 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 7910, 7912, 7914, and 7916, a bird's-eye view image of the vehicle 7900 viewed from above can be obtained.

[0149] The outside vehicle information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The outside vehicle information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and above the windshield inside the vehicle cabin of the vehicle 7900 may be, for example, LIDAR devices. These outside vehicle information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

[0150] Returning to FIG. 18 , the explanation will be continued. The outside vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside vehicle information detection unit 7400 also receives detection information from the connected outside vehicle information detection unit 7420. If the outside vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside vehicle information detection unit 7400 emits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. Based on the received information, the outside vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, text on the road, etc. Based on the received information, the outside vehicle information detection unit 7400 may also perform environment recognition processing for recognizing rainfall, fog, road conditions, etc. Based on the received information, the outside vehicle information detection unit 7400 may also calculate the distance to an object outside the vehicle.

[0151] The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, vehicles, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by combining image data captured by different image capturing units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different image capturing units 7410.

[0152] The interior information detection unit 7500 detects information inside the vehicle. A driver state detection unit 7510 that detects the driver's state is connected to the interior information detection unit 7500, for example. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound from within the vehicle cabin. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or the driver gripping the steering wheel. The interior information detection unit 7500 may calculate the driver's level of fatigue or concentration based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The interior information detection unit 7500 may perform processing such as noise canceling on the collected audio signal.

[0153] The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. An input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 may be implemented by a device that can be operated by a passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input through a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a personal digital assistant (PDA) that can operate the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information using gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and the like operate this input unit 7800 to input various data to the vehicle control system 7000 and to instruct processing operations.

[0154] The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as an HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0155] The general-purpose communication I / F 7620 is a general-purpose communication I / F that mediates communication with various devices present in the external environment 7750. The general-purpose communication I / F 7620 may implement a cellular communication protocol such as GSM (Global System of Mobile communications), WiMAX (registered trademark), LTE (Long Term Evolution), or LTE-Advanced (LTE-A), or other wireless communication protocols such as a wireless LAN (also referred to as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). The general-purpose communication I / F 7620 may connect to a device (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I / F 7620 may connect to a terminal located near the vehicle (e.g., a terminal of a driver, pedestrian, or store, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.

[0156] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol designed for use in vehicles. The dedicated communication I / F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), which is a combination of a lower layer IEEE 802.11p and an upper layer IEEE 1609, DSRC (Dedicated Short Range Communications), or a cellular communication protocol. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0157] The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites), and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

[0158] The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations or the like installed on the road, and acquires information such as the current location, congestion, road closures, required travel time, etc. The function of the beacon receiving unit 7650 may be included in the dedicated communication I / F 7630 described above.

[0159] The in-vehicle device I / F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I / F 7660 may establish wireless connections using wireless communication protocols such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). Furthermore, the in-vehicle device I / F 7660 may establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-Definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or a wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I / F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

[0160] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals in accordance with a predetermined protocol supported by the communication network 7010.

[0161] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 in accordance with various programs based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. For example, the microcomputer 7610 may calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output control commands to the drivetrain control unit 7100. For example, the microcomputer 7610 may perform cooperative control aimed at realizing functions of an Advanced Driver Assistance System (ADAS), including vehicle collision avoidance or impact mitigation, following driving based on the following distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may perform cooperative control for the purpose of autonomous driving, in which the vehicle travels autonomously without relying on driver operation, by controlling a driving force generating device, a steering mechanism, a braking device, etc. based on information acquired about the vehicle's surroundings.

[0162] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people, and create local map information including information about the vicinity of the vehicle's current location, based on information acquired via at least one of the general-purpose communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I / F 7660, and the in-vehicle network I / F 7680. Furthermore, the microcomputer 7610 may predict dangers, such as a vehicle collision, the approach of a pedestrian, or entry into a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or turning on a warning lamp.

[0163] The audio / image output unit 7670 transmits at least one audio and / or image output signal to an output device capable of visually or audibly notifying vehicle occupants or the outside of the vehicle of information. In the example of FIG. 18 , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as examples of the output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may also be other devices, such as headphones, a wearable device such as an eyeglass-type display worn by the occupant, a projector, or a lamp. When the output device is a display device, the display device visually displays results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats, such as text, images, tables, and graphs. When the output device is an audio output device, the audio output device converts audio signals, such as reproduced audio data or acoustic data, into analog signals and audibly outputs the analog signals.

[0164] In the example shown in FIG. 18 , at least two control units connected via the communication network 7010 may be integrated into a single control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by one of the control units may be performed by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by one of the control units. Similarly, a sensor or device connected to one of the control units may be connected to another control unit, and multiple control units may transmit and receive detection information to and from each other via the communication network 7010.

[0165] A computer program for realizing each function of the image processing system 1 according to this embodiment described with reference to FIG. 1 can be implemented in any control unit or the like. A computer-readable recording medium storing such a computer program can also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network without using a recording medium.

[0166] In the vehicle control system 7000 described above, the image processing system 1 according to the present embodiment described with reference to Fig. 1 can be applied to the imaging unit 7410 of the application example shown in Fig. 18. This enables high-precision and high-speed extraction of the focus range, etc.

[0167] Furthermore, at least some of the components of the image processing system 1 described using Fig. 1 may be realized in a module (for example, an integrated circuit module configured on a single die) for the integrated control unit 7600 shown in Fig. 18. Alternatively, the image processing system 1 described using Fig. 1 may be realized by multiple control units of the vehicle control system 7000 shown in Fig. 18.

[0168] The present technology may be configured as follows: (1) A photodetector including: a plurality of pairs of photoelectric conversion elements that each detect incident photons; and a plurality of pixel circuits that, for each of the plurality of pairs of photoelectric conversion elements, output the number of photons detected by the pair of photoelectric conversion elements that is determined to be in focus by comparing the number of photons detected by the pair of photoelectric conversion elements, and do not output the number of photons detected by the pair of photoelectric conversion elements that is not determined to be in focus. (2) The photodetector according to (1), including a plurality of signal generation circuits that are provided for each of the pairs and generate a plurality of output control signals that become a predetermined logic when the pair is determined to be in focus. (3) The photodetector according to (2), wherein each of the plurality of pixel circuits determines whether each pair of photoelectric conversion elements is in focus during a first period within an exposure period of the plurality of pairs of photoelectric conversion elements, and outputs the number of photons detected by the pair of photoelectric conversion elements that is determined to be in focus during a second period within the exposure period after the first period. (4) The photodetector according to (3), further comprising: a plurality of counters, each including a first counter and a second counter, that count the number of photons detected by the pair of two photoelectric conversion elements during the first period, and the signal generating circuit generates the output control signal by comparing at least some bits of a plurality of bit strings that represent the number of photons counted by the first counter and the second counter. (5) The photodetector according to any one of (1) to (4), further comprising: a first light-shielding member that shields one of the pair of two photoelectric conversion elements and a second light-shielding member that shields the other of the pair of photoelectric conversion elements. (6) The photodetector according to (5), further comprising: a plurality of pairs of photoelectric conversion elements each having the first light-shielding member and the second light-shielding member, and arranged between a plurality of photoelectric conversion elements that detect incident photons without determining whether or not the photoelectric conversion elements are in focus. (7) The photodetector according to (2), further comprising: an optical member that guides incident light to each of the pair of two photoelectric conversion elements.(8) The photodetector according to (7), wherein the plurality of pairs of photoelectric conversion elements determine whether or not they are in focus and detect the number of incident photons, and the plurality of pixel circuits determine whether or not each pair is in focus during a first period that is a part of an exposure period of the plurality of pairs of photoelectric conversion elements, and output the number of photons detected by the two photoelectric conversion elements of the pair determined to be in focus during a second period within the exposure period after the first period. (9) The photodetector according to (8), wherein the photodetector includes, for each of the plurality of pairs of photoelectric conversion elements, one counter that sequentially counts the number of photons detected by the two photoelectric conversion elements of the pair during the first period, and two holders that separately hold the two numbers of photons counted sequentially by the one counter, and the signal generation circuit generates the output control signal by comparing at least some bits of a plurality of bit strings representing the two numbers of photons held in the two holders. (10) The photodetector according to (4) or (9), wherein the signal generation circuit compares two or more bit strings among a plurality of bit strings representing the number of photons counted by the counter to generate the two or more bit output control signal, and each bit of the output control signal is a result of comparing different bits of the bit strings being compared. (11) The photodetector according to (4), (9), or (10), wherein the signal generation circuit stops the counting operation of the counter during the second period when at least some of the bits among the plurality of bit strings representing the number of photons do not match. (12) The photodetector according to (4), (9), or (10), wherein the signal generation circuit stops the counting operation of the counter and stops photoelectric conversion operations of the plurality of photoelectric conversion elements during the second period when at least some of the bits among the plurality of bit strings representing the number of photons do not match. (13) A photodetection device comprising: a plurality of photoelectric conversion elements that respectively detect incident photons; and a plurality of pixel circuits that thin out some of the photoelectric conversion elements having approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements during a first period within an exposure period, and output the number of photons detected by the photoelectric conversion elements that have not been thinned out during a second period within the exposure period that follows the first period.(14) The photodetector according to (13), wherein the plurality of pixel circuits, when the amounts of incident light on two or more pixels arranged identically or adjacently among the plurality of photoelectric conversion elements are approximately equal, thin out some of the photoelectric conversion elements included in the two or more pixels and output the number of photons detected by the remaining photoelectric conversion elements. (15) The photodetector according to (13) or (14), further comprising a signal generation circuit that generates an output control signal of a predetermined logic when thinning out some of the photoelectric conversion elements having approximately the same amount of incident light. (16) The photodetector according to (15), further comprising a first counter and a second counter that count the number of photons output from first and second photoelectric conversion elements included in neighboring pixels among the plurality of photoelectric conversion elements, and the signal generation circuit generates the output control signal by comparing at least some bits of a plurality of bit strings representing the number of photons counted by the first counter and the second counter. (17) The photodetector according to (16), wherein the signal generation circuit stops the counting operation of at least one of the first counter or the second counter during the second period when the output control signal becomes the predetermined logic. (18) The photodetector according to (16), wherein the signal generation circuit stops the counting operation of at least one of the first counter or the second counter during the second period and stops photoelectric conversion operations of at least some of the photoelectric conversion elements of the plurality of photoelectric conversion elements during the second period when the output control signal becomes the predetermined logic. (19) The photodetector according to any one of (1) to (18), wherein the photoelectric conversion elements include SPADs (Single Photon Avalanche Diodes).(20) An image processing system comprising: a photodetection device having a plurality of photoelectric conversion elements that respectively detect incident photons; and a plurality of pixel circuits that, for each pair of photoelectric conversion elements among the plurality of photoelectric conversion elements, output the number of photons detected by two paired photoelectric conversion elements that are determined to be in an in-focus state by comparing the number of photons detected by the two paired photoelectric conversion elements, and do not output the number of photons detected by paired photoelectric conversion elements that are not determined to be in an in-focus state, or thin out some photoelectric conversion elements that have approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements in a first period within an exposure period, and output the number of photons detected by photoelectric conversion elements that are not thinned out in a second period within the exposure period that follows the first period; and an image generation unit that generates image data based on the number of photons detected by the plurality of pixel circuits.

[0169] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0170] 1, 1a, 1b Image processing system, 2 Logic processing unit, 3 Image output unit, 10 Photodetector, 11, 11a Imaging unit, 12, 12a, 12b Illuminance determination unit, 13, 62a, 62b, 74 Holder, 14 Signal transfer unit, 20, 20a, 20b, 20c Pixel, 21, 21a, 21b Photoelectric conversion element, 22, 22a, 22b Transistor, 23, 23a, 23b Pulse generation circuit, 24, 24a, 24b Counter, 25, 25a, 25b Pixel circuit, 30 Imaging range, 31 Subject, 32 Focusing range, 41 Pixel chip, 42 Logic chip, 43 First pixel chip, 44 Second pixel chip, 51, 51a On-chip lens, 52 Color filter, 53a, 53b Light shielding member, 54 Light absorption unit, 55 charge multiplication unit, 61a, 61b, 81a, 81b changeover switch, 63, 63a signal generation circuit, 64, 65, 66, 66a, 66b logic circuit, 71 D flip-flop circuit, 72 repeater, 73 GC generation circuit, 91 illuminance complement unit

Claims

1. A photodetection device comprising: multiple pairs of photoelectric conversion elements that each detect incident photons; and multiple pixel circuits that, for each pair of the multiple pairs of photoelectric conversion elements, output the number of photons detected by two paired photoelectric conversion elements that are determined to be in focus by comparing the number of photons detected by the two paired photoelectric conversion elements, and do not output the number of photons detected by a pair of photoelectric conversion elements that are not determined to be in focus.

2. The photodetector according to claim 1, further comprising a plurality of signal generating circuits provided for each of the pairs, for generating a plurality of output control signals that assume a predetermined logic level when the in-focus state is determined.

3. The photodetection device according to claim 2, wherein each of the plurality of pixel circuits determines whether each of the plurality of pairs of photoelectric conversion elements is in focus during a first period within the exposure period of the plurality of pairs of photoelectric conversion elements, and outputs the number of photons detected by the two photoelectric conversion elements of the pair determined to be in focus during a second period within the exposure period after the first period.

4. The photodetector according to claim 3, further comprising a plurality of counters including a first counter and a second counter for counting the number of photons detected by the two photoelectric conversion elements of the pair during the first period for each of the pairs, and wherein the signal generating circuit generates the output control signal by comparing at least some of the bits of a plurality of bit strings representing the number of photons counted by the first counter and the second counter.

5. The photodetector according to claim 1, further comprising, for each pair, a first light-shielding member that shields one of the paired photoelectric conversion elements from light, and a second light-shielding member that shields the other of the paired photoelectric conversion elements from light.

6. The optical detection device according to claim 5, wherein the plurality of pairs of photoelectric conversion elements having the first light-shielding member and the second light-shielding member are arranged between a plurality of photoelectric conversion elements that detect incident photons without determining whether or not they are in focus.

7. The photodetector according to claim 2, further comprising an optical member for guiding incident light to each of the two photoelectric conversion elements in each pair of the plurality of pairs of photoelectric conversion elements.

8. The photodetection device according to claim 7, wherein the plurality of pairs of photoelectric conversion elements determine whether or not they are in focus and detect the number of incident photons, and the plurality of pixel circuits determine whether or not each pair is in focus during a first period that is part of the exposure period of the plurality of pairs of photoelectric conversion elements, and output the number of photons detected by the two photoelectric conversion elements of the pair that have been determined to be in focus during a second period within the exposure period after the first period.

9. A photodetector according to claim 8, further comprising: one counter for each of said plurality of pairs of photoelectric conversion elements, which counts in sequence the number of photons detected by said pair of two photoelectric conversion elements during said first period; and two holders for separately holding the two numbers of photons counted in sequence by said one counter; and said signal generating circuit generates said output control signal by comparing at least some bits of a plurality of bit strings representing said two numbers of photons held by said two holders.

10. The photodetector device of claim 4, wherein the signal generating circuit compares two or more bit strings among a plurality of bit strings representing the number of photons counted by the counter to generate the two or more bit output control signal, and each bit of the output control signal is the result of comparing different bits of the bit strings being compared.

11. The photodetector according to claim 4, wherein the signal generating circuit stops the counting operation of the counter during the second period when at least some of the bits in the multiple bit strings representing the number of photons do not match.

12. The photodetector device according to claim 4, wherein the signal generating circuit stops the counting operation of the counter and stops the photoelectric conversion operation of the photoelectric conversion elements during the second period when at least some of the bits in the bit strings representing the number of photons do not match.

13. A photodetection device comprising: a plurality of photoelectric conversion elements that each detect incident photons; and a plurality of pixel circuits that thin out some of the photoelectric conversion elements having approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements during a first period within an exposure period, and output the number of photons detected by the photoelectric conversion elements that were not thinned out during a second period within the exposure period after the first period.

14. The photodetector device according to claim 13, wherein, when the amounts of incident light on two or more identical or closely arranged pixels among the plurality of photoelectric conversion elements are approximately equal, the plurality of pixel circuits thin out some of the photoelectric conversion elements included in the two or more pixels and output the number of photons detected by the remaining photoelectric conversion elements.

15. The photodetector according to claim 13, further comprising a signal generating circuit that generates an output control signal of a predetermined logic when thinning out some of the photoelectric conversion elements having approximately the same amount of incident light.

16. The photodetector according to claim 15, further comprising a first counter and a second counter that count the number of photons output from a first photoelectric conversion element and a second photoelectric conversion element included in a neighboring pixel among the plurality of photoelectric conversion elements, and the signal generation circuit generates the output control signal by comparing at least some bits among a plurality of bit strings representing the number of photons counted by the first counter and the second counter.

17. The photodetector according to claim 16, wherein the signal generating circuit stops the counting operation of at least one of the first counter and the second counter during the second period when the output control signal reaches the predetermined logic level.

18. The photodetector device according to claim 16, wherein when the output control signal reaches the predetermined logic level, the signal generating circuit stops the counting operation of at least one of the first counter or the second counter during the second period, and stops the photoelectric conversion operation of at least some of the photoelectric conversion elements of the plurality of photoelectric conversion elements.

19. The photodetector according to claim 1, wherein the photoelectric conversion element includes a SPAD (Single Photon Avalanche Diode).

20. An image processing system comprising: a photodetection device having a plurality of photoelectric conversion elements that each detect incident photons; and a plurality of pixel circuits that, for each pair of photoelectric conversion elements among the plurality of photoelectric conversion elements, output the number of photons detected by two paired photoelectric conversion elements that are determined to be in focus by comparing the number of photons detected by the two paired photoelectric conversion elements, and do not output the number of photons detected by paired photoelectric conversion elements that are not determined to be in focus, or thin out some photoelectric conversion elements with approximately the same amount of incident light based on the number of photons detected by the plurality of photoelectric conversion elements during a first period within an exposure period, and output the number of photons detected by photoelectric conversion elements that are not thinned out during a second period within the exposure period after the first period; and an image generation unit that generates image data based on the number of photons detected by the plurality of pixel circuits.

Images