Light detection device

Abstract

[Problem] To reduce pixel size and improve S / N ratio. [Solution] A light detection device is provided with: a plurality of pixels each having a photoelectric conversion element; a first circuit that is provided for each pixel group including two or more pixels among the plurality of pixels, and that adds or counts signals output from at least one pixel included in the pixel group; and a second circuit that performs, in parallel with the first circuit, processing that adds or counts signals output from at least one pixel among the pixels included in the pixel group, where at least some of the pixels differ from the first circuit.

Description

Optical detection device

[0001] The present disclosure relates to an optical detection device.

[0002] With the progress of semiconductor integrated circuit technology, high resolution has been achieved by reducing the pixel size and increasing the number of pixels of image sensors. When the pixel size is reduced, the amount of charge that can be accumulated in one pixel is limited. Therefore, a technique for improving the dynamic range by sharing one pixel circuit among multiple pixels is known.

[0003] In addition, a technique for performing autofocus adjustment by dividing a pixel into two and using it as a phase difference detection pixel has been put into practical use.

[0004] For example, Patent Document 1 discloses an imaging device provided with two avalanche photodiodes (hereinafter referred to as APDs) and two counters in a unit pixel. In one counter, the number of photons detected by one APD is counted, and in the other counter, the number of photons detected by both APDs is counted.

[0005] In addition, Patent Document 2 discloses an imaging device provided with two APDs, two counters, and two selectors in a unit pixel. In the first counting mode, each counter counts the number of photons detected by the corresponding APD. In the second counting mode, one counter counts the number of photons detected by both APDs, and the other counter counts the pulses of the carry signal of one counter.

[0006] Japanese Patent Application Laid-Open No. 2020-127122, Japanese Patent Application Laid-Open No. 2020-145502

[0007] However, in both Patent Documents 1 and 2, since a counter is provided for each APD, it is difficult to reduce the pixel size. In particular, increasing the number of bits of the counter is effective for improving the S / N ratio. However, as the number of bits increases, the circuit area of the counter increases. Therefore, it is not easy to improve the S / N ratio in Patent Documents 1 and 2.

[0008] Therefore, the present disclosure provides an optical detection device capable of reducing the pixel size and improving the S / N ratio.

[0009] To solve the above problems, the present disclosure provides a photodetector comprising: a plurality of pixels, each having a photoelectric conversion element; a first circuit provided for each pixel group including two or more pixels from the plurality of pixels, which adds or counts signals output from at least one pixel included in the pixel group; and a second circuit that performs in parallel with the first circuit a process of adding or counting signals output from at least one pixel included in the pixel group, which differs from the first circuit in at least some of its pixels.

[0010] The first circuit may output a compressed signal obtained by compressing the signals output from two or more pixels included in the pixel group, while the second circuit may output an uncompressed signal obtained by one pixel included in the pixel group.

[0011] The first circuit may have a switching circuit that randomly selects two or more pixels included in the pixel group, or switches between the types of the two or more pixels.

[0012] The first circuit and the second circuit may add or count signals output from at least one common pixel.

[0013] The first circuit and the second circuit may add or count signals output from different pixels.

[0014] The pixel group comprises one or more first pixels and one or more second pixels. The first circuit may add or count the signals output from the one or more first pixels, and the second circuit may add or count the signals output from the one or more second pixels.

[0015] The first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon, the first circuit has a first counter that counts the pulse signals output from the one or more first pixels included in the pixel group, and the second circuit may have a second counter that counts the pulse signals output from the one or more second pixels included in the pixel group.

[0016] The first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon; the first circuit has a first counter that counts the pulse signals output from the one or more first pixels and the one or more second pixels included in the pixel group; and the second circuit may have a second counter that counts the pulse signals output from the one or more second pixels included in the pixel group.

[0017] The second pixel may have different photoelectric conversion characteristics than the first pixel.

[0018] The second pixel may have a smaller photoelectric conversion region than the first pixel.

[0019] The second pixel may have lower sensitivity than the first pixel.

[0020] The first pixel is a pixel that detects the amount of incident light or the presence or absence of incident light, and the second pixel may be a pixel that detects a change in the amount of incident light.

[0021] The pixel group comprises at least one pair of first and second pixels, and the pair of first and second pixels may be phase difference detection pixels.

[0022] The system may also include a first control unit for controlling the exposure period or dead time of one or more first pixels, and a second control unit for controlling the exposure period or dead time of one or more second pixels.

[0023] The device comprises a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of the phase difference detection pixels, and the first counter counts the signals output from the N first pixels in the first pixel group and the signals output from the N second pixels in the second pixel group for each first pixel group containing N (where N is an integer of 2 or more and less than or equal to M) of the M first pixels included in the pair of M, and the second counter may count the signals output from the N second pixels in the second pixel group for each second pixel group containing N of the M second pixels.

[0024] The device comprises a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of the phase difference detection pixels, the first counter counts the signals output from the N first pixels included in the first pixel group for each first pixel group containing N (where N is an integer of 2 or more and less than or equal to M) of the M first pixels included in the pair of M, and the second counter may count the signals output from the N second pixels included in the second pixel group for each second pixel group containing N of the M second pixels included in the pair of M.

[0025] The device may also include a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of which are phase difference detection pixels, and for each first pixel group containing N (where N is an integer of 1 or more) of the M first pixels included in the M pair, the N first counters count the signals output from each of the N first pixels, and for each first pixel group containing N of the M second pixels included in the M pair, the M second counters count the signals output from each of the N second pixels.

[0026] The system comprises a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of which is a phase difference detection pixel. The first counter counts the M signals output from each of the M first pixels in the M pair and the M signals output from each of the M second pixels. The second counter may count the M signals output from each of the M second pixels in the M pair.

[0027] The system comprises N sets (N is an integer of 2 or more) of M pairs (M is an integer of 1 or more) of the first and second pixels, each of which is a phase difference detection pixel. The first counter counts the signal output from one of the M first pixels for each of the N sets, and the second counter counts the signal output from one of the M second pixels for each of the N sets.

[0028] The photoelectric conversion element may be a SPAD (Single Photon Avalanche Diode).

[0029] A block diagram showing the configuration of a photodetection system including the photodetector according to this disclosure. A diagram showing the planar configuration of the imaging unit according to this disclosure. A block diagram showing the internal configuration of the imaging unit according to the first embodiment. A circuit diagram showing an example of the specific configuration of the first and second circuits. A diagram showing an example of randomly selecting a plurality of pixels included in the first circuit from a group of pixels. Detailed circuit diagrams of the first and second circuits in Figure 5. A diagram showing an example of a group of pixels including 4x4 pixels. A diagram showing a matrix that represents the operation of individually selecting whether or not to count the output of each pixel included in the group of pixels in the first circuit. A circuit diagram showing an example of the specific configuration of the first and second circuits. A diagram showing the unit of compression of special pixels. Circuit diagrams of the first and second circuits that compress and count both normal pixels and special pixels. A diagram showing the compression range in a pixel having 2x1 OCL. Circuit diagrams of the first and second circuits corresponding to Figure 12. A diagram showing a group of pixels that perform compressed counting by adjusting the S / N ratio. Circuit diagrams of the first and second circuits that have a function to adjust the S / N ratio of normal pixels and special pixels. A diagram showing the pixel arrangement of a Dual PD. Circuit diagrams of the first and second circuits in Dual PD. Circuit diagram relating to one modification of Figure 17. Diagram showing a pixel configuration in which on-chip lenses are arranged in units of 2x2 pixels of the same color. Circuit diagrams of the first and second circuits in 2x2 OCL. Circuit diagram relating to the first modification of Figure 20. Diagram showing an example of compressed counting in units of a 4x8 pixel area. Diagram showing an example of compressed counting relating to one modification of Figure 22. Circuit diagrams of the first and second pixel groups equipped with a switch and first to fourth counters. Diagram showing the first and second pixel groups in the pixel configuration of Dual PD. Diagram showing the first and second pixel groups including pixels different from those in Figure 25A. Diagram showing the first and second pixel groups in the pixel configuration of light-shielding ZAF. Diagram showing the first and second pixel groups with a different pixel configuration from Figure 26A. Circuit diagram relating to a second modification of the pixel configuration of 2x2 OCL. Diagram showing a third modification of the pixel configuration of 2x2 OCL. A block diagram of the pixel group relating to the third modification. A diagram showing an example of the circuit configuration of a special pixel. A diagram showing the first example of the circuit configuration of a normal pixel. A diagram showing the second example of the circuit configuration of a normal pixel. A diagram showing the first example of compressed counting in the pixel configuration of an Octa PD. A diagram showing the second example of compressed counting in the pixel configuration of an OCTA PD. A circuit diagram of the pixel group performing the compressed counting in the first example of Figure 33A. A circuit diagram of the pixel group relating to one modification of Figure 34.A block diagram showing an example of the general configuration of a vehicle control system. An explanatory diagram showing an example of the installation location of the external information detection unit and the imaging unit.

[0030] The embodiments of the photodetector will be described below with reference to the drawings. While the main components of the photodetector will be described below, there may be components and functions not shown or described in the drawings. The following description does not exclude any components or functions not shown or described.

[0031] Figure 1 is a block diagram showing the configuration of a photodetection system 2 including a photodetector 1 according to this disclosure. The photodetector 1 is a device that detects incident light. The photodetection system 2 is a device that performs phase difference detection and generates image data based on the incident light detected by the photodetector 1. Phase difference detection is used, for example, to perform autofocus adjustment.

[0032] The light detection system 2 includes a light detection device 1, a logic processing unit 3, and an image output unit 4. The light detection device 1 includes an imaging unit 5, a phase difference detection unit 6, a holding unit 7, and a signal transfer unit 8.

[0033] The imaging unit 5 has a plurality of pixels arranged in a two-dimensional direction. Figure 2 is a diagram showing the planar configuration of the imaging unit 5 according to this disclosure. In this specification, the left-right (horizontal) direction in Figure 2 is referred to as the first direction X, and the up-down (vertical) direction in Figure 2 is referred to as the second direction Y. In this specification, the plurality of pixels arranged in a two-dimensional direction as shown in Figure 2 is also referred to as the pixel array. In the imaging unit 5, the number of pixels arranged in the first direction X and the second direction Y is arbitrary.

[0034] Each of the multiple pixels has a photoelectric conversion element that detects incident photons. Each photoelectric conversion element is, for example, a SPAD (Single Photon Avalanche Diode). Alternatively, each photoelectric conversion element may be a photodiode that accumulates charge in proportion to the amount of incident light.

[0035] Multiple pixels can selectively receive light containing specific color information by providing color filters or the like. Figure 2 shows an example where pixels PX that receive red (R) light, pixels PX that receive blue (B) light, and pixels PX that receive green (G) light are arranged. Note that multiple pixels PX may also receive incident light without distinguishing between colors, without providing color filters.

[0036] At least some of the multiple pixels PX are, for example, phase-difference detection pixels. In a phase-difference detection pixel, half of the pixel PX area is shielded from light. As shown in Figure 2, the shielded area of ​​the phase-difference detection pixel (the black area in Figure 2) is provided in, for example, the left half, right half, upper half, or lower half of the pixel PX. In this case, by pairing a phase-difference detection pixel with its left half shielded with a phase-difference detection pixel with its right half shielded, phase difference information in the left-right direction can be obtained. Similarly, by pairing a phase-difference detection pixel with its upper half shielded with a phase-difference detection pixel with its lower half shielded, phase difference information in the up-down direction can be obtained. By combining the left-right and up-down phase difference information, highly accurate autofocus adjustment can be performed.

[0037] Figure 2 shows an example where some pixels PX in the pixel array are phase-difference detection pixels, but all pixels PX in the pixel array may also be phase-difference detection pixels.

[0038] The imaging unit 5 counts the number of photons detected for each pixel PX. The number of photons detected for each pixel PX is stored in the storage unit 7 in Figure 1 and then transferred to the logic processing unit 3 via the signal transfer unit 8. The phase difference detection unit 6 in Figure 1 detects phase difference information by comparing the number of photons detected for two paired phase difference detection pixels and determines whether or not the image is in focus. The result determined by the phase difference detection unit 6 may be notified to at least one of the imaging unit 5 or the storage unit 7. For example, the storage unit 7 can generate only image data in focus by not storing the number of photons detected when it is determined that the image is not in focus.

[0039] The phase difference detection unit 6 may be provided outside the photodetector 1. Furthermore, the phase difference detection unit 6 is not mandatory and may be omitted. The logic processing unit 3 generates image data based on the number of photons detected output from the imaging unit 5. The image output unit 4 outputs the image data generated by the logic processing unit 3, for example, in frame units.

[0040] The imaging unit 5 according to this disclosure can output, for example, a compressed signal for each pixel group 10 containing two or more pixels PX, and an uncompressed signal for each pixel PX. The rectangular frame in Figure 2 shows an example of a pixel group 10, which is the unit of compression. The pixel group 10 in Figure 2 contains 2 x 2 pixels PX. Each pixel group 10 may or may not contain phase difference detection pixels. Two adjacent pixel groups 10 may be shifted by one pixel PX or by multiple pixels PX. Since green pixels PX are often used for phase difference detection pixels, Figure 2 shows an example in which each pixel group 10 contains four green pixels PX.

[0041] (First Embodiment) The imaging unit 5 of the light detection device 1 according to the first embodiment has normal pixels PX1 and special pixels PX2.

[0042] Figure 3 is a block diagram showing the internal configuration of the imaging unit 5 according to the first embodiment. Figure 3 illustrates the internal configuration of one pixel group 10 of the imaging unit 5. The imaging unit 5 has a first circuit 11 and a second circuit 12 for each pixel group 10.

[0043] The first circuit 11 adds or counts signals output from at least one pixel PX included in the pixel group 10. In the present embodiment, an example in which the pixel PX has a SPAD will be mainly described. However, the pixel PX may have a photoelectric conversion element (for example, a photodiode) that accumulates charges according to the incident light amount. In this case, for example, a signal obtained by performing analog-digital conversion (AD conversion) for each pixel PX is output. When the pixel PX has a SPAD or an APD, the first circuit 11 counts signals output from at least some of the pixels PX included in the pixel group 10. Further, when the pixel PX has a photodiode, the first circuit 11 adds signals output from at least some of the pixels PX included in the pixel group 10. In a more specific example, the first circuit 11 adds or counts signals output from four pixels PX included in the pixel group 10.

[0044] The second circuit 12 performs, in parallel with the first circuit 11, a process of adding or counting signals output from at least one pixel PX that is at least partially different from the first circuit 11 among the pixels PX included in the pixel group 10. In a more specific example, the second circuit 12 adds or counts signals output from one pixel PX included in the pixel group 10. The first circuit 11 outputs a signal obtained by compressing signals output from two or more pixels PX included in the pixel group 10. The second circuit 12 outputs a signal output from one pixel PX included in the pixel group 10 without compressing it.

[0045] In this specification, the pixel PX that the second circuit 12 adds or counts is sometimes referred to as a special pixel PX2, and the pixel PX that the first circuit 11 adds or counts and that is other than the special pixel PX2 is sometimes referred to as a normal pixel PX1. In the example of FIG. 3, the first circuit 11 adds or counts signals output from the normal pixel PX1 and the special pixel PX2, and the second circuit 12 adds or counts signals output from the special pixel PX2.

[0046] The special pixel PX2 is, for example, a phase difference detection pixel as described above. The phase difference detection pixel may be a divided pixel obtained by dividing the normal pixel PX1 into two halves, or a light-shielding pixel obtained by shielding half of the normal pixel PX1. Further, the special pixel PX2 is not limited to a phase difference detection pixel, and may be a pixel PX having a photoelectric conversion characteristic different from that of the normal pixel PX1. More specifically, the special pixel PX2 may be a pseudo pixel whose photoelectric conversion characteristic is pseudo-differentiated from that of the normal pixel PX1. The pseudo pixel may be, for example, a pseudo light-shielding pixel whose sensitivity is intentionally reduced by adjusting the cathode voltage of the SPAD. Further, the special pixel PX2 may be an EVS (Event Vision Sensor) pixel that detects a change in the incident light amount. Furthermore, the special pixel PX2 may be a normal pixel that is not compressed.

[0047] As described above, the first circuit 11 and the second circuit 12 add or count signals output from at least one common pixel (special pixel). Further, the pixel group 10 includes one or more normal pixels (first pixels) and one or more special pixels (second pixels), the first circuit 11 adds or counts signals output from the one or more normal pixels, and the second circuit 12 adds or counts signals output from the one or more special pixels.

[0048] FIG. 4 is a circuit diagram showing an example of a specific configuration of the first circuit 11 and the second circuit 12. FIG. 4 shows an example in which each pixel PX has a SPAD 13. More specifically, each pixel PX includes a SPAD 13, a recharge circuit 14, and an inverter 15. The recharge circuit 14 performs an operation of initializing the cathode voltage of the SPAD 13 after the SPAD 13 detects a photon. In this specification, the detection of a photon by the SPAD 13 may be referred to as firing. When the SPAD 13 fires, it temporarily reduces the signal level of the output signal. The inverter 15 connected to the output node of the SPAD 13 outputs, for example, a high-level pulse signal when the SPAD 13 fires.

[0049] As shown in Figure 4, the first circuit 11 includes, for example, a four-input OR circuit 16 and a first counter 17. The OR circuit 16 calculates the OR of pulse signals output from four pixels PX included in the pixel group 10. The OR circuit 16 outputs a number of pulse signals corresponding to the number of photons detected by the four pixels PX. The first counter 17 counts the number of pulse signals output from the OR circuit 16. For example, of the four pixels PX input to the OR circuit 16, three are normal pixels PX1 and the remaining one is a special pixel PX2. Below, an example is described in which the special pixel PX2 is a light-shielding pixel 26 used for phase difference detection.

[0050] The second circuit 12 includes, for example, a second counter 18. The second counter 18 counts the pulse signals output from one pixel PX (special pixel PX2) included in the pixel group 10.

[0051] Thus, normal pixels and special pixels output pulse signals when the photoelectric conversion element detects the incidence of a photon. The first circuit 11 has a first counter 17 that counts the pulse signals output from one or more normal pixels included in the pixel group, and the second circuit 12 has a second counter 18 that counts the pulse signals output from one or more special pixels included in the pixel group. More specifically, normal pixels and special pixels output pulse signals when the photoelectric conversion element detects the incidence of a photon, the first circuit 11 has a first counter 17 that counts the pulse signals output from one or more normal pixels and one or more special pixels included in the pixel group 10, and the second circuit 12 has a second counter 18 that counts the pulse signals output from one or more special pixels included in the pixel group.

[0052] If the count value of the first counter 17 is C[0] and the count value of the second counter 18 is C[1], then the following equation (1) holds.

[0053] C[0] ← C[0] + gain × C[1] …(1) The gain is a coefficient that takes into account that the light-shielding pixel 26 has a lower photoelectric conversion efficiency than the normal pixel PX1. By correcting the count value of the second counter 18 with the coefficient gain, the signal of the four pixels PX included in the pixel group 10 can be output using equation (1). The second circuit 12 counts the pulse signal output from the special pixel PX2 with the second counter 18 without compression. The count value of the second counter 18 can be used, for example, for phase difference detection.

[0054] The multiple pixels PX included in the first circuit 11 shown in Figure 3 may be placed at predetermined pixel positions within the pixel group 10, or they may be randomly selected from the pixel group 10, or the pixel positions selected from the pixel group 10 may be switchable.

[0055] Figure 5 shows an example in which multiple pixels PX included in the first circuit 11 are randomly selected from the pixel group 10. In Figure 5, the first circuit 11 has four first counters 17 that count the number of pulse signals output from, for example, four to five pixels PX randomly selected from the pixel group 10 which includes 4x4 pixels PX. The second circuit 12 has two second counters 18 that count the pulse signals output from two special pixels PX2 included in the pixel group 10 separately from each other. In Figure 5, for simplification, an example is shown in which the 16 pixels PX included in the pixel group 10 are the same color. If the pixel group 10 includes RGB pixels PX, the first circuit 11 can count the pulse signals output from pixels PX randomly selected for each color using the first counters 17.

[0056] Figure 6 is a detailed circuit diagram of the first circuit 11 and the second circuit 12 of Figure 5. Four vertical signal lines VSL are arranged for each pixel group 10 arranged in the second direction (vertical direction) Y. Each of these four vertical signal lines VSL is connected to a separate first counter 17. Each of the four first counters 17 counts the number of pulse signals output from four pixels PX randomly selected from the pixel group 10. The second circuit 12 has two second counters 18 connected to each of the two vertical signal lines VSL. Each second counter 18 counts the number of pulse signals output from the corresponding special pixel PX2.

[0057] In Figure 6, the recharge circuit 14 connected to the cathode of SPAD 13 in the pixel PX is replaced with a switch 30, but the recharge circuit 14 may be connected instead of the switch 30. The function of the switch 30 will be explained later in Figure 15.

[0058] Figures 7 to 9 show examples of how the type of pixel PX selected by the first circuit 11 from the pixel group 10 can be switched by a switch 27. Figure 7 shows an example of a pixel group 10 including a 4x4 pixel PX. The 16 pixels PX included in the pixel group 10 are denoted as X0 to X15. Figure 8 is a matrix equation representing the operation of individually selecting whether or not to count the output of each pixel PX included in the pixel group 10 by the first circuit 11. Each element of the 4x16 matrix on the far left can be 1 or 0. For example, 1 means that the first circuit 11 counts it, and 0 means that the first circuit 11 does not count it. Y0 to Y3 represent four first counters 17.

[0059] Figure 9 is a circuit diagram showing an example of the specific configuration of the first circuit 11 and the second circuit 12. A switch 27 is connected to the output node of each pixel PX. The pulse signal output from each pixel PX is transmitted via the switch 27 on the corresponding vertical signal line VSL. By individually switching each switch 27, it is possible to arbitrarily select the type of pixel PX to be counted by the four first counters 17. The first circuit 11 in Figure 9 can randomly or arbitrarily switch two or more pixel PXs included in the pixel group 10 by individually switching the switches 27.

[0060] (Unit of compression of special pixels PX2) At least some of the multiple pixels PX arranged in the pixel array are special pixels PX2. If the ratio of special pixels PX2 to the total number of pixels PX in the pixel array is large, compression processing such as downsampling of special pixels PX2 may be performed.

[0061] Figure 10 is a diagram showing the units of compression for special pixels PX2. Figure 10 shows an example where the special pixels PX2 are light-shielding pixels 26. The light-shielding pixels 26 in Figure 10 come in two types: horizontal and vertical. The vertical light-shielding pixels 26 are placed in the left half or right half of the pixel area. The horizontal light-shielding pixels 26 are placed in the upper half or lower half of the pixel area. The first compression range 28a in Figure 10 compresses the four pixels PX at the corners of a 3x3 pixel group 10. The first compression range 28a includes two light-shielding pixels 26, one horizontal and one vertical. The second compression range 28b compresses the four pixels PX at the corners of a 7x7 pixel group 10. The second compression range 28b includes four vertical (left half) light-shielding pixels 26. The third compression range 28c is the same size as the second compression range 28b, but it is shifted 2 pixels PX to the right, and includes four horizontally elongated (upper half) light-shielding pixels 26.

[0062] Figure 11 is a circuit diagram of the first circuit 11 and the second circuit 12, which compress and count both normal pixels PX1 and special pixels PX2. Figure 11 shows an example where the special pixel PX2 is a light-shielding pixel 26. Both the first circuit 11 and the second circuit 12 compress multiple pixels PX and count the number of pulse signals.

[0063] The first circuit 11 has a first counter 17 that counts the number of pulse signals output from two normal pixels PX1 and two light-shielding pixels 26 included in the pixel group 10 (first compression range 28a in Figure 10). The second circuit 12 has a second counter 18 that counts the number of pulse signals output from four vertically elongated light-shielding pixels 26 included in the second compression range 28b in Figure 10, or a second counter 18 that counts the number of pulse signals output from four horizontally elongated light-shielding pixels 26 included in the third compression range 28c in Figure 10.

[0064] (Correction of Compressed Count Results) In a pixel PX having a 2-pixel size on-chip lens (hereinafter referred to as 2×1 OCL) 29, blue pixels PX and green pixels PX may get mixed together when performing compressed counting. Figure 12 is a diagram showing the compression range in a pixel PX having a 2×1 OCL. In the pixel group 10 which is the compression range in Figure 12, one green pixel PX and three blue pixels PX are arranged at each of the four corners.

[0065] Figure 13 is a circuit diagram of the first circuit 11 and the second circuit 12 corresponding to Figure 12. The first circuit 11 has a four-input OR gate 16 and a first counter 17 that counts the number of pulse signals output from three blue pixels PX and one green pixel PX. The second circuit 12 has a second counter 18 that counts the number of pulse signals output from one green pixel PX.

[0066] The count value of the first counter 17 contains a mixture of the pulse counts of blue pixels PX and green pixels PX. Therefore, it is desirable to correct the count value C[0] of the first counter 17 using, for example, the following equation (2), where C[1] is the count value of the second counter 18.

[0067] C[0] ← (C[0] - C[1]) × 4 / 3 … (2) (Adjustment of S / N ratio between normal pixel PX1 and special pixel PX2) The required S / N ratio may differ between normal pixel PX1 and special pixel PX2. For example, if special pixel PX2 is a light-shielding pixel 26, the amount of incident light on the light-shielding pixel 26 is less than that on normal pixel PX1, so it is necessary to increase the S / N ratio at low light levels.

[0068] Figure 14 shows a pixel group 10 that performs compressed counting by adjusting the signal-to-noise ratio. The pixel group 10 may contain a mixture of normal pixels PX1 and special pixels PX2 (in this case, light-shielding pixels 26). By making the signal-to-noise ratio different between normal pixels PX1 and special pixels PX2, the accuracy of the count value obtained by compressed counting can be improved.

[0069] Figure 15 is a circuit diagram of a first circuit 11 and a second circuit 12 that have the function of adjusting the signal-to-noise ratio of a normal pixel PX1 and a special pixel PX2. The first circuit 11 in Figure 15 has a four-input OR gate 16 and a first counter 17 that counts the number of pulse signals output from three normal pixels PX1 and one special pixel PX2 (in this case, a light-shielding pixel 26). The second circuit 12 has a second counter 18 that counts the number of pulse signals output from the light-shielding pixel 26.

[0070] A normal pixel PX1 has a SPAD 13, an inverter 15, and a first switch 30a. A light-shielding pixel 26 has a SPAD 13, an inverter 15, and a second switch 30b. The first switch 30a and the second switch 30b are connected to the cathodes of the corresponding SPAD 13. The first switch 30a functions as a first control unit that switches and controls the exposure period or dead time of the normal pixel PX1. The second switch 30b functions as a second control unit that switches and controls the exposure period or dead time of the special pixel PX2.

[0071] When the first switch 30a or the second switch 30b is turned on, the cathode of the corresponding SPAD 13 returns to the initialization voltage. By changing the switching frequency of the first switch 30a or the second switch 30b, the dead time of the normal pixel PX1 or the special pixel PX2 (light-shielding pixel 26) can be adjusted, and the signal-to-noise ratio with respect to the amount of incident light can be adjusted.

[0072] When the special pixel PX2 is a light-shielding pixel 26, the amount of light incident on the light-shielding pixel 26 is less than that on the normal pixel PX1. Therefore, it is necessary to make the signal-to-noise ratio for low light levels higher than that of the normal pixel PX1, and it is desirable to increase the switching frequency of the second switch 30b.

[0073] As mentioned above, since the required signal-to-noise ratio differs between the normal pixel PX1 and the special pixel PX2, the number of bits in the first counter 17 and the second counter 18 may be different. For example, the second counter 18, which requires a high signal-to-noise ratio, may have more bits than the first counter 17.

[0074] Thus, in the first embodiment, since the first circuit 11 that adds or counts signals output from multiple normal pixels PX1 and the second circuit 12 that adds or counts signals output from special pixels PX2 are operated in parallel, the outputs of normal pixels PX1 and special pixels PX2 can be acquired in parallel, and the frame rate can be increased. In the first circuit 11, since multiple pixels PX share one first counter 17, compressed sensing can be performed with a small circuit size.

[0075] Special pixels PX2 are, for example, phase-difference detection pixels, which can achieve both phase-difference detection and compressed sensing.

[0076] (Second Embodiment) The imaging unit 5 of the light detection device 1 according to the second embodiment includes a plurality of paired phase difference detection pixels. The imaging unit 5 according to the second embodiment includes a first circuit 11 that counts the number of pulse signals output from one or more of the plurality of phase difference detection pixels, and a second circuit 12 that counts the number of pulse signals output from one or more other phase difference detection pixels. Each of the first circuit 11 and the second circuit 12 shares one counter (first counter 17 or second counter 18) with a plurality of pixels PX. Since there are various variations in the pixel configuration in which a plurality of phase difference detection pixels are arranged, typical pixel configurations will be described in order below.

[0077] (Acquiring phase difference information in Dual PD) The method of acquiring phase difference information from two adjacent pixels PX is sometimes called Dual PD. Figure 16 is a diagram showing the pixel array of Dual PD. In Dual PD compressed counting, for example, as shown in pixel group 10 in Figure 16, compressed counting is performed on four adjacent pixels PX of the same color.

[0078] Figure 17 is a circuit diagram of the first circuit 11 and the second circuit 12 in the Dual PD. The first circuit 11 in Figure 17 includes a first OR circuit 16a that takes the sum of four pulse signals output from the left pixel PX of four pairs of pixels PX at the corners of the pixel group 10 that are the target of the compressed count, a second OR circuit 16b that takes the sum of four pulse signals output from the right pixel PX of four pairs of pixels PX, a two-input third OR circuit 16c that takes the sum of the first and second OR circuits 16b, and a first counter 17.

[0079] The second circuit 12 includes a four-input OR gate 16b shared with the first circuit 11, and a second counter 18.

[0080] The first counter 17 outputs the count value C[0] of all pulse signals of the four pairs of pixels PX at the corners of the pixel group 10 that are subject to compressed counting. The second counter 18 outputs the count value C[1] of the pulse signal of the rightmost pixel PX of the four pairs of pixels PX.

[0081] The count value C[2] of the pulse signal of the leftmost pixel PX of the four pairs of pixels PX can be easily calculated by C[0] - C[1].

[0082] In Figures 16 and 17, compressed counting is performed using eight pixel PX located at the four corners of a 3x6 pixel PX pixel group 10. Generalizing this, the pixel group 10 comprises M pairs (where M is an integer greater than or equal to 1) of first and second pixel PX pixels. The first counter 17 counts the signals output from the N first pixel PX pixels in the first pixel group 10 and the signals output from the N second pixel PX pixels in the second pixel group 10 for each first pixel group 10 containing N (where N is an integer greater than or equal to 2 and less than or equal to M) of the M first pixel PX pixels in the M pairs, and each second pixel group 10 containing N of the M second pixel PX pixels. The second counter 18 counts the signals output from the N second pixel PX pixels in each second pixel group 10 containing N of the M second pixel PX pixels in the second pixel group 10.

[0083] Figure 18 is a circuit diagram relating to a modified example of Figure 17. The first circuit 11 in Figure 18 includes a first OR circuit 16a that takes the sum of four pulse signals output from the left pixel PX of the four pairs of pixels PX at the corners of the pixel group 10 that are subject to compression counting, and a first counter 17 that counts the number of pulse signals output from the first OR circuit 16a. The second circuit 12 includes a second OR circuit 16b that takes the sum of four pulse signals output from the right pixel PX of the four pairs of pixels PX at the corners of the pixel group 10 that are subject to compression counting, and a second counter 18 that counts the number of pulse signals output from the second OR circuit 16b.

[0084] The first counter 17 outputs the count value C[0] of the pulse signal of the left pixel PX of the four pairs of pixels PX at the corners of the pixel group 10 that are subject to compression counting. The second counter 18 outputs the count value C[1] of the pulse signal of the right pixel PX of the four pairs of pixels PX at the corners of the pixel group 10 that are subject to compression counting. By adding the count values ​​C[0] and C[1] together, the count values ​​C[2] of the pulse signals of all eight pixels PX can be easily calculated.

[0085] Thus, the first circuit 11 and the second circuit 12 in Figure 18 add or count signals output from different pixels PX.

[0086] In Figures 16 and 18, compressed counting is performed using eight pixels PX located at the four corners of a 3x6 pixel PX pixel group 10. Generalizing this, the first counter 17 counts the signals output from the N first pixels PX included in the first pixel group 10 for each first pixel group 10 containing N pixels (where N is an integer between 2 and M) out of the M first pixels PX included in the M pairs. The second counter 18 counts the signals output from the N second pixels PX included in the second pixel group 10 for each second pixel group 10 containing N pixels out of the M second pixels PX included in the M pairs.

[0087] (Acquiring phase difference information for 2x2 OCLs) Figure 19 shows a pixel configuration in which on-chip lenses (OCLs) are arranged in units of 2x2 pixels PX of the same color. Hereafter, the pixel configuration in Figure 19 will be referred to as 2x2 OCL. In the 2x2 OCL pixel configuration, phase difference information is detected in the left half of the 2x2 pixel PX and the right half of the 2x2 pixel PX. The 4 pixels PX x 4 located at each corner of the pixel group 10 in Figure 19 are the target of the compressed count.

[0088] Figure 20 is a circuit diagram of the first circuit 11 and the second circuit 12 in a 2x2 OCL. The first circuit 11 in Figure 20 has a 4-input first OR circuit 16a that outputs the OR signal of a total of 8 pulse signals output from 2 pixels PX×4 at the upper left and lower left corners of each corner of the pixel group 10, a 4-input second OR circuit 16b that outputs the OR signal of a total of 8 pulse signals output from 2 pixels PX×4 at the upper right and lower right corners of each corner of the pixel group 10, a 2-input third OR circuit 16c that outputs the OR signal of the outputs of these two OR circuits 16, and a first counter 17 that counts the number of pulse signals output from the third OR circuit 16c. The second circuit 12 has a 4-input second OR circuit 16b shared with the first circuit 11, and a second counter 18 that counts the number of pulse signals output from the second OR circuit 16b.

[0089] The count value C[0] of the first counter 17 is the count value of all pulse signals output from the four pixels PX at each corner of the pixel group 10. The count value C[1] of the second counter 18 is the count value of pulse signals output from the two pixels PX in the right half of the four pixels PX at each corner. The count value C[2] of pulse signals output from the two pixels PX in the left half of the four pixels PX at each corner is represented by C[0] - C[1].

[0090] Figure 21 is a circuit diagram relating to the first modified example of Figure 20. The first circuit 11 in Figure 21 has a 4-input first OR circuit 16a that outputs a logical OR signal of a total of 8 pulse signals output from 2 pixels PX×4 at the upper left and lower left corners of each corner of the pixel group 10, and a first counter 17 that counts the number of pulse signals output from the first OR circuit 16a. The second circuit 12 has a 4-input second OR circuit 16b that outputs a logical OR signal of a total of 8 pulse signals output from 2 pixels PX×4 at the upper right and lower right corners of each corner of the pixel group 10, and a second counter 18 that counts the number of pulse signals output from the second OR circuit 16b.

[0091] The count value C[0] of the first counter 17 is the count value of the pulse signal output from the left half of the four pixels PX at each corner of the pixel group 10. The count value C[1] of the second counter 18 is the count value of the pulse signal output from the right half of the four pixels PX at each corner of the pixel group 10. By taking the sum of C[0] and C[1], the total count value of the pulse signals output from the four pixels PX at each corner can be easily calculated.

[0092] When performing compressed counting using a 2x2 pixel PX unit corresponding to a 2x2 OCL, the area subject to compressed counting is not limited to that shown in Figure 19.

[0093] Figure 22 shows an example of compressed counting performed on a region of 4 pixels vertically x 8 pixels horizontally. This region is provided with first and second pixel groups 10a and 10b. The first pixel group 10a has a first counter 17 and a second counter 18. The first counter 17 counts the number of pulse signals output from 16 pixels PX located in the left half of the region. The second counter 18 counts the number of pulse signals output from 8 pixels PX located in the right half of the aforementioned 16 pixels PX.

[0094] The count value C[0] of the first counter 17 and the count value C[1] of the second counter 18 are expressed by the following equations (3) and (4). In equations (3) and (4), the pulse signals output from the 16 pixels PX are denoted as X00 to X03, X10 to X13, X20 to X23, and X30 to X33.

[0095] C[0] = X00 + X01 + X02 + X03 + X10 + X11 + X12 + X13 + X20 + X21 + X22 + X23 + X30 + X31 + X32 + X33 ... (3) C[1] = X01 + X03 + X11 + X13 + X21 + X23 + X31 + X33 ... (4) The second pixel group 10b has a third counter and a fourth counter. The third counter counts the number of pulse signals output from 16 pixels PX located in the left half of the region. The fourth counter counts the number of pulse signals output from 8 pixels PX located in the right half of the aforementioned 16 pixels PX.

[0096] The count value C[2] of the third counter and the count value C[3] of the fourth counter are expressed by the following equations (5) and (6). In equations (5) and (6), the pulse signals output from the 16 pixels PX are designated as X40 to X43, X50 to X53, X60 to X63, and X70 to X73.

[0097] C[2] = X40 + X41 + X42 + X43 + X50 + X51 + X52 + X53 + X60 + X61 + X62 + X63 + X70 + X71 + X72 + X73 ... (5) C[3] = X41 + X43 + X51 + X53 + X61 + X63 + X71 + X73 ... (6) Of the 32 pixels PX1 to PX32 in Figure 22, pixels PX1 to PX4 are M1, pixels PX5 to PX8 are M2, pixels PX9 to PX12 are M3, pixels PX13 to PX16 are M4, pixels PX17 to PX20 are M5, pixels PX21 to PX24 are M6, pixels PX25 to PX28 are M7, and pixels PX29 to PX32 are M8. In Figure 22, the first pixel group 10a includes M1 to M4, and the second pixel group 10b includes M5 to M8. However, the multiple pixels PX included in the first pixel group 10a and the second pixel group 10b do not necessarily have to be the same as in Figure 22. For example, the first pixel group 10a may include M1, M3, M5, and M7, and the second pixel group 10b may include M2, M4, M6, and M8. In this case, the first counter 17 counts the number of pulse signals for M1, M3, M5, and M7; the second counter 18 counts the number of pulse signals output from the right half of the pixels PX for M1, M3, M5, and M7; the third counter counts the number of pulse signals for M2, M4, M6, and M8; and the fourth counter counts the number of pulse signals output from the right half of the pixels PX for M2, M4, M6, and M8.

[0098] The range subject to compression counting is not limited to Figure 22. Figure 23 shows an example of compression counting according to one modification of Figure 22. In this modification, compression counting is performed on a zigzag-shaped group of pixels 10, as shown by the dashed line in Figure 23.

[0099] In Figure 23, the first pixel group 10a includes M1, M4, M5, and M6, and the second pixel group 10b includes M2, M3, M7, and M8. The count value C[0] of the first counter 17, the count value C[1] of the second counter 18, the count value C[2] of the third counter, and the count value C[3] of the fourth counter are expressed by the following equations (7) to (10).

[0100] C [0] = X00 + X01 + X02 + X03 + X30 + X31 + X32 + X33 + X40 + X41 + X42 + X43 + C [2] = X20 + X21 + X22 + X23 + X10 + X11 + X12 + X13 + X60 + X61 + X62 + X63 + Figure 23 is an example, and the types of pixels PX included in the first pixel group 10a and the second pixel group 10b can be arbitrarily set.

[0101] To arbitrarily switch the types of pixels PX included in the first pixel group 10a and the second pixel group 10b, a switch 27 can be provided between each pixel PX and the vertical signal line VSL.

[0102] Figure 24 is a circuit diagram of the first pixel group 10a and the second pixel group 10b, each comprising switches 27 and first to fourth counters. Four switches 27 are connected between the output node of each pixel PX and four vertical signal lines VSL. By individually switching the on / off state of the four switches 27 for each pixel PX, the types of pixels PX included in the first pixel group 10a and the second pixel group 10b can be arbitrarily switched.

[0103] If four switches 27 are provided for each of the pixels PX, the circuit size will increase. Therefore, switches 27 may be provided for only some of the pixels PX.

[0104] Figures 22 and 23 illustrate the compressed counting method for a 2x2 OCL pixel configuration, but compressed counting can also be performed for any pixel configuration such as Dual PD, Octa PD, light-shielding ZAF, and 2x1 OCL. Dual PD and light-shielding ZAF will be described below.

[0105] Figure 25A shows the first pixel group 10a and the second pixel group 10b in the pixel configuration of Dual PD. In Dual PD, phase difference detection is performed using two pixels PX on the left and right. In the Dual PD pixel configuration, compressed counting is performed using a total of 16 pixels PX, which is 2 pixels PX vertically and 8 pixels PX horizontally. Within the 2 pixels PX vertically and 8 pixels PX horizontally, there are M1 to M8. Each of M1 to M8 includes two pixels PX on the left and right. The first pixel group 10a includes M1 to M4. The second pixel group 10b includes M5 to M8. The eight pixels PX1 to PX8 included in the first pixel group 10a output pulse signals X00, X01, X10, X11, X20, X21, X30, and X31. The eight pixels PX9 to PX16 included in the second pixel group 10b output pulse signals X40, X41, X50, X51, X60, X61, X70, and X71.

[0106] The count value C[0] of the first counter 17, the count value C[1] of the second counter 18, the count value C[2] of the third counter, and the count value C[3] of the fourth counter are expressed by the following equations (10) to (13).

[0107] C[0] = X00 + X01 + X10 + X11 + X20 + X21 + X30 + X31 ... (10) C[1] = X01 + X11 + X21 + X31 ... (11) C[2] = X40 + X41 + X50 + X51 + X60 + X61 + X70 + X71 ... (12) C[3] = X41 + X51 + X61 + X71 ... (13) Figure 25B shows a first pixel group 10a and a second pixel group 10b that include different pixels PX than those in Figure 25A. The first pixel group 10a in Figure 25B includes M1, M4, M5, and M6. The second pixel group 10b includes M3, M2, M7, and M8. The count value C[0] in Figure 25B, the count value C[1] of the second counter 18, the count value C[2] of the third counter, and the count value C[3] of the fourth counter are expressed by the following equations (14) to (17).

[0108] C[0] = X00 + X01 + X30 + X31 + X40 + X41 + X50 + X51 ... (14) C[1] = X01 + X31 + X41 + X51 ... (15) C[2] = X20 + X21 + X10 + X11 + X60 + X61 + X70 + X71 ... (16) C[3] = X21 + X11 + X61 + X71 ... (17) Figure 25B shows the first pixel group 10a and the second pixel group 10b, which include different pixels PX than those in Figure 25A. The first pixel group 10a in Figure 25B includes M1, M4, M5, and M6. The second pixel group 10b includes M3, M2, M7, and M8.

[0109] Figure 26A shows the first pixel group 10a and the second pixel group 10b in the pixel configuration of a light-shielding ZAF. In the light-shielding ZAF, the first pixel group 10a and the second pixel group 10b are provided within a region of 2 pixels vertically PX × 4 pixels horizontally. The first pixel group 10a has pixels PX1 to PX4, and the right half of pixel PX2 is light-shielded. The second pixel group 10b has pixels PX5 to PX8, and the left half of pixel PX7 is light-shielded. Pixels PX1 to PX8 output pulse signals X0 to X8.

[0110] The count values ​​C[0] of the first counter 17, C[1] of the second counter 18, C[2] of the third counter, and C[3] of the fourth counter in Figure 26A are expressed by the following equations (14) to (17).

[0111] C[0] = X0 + X1 + X2 + X3 ... (14) C[1] = X1 ... (15) C[2] = X4 + X5 + X6 + X7 ... (16) C[3] = X6 ... (17) Figure 26B shows a first pixel group 10a and a second pixel group 10b with a different pixel configuration than Figure 26A. The first pixel group 10a in Figure 26B includes pixels PX1, PX4, PX5, and PX6. The second pixel group 10b includes pixels PX3, PX2, PX7, and PX8.

[0112] The count values ​​C[0] of the first counter 17, C[1] of the second counter 18, C[2] of the third counter, and C[3] of the fourth counter in Figure 26B are expressed by the following equations (18) to (21).

[0113] C[0] = X0 + X3 + X4 + X5 ... (18) C[1] = X1 ... (19) C[2] = X2 + X1 + X6 + X7 ... (20) C[3] = X6 ... (21) The specific circuit for compressed counting in the 2x2 OCL pixel configuration shown in Figure 19 is not limited to Figure 20 or Figure 21, and various modifications are possible.

[0114] Figure 27 is a circuit diagram relating to a second modified example of a 2x2 OCL pixel configuration. In the second modified example, the pixel group 10 has four circuits corresponding to the four corners. The first circuit 11 has a four-input OR circuit 16 that calculates the OR of pulse signals output from the upper left pixel PX of the four pixels PX at each corner, and a first counter 17. The second circuit 12 has a four-input OR circuit 16 that calculates the OR of pulse signals output from the lower left pixel PX of the four pixels PX at each corner, and a second counter 18. The third circuit 31 has a four-input OR circuit 16 that calculates the OR of pulse signals output from the upper right pixel PX of the four pixels PX at each corner, and a third counter. The fourth circuit 32 has a four-input OR circuit 16 that calculates the OR of pulse signals output from the lower right pixel PX of the four pixels PX at each corner, and a fourth counter.

[0115] The phase difference information on the left is represented by the sum of the count value C[0] of the first counter 17 and the count value C[1] of the second counter 18. The phase difference information on the right is represented by the sum of the count value C[2] of the third counter and the count value C[3] of the fourth counter. Furthermore, the phase difference information on the top is represented by the sum of the count value C[0] of the first counter 17 and the count value C[2] of the third counter. The phase difference information on the bottom is represented by the sum of the count value C[1] of the second counter 18 and the count value C[3] of the fourth counter.

[0116] Thus, in the second modified example, all pixels in the PX can detect vertical and horizontal phase difference information, but the scale of the counter becomes larger than in Figures 20 and 21.

[0117] Generalizing the 2x2OCL pixel configuration in Figure 27, the pixel group 10 has M pairs (where M is an integer greater than or equal to 1) of first pixels PX and second pixels PX. The first circuit 11 has N first counters 17 for each first pixel group 10a containing N (where N is an integer greater than or equal to 1) of the M first pixels PX included in the M pairs, which count the signals output from each of the N first pixels PX. The second circuit 12 has M second counters 18 for each first pixel group 10a containing N second pixels PX included in the M pairs, which count the signals output from each of the N second pixels PX.

[0118] Figure 28 shows a third modified example of a 2x2 OCL pixel configuration. In this third modification, some of the pixels PX are pseudo-light-shielding pixels 26d. A pseudo-light-shielding pixel 26d is a pixel PX that is electrically light-shielded rather than physically light-shielded. For example, a switch 27 is provided between the output node of the pixel PX and the vertical signal line VSL, and the switch 27 is turned off when light-shielding is desired. As a result, the pulse signal output from the pixel PX is not supplied to the vertical signal line VSL. Alternatively, when light-shielding is desired, the inverter 15 connected to the cathode of the SPAD 13 in the pixel PX may be disabled so that the pulse signal is not output from the inverter 15.

[0119] In the third modification, compressed counting is performed using the pixel group 10 shown in Figure 28 as the unit. The number of pulse signals output from the four pixels PX located at each of the four corners of the pixel group 10 is counted.

[0120] Figure 29 is a block diagram of a pixel group 10 according to a third modified example. The pixel group 10, which has a 2x2 OCL pixel configuration, has a first circuit 11 and a second circuit 12. The first circuit 11 has a four-input OR circuit 16 that calculates the OR of pulse signals output from three normal pixels PX1 and a pulse signal output from one special pixel PX2, and a first counter 17 that counts the number of pulse signals output from the OR circuit 16. The second circuit 12 has a second counter 18 that counts the number of pulse signals output from one special pixel PX2.

[0121] The special pixel PX2 is a pseudo-light-shielding pixel 26d shown in Figure 28. Since the pseudo-light-shielding pixel 26d generates a light-shielding state electrically, there is no risk of a decrease in the signal-to-noise ratio compared to the normal pixel PX1.

[0122] In the third modification, the proportion of pseudo-light-shielding pixels 26d is reduced, which reduces the power required to acquire phase difference information, but it is not possible to acquire phase difference information for all pixels PX.

[0123] Figure 30 shows an example of the circuit configuration of a special pixel PX2. The special pixel PX2 has a pseudo-light-shielding pixel 26d. The four pixel PXs shown in Figure 30 can be used as normal pixels PX1 or pseudo-light-shielding pixels 26d. The cathodes of the two upper SPADs 13 are connected to a first inverter 15 and a first recharge circuit 14. The cathodes of the two lower SPADs 13 are connected to a second inverter 15 and a second recharge circuit 14. The output nodes of the first inverter 15 and the second inverter 15 are connected to a two-input OR circuit 16. In parallel with the OR circuit 16 outputting the OR signal of the pulse signals output from the four pixel PXs, the second inverter 15 outputs the OR signal of the pulse signals output from the two lower pixel PXs. The pulse signals output from the second inverter 15 can be used as phase difference information.

[0124] Figure 31 shows a first example of the circuit configuration of a normal pixel PX1. The normal pixel PX1 according to the first example has an inverter 15 and a recharge circuit 14 connected to the cathodes of four SPADs 13 for compressed counting. The inverter 15 can output a logical OR signal of the four pulse signals output from the four SPADs 13. In the circuit configuration of the normal pixel PX1 in Figure 31, the number of inverters 15 can be reduced and the logical OR circuit 16 can be omitted, thus reducing the size of the circuit.

[0125] Figure 32 shows a second example of the circuit configuration of a normal pixel PX1. The normal pixel PX1 according to the second example has a first inverter 15 and a first recharge circuit 14 connected to the cathodes of the two upper SPADs 13, a second inverter 15 and a second recharge circuit 14 connected to the cathodes of the two lower SPADs 13, and a two-input OR circuit 16 connected to the output nodes of the first inverter 15 and the second inverter 15. Since the circuit configuration of the second example is similar to the configuration of the special pixel PX2 in Figure 30, the electrical characteristics (e.g., signal propagation time, etc.) of the normal pixel PX1 and the special pixel PX2 can be matched.

[0126] (Acquiring phase difference information of Octa PD) Figure 33A shows a first example of compressed counting in the pixel configuration of Octa PD. In the pixel configuration of Octa PD, pixel groups 10, each containing 4x2 pixels PX of the same color, are arranged in a Bayer array. Two adjacent pixels PX are phase difference detection pixels. Each pixel group 10 has four pairs of phase difference detection pixels. In the first example, compressed counting is performed for each green pixel group 10.

[0127] Figure 33B shows a second example of compressed counting in the pixel configuration of OCTA PD. In the second example, compressed counting is performed using four adjacent green pixel groups 10 as units. The second example performs compressed counting over a wider range than the first example. By combining images acquired with these varying compression patterns, it is expected that the image resolution will be improved.

[0128] Figure 34 is a circuit diagram of the pixel group 10 performing the first example of compression counting in Figure 33A. The pixel group 10 in Figure 34 has a first circuit 11 and a second circuit 12. The first circuit 11 has a 4-input first OR circuit 16a that takes the logical OR of pulse signals output from the four left-hand pixels PX included in the pixel group 10, a 4-input second OR circuit 16b that takes the logical OR of pulse signals output from the four right-hand pixels PX included in the pixel group 10, a 2-input third OR circuit 16c that takes the logical OR of pulse signals output from the first OR circuit 16a and the second OR circuit 16b, and a first counter 17 that counts the number of pulse signals output from the third OR circuit 16c. The second circuit 12 has a second OR circuit 16b shared with the first circuit 11, and a second counter 18 that counts the number of pulse signals output from the second OR circuit 16b.

[0129] The count value C[0] of the first counter 17 is the total number of pulse signals output from the pixel group 10. The count value C[1] of the second counter 18 is the number of pulse signals output from the rightmost pixel PX included in the pixel group 10. The count value C[2] of the pulse signals output from the leftmost pixel PX included in the pixel group 10 is C[0] - C[1].

[0130] Generalizing the first example, the pixel group 10 has M pairs (where M is an integer greater than or equal to 1) of first pixels PX and second pixels PX. The first counter 17 counts the M signals output from each of the M first pixels PX included in the M pairs, and the M signals output from each of the M second pixels PX. The second counter 18 counts the M signals output from each of the M second pixels PX included in the M pairs.

[0131] Generalizing the second example, the first counter 17 counts the signal output from one first pixel PX contained in M ​​first pixels PX for each of the N sets. The second counter 18 counts the signal output from one second pixel PX contained in M ​​second pixels PX for each of the N sets.

[0132] Figure 35 is a circuit diagram of a pixel group 10 relating to a modified example of Figure 34. The pixel group 10 in Figure 35 has a first circuit 11 and a second circuit 12. The first circuit 11 has a 4-input first OR circuit 16a that takes the logical OR of pulse signals output from the 4 left pixels PX included in the pixel group 10, a 4-input second OR circuit 16b that takes the logical OR of pulse signals output from the 4 right pixels PX included in the pixel group 10, a first counter 17 that counts the number of pulse signals output from the first OR circuit 16a, and a second counter 18 that counts the number of pulse signals output from the second OR circuit 16b.

[0133] The count value C[0] of the first counter 17 is the number of pulse signals output from the leftmost pixel PX included in the pixel group 10. The count value C[1] of the second counter 18 is the number of pulse signals output from the leftmost pixel PX included in the pixel group 10. The total number of pulse signals output from all pixels PX included in the pixel group 10, C[2], is C[0] + C[1].

[0134] Thus, in the second embodiment, the pulse signals output from a pair of phase difference detection pixels are divided and counted by the first circuit 11 and the second circuit 12. In each of the first circuit 11 and the second circuit 12, one counter is shared by two or more phase difference detection circuits, so compressed counting can be performed with a small circuit size. Furthermore, by providing a switch 27 between each pixel PX and the vertical signal line VSL, the type of pixel PX whose pulse signal is counted by the first circuit 11 or the second circuit 12 can be switched.

[0135] <Examples of application to mobile devices> The technology disclosed herein (this technology) can be applied to various products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0136] Figure 36 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0137] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 36, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0138] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0139] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0140] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0141] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0142] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0143] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0144] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0145] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0146] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 36, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0147] Figure 37 shows an example of the installation position of the imaging unit 12031.

[0148] In Figure 37, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0149] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0150] Figure 37 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0151] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0152] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0153] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0154] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0155] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein may be applied to the imaging unit 12031, etc., among the configurations described above.

[0156] Furthermore, this technology can take the following configurations: (1) A light detection device comprising: a plurality of pixels, each having a photoelectric conversion element; a first circuit provided for each pixel group including two or more pixels from the plurality of pixels, which adds or counts signals output from at least one pixel included in the pixel group; and a second circuit that performs in parallel with the first circuit a process of adding or counting signals output from at least one pixel included in the pixel group, which differs from the first circuit in at least some of its pixels. (2) The light detection device according to (1), wherein the first circuit outputs a compressed signal of signals output from two or more pixels included in the pixel group, and the second circuit outputs an uncompressed signal of signals output from one pixel included in the pixel group. (3) The light detection device according to (2), wherein the first circuit has a switching circuit that randomly selects the two or more pixels included in the pixel group, or switches the types of the two or more pixels. (4) The light detection device according to any one of (1) to (3), wherein the first circuit and the second circuit add or count signals output from at least one common pixel. (5) The light detection device according to any one of (1) to (3), wherein the first circuit and the second circuit add or count signals output from pixels that are different from each other. (6) The light detection device according to any one of (1) to (5), wherein the pixel group has one or more first pixels and one or more second pixels, the first circuit adds or counts signals output from the one or more first pixels, and the second circuit adds or counts signals output from the one or more second pixels. (7) The photodetector according to (6), wherein the first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon, the first circuit has a first counter for counting pulse signals output from the one or more first pixels included in the pixel group, and the second circuit has a second counter for counting pulse signals output from the one or more second pixels included in the pixel group.(8) The photodetector according to (6), wherein the first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon, the first circuit has a first counter for counting the pulse signals output from the one or more first pixels and the one or more second pixels included in the pixel group, and the second circuit has a second counter for counting the pulse signals output from the one or more second pixels included in the pixel group. (9) The photodetector according to any one of (6) to (8), wherein the second pixel has different photoelectric conversion characteristics than the first pixel. (10) The photodetector according to (9), wherein the second pixel has a smaller photoelectric conversion area than the first pixel. (11) The photodetector according to (9), wherein the second pixel has lower sensitivity than the first pixel. (12) The photodetector according to (9), wherein the first pixel is a pixel that detects the amount of incident light or the presence or absence of incident light, and the second pixel is a pixel that detects a change in the amount of incident light. (13) The photodetector according to (7) or (8), wherein the pixel group has at least one pair of first pixels and second pixels, and the pair of first pixels and second pixels are phase difference detection pixels. (14) The photodetector according to any one of (6) to (8), comprising: a first control unit for controlling the exposure period or dead time of one or more first pixels; and a second control unit for controlling the exposure period or dead time of one or more second pixels. (15) The light detection device according to (13), comprising M pairs of first pixels and second pixels, each of which is a phase difference detection pixel, wherein the first counter counts the signals output from the N first pixels included in the first pixel group and the signals output from the N second pixels included in the second pixel group for each first pixel group including N pixels (N is an integer of 2 or more and less than or equal to M) from the M first pixels included in the M pairs and for each second pixel group including the N pixels from the M second pixels, and the second counter counts the signals output from the N second pixels included in the second pixel group for each second pixel group including the N pixels from the M second pixels.(16) The photodetector according to (13), comprising a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of which are phase difference detection pixels, wherein the first counter counts the signals output from the N first pixels included in the first pixel group for each first pixel group including N (where N is an integer of 2 or more and less than or equal to M) of the M first pixels included in the pair of M, and the second counter counts the signals output from the N second pixels included in the second pixel group for each second pixel group including the N of the M second pixels included in the pair of M. (17) The light detection device according to (13), comprising: (17) a first pixel and a second pixel of M pairs (where M is an integer of 1 or more), each of which is a phase difference detection pixel; the N first counters for each first pixel group containing N (where N is an integer of 1 or more) of the M first pixels included in the M pair; and the M second counters for each first pixel group containing N of the M second pixels included in the M pair, each of which is a light detection device according to (13). (18) The light detection device according to (13), comprising M pairs of first pixels and second pixels, each of which is a phase difference detection pixel, wherein the first counter counts the M signals output from each of the M first pixels included in the M pair and the M signals output from each of the M second pixels, and the second counter counts the M signals output from each of the M second pixels included in the M pair. (19) The light detection device according to (13), comprising N pairs (N is an integer of 2 or more) of first pixels and second pixels, each of which is a phase difference detection pixel, wherein the first counter counts the signal output from one of the M first pixels included in each of the N pairs, and the second counter counts the signal output from one of the M second pixels included in each of the N pairs.(20) The photodetector according to any one of (1) to (19), wherein the photoelectric conversion element is a SPAD (Single Photon Avalanche Diode).

[0157] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents.

[0158] 1. Light detection device, 2. Light detection system, 3. Logic processing unit, 4. Image output unit, 5. Imaging unit, 6. Phase difference detection unit, 7. Holding unit, 8. Signal transfer unit, 10. Pixel group, 10a. First pixel group, 10b. Second pixel group, 11. First circuit, 12. Second circuit, 14. Recharge circuit, 15. Inverter, 16. OR circuit, 16a. First OR circuit, 16b. Second OR circuit, 16c. Third OR circuit, 17. First counter, 18. Second counter, 19. Pulse generation circuit, 20. Resistor element, 21. Inverter, 22. D-type flip-flop (hereinafter, D-F / F), 23. Feedback circuit, 24. Active element (PMOS transistor), 25. AND circuit, 26. Light-shielding pixel, 26d. Pseudo-light-shielding pixel, 27. Switch, 28a. First compression range, 28b. Second compression range, 28c. Third compression range, 29 2 x 1 OCL, 30 Switch, 30a First switch, 30b Second switch, 31 Third circuit, 32 Fourth circuit

Claims

1. A light detection device comprising: a plurality of pixels, each having a photoelectric conversion element; a first circuit provided for each pixel group including two or more pixels from the plurality of pixels, which adds or counts signals output from at least one pixel included in the pixel group; and a second circuit that performs in parallel with the first circuit a process of adding or counting signals output from at least one pixel included in the pixel group, which differs from the first circuit in at least some of its pixels.

2. The photodetector according to claim 1, wherein the first circuit outputs a compressed signal obtained by compressing the signals output from two or more pixels included in the pixel group, and the second circuit outputs an uncompressed signal obtained by compressing the signal output from one pixel included in the pixel group.

3. The photodetector according to claim 2, wherein the first circuit has a switching circuit that randomly selects two or more pixels included in the pixel group, or switches the types of the two or more pixels.

4. The photodetector according to claim 1, wherein the first circuit and the second circuit add or count signals output from at least one common pixel.

5. The photodetector according to claim 1, wherein the first circuit and the second circuit add or count signals output from different pixels.

6. The photodetector according to claim 1, wherein the pixel group comprises one or more first pixels and one or more second pixels, the first circuit adds or counts signals output from the one or more first pixels, and the second circuit adds or counts signals output from the one or more second pixels.

7. The photodetector according to claim 6, wherein the first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon, the first circuit has a first counter for counting pulse signals output from the one or more first pixels included in the pixel group, and the second circuit has a second counter for counting pulse signals output from the one or more second pixels included in the pixel group.

8. The photodetector according to claim 6, wherein the first pixel and the second pixel output a pulse signal when the photoelectric conversion element detects the incidence of a photon, the first circuit has a first counter for counting the pulse signals output from the one or more first pixels and the one or more second pixels included in the pixel group, and the second circuit has a second counter for counting the pulse signals output from the one or more second pixels included in the pixel group.

9. The photodetector according to claim 6, wherein the second pixel has different photoelectric conversion characteristics from the first pixel.

10. The photodetector according to claim 9, wherein the second pixel has a photoelectric conversion region smaller than that of the first pixel.

11. The photodetector according to claim 9, wherein the second pixel has lower sensitivity than the first pixel.

12. The light detection device according to claim 9, wherein the first pixel is a pixel that detects the amount of incident light or the presence or absence of incident light, and the second pixel is a pixel that detects a change in the amount of incident light.

13. The photodetector according to claim 7, wherein the pixel group has at least one pair of first pixels and second pixels, and the pair of first pixels and second pixels are phase difference detection pixels.

14. The photodetector according to claim 6, comprising: a first control unit for controlling the exposure period or dead time of one or more first pixels; and a second control unit for controlling the exposure period or dead time of one or more second pixels.

15. The photodetector according to claim 13, comprising M pairs of first pixels and second pixels, each of which is a phase difference detection pixel, wherein the first counter counts the signals output from the N first pixels included in the first pixel group and the signals output from the N second pixels included in the second pixel group for each first pixel group including N pixels (N is an integer of 2 or more and less than or equal to M) from the M first pixels included in the M pairs and for each second pixel group including the N pixels from the M second pixels, and the second counter counts the signals output from the N second pixels included in the second pixel group for each second pixel group including the N pixels from the M second pixels.

16. The photodetector according to claim 13, comprising a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of which are phase difference detection pixels, wherein the first counter counts the signals output from the N first pixels included in the first pixel group for each first pixel group including N (where N is an integer of 2 or more and less than or equal to M) of the M first pixels included in the pair of M, and the second counter counts the signals output from the N second pixels included in the second pixel group for each second pixel group including the N of the M second pixels included in the pair of M.

17. The photodetector according to claim 13, comprising: a first pixel and a second pixel of a first pixel and a second pixel of a first pixel group M (where M is an integer of 1 or more), each of which is a phase difference detection pixel; N first counters for each first pixel group containing N (where N is an integer of 1 or more) of the M first pixels included in the M pair; and M second counters for each first pixel group containing N of the M second pixels included in the M pair, each of which is a second counter for the signals output from each of the N second pixels.

18. The photodetector according to claim 13, comprising a first pixel and a second pixel in a pair of M (where M is an integer of 1 or more) each of which is a phase difference detection pixel, wherein the first counter counts the M signals output from each of the M first pixels included in the pair of M and the M signals output from each of the M second pixels, and the second counter counts the M signals output from each of the M second pixels included in the pair of M.

19. The photodetector according to claim 13, comprising N sets (N is an integer of 2 or more) of M pairs (M is an integer of 1 or more) of the first pixels and the second pixels, each of which is a phase difference detection pixel, wherein the first counter counts the signal output from one of the M first pixels for each of the N sets, and the second counter counts the signal output from one of the M second pixels for each of the N sets.

20. The photodetector according to claim 1, wherein the photoelectric conversion element is a SPAD (Single Photon Avalanche Diode).

Images