Solid-state imaging device and imaging system

Abstract

To provide a solid-state imaging device capable of removing the influence of background light while suppressing generation of artifacts in a captured image.SOLUTION: The present invention includes a solid-state imaging device having a plurality of pixels 100, each including: a photoelectric conversion unit 110 for detecting photons and outputting a signal according to the number of the photons; and an arithmetic processing unit 120 for adding an output signal of the photoelectric conversion unit 110 corresponding to pulse light and ambient light received by the pixel 100 during a first period within one frame period, and subtracting an output signal of the photoelectric conversion unit 110 corresponding to ambient light received by the pixel 100 during a second period within the one frame period.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a solid-state imaging device and an imaging system. [Background technology]

[0002] Near-infrared image sensors are used in driver monitoring systems that perform facial recognition of drivers inside automobiles.

[0003] As a related technique, an image processing device such as that shown in Patent Document 1 below has been proposed from the viewpoint of suppressing the influence of background light on a captured image. The image processing device disclosed in Patent Document 1 generates a first IR image captured with a pulse wave turned on and a second IR image captured with the pulse wave turned off, and corrects the first IR image based on the second IR image. With this configuration, the influence of background light can be removed by subtracting the second IR image from the first IR image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-051042 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above image processing device, there is a problem that, because there is a large time difference between the two IR images, if the subject (for example, the driver) moves during shooting, artifacts are likely to occur at the edge of the subject.

[0006] The present invention has been made in view of the above-mentioned problems, and therefore an object of the present invention is to provide a solid-state imaging device and an imaging system that can eliminate the influence of background light and suppress the occurrence of artifacts in captured images. [Means for solving the problem]

[0007] The above object of the present invention can be achieved by the following means.

[0008] (1) A solid-state imaging device having a plurality of pixels, each of which has a photoelectric conversion unit that detects photons and outputs a signal corresponding to the number of photons, and an arithmetic processing unit that adds output signals of the photoelectric conversion unit for pulsed light and external light received by the pixel during a first period within one frame period, and subtracts output signals of the photoelectric conversion unit for external light received by the pixel during a second period within the one frame period.

[0009] (2) The solid-state imaging device described in (1) above, wherein the photoelectric conversion unit includes a single-photon avalanche diode and outputs a number of pulse signals corresponding to the number of photons, and the arithmetic processing unit includes an up / down counter that up-counts the pulse signals during the first period and down-counts the pulse signals during the second period.

[0010] (3) The solid-state imaging device described in (1) or (2) above, wherein the first period is a period during which the pulsed light is on within one cycle of the blinking of the pulsed light, the second period is a period during which the pulsed light is off within one cycle of the blinking of the pulsed light, and the calculation processing unit performs a process of adding the output signal and a process of subtracting the output signal within one cycle of the blinking of the pulsed light.

[0011] (4) A solid-state imaging device as described in (3) above, wherein during a period in which the pixel continuously receives the pulsed light, the arithmetic processing unit repeatedly performs a process of adding the output signal and a process of subtracting the output signal.

[0012] (5) The solid-state imaging device according to (1) or (2) above, including a plurality of the arithmetic processing units corresponding to a plurality of time-shared output signals.

[0013] (6) A solid-state imaging device as described in (5) above, wherein the first period is a period during which the pixel continuously receives the pulsed light, and the second period is a period during which the pixel does not receive the pulsed light, and each of the arithmetic processing units performs processing to add the output signal during the first period and processing to subtract the output signal during the second period.

[0014] (7) the first period and the second period are repeated periodically within the one frame period; The solid-state imaging device according to (6), wherein each of the arithmetic processing units repeatedly performs the process of adding the output signal and the process of subtracting the output signal within one frame period.

[0015] (8) The solid-state imaging device according to (1) or (2), wherein the arithmetic processing unit performs a process of subtracting the output signal before a process of adding the output signal.

[0016] (9) An imaging system having a light source device that irradiates a subject with pulsed light and a solid-state imaging device that receives the pulsed light reflected by the subject, wherein the solid-state imaging device has a plurality of pixels, each of which has a photoelectric conversion unit that detects photons and outputs a signal corresponding to the number of photons, and an arithmetic processing unit that adds up output signals from the photoelectric conversion unit for the pulsed light and external light received by the pixel during a first period within one frame period, and subtracts the output signal from the photoelectric conversion unit for the external light received by the pixel during a second period within the one frame period.

[0017] (10) The imaging system described in (9) above, wherein the photoelectric conversion unit includes a single-photon avalanche diode and outputs a number of pulse signals corresponding to the number of photons, and the calculation processing unit includes an up / down counter that up-counts the pulse signals during the first period and down-counts the pulse signals during the second period.

[0018] (11) The imaging system described in (9) or (10) above, wherein the first period is a period during which the pulsed light is on within one cycle of the blinking of the pulsed light, the second period is a period during which the pulsed light is off within one cycle of the blinking of the pulsed light, and the calculation processing unit performs a process of adding the output signal and a process of subtracting the output signal within one cycle of the blinking of the pulsed light.

[0019] (12) The imaging system described in (11) above, wherein during a period in which the light source device continuously irradiates the subject with the pulsed light, the calculation processing unit repeatedly performs a process of adding the output signal and a process of subtracting the output signal.

[0020] (13) The imaging system according to (9) or (10), wherein the solid-state imaging device includes a plurality of the arithmetic processing units corresponding to a plurality of time-divided output signals.

[0021] (14) The imaging system described in (13) above, wherein the first period is a period during which the light source device continuously irradiates the subject with pulsed light, and the second period is a period during which the light source device stops irradiating the subject with pulsed light, and each of the arithmetic processing units performs processing to add the output signal during the first period and processing to subtract the output signal during the second period.

[0022] (15) The imaging system described in (14) above, wherein the first period and the second period are repeated periodically within one frame period, and each of the calculation processing units repeatedly performs the process of adding the output signal and the process of subtracting the output signal within one frame period.

[0023] (16) The imaging system according to (9) or (10) above, wherein the arithmetic processing unit performs the subtraction process on the output signal before the addition process on the output signal. [Effects of the Invention]

[0024] According to the present invention, processing to remove the effects of external light (background light) is performed at a time interval shorter than the time it takes for the captured image to be generated, thereby removing the effects of background light and suppressing the occurrence of artifacts in the captured image. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of an imaging system. [Figure 2] 1 is a circuit diagram showing a schematic configuration of a pixel according to a first embodiment. [Figure 3] 10 is a timing chart showing an example of the operation of the imaging system. [Figure 4] FIG. 10 is a circuit diagram showing a schematic configuration of a pixel according to a second embodiment. [Figure 5] 10 is a timing chart showing an example of the operation of the imaging system. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following drawings, the same reference numerals refer to the same components, and the size of each component in the drawings may be exaggerated for clarity and convenience. Meanwhile, the embodiments described below are merely examples, and various modifications are possible from such embodiments.

[0027] Hereinafter, the terms "upper" and "above" may include not only what is directly above in contact with something, but also what is above without contact. Similarly, the terms "lower" and "below" may include not only what is directly below in contact with something, but also what is below without contact.

[0028] The singular expression includes the plural expression unless the context clearly dictates otherwise. Furthermore, when a part "includes," "comprises," or "has" a certain element, it does not mean that it excludes other elements, but that it may further include other elements, unless otherwise specified to the contrary.

[0029] Unless explicitly stated or stated to the contrary, steps constituting a method may be performed in any suitable order, and are not necessarily limited to the order of the steps described. The use of any examples or exemplary terms is merely for the purpose of illustrating the technical idea, and the scope of the invention is not limited by the claims, and should not be construed as being limited by said examples or exemplary terms.

[0030] In the following description, when ordinal numbers such as "first" and "second" are used, unless otherwise specified, they are used for convenience and do not stipulate any particular order.

[0031] (First embodiment) A first embodiment of the present invention will be described below with reference to Figures 1 to 3. Figure 1 is a diagram showing a schematic configuration of an imaging system 1 according to the first embodiment. The imaging system 1 is used in an environment where external light such as sunlight is present.

[0032] 1, the imaging system 1 includes a light source device 10, a solid-state imaging device 20, and a signal processing device 30. The light source device 10, the solid-state imaging device 20, and the signal processing device 30 are electrically connected to each other.

[0033] The light source device 10 irradiates the subject 50 with pulsed light L1. The light source device 10 is, for example, a semiconductor laser, and irradiates the subject 50 with pulsed light L1 in the near-infrared wavelength band. The light source device 10 irradiates the subject 50 with pulsed light L1 that flashes at a predetermined frequency (for example, 1 kHz to 10 kHz). The light source device 10 may include an optical element (not shown).

[0034] The solid-state imaging device 20 receives pulsed light L1 reflected by the subject 50. The solid-state imaging device 20 also receives external light L2, such as sunlight. The solid-state imaging device 20 includes a plurality of pixels 100 arranged in an array. Each pixel 100 includes a single photon avalanche diode (SPAD) capable of detecting a single photon. Each pixel 100 is controlled by a drive circuit 21, and the value of each pixel 100 is output to the outside as a pixel signal by a signal output circuit 22. The solid-state imaging device 20 may include optical elements (not shown).

[0035] The signal processing device 30 performs various processes on the pixel signals output from the solid-state imaging device 20. The signal processing device 30 generates a captured image of the subject 50 from the pixel signals output from the solid-state imaging device 20.

[0036] The imaging system 1 may include components other than those described above, or may not include some of the components described above. For example, the signal processing device 30 may be provided inside the solid-state imaging device 20 as a signal processing circuit.

[0037] Next, the pixel 100 of the solid-state imaging device 20 will be described with reference to Fig. 2. Fig. 2 is a circuit diagram showing a schematic configuration of the pixel 100. As shown in Fig. 2, the pixel 100 has a photoelectric conversion unit 110 and an arithmetic processing unit 120.

[0038] <Photoelectric conversion unit 110> The photoelectric conversion unit 110 includes a SPAD 111, a transistor 112, and an inverter 113. The photoelectric conversion unit 110 detects photons using the SPAD 111, and outputs a pulse signal according to the number of detected photons.

[0039] The cathode of the SPAD 111 is connected to the drain of the transistor 112, and the anode of the SPAD 111 is connected to a negative voltage source (not shown). The source of the transistor 112 is connected to a positive voltage source (not shown). The junction between the SPAD 111 and the transistor 112 is connected to the input terminal of the inverter 113, and the output terminal of the inverter 113 is connected to the arithmetic processing unit 120. The inverter 113 has a function of converting the analog signal supplied from the SPAD 111 into a pulse signal. Note that the configuration of the photoelectric conversion unit 110 itself is well known, so a detailed description of the configuration of the photoelectric conversion unit 110 will be omitted.

[0040] The photoelectric conversion unit 110 of this embodiment detects photons from the pulsed light L1 and external light L2 received by the pixel 100 using the SPAD 111, and outputs pulse signals whose number corresponds to the number of detected photons. Specifically, during a first period in which the pulsed light is on during one cycle of blinking of the pulsed light, the photoelectric conversion unit 110 detects photons from the pulsed light L1 and external light L2, and outputs pulse signals whose number corresponds to the number of detected photons. Meanwhile, during a second period in which the pulsed light is off during one cycle of blinking of the pulsed light, the photoelectric conversion unit 110 detects photons from the external light L2, and outputs pulse signals whose number corresponds to the number of detected photons. The number of pulse signals indicates the amount (intensity) of light received by the pixel 100 during the first or second period.

[0041] <Arithmetic processing unit 120> The arithmetic processing unit 120 is an up-down counter that counts up or down the pulse signal. The arithmetic processing unit 120 counts up the pulse signal output from the photoelectric conversion unit 110 during a first period when the pulsed light L1 is on, and counts down the pulse signal output from the photoelectric conversion unit 110 during a second period when the pulsed light L1 is off. In other words, the arithmetic processing unit 120 counts up the pulse signal during a period when the pixel 100 receives both the pulsed light L1 and the external light L2, and counts down the pulse signal during a period when the pixel 100 receives only the external light L2. With this configuration, a count value indicating the amount of light containing only the external light component is subtracted from a count value indicating the amount of light containing both the pulsed light component and the external light component, thereby obtaining a count value indicating the amount of light containing only the pulsed light component.

[0042] The arithmetic processing unit 120 of this embodiment performs a process of up-counting the pulse signal and a process of down-counting the pulse signal for each cycle of blinking of the pulsed light L1 during the period when the pulsed light L1 is continuously irradiated from the light source device 10 to the subject 50. The final count value of the arithmetic processing unit 120 is a value (integrated value) indicating the light amount of the pulsed light L1 during the period when the pulsed light L1 is continuously irradiated to the subject 50, and is output to the signal processing device 30 as a pixel signal.

[0043] Next, the operation of the imaging system 1 according to this embodiment will be described in detail with reference to Fig. 3. Fig. 3 is a timing chart showing an example of the operation of the imaging system 1.

[0044] The upper part of Fig. 3 shows the light emission operation of the pulsed light L1 emitted from the light source device 10, and the lower part of Fig. 3 shows the change in the count value of the arithmetic processing unit 120. The period surrounded by the rectangular dashed line in Fig. 3 indicates the count period during which the arithmetic processing unit 120 counts up or down the pulse signal. The count period is the up-count period T during which the arithmetic processing unit 120 counts up the pulse signal. UP and a down-count period T during which the arithmetic processing unit 120 counts down the pulse signal. DOWNIn one cycle of flashing of the pulsed light, the period during which the pulsed light is on is defined as a first period T1, and the period during which the pulsed light is off is defined as a second period T2. UP ,T DOWN is preferably set to a period slightly shorter than the first and second periods T1 and T2 (to allow for a margin for timing deviation).

[0045] In the imaging system 1 of this embodiment, during the period in which pulsed light L1 flashing at a predetermined frequency is continuously irradiated onto the subject 50, the calculation processing unit 120 performs a process of up-counting the pulse signal and a process of down-counting the pulse signal for each cycle of the flashing of the pulsed light.

[0046] 3, in the first cycle of blinking of the pulsed light L1, the arithmetic processing unit 120 first down-counts the pulse signal output from the photoelectric conversion unit 110 during a second period T2 in which the pulsed light is turned off. Then, the arithmetic processing unit 120 up-counts the pulse signal output from the photoelectric conversion unit 110 during a first period T1 in which the pulsed light is turned on.

[0047] Similarly, in the next cycle of blinking of the pulsed light L1, the arithmetic processing unit 120 down-counts the pulse signal output from the photoelectric conversion unit 110 during a second period T2 when the pulsed light is off. Subsequently, the arithmetic processing unit 120 up-counts the pulse signal output from the photoelectric conversion unit 110 during a first period T1 when the pulsed light is on.

[0048] The arithmetic processing unit 120 repeatedly performs the above process during the period in which the pulsed light L1 is continuously irradiated onto the subject 50. The number of pulse signals that is counted up is a value indicating the amount of light that includes a pulsed light component and an external light component, and the number of pulse signals that is counted down is a value indicating the amount of light that includes only the external light component.

[0049] 3, the count value of the arithmetic processing unit 120 increases and decreases repeatedly over time, gradually increasing by the number of pulse signals corresponding to the light intensity of the pulsed light component. The final count value (accumulated value) CV of the arithmetic processing unit 120 is a value indicating the light intensity of only the pulsed light component from which the external light component has been removed, and can be output to the signal processing device 30 as the pixel signal of the pixel 100. The signal processing device 30 generates a captured image of the subject 50 from the pixel signals of the multiple pixels 100.

[0050] As described above, in the imaging system 1 of this embodiment, the amount of pulsed light reflected by the subject 50 is detected for each cycle of the blinking of the pulsed light while removing external light components. Here, one cycle of the pulsed light is significantly shorter than the period (one frame period) in which a captured image is generated. Therefore, even if the subject 50 moves during this period, multiple on periods and multiple off periods of the pulsed light within one frame can be considered to overlap as a whole, making it less likely that artifacts will occur in the captured image. Therefore, the imaging system 1 of this embodiment can reduce the occurrence of artifacts in the captured image while removing the influence of external light L2.

[0051] Additionally, in the imaging system 1 of this embodiment, processing to remove the influence of external light L2 is performed within the pixel 100 and within the time required to acquire one frame image. Therefore, the imaging system 1 of this embodiment does not require an image processing device that performs differential processing on two captured images, simplifying the system configuration.

[0052] Furthermore, in the imaging system 1 of this embodiment, photons of pulsed light are detected by the SPAD 111. Therefore, according to the imaging system 1 of this embodiment, the detection accuracy of pulsed light is improved compared to when a photodiode is used.

[0053] Furthermore, in the imaging system 1 of this embodiment, the process of down-counting the pulse signal is performed before the process of up-counting the pulse signal. With this configuration, the negative side of the up-down counter (arithmetic processing unit 120) only needs to cover the range of the integrated value of the external light component, and the positive side only needs to cover the range of the integrated value of the pulse light component, thereby minimizing the counter size of the up-down counter. That is, when starting with up-counting (when light is irradiated), the maximum count number of the up-counting is the external light component + reflected light component, but when starting with down-counting (when not irradiated), the maximum down-counting is the external light component and the maximum up-counting is the reflected light component, so it is determined by the larger of the external light component and the reflected light component. Therefore, the counter size can be made smaller when starting with down-counting (when not irradiated).

[0054] Note that the higher the frequency of the pulsed light L1, the smaller the time difference between the first period T1 and the second period T2, further suppressing the occurrence of artifacts. When using high-frequency pulsed light L1, the light-receiving period (first or second period T1, T2) becomes shorter, so the signal amount of the external light component becomes smaller, and the count value of the external light component also becomes smaller. Therefore, according to the imaging system 1 of this embodiment, the counter size of the up / down counter (arithmetic processing unit 120) can be made smaller as the frequency of the pulsed light L1 becomes higher.

[0055] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to Figures 4 and 5. In the second embodiment, the imaging system 1 of the present invention is applied to an iToF (Indirect Time of Flight) ranging system. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and their description will be omitted.

[0056] 4 is a circuit diagram showing a schematic configuration of a pixel 100 according to this embodiment. Note that the solid-state imaging device 20 according to this embodiment is a so-called multi-tap image sensor, and the following description will be given taking as an example a case where the pixel 100 has four taps.

[0057] 4, the pixel 100 includes a photoelectric conversion unit 110, first to fourth arithmetic processing units 121 to 124, and first to fourth switches 131 to 134. The photoelectric conversion unit 110 is connected to the first to fourth arithmetic processing units 121 to 124 via the first to fourth switches 131 to 134.

[0058] The photoelectric conversion unit 110 includes a SPAD 111 that detects photons. The first to fourth arithmetic processing units 121 to 124 are up / down counters that up-count or down-count the pulse signals output from the photoelectric conversion unit 110. The first to fourth switches 131 to 134 switch between the first to fourth arithmetic processing units 121 to 124 that are connected to the photoelectric conversion unit 110. The first to fourth arithmetic processing units 121 to 124 correspond to the first to fourth taps, respectively.

[0059] In the imaging system 1 of this embodiment configured as described above, pulsed light L1 is intermittently irradiated onto the subject 50 from the light source device 10. Since this embodiment is applied to iToF, the pulsed light L1 preferably flashes at a frequency of several tens of MHz to several hundreds of MHz. Then, during a first period when the light source device 10 is irradiating the subject 50 with pulsed light, the pulse signal output from the photoelectric conversion unit 110 is counted up. On the other hand, during a second period when the irradiation of the pulsed light onto the subject 50 is stopped, the pulse signal output from the photoelectric conversion unit 110 is counted down. The first period when the pulsed light is irradiated and the second period when the irradiation of the pulsed light is stopped are repeated at a predetermined frequency (for example, 1 kHz to 10 kHz).

[0060] FIG. 5 is a timing chart showing an example of the operation of the imaging system 1 according to this embodiment. In FIG. 5, "Light" indicates the drive signal of the light source device 10, and "Tap1" to "Tap4" indicate the drive signals of the first to fourth switches 131 to 134, which are driven with a phase difference of 90 degrees. "Up" indicates the drive signal for the up-counting operation of the first to fourth arithmetic processing units 121 to 124, and "Down" indicates the drive signal for the down-counting operation of the first to fourth arithmetic processing units 121 to 124. In this embodiment, an example of four taps will be described, but the number of taps can be increased to achieve a finer phase difference. When the number of taps is n, the phase difference can be configured as 360 / n degrees.

[0061] In the imaging system 1 of this embodiment, a first period during which pulsed light is irradiated onto the subject 50 and a second period during which irradiation of the pulsed light onto the subject 50 is stopped are periodically repeated within one frame period. The first to fourth arithmetic processing units 121 to 124 up-count the pulse signal during the first period and down-count the pulse signal during the second period.

[0062] More specifically, as shown in FIG. 5, first, in a second period in which irradiation of the subject 50 with pulsed light is stopped, the first to fourth switches 131 to 134 are turned on, and the first to fourth arithmetic processing units 121 to 124 are connected to the photoelectric conversion unit 110. At the same time, the down-count operation is turned on, and the first to fourth arithmetic processing units 121 to 124 down-count the pulse signals output from the photoelectric conversion unit 110. The first to fourth arithmetic processing units 121 to 124 count down the pulse signals output from the photoelectric conversion unit 110 during a predetermined down-count period T DOWN As a result, the first to fourth arithmetic processing units 121 to 124 count down the count value indicating the amount of light containing only the external light component.

[0063] Subsequently, during a first period in which the subject 50 is continuously irradiated with pulsed light, the first switch 131, the third switch 133, the second switch 132, and the fourth switch 134 are each turned on with a phase difference of 90 degrees, and the first to fourth arithmetic processing units 121 to 124 are each connected to the photoelectric conversion unit 110 with a phase difference of 90 degrees. During this period, the up-count operation is turned on, and the first to fourth arithmetic processing units 121 to 124 up-count the pulse signals output from the photoelectric conversion unit 110 while the corresponding switches 131 to 134 are in the on state. During the first period, the first switch 131, the third switch 133, the second switch 132, and the fourth switch 134 are repeatedly turned on in that order in synchronization with the blinking operation of the pulsed light L1. The first to fourth arithmetic processing units 121 to 124 are connected to the photoelectric conversion unit 110 with a phase difference of 90 degrees, and the first to fourth arithmetic processing units 121 to 124 are connected to the photoelectric conversion unit 110 with a phase difference of 90 degrees. During this period, the up-count operation is turned on, and the first to fourth arithmetic processing units 121 to 124 up-count the pulse signals output from the photoelectric conversion unit 110 while the corresponding switches 131 to 134 are in the on state. During the first period, the first switch 131, the third switch 133, the second switch 132, and the fourth switch 134 are repeatedly turned on in that order in synchronization with the blinking operation of the pulsed light L1. UP As a result, the pulse signals for the light including the pulsed light component and the external light component are time-divided with a phase difference of 90 degrees, and the first to fourth arithmetic processing units 121 to 124 count up the count values ​​indicating the light amounts of the light time-divided into four taps.

[0064] The arithmetic processing unit 120 repeatedly performs the above process during a period (e.g., one frame period) during which the pulsed light L1 is intermittently irradiated onto the subject 50. The number of pulse signals that are counted up is a value indicating the amount of light containing a pulsed light component and an external light component, and the number of pulse signals that are counted down is a value indicating the amount of light containing only the external light component. The final count value of the first to fourth arithmetic processing units 121 to 124 is a value (integrated value) indicating the amount of light containing only the pulsed light component during the period during which the pulsed light L1 is intermittently irradiated onto the subject 50, and can be output to the signal processing device 30 as a pixel signal. The signal processing device 30 calculates the phase difference between the irradiated pulsed light and the received pulsed light from the pixel signals of the four taps, calculates the distance to the subject 50 from the phase difference information, and generates a ranging image of the subject 50. Note that the technology for calculating the distance to the subject from the pixel signals of the four taps and generating a ranging image is a well-known technology, so a detailed description thereof will be omitted.

[0065] As described above, according to the imaging system 1 of this embodiment, during the period in which pulsed light is intermittently irradiated onto the subject 50, the amount of pulsed light reflected by the subject 50 is detected while external light components are removed. Here, the cycle in which the process of counting up the pulse signal and the process of counting down the pulse signal are performed is sufficiently short compared to the period in which a ranging image is generated, so even if the subject 50 moves during this period, artifacts are unlikely to occur in the ranging image. Therefore, according to the imaging system 1 of this embodiment, it is possible to suppress the occurrence of artifacts in the ranging image while removing the influence of external light L2.

[0066] If a similar external light removal is performed in a general iToF ranging system using a photodiode, it is necessary to time-divide the output signals with a phase difference of 90 degrees even during the period when only the external light component is received, taking into account the imbalance of electric charges between the taps. However, in the imaging system 1 of this embodiment, since the pulse signals digitized by the photoelectric conversion unit 110 are counted, it is not necessary to consider the imbalance of electric charges. Therefore, in the imaging system 1 of this embodiment, it is not necessary to time-divide the output signals during the period when only the external light component is received, and the period when only the external light component is received can be halved compared to a general iToF ranging system. Therefore, according to the imaging system 1 of this embodiment, the second period (or the down-count period T DOWN ) is the first period (or the up-count period T UP ), which results in a reduction in the total light receiving time and an improvement in the frame rate.

[0067] Furthermore, in the imaging system 1 of this embodiment, the process of down-counting the pulse signal is performed before the process of up-counting the pulse signal. With this configuration, the counter sizes of the four up-down counters (first to fourth arithmetic processing units 121 to 124) can be minimized.

[0068] The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims.

[0069] For example, in the first and second embodiments described above, an example was given in which the photoelectric conversion unit 110 outputs a pulse signal whose number corresponds to the number of photons. However, the output signal is not limited to a pulse signal whose number corresponds to the number of photons, and may be a voltage signal whose magnitude corresponds to the number of photons. In this case, the arithmetic processing unit 120 performs a process of adding a voltage value in a first period and a process of subtracting a voltage value in a second period.

[0070] In the above-described first and second embodiments, the case where the photoelectric conversion unit 110 is provided with the SPAD 111 has been described as an example. However, the photodetection element for detecting photons is not limited to a SPAD, and may be an avalanche photodiode (APD).

[0071] In the first and second embodiments described above, the process of down-counting the pulse signal is performed before the process of up-counting the pulse signal. However, the process of up-counting the pulse signal may be performed before the process of down-counting the pulse signal. [Explanation of symbols]

[0072] 1 imaging system, 10 light source device, 20 solid-state imaging device, 30 signal processing device, 50 subjects, 100 pixels, 110 photoelectric conversion unit, 111 Single-photon avalanche diode, 112 transistors, 113 inverter, 120,121,122,123,124 Processing unit; 131,132,133,134 switches, T1 1st period, T2 second period, T UPUp-count period, T DOWN Down count period.

Claims

1. A solid-state imaging device having a plurality of pixels, Each of the plurality of pixels is a photoelectric conversion unit that detects photons and outputs a signal according to the number of the photons; an arithmetic processing unit that adds together output signals of the photoelectric conversion unit for pulsed light received by the pixel and external light during a first period within one frame period, and subtracts output signals of the photoelectric conversion unit for external light received by the pixel during a second period within the one frame period; A solid-state imaging device comprising:

2. the photoelectric conversion unit includes a single-photon avalanche diode and outputs pulse signals whose number corresponds to the number of the photons; 2. The solid-state imaging device according to claim 1, wherein the arithmetic processing section includes an up-down counter that counts up the pulse signal during the first period and counts down the pulse signal during the second period.

3. the first period is a period during which the pulsed light is turned on in one cycle of blinking of the pulsed light, and the second period is a period during which the pulsed light is turned off in one cycle of blinking of the pulsed light, 3. The solid-state imaging device according to claim 1, wherein the arithmetic processing unit performs an addition process and a subtraction process of the output signal during one blinking cycle of the pulsed light.

4. 4. The solid-state imaging device according to claim 3, wherein the arithmetic processing unit repeatedly performs a process of adding the output signal and a process of subtracting the output signal during a period in which the pixel continuously receives the pulsed light.

5. 3. The solid-state imaging device according to claim 1, further comprising a plurality of the arithmetic processing sections corresponding to a plurality of time-shared output signals.

6. the first period is a period during which the pixel continuously receives the pulsed light, and the second period is a period during which the pixel does not receive the pulsed light, The solid-state imaging device according to claim 5 , wherein each of the arithmetic processing units performs a process of adding the output signal during the first period and a process of subtracting the output signal during the second period.

7. the first period and the second period are repeated periodically within one frame period, 7. The solid-state imaging device according to claim 6, wherein each of the arithmetic processing units repeatedly performs the process of adding the output signal and the process of subtracting the output signal within one frame period.

8. 3. The solid-state imaging device according to claim 1, wherein the arithmetic processing unit performs a subtraction process on the output signal before performing an addition process on the output signal.

9. An imaging system including a light source device that irradiates a subject with pulsed light and a solid-state imaging device that receives the pulsed light reflected by the subject, the solid-state imaging device has a plurality of pixels, Each of the plurality of pixels is a photoelectric conversion unit that detects photons and outputs a signal according to the number of the photons; an arithmetic processing unit that adds together output signals of the photoelectric conversion unit for pulsed light received by the pixel and external light during a first period within one frame period, and subtracts output signals of the photoelectric conversion unit for external light received by the pixel during a second period within the one frame period; An imaging system comprising:

10. the photoelectric conversion unit includes a single-photon avalanche diode and outputs pulse signals whose number corresponds to the number of the photons; The imaging system according to claim 9 , wherein the arithmetic processing unit includes an up-down counter that counts up the pulse signal during the first period and counts down the pulse signal during the second period.

11. the first period is a period during which the pulsed light is turned on in one cycle of blinking of the pulsed light, and the second period is a period during which the pulsed light is turned off in one cycle of blinking of the pulsed light, 11. The imaging system according to claim 9, wherein the arithmetic processing unit performs an addition process and a subtraction process of the output signal during one blinking cycle of the pulsed light.

12. The imaging system according to claim 11 , wherein the arithmetic processing unit repeatedly performs a process of adding the output signal and a process of subtracting the output signal during a period in which the light source device continuously irradiates the subject with the pulsed light.

13. 11. The imaging system according to claim 9, wherein the solid-state imaging device includes a plurality of the arithmetic processing units corresponding to a plurality of time-shared output signals.

14. the first period is a period during which the light source device continuously irradiates the subject with pulsed light, and the second period is a period during which the light source device stops irradiating the subject with pulsed light, The imaging system according to claim 13 , wherein each of the arithmetic processing units performs a process of adding the output signal during the first period and a process of subtracting the output signal during the second period.

15. the first period and the second period are repeated periodically within one frame period, The imaging system according to claim 14 , wherein each of the arithmetic processing units repeatedly performs the process of adding the output signal and the process of subtracting the output signal within one frame period.

16. The imaging system according to claim 9 , wherein the arithmetic processing unit performs the subtraction process on the output signal before the addition process on the output signal.

Images