Image sensing apparatus and imaging apparatus including same

Abstract

The invention relates to an image sensing device and an imaging device including the same. The image sensing apparatus includes: a pixel configured to generate a pixel signal having a single photon avalanche diode (SPAD) pulse by detecting incident light; a time-to-digital converter (TDC) configured to generate time-to-digital converter (TDC) data representative of a time of flight (TOF) of the SPAD pulse; and a TDC memory configured to store TDC data in a cell memory determined from among the plurality of cell memories according to a number of occurrences of the SPAD pulse.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0080662, filed on June 22, 2023, the disclosure of which is incorporated herein by reference in its entirety as a part of the disclosure of this patent document. Technical Field

[0003] The present technology and embodiments generally relate to an image sensing device capable of detecting a distance to a target object and an imaging device including the image sensing device. Background Art

[0004] Image sensing devices are devices that capture optical images by converting light into electrical signals using photosensitive semiconductor materials that react to light. With the development of the automotive, medical, computer and communication industries, the demand for high-performance image sensing devices is increasing in various devices such as smartphones, digital cameras, game consoles, IOT (Internet of Things), robots, security cameras and medical micro cameras.

[0005] In recent years, image sensing devices have been actively used not only to acquire color images but also to sense the distance to a target object. In particular, among various image sensing devices, image sensing devices based on a time-of-flight (ToF) method that can directly or indirectly measure the time required for irradiated light to reflect from an object and return to an image sensor have been widely used. Summary of the invention

[0006] According to an embodiment of the disclosed technology, an image sensing device may include: a pixel configured to generate a pixel signal having a single photon avalanche diode (SPAD) pulse by detecting incident light; a time-to-digital converter (TDC) configured to generate time-to-digital converter (TDC) data representing a time of flight (TOF) of the SPAD pulse; and a TDC memory configured to store the TDC data in a unit memory determined from among a plurality of unit memories according to the number of occurrences of the SPAD pulse.

[0007] According to another embodiment of the disclosed technology, an image sensing device may include: a memory block provided with a plurality of unit memories; and a memory control block configured to control the memory block to store time-to-digital converter (TDC) data representing a time of flight (TOF) of a single photon avalanche diode (SPAD) pulse, the SPAD pulse being stored in a unit memory determined from among the plurality of unit memories based on a number of occurrences of the SPAD pulse generated by detecting incident light.

[0008] According to another embodiment of the disclosed technology, an imaging device may include: an image sensing device configured to store time-to-digital converter (TDC) data representing the time of flight (TOF) of a single photon avalanche diode (SPAD) pulse, the SPAD pulse being stored in a unit memory determined from among a plurality of unit memories based on a number of occurrences of the SPAD pulse generated by detecting incident light; and an image signal processor configured to calculate a distance between a pixel generating the SPAD pulse and a target object based on the TDC data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] When with Figure 1 The above-described and other features and advantageous aspects of the disclosed technology will become apparent when considered together with reference to the following detailed description.

[0010] Figure 1 is a block diagram illustrating an example of an imaging device based on some embodiments of the disclosed technology.

[0011] Figure 2 is a method for showing some embodiments of the disclosed technology for Figure 1 A circuit diagram of an example of constituent elements for generating a candidate time of flight (TOF) in an image sensing device shown.

[0012] Figure 3 is a diagram showing some embodiments of the disclosed technology for Figure 2 The constituent elements shown are a timing diagram of example operations for generating a candidate time of flight (TOF).

[0013] Figure 4 Some embodiments based on the disclosed technology are shown Figure 2 A schematic diagram of an example of a time-to-digital converter (TDC) memory is shown.

[0014] Figure 5 Some embodiments based on the disclosed technology are shown Figure 4 A timing diagram of an example operation of a TDC memory is shown.

[0015] Figure 6 is a method for showing some embodiments of the disclosed technology for Figure 1 A circuit diagram of another example of constituent elements for generating a candidate time of flight (TOF) in an image sensing device is shown.

[0016] Figure 7 Some embodiments based on the disclosed technology are shown Figure 6 A timing diagram of an example operation of a single photon avalanche diode (SPAD) is shown. DETAILED DESCRIPTION

[0017] The present application provides embodiments and examples of an image sensing device capable of detecting the distance of a target object and an imaging device including the image sensing device, which can be used in a configuration to substantially solve one or more technical problems or engineering problems and alleviate limitations or disadvantages encountered in some other image sensing devices. Some embodiments of the disclosed technology relate to an image sensing device capable of reducing unnecessary power consumption and an imaging device including the image sensing device. The disclosed technology provides multiple embodiments of an image sensing device and an imaging device including the image sensing device that can reduce peak current by minimizing the number of unit memories configured to operate when time-to-digital converter (TDC) data corresponding to a single-photon avalanche diode (SPAD) pulse is stored.

[0018] Reference will now be made in detail to embodiments of the disclosed technology, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. Although the present disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are illustrated by way of example in the accompanying drawings. However, the disclosure should not be construed as being limited to the embodiments described herein.

[0019] Hereinafter, various embodiments will be described with reference to the accompanying drawings. However, it should be understood that the disclosed technology is not limited to specific embodiments, but includes various modifications, equivalents and / or alternatives of the embodiments. The embodiments of the disclosed technology can provide various effects that can be directly or indirectly recognized by the disclosed technology.

[0020] Various embodiments of the disclosed technology relate to an image sensing device capable of reducing unnecessary power consumption and an imaging device including the image sensing device.

[0021] It is to be understood that both the foregoing general description and the following detailed description of the disclosed technology are illustrative and explanatory and are intended to provide further explanation of the disclosure as claimed.

[0022] Figure 1 is a block diagram illustrating an example of an imaging device (ID) based on some embodiments of the disclosed technology.

[0023] refer to Figure 1 , an imaging device (ID) may refer to a device, such as a digital still camera for taking still images or a digital video camera for taking moving images. For example, an imaging device (ID) may be implemented as a digital single-lens reflex (DSLR) camera, a mirrorless camera, or a smart phone, etc. An imaging device (ID) may include a device having a lens and an image pickup element, so that the device can capture (or photograph) a target object and thus can create an image of the target object. In some embodiments, an imaging device (ID) may be implemented as a lidar sensor.

[0024] The imaging device (ID) may include an image sensing device 100 and an image signal processor (ISP) 150 .

[0025] The image sensing device 100 may be a complementary metal oxide semiconductor image sensor (CIS) for converting incident light into an electrical signal. The image sensing device 100 may include a light source 10, a lens module 20, a light source driver 30, a pixel array 110, a sensor driver 120, a readout circuit 130, and a timing controller 140.

[0026] The light source 10 can emit light to the target object 1 upon receiving a control signal from the light source driver 30. The light source 10 can be a laser diode (LD) or a light emitting diode (LED) for emitting light having a specific wavelength band (e.g., near infrared (NIR) light, infrared (IR) light, or visible light), or can be any one of a near infrared laser (NIR), a point light source, a monochromatic light source combined with a white lamp or a monochromator, and a combination of other laser sources. For example, the light source 10 can emit infrared light having a wavelength of 800nm ​​to 1000nm. In addition, the light emitted from the light source 10 can be a pulsed light having a predetermined period, amplitude, and pulse width. Although for ease of description, Figure 1 Only one light source 10 is shown, but the scope or spirit of the disclosed technology is not limited thereto, and a plurality of light sources may also be arranged near the lens module 20 .

[0027] The lens module 20 may collect light reflected from the target object 1 and may allow the collected light to be focused on the pixel (PX) of the pixel array 110. For example, the lens module 20 may include a focusing lens having a surface formed of glass or plastic or another cylindrical optical element having a surface formed of glass or plastic. The lens module 20 may include a plurality of lenses arranged to focus on an optical axis.

[0028] The light source driver 30 may drive the light source 10 under the control of the timing controller 140. In particular, the light source driver 30 may control the waveform (eg, period, amplitude, pulse width, etc.) of the emission light (EL) output from the light source 10.

[0029] The pixel array 110 may include a plurality of pixels (PX) arranged continuously in a two-dimensional (2D) matrix structure (e.g., arranged continuously in a column direction and / or a row direction). Each of the plurality of pixels (PX) may generate a pixel signal by sensing incident light sensed via the lens module 20 under the control of the sensor driver 120.

[0030] Each pixel (PX) may be an infrared pixel for generating a pixel signal by sensing incident light, the incident light including reflected light (RL) generated when light (EL) emitted from the light source 10 is reflected from the target object 1. The incident light may be light incident on the pixel array 110 through the lens module 20. According to an embodiment, the infrared pixel may be a depth pixel for calculating the distance to the target object 1. According to another embodiment, the infrared pixel may include a pixel for generating an infrared image by simply sensing reflected infrared light from a scene without sensing reflected visible light. According to yet another embodiment, the pixel (PX) may include a pixel for generating a color image by sensing reflected visible light sensed from a scene. Hereinafter, it will be described assuming that each pixel (PX) is a single photon avalanche diode (SPAD) pixel for detecting the distance to the target object 1 according to a direct time of flight (ToF) method. Reference will be made to the description from Figure 2 A more detailed structure and operation of each unit pixel (PX) are described with reference to the accompanying drawings.

[0031] The sensor driver 120 may drive the pixels (PX) of the pixel array 110 in response to the timing signals output from the timing controller 140. For example, the sensor driver 120 may generate a control signal capable of selecting and controlling the pixels (PX) in at least one row line among a plurality of row lines included in the pixel array 110.

[0032] The readout circuit 130 can process the pixel signal received from the pixel array 110 under the control of the timing controller 140, and can generate and store depth data for detecting the distance of the target object 1. Specifically, the readout circuit 130 can calculate the candidate flight time (TOF) corresponding to the SPAD pulse generated by each pixel (PX) sensing the incident light (including reflected light), and can store the candidate flight time (TOF) corresponding to the SPAD pulse in units of subframes. The readout circuit 130 can transmit the candidate flight time (TOF) stored in units of subframes to the image signal processor 150 under the control of the timing controller 140. In some embodiments, for ease of description, the candidate flight time (TOF) can be used interchangeably with the flight time as needed.

[0033] The timing controller 140 may generate timing signals to control the light source driver 30, the sensor driver 120, and the readout circuit 44. In some embodiments, the timing controller 140 may generate timing signals according to data received from the readout circuit 130 or a request received from the image signal processor 150. In some embodiments, the timing controller 140 may include a logic control circuit, a phase-locked loop (PLL) circuit, a timing control circuit, a communication interface circuit, etc.

[0034] The image signal processor 150 may perform at least one image signal processing on the image data (IDATA) received from the image sensing device 100, and thus may generate processed image data. The image signal processor 150 may reduce noise of the image data (IDATA), and may perform various image signal processing (e.g., interpolation of the image data (IDATA), lens distortion correction, etc.) for improving the image quality of the image data.

[0035] The image data (IDATA) may include a candidate time of flight (TOF) stored for each subframe. The image signal processor 150 may generate a histogram for one frame by accumulating the data stored in units of subframes, and may determine a target time of flight (TOF) for one frame based on the histogram. The target time of flight (TOF) may be determined for each pixel (PX), and the image signal processor 150 may determine the target distance (i.e., the distance to the target object 1 detected by each pixel PX) based on the target time of flight (TOF) for each pixel (PX). A set (or collection) of target distances for pixels (PX) included in the pixel array 110 may be referred to as a depth image and may be included in the processed image data.

[0036] The image signal processor 150 may transmit the processed image data to a host device (not shown). The host device (not shown) may be a processor (e.g., an application processor) for processing image signal processed (ISP) image data received from the image signal processor 150, a memory (e.g., a nonvolatile memory) for storing the ISP image data, or a display device (e.g., a liquid crystal display (LCD)) for visually displaying the ISP image data.

[0037] The image signal processor 150 may transmit a control signal for controlling the operation (whether to operate, operation timing, operation mode, etc.) of the image sensing device 100 to the image sensing device 100 .

[0038] Figure 2 is a method for showing some embodiments of the disclosed technology for Figure 1 A circuit diagram of an example of constituent elements for generating a candidate time of flight (TOF) in an image sensing device shown.

[0039] refer to Figure 2 , the image sensing device 100 may include a component element 200 for generating a candidate time of flight (TOF). The component element 200 for generating a candidate time of flight (TOF) may include a pixel 210, a time-to-digital converter (TDC) 220, and a TDC memory 230. According to an embodiment, the pixel 210 may be included in Figure 1The TDC 220 and the TDC memory 230 may be included in the pixel array 110, and the TDC 220 and the TDC memory 230 may be included in the readout circuit 130. According to another embodiment, at least one of the TDC 220 and the TDC memory 230 may be included in the pixel array 110.

[0040] Pixel 210 may mean Figure 1 The pixel 210 may be substantially the same as the pixel (PX) described in FIG. Pixel 210 may include a single photon avalanche diode (SPAD), a quenching circuit (Quenching Circuit, QX) and a pixel inverter (IV_P).

[0041] The SPAD may detect a single photon of reflected light (RL) reflected by the target object 1 , and may generate a voltage pulse corresponding to the detected single photon.

[0042] SPAD can be used as a photodiode including a photosensitive PN junction. In SPAD, avalanche breakdown can be triggered by a single photon received in Geiger mode, so that SPAD can generate a voltage pulse, the single photon receives a reverse bias voltage, at which the cathode to anode voltage is higher than the breakdown voltage. As described above, the process of triggering avalanche breakdown by a single photon to generate a voltage pulse is hereinafter referred to as an avalanche process.

[0043] One terminal of the SPAD can receive a first bias voltage (V bd ), for applying a reverse bias voltage to the SPAD that can operate the SPAD in Geiger mode. For example, the first bias voltage (V bd ) may be a voltage at which the first bias voltage (V bd ) minus the second bias voltage (Vex) becomes a negative voltage having an absolute value higher than the breakdown voltage absolute value. The other end of the SPAD may be connected to a sensing node (SN), and the SPAD may output a voltage pulse generated by sensing a single photon to the sensing node (SN).

[0044] The quenching circuit (QX) may perform a quenching operation in which the voltage of the sensing node (SN) is returned to the second bias voltage (V ex ) (or the SPAD returns to Geiger mode). One terminal of the quenching circuit (QX) may receive a second bias voltage (V ex ), for applying a reverse bias voltage to the SPAD that can operate the SPAD in Geiger mode. For example, the second bias voltage (V ex ) may be a voltage at which the first bias voltage (V bd ) minus the second bias voltage (Vex ) and the value obtained becomes a negative voltage having an absolute value higher than the breakdown voltage. The other terminal of the quenching circuit (QX) may be connected to the sensing node (SN), and when the quenching circuit (QX) is turned on, the voltage of the sensing node (SN) may return to the second bias voltage (V ex ).

[0045] In some embodiments, the quenching circuit (QX) may be implemented as a P-channel metal oxide semiconductor (PMOS) transistor that receives a quenching control voltage (V qch ). Quenching control voltage (V qch ) may: have a turn-on voltage when the voltage of the sensing node (SN) is lowered due to the generation of the voltage pulse, and have a turn-on voltage when the voltage of the sensing node (SN) returns to the second bias voltage (V ex ) has a turn-off voltage. For example, the quench control voltage (V gch ) may be supplied from the sensor driver 120 or may be supplied from the sensing node (SN).

[0046] The pixel inverter (IV_P) can sample the analog voltage pulse generated from the sensing node (SN), and can convert the sampled analog voltage pulse into a digital pulse signal (i.e., a SPAD pulse) having a waveform opposite to that of the analog voltage pulse. Here, the sampling method can be implemented as a method of converting a voltage pulse into a pulse signal having a logic level of "0" or "1" according to whether the level of the voltage pulse is equal to or higher than the threshold level, but the scope or spirit of the disclosed technology is not limited thereto. The analog voltage pulse generated at the sensing node (SN) can have a waveform immediately from the second bias voltage (V ex ) is reduced and then returns to the second bias voltage (V ex ). Therefore, since the pixel inverter (IV_P) converts the voltage pulse into a digital pulse signal having a waveform opposite to that of the voltage pulse, the pixel inverter (IV_P) may have a waveform as follows: a pixel signal (PX_OUT) corresponding to the output of the pixel inverter (IV_P) has a logic level 0, then immediately converts to a logic level 1 (or a logic high level) when the voltage pulse is generated, and finally returns to a logic level 0 (or a logic low level).

[0047] In some embodiments, the pulse of the pixel signal (PX_OUT) generated by the pixel inverter (IV_P) can be referred to as a single photon avalanche diode (SPAD) pulse.

[0048] The TDC 220 may calculate the time delay between the SPAD pulse of the pixel signal (PX_OUT) output from the pixel 210 and the reference pulse of the emission light (EL), and may generate TDC data (TDC_OUT) representing the time delay. In some embodiments, the TDC 220 may generate the TDC data (TDC_OUT) by performing a counting operation from the time point of the reference pulse to the time point of the SPAD pulse. The TDC 220 may obtain the time point at which the reference pulse of the emission light (EL) is generated from the timing controller 140 that controls the light source driver 30, or may consider a predetermined time point (e.g., a time point before a predetermined time from the start point of the subframe) as the generation time point of the reference pulse. The TDC data (TDC_OUT) output by the TDC 220 may represent the candidate time of flight (TOF) described above, and in the following description, the TDC data (TDC_OUT) and the candidate time of flight (TOF) may be used interchangeably in a similar concept.

[0049] The TDC 220 may be activated or deactivated according to a TDC enable signal (TDC_EN) supplied from the timing controller 140 .

[0050] The TDC enable signal (TDC_EN) may have an activation level (e.g., logic level 1) when it is necessary to generate TDC data (TDC_OUT) for the pixel 210 (e.g., during each subframe), and may have a deactivation level (e.g., logic level 0) when it is not necessary to generate TDC data (TDC_OUT) for the pixel 210 (e.g., when the storage capacity of the TDC memory 230 is full). The TDC memory 230 may store the TDC data (TDC_OUT) and may output memory output data (MEM_OUT) to the image signal processor 150 in predetermined units (e.g., in units of subframes). Reference will be made later to Figure 4 A more detailed structure and operation of the TDC memory 230 is described.

[0051] In addition, although Figure 2 It is known that the SPAD receives a first bias voltage (V bd ), but other embodiments are also possible, and it should be noted that the SPAD can also receive a first bias voltage (V bd ). However, in this case, the first bias voltage (V bd ) and the voltage level of the second bias voltage (Vex) can be determined so that the SPAD can operate in Geiger mode.

[0052] Figure 3 is a diagram showing some embodiments of the disclosed technology for Figure 2 The constituent elements shown are a timing diagram of example operations for generating a candidate time of flight (TOF).

[0053] refer to Figure 3 , the target flight time (TOF) corresponding to the distance to the target object 1 detected by the pixel 210 can be determined based on the data acquired within one frame, and in the disclosed technology, for ease of description, it can be assumed that one frame includes four subframes (SF1~SF4). In other words, in order to more accurately determine the target flight time (TOF), the target flight time (TOF) can be determined by four detection behaviors instead of one detection behavior, which can reduce the influence of noise caused by ambient light. In addition, the number of subframes included in one frame can be predetermined to be 2 or more according to the characteristics required by the image sensing device 100 (e.g., sensing accuracy, sensing speed, sensing environment, etc.), or can be changed according to a control signal from an external device (e.g., a timing controller 140 or an image signal processor 150). In addition, the number of subframes included in one frame can be limited to a predetermined number of subframes, so that the time delay between the pulse of the emitted light (EL) and the pulse of the reflected light (RL) can be maintained unchanged, and the reflected light (RL) is generated when the emitted light (EL) is reflected from the target object 1. This is because, when the time delay is changed, the accuracy of detecting the pulse of the reflected light (RL) may be reduced regardless of whether the influence of noise caused by ambient light is reduced.

[0054] In addition, the image sensing device 100 can detect multiple SPAD pulses (according to the multi-event scheme) and can store candidate flight time (TOF) values ​​corresponding to the multiple SPAD pulses, instead of detecting only one SPAD pulse in each subframe SF1 to SF4 (according to the single-event scheme). At this time, the number of unit memories (M1 to Mn, where "n" is an integer of 2 or more) included in the TDC memory 230 that can store TDC data (TDC_OUT) can be the maximum number of SPAD pulses that can store candidate flight time (TOF).

[0055] First, in the first subframe ( SF1 ), a time delay depending on the distance between the pixel 210 and the target object 1 may occur between a pulse of emission light (EL) and a pulse of reflection light (RL) reflected from the target object 1 .

[0056] The pixel 210 may detect a single photon of incident light including reflected light (RL) to generate a pixel signal (PX_OUT) including a plurality of SPAD pulses. Here, the remaining SPAD pulses (represented by a dotted line) generated by the pulse of the reflected light (RL) except for the second SPAD pulse (represented by a solid line) may correspond to noise components caused by internal factors and external factors (such as ambient light, etc.).

[0057] The TDC 220 may calculate each candidate time of flight (TOF) (or each time delay) of each of a plurality of SPAD pulses of the pixel signal (PX_OUT), and may generate TDC data (TDC_OUT) including each candidate time of flight (TOF).

[0058] The TDC memory 230 may include a plurality of unit memories (M1 to Mn), each of which may store TDC data (TDC_OUT) representing a candidate time of flight (TOF). The plurality of unit memories (M1 to Mn) may respectively store candidate time of flight (TOF) values ​​of the TDC data (TDC_OUT). At this time, among the plurality of unit memories (M1 to Mn), the first unit memory M1 may store a candidate time of flight (TOF) corresponding to the first SPAD pulse, and the second unit memory M2 may store a candidate time of flight (TOF) corresponding to the second SPAD pulse. That is, the mth unit memory Mm (where m is an integer of "1≤m≤n") among the plurality of unit memories (M1 to Mn) may store a candidate time of flight (TPF) corresponding to the mth SPAD pulse.

[0059] The TDC memory 230 may store candidate time of flight (TOF) values ​​corresponding to the SPAD pulse detected in the first subframe SF1 and may transmit the candidate time of flight (TOF) values ​​corresponding to the SPAD pulse detected in the first subframe SF1 to the image signal processor 150 after the first subframe SF1 has passed.

[0060] like Figure 3 As shown, the image signal processor 150 may convert candidate time-of-flight (TOF) values ​​corresponding to the SPAD pulse detected in the first subframe SF1 into histogram values ​​and may store the histogram values.

[0061] Next, in the second subframe SF2, the pixel 210 can detect a single photon of the reflected light (RL) to generate a pixel signal (PX_OUT) including multiple SPAD pulses, and the TDC 220 can calculate the candidate flight time (TOF) of each of the multiple SPAD pulses of the pixel signal (PX_OUT) to generate TDC data (TDC_OUT) including each candidate flight time (TOF).

[0062] The TDC memory 230 may store candidate time of flight (TOF) values ​​corresponding to the SPAD pulse detected in the second subframe SF1 and may transmit the candidate time of flight (TOF) values ​​corresponding to the SPAD pulse detected in the second subframe SF2 to the image signal processor 150 after the second subframe SF2 passes.

[0063] like Figure 3 As shown, the image signal processor 150 may accumulate candidate time-of-flight (TOF) values ​​corresponding to the SPAD pulses detected in the second subframe SF2 to form a histogram indicating accumulated values, and may store the resulting histogram values.

[0064] In each of the third subframe SF3 and the fourth subframe SF4 , the image sensing device 100 and the image signal processor 150 may perform the same operation as that in the second subframe SF2 .

[0065] After the fourth subframe SF4, the image signal processor 150 may obtain a histogram in which candidate TOF values ​​corresponding to the SPAD pulses detected in the first to fourth subframes (SF1-SF4) are accumulated. The image signal processor 150 may then determine the target flight time (TOF) corresponding to the pulse of the reflected light (RL) detected by the pixel 210 based on the accumulated histogram.

[0066] In some embodiments, candidate time of flight (TOF) values ​​corresponding to SPAD pulses detected in the first to fourth subframes (SF1-SF4) may be assigned to time bins divided into predetermined time units, and a time corresponding to a time bin including the largest number of candidate time of flight (TOF) values ​​may be determined as a target time of flight (TOF). Figure 3 In the example of , the time bins may be measured based on the segmented units and the histogram values ​​for the TDC data TDC_OUT.

[0067] exist Figure 3 In the example of FIG. 1 , a candidate time of flight (TOF) corresponding to a SPAD pulse detecting a pulse of reflected light (RL) may be displayed in black, and a candidate time of flight (TOF) corresponding to a SPAD pulse detecting a single photon of a noise component may be displayed in shaded form. Figure 3 As shown, the time corresponding to the twelfth time bin including four candidate TOF values ​​may be determined as the target time of flight (TOF).

[0068] Figure 4 Some embodiments based on the disclosed technology are shown Figure 2A schematic diagram of an example of a time-to-digital converter (TDC) memory 230 is shown.

[0069] refer to Figure 4 , the TDC memory 230 may include a memory block 240 and a memory control block 250 .

[0070] The memory block 240 may include first to eighth unit memories ( M1 ˜ M8 ). Figure 4 The example shows Figure 3 An example case is shown in which the number (n) of unit memories is 8 (ie, n=8).

[0071] Each of the first to eighth unit memories (M1-M8) may be implemented as a D flip-flop, and may have a storage capacity (eg, 1 byte) capable of storing TDC data (TDC_OUT) indicating one candidate time of flight (TOF).

[0072] Each of the first to eighth unit memories (M1 to M8) may include a clock enable terminal (CLK), an input terminal (D), and an output terminal (Q). The clock enable terminal (CLK) of each of the first to eighth unit memories (M1 to M8) may receive a corresponding memory enable signal (ME1 to ME8), the input terminal (D) of each of the first to eighth unit memories (M1 to M8) may receive TDC data (TDC_OUT), and the output terminal (Q) of each of the first to eighth unit memories (M1 to M8) may output data currently stored in each of the first to eighth unit memories (M1 to M8) as first memory output data to eighth memory output data (M1_OUT to M8_OUT). Specifically, the clock enable terminal (CLK) of the first unit memory M1 can receive the memory enable signal ME1, the input terminal (D) of the first unit memory M1 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the first unit memory M1 can output the data currently stored in the first unit memory M1 as the first memory output data (M1_OUT). The clock enable terminal (CLK) of the second unit memory M2 can receive the memory enable signal ME2, the input terminal (D) of the second unit memory M2 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the second unit memory M2 can output the data currently stored in the second unit memory M2 as the second memory output data (M2_OUT). The clock enable terminal (CLK) of the third unit memory M3 can receive the memory enable signal ME3, the input terminal (D) of the third unit memory M3 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the third unit memory M3 can output the data currently stored in the third unit memory M3 as the third memory output data (M3_OUT). The clock enable terminal (CLK) of the fourth unit memory M4 can receive the memory enable signal ME4, the input terminal (D) of the fourth unit memory M4 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the fourth unit memory M4 can output the data currently stored in the fourth unit memory M4 as the fourth memory output data (M4_OUT). The clock enable terminal (CLK) of the fifth unit memory M5 can receive the memory enable signal ME5, the input terminal (D) of the fifth unit memory M5 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the fifth unit memory M5 can output the data currently stored in the fifth unit memory M5 as the fifth memory output data (M5_OUT).The clock enable terminal (CLK) of the sixth unit memory M6 can receive the memory enable signal ME6, the input terminal (D) of the sixth unit memory M6 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the sixth unit memory M6 can output the data currently stored in the sixth unit memory M6 as the sixth memory output data (M6_OUT). The clock enable terminal (CLK) of the seventh unit memory M7 can receive the memory enable signal ME7, the input terminal (D) of the seventh unit memory M7 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the seventh unit memory M7 can output the data currently stored in the seventh unit memory M7 as the seventh memory output data (M7_OUT). The clock enable terminal (CLK) of the eighth unit memory M8 can receive the memory enable signal ME8, the input terminal (D) of the eighth unit memory M8 can receive the TDC data (TDC_OUT), and the output terminal (Q) of the eighth unit memory M8 can output the data currently stored in the eighth unit memory M8 as the eighth memory output data (M8_OUT).

[0073] Each of the first to eighth unit memories (M1 to M8) may store TDC data (TDC_OUT) received through the input terminal (D) at a rising edge of a corresponding enable signal (ME1 to ME8), and may maintain and output the stored TDC data (TDC_OUT).

[0074] The memory control block 250 may include a memory inverter (IV_M), a counter 252 , a decoder 254 , and a plurality of comparators 256 - 1 to 256 - 8 .

[0075] The memory inverter (IV_M) may receive the pixel signal (PX_OUT), invert the pixel signal (PX_OUT), and thus output an inverted pixel signal (PX_OUTb).

[0076] The counter 252 may include a clock enable terminal (CLK) for receiving an inverted pixel signal (PX_OUTb), an input terminal (D) for receiving a TDC enable signal (TDC_EN), and an output terminal (Q) for outputting a counter value (CNT_S) currently stored in the counter 252. Figure 4 In the example of , the counter 252 may be implemented as a D flip-flop based synchronous counter that stores 3 bits to count to a number of 8, indicating the maximum number of TDC data (TDC_OUT) that may be stored in the memory block 240 .

[0077] When the pixel signal PX_OUT has a falling edge, the inverted pixel signal (PX_OUTb) may have a rising edge, and when the TDC enable signal (TDC_EN) has a logic level 1 at the rising edge of the inverted pixel signal (PX_OUTb), the counter 252 may increase the currently stored counter value (CNT_S) by 1.

[0078] The decoder 254 may receive the counter value (CNT_S) and may generate decoder output data (DEC) corresponding to the counter value (CNT_S). In some embodiments, the decoder 254 may be implemented as (P×2 P ) decoder, which receives a counter value (CNT_S) including P bits (wherein "P" is an integer of 1 or greater) and generates a 2-bit decoder corresponding to the counter value (CNT_S). P The decoder outputs data (DEC) of bits. Figure 4 In the example of FIG. 1 , the decoder 254 may receive a 3-bit counter value (CNT_S) and may generate a decoder output data (DEC<0:7>) consisting of 8 (=23) bits. The first decoder bit to the eighth decoder bit (DEC<0:7>) of the decoder output data (DEC<0:7>) are <0> ~DEC <7> ) may have a logic level “1” in response to the counter value (CNT_S), and the remaining decoder bit bits except the one decoder bit may have a logic level “0”.

[0079] Each of the plurality of comparators 256-1 to 256-8 may convert the first decoder bit DEC of the decoder output data DEC<0:7> into a decoder bit DEC<0:7>. <0> To the 8th decoder bit DEC <7> Any one of the first decoder bits DEC is compared with the inverted pixel signal (PX_OUTb) and each of the memory enable signals ME1 to ME8 can be generated. Specifically, the comparator 256-1 can compare the first decoder bit DEC <0> The comparator 256-2 can compare the second decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate the memory enable signal ME1. <1> The comparator 256-3 can compare the third decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate the memory enable signal ME2. <2> The comparator 256-4 can compare the fourth decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate a memory enable signal ME3. <3> The comparator 256-5 can compare the 5th decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate the memory enable signal ME4. <4> The comparator 256-6 can compare the sixth decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate a memory enable signal ME5. <5> The comparator 256-7 can compare the seventh decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate the memory enable signal ME6. <6> The comparator 256-8 can compare the eighth decoder bit DEC with the inverted pixel signal (PX_OUTb) and generate the memory enable signal ME7. <7> The memory enable signal ME8 is generated by comparing the inverted pixel signal (PX_OUTb). In some embodiments, each of the plurality of comparators 256-1 to 256-8 can be implemented as a logic AND gate. In this case, when the first decoder bit DEC of the decoder output data DEC<0:7> is <0> To the eighth decoder bit DEC <7> When any one of the inverted pixel signal (PX_OUTb) has a logic level of 1, the corresponding memory enable signal (ME1-ME8) may have a logic level of 1. In addition, when the first decoder bit DEC of the decoder output data DEC<0:7> <0> To the eighth decoder bit DEC <7> When any one of the inverted pixel signal (PX_OUTb) and the first decoder bit DEC does not have a logic level of "1" (ie, when the inverted pixel signal (PX_OUTb) and the first decoder bit DEC <0> To the eighth decoder bit DEC <7> has a logic level of 0), the corresponding memory enable signal (ME1-ME8) may have a logic level of "0".

[0080] Since the first decoder bit DEC of the decoder output data DEC<0:7> <0> To the eighth decoder bit DEC <7> Each of the memory enable signals (ME1-ME8) indicates any one of the plurality of unit memories M1-M8, so the first decoder bit DEC <0> To the eighth decoder bit DEC <7> The memory enable signals ME1 to ME8 may be defined as memory addresses as needed.

[0081] Figure 5 Some embodiments based on the disclosed technology are shown Figure 4 A timing diagram of an example operation of a TDC memory is shown.

[0082] refer to Figure 5 , assuming that the pixel signal (PX_OUT) includes 8 SPAD pulses in one subframe, and the inverted pixel signal (PX_OUTb) may have a waveform opposite to that of the pixel signal (PX_OUT). In addition, the TDC enable signal (TDC_EN) may maintain a logic level of "1" during the subframe.

[0083] like Figure 5 As shown, the counter value (CNT_S) that increases by 1 at the rising edge of the inverted pixel signal (PX_OUTb) may include a first counter bit CNT_S <0> To the third counter bit CNT_S <2> , the first counter bit (CNT_S <0> ) can be the least significant bit (LSB), and the third counter bit (CNT_S <2> ) may be the most significant bit (MSB). Therefore, according to the first counter bit CNT_S <0> To the third counter bit CNT_S <2> The counter value (CNT_S) can represent values ​​from 0 to 7 in sequence.

[0084] The first decoder bit (DEC) of the decoder output data (DEC<0:7>) <0> ) may have a logic level of "1" in response to the counter value (CNT_S) "0", except for the first decoder bit (DEC <0> ) The remaining decoder bits except can have logic level "0".

[0085] The second decoder bit (DEC) of the decoder output data (DEC<0:7>) <1> ) may have a logic level of “1” in response to the counter value (CNT_S) “1”, except for the second decoder bit (DEC <1> ) The remaining decoder bits except can have logic level "0".

[0086] That is, in response to the (k+1)th decoder bit (DEC) of the decoder output data (DEC<0:7>) of k counter values ​​(CNT_S) (where “k” is any one of 0 to 7), <k>) and except for the (k+1)th decoder bit (DEC <k>) The remaining decoder bits except can have logic level "0".

[0087] The corresponding memory enable signal (ME1-ME8) can be obtained by connecting the inverted pixel signal (PX_OUTb) to the first decoder bit DEC <0> To the 8th decoder bit DEC <7> . Therefore, when TDC data (TDC_OUT) for one pulse of the pixel signal (PX_OUT) is stored in the memory block 240, only one unit memory among the unit memories (M1 to M8) may be activated so that the TDC data (TDC_OUT) may be stored, and the remaining unit memories may be deactivated. Therefore, the peak current that may occur when a plurality of unit memories (M1 to M8) are operated simultaneously may be significantly reduced, thereby reducing power consumption.

[0088] In addition, the eighth decoder bit (DEC) of the decoder output data (DEC<0:7>) <7> ) may be provided to the timing controller 140 and then may be converted to a logic level 1, so that the timing controller 140 may determine that the storage capacity of the TDC memory 230 is full and may allow the TDC enable signal (TDC_EN) to be converted to a deactivated level after a predetermined time has passed.

[0089] As described above, the TDC memory 230 may select any one unit memory from among the plurality of unit memories M1 to M8 according to the number of occurrences of the SPAD pulse, activate only the unit memory determined according to the number of occurrences of the SPAD pulse, and store TDC data (TDC_OUT).

[0090] Figure 6 is a method for showing some embodiments of the disclosed technology for Figure 1 A circuit diagram of another example of constituent elements for generating a candidate time of flight (TOF) in an image sensing device is shown.

[0091] refer to Figure 6 , the image sensing device 100 may include a component 200 for generating a candidate time of flight (TOF). The component 200 for generating a candidate time of flight (TOF) may include a pixel 310, a time-to-digital converter (TDC) 320, a TDC memory 330, and a single photon avalanche diode (SPAD) controller 340. According to one embodiment, the pixel 310 may be included in Figure 1 The pixel array 110 may include the TDC 320, the TDC memory 330, and the SPAD controller 340, and may be included in the readout circuit 130. According to another embodiment, at least one of the TDC 320, the TDC memory 330, and the SPAD controller 340 may be included in the pixel array 110.

[0092] Pixel 310 may mean Figure 1 Pixel 310 may include a SPAD, a quenching circuit (QX), a pixel inverter (IV_P), and a control transistor (CX). That is, compared to Figure 2 The pixel 210 of the present invention may further include a control transistor (CX), and the remaining components except the control transistor (CX) may be connected to the pixel 310. Figure 2 The structure and operation of the pixel 210 are substantially the same, and therefore, for the sake of brevity, redundant descriptions thereof will be omitted herein.

[0093] The control transistor (CX) may include a transistor configured to receive a SPAD control voltage (V c ) and can be connected between the sensing node (SN) and a ground terminal supplying a ground voltage (e.g., 0V).

[0094] The control transistor (CX) can respond to the SPAD control voltage (V c ) and is turned on so that the sensing node (SN) can be connected to the ground terminal. Since the sensing node (SN) connected to the ground terminal has a ground voltage, the amplitude of the reverse bias voltage applied to the SPAD can be reduced, so that the SPAD can operate in a linear mode instead of a Geiger mode. Therefore, the SPAD may not respond to single photons, so that power consumption caused by the SPAD avalanche process does not occur.

[0095] In addition, the control transistor (CX) can be turned off in response to the SPAD control voltage (Vc) having a logic level 0 (ie, an off voltage), so that the sensing node (SN) can be separated from the ground terminal, and the SPAD can be operated in the same manner as in FIG. Figure 2 The SPAD shown operates in the same manner.

[0096] The structure and operation of TDC 320 and TDC memory 330 may be similar to Figures 2 to 5 The structures and operations of the TDC 220 and the TDC memory 230 are substantially the same, and therefore, for the sake of brevity, redundant descriptions thereof will be omitted herein.

[0097] The SPAD controller 340 may generate a SPAD control voltage (V c Then, when the SPAD controller 340 detects the eighth decoder bit (DEC <7> ) rising edge, the SPAD control voltage (V c ) can be switched from 0 to 1, and the SPAD can be operated in linear mode. Thereafter, when the corresponding subframe ends, the SPAD controller 340 can allow the SPAD control voltage (V c ) is converted from "1" to "0", so that the sensing node (SN) can be separated from the ground terminal and the quenching control voltage (V qch ) may have a turn-on voltage. Therefore, the sensing node (SN) may have a second bias voltage (V ex ), so that the SPAD can enter Geiger mode.

[0098] That is, the SPAD controller 340 may generate a SPAD control voltage (V c More specifically, the SPAD controller 340 may be configured to detect the presence of a SPAD pulse based on the eighth decoder bit (DEC ) indicating that the SPAD pulse has occurred to fill the storage capacity of the TDC memory 330 . <7> ) to generate the SPAD control voltage (V c ).

[0099] Figure 7 Some embodiments based on the disclosed technology are shown Figure 6 A timing diagram of an example operation of a single photon avalanche diode (SPAD) is shown.

[0100] refer to Figure 7 , since single photons (ie, portions indicated by bars) are continuously included in the incident light (IL) within one subframe, the pixel signal (PX_OUT) can continuously generate SPAD pulses.

[0101] When the counter value (CNT_S) reaches a value of "7" while the TDC memory 330 stores TDC data (TDC_OUT) corresponding to the SPAD pulse, the TDC memory 330 may set the eighth decoder bit (DEC <7> ) is converted to a logic level “1”. Therefore, the timing controller 140 may convert the TDC enable signal TDC_EN to a deactivation level to deactivate the TDC 320, so that unnecessary power consumption of the TDC 320 and the TDC memory 330 may be prevented (ie, Figure 7 "Memory power saving" in the control panel).

[0102] In addition, when the eighth decoder bit (DEC <7> ) is converted to logic level "1", the SPAD controller 340 can c ) is switched from a logic level of "0" to a logic level of "1". The SPAD control voltage (V c ), the SPAD may enter a linear mode such that the SPAD may not react to a single photon of incident light (IL) (ie, the SPAD is turned off). Thus, power consumption in the pixel 310 caused by the avalanche process of the SPAD may be prevented (ie, Figure 7 Pixel Power Saving).

[0103] The image sensing device 100 according to an embodiment of the disclosed technology may reduce a peak current by minimizing the number of unit memories that operate when TDC data corresponding to a SPAD pulse is stored.

[0104] In addition, when the storage capacity of the TDC memory becomes full, the SPAD is turned off, so that the amount of power to be consumed in each pixel can be minimized.

[0105] The number of subframes per frame, the number of unit memories, the counter value, and the number of bits of decoder output data described in the embodiments of the disclosed technology are merely examples and may be changed within the scope to which the technical idea of ​​the disclosed technology can be applied.

[0106] As can be apparent from the above description, an image sensing device based on some embodiments of the disclosed technology and an imaging device including the image sensing device can reduce peak current by minimizing the number of unit memories configured to operate when time-to-digital converter (TDC) data corresponding to a single photon avalanche diode (SPAD) pulse is stored.

[0107] Embodiments of the disclosed technology can provide various effects that can be directly or indirectly identified through the above-mentioned patent documents.

[0108] Although a number of exemplary embodiments have been described, it should be appreciated that modifications and enhancements to the disclosed embodiments and other embodiments may be devised based upon what is described and / or illustrated in this patent document.< / k> < / k>

Claims

1. An image sensing device, comprising: A pixel that: generates a pixel signal with a SPAD pulse by detecting incident light, where SPAD refers to a single photon avalanche diode; A TDC that: generates TDC data representing the TOF of the SPAD pulses, TDC refers to time to digital converter, TOF refers to time of flight; and A TDC memory stores the TDC data in a unit memory determined from among a plurality of unit memories according to the number of occurrences of the SPAD pulse.

2. The image sensing device according to claim 1, wherein: The TOF is the time delay between a reference pulse of emission light irradiating a target object and the SPAD pulse.

3. The image sensing device according to claim 2, wherein: The TDC generates the TDC data by performing a counting operation from a time point when the reference pulse is generated to a time point when the SPAD pulse is generated.

4. The image sensing device according to claim 1, wherein: The TDC memory also includes: A memory control block controls whether to activate each of the plurality of unit memories.

5. The image sensing device according to claim 4, wherein: The memory control block deactivates remaining unit memories except the determined unit memory among the plurality of unit memories.

6. The image sensing device according to claim 4, wherein: The memory control block comprises: a memory inverter that: receives the pixel signal and generates an inverted pixel signal having a waveform opposite to that of the pixel signal; a counter that: increases a currently stored counter value at a rising edge of the inverted pixel signal; and A decoder, which generates decoder output data in response to the counter value, the decoder output data including any one decoder bit having a logic high level and the remaining decoder bits each having a logic low level.

7. The image sensing device according to claim 6, wherein: The decoder is (P×2 P ) decoder, which receives a counter value including P bits and generates a counter value including 2 P bits of decoder output data, where P is an integer of 1 or greater.

8. The image sensing device according to claim 6, wherein: The memory control block also includes: A plurality of comparators, each of the plurality of comparators compares the inverted pixel signal with each of the decoder bits to generate a memory enable signal.

9. The image sensing device according to claim 8, wherein: Each of the plurality of unit memories is activated in response to a rising edge of a corresponding memory enable signal received from a comparator corresponding to each of the plurality of unit memories and stores the TDC data.

10. The image sensing device according to claim 8, wherein: Each of the plurality of unit memories is a D flip-flop that receives a corresponding memory enable signal from a comparator corresponding to each of the plurality of unit memories through a clock terminal thereof.

11. The image sensing device according to claim 1, wherein: The pixels include: A SPAD that: generates a voltage pulse at a sensing node by detecting a single photon of the incident light in a Geiger mode; a quenching circuit that performs a quenching operation for returning the SPAD to Geiger mode; and A pixel inverter is provided for converting the voltage pulse into the SPAD pulse, wherein the SPAD pulse has a waveform opposite to that of the voltage pulse.

12. The image sensing device according to claim 11, in, One terminal of the SPAD receives a first bias voltage, wherein one terminal of the quenching circuit receives a second bias voltage, and Wherein, an absolute value of a value obtained by subtracting the second bias voltage from the first bias voltage is greater than an absolute value of a breakdown voltage of the SPAD.

13. The image sensing device according to claim 11, further comprising: A SPAD controller that: generates a SPAD control voltage based on the number of occurrences of the SPAD pulse; The pixel further includes a control transistor, and the control transistor supplies a ground voltage to the sensing node according to the SPAD control voltage.

14. The image sensing device according to claim 1, further comprising: a light source driver that drives the light source to output emission light having a predetermined waveform; a sensor driver that drives the pixels; as well as A timing controller controls the sensor driver, the TDC, and the TDC memory.

15. An image sensing device, comprising: a memory block including a plurality of unit memories; as well as A memory control block, which: controls the memory block to store TDC data representing TOF of a SPAD pulse, the SPAD pulse being stored in a unit memory, the unit memory being determined from among a plurality of unit memories according to the number of occurrences of the SPAD pulse generated by detecting incident light, SPAD referring to a single photon avalanche diode, TOF referring to time of flight, and TDC referring to a time to digital converter.

16. The image sensing device according to claim 15, wherein: The memory control block comprises: a memory inverter which: receives a pixel signal including the SPAD pulse and generates an inverted pixel signal having a waveform opposite to that of the pixel signal; a counter that: increases a currently stored counter value at a rising edge of the inverted pixel signal; and A decoder, which generates decoder output data in response to the counter value, the decoder output data including any one decoder bit having a logic high level and the remaining decoder bits each having a logic low level.

17. The image sensing device according to claim 16, wherein: The decoder is (P×2 P ) decoder, the decoder receives a counter value including P bits and generates a counter value including 2 P bits of decoder output data, where P is an integer of 1 or greater.

18. The image sensing device according to claim 16, wherein: The memory control block also includes: A plurality of comparators, each of the plurality of comparators compares the inverted pixel signal with each of the decoder bits to generate a memory enable signal.

19. The image sensing device according to claim 18, wherein: Each of the plurality of unit memories is activated in response to a rising edge of a corresponding memory enable signal received from a comparator corresponding to each of the plurality of unit memories and stores the TDC data.

20. An imaging device comprising: An image sensing device, which: stores TDC data representing TOF of a SPAD pulse, the SPAD pulse being stored in a unit memory, the unit memory being determined from among a plurality of unit memories according to the number of occurrences of the SPAD pulse generated by detecting incident light, SPAD refers to a single photon avalanche diode, TOF refers to time of flight, and TDC refers to a time to digital converter; and An image signal processor is configured to calculate a distance between a pixel generating the SPAD pulse and a target object based on the TDC data.

Images