Imager pixel
By adopting dynamic LSB and ratio reading systems in the imager, the difficulty in capturing visual features of low-level depth imagers under fluctuating light conditions is solved, and the expansion of the light condition range and the improvement of image classification accuracy is achieved, while reducing power consumption.
Patent Information
- Application Number
- CN202380057541.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-06-07
- Filing Date
- 2023-06-07
- Publication Date
- 2025-03-14
AI Technical Summary
Under fluctuating lighting conditions, low-level depth imagers are difficult to effectively capture visual features, resulting in a decrease in image classification accuracy, especially in scenarios where the actual dynamic range is much smaller than the full scale range.
The imager design with dynamic LSB is adopted, and the ratio reading of pixel current is realized through a ratio reading system and cluster-level pipeline technology. The LSB is dynamically adjusted to adapt to the light intensity of the scene, thereby expanding the range of light conditions.
The expansion of the light condition range at low-bit depth is achieved, the accuracy of image classification decreases from dark to bright remains small, and the imager can operate at a μW range of power, reducing power consumption.
Smart Images
Figure CN119631422A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates generally to imagers and, in particular but not exclusively, to pixels that facilitate dynamic LSB adaptation and radiometric readout. Background Art
[0002] Imagers with pure energy harvesting enable battery suppression for low-cost, miniaturized, and long-lived sensor nodes for distributed vision systems, where image sensing and cloud AI interpretation allow visual data to be captured and valuable insights to be generated. To reduce power, extremely low precision (e.g., down to 4 bits) is explored in AI accelerators. Similarly, low imager bit depths are being explored to close the gap. As a disadvantage, low bit depth N flattens the visual features of interest under fluctuating lighting conditions. If the actual dynamic range in the scene is much smaller than the full-scale range FS (dark scenes), most pixels are flattened into a few bins with low intensity, and classification becomes difficult at a relatively large least significant bit (LSB) = FS / 2N. Conversely, in scenes with a large actual dynamic range (bright scenes), a small and fixed LSB brings the opposite problem, i.e., misclassification again, because most pixels are instead saturated to the highest value ( Figure 1 ).
[0003] It would be desirable to overcome or ameliorate at least one of the problems described above, or at least provide a useful alternative. Summary of the invention
[0004] This paper describes a novel imager design with μW peak power for acquisition-only operation. The LSB is dynamically adapted to the scene's light intensity to achieve a large bit-depth reduction, avoiding the traditional over-marginalization of the dynamic range across actual light intensities at fixed LSBs. Ratiometric readout of pixel current eliminates threshold voltage mismatches. A 256 × 256 pixel 180-nm imager exhibits 5-μW power at 1fps and 4 bits while achieving ImageNet classification accuracy down to the percentile over a 75-dB ambient light range on both raw and brightness-adjusted images.
[0005] An imager with dynamic LSB is proposed to achieve low-bit depth sensing and extend the range of light conditions (75-dB range, Figure 1 ), over the range of light conditions, the drop in accuracy of image classification from dark to light remains small.
[0006] Imager pixels are disclosed, including:
[0007] A forward biased photodetector having a readout transistor for light-to-current conversion;
[0008] Ratiometric reading systems, including current to pulse width converters, are used for:
[0009] Operate in light mode to:
[0010] generating a first current proportional to the photodetector current and the leakage current when exposed to the photodetector voltage; and
[0011] converting the first current into a first pulse width; and
[0012] Operate in Least Significant Bit (LSB) mode to:
[0013] generating a second current proportional to a least significant bit (LSB) current and a drain current when exposed to a gate-source voltage; and
[0014] converting the second current into a second pulse width;
[0015] a converter circuit for converting the first pulse width into counts (CNTB) and converting the second pulse width into counts (CNTA); and
[0016] A counting circuit is provided for providing a radiometric reading based on the ratio of CNTB to CNTA.
[0017] Also disclosed is a system comprising:
[0018] a plurality of rows, each row comprising a plurality of pixels as described above, wherein the plurality of rows are separated into a plurality of clusters;
[0019] Row controller, for frames:
[0020] sequentially operating the pixels in the cluster in an LSB mode, a light mode, and an average mode, and then idling the pixels until the next frame; and
[0021] operating in an averaging mode by averaging pixel values within a frame to produce an average pixel value; and
[0022] A capacitor is provided for each cluster, the capacitor being used to store a voltage corresponding to an average pixel value of the cluster.
[0023] Advantageously, a forward biased photodetector with a subthreshold readout transistor can be provided. This enables a nearly linear light-to-current conversion.
[0024] Advantageously, embodiments provide ratiometric readout with dual sensing modes. This mitigates threshold voltage mismatches - for example, on MPDs.
[0025] Advantageously, embodiments provide cluster-level pipelining. This eliminates the timing overhead due to dual sensing mode switching. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Embodiments of the invention will now be described by way of non-limiting examples with reference to the accompanying drawings, in which:
[0027] Figure 1 The limitations of conventional fixed least significant bit (LSB) imagers at low bit depths are shown;
[0028] Figure 2 Schematically illustrates a dual sensing mode 12T pixel architecture and a ratiometric readout scheme;
[0029] Figure 3 shows the proposed architecture of low bit depth imager with dynamic LSB;
[0030] Figure 4 It is the timing diagram of the cluster-level pipeline;
[0031] Figure 5 shows the voltage threshold (VTH) mismatch plot measured from CNTB / CNTA (uniform plot, with and without ratiometric readout);
[0032] Figure 6 The measured LSB pulse width (PWLSB) adaptation for light intensity (>75dB) is shown;
[0033] Figure 7 is an example of under-acquisition: the first image measured is misclassified with reduced contrast brightness; and
[0034] Figure 8 It shows that the classification accuracy of dynamic LSB is improved compared with fixed LSB. DETAILED DESCRIPTION
[0035] This paper describes a Least Significant Bit (LSB) imager that enables low bit depth sensing and extends the range of light conditions in which the accuracy drop in image classification from dark to bright remains small. The LSB imager also enables acquisition-only systems to operate in the μW range.
[0036] refer to Figure 2 , an imager pixel 200 is provided. The pixel 200 includes a forward biased photodetector 202 having a subthreshold readout transistor 204 for light to current conversion. The pixel 200 also includes a ratiometric readout system 206 including a current to pulse width converter. The imager pixel operates in a light sensing mode and an LSB sensing mode.
[0037] For the purposes of this discussion, "light sensing mode," "light mode," and similar modes refer to when the photodetector is exposed to a photodetector voltage, and "LSB sensing mode," "LSB mode," and similar modes refer to when the photodetector is exposed to a gate-source voltage that is obtained from a non-mismatched pixel exposed to 1-LSB light intensity.
[0038] In the light sensing mode, the MPD is exposed to the photodetector voltage. When exposed to the photodetector voltage, the MPD generates a first current (IMPD) proportional to the photodetector current (IPD) and the leakage current (Ilkg). The MPD then generates a first current (IMPD) proportional to the photodetector current (IPD) and the leakage current (Ilkg). Figure 3 ) is compared within the pixel to convert the first current into a first pulse width PWlight. Then, PWlight is converted into a count CNTB based on the system clock.
[0039] In LSB sensing mode, MPD is exposed to the gate-source voltage (V GS,LSB This generates the same LSB current (I LSB ) and leakage current (I lkg ) is proportional to the second current (I MPD ). GS,LSB Produced by non-mismatched (ie, threshold voltage non-mismatched) pixels exposed to 1-LSB light intensity, non-mismatched pixels exposed to 1-LSB light intensity are derived as true fractions of the average light level within the frame. As described above, the current IMPD is converted to pulse width PWLSB and then to counts CNTA.
[0040] The pixel 200 further includes a converter circuit 210, which converts the first pulse width PW light Convert to count (CNTB), and convert the second pulse width PW LSB Pixel 200 also includes counting circuitry for providing a radiometric reading based on the ratio of CNTB to CNTA. Thus, the ratio PW light / PW LSB =CNTB / CNTA is independent of leakage and hence threshold voltage mismatch. Figure 2, without a digital divider, the ratio is evaluated digitally by using the count CNTA in LSB mode as the modulus of CNTB in light mode, and incrementing a third counter CNTC only at each CNTA cycle while counter B keeps incrementing. Thus, the final value of counter C (CNTC) is CNTB / CNTA, thereby achieving a ratiometric reading.
[0041] Also disclosed herein are systems comprising multiple rows, wherein each row comprises multiple pixels, such as imager pixel 200 as described above (i.e., multiple rows of pixels), wherein the multiple rows are separated into multiple clusters. Rows can be clustered by proximity—for example, every N adjacent rows can form a cluster. Alternatively, pixels in a row of pixels can be clustered together based on proximity—for example, a pixel and eight pixels surrounding the pixel (if the pixels are in a regular grid pattern) form a cluster—and thus any row can include pixels from multiple clusters.
[0042] The system also includes a row controller that sequentially operates pixels in the cluster in an LSB mode, a light mode, and an average mode, and then idles the pixels until the next frame. The row controller is based on the V avg The resulting average pixel value is dynamically scaled by the LSB. Figure 4 , the cluster-level pipeline masks the timing overhead of the LSB to light mode switch. The row controller also operates in an averaging mode by averaging the pixel values within a frame to produce an average pixel value. In cluster-level pixel averaging, the photodetectors are short-circuited by the switch LSBSEL. The accumulated current of the photodetectors adds the aggregate photodetector voltage V cluster is set to the value of a single photodetector, which has a current equal to the average value of the photodetectors. Then, V cluster is stored in capacitor C cluster When accessed, capacitor C is gradually reallocated with the capacitors of subsequent clusters (i.e., clusters subsequently used in cluster-level averaging). cluster The charge on the cluster is thus averaged over V cluster ( Figure 3 ). At the end of the frame, the voltage V AVG is the average open circuit voltage of all photodetectors on the array.The system also includes a capacitor for each cluster for storing a voltage corresponding to the average pixel value of the cluster.
[0043] Finally, V AVG is applied to the appropriately sized pixel transistor replica MPD to generate the average pixel current I AVG The LSB adaptive loop sets the I LSB =I AVG / 8. The LSB adaptive loop also generates the corresponding V GS,LSB Furthermore, the LSB adaptive loop can scale the ramp full-scale value to 2I AVG , to I AVG Keep at half the dynamic range. Frame end V GS,LSB Update V AVG The current is transferred from the storage capacitor to the switched current mirror ( Figure 3 ).
[0044] Measurements on a 180-nm test chip show that ratiometric readout reduces the standard deviation of the digital output caused by mismatches across pixels by a factor of 11 ( Figure 6 ). LSB current I LSB (exist Figure 7 PW LSB The 4-bit LSB (measured) linearly tracks scene light intensity over a >75 dB range of ambient light (averaged from 15 lux to 60,000 lux within the frame), significantly exceeding the fixed LSB maximum of 24 dB at 4-bit resolution. 2 Solar cells, Figure 8 ), compared with the conventional fixed LSB, the dynamic LSB capture maintains the correct MobileNetV2 neural network classification at lower brightness. Iterated on the randomly sampled ImageNet dataset, the dynamic LSB enables up to 24% higher classification accuracy than the fixed LSB in dark scenes (Figure 9).
[0045] Table I provides a performance comparison with existing imager technology. The proposed imager with dynamic LSB enables operation in the μW range for pure acquisition operation, thanks to the lowest 0.08nW / pixel power, which is 3.75-47.5 times lower than the known pixel power with the lowest bit depth and three orders of magnitude lower than higher bit depth imagers. While maintaining a linear mapping, the >75-dB ambient light range is substantially higher than the logarithmic imager by 15.7dB without the need for imager-specific processing as in the 16×10 array (ISSCC'18) at 8-10 bits.
[0046] Table I. Summary of properties (best or unique properties are in bold)
[0047]
[0048]
[0049] It will be apparent that many further modifications and permutations of various aspects of the described embodiments are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.
[0050] In this specification and the claims that follow, unless the context requires otherwise, the word "comprise" and variations such as "comprises" and "comprising", will be understood to imply the inclusion of stated integers or steps or groups of integers or steps but not the exclusion of any other integers or steps or groups of integers or steps.
[0051] Reference in this specification to any prior publication (or information derived from any prior publication), or reference to any known matter, is not and should not be taken as an acknowledgement or acknowledgment or any form of suggestion that the prior publication (or information derived from any prior publication) or known matter forms part of the common general knowledge in the field to which this specification is intended to relate.
Claims
1. An imager pixel, comprising: a forward biased photodetector having a readout transistor for light-to-current conversion; A ratiometric readout system, including a current to pulse width converter, the ratiometric readout system being used to: Operate in light mode to: generating a first current proportional to the photodetector current and the leakage current when exposed to the photodetector voltage; as well as converting the first current into a first pulse width; as well as Operate in LSB mode to: generating a second current proportional to the least significant bit (LSB) current and the drain current when exposed to a gate-source voltage; as well as converting the second current into a second pulse width; a converter circuit for converting the first pulse width into a count CNTB and converting the second pulse width into a count CNTA; as well as A counting circuit is provided for providing a radiometric reading based on the ratio of CNTB to CNTA.
2. The pixel according to claim 1, wherein: The read transistor is a subthreshold read transistor.
3. The pixel according to claim 1 or 2, wherein: The first pulse width and the second pulse width are proportional pulse widths generated by in-pixel comparison with a ramp current.
4. The pixel according to any one of claims 1 to 3, wherein: The gate-source voltage corresponds to a non-mismatched pixel exposed to 1-LSB intensity light.
5. The pixel according to claim 4, wherein: The non-mismatched pixels exposed to the 1-LSB intensity of light are derived as true fractions of the average light level within the frame.
6. The pixel according to any one of claims 1 to 5, wherein: The ratio is determined by using the CNTA in the LSB pattern as a modulus of the CNTB in the optical pattern and incrementing a counter C every CNTA cycle, with the CNTB being incremented during every CNTA cycle.
7. A system comprising: a plurality of rows, each row comprising a plurality of pixels according to any one of claims 1 to 6, wherein the plurality of rows are separated into a plurality of clusters; Row controller, for frames: sequentially operating the pixels in the cluster in an LSB mode, a light mode, and an average mode, and then idling the pixels until a next frame; and operating in an averaging mode by averaging pixel values within the frame to produce an average pixel value; and A capacitor is provided for each cluster, the capacitor being used to store a voltage corresponding to the average pixel value of the cluster.
8. The system according to claim 7, wherein: The row controller dynamically scales the LSBs based on the average pixel value.
9. The system according to claim 7 or 8, wherein: The row controller averages the pixel values within the frame by: performing a first averaging process by averaging pixel values within each cluster to generate a cluster-level average pixel value, and storing a voltage corresponding to the cluster-level average pixel value in a capacitor corresponding to each cluster; as well as A second averaging process is performed by stepwise averaging the cluster-level average pixel values over the clusters.
10. The system according to claim 9, wherein: The row controller short-circuits the photodetector of each pixel in each cluster involved in the second averaging process.
11. The system according to claim 9 or 10, wherein: The second averaging process involves, in a stepwise averaging case, stepwise averaging the cluster-level average pixel values over the clusters by stepwise redistributing the voltage from each cluster over the capacitors of subsequent clusters.
12. A system according to any one of claims 7 to 11, wherein: The row controller generates the average open circuit voltage V of all photodetectors in the array based on the voltage avg , each voltage corresponds to the average pixel value of each of the clusters.
13. A system according to any one of claims 7 to 12, wherein: The row controller: From V avg generating an average pixel current value; as well as Based on the average pixel current value, an LSB current is set, a gate-source voltage is generated, and a ramp current is scaled.
14. The system according to claim 13, wherein: After processing the frame, the gate-source voltage will be V avg Transmitted to the switched current mirror to process the next frame.