Image sensor with multiple superpixels
Through the combination of superpixel technology and local ADC, the time redundancy problem in image sensors is solved, efficient data acquisition and low power consumption are achieved, and suitable for high-speed visual applications.
Patent Information
- Application Number
- CN202411073205.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2018-12-17
- Filing Date
- 2019-06-27
- Publication Date
- 2024-11-26
AI Technical Summary
Existing image sensors have time redundancy problems when acquiring scene visual information, resulting in increased data volume, increased hardware complexity and power consumption, and existing compression technologies are unable to provide efficient video output in real time.
Using superpixel technology, automatic sampling through condition detectors, independent exposure measurement and analog to digital conversion, reduce redundant data, improve exposure measurement rate for each pixel, and include local ADC in each superpixel to directly output digital data .
Significantly reduces data volume, improves image sensor efficiency, allows for higher exposure measurement rates and higher information content, while reducing power consumption and post-processing load.
Smart Images

Figure CN119031259A_ABST
Abstract
Description
[0001] Divisional application statement
[0002] This application is a divisional application of the Chinese invention patent application with application number 201980056827.5 and invention name “Image sensor with multiple superpixels”, which claims priority to U.S. patent application No. 62 / 690,948 filed on June 27, 2018 and U.S. patent application No. 62 / 780,913 filed on December 17, 2018, and PCT international application No. PCT / EP2019 / 067264 filed on December 17, 2018, and entered China on February 26, 2021. Technical Field
[0003] The present disclosure relates generally to the field of image sensing. More specifically and without limitation, the present disclosure relates to systems and methods for providing superpixels and implementing and using image sensors having superpixels. The image sensors and techniques disclosed herein can be used in a variety of applications and vision systems, such as security systems, autonomous vehicles, and other systems that benefit from fast and efficient motion detection and motion event-driven image data acquisition. Background Art
[0004] Existing image sensors use multiple pixels, including semiconductor charge-coupled devices (CCDs), complementary metal-oxide semiconductor (CMOS) sensors, N-type metal-oxide-semiconductor (NMOS) sensors, or other sensors, to capture digital images of scenes.
[0005] These conventional image sensors can acquire temporally quantized visual information at a predetermined frame rate. Therefore, each typical frame carries information from all pixels, regardless of whether the information has changed since the previous frame was acquired. Depending on the dynamic content of the scene, this approach usually leads to a more or less high degree of redundancy in the recorded image data, because although all unchanged pixel values are known, all of these unchanged pixel values are also re-acquired and re-sent each time. In conventional image sensors, these pixels usually cannot be sure that the values they measure have not changed. As modern image sensors develop to higher and higher spatial and temporal resolutions, this problem tends to worsen. Moreover, the hardware required for data post-processing usually increases in complexity and cost, the demand for transmission bandwidth and data storage capacity also surges, and power consumption also increases, leading to limitations in many vision applications from high-speed industrial vision systems to mobile, battery-powered consumer devices.
[0006] One existing method for reducing temporal redundancy in video data is inter-frame difference coding. This form of video compression typically involves transmitting, frame by frame, only pixel values that exceed a defined intensity change threshold after an initial keyframe. Existing inter-frame difference systems typically rely on the acquisition and processing of complete frames of image data and are unable to self-consistently suppress temporal redundancy and provide real-time compressed video output from the sensor.
[0007] Indeed, many vision applications do not require a complete image, but benefit from fast image data from the portion of the scene where changes or motion are detected. For example, a security system or other vision system may be interested only in motion data, and not in the portion of the image without motion. In another example, an autonomous vehicle must process captured data quickly and efficiently in order to make decisions comparable to human perception times (typically in milliseconds). This efficiency is limited when large amounts of data must be discarded (e.g., via post-processing) in order to obtain the portion of captured data that is relevant to the situation.
[0008] Other existing approaches include time-based asynchronous image sensors, whose pixels generate detections in response to specific conditions determined by detectors within the pixel. However, such pixels still output analog signals or time-based signals derived from such analog signals. When high activity occurs in the scene, these output signals are prone to losing image quality and must be post-processed into digital data for practical use. Accordingly, the post-processing cost increases significantly as the spatial and temporal resolution increases. Summary of the invention
[0009] Embodiments of the present disclosure may provide a method and apparatus for continuously acquiring visual information of an observed scene at a high temporal resolution, thereby generating a smaller amount of data by reducing redundancy. Therefore, the data generated does not include continuous frames containing image information of all pixels, but may include a stream of changes and light intensity information of each pixel, which are recorded and transmitted when a change in light intensity in the field of view of the pixel exceeds a threshold value, which has been detected by the individual pixel itself. Accordingly, embodiments of the present disclosure may significantly reduce the data generated by suppressing temporal redundancy in picture information typical of conventional image sensors, while still providing data containing comparable, or even higher, information content.
[0010] Since the data output from the pixel array is reduced by suppressing redundant data as described above, it is possible to increase the rate of exposure measurement at each pixel, for example from 30 times per second or 60 times per second (the frame rate of a typical CMOS image sensor) to a higher rate of, for example, 1000 times per second, 10,000 times per second, and above 10,000 times per second, etc., the rate for each pixel being independently dependent on the activity in the field of view of the respective pixel detected by the pixel's state detector (as described below) that can detect conditions at such a rate. Conditional readout and analog-to-digital (AD) conversion of exposure measurements from each pixel at such a rate is a challenging task that is difficult to address using standard techniques in which the AD conversion is performed external to the pixel array (either at the column level or at the sensor level).
[0011] Embodiments of the present disclosure may use pixel-independent automatic sampling (self-triggering) superpixels. As used herein, automatic sampling may refer to each independent pixel autonomously deciding when to acquire image information. For example, a pixel may be equipped with a (condition) detector (Condition Detector, CD) that detects the condition of the input light signal of the pixel individually and independently of the detectors of other pixels (e.g., a specific size of intensity change) and initiates exposure measurement in the exposure measurement (Exposure Measurement, EM) component of the pixel. This functionality may allow for pixel-independent measurement and / or communication of new light exposure levels when the light impinging on the photosensitive element (e.g., photodiode) of the pixel meets defined criteria (e.g., due to a change that exceeds a threshold since the previous measurement, the event of the change is detected by the pixel itself (e.g., by its (condition) detector)).
[0012] In order to benefit from the latest advances in photodiode and CMOS pixel technology (e.g., using pinned photodiode (PPD) pixels, global shutter pixels, etc.), the EM component of the superpixel of the embodiments of the present disclosure may be based on a standard (4T, 5T) active pixel sensor (APS) pixel, and may not share a photodiode with a CD. For example, one possible arrangement is to combine a CD with an EM into a "superpixel".
[0013] The results of the exposure measurements of the EM components typically need to be converted to digital form at some point. A flexible solution is to include a complete ADC within each superpixel. This arrangement with a local ADC in each pixel can avoid having to drive long columns of readout lines with analog signals, which typically takes time and consumes power. In addition, the pixel-level ADC can allow multiple exposures in a single shot for High-Definition Resolution (HDR) and / or Red / Green / Blue (RGB) color acquisition, as further described below.
[0014] In view of the foregoing, embodiments of the present disclosure provide superpixels for use in image sensors. As disclosed herein, superpixels may include an analog-to-digital converter to directly output digital data or measurements. Accordingly, the amount of post-processing of the signal can be significantly reduced, thereby reducing the amount of data and improving the efficiency of the sensor. In addition, in order to achieve this conversion without exponentially increasing power consumption, the converter can be power cycled by a condition detector within each superpixel. This allows the converter to be actually implemented in each superpixel. In addition, the analog-to-digital converter of each superpixel can be triggered by an external control signal. Accordingly, readout values can be pulled for areas of interest and on demand rather than based on detected triggers or conditions.
[0015] In some embodiments, superpixels are provided that include multiple exposure measurement circuits paired with a single condition detector. By arranging superpixels in this manner, high resolution and / or color images can be detected and read out from an image sensor incorporating such superpixels. The image is read out directly as digital data, thereby reducing the need for analog-to-digital conversion by auxiliary circuitry.
[0016] According to an exemplary embodiment of the present disclosure, an image sensor is provided, which includes a plurality of superpixels. Each superpixel may include a first photosensitive element and a detector, the detector being electrically connected to the first photosensitive element and configured to generate a trigger signal when an analog signal proportional to the brightness of light incident on the first photosensitive element matches a condition. The superpixel may also include a second photosensitive element and an exposure measurement circuit, the exposure measurement circuit being electrically connected to the second photosensitive element and configured to convert the analog signal proportional to the brightness of light incident on the second photosensitive element into a digital signal. Further, the superpixel may include a logic circuit, the logic circuit being electrically connected to the detector and the exposure measurement circuit, and configured to enable the exposure measurement circuit (or a component of the exposure measurement circuit) in response to the trigger signal, and disable the exposure measurement circuit (or a component of the exposure measurement circuit) when the digital signal is read out from the exposure measurement circuit.
[0017] In some embodiments, the exposure measurement circuit may include an analog-to-digital circuit configured to convert the analog signal proportional to the brightness of the light incident on the second photosensitive element into the digital signal. For example, the analog-to-digital circuit may include a digital ramp analog-to-digital circuit. In such an embodiment, the analog-to-digital circuit may include a single slope ramp analog-to-digital circuit.
[0018] In any of the above embodiments, the analog-to-digital circuit may be further configured to output the digital signal to an external readout system. In such an embodiment, the analog-to-digital circuit may be further configured to be disabled after the output. For example, the analog-to-digital circuit may be further configured to send a signal in response to the output to reset the logic circuit so that the analog-to-digital circuit is disabled.
[0019] In any of the above embodiments, the external readout system may request the output from the analog-to-digital circuit based on a clock circuit.
[0020] In any of the above embodiments, the analog-to-digital conversion circuit may also be configured to send a signal to reset the logic circuit after it has received a signal from the logic circuit to initiate conversion of the analog signal to a digital signal. Additionally or alternatively, the logic circuit may also be configured to enable the exposure measurement circuit (or its components) in response to an external control signal.
[0021] In any of the above embodiments, the detector may be further configured to output the trigger signal to an external readout system. In such an embodiment, the external readout system may be configured to send a response signal to the detector in response to the trigger signal, and the detector may be further configured to reset in response to the response signal.
[0022] In any of the above embodiments, the logic circuit may include control logic electrically connected to the detector and a switch controlled by the control logic. In any of the above embodiments, the control logic may also be electrically connected to an analog-to-digital converter of the exposure measurement circuit.
[0023] In any of the above embodiments, the logic circuit may include control logic electrically connected to the detector, and logic gates connected to the control logic and to an external controller. In any of the above embodiments, the control logic may also be electrically connected to an analog-to-digital converter of the exposure measurement circuit.
[0024] In any of the above embodiments, the external readout system may request an output from the exposure measurement circuit based on a circuit clock, and the exposure measurement circuit may be configured to output a digital signal in response to the request. For example, the digital signal may indicate the time that the detector has not been triggered since a previous request. In such an embodiment, upon receiving a threshold number of digital signals indicating that the detector has not been triggered, the external readout system may send at least one control signal to cause the exposure measurement circuit to convert an analog signal proportional to the brightness of light incident on the second photosensitive element into digital data.
[0025] According to another exemplary embodiment of the present disclosure, an image sensor is provided, the image sensor comprising a plurality of super pixels. Each super pixel may include: a first photosensitive element; a detector electrically connected to the first photosensitive element and configured to generate a trigger signal when an analog signal proportional to the brightness of light incident on the first photosensitive element matches a condition; a plurality of second photosensitive elements; and at least one exposure measurement circuit electrically connected to the plurality of second photosensitive elements and configured to convert an analog signal proportional to the brightness of light incident on one or more of the plurality of second photosensitive elements into a digital signal in response to the trigger signal.
[0026] In some embodiments, the plurality of second photosensitive elements may output respective analog signals to separate capacitor nodes of the exposure measurement circuit. In such embodiments, at least two of the plurality of second photosensitive elements may have different gains.
[0027] Additionally or alternatively, the plurality of second photosensitive elements may integrate charge onto respective nodes at different times. In such an embodiment, the plurality of second photosensitive elements may output respective analog signals to the same capacitor node of the exposure measurement circuit.
[0028] In any of the above embodiments, at least two of the plurality of second photosensitive elements may include different optical filters (e.g., density filters or polarization filters, etc.). Additionally or alternatively, at least two of the plurality of second photosensitive elements may include different color filters (or other wavelength filters, such as infrared cut-off filters or infrared pass filters, etc.).
[0029] In any of the above embodiments, the at least one exposure measurement circuit may include an analog-to-digital circuit configured to convert the analog signal proportional to the brightness of the light incident on one or more of the plurality of second photosensitive elements into the digital signal. In such an embodiment, the analog-to-digital circuit may include a digital ramp analog-to-digital circuit, such as a single slope ramp analog-to-digital circuit. Additionally or alternatively, the analog-to-digital circuit may include at least one comparator. In such an embodiment, the comparator may include a first transistor of the analog-to-digital circuit, the first transistor coupled to at least one transistor associated with each of the plurality of second photosensitive elements. Such an embodiment may also include at least one reset transistor associated with each of the plurality of second photosensitive elements, and the at least one exposure measurement circuit may also be configured to output a reset voltage of each reset transistor before outputting the digital signal. For example, the at least one exposure measurement circuit may sequentially output the reset voltage of each reset transistor. Additionally or alternatively, the readout circuit may be configured to perform Correlated Double Sampling (CDS) correction on noise and mismatch on the digital signal using the output reset voltage.
[0030] According to another exemplary embodiment of the present disclosure, a superpixel for use in an image sensor is provided. The superpixel may include an asynchronous unit, the asynchronous unit including a first photosensitive element and a detector, the detector being electrically connected to the first photosensitive element and configured to generate a trigger signal when an analog signal proportional to the brightness of light incident on the first photosensitive element matches a condition. The superpixel may also include a synchronization unit, the synchronization unit including at least one second photosensitive element and an exposure measurement circuit, the exposure measurement circuit being electrically connected to the at least one second photosensitive element and configured to convert the analog signal proportional to the brightness of light incident on the at least one second photosensitive element into a digital signal. The superpixel may also include a logic circuit, the logic circuit being electrically connected to the asynchronous unit and configured to enable the synchronization unit in response to the trigger signal, and disable the synchronization unit when a digital signal is read out from the synchronization unit.
[0031] According to another exemplary embodiment of the present disclosure, a method for controlling an image sensor having a plurality of superpixels is provided. The method may include: receiving a first analog signal proportional to light incident on a first photosensitive element; generating a trigger signal when the first analog signal matches a condition; enabling an exposure measurement circuit in response to the trigger signal; receiving at least one second analog signal proportional to light incident on at least one second photosensitive element; using the exposure measurement circuit, converting the at least one second analog signal into a digital signal; disabling the exposure measurement circuit after converting the at least one second analog signal into the digital signal; and outputting the digital signal to an external readout system.
[0032] In some embodiments, the conversion may be performed using an analog-to-digital circuit included in the exposure measurement circuit. In such embodiments, the analog-to-digital circuit may include a digital ramp analog-to-digital circuit. For example, the analog-to-digital circuit may include a single slope ramp analog-to-digital circuit.
[0033] In some embodiments, the method may further include: outputting the digital signal to an external readout system using the analog-to-digital conversion circuit. In such embodiments, the method may further include: disabling the analog-to-digital conversion circuit after the output.
[0034] In any of the above embodiments, the method may further include: receiving a request for an output from the external readout system based on a clock circuit.
[0035] In any of the above embodiments, the method may further include: using the analog-to-digital circuit to send a signal in response to the output to reset the logic circuit so that the analog-to-digital circuit is disabled. Additionally or alternatively, the method may further include: after completing the conversion of the analog signal to the digital signal, using the analog-to-digital circuit to send a signal to reset the logic circuit.
[0036] In any of the above embodiments, the method may further include: using the analog-to-digital conversion circuit to enable the exposure measurement circuit in response to an external control signal.
[0037] In any of the above embodiments, the method may further include: using the detector to output the trigger signal to an external readout system. In such an embodiment, the method may further include: using the detector to receive a response signal from the external readout system in response to the trigger signal; and resetting the detector in response to the response signal.
[0038] In any of the above embodiments, the logic circuit may include control logic electrically connected to the detector and a switch controlled by the control logic. In any of the above embodiments, the control logic may also be electrically connected to an analog-to-digital converter of the exposure measurement circuit.
[0039] In any of the above embodiments, the logic circuit may include control logic electrically connected to the detector and logic gates connected to the control logic and to an external controller. In any of the above embodiments, the control logic may also be electrically connected to an analog-to-digital converter of the exposure measurement circuit.
[0040] In such an embodiment, the method may further include receiving a request for the exposure measurement circuit output according to a circuit clock from an external readout system, and outputting a digital signal in response to the request. For example, the digital signal may indicate the time that the detector has not been triggered since the last request. In such an embodiment, the method may further include receiving at least one control signal from the external readout system to cause the exposure measurement circuit to convert an analog signal proportional to the brightness of light incident on the second photosensitive element into digital data when a threshold number of digital signals indicating that the detector has not been triggered are received.
[0041] According to another exemplary embodiment of the present disclosure, an image sensor is provided, the image sensor including a plurality of superpixels. Each superpixel may include a first photosensitive element and a detector, the detector being electrically connected to the first photosensitive element and configured to generate a trigger signal when a first analog signal proportional to the brightness of light incident on the first photosensitive element matches a condition. Each superpixel may also include a second photosensitive element and an exposure measurement circuit, the exposure measurement circuit being electrically connected to the second photosensitive element and configured to convert a second analog signal proportional to the brightness of light incident on the second photosensitive element into a digital signal. The superpixel may also include at least one latch, the at least one latch being configured to be enabled during the conversion of the second analog signal to the digital signal when the value of a third analog signal proportional to the brightness of light incident on the first photosensitive element meets the condition. The exposure measurement circuit may be configured to convert the second analog signal in a first cycle in response to the trigger signal, and convert a fourth analog signal proportional to the brightness of light incident on the second photosensitive element into a digital signal in a second cycle in response to the enabling of the at least one latch. In some embodiments, the exposure measurement circuit may include an analog-to-digital circuit configured to convert the second analog signal and the fourth analog signal proportional to the brightness of the light incident on the second photosensitive element into the digital signal.
[0042] In such an embodiment, the at least one latch may include a logic circuit having a first latch and a second latch, the first latch being configured to activate the first cycle of the analog-to-digital circuit in response to the trigger signal, the second latch being configured to activate and clear in response to a signal from the analog-to-digital circuit; and the analog-to-digital circuit may be configured to set the second latch and clear the first latch when starting the first cycle. Furthermore, when the second latch is set, an external readout system may utilize the analog-to-digital circuit to initiate a readout. Additionally or alternatively, the analog-to-digital circuit may be configured to start the second cycle in response to activation of the first latch during the first cycle. Additionally or alternatively, the analog-to-digital circuit may also be configured to be disabled after the first cycle when the first latch is not activated.
[0043] In some embodiments, the at least one latch may be configured to be activated in response to the trigger signal and cleared in response to a signal from the analog-to-digital circuit; the analog-to-digital circuit may be configured to poll the at least one latch at the beginning of each cycle to determine whether to start conversion; and the analog-to-digital circuit may be configured to set an internal latch and clear the at least one latch when starting conversion. In such an embodiment, when the internal latch is set, an external readout system may utilize the analog-to-digital circuit to initiate readout. Additionally or alternatively, the internal latch may include a single latch of a plurality of latches used in the conversion. Additionally or alternatively, the analog-to-digital circuit may also be configured to be disabled during a cycle, at the beginning of which the at least one latch is not activated.
[0044] In some embodiments, the at least one latch may be external to the first photosensitive element, the detector, the second photosensitive element, the exposure measurement circuit, and the logic circuit. In such an embodiment, the at least one latch for each superpixel may form a memory array external to the plurality of superpixels. In such an embodiment, the at least one latch is connected to the exposure measurement circuit using at least one of: a direct wiring connection, or one or more arbiters that associate the memory address of the at least one latch with the address of the corresponding superpixel. Additionally or alternatively, the at least one latch may be configured to be activated in response to the trigger signal and to be cleared in response to a signal from an external reader; the analog-to-digital circuit may be configured to perform a conversion in each cycle; and the external reader may be configured to read a conversion from the analog-to-digital circuit only when the at least one latch is activated.
[0045] In any of the above embodiments, the at least one latch may include at least one bit of a static random access memory (SRAM).
[0046] According to another exemplary embodiment of the present disclosure, a readout system for an image sensor is provided. The readout system may include: a first array of measurement readers configured to output digital signals; and a second array of latches associated with the measurement readers and configured to be activated when a trigger signal is received from the associated measurement readers. The readout system may also include a clock circuit configured to clock one or more measurement readers in the first array of measurement readers based on the activated latches in the second array. The second array of latches may be configured to be deactivated when being clocked.
[0047] In some embodiments, the readout system may further include a plurality of switches configured to connect the clock circuit to the measurement readout of the first array when the corresponding latch is activated. Additionally or alternatively, the second array may be external to the first array. For example, the second array includes a static random access memory (SRAM) array of one bit or larger units.
[0048] In any of the above embodiments, the second array may be formed on the same chip as the first array. Additionally or alternatively, the clock circuit may be further configured to skip a clock cycle when no latch in the second array is activated.
[0049] Other purposes and advantages of the present disclosure will be set forth in part in the following detailed description, and in part will become apparent from the description, or may be understood through the practice of the present disclosure. The purposes and advantages of the present disclosure will be realized and obtained through the elements and combinations specifically pointed out in the appended claims.
[0050] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The accompanying drawings, which constitute a part of this specification, illustrate various embodiments and, together with the description, serve to explain the principles and features of the disclosed embodiments. In the drawings:
[0052] Figure 1A is a schematic diagram of an exemplary superpixel according to an embodiment of the present disclosure.
[0053] Figure 1B is a schematic diagram of another exemplary superpixel according to an embodiment of the present disclosure.
[0054] Figure 1C is a schematic diagram of another exemplary superpixel according to an embodiment of the present disclosure.
[0055] Figure 1D is a schematic diagram of yet another exemplary superpixel according to an embodiment of the present disclosure.
[0056] Figure 2A is a schematic diagram of an exemplary exposure measurement subcircuit according to an embodiment of the present disclosure.
[0057] Figure 2B is a schematic diagram of three exposure measurement subcircuits with separate nodes for synchronized exposure according to an embodiment of the present disclosure.
[0058] Figure 2Cis a schematic diagram of three exposure measurement subcircuits with separate nodes for separate exposures according to an embodiment of the present disclosure.
[0059] Figure 2D is a schematic diagram of three exposure measurement sub-circuits with a common node according to an embodiment of the present disclosure.
[0060] Figure 2E is a schematic diagram of three exposure measurement sub-circuits sharing a column-level analog-to-digital converter according to an embodiment of the present disclosure.
[0061] Figure 2F is a schematic diagram of three exposure measurement subcircuits sharing a column-level analog-to-digital converter as well as a floating diffusion, a reset transistor, a source follower transistor, and a pixel select transistor according to an embodiment of the present disclosure.
[0062] Figure 2G is a schematic diagram of three exposure measurement sub-circuits that share a column-level analog-to-digital converter and operate in a rolling shutter mode according to an embodiment of the present disclosure.
[0063] Figure 2H is a schematic diagram of three exposure measurement subcircuits sharing a column-level analog-to-digital converter and operating in rolling shutter mode, as well as a floating diffusion, a reset transistor, a source follower transistor, and a pixel select transistor according to an embodiment of the present disclosure.
[0064] Figure 3A is a schematic diagram of an exemplary analog-to-digital converter according to an embodiment of the present disclosure.
[0065] Figure 3B is a schematic diagram of an exemplary voltage comparator that is part of an exemplary analog-to-digital converter that shares at least one transistor with an exposure measurement subcircuit according to an embodiment of the present disclosure.
[0066] Figure 4 is a flowchart of an exemplary method for controlling an image sensor having multiple superpixels according to an embodiment of the present disclosure.
[0067] Figure 5A According to an embodiment of the present disclosure, Figure 2A Schematic diagram of possible timing of signals controlling light intensity measurement and analog-to-digital conversion of light intensity measurement results in exposure measurement for sub-pixel and super-pixel ADCs in the case of a single exposure measurement sub-pixel shown in FIG.
[0068] Figure 5B According to an embodiment of the present disclosure, Figure 2B ( Figure 2BA schematic diagram of possible timing of signals controlling light intensity measurement and analog-to-digital conversion of light intensity measurement results in exposure measurement of sub-pixels and super-pixel ADCs in the case of two connected exposure measurement sub-pixels shown in three exposure measurement sub-pixels.
[0069] Figure 5C According to an embodiment of the present disclosure, Figure 2C ( Figure 2C A schematic diagram of possible timing of signals controlling light intensity measurement and analog-to-digital conversion of light intensity measurement results in exposure measurement of sub-pixels and super-pixel ADCs in the case of two connected exposure measurement sub-pixels shown in three exposure measurement sub-pixels.
[0070] Figure 5D According to an embodiment of the present disclosure, Figure 2D ( Figure 2D A schematic diagram of possible timing of signals controlling light intensity measurement and analog-to-digital conversion of light intensity measurement results in exposure measurement of sub-pixels and super-pixel ADCs in the case of two connected exposure measurement sub-pixels shown in three exposure measurement sub-pixels.
[0071] Figure 5E According to an embodiment of the present disclosure, Figure 3B Schematic diagram of possible timing of signals controlling light intensity measurement and analog-to-digital conversion of light intensity measurement results in exposure measurement for sub-pixel and super-pixel ADCs in the case of the comparator and exposure measurement sub-pixel shown in FIG.
[0072] Fig. 6A is a schematic diagram of an exemplary condition detector according to an embodiment of the present disclosure.
[0073] Figure 6B is a schematic diagram of another exemplary condition detector according to an embodiment of the present disclosure.
[0074] Figure 7 According to the embodiment of the present disclosure, Figure 6B Illustration of the triggering generated by the detector.
[0075] Figure 8 is a schematic diagram of different possible geometric arrangements of the photodiode and exposure measurement subcircuit of a condition detector according to an embodiment of the present disclosure, as well as a schematic diagram of the connections from the photodiode to the associated circuitry.
[0076] Fig. 9 is a schematic diagram of an exemplary superpixel provided for high-dynamic range (HDR) imaging and / or red-green-blue (RGB) imaging according to an embodiment of the present disclosure.
[0077] Fig. 10A is a schematic diagram of a clock-controlled readout circuit according to an embodiment of the present disclosure.
[0078] Fig. 10B is a schematic diagram of a clock-controlled readout circuit implemented with an enable latch according to an embodiment of the present disclosure.
[0079] Fig.11 is a schematic diagram of an image sensor using a column-level analog-to-digital converter.
[0080] Fig.12 is a schematic diagram of the general time relationship between any asynchronous detection event (CD) and the associated (triggered) exposure measurement (EM) in a continuous control cycle N according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0081] The disclosed embodiments relate to systems and methods for providing superpixels and implementing and using image sensors having superpixels. Advantageously, these exemplary embodiments can provide fast and efficient image and event sensing with direct output in a digital format. The disclosed embodiments can be implemented and used in a variety of applications and vision systems, such as security systems, autonomous vehicles, and other systems that benefit from fast and efficient motion detection and motion event driven image data acquisition.
[0082] Although embodiments of the present disclosure are generally described with reference to imaging systems, it should be understood that such systems may be part of a camera, LIDAR, or other imaging system.
[0083] Figure 1A 1 is a schematic diagram of an exemplary superpixel 100 according to an embodiment of the present disclosure. As used herein, a "superpixel" refers to the smallest element of an image sensor that outputs digital data based on light incident on the superpixel. In some embodiments, a superpixel can be larger or include more components than a traditional "pixel" because it can include two or more photosensitive elements, such as element 101 and element 103, as well as a condition detector 105, an ADC 109, and control and communication logic, such as 111, 113.
[0084] The photosensitive element may include a photodiode (e.g., a pn junction or a PIN structure) or any other element configured to convert light into an electrical signal. Each photodiode (e.g., element 101 or element 103) may generate a current (e.g., I ph ).
[0085] As in Figure 1AAs shown in the example of FIG. 1 , the superpixel includes a synchronous unit 100a and an asynchronous unit 100b. The asynchronous unit 100b may include a photosensitive element 101 (PD CD , for example, a photodiode), and the synchronization unit 100a may include at least one photosensitive element 103 (PD EM , for example, a photodiode).
[0086] The asynchronous unit 100b may further include a condition detector 105 (Condition Detector, CD). Figure 1A As shown in the example of FIG. 1 , the detector 105 is electrically connected to the first photosensitive element 101 (PD CD ), and is configured to generate a trigger signal (at the time of the trigger) when an analog signal proportional to the brightness of light incident on the first photosensitive element 101 matches a condition. Figure 1A ). For example, the condition may include whether the analog signal exceeds a threshold (eg, a voltage level or a current level). The analog signal may include a voltage signal or a current signal.
[0087] The synchronization unit 100a may include an exposure measurement sub-pixel 107. The exposure measurement sub-pixel 107 may be configured to generate a signal that is incident on the second photosensitive element 103 (PD EM ) is an analog measurement proportional to the brightness of the light on the Figure 1A is depicted as a voltage signal V FD , however, the analog measurement value may alternatively include a current signal. The synchronization unit 100a may also include a device for converting the analog measurement value into digital data (in Figure 1A The exposure measurement sub-pixel 107 and the ADC 109 may be combined to form an exposure measurement circuit. Accordingly, the exposure measurement circuit may be electrically connected to the second photosensitive element 103 (PD EM ), and is configured to convert an analog signal proportional to the brightness of light incident on the second photosensitive element 103 into a digital signal. Unlike the asynchronous unit 100b that outputs based on the condition detected by the condition detector CD, the synchronous unit 100a may, for example, be clock-controlled so that digital data is output from the ADC only according to the clock cycle. In some embodiments, the exposure measurement sub-pixel 107 may also be clock-controlled so that the exposure measurement sub-pixel 107 from the second photosensitive element 103 (PD) is output only according to the clock cycle. EM ) signal into an analog signal.
[0088] like Figure 1AAs further shown in FIG. 1 , upon detecting a condition, the condition detector 105 (CD) may send a signal (at Figure 1A The control logic 111 may trigger a switch (not shown) and / or a logic gate 113 (in the example marked as "set") to form part of the asynchronous unit 100b. Figure 1A is depicted as an OR gate in the example) to enable (or Figure 1A 109. As used herein, "enable" may refer to activation of the ADC 109 so that it can perform conversions using input ADC control signals, ramp codes, etc. Accordingly, ADC control signals, ramp codes, etc. may be continuously input to the ADC 109 for use when enabled.
[0089] The combination of the control logic 111 and the switches (not shown) and / or logic gates 113 may be referred to as a “logic circuit”. Accordingly, the logic circuit may be electrically connected to the detector 105 and the exposure measurement circuit and configured to respond to a trigger signal (in Figure 1A In some embodiments, the exposure measurement circuit (or, in particular, the ADC 109 of the exposure measurement circuit) is enabled when the exposure measurement circuit (or, in particular, the ADC 109 of the exposure measurement circuit) is enabled when a digital signal (labeled as “set” in the example of FIG. 1 ) is read out from the exposure measurement circuit (labeled as “digital pixel data”). In some embodiments, the logic circuit may temporarily enable the exposure measurement circuit (or, in particular, the ADC 109 of the exposure measurement circuit) so that the exposure measurement circuit (or, in particular, the ADC 109 of the exposure measurement circuit) is automatically disabled after the digital signal is output.
[0090] Accordingly, after reading out the digital data to an external readout system (not shown), the ADC 109 may generate a reset signal (at Figure 1A The reset signal is sent to the control logic 111. The control logic 111 may thus trigger a switch (not shown) and / or logic gate 113 to disable (or otherwise reset) the ADC 109. Figure 1A As used herein, “disable” may refer to deactivating the ADC 109 to perform conversions using input ADC control signals, ramp codes, etc. However, after disabling, ADC control signals, ramp codes, etc. may still be input to the ADC 109.
[0091] In some embodiments, the logic circuit of the control logic 111 may temporarily enable the synchronization unit 100a (or, in particular, the ADC 109 of the synchronization unit 100a) so that the synchronization unit 100a (or, in particular, the ADC 109 of the synchronization unit 100a) is automatically disabled after outputting the digital signal. Figure 1A After being enabled by the logic circuit, the reset signal (in Figure 1A ) may be sent from ADC 109 to the logic circuit of control logic 111 so that the logic circuit is ready to receive a new “set” signal from condition detector 105 .
[0092] like Figure 1A As further shown in FIG. 1 , the condition detector 105 (CD) may transmit a trigger signal to an external readout system (in Figure 1A The condition detector 105 (CD) may receive a response signal (shown as a "request" in the example of Figure 1A The external readout system may also send control signals to the exposure measurement sub-pixel 107 and / or the ADC 109. All communications with the external readout system may be controlled by a protocol such as the Address-Event Representation (AER) protocol and / or may be controlled by clock cycles. Accordingly, the external readout system may include an event readout system configured to asynchronously read data from the super-pixel and / or may include a clock-controlled readout system configured to read data from the super-pixel during a predetermined clock cycle.
[0093] like Figure 1A As further shown in , external control can also be provided to a switch (not shown) and / or logic gate 113 (or to control logic 111) to activate readout. For example, if superpixel 100 does not return a readout value after a threshold number of clock cycles (e.g., if a condition is not met after a threshold number of clock cycles), the external readout system can send a control signal to force readout. Additionally or alternatively, the external readout system can operate in a standard frame mode so that a superpixel or at least a group of superpixels (e.g., superpixel 100) is triggered to capture a full frame or a partial frame regardless of whether a condition is detected.
[0094] Figure 1B is another exemplary superpixel 100 ′ according to an embodiment of the present disclosure. Figure 1B The function of superpixel 100′ is similar to Figure 1A The superpixel 100 further includes a switch 115 (S) together with an OR gate 113 in the control circuit. EM ). Accordingly, by using an external control signal applied to the switch 115, Figure 1B The superpixel 100' may also suppress the exposure measurement subpixel 107 whenever the ADC 109 is disabled.
[0095] Figure 1C Yet another exemplary superpixel 100 ″ is depicted in accordance with an embodiment of the present disclosure. Figure 1C The SuperPixel 100″ functions similarly to Figure 1A 107a, 107b, and 107c. Although three exposure measurement sub-pixels are depicted, Figure 1C The embodiments depicted in can be implemented with any number of sub-pixels paired with ADC 109. Accordingly, Figure 1C Embodiments of the invention may allow for capturing more accurate data, for example, by averaging, summing, or otherwise combining measurements from multiple exposure-measuring sub-pixels and then converting the combined measurements into digital signals. Additionally or alternatively, the measurements from multiple exposure-measuring sub-pixels may be converted sequentially into digital signals to achieve higher resolution.
[0096] Figure 1D Yet another exemplary superpixel 100'" is depicted in accordance with an embodiment of the present disclosure. Figure 1D The function of superpixel 100″′ is similar to Figure 1C 100″, but also includes a latch 115. The latch 115 may, for example, include a switch (e.g., one or more transistors configured to function as a switch), one or more bits of a static random access memory (SRAM), etc. Although depicted as being separate from the ADC 109, in some embodiments, the latch 115 may include at least one of the n-bit latches provided in the ADC 109 (e.g., as described below with respect to Figure 3A ADC 300 as depicted).
[0097] In one embodiment, the trigger signal (at Figure 1D In the example marked as "set" in FIG. 1 , the control logic 111 can cause the control logic 111 to activate during the first cycle N-1. In response, the control logic 111 can turn on the enable signal (in Figure 1DIn the example of FIG. 100 , the ADC 109 may send a signal to the control logic 111 (in the example of FIG. 101 ) to the ADC 109. In response, the ADC 109 may begin converting analog signals from one or more exposure measurement subpixels (e.g., subpixels 107a, 107b, and 107c) in the second cycle N. In addition, the ADC 109 may set the latch 115 and send a signal to the control logic 111 (in the example of FIG. 101 ) to start the conversion of analog signals from one or more exposure measurement subpixels (e.g., subpixels 107a, 107b, and 107c) in the second cycle N. Figure 1D In the example of FIG. 1 , the trigger signal is marked as “cleared” so that the control logic 111 is ready to generate a new trigger signal at any time during the cycle N. Accordingly, Fig.12 As depicted in the example of and explained further below, a new exposure / conversion can already be performed in the third cycle N+1. The latch 115 is further connected to the readout system (e.g., via Figure 1D ) to signal the readout system to perform readout at the end of cycle N. Accordingly, when the readout system pulls the digital signal from ADC 109 at the end of cycle N, latch 115 can be cleared by ADC 109.
[0098] In another embodiment, control logic 111 may activate latch 115 instead of sending a "power-on" signal to ADC 109. In such an embodiment, at the beginning of each cycle, ADC 109 may poll the state of latch 115 to determine whether to perform conversions in that cycle. In an embodiment where ADC 109 is disabled rather than running continuously, ADC 109 may be disabled for a cycle if latch 115 is not activated at the beginning of the cycle. If ADC 109 polls latch 115 and latch 115 is enabled, ADC 109 may clear latch 115 and begin converting analog signals from one or more exposure measurement sub-pixels (e.g., sub-pixels 107a, 107b, and 107c). In such an embodiment, ADC 109 may further set an internal status bit to notify the readout system to perform a readout at the end of the cycle. After the readout, ADC 109 may disable itself and, at the beginning of the next cycle, again poll latch 115 to determine whether to re-enable another cycle. Alternatively, the ADC 109 may poll the latch 115 before disabling to determine whether to remain enabled for another conversion or to disable.
[0099] In another embodiment ( Figure 1D), latch 115 may be external to superpixel 100′″. In such an embodiment, superpixel 100′″ may exclude control logic 111. Alternatively, detector 105 may activate latch 115 in response to a condition being met, and ADC 109 may continue to perform conversions in each cycle. Accordingly, the readout system may use latch 115 to determine whether to perform a readout at the end of each cycle. Otherwise, the conversion performed by ADC 109 may be deleted and not read out. Although such an embodiment may use additional power for each ADC cycle, superpixel 100′″ may be made smaller by eliminating control logic 111 and using a smaller external latch array (e.g., latch 115).
[0100] In embodiments where latch 115 is external to superpixel 100′″, latch 115 may be connected to superpixel 100′″ using at least one of: a direct wiring connection, or one or more arbitrators. For example, latch 115 may form part of an external latch array such that one or more arbitrators may associate a memory address of latch 115 with an address of a corresponding superpixel (e.g., superpixel 100′″).
[0101] Although three exposure measurement sub-pixels are depicted (e.g., sub-pixels 107a, 107b, and 107c), Figure 1D The embodiments depicted in can be implemented with any number of sub-pixels paired with ADC 109, for example, one sub-pixel, two sub-pixels, four sub-pixels, etc.
[0102] Figure 1A , Figure 1B , Figure 1C and Figure 1D The exemplary superpixels 100, 100', 100", and 100'" can be arranged into one or more arrays, respectively, to form an image or event sensor. For example, the superpixels can be arranged into one or more rows and one or more columns with corresponding row and column clock circuits for reading out digital data from an ADC to an external readout system. In another example, the superpixels can be arranged in a star pattern with a clock circuit for each node of the star for reading out digital data from an ADC to an external readout system.
[0103] Figure 2A is used for superpixels (e.g. Figure 1A Superpixel 100 or Figures 1B to 1D Schematic diagram of an exposure measurement sub-pixel 200 in any of the other super-pixels of FIG. Figure 2A As shown in FIG. 1 , a sub-pixel 200 may include a photosensitive element 201 (PD EM), the photosensitive element 201 generates a signal proportional to the light incident on the element 201 (e.g., I ph ). The signal may be integrated onto node 203 (FD). For example, the signal may be integrated onto a node of element 201 (eg, photodiode node PD) at a predetermined time. EM The integrated charge can be transferred from the parasitic capacitance of element 201 to node 203 (FD) through transistor 205 (TG). The gate of transistor 205 (TG) can be Figure 2A Accordingly, transistor 205 (TG) generates a voltage output signal V by transmitting the integrated signal to node 203 (FD). FD .like Figure 2A As depicted in FIG. 1 , node 203 may have a capacitance due to capacitor 207 and / or parasitic capacitance at the node. For example, in the case of signal V FD After being transmitted to the ADC (not shown) via the switch 211 (and / or one or more transistors configured to act as a switch), the second transistor 209 (RS) can reset the node 203 for a new measurement. The gate of the transistor 211 (RS) can be Figure 2A The signal marked "reset" controls the
[0104] Figure 2B is a schematic diagram of a system 250 that includes three exposure measurement subpixels having separate sense nodes and having common transfer and reset signals to allow simultaneous exposure (i.e., common start and end of integration). Different sense nodes can be configured to have different capacitances to allow different charge-to-voltage conversion gains. This can be beneficial, for example, for high dynamic range operation. System 250 can be used for superpixels (e.g., Figure 1C In the superpixel 100″). Figure 2B As shown in FIG. , each photosensitive element (eg, element 251 (PD EM1 ), element 253 (PD EM2 ) and component 255 (PD EM3 )) may have its corresponding signal (e.g., Figure 2B The example shown in I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3) on different parasitic capacitances of the cathode of ). Accordingly, transistors 257a, 257b and 257c (TG1, TG2 and TG3, respectively) generate voltage output signals V by transmitting the integrated signals to nodes 259a, 259b and 259c (FD1, FD2 and FD3, respectively). FD .like Figure 2B As shown in , node 259a may have a capacitance due to capacitor 261a and / or parasitic capacitance at the node; node 259b may have a capacitance due to capacitor 261b and / or parasitic capacitance at the node; and node 259c may have a capacitance due to capacitor 261c and / or parasitic capacitance at the node. In addition, Figure 2B In the example above, all transistors are controlled by the same signal (labeled "Transfer"). For example, in the case of signal V FD After being transmitted to the ADC (not shown), the second set of transistors 263a, 263b and 263c (RS1, RS2 and RS3, respectively) can reset nodes 259a, 259b and 259c, respectively, for a new measurement. The gates of transistors 263a, 263b and 263c (RS1, RS2 and RS3, respectively) can be reset by Figure 2B Each node can be connected to a different path component, such as a switch (e.g., switch 265a (S FD1 ), switch 265b (S FD2 ) and switch 265c (S FD3 ) and / or one or more transistors configured to act as a switch) to allow simultaneous exposure and sequential transfer to an ADC (not shown).
[0105] Figure 2C 2 is a schematic diagram of a system 270 including three exposure measurement sub-pixels having separate sensing nodes and having separate transmission signals to allow different exposure times (i.e., a common start time for integration but different end times). The conversion gains of the different sensing nodes can be configured to be the same so that a high dynamic range can be obtained with different integration times. Alternatively, similar to Figure 2B In the system 250, the conversion gain of different sensing nodes can be configured to be different. The system 270 can be used for super pixels (e.g., Figure 1C In the superpixel 100″). Figure 2C As shown in FIG. , each photosensitive element (eg, element 271 (PD EM1 ), element 273 (PD EM2 ) and element 275 (PD EM3 )) may have its corresponding signal (e.g., Figure 2C The example shown in Iph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on different parasitic capacitances of the cathode of ). Accordingly, transistors 277a, 277b and 277c (TG1, TG2 and TG3, respectively) generate voltage output signals V by transmitting the integrated signals to nodes 279a, 279b and 279c (FD1, FD2 and FD3, respectively). FD .like Figure 2C As depicted in FIG, node 279a may have a capacitance due to capacitor 281a and / or parasitic capacitance at the node; node 279b may have a capacitance due to capacitor 281b and / or parasitic capacitance at the node; and node 279c may have a capacitance due to capacitor 281c and / or parasitic capacitance at the node. In addition, Figure 2C In the example of FIG. 1 , each transistor is controlled by a corresponding signal (e.g., labeled “Transfer 1 ”, “Transfer 2 ”, and “Transfer 3 ”). For example, in the case of signal V FD After being transmitted to the ADC (not shown), the second set of transistors 283a, 283b and 283c (RS1, RS2 and RS3, respectively) can reset nodes 279a, 279b and 279c, respectively, for a new measurement. The gates of transistors 283a, 283b and 283c (RS1, RS2 and RS3, respectively) can be reset by Figure 2B Each node can be connected to different paths, such as switches (e.g., switches 285a (S FD1 )、switch 285b(S FD2 ) and switch 285c (S FD3 ) and / or one or more transistors configured to act as switches) to allow sequential transfer to an ADC (not shown).
[0106] Figure 2D 2 is a schematic diagram of a system 290 including three exposure measurement sub-pixels that share the same sensing node and have separate transmission signals to allow different exposure times (i.e., a common start time for integration but different end times). System 290 can be used for superpixels (e.g., Figure 1C In the superpixel 100″). Figure 2D As shown in FIG. , each photosensitive element (eg, element 291 (PD EM1 ), element 293 (PD EM2 ) and element 295 (PDEM3 )) may have its corresponding signal (e.g., I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on the parasitic capacitance of the cathode of ). Accordingly, transistors 277a, 277b and 277c (TG1, TG2 and TG3, respectively) generate voltage output signals V by transmitting the integrated signals to node 297 (FD), respectively. FD .like Figure 2D As depicted in FIG. 2 , node 297 may have a capacitance due to capacitor 298 and / or parasitic capacitance at the node. Figure 2D In the example of FIG. 1 , each transistor is controlled by a corresponding signal (e.g., labeled “Transfer 1 ,” “Transfer 2 ,” and “Transfer 3 ”). The second transistor 299 (RS1) can reset node 297 for a new measurement, for example, after sequentially transmitting the integrated signal to an ADC (not shown). The gate of transistor 299 (RS1) can be controlled by Figure 2D is controlled by the corresponding signal marked as "reset" in.
[0107] Figure 2E is a schematic diagram of a system 270′ including three exposure measurement subpixels with separate sensing nodes and with separate transmission signals to allow different exposure times (i.e., a common start time for integration but different end times). System 270′ can be used with column-level ADC and external latch operation. The readout trigger is still controlled by the condition detector of the superpixel.
[0108] Accordingly, in Figure 2E In , the conversion gains of different sensing nodes can be configured to be the same so that a high dynamic range can be obtained by using different integration times. Figure 2B The conversion gain of different sensing nodes of the system 250 can be configured to be different. The system 270′ can be used for the superpixel (e.g., Figure 1C Furthermore, system 270′ may be used in a superpixel having a conditional detector that outputs a trigger signal but does not control an exposure measurement circuit, but instead the exposure measurement circuit is controlled by an external readout trigger (e.g., a standard frame mode that implements readout of the full pixel array).
[0109] like Figure 2E As shown in FIG. , each photosensitive element (eg, element 271 (PD EM1 ), element 273 (PDEM2 ) and element 275 (PD EM3 )) may have its corresponding signal (e.g., I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on different parasitic capacitances of the cathode of ). Accordingly, transistors 277a, 277b and 277c (TGX1, TGX2 and TGX3, respectively) generate voltage output signals V by transmitting the integrated signals to nodes 279d, 279e and 279f (MEM1, MEM2 and MEM3, respectively). FD .like Figure 2E As depicted in FIG, node 279d may have a capacitance due to capacitor 281d and / or parasitic capacitance at the node; node 279e may have a capacitance due to capacitor 281e and / or parasitic capacitance at the node; and node 279f may have a capacitance due to capacitor 281f and / or parasitic capacitance at the node. Figure 2E In the example of Figure 2E 270 ′). For example, when a row including system 270 ′ is selected, a global periodic external signal may activate signal “Transfer X”. Accordingly, such charge transfer may occur on a first fixed clock cycle.
[0110] In the second fixed period, such as Figure 2E As shown in FIG. 1 , transistors 277d, 277e, and 277f (TGY1, TGY2, and TGY3, respectively) generate a voltage output signal V by transmitting the integrated signal to nodes 279a, 279b, and 279c (FD1, FD2, and FD3, respectively). FD .like Figure 2E As depicted in FIG, node 279a may have a capacitance due to capacitor 281a and / or parasitic capacitance at the node; node 279b may have a capacitance due to capacitor 281b and / or parasitic capacitance at the node; and node 279c may have a capacitance due to capacitor 281c and / or parasitic capacitance at the node. In addition, Figure 2E In the example of , each transistor is controlled by a second global periodic external signal (eg, labeled as “transmission Y1 ,” “transmission Y2 ,” and “transmission Y3 ,” respectively).
[0111] The third set of transistors 287a, 287b, and 287c (TOV1, TOV2, and TOV3, respectively) can be used as overflow gates to reset the photodiodes and ensure that the transmitted voltage V FD Does not invert the sign before output.
[0112] A fourth group of transistors 289a, 289b, and 289c (SF1, SF2, and SF3, respectively) and 291a, 291b, and 291c (SEL1, SEL2, and SEL3, respectively) can control the selection and readout of the exposure measurement circuit. For example, in an embodiment where the exposure measurement is triggered by a condition detector, transistors 291a, 291b, and 291c (SEL1, SEL2, and SEL3, respectively) can respond to the trigger signal (labeled "Select 1", "Select 2", and "Select 3", respectively) to activate the measurement from the exposure measurement circuit corresponding to the condition detector. In response, transistors 289a, 289b, and 289c (SF1, SF2, and SF3, respectively) can act as source followers of nodes 279a, 279b, and 279c (FD1, FD2, and FD3, respectively). Additionally or alternatively, for example, when an exposure measurement circuit is not triggered in a threshold number of clock cycles, an external readout system (not shown) can select an exposure measurement circuit regardless of its corresponding condition detector. In such an embodiment, the condition detector can also activate an external latch so that the readout system can use the external latch to determine which exposure measurement circuits should be clocked (e.g., using corresponding row transfer signals and column transfer signals). Alternatively, the readout system can clock all exposure measurement circuits and then use post-processing to throw away any measurement values from exposure measurement circuits that have corresponding external latches that are not enabled.
[0113] Finally, with Figure 2C The system 270 is similar, for example, in which the signal V FD After being transmitted to the ADC (not shown), a set of transistors 283a, 283b and 283c (RS1, RS2 and RS3, respectively) can reset nodes 279a, 279b and 279c, respectively, for a new measurement. The gates of transistors 283a, 283b and 283c (RS1, RS2 and RS3, respectively) can be connected by Figure 2E It is controlled by the corresponding signals marked as "RESET 1", "RESET 2" and "RESET 3" respectively.
[0114] Figure 2Fis a schematic diagram of a system 290′ including three exposure measurement subpixels that share the same sensing node and have separate transmission signals to allow different exposure times (i.e., a common start time for integration but different end times). System 290′ can be used with column-level ADC and external latch operations. The readout trigger is still controlled by the condition detector of the superpixel.
[0115] Alternatively, system 290' may be used for superpixels of the present disclosure (e.g., Figure 1C Further, system 290′ can be used in a superpixel having a conditional detector that outputs a trigger signal but does not control the exposure measurement circuit, but instead the exposure measurement circuit is controlled by an external readout trigger (e.g., a standard frame mode that implements readout of the entire pixel array).
[0116] like Figure 2F As shown in FIG. , each photosensitive element (eg, element 271 (PD EM1 ), element 273 (PD EM2 ) and element 275 (PD EM3 )) may have its corresponding signal (e.g., I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on the parasitic capacitance of the cathode of ). Accordingly, transistors 277a, 277b and 277c (TGX1, TGX2 and TGX3, respectively) generate voltage output signals V by simultaneously transmitting the integrated signals to nodes 279d, 279e and 279f (MEM1, MEM2 and MEM3, respectively). FD .like Figure 2F As depicted in FIG, node 279d may have a capacitance due to capacitor 281a and / or parasitic capacitance at the node; node 279e may have a capacitance due to capacitor 281b and / or parasitic capacitance at the node; and node 279f may have a capacitance due to capacitor 281c and / or parasitic capacitance at the node. Figure 2F In the example of Figure 2F 290 ′). For example, when a row including system 290 ′ is selected, a global periodic external signal may activate signal “Transfer X”. Accordingly, such charge transfer may occur on a first fixed clock cycle.
[0117] In the second fixed period, such as Figure 2F As shown in FIG. 1 , transistors 277 d, 277 e, and 277 f (TGY1, TGY2, and TGY3, respectively) generate a voltage output signal V by transmitting the integrated signal to node 297 (FD). FD .like Figure 2F As depicted in FIG. 2 , node 297 may have a capacitance due to capacitor 298 and / or parasitic capacitance at the node. Figure 2F In the example of , each transistor is controlled by a second periodic external signal that sequentially triggers three transmissions (eg, labeled as “transmission Y1 ,” “transmission Y2 ,” and “transmission Y3 ,” respectively).
[0118] The third set of transistors 287a, 287b, and 287c (TOV1, TOV2, and TOV3, respectively) can be used as overflow gates to reset the photodiodes and ensure that the transmitted voltage V FD Does not invert the sign before output.
[0119] A fourth set of transistors 289 (SF) and 291 (SEL) can control the selection and readout of exposure measurement circuits. For example, in an embodiment where exposure measurement is triggered by a condition detector, transistor 291 (SEL) can respond to a trigger signal (labeled "SELECT") to activate measurements from the exposure measurement circuit corresponding to the condition detector. In response, transistor 289 (SF) can act as a source follower for node 297 (FD). Additionally or alternatively, for example, when an exposure measurement circuit is not triggered in a threshold number of clock cycles, an external readout system (not shown) can select an exposure measurement circuit regardless of its corresponding condition detector. In such an embodiment, the condition detector can also activate an external latch so that the readout system can use the external latch to determine which exposure measurement circuits should be clocked (e.g., using corresponding row transfer signals and column transfer signals). Alternatively, the readout system can clock all exposure measurement circuits and then use post-processing to throw away any measurements from exposure measurement circuits that have corresponding external latches that are not enabled.
[0120] The second transistor 299 (RS1) can reset the node 297 (FD) for a new measurement, for example, after sequentially transmitting the integrated signal to an ADC (not shown). The gate of the transistor 299 (RS1) can be formed by Figure 2F is controlled by the corresponding signal marked as "reset" in.
[0121] Figure 2Gis a schematic diagram of a system 250′ including three exposure measurement sub-pixels with separate sense nodes and with separate transmission signals. Different sense nodes can be configured to have different capacitances to allow different charge-to-voltage conversion gains. Such an arrangement may be beneficial for high dynamic range operation. System 250′ can be used with column-level ADC and external latch operation. The readout trigger is still controlled by the condition detector of the superpixel.
[0122] System 250' may be used for superpixels of the present disclosure (e.g., Figure 1C Furthermore, the system 250′ may be used in a superpixel 100″ having a conditional detector that outputs a trigger signal but does not control an exposure measurement circuit, but rather an exposure measurement circuit that is controlled by an external readout trigger (e.g., a standard frame mode that implements readout of the complete pixel array).
[0123] like Figure 2G As shown in FIG. , each photosensitive element (eg, element 251 (PD EM1 ), element 253 (PD EM2 ) and component 255 (PD EM3 )) may have its corresponding signal (e.g., I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on different parasitic capacitances of the cathode of ). Accordingly, transistors 257a, 257b and 257c (TG1, TG2 and TG3, respectively) generate voltage output signals V by transmitting the integrated signals to nodes 259a, 259b and 259c (FD1, FD2 and FD3, respectively). FD .like Figure 2G As depicted in , node 259a may have a capacitance due to capacitor 261a and / or parasitic capacitance at the node; node 259b may have a capacitance due to capacitor 261b and / or parasitic capacitance at the node; and node 259c may have a capacitance due to capacitor 261c and / or parasitic capacitance at the node. After the sensing node is reset, the transmission is controlled by the first periodic external signal.
[0124] Before a new readout, for example, the signal V FDBefore being transmitted to the ADC (not shown) and before a new measurement, the second set of transistors 263a, 263b and 263c (respectively RS1, RS2 and RS3) can reset nodes 259a, 259b and 259c, respectively. The gates of transistors 263a, 263b and 263c (respectively RS1, RS2 and RS3) can be reset by Figure 2G It is controlled by a second periodic external signal marked as "RESET 1", "RESET 2" and "RESET 3" respectively.
[0125] A third set of transistors 289a, 289b, and 289c (SF1, SF2, and SF3, respectively) and 291a, 291b, and 291c (SEL1, SEL2, and SEL3, respectively) can control the selection and readout of the exposure measurement circuit. For example, in an embodiment where the exposure measurement is triggered by a condition detector, transistors 291a, 291b, and 291c (SEL1, SEL2, and SEL3, respectively) can respond to the trigger signal (labeled "Select 1", "Select 2", and "Select 3", respectively) to activate the measurement from the exposure measurement circuit corresponding to the condition detector. In response, transistors 289a, 289b, and 289c (SF1, SF2, and SF3, respectively) can act as source followers of nodes 279a, 279b, and 279c (FD1, FD2, and FD3, respectively). Additionally or alternatively, for example, when an exposure measurement circuit is not triggered in a threshold number of clock cycles, the external readout system can select an exposure measurement circuit regardless of its corresponding condition detector. In such an embodiment, the condition detector can also activate an external latch so that the readout system can use the external latch to determine which exposure measurement circuits should be clocked (e.g., using corresponding row transfer signals and column transfer signals). Alternatively, the readout system can clock all exposure measurement circuits and then use post-processing to throw away any measurements from exposure measurement circuits that have corresponding external latches that are not enabled.
[0126] Figure 2H is a schematic diagram of a system 290" that includes three exposure measurement subpixels that share the same sensing node and have separate transmission signals to allow different exposure times (ie, a common start time for integration but different end times).
[0127] System 290″ can be used with column-level ADC and external latch operations. The readout trigger is still controlled by the condition detector of the superpixel. Optionally, system 290″ can be used with superpixels of the present disclosure (e.g., Figure 1CFurthermore, the system 290" may be used in a superpixel 100" having a conditional detector that outputs a trigger signal but does not control an exposure measurement circuit, but instead the exposure measurement circuit is controlled by an external readout trigger (e.g., a standard frame mode that implements reading out the complete pixel array).
[0128] like Figure 2H As shown in FIG. , each photosensitive element (eg, element 271 (PD EM1 ), element 273 (PD EM2 ) and element 275 (PD EM3 )) may have its corresponding signal (e.g., I ph1 ,I ph2 and I ph3 ), the corresponding signal is integrated to the nodes of each element (e.g., the corresponding photodiode node (e.g., PD EM1 , PD EM2 and PD EM3 ) on the parasitic capacitance of the cathode of ). Accordingly, transistors 277a, 277b and 277c (TG1, TG2 and TG3, respectively) generate a voltage output signal V by sequentially transmitting the integrated signal to node 297 (FD). FD .like Figure 2H As depicted in FIG. 2 , node 297 may have a capacitance due to capacitor 298 and / or parasitic capacitance at the node. Figure 2H In the example of FIG. 1 , the transistors are controlled by a first periodic external signal (e.g., via signals labeled “Transfer 1 ,” “Transfer 2 ,” and “Transfer 3 ,” respectively). The second transistor 299 (RS1) can reset the node 297 for a new measurement, for example, before the signal VFD is transmitted to the ADC (not shown) and before a new measurement. The gate of the transistor 299 (RS1) can be controlled by Figure 2H The second periodic external signal marked as "reset" is controlled.
[0129] Although three exposure measurement sub-pixels are depicted, FIG. 2B to FIG. 2H The structure depicted in can be implemented for any number of exposure measurement sub-pixels. For example, two exposure measurement sub-pixels can be used, or more than three exposure measurement sub-pixels can be used.
[0130] Figure 3A is used for superpixels (e.g. Figure 1A Superpixel 100 or Figures 1B to 1D Schematic diagram of an analog-to-digital converter 300 in any of the other superpixels of FIG. Figure 3AAs shown in , ADC 300 may include an n-bit ramp analog-to-digital converter. Thus, the ADC may be a digital ramp analog-to-digital converter, such as a single slope ramp or multi-slope ramp type analog-to-digital converter. A person of ordinary skill will appreciate that other ADCs may be used in superpixels according to the present disclosure.
[0131] like Figure 3A As further shown in FIG. 1 , the control logic of the control circuit of the superpixel controls the ADC so that digital data is output from the ADC only when the ADC is enabled. In addition, the enablement (or “power-up”) caused by the condition detector of the superpixel can cause the ADC 300 to activate, for example, by supplying power to the comparator 303 of the ADC 300. Externally, a ramp voltage (V ramp ) and the output signal (V FD ) are fed from an external control circuit to a comparator 303 of the ADC 300. A ramp code (e.g., to control the timing between ramp increases) may also be fed externally to control the ADC. Other ramp controls (labeled "controls") may also be input externally to the ADC. Accordingly, a number of latches 305 (determined by the number of bits) are set using data from the comparator 303 and read and write controls based on the ramp code. The contents of the latches 305 may be read out (e.g., by an external readout system (not shown)), and the ADC control logic 301 may send a "clear" signal to the control logic (not shown) of the control circuit of the superpixel.
[0132] Figure 3B Depicts the use of superpixels (e.g., Figure 1C The alternative analog-to-digital converter 350 in the superpixel 100″) is Figure 3B In the example, Figure 3B The comparator 360 may include a pair of transistors such that a first transistor 361a cooperates with at least one second transistor of a plurality of sub-pixels (eg, a second transistor 361b of the sub-pixel 370 or a second transistor 361c of the sub-pixel 380, etc.).
[0133] Figure 3B The multiple sub-pixels can be configured similarly to the above FIG. 2A to FIG. 2H Any of the examples depicted in . Figure 3B In the example, the sub-pixel 370 includes a photosensitive element PD EM1 , the photosensitive element PD EM1A signal is generated which is integrated onto node 379 (FD) through transistor 375 (TGT1). In addition, a second transistor 373 (RST1) can reset node 379 (FD) for a new measurement, for example, after the signal is transmitted to comparator 360 of the ADC. A third transistor 377 (OVGT1) can be used as an overflow gate to ensure that the transmitted voltage VFD does not reverse sign when used by comparator 360 of the ADC, and a fourth transistor 371 (SEL1) can control the selection and readout of sub-pixel 370. Similarly, in Figure 3B In the example, the sub-pixel 380 includes a photosensitive element PD EM2 , the photosensitive element PD EM2 A signal is generated which is integrated to a node 389 (FD) through a transistor 385 (TGT2). In addition, a second transistor 383 (RST2) can reset the node 389 (FD) for a new measurement, for example, after the signal is transmitted to the comparator 360 of the ADC. A third transistor 387 (OVGT2) can be used as an overflow gate to ensure that the transmitted voltage VFD does not reverse sign when used by the comparator 360 of the ADC, and a fourth transistor 381 (SEL2) can control the selection and readout of the sub-pixel 370.
[0134] Figure 4 is a flow chart of an exemplary method 400 for controlling an image sensor having multiple superpixels according to an embodiment of the present disclosure. Figure 4 The method 400 may use any superpixel disclosed herein (e.g. Figure 1A Superpixel 100 or Figures 1B to 1D to any superpixel in the other superpixels of .
[0135] At step 401, the superpixel may receive a first analog signal proportional to light incident on a first photosensitive element. For example, as explained above, the photosensitive element 101 may generate a first analog signal proportional to the light incident thereon (e.g., I ph or voltage signal).
[0136] At step 403, the superpixel may generate a trigger signal when the first analog signal matches the condition. For example, as explained above, the condition detector 105 (CD) may generate a trigger signal (e.g., "set" in the above example) when the first analog signal exceeds a threshold. In some embodiments, the condition detector (CD) of the superpixel may detect the condition asynchronously with the system clock.
[0137] At step 405, in response to the trigger signal, the superpixel may enable the exposure measurement circuit. For example, as explained above, the control logic 111 may enable the ADC 109 in response to the trigger signal (“set”). In some embodiments, the exposure measurement subpixel 107 (or exposure measurement subpixels 107a, 107b, or 107c, etc.) is not controlled by the system clock (e.g., does not receive periodic control signals such as “reset” and “transfer” as explained above with respect to the example of FIG. 2 ), and the control logic 111 may also enable the exposure measurement subpixel 107 (or exposure measurement subpixels 107a, 107b, or 107c, etc.) in response to the trigger signal (“set”). For example, the control logic 111 may enable the exposure measurement subpixel 107 (or exposure measurement subpixel 107a, 107b, or 107c, etc.) by enabling the subpixel to receive the periodic control signal, thereby integrating the signal from the photosensitive element (as explained above with respect to the example of FIG. 2 ).
[0138] At step 407, the superpixel may receive at least one second analog signal proportional to the light incident on at least one second photosensitive element. For example, as explained above with respect to the example of FIG. 1 , photosensitive element 103 may generate a second analog signal (e.g., I ph As explained above with respect to step 405, in some embodiments, the exposure measuring sub-pixel 107 (or the exposure measuring sub-pixel 107a, 107b, or 107c, etc.) may receive the second analog signal but not integrate it until a periodic control signal is received in response to a trigger signal from the condition detector 105. In other embodiments, the exposure measuring sub-pixel 107 (or the exposure measuring sub-pixel 107a, 107b, or 107c, etc.) may receive the periodic control signal according to a system clock, and thus integrate the received second analog signal according to the periodic control signal.
[0139] At step 409, the superpixel may convert at least one second analog signal into a digital signal using the exposure measurement circuit. For example, as explained above with respect to the example of FIG. 1 , after enabling ADC 109, ADC 109 may convert the second analog signal (e.g., V FD ) is converted into digital data (e.g., “digital pixel data”) and output to an external readout system.
[0140] At step 411, after converting the at least one second analog signal to a digital signal, the superpixel may disable the exposure measurement circuit. For example, as explained above with respect to the example of FIG. 1, ADC 109 may generate a “clear” signal in response to the conversion and / or output, which control logic 111 may use to disable ADC 109. Additionally or alternatively, after the conversion and / or output, ADC 109 may automatically stop responding to an external control signal (e.g., V ramp or ramp code, etc.) response.
[0141] Figure 5A It is a description control Figure 1A or Figure 1B The signal of the superpixel and the Figure 1A or Figure 1B A diagram showing the timing of the signal generated by a superpixel using Figure 2A Sub-pixel system for exposure measurement. Figure 5A As depicted in FIG. 1 , a reset signal may be sent by an external readout system, for example, based on a system clock, to the exposure measurement subpixel 107. Furthermore, a transfer signal may be sent by an external readout system, for example, based on a system clock, to the exposure measurement subpixel 107. The exposure measurement subpixel 107 may thus begin integrating the analog signal from the photosensitive element 103 of the exposure measurement subpixel into an analog signal V provided to the ADC 109. FD In addition, despite Figure 5A 109, but may send a signal from condition detector 105 to ADC, for example, in response to detecting that the analog signal from photosensitive element 101 of condition detector 105 matches a condition. Accordingly, ADC 109 may begin to ramp V FD Convert to digital data.
[0142] In an embodiment using a ramp ADC, a ramp code may be provided from an external readout system to ADC 109. The corresponding latch of ADC 109 may use the ramp code to convert the analog signal V FD ADC 109 may then use the readout peripheral to output the data and reset the latch of ADC 109 for another conversion. Figure 5A Not shown in FIG. 1 , the ADC 109 may thereafter be automatically disabled after converting the analog signal / outputting the digital data.
[0143] Figure 5B Yes Control Figure 1C The signal of the superpixel and the Figure 1C A diagram showing the timing of the signal generated by a superpixel using Figure 2B Sub-pixel system for exposure measurement. Figure 5B The signal and Figure 5A The signal V is similar to that of ADC 109, but includes two analog signals V from two exposure measurement sub-pixels 107a and 107b paired to a single ADC 109. FD1 and V FD2 .like Figure 5B As shown in , the integration is simultaneous, while the readout is sequential. Accordingly, the sequential readout includes complementary measurements obtained at the same time. Therefore, the gain and / or color filter of the exposure measurement sub-pixel can be different in order to obtain complementary data.
[0144] Figure 5C Yes Control Figure 1C The signal of the superpixel and the Figure 1C A diagram showing the timing of the signal generated by a superpixel using Figure 2C Sub-pixel system for exposure measurement. Figure 5C The signal and Figure 5A The signal V is similar to that of ADC 109, but includes two analog signals V from two exposure measurement sub-pixels 107a and 107b paired to a single ADC 109. FD1 and V FD2 .like Figure 5C As shown in , the integration is simultaneous, but over different times (i.e., "exposure time 1" and "exposure time 2"). Accordingly, the sequential readouts include complementary measurements taken at different times. Therefore, complementary data can be used for high dynamic range imaging.
[0145] Figure 5D Yes Control Figure 1C The signal of the superpixel and the Figure 1C A diagram showing the timing of the signal generated by a superpixel using Figure 2D Sub-pixel system for exposure measurement. Figure 5D The signal and Figure 5A The signal is similar to that of FD is based on two exposure measurement sub-pixels 107a and 107b that are paired to a single ADC 109 and sequentially transmitted to the same sensing node. Figure 5D As shown in , the integration is simultaneous, but over different times (i.e., "exposure time 1" and "exposure time 2"). Accordingly, the sequential readouts include complementary but overlapping (in time) measurements taken at different times. Thus, the complementary data can be used for high dynamic range imaging.
[0146] Figure 5E is a diagram of the timing of the signals controlling a superpixel and the signals generated by the superpixel, the superpixel using Figure 3B Sub-pixel system for exposure measurement. Figure 5EAs depicted in FIG. 1 , a reset signal may be sent by an external readout system, for example, based on a system clock, to the exposure measuring sub-pixels 370 and 380. In addition, a transfer signal may be sent by an external readout system, for example, based on a system clock, to the exposure measuring sub-pixels 370 and 380. The exposure measuring sub-pixels 370 and 380 may thus begin integrating the analog signal from the photosensitive element of the exposure measuring sub-pixel into the analog signal of the comparator 360 provided to the ADC. In addition, although Figure 5E , but a signal from condition detector 105 may be sent to the ADC, for example, in response to detecting that an analog signal from a photosensitive element of condition detector 105 matches a condition. Accordingly, the ADC may begin converting the analog signals from sub-pixels 370 and 380 into digital data, for example, using a ramp program.
[0147] like Figure 5E As further depicted in FIG, during the integration of the analog signals from sub-pixels 370 and 380, a reset voltage from sub-pixels 370 and 380 can be output to a readout system. In some embodiments, the ADC can convert the reset voltage into a digital signal before output. Thereafter, an analog signal proportional to the light incident on the photosensitive elements of sub-pixels 370 and 380 can be output. Accordingly, during Figure 5E In an embodiment of the present invention, the output analog signal and the reset voltage may be used to perform one or more noise suppression and / or mismatch correction.
[0148] Fig. 6A is a superpixel used in the present disclosure (e.g. Figure 1A Superpixel 100 or Figures 1B to 1D Schematic diagram of an exemplary conditional detector 600 in any superpixel of other superpixels, etc. Fig. 6A As shown in FIG. 6 , an array of transistors 603 (eg, M p11 、M p12 、M p13 、M p14 、M p21 、M p22 、M p23 、M p24 、M p31 、M p32 、M p33 and M p34 ) can amplify the output I of the photosensitive element 601 ph The analog signal V ph (PD CD )(For example, V ph Can be with I ph logarithm), and outputs the amplified signal V gp1 Array 603 has a receiving V phThe input terminals and output V gp1 In addition to the output terminal, it can also have one or more control terminals, such as V qDC The control terminal can control the V by, for example, shifting the offset of the input (and / or output). ph Additional terminal V q and V DD The magnification level of array 603 may be controlled.
[0149] The condition detector 600 may further include a pair of comparators 605 with hysteresis. The pair of comparators 605 may output V O + and V O - , which can respectively activate the current source 607a (V pch ) and 607b(V nch ). The current sources 607a and 607b can generate a current of a fixed value and can charge (or discharge) the capacitor 609. The voltage of the capacitor 609 is controlled via a voltage follower, for example, via the control terminal V qDC is fed to the array 603. Accordingly, as the light incident on the photosensor 601 increases, the current from the source 607a increases, thereby increasing the offset of the amplifier formed by the array 603. Accordingly, the input to the pair of comparators 605 will decrease until it reaches zero, and the logic 611 will output a "set" signal. Similarly, as the light incident on the photosensor 601 decreases, the current from the source 607b increases, thereby reducing the offset of the amplifier formed by the array 603. Accordingly, the input to the pair of comparators 605 will increase until it reaches zero, and the logic 611 will output a "set" signal. In some embodiments, the logic 611 will also participate in the handshake (labeled "request" and "reply") (with the external readout system).
[0150] Figure 6B is a superpixel used in the present disclosure (e.g. Figure 1A Schematic diagram of another exemplary condition detector 650 in a superpixel 100, etc. Figure 6B As shown in FIG. 6 , the condition detector 650 includes a photosensitive element 601 (PD CD ) of amplifier 651(-A1). Amplifier 651(-A1) may be a unity gain amplifier. In some embodiments, amplifier 651(-A1) may be inverting. Capacitor 653(C1) may be connected to amplifier 651(-A1).
[0151] The condition detector 650 may further include an amplifier 655 (-A2), a capacitor 657 (C2), and a reset switch 659 connected to a common node in parallel with the capacitor 653 (C1). Accordingly, whenever the condition detector 650 receives a control signal (e.g., an acknowledgement signal from an external readout system), the reset switch 659 may be closed to short-circuit the detector and prepare for a new detection. Alternatively, as described below, the condition detector 650 may be configured to automatically reset after outputting a set signal.
[0152] Amplifier 651 (-A1) (which may be an inverting amplifier) amplifies the changes in voltage from capacitor 653 (C1) to cause these changes to deviate from the V diff The voltage level defined by V is defined by the voltage level of capacitor 651 (C1) at the previous reset signal. In some embodiments, amplifier 651 can be controlled by transistor 661 so that amplifier 651 amplifies only in response to control (e.g., a set signal from a logic circuit of the superpixel, etc.). Whenever V diff When the threshold is exceeded in the negative direction, a “-event” trigger signal is sent to the logic and AER handshake circuits, and a set signal is sent down to the superpixel control logic. Similarly, whenever V diff When a threshold (which may be the same or different than the threshold above) is crossed in the positive direction, a "+ event" trigger is sent to the logic and AER handshake circuits, and a set signal is sent down to the superpixel's control logic. Accordingly, the detector can detect increases in intensity above one threshold and decreases in intensity below the same or a different threshold.
[0153] Even if 6A or Figure 6B The architecture of the invention is described, and any suitable detector is suitable for performing the process of converting incident light to a photosensitive element (such as a PD) CD ) with one or more predetermined conditions (e.g., one or more threshold values). Accordingly, any combination of transistors, capacitors, switches, and / or other circuit components arranged to perform such comparisons may be used in the image sensor of the present disclosure.
[0154] Figure 7 Is Figure 6B Illustration of the triggering generated by the detector. Figure 7 In, V p represents the voltage at capacitor 651 (C1) of detector 650 and is therefore the same voltage as the incident light on photosensitive element 601 (PD CD ) is proportional to the light on the diff is the voltage signal of the detector 650, which is V p is an inverted and amplified version of the photosensitive element 601 (PDCD ) is proportional to the output. Figure 7 As further described in V p increases (depicted as “+ events”) and V p A decrease in V (depicted as a “- event”) may result in the generation of a set signal. diff Yes V p is the inverted and amplified version of p The increase of V diff and vice versa. Figure 7 It is shown how the "+threshold" can be set to be the same (in magnitude) as the "-threshold", but it can also be set to be different. Finally, Figure 7 It is shown that after each trigger (e.g., by closing the reset switch 659), V diff How it is reset to a baseline (depicted as the "reset level").
[0155] Figure 8 Schematic diagrams of different possible geometrical arrangements 810, 820 and 830 of the photodiodes and exposure measurement subpixels (e.g., EM 107) of a condition detector (e.g., CD 101), and the connections from the photodiodes to the associated circuitry (e.g., CD 105) and (e.g., 107). Figure 8 As shown in , multiple exposure measurement sub-pixels (EMs) (e.g., three EMs for superpixel 810, eight EMs for superpixel 820, or five EMs for superpixel 830, etc.) can be paired with a single condition detector (CD) in a superpixel. Figure 8 As shown in the superpixel 810 of FIG. 8 , the multiple EMs can all be connected to the same analog-to-digital converter (ADC) in the superpixel, resulting in sequential readout of measurements from the EMs. Although depicted for superpixel 810, the same connection scheme can be used for Figure 8 The superpixel 820 and superpixel 830 depicted in FIG. FIG. 2B to FIG. 2D As explained, the EMs may be integrated separately or overlappingly, and may be integrated at the same time or at different times. Accordingly, various characteristics of the EMs in a single superpixel may be varied, as explained below.
[0156] Fig. 9 Schematic diagram of an exemplary superpixel provided for high-dynamic range (HDR) imaging and / or red-green-blue (RGB) imaging. Fig. 9 As shown in , different ranges can be used for different EMs on superpixels (e.g., Fig. 9910 by varying the integration time across the EMs). Alternatively, color filters can be applied to different EMs on the superpixel to automatically result in colored digital data being read out from the sensor. Examples of combinations of color filters used at different ranges are shown as Fig. 9 Superpixel 920 and superpixel 930.
[0157] In addition, despite the Fig. 9 Not shown, however, as mentioned above FIG. 2B to FIG. 2D As explained, the conversion gains of multiple EMs in a superpixel may also be varied to increase the diversity of data extracted from a single superpixel. In addition to or in lieu of color filters, the conversion gain may also be varied.
[0158] The output from an image sensor constructed using superpixels of the present disclosure can provide synchronous digital data based on asynchronous analog data rather than synchronous analog data as in many existing image sensors. Moreover, the analog data can be converted to digital data at the pixel level of the image sensor of the present disclosure rather than at the chip level as in many existing image sensors. Accordingly, the digital data can be used directly to construct a differential image (e.g., best suited for encoding video) without the need for post-processing as in existing image sensors. In addition, the digital data output from the image sensor can be significantly reduced by reading out only pixels that meet certain conditions at least clock cycles instead of reading out from all pixels.
[0159] To manufacture different portions of a pixel constructed in accordance with the present disclosure, control detectors, exposure measurement sub-pixels, and other photodiode-related circuits may require a particular voltage range, while other circuits (e.g., control logic and ADC, etc.) may require a different voltage range, e.g., to perform miniaturization and / or scalability of manufacturing. Accordingly, the supply voltage to the photodiode-related circuits may be different and higher than the supply voltage supplied to the other circuits.
[0160] Accordingly, in some embodiments, different semiconductor technologies may be used to implement circuits related to the photodiode and other circuits of the image sensor, respectively. For example, two semiconductor process flows (which may be based on different basic technologies, have different minimum feature sizes, or have different supply voltages, etc.) may be optimized for the two circuit types, respectively. Then, wafer-to-wafer stacking technology may be used to integrate the results of these process flows.
[0161] Fig. 10A An example readout system 1000 is shown in accordance with an embodiment of the present disclosure. The readout system 1000 can be used with an array of superpixels, such as any of the superpixels described herein (e.g., Figure 1A Super Pixel 100, Figure 1B Superpixel 100′, Figure 1C Superpixel 100″ or Figure 1D Superpixel 100″′). Fig. 10A In the example of FIG. 1 , multiple external latches 1001 (eg, similar to Figure 1D The latch 115 described in the foregoing description may store an indication of whether the condition detectors of the various superpixels were triggered prior to the current clock cycle. The readout system 1000 may also include logic circuitry 1003 for implementing clock-controlled readout and a memory array, such as a static random access memory (SRAM) 1005, for storing digital values from a plurality of superpixels. Although described as an SRAM array, the memory array for storing digital values (and the memory arrays described herein and with reference to Figure 1D The enable latch array) may include any non-volatile memory, such as a dynamic random access memory (DRAM), a read-only memory (ROM), a hard disk drive, or a flash memory. The readout system 1000 may include a shift register chain of multiple n-bit registers (e.g., registers 1009-1, 1009-2, 1009-3, ..., 1009-m), each register receiving an output from an n-bit latch (e.g., latch 1007-1, 1007-2, 1007-3, ..., 1007-m) of the ADC of a corresponding superpixel. Accordingly, in Fig. 10A In the example of , there are m superpixels sending outputs to m registers. However, in alternative embodiments, one or more ADCs may share one or more registers.
[0162] like Fig. 10A As shown in , logic 1003 may cause n-bit registers (e.g., registers 1009-1, 1009-2, 1009-3, ..., 1009-m) to transfer the contents stored therein to memory 1005 at each clock cycle. In some embodiments, registers 1009-1, 1009-2, 1009-3, ..., 1009-m may first perform a Correlated Double Sampling (CDS) operation, for example, by subtracting a reset voltage output from a corresponding superpixel from a digital signal output therefrom. In addition, Fig. 10A In the example of , logic 1003 may skip a clock cycle if any of registers 1009 - 1 , 1009 - 2 , 1009 - 3 , . . . , 1009 - m do not have any new information, for example, as indicated by enable latch 1001 .
[0163] Fig. 10B Another example of a readout system 1000' is depicted in accordance with an embodiment of the present disclosure. The system 1000' is similar to Fig. 10A The system 1000 of FIG. 1001 is similar to the system 1000 of FIG. 1002 , but includes a plurality of bits 1011-1, 1011-2, ..., 1009-(m-1), 1009-m. For example, each bit may include an enable latch for each corresponding superpixel. Alternatively, each bit may copy a state (e.g., an "on" state or an "off" state or storage of an analog signal from a condition detector) from a latch 115 of each corresponding superpixel. Fig. 10B As depicted in , the multiple bits can activate corresponding switches (or at least one transistor configured to act as a switch) when a trigger signal is received from the corresponding superpixel, so that during each clock cycle, only the register having the corresponding activated bits is transferred to the memory 1005.
[0164] Fig.11 An exemplary image array 1100 is depicted that implements a column-level ADC 1107 instead of an ADC in each superpixel (e.g., superpixel 1101a, 1101b, etc.). In some embodiments, Figure 2E Exposure measurement system 270′, Figure 2F 290′ Figure 2G 250′ and / or Figure 2H 290″ may be used in array 1100. Additionally or alternatively, ADC 1107 may use Figure 3B System 350.
[0165] like Fig.11 As depicted in , the condition detector of a superpixel (e.g., superpixel 1101a, 1101b, etc.) can be activated and communicate with the condition detector readout 1105 to transmit a corresponding trigger signal to enable latch 1109. In addition, after the exposure measurement circuit of the superpixel (e.g., superpixel 1101a, 1101b, etc.) performs integration and the column-level ADC 1107 converts the corresponding integrated charge into a digital signal based on the enable latch 1109, the exposure measurement control 1103 can deactivate the corresponding enable latch 1109 to prepare for another trigger signal.
[0166] Fig.12 For example, Figure 1D107a, 107b, and 107c, in a superpixel. Each peak in "CD Event" represents the time at which the condition detector 105 detects that the analog signal from the photosensitive element 101 matches the condition, while each sloped portion of "EM Measurement" represents the conversion of the analog signal from one or more photosensitive elements of the subpixels 107a, 107b, and 107c to a digital signal performed by the ADC 109. Fig.12 As depicted in , by using latches, such as Figure 1D The latch 115 of FIG. 114 , detection of a positive state during a transition and output of a digital signal may result in another transition and output during the next clock cycle. Fig.12 The second detection during the N+2 conversion cycle results in Fig.12 However, because the latch is already activated / enabled, the positive state detection while waiting for the transition in the previous state detection will be ignored. For example, because the first detection has triggered the transition cycle in cycle N+2, Fig.12 A second detection during the N+1 cycle will not result in a switching cycle.
[0167] The foregoing description is presented for the purpose of illustration. It is not exhaustive and is not limited to the precise form or embodiments disclosed. Modifications and adjustments to the embodiments will be apparent from consideration of the specification and practice of the disclosed embodiments. For example, the described implementations include hardware, but the systems and methods according to the present disclosure may be implemented with hardware and software. In addition, although certain components have been described as being coupled to each other, these components may be integrated or distributed with each other in any suitable manner.
[0168] In addition, although illustrative embodiments have been described herein, the scope includes any and all embodiments based on equivalent elements, modifications, omissions, combinations (e.g., combinations of aspects in the various embodiments), adaptations, and / or changes of the present disclosure. The elements in the claims are to be interpreted broadly based on the language used in the claims, and are not limited to the examples described in this specification or during the prosecution of this application, which examples should be interpreted as non-exclusive. In addition, the steps of the disclosed methods may be modified in any manner including reordering steps and / or inserting or deleting steps.
[0169] According to the detailed description, the features and advantages of the present disclosure are obvious, and therefore, the appended claims are intended to cover all systems and methods that fall within the true spirit and scope of the present disclosure. As used herein, the indefinite articles "a" and "an" mean "one or more". Similarly, the use of plural forms does not necessarily represent plural forms unless it is clearly plural in a given context. Unless otherwise clearly indicated, words such as "and" or "or" mean "and / or". In addition, since many modifications and variations will easily occur through the study of the present disclosure, it is not desired to limit the present disclosure to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalent forms that may be adopted fall within the scope of the present disclosure.
[0170] Other embodiments will be apparent from consideration of the specification and practice of the embodiments disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosed embodiments being indicated by the following claims.
Claims
1. An image sensor comprising a plurality of superpixels, each superpixel comprising: a first photosensitive element; a detector electrically connected to the first photosensitive element and configured to: detecting when an analog signal proportional to the brightness of light incident on the first photosensor matches a condition; generating a trigger signal when the analog signal proportional to the brightness of light incident on the first photosensor matches the condition; as well as communicating with a conditional detector readout to transmit the trigger signal to set a latch external to the superpixel, wherein the latch is configured to store a digital indication of whether the detector is triggered; at least one second photosensor; and at least one exposure measurement subcircuit electrically connected to the at least one second photosensor and configured to: integrating the analog signal proportional to the brightness of light incident on the at least one second photosensor; and The integrated analog signal is transmitted to an analog-to-digital conversion circuit in response to at least one periodic control signal.
2. The image sensor according to claim 1, wherein: The analog-to-digital circuit is shared by at least two superpixels of the image sensor.
3. The image sensor according to claim 1, wherein: The at least one periodic control signal includes at least one of a row signal and a column signal.
4. The image sensor according to claim 1, wherein: The at least one periodic control signal includes an external readout trigger, and the at least one exposure measurement subcircuit is configured to transmit the integrated analog signal to the analog-to-digital conversion circuit in response to the external readout trigger.
5. The image sensor according to claim 1, wherein: At least two superpixels of the image sensor share a node.
6. The image sensor according to claim 1, wherein: The latch setting is configured to enable the analog-to-digital circuit, and wherein the analog-to-digital circuit is shared by at least two superpixels.
7. The image sensor according to claim 6, wherein: The at least one exposure measurement subcircuit is configured to unset the latch in response to the integrated analog signal having been converted to a digital signal.
8. The image sensor according to claim 6, wherein: Each detector corresponding to each superpixel is associated with a different latch connected to the analog-to-digital circuit shared by the at least two superpixels.
9. The image sensor according to claim 8, wherein: The latch is configured to determine, based on the latch being set, from which exposure measurement subcircuit of the at least one exposure measurement subcircuit the integrated analog signal is to be transmitted, and to enable each determined exposure measurement subcircuit to transmit the integrated analog signal to the analog-to-digital conversion circuit.
10. The image sensor according to claim 8, wherein: The latch is configured to enable transmission of the integrated analog signal from the determined associated one of the at least one exposure measurement subcircuit to the analog-to-digital conversion circuit.
11. The image sensor according to claim 6, wherein: The analog-to-digital circuit shared by the at least two superpixels of the image sensor is configured to receive the integrated analog signal and convert the integrated analog signal into a digital signal based on the state of the latch.
12. The image sensor according to claim 11, wherein: The at least one exposure measurement subcircuit is enabled in response to the trigger signal and is disabled when the integrated analog signal is transmitted from the at least one exposure measurement subcircuit.
13. The image sensor according to claim 6, wherein: Two or more of the at least one exposure measurement subcircuit are associated with a latch connected to the analog-to-digital circuit shared by the at least two superpixels.
14. The image sensor according to claim 1, wherein: The detector is also configured to output the trigger signal to an external readout system.
15. The image sensor according to claim 14, wherein: The detector is configured to receive an acknowledgement signal in response to the trigger signal sensed from the condition detector, and the detector is further configured to reset in response to receiving the acknowledgement signal.
16. The image sensor according to claim 1, wherein: The analog-to-digital circuit is configured to receive the integrated analog signal and convert the integrated analog signal into a digital signal.
17. The image sensor according to claim 1, wherein: The at least one exposure measurement subcircuit is configured to integrate the analog signal onto a node and is further configured to transmit the integrated analog signal from the node to the analog-to-digital circuit in response to the at least one periodic control signal.
18. A method for controlling an image sensor comprising a plurality of superpixels, the method comprising: using a detector of at least one superpixel to detect when an analog signal proportional to the brightness of light incident on the first photosensitive element matches a condition; the detector generates a trigger signal when the analog signal proportional to the brightness of light incident on the first photosensor matches the condition; transmitting the trigger signal from the detector to a conditional detector readout to set a latch external to a superpixel in the plurality of superpixels, wherein the latch is configured to store a digital indication of whether the detector is triggered; integrating, using at least one exposure measurement subcircuit of the superpixel, the analog signal proportional to the brightness of light incident on at least one second photosensitive element; and Using the at least one exposure measurement subcircuit, the integrated analog signal is transmitted to an analog-to-digital conversion circuit in response to at least one periodic control signal.
19. The method according to claim 18, wherein: The analog-to-digital circuit is shared by at least two superpixels of the image sensor.
20. The method according to claim 18, wherein: The at least one periodic control signal includes at least one of a row signal or a column signal.
21. The method according to claim 18, wherein: The at least one periodic control signal comprises an external readout trigger, and wherein the method further comprises: The integrated analog signal is transmitted to the analog-to-digital circuit in response to the external readout trigger using the at least one exposure measurement subcircuit.
22. The method of claim 18, further comprising: The latch is set to enable the analog-to-digital circuit, wherein the analog-to-digital circuit is shared by at least two superpixels.
23. The method according to claim 22, further comprising: Using the at least one exposure measurement subcircuit, the latch is unset in response to the integrated analog signal having been converted to a digital signal.
24. The method according to claim 22, wherein: Each detector is associated with a different latch connected to the analog-to-digital circuit shared by the at least two superpixels, and wherein the method further comprises: determining from which of the at least one exposure measurement subcircuit the integrated analog signal is to be transmitted based on the latch being set; and Each determined exposure measurement subcircuit is enabled to transmit the integrated analog signal to the analog-to-digital conversion circuit.
25. The method according to claim 24, further comprising: reading the integrated analog signal from each exposure measurement subcircuit of the at least one exposure measurement subcircuit; determining which of the at least one exposure measurement subcircuit is associated with a set latch based on reading the integrated analog signal; as well as The integrated analog signal is transmitted from the determined one of the at least one exposure measurement subcircuit associated with the set latch to the analog-to-digital conversion circuit.
26. The method of claim 23, further comprising: Based on the state of the latch, the analog-to-digital circuit shared by the at least two superpixels of the image sensor is enabled to receive the integrated analog signal and convert the integrated analog signal into a digital signal.
27. The method according to claim 26, further comprising: enabling the at least one exposure measurement subcircuit in response to the trigger signal; and The at least one exposure measurement subcircuit is disabled when the integrated analog signal is transmitted from the at least one exposure measurement subcircuit.
28. The method of claim 18, further comprising: The trigger signal is output to the condition detector readout using the detector.
29. The method according to claim 28, further comprising: receiving, using the detector, a response signal in response to the trigger signal read out from the condition detector; as well as The detector is reset in response to receiving the acknowledge signal.
30. The method of claim 18, further comprising: The analog-to-digital circuit is configured to receive the integrated analog signal and convert the integrated analog signal into a digital signal.
31. The method of claim 18, further comprising: integrating the analog signal onto a node using the at least one exposure measurement subcircuit; as well as The integrated analog signal is transmitted from the node to the analog-to-digital circuit in response to the at least one periodic control signal.
