A differential image sensor with digital pixel storage

Abstract

The present invention belongs to the technical field of image sensors, and particularly relates to a differential image sensor with digital pixel storage, which includes a pixel array and a plurality of acquisition circuits. Each of the acquisition circuits includes at least one sensor circuit for generating a sensor signal VSIG according to a light signal illuminating at least one pixel of the corresponding light sensor; at least one analog-to-digital conversion circuit for generating a current digital signal according to the sensor signal VSIG; at least one digital storage circuit for storing a previous digital signal; at least one digital comparison circuit for comparing the level of the current digital signal with the level of the previous digital signal and determining whether the current digital signal has a changing level; and at least one digital output circuit for generating an event signal and outputting it when the current digital signal has a changing level.

Description

Technical Field

[0001] The invention belongs to the technical field of image sensors, and in particular relates to a differential image sensor with digital pixel storage. Background Art

[0002] Image sensors having one ADC per pixel or per small subset of pixels are well known in the prior art and these are typically used for applications where significant processing is done in the pixel, such as X-ray particle tracking or satellite imaging.

[0003] In addition, image sensors that use image pixel level change detection, storage and comparison in the analog domain to generate events are well known. Comparison and storage in the analog domain provide advantages and disadvantages over digital methods. The advantage is that it can be implemented based on a compact structure, and the disadvantage is gradual storage degradation, including "hot pixels", sensitivity to mismatch, and reduced flexibility of light-to-electrical conversion functions.

[0004] Image sensors that share circuitry between pixels are well known, for example, have been proposed by Suh et al., 2020. Sharing is crucial in image sensor design, and sharing enables the creation of dense pixel arrays and high pixel resolution.

[0005] CMOS image sensors are mass-produced with specialized processes in optimized foundry processes, and CMOS digital logic devices are mass-produced with optimized foundry processes. The density of both processes is increasing, especially the density of digital logic processes, and the increased density allows the increase of digital circuits associated with pixel circuits in image sensors. In addition, digital storage at the front stage in the pixel circuit has advantages in terms of storage fidelity and subsequent processing at the storage level.

[0006] Serrano-Gotaredona et al. addressed the mismatch in standard analog memory implementations of dynamic vision sensors using transistors in subthreshold operation. Since operation in this transistor mode only works properly over a limited range, they used global analog feedback to the analog portion of the acquisition circuitry of the entire array to obtain satisfactory operation. Summary of the invention

[0007] Improved differential image sensors with digital pixel storage may be desired, and in particular, it may be desirable to use digital storage and event counting in a time-sequential manner, which is particularly suitable for resource sharing between pixels without performance degradation. In particular, it may be desirable to modulate analog circuit parameters based on stored digital levels or event outputs, because this feedback is effectively implemented with switches under digital control. In particular, it may be desirable to modulate analog circuit parameters based on locally stored pixel digital levels or event outputs, because this feedback is effectively implemented with switches under digital control. In particular, it may be desirable to enable area-optimized small pixel structures, resulting in high-resolution and cost-optimized sensors. In particular, it may be desirable to have improved incremental image sensors with pixel circuits having digital storage of representations of previous illumination intensities in the pixel. In particular, it may be desirable to enable area-optimized small pixel structures that take advantage of digital intra-pixel conversion and storage, resulting in high-resolution and cost-optimized sensors.

[0008] This need is met by the subject matter of the independent claims. Advantageous embodiments are defined in the dependent claims.

[0009] One aspect of the present invention relates to a differential image sensor with digital pixel storage, comprising a pixel array and a plurality of acquisition circuits, wherein the pixel array comprises a plurality of unit pixels, and each of the acquisition circuits corresponds to at least one unit pixel; wherein each of the acquisition circuits comprises:

[0010] at least one sensor circuit, each of the sensor circuits comprising a light sensor, the light sensor configured to generate a sensor signal VSIG based on a light signal illuminating at least one pixel corresponding to the light sensor;

[0011] at least one analog-to-digital conversion circuit, the analog-to-digital conversion circuit being configured to generate a current digital signal according to the sensor signal VSIG;

[0012] at least one digital storage circuit, the digital storage circuit being configured to store a previous digital signal, wherein the previous digital signal is a digital signal generated according to a previous sensor signal VSIG;

[0013] at least one digital comparison circuit, the digital comparison circuit being used to compare the level of the current digital signal with the level of the previous digital signal and determine whether the current digital signal has a changed level;

[0014] At least one digital output circuit, the digital output circuit is used to generate and output an event signal when the current digital signal has a changed level.

[0015] In some embodiments, one acquisition circuit corresponds to at least two unit pixels adjacent to each other.

[0016] In some embodiments, the sensor circuit generates the sensor signal VSIG in response to a control signal, and the sensor signal VSIG changes its correspondence with the light intensity based on the control of the control signal, and / or,

[0017] The sensor circuit generates the sensor signal VSIG in response to a control signal during operation, and the sensor signal VSIG changes its correspondence with light intensity based on the control of the control signal, wherein one control signal acts on the entire pixel array or a local unit pixel in the pixel array.

[0018] In some embodiments, the sensor signal VSIG generated by the sensor circuit has any one of the following corresponding relationships with the light intensity:

[0019] The sensor signal VSIG is linearly related to the light intensity, or,

[0020] The sensor signal VSIG signal is nonlinearly correlated with the light intensity, wherein the nonlinear correlation includes but is not limited to a logarithmic change; or,

[0021] The sensor signal VSIG and the light intensity are in a combination of nonlinear and linear correlation; wherein the above corresponding relationship changes in response to a control signal.

[0022] In some embodiments, a) the analog-to-digital conversion circuit includes a comparator, and the comparator is used to compare the sensor signal VSIG with any of the following scanning signals:

[0023] aa) an analog scan input signal;

[0024] bb) continuous analog scan input signals, wherein each analog scan input signal provides a different ramp signal slope;

[0025] Wherein, a single or continuous analog scan input signal is used to provide to at least one of the analog-to-digital conversion circuits; and / or,

[0026] b) The analog-to-digital conversion circuit is further used to provide at least one digital code in parallel with the scanning signal to at least one unit pixel, and the digital code includes but is not limited to Gray code.

[0027] In some embodiments, the scanning signal is a linear ramp, or,

[0028] The scanning signal is a nonlinear ramp, or,

[0029] The scanning signal changes during operation, and / or,

[0030] The period of the scanning signal varies during operation, and / or,

[0031] The scanning signal is interrupted at a specific time during operation, and the specific time is set or preset during operation.

[0032] In some embodiments, at least one reference pixel is further included, and the reference pixel is used to define a ramp signal; wherein the reference pixel is disposed outside the pixel array.

[0033] In some embodiments, the digital comparison circuit compares the level of the analog signal with the previous digital signal during the scanning period of the analog signal; or,

[0034] The digital comparison circuit compares the level of the analog signal with that of the previous digital signal after scanning.

[0035] In some embodiments, the digital comparison circuit uses dynamic logic to compare the levels, or,

[0036] The digital comparison circuit uses static logic to compare levels.

[0037] In some embodiments, the digital output circuit is used to output the output signal according to the comparison result of the digital comparison circuit in adjacent pixels, or,

[0038] The digital output circuit is used to generate an output signal based on a function of adjacent pixels in a fixed configuration or in a configuration that changes during operation, where the function includes but is not limited to an average function.

[0039] In some embodiments, the digital output circuit is used to generate an event signal based on the output of adjacent pixels, or,

[0040] The output circuit is used to generate an event signal based on the level stored in the adjacent pixel, or,

[0041] The output circuit is used to generate an event signal according to a plurality of pre-stored storage values.

[0042] In some embodiments, the digital output circuit is used to generate an event signal indicating a direction of a level change, and / or,

[0043] The digital output circuit is adapted to generate an event signal indicating only one direction of change of a change level, and / or,

[0044] The digital output circuit is used to generate an event signal indicating the magnitude of the change level, and / or,

[0045] The digital output circuit is used to generate a light intensity indicative of a changed level before and / or after the change.

[0046] In some embodiments, the digital storage circuit is used to provide a stored previous digital signal at an output line of the pixel array, and / or,

[0047] The digital storage circuit is used to selectively provide a stored previous digital signal to a unit pixel having an event output on an output line of the pixel array, and / or,

[0048] the digital storage circuit is for providing a stored previous digital signal at an output line using a time column line, and / or,

[0049] The digital storage circuit is used to write the stored previous digital signal into the unit pixel, and / or,

[0050] The digital storage circuit is used to write a data stream into a previous digital signal;

[0051] The digital storage circuit is used to write from a data stream as an event stream to a stored previous digital signal.

[0052] In some embodiments, the sensor circuit receives feedback from the digital storage circuit corresponding to the digital level stored by the digital storage circuit or the digital level stored by the adjacent pixel, and the sensor circuit generates the sensor signal VSIG based on the feedback card, and / or,

[0053] The analog-to-digital conversion circuit receives feedback from the level of a previously stored digital signal or the level of a previously stored digital signal of an adjacent unit pixel, and adjusts the output current digital signal according to the feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 A circuit structure block diagram of a differential image sensor with digital pixel storage provided by an embodiment of the present invention.

[0056] Among them, the reference numerals in the figure are:

[0057] 100, sensor circuit; 200, analog-to-digital conversion circuit; 300, digital storage circuit; 400, digital comparison circuit; 500, digital output circuit. DETAILED DESCRIPTION

[0058] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0059] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0060] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0061] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0062] In one embodiment of the present invention, Figure 1 As shown, a differential image sensor with digital pixel storage is provided, including a pixel array and a plurality of acquisition circuits, wherein the pixel array includes a plurality of unit pixels, each of the acquisition circuits corresponds to at least one unit pixel, and the acquisition circuit is also referred to as a pixel circuit in the entire specification. Wherein, each of the unit pixels is usually organized into a two-dimensional grid with "rows" and "columns" to form the pixel array. Alternatively, each of the unit pixels is arranged in an irregular shape to form the pixel array.

[0063] Among them, each of the acquisition circuits includes a sensor circuit 100, an analog-to-digital conversion circuit 200, a digital storage circuit 300, a digital comparison circuit 400 and a digital output circuit 500, and the sensor circuit 100, the analog-to-digital conversion circuit 200, the digital storage circuit 300, the digital comparison circuit 400 and the digital output circuit 500 are at least one.

[0064] The sensor circuits 100 each include a light sensor configured to generate a sensor signal VSIG according to a light signal illuminating at least one pixel corresponding to the light sensor. The light sensor may generate the sensor signal VSIG continuously or periodically.

[0065] The analog-to-digital conversion circuit 200 is configured to generate a current digital signal according to the sensor signal VSIG.

[0066] The digital storage circuit 300 is used to store a previous digital signal. The previous digital signal is a digital signal generated according to a previous sensor signal VSIG. The previous sensor signal VSIG corresponds to a signal generated and stored in the digital storage circuit 300 in any previous sampling cycle. Alternatively, the previous digital signal can also be provided and stored externally. In order to ensure the timeliness of data storage, and considering that the level of the digital signal does not degrade over time, the previous digital signal is stored, and the storage circuit of the previous digital signal can be physically implemented in a smaller volume, and the previous digital signal can be used more conveniently and efficiently, such as combination and comparison with other pixels or other signals.

[0067] The digital comparison circuit 400 is used to compare the level of the current digital signal with the level of the previous digital signal, and determine whether the current digital signal has a changed level. The digital output circuit 500 is used to generate and output an event signal when the current digital signal has a changed level.

[0068] In another embodiment of the present invention, the analog-to-digital conversion circuit 200 is a single-slope analog-to-digital converter.

[0069] The analog-to-digital conversion circuit 200 uses different storage ramps to compare with comparison ramps to achieve analog-to-digital conversion, and the comparison between the storage ramp and the comparison ramp generates a difference function.

[0070] Wherein, in another embodiment of the present invention, Figure 1As shown, the analog-to-digital conversion circuit 200 includes a comparator for comparing the sensor signal VSIG with a scanning signal. The conversion from the analog signal (VSIG) to the digital signal is performed by the comparator and the scanning signal applied to the pixel array. The digital representation is copied to the digital storage circuit 300 according to the point at which the comparator output switches.

[0071] The scan signal may be an analog scan input signal, or continuous analog scan input signals, wherein each analog scan input signal provides a different ramp signal slope, and the ramp signal slope is the slope of the storage ramp.

[0072] By providing different ramp signal slopes, the analog scan input signal with different ramp signal slopes is compared with the sensor signal VSIG, thereby improving the accuracy of converting the sensor signal VSIG into a digital signal.

[0073] In another embodiment of the present invention, when the storage ramp and the comparison ramp are compared, each comparison generates a difference value between the two, and the difference value is a voltage difference. The voltage differences are aggregated to form a difference function, which is used to characterize the difference between the storage ramp and the comparison ramp.

[0074] When the slope of the storage ramp is set to be constant, if the slope of the comparison ramp is also constant, there is a specific voltage difference between the storage ramp and the comparison ramp, and the specific voltage difference is constant. This situation is applicable to when the slope of the storage ramp is set to change, the slope of the comparison ramp is constant or changes, and when the slope changes, the levels of the storage ramp and the comparison ramp change. At this time, there is a specific voltage difference between the storage ramp and the comparison ramp, and the specific voltage difference changes with the change of the levels of the storage ramp and the comparison ramp.

[0075] In another embodiment of the present invention, the storage ramp is generated from a digital signal pre-stored in the digital storage circuit 300, and the pre-stored digital signal is generated by a VSIG signal at a previous moment or a previous time period, and is used to characterize the light intensity at a previous moment or a previous time period.

[0076] Next, the stored slope is compared with the comparison slope generated from the current VSIG signal at the current moment, so that a digital signal is generated when the light intensity signal changes.

[0077] In this way, when the storage slope comes from the previous moment, the slope of the storage slope is constant, and when the storage slope comes from the previous time period, the slope of the storage slope is variable. In this way, based on the digital signal of the back end, the current analog-to-digital signal conversion is regulated to improve the conversion efficiency and reduce the occurrence of redundant events.

[0078] In another embodiment of the present invention, the storage slope is pre-set and the slope slope is changed in real time. The change in the slope slope is based on the difference value generated by the comparison between the previous moment and the comparison slope. By changing the slope slope, the difference value between the changed storage slope and the comparison slope is reduced, thereby realizing the change of the difference function.

[0079] In another embodiment of the present invention, the storage slope and the comparison slope are used according to the state of the pixel, wherein the change of the state of the pixel reverses the definition of the storage slope and the comparison slope. That is, as the state of the pixel changes, the comparison slope at the current moment will be defined as the storage slope at the next moment, thereby realizing the judgment of whether the light intensity at the current moment and the next moment changes.

[0080] If digital code A is recorded during the storage ramp, digital code B is read during the comparison ramp, and B≥A, then VSIG during the comparison period is greater than VSIG, that is, the difference between the ramps represents the change in the VSIG signal between the current moment and the previous moment, and thus represents the change in light intensity of the external environment.

[0081] In another embodiment of the invention, a storage ramp is compared with one or more comparison ramps depending on the state of the pixel.

[0082] Alternatively, a plurality of stored ramps are compared to one or more comparison ramps.

[0083] A storage ramp is compared with a comparison ramp to generate a difference function, and a storage ramp is compared with multiple comparison ramps to generate a difference function respectively.

[0084] When a plurality of stored ramps are compared with a comparison ramp, a difference function is generated in each case.

[0085] When a plurality of storage ramps are compared with a plurality of comparison ramps, a difference function is generated respectively.

[0086] In another embodiment of the present invention, one storage ramp is compared with one or more comparison ramps, or multiple storage ramps are compared with one or more comparison ramps; wherein the comparison of one storage ramp and one comparison ramp produces a difference function; and the comparison between the storage ramp and the comparison ramp is not affected by the pixel state.

[0087] In another embodiment of the present invention, a digital comparison based on the storage ramp and the comparison ramp is performed during the ramping period, and the digital comparison value generated by the digital comparison is not stored.

[0088] The ramp device stores the time when the ramp and the comparison ramp exist, and only compares without storing the comparison value temporarily, so as to suppress the signal change caused by the local light intensity change and reduce the occurrence of invalid events.

[0089] In another embodiment of the present invention, a digital comparison based on the storage ramp and the comparison ramp is performed after the ramp period, and a digital comparison value resulting from the digital comparison is stored.

[0090] The acquisition circuit can detect the light intensity that changes over a period of time. If the digital comparison circuit 400 detects that there is a difference between the previous digital signal stored previously and the subsequent digital signal, and the difference meets a preset given standard, an event is generated, that is, an event signal is generated, and the fact of the change is reported as event information to a device at the output end, such as an image sensor.

[0091] In another embodiment of the present invention, one acquisition circuit corresponds to at least two unit pixels, that is, one acquisition circuit can perform signal acquisition on at least two unit pixels. Preferably, at least two unit pixels are adjacent to each other.

[0092] In another embodiment of the present invention, Figure 1 As shown, the sensor circuit 100 generates the sensor signal VSIG in response to a control signal, and the sensor signal VSIG changes its correspondence with the light intensity based on the control of the control signal, and / or, the sensor circuit 100 generates the sensor signal VSIG in response to a control signal during operation, and the sensor signal VSIG changes its correspondence with the light intensity based on the control of the control signal.

[0093] The control signal acts on the entire pixel array or a local unit pixel in the pixel array. The operation period is the use period of the differential image sensor with digital pixel storage.

[0094] In another embodiment of the present invention, the sensor signal VSIG generated by the sensor circuit 100 and the light intensity have any one of the following corresponding relationships:

[0095] The sensor signal VSIG signal is linearly correlated with the light intensity, or the sensor signal VSIG signal is nonlinearly correlated with the light intensity, wherein the nonlinear correlation includes but is not limited to logarithmic change; or the sensor signal VSIG signal is a combination of nonlinear and linear correlation with the light intensity; wherein the above corresponding relationship changes in response to a control signal.

[0096] The sensor signal VSIG is linearly related to the light intensity, which means that the optical illumination intensity signal is converted into an analog electrical signal (VSIG) with a linear relationship. This conversion relationship is suitable for application environments with low light and little light change.

[0097] The sensor signal VSIG is nonlinearly related to the light intensity, which means that the optical illumination intensity signal is converted into an analog electrical signal with a logarithmic relationship. This conversion relationship is advantageous in terms of the high dynamic range of the optical illumination signal, because the output analog signal will not be saturated over a wide range of illumination intensities, so as to more truly reflect the changes in the optical illumination intensity.

[0098] The sensor signal VSIG has a combination of nonlinear and linear correlation with the light intensity, which means that the optical illumination intensity signal is converted into an analog electrical signal having a combination of a logarithmic relationship and a linear relationship or a similar nonlinear functional relationship.

[0099] During actual operation, the specific conversion correspondence is selected or set according to actual needs.

[0100] Specifically, a signal source, such as a signal generating circuit, may be separately provided to provide the control signal, and based on actual use, the control signal is a signal that changes from time to time. The change of the control signal is derived from the global light intensity or the actual operation during operation. Based on the control signal, the corresponding relationship between the sensor signal VSIG and the light intensity is converted to meet the use requirements of data in different scenarios.

[0101] The single or continuous analog scan input signal is used to provide at least one of the analog-to-digital conversion circuits 200. That is, a scan signal can scan one or more analog-to-digital conversion circuits 200 in one acquisition circuit, or scan one or more analog-to-digital conversion circuits 200 in multiple acquisition circuits at the same time, thereby improving the analog-to-digital conversion efficiency and the data processing speed.

[0102] And / or, b) the analog-to-digital conversion circuit 200 is further used to provide a digital code in parallel with the scanning signal to at least one pixel, and the digital code stored in the digital storage circuit 300 is a digital code value output by the comparator after comparison. The digital code is preferably a Gray code, which makes full use of the Gray code without synchronization with the digital storage, and only one bit changes at a time, and the codes on either side of the Gray code are valid, and the Gray code has fewer edges, which reduces power consumption during data processing.

[0103] In another embodiment of the present invention, the digital output circuit 500 is used to generate and output an event signal when the current digital signal has a changing level and the changing level exceeds a preset specific threshold, wherein the specific threshold is set based on a fixed intensity change rate of the optical signal.

[0104] The light signal is the signal emitted by the artificial light source. Based on the different voltages of the power supply, the artificial light source usually has a regularly changing intensity. That is, the artificial light sources on the market have regularly changing intensities due to the limitations of their own structure and control. For example, the fixed intensity of an incandescent lamp changes by 14%, the fixed intensity of a fluorescent lamp changes by 8%, and the fixed intensity of an LED changes by 9%. If the digital output circuit 500 generates an event signal and outputs the required specific threshold value and is set to be less than the fixed intensity change of the artificial light source, such as 2%-3%, the modulation of the artificial lighting will easily lead to a large number of redundant events.

[0105] Therefore, in order to suppress or eliminate redundant events caused by fixed intensity changes of the artificial light source, it is necessary to set the specific threshold based on the fixed intensity change rate of the light signal.

[0106] Specifically, when there is more than one optical signal, the specific threshold is greater than the fixed intensity change rate corresponding to the optical signal, thereby reducing the generation of redundant events.

[0107] When there are multiple optical signals, and each of the optical signals has a different fixed intensity change rate, the specific threshold is greater than the largest fixed intensity change rate among the fixed intensity change rates, thereby ensuring that it is not affected by the fixed intensity change rates of all the optical signals themselves. This setting is suitable for the situation where the number of inherent optical signals in a known background environment and the redundant events caused by all inherent optical signals in the background need to be shielded.

[0108] For example, when conducting relevant experiments, the background environment of the experiment has three inherent optical signals. In order to obtain the event changes caused by optical signals other than the three optical signals in the background environment, it is necessary to shield the redundant events caused by the three optical signals in the background environment during the experiment. Therefore, at this time, setting the specific threshold value greater than the largest fixed intensity change rate among the three fixed intensity change rates corresponding to the three optical signals in the background environment also meets the experimental requirements.

[0109] In this case, the fixed intensity change rate of the optical signal other than the three optical signals in the background environment is greater than the specific threshold value. The specific threshold value is generally set slightly greater than the largest fixed intensity change rate among the three fixed intensity change rates corresponding to the three optical signals in the background environment, so as to ensure that the acquisition of events caused by the optical signal of the target is not affected under the premise of shielding redundant events generated by the optical signal in the background.

[0110] When testing the differential image sensor with digital pixel storage and synchronous sampling,

[0111] In another embodiment of the present invention, the specific threshold value may be preset or set in real time according to changes in the optical signal during operation.

[0112] In another embodiment of the present invention, the repetition rate of the scanning signal is pre-set and used to reduce or eliminate redundant events caused by changes in light intensity of the light signal illuminating the light sensor. Specifically, the repetition rate of the scanning signal is the frequency at which the scanning signal is provided to the analog-to-digital conversion circuit 200.

[0113] The repetition rate of the scanning signal is set based on at least one preset repetition rate, and the preset repetition rate matches the modulation frequency of the light signal irradiating the light sensor.

[0114] It should be understood that the repetition rate of the scanning signal is selected from one or more preset repetition rates, each of which corresponds to the light source modulation of the light signal irradiating the light sensor. It should be understood that when the fixed intensity change rate of the light signal itself is constant, the redundant events generated in a time period are certain, and in this time period, if the scanning signal continues to scan and the repetition rate is always higher than the modulation frequency of the light signal irradiating the light sensor, then all redundant events generated in the event segment will be acquired. Therefore, in order to reduce the number of redundant events, it is necessary to adjust the repetition rate of the scanning signal.

[0115] When the modulation frequency of the light signal irradiating the light sensor is multiple, such as 50HZ, 100HZ and 200HZ, that is, at 50HZ, 100HZ and 200HZ, the number of intensity changes of the corresponding light intensity in one cycle is 100 times, 200 times and 400 times, respectively, and the number of redundant events generated is 100, 200 and 400. At this time, in order to reduce redundant events, the repetition rate of the scanning signal is set based on the repetition rates corresponding to 50HZ, 100HZ and 200HZ, such as the repetition rate can be set to be less than the number of intensity changes of the light intensity, such as setting the repetition rate of the scanning signal to scan 25 times in one cycle, and obtaining one redundant event in one scan, and obtaining 25 redundant events in total, thereby reducing the number of redundant events.

[0116] Preferably, the repetition rate of the scanning signal is a multiple of the modulation frequency of the light signal irradiating the light sensor.

[0117] In another embodiment of the present invention, the scanning signal is a linear ramp, or a nonlinear ramp, and the nonlinear ramp includes but is not limited to an exponential ramp. The scanning signal can be generated centrally, and considering that the light intensity mostly changes nonlinearly, it is preferred to set the scanning signal to be nonlinear, so the signal generating device on the device can be used to generate the scanning signal to improve the sharing of the hardware circuit and save the volume required for the hardware circuit.

[0118] In another embodiment of the present invention, the scanning signal is changed during operation, and / or the period of the scanning signal is changed during operation, and / or the scanning signal is interrupted at a specific time during operation, and the specific time is set or preset during operation. The scanning signal of the linear ramp or nonlinear ramp can be changed during operation to match different sensor signals VSIG to achieve real-time matching and comparison.

[0119] Specifically, the scanning signal is repeatedly scanned out according to a preset scanning rule during operation. During operation, the repetition of the scanning signal can be interrupted or extended as needed to change the repetition rate of the scanning.

[0120] In another embodiment of the present invention, Figure 1 As shown, the differential image sensor with digital pixel storage further includes at least one reference pixel, and the reference pixel is used to define a ramp signal. The reference pixel is preset or automatically set in real time during operation, and is associated with the sensor signal VSIG. The reference pixel can be set after a specific numerical representation or a numerical range representation of the sensor signal VSIG is obtained, and the reference pixel is used to define the ramp signal, so as to achieve a more accurate and non-redundant comparison, so as to more efficiently promote subsequent data generation.

[0121] In another embodiment of the present invention, the reference pixel is arranged outside the pixel array.

[0122] In another embodiment of the present invention, Figure 1 As shown, the digital comparison circuit 400 compares the level of the previous digital signal with that of the previous digital signal during the scanning of the analog signal; or, the digital comparison circuit 400 compares the level of the previous digital signal with that of the previous digital signal after the scanning of the analog signal.

[0123] When the digital comparison circuit 400 compares the level of the previous digital signal with the level of the previous digital signal during the scanning period of the analog signal, it is not necessary to store the A / D output value, but only the comparison result.

[0124] In the comparison process after scanning the analog signal, a sequence, such as bitwise, may be used.

[0125] In another embodiment of the present invention, the digital comparison circuit 400 uses dynamic logic to compare the levels, or the digital comparison circuit 400 uses static logic to compare the levels.

[0126] When the comparison is required in a low power consumption state, static logic is selected for level comparison. When the comparison speed is required to be fast and the output result needs to be maintained for a long time, dynamic logic is selected for level comparison.

[0127] In another embodiment of the present invention, feedback from the results of previous comparisons is used to perform digital comparisons, ie, comparisons of the next moment with the previous moment, thereby facilitating noise suppression and reducing the generation of false events.

[0128] In another embodiment of the present invention, the comparison of digital signals and the output of event signals require specific differences to be realized, thereby reducing the number of events generated and improving the resolution of events. The specific differences may be generated by the signal level, operation mode, level of adjacent pixels, previous level of pixels, previous time of pixels, and events generated by adjacent pixels.

[0129] In another embodiment of the present invention, the digital output circuit 500 is used to output the output signal according to the comparison result of the digital comparison circuit 400 in adjacent pixels, or, the digital output circuit 500 is used to generate an output signal according to a function of adjacent pixels in a fixed configuration or a function in a configuration that changes during operation, wherein the function includes but is not limited to an average function.

[0130] In another embodiment of the present invention, the digital output circuit 500 is used to generate an event signal based on the output of an adjacent pixel, or the output circuit is used to generate an event signal based on the level stored in the adjacent pixel, or the output circuit is used to generate an event signal based on a plurality of pre-stored storage values.

[0131] In another embodiment of the present invention, Figure 1 As shown, the digital output circuit 500 is used to generate an event signal indicating a direction of a level change, and / or, the digital output circuit 500 is used to generate an event signal indicating only a change direction of a changed level, and / or, the digital output circuit 500 is used to generate an event signal indicating an amplitude of a changed level, and / or, the digital output circuit 500 is used to generate a light intensity indicating a changed level before and / or after the change.

[0132] In another embodiment of the present invention, the generation of an event signal of one acquisition circuit depends on the event signal of another adjacent acquisition circuit, thereby filtering the events to reduce the number of false events. Alternatively, the generation of an event signal of one acquisition circuit depends on the digital level stored by another adjacent acquisition circuit, thereby filtering the events to provide higher sensitivity near the edge.

[0133] In another embodiment of the present invention, generation of an event signal of an acquisition circuit depends on a plurality of digital signals stored in a digital storage circuit 300 in the acquisition circuit.

[0134] In another embodiment of the present invention, when the output of the event signal is a single event bit, such as an up event, it represents an increased light intensity. When the output of the event signal is two event bits, such as up and down, the event output can indicate whether the light intensity or pixel signal VSIG is greater than or less than the stored light intensity level or stored VSIG. The event output also includes the change amplitude of the digital signal, based on which the image can be perfectly reconstructed from the event information, thereby obtaining the currently detected image.

[0135] In another embodiment of the present invention, Figure 1 As shown, the digital storage circuit 300 is used to provide a stored previous digital signal at an output line of a pixel array, and / or, the digital storage circuit 300 is used to selectively provide a stored previous digital signal for a unit pixel with an event output on an output line of a pixel array, and / or, the digital storage circuit 300 is used to provide a stored previous digital signal at an output line using a time column line, and / or, the digital storage circuit 300 is used to write the stored previous digital signal to the unit pixel, and / or, the digital storage circuit 300 is used to write a data stream to the previous digital signal; the digital storage circuit 300 is used to write from a data stream as an event stream to a stored previous digital signal.

[0136] In another embodiment of the present invention, a stored digital level can be written into the pixel arrangement as a reference for comparison, and a data stream can be written into the stored digital level to intentionally provide a time-related reference for event generation, wherein the stored digital level is the previous digital signal.

[0137] In another embodiment, the data stream is typically generated by a sensor.

[0138] The digital storage circuit 300 provides at least one output line, which is provided to at least one digital output circuit 500 , so that one digital output circuit 500 or multiple digital output circuits 500 can share the same output line.

[0139] The previous digital signal stored in the digital storage circuit 300 can be written by an external pixel array, or the signal output at the previous moment can be written into the stored previous digital signal, thereby characterizing the signal change process. For example, when the digital signal output at time t1 is p, then at the next time t2, time t1 is the previous moment, and the output at time t1 is p, then p is the previous digital signal for time t2. The previous digital signal is used for reference. At time t2, it is output only when the signal changes compared with the reference, that is, compared with time t1, and then the event signal is output, that is, the signal is output only when there is a change, thereby reducing the generation of redundant data.

[0140] In another embodiment of the present invention, the digital storage values ​​from the target area are selectively read, ie, by random access.

[0141] In another embodiment of the present invention, a sampling rate may also be set, and the sampling rate is the frequency of the acquisition of event signal generation. The sampling rate may be changed from time to time during operation based on the electrical configuration, such as the hardware circuit. Different sampling rates also determine the difference in power consumption and event generation rate. When there is no event signal or non-use time for a long time, an extremely low power consumption mode may be achieved by reducing the sampling rate, or the power consumption may be reduced by completely interrupting the sampling for a period of time.

[0142] In another embodiment of the present invention, the sensor circuit 100 receives feedback corresponding to the digital level stored in the digital storage circuit 300 or the digital level stored in the adjacent pixel from the digital storage circuit 300, and the sensor circuit 100 generates the sensor signal VSIG based on the feedback card, and / or the analog-to-digital conversion circuit 200 receives feedback from the level of the stored previous digital signal or the level of the previous digital signal stored in the adjacent unit pixel, and adjusts the output current digital signal according to the feedback. By setting the feedback, the generated sensor signal VSIG is more accurate.

[0143] In another embodiment of the present invention, the sensor circuit 100 is used to provide an analog representation of the sensor signal VSIG, and / or the differential image sensor with digital pixel storage is composed of multiple semiconductor layers, each of which has a different optimized function.

[0144] Specifically, the semiconductor process type of each layer is optimized for the function of the layer. For example, the performance is improved, such as improving heat dissipation. Alternatively, the silicon area of ​​the semiconductor layer is reduced to meet the actual use requirements in accordance with the requirements of device size and device cost, as long as the physical structure of the changed semiconductor layer can improve the accuracy and efficiency of light intensity changes and reduce the electrical interference between the control signal line and the photoelectric sensor circuit 100.

[0145] The output mode of the event signal includes a continuous output mode and an intermittent output mode, and the output mode of the event signal switches between the continuous output mode and the intermittent output mode in response to an interrupt control signal, and the interrupt control signal is generated by the digital output circuit 500. By setting the continuous output mode, it is suitable for the situation where the event signal is generated at a high frequency, that is, a high power consumption state. The intermittent output mode is suitable for a low power consumption state, that is, when the frequency of event signal generation is not high, or it is understood that the event signal is not generated for a long time.

[0146] When in the intermittent output mode, the event signal is intermittently output at a preset specific time interval. That is, the event signal is first stored and lasts for a period of time, and then the event signal is output at the same time after being stored for a period of time. The specific time interval is set by yourself, such as 5 minutes, 10 minutes, even half an hour, or 6 hours. Taking 5 minutes as an example, the event signal is output every 5 minutes.

[0147] In this way, by utilizing the intermittent output and the characteristics that the digital signal is easy to store in the digital output circuit 500 without data loss, low power consumption is ensured while ensuring lossless output of events.

[0148] When in the continuous output mode, the event signal is output immediately when a level change of the current digital signal is detected.

[0149] In another embodiment of the present invention, when the digital output circuit 500 detects that the current digital signal has a level change within a first specific time period and the frequency of the level change does not exceed a preset frequency, the digital output circuit 500 controls the output mode of the event signal to be an intermittent output mode. The first specific time period is set as needed, such as 1 minute. The preset frequency is pre-set, and when the frequency of the level change does not exceed the preset frequency, it indicates that the frequency of generating the event signal at this time is low, so it can be set to a low power consumption mode, that is, an intermittent output mode.

[0150] In another embodiment of the present invention, when the digital output circuit 500 detects that the current digital signal has a changing level and the frequency of the level change exceeds a preset frequency within a first specific time period, the frequency of event signal generation is high and it is a high power consumption mode. At this time, the digital output circuit 500 controls the output mode of the event signal to a continuous output mode.

[0151] In another embodiment of the present invention, when the digital storage circuit 300 does not obtain the digital signal generated according to the previous sensor signal VSIG within the second specific time period, it means that no new digital signal is generated within the second specific time period, and thus it means that the external environment has not changed, that is, no new event signal will be generated at this time, and it can be determined that this is a low power consumption mode, and thus a first command signal is generated and sent to the digital output circuit 500, and the digital output circuit 500 controls the output mode of the event signal to an intermittent output mode according to the first command signal.

[0152] Furthermore, when the digital storage circuit 300 obtains at least one digital signal generated according to the previous sensor signal VSIG within a second specific time period, it indicates that the external environment continues to change, so a second command signal is generated and sent to the digital output circuit 500. The digital output circuit 500 controls the output mode of the event signal to a continuous output mode according to the second command signal.

[0153] In addition, the storage of digital signals will not degrade over time, and information can be retained indefinitely without losing information or needing to refresh image frames. In certain applications, such as monitoring, different trigger environments, images often do not change for a long time, and by setting the switching of the event output mode, the observation rate can be greatly reduced and power can be saved.

[0154] In another embodiment of the present invention, each of the acquisition circuits may be connected to a common or multiple common event output lines. A group of acquisition circuits may share any one of the digital output circuit 500, the digital comparison circuit 400, the digital storage circuit 300, and the analog-to-digital conversion circuit 200.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A differential image sensor with digital pixel storage, It is characterized in that The invention comprises a pixel array and a plurality of acquisition circuits, wherein the pixel array comprises a plurality of unit pixels, and each of the acquisition circuits corresponds to at least one unit pixel; wherein each of the acquisition circuits comprises: at least one sensor circuit, each of the sensor circuits comprising a light sensor, the light sensor configured to generate a sensor signal VSIG based on a light signal illuminating at least one pixel corresponding to the light sensor; at least one analog-to-digital conversion circuit, the analog-to-digital conversion circuit being used to generate a current digital signal according to the sensor signal VSIG, wherein the analog-to-digital conversion circuit is a single-slope analog-to-digital converter; at least one digital storage circuit, the digital storage circuit being configured to store a previous digital signal, wherein the previous digital signal is a digital signal generated according to a previous sensor signal VSIG; at least one digital comparison circuit, the digital comparison circuit being used to compare the level of the current digital signal with the level of the previous digital signal and determine whether the current digital signal has a changed level; at least one digital output circuit, the digital output circuit being used to generate and output an event signal when the current digital signal has a changed level; Wherein, the analog-to-digital conversion circuit uses different storage ramps to compare with comparison ramps to achieve analog-to-digital conversion, and the comparison of the storage ramp and the comparison ramp generates a difference function; The sensor circuit generates the sensor signal VSIG in response to a control signal, and the sensor signal VSIG changes its corresponding relationship with the light intensity based on the control of the control signal, and / or, The sensor circuit generates the sensor signal VSIG in response to a control signal during operation, and the sensor signal VSIG changes its corresponding relationship with light intensity based on the control of the control signal, wherein one control signal acts on the entire pixel array or a local unit pixel in the pixel array; The sensor signal VSIG generated by the sensor circuit has any one of the following corresponding relationships with the light intensity: The sensor signal VSIG is linearly related to the light intensity, or, The sensor signal VSIG signal is nonlinearly correlated with the light intensity, wherein the nonlinear correlation includes but is not limited to a logarithmic change; or, The sensor signal VSIG and the light intensity are in a combination of nonlinear and linear correlation; wherein the above corresponding relationship changes in response to a control signal.

2. The differential image sensor with digital pixel storage according to claim 1, It is characterized in that There is a specific voltage difference between the storage ramp and the comparison ramp, and the specific voltage difference is constant.

3. The differential image sensor with digital pixel storage according to claim 1, It is characterized in that There is a specific voltage difference between the storage ramp and the comparison ramp, and the specific voltage difference changes as the levels of the storage ramp and the comparison ramp change.

4. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that The storage ramp and the comparison ramp are used depending on the state of the pixel, wherein a change in the state of the pixel inverts the definition of the storage ramp and the comparison ramp.

5. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that Depending on the state of the pixel, a storage ramp is compared with one or more comparison ramps, or, comparing a plurality of stored ramps to one or more comparison ramps; Therein, a storage ramp and a comparison ramp are compared to produce a difference function.

6. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that A stored ramp is compared to one or more comparison ramps, or comparing a plurality of stored ramps to one or more comparison ramps; wherein a storage ramp and a comparison ramp are compared to generate a difference function; Furthermore, the comparison between the storage ramp and the comparison ramp is not affected by the pixel state.

7. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that During the ramping period, a digital comparison based on the storage ramp and the comparison ramp is performed, and a digital comparison value resulting from the digital comparison is not stored.

8. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that A digital comparison based on the stored ramp and the comparison ramp is performed after the ramp period, and a digital comparison value resulting from the digital comparison is stored.

9. The differential image sensor with digital pixel storage according to any one of claims 1 to 3, It is characterized in that One acquisition circuit corresponds to at least two unit pixels adjacent to each other.

Images