A differential image sensor with digital pixel storage and synchronous sampling

Abstract

The present invention belongs to the technical field of image sensors, and particularly relates to a differential image sensor with digital pixel storage and synchronous sampling, including a pixel array and a plurality of acquisition circuits. Each of the acquisition circuits includes at least one sensor circuit for generating a sensor signal VSIG according to a light signal illuminating at least one pixel of the corresponding light sensor; at least one analog-to-digital conversion circuit for generating a current digital signal according to the sensor signal VSIG; at least one digital storage circuit for storing a previous digital signal; at least one digital comparison circuit for comparing the level of the current digital signal with the level of the previous digital signal and determining whether there is a changing level in the current digital signal; and at least one digital output circuit for generating an event signal and outputting it when there is a changing level in the current digital signal.

Description

Technical Field

[0001] The present invention belongs to the technical field of image sensors, and in particular relates to a differential image sensor with digital pixel storage and synchronous sampling. Background Art

[0002] In the prior art, image sensors in which each pixel or each small subset of pixels has an ADC are well-known, and these are generally used in applications where important processing is carried out in pixels, such as X-ray particle track tracing or satellite imaging.

[0003] In addition, image sensors that use image pixel-level change detection, storage, and comparison in the analog domain to generate events are well-known. Comparison and storage in the analog domain have both advantages and disadvantages compared to digital methods. The advantages are that they can be implemented based on a compact structure, while the disadvantages include gradual storage degradation, including "hot pixels", sensitivity to mismatches, and reduced flexibility of the photoelectric conversion function.

[0004] Image sensors that share circuitry between pixels are well-known. For example, Suh et al. proposed it in 2020. Sharing is crucial in image sensor design, and based on sharing, a dense pixel array and high pixel resolution can be created.

[0005] CMOS image sensors are mass-produced in a specialized process in an optimized foundry process, and CMOS digital logic devices are mass-produced in an optimized foundry process. The density of both processes is increasing, especially the density of the digital logic process. The increased density allows for an increase in the digital circuitry associated with the pixel circuitry in the image sensor. In addition, digital storage in the front stage of the pixel circuitry has advantages in terms of storage fidelity and subsequent processing at the storage level.

[0006] Serrano Gotalredoña et al. solved the problem of mismatches in the standard analog storage implementation of dynamic vision sensors that use transistors in subthreshold operation. Since the operation in this transistor mode can only work properly within a limited range, they use global analog feedback to the analog part of the acquisition circuitry of the entire array to obtain satisfactory operation. Summary of the Invention

[0007] An incremental image sensor with digital pixel storage and synchronous sampling may need to be improved. In particular, it may be necessary to use digital storage and event calculation in a chronological manner, which is particularly suitable for resource sharing between pixels without degrading performance. In particular, it may be necessary to modulate analog circuit parameters based on stored digital levels or event outputs, as this feedback is effectively implemented by switches under digital control. In particular, it may be necessary to modulate analog circuit parameters based on the digital levels or event outputs of locally stored pixels, as this feedback is effectively implemented by switches under digital control. In particular, it may be necessary to be able to implement a small pixel structure for area optimization, resulting in a high-resolution and cost-optimized sensor. In particular, it may be necessary to improve the incremental image sensor with pixel circuits that have digital storage of the representation of the previous illumination intensity in the pixels. In particular, it may be necessary to utilize the advantages of digital pixel conversion and storage to implement a small pixel structure for area optimization, resulting in a high-resolution and cost-optimized sensor.

[0008] The subject matter of the independent claims can meet this need. Advantageous embodiments are defined in the dependent claims.

[0009] One aspect of the present invention relates to an incremental image sensor with digital pixel storage and synchronous sampling, including a pixel array and a plurality of acquisition circuits. The pixel array includes a plurality of unit pixels, and each acquisition circuit corresponds to at least one unit pixel; wherein each acquisition circuit includes:

[0010] At least one sensor circuit, each sensor circuit including a photosensor for generating a sensor signal VSIG according to a light signal illuminating at least one pixel corresponding to the photosensor;

[0011] At least one analog-to-digital conversion circuit for generating a current digital signal according to the sensor signal VSIG;

[0012] At least one digital storage circuit for storing a previous digital signal, where the previous digital signal is a digital signal generated according to a previous sensor signal VSIG;

[0013] At least one digital comparison circuit for comparing the level of the current digital signal with the level of the previous digital signal and determining whether the current digital signal has a changing level;

[0014] At least one digital output circuit for generating an event signal and outputting it when the current digital signal has a changing level.

[0015] In some embodiments, one acquisition circuit corresponds to at least two adjacent unit pixels.

[0016] In some embodiments, the sensor circuit generates the sensor signal VSIG in response to a control signal, and the sensor signal VSIG changes its corresponding relationship with the light intensity based on the control of the control signal, and / or,

[0017] The sensor circuit generates the sensor signal VSIG in response to a control signal during operation, and the sensor signal VSIG changes its corresponding relationship with the light intensity based on the control of the control signal, wherein one control signal acts on the entire pixel array or a local unit pixel in the pixel array.

[0018] In some embodiments, the sensor signal VSIG generated by the sensor circuit has any one of the following corresponding relationships with the light intensity:

[0019] The sensor signal VSIG is linearly correlated with the light intensity, or,

[0020] The sensor signal VSIG is non-linearly correlated with the light intensity, wherein the non-linear correlation includes but is not limited to logarithmic variation; or,

[0021] The sensor signal VSIG has a combined relationship of non-linear and linear correlation with the light intensity; wherein the above corresponding relationship changes in response to a control signal.

[0022] In some embodiments, a) the analog-to-digital conversion circuit includes a comparator, and the comparator is used to compare the sensor signal VSIG with any one of the following scanning signals:

[0023] aa) an analog scanning input signal;

[0024] bb) consecutive analog scanning input signals, wherein each analog scanning input signal provides a different slope of the ramp signal;

[0025] Wherein, the single or consecutive analog scanning input signals are used to be provided to at least one of the analog-to-digital conversion circuits; and / or,

[0026] b) The analog-to-digital conversion circuit is further used to provide at least one digital code parallel to the scanning signal to at least one unit pixel, and the digital code includes but is not limited to Gray code.

[0027] In some embodiments, the scanning signal is a linear ramp, or,

[0028] The scanning signal is a non-linear ramp, or,

[0029] The scanning signal changes during operation, and / or,

[0030] The period of the scanning signal changes during operation, and / or,

[0031] The scanning signal is interrupted at a specific time during operation, and the specific time is set or preset during operation.

[0032] In some embodiments, it further includes at least one reference pixel for defining a ramp signal; wherein, the reference pixel is disposed outside the pixel array.

[0033] In some embodiments, the digital comparison circuit compares the level with that of the previous digital signal during the scanning of the analog signal; or,

[0034] The digital comparison circuit compares the level with that of the previous digital signal after the scanning of the analog signal.

[0035] In some embodiments, the digital comparison circuit uses dynamic logic to compare levels, or,

[0036] The digital comparison circuit uses static logic to compare levels.

[0037] In some embodiments, the digital output circuit is used to output the output signal according to the comparison result of the digital comparison circuit in adjacent pixels, or,

[0038] The digital output circuit is used to generate an output signal according to a function of adjacent pixels in a fixed configuration or a function in a configuration that changes during operation, wherein the function includes but is not limited to an averaging function.

[0039] In some embodiments, the digital output circuit is used to generate an event signal according to the outputs of adjacent pixels, or,

[0040] The output circuit is used to generate an event signal according to the levels stored in adjacent pixels, or,

[0041] The output circuit is used to generate an event signal according to a plurality of pre-stored stored values.

[0042] In some embodiments, the digital output circuit is used to generate an event signal indicating the direction of the level change, and / or,

[0043] The digital output circuit is applied to generate an event signal indicating only one change direction of one changed level, and / or,

[0044] The digital output circuit is used to generate an event signal indicating the amplitude of the changed level, and / or,

[0045] The digital output circuit is used to generate an event signal indicating the light intensity before and / or after the level change.

[0046] In some embodiments, the digital storage circuit is used to provide a stored previous digital signal at the output line of the pixel array, and / or,

[0047] the digital storage circuit is used to selectively provide a stored previous digital signal for a unit pixel having an event output on the output line of the pixel array, and / or,

[0048] the digital storage circuit is used to provide a stored previous digital signal at the output line using a time column line, and / or,

[0049] the digital storage circuit is used to write the stored previous digital signal into a unit pixel, and / or,

[0050] the digital storage circuit is used to write a data stream into the previous digital signal;

[0051] the digital storage circuit is used to write from a data stream as an event stream into the stored previous digital signal.

[0052] In some embodiments, the sensor circuit receives feedback corresponding to the digital level stored in the digital storage circuit or the digital level of an adjacent pixel stored in the digital storage circuit, and the sensor circuit generates a sensor signal VSIG based on the feedback, and / or,

[0053] The analog-to-digital conversion circuit receives feedback from the level of the stored previous digital signal or the level of the previous digital signal stored in an adjacent unit pixel, and adjusts the current digital signal output according to the feedback. Description of the Drawings

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 It is a circuit structure block diagram of a differential image sensor with digital pixel storage and synchronous sampling provided for the embodiments of the present invention.

[0056] Among them, the reference numerals in the drawings:

[0057] 100, sensor circuit; 200, analog-to-digital conversion circuit; 300, digital storage circuit; 400, digital comparison circuit; 500, digital output circuit. Detailed Embodiments

[0058] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0059] In the description of the embodiments of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.

[0060] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, the meaning of "a plurality" is two or more unless otherwise specifically defined.

[0061] In the embodiments of the present invention, unless otherwise clearly defined and limited, the terms "mounted", "connected", "coupled", "fixed", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0062] In one embodiment of the present invention, as Figure 1 shown, a differential image sensor with digital pixel storage and synchronous sampling is provided, which includes a pixel array and a plurality of acquisition circuits. The pixel array includes a plurality of unit pixels, and each acquisition circuit corresponds to at least one unit pixel. The acquisition circuit is also referred to as a pixel circuit throughout the specification. Among them, each of the unit pixels is usually organized into a two-dimensional grid having "rows" and "columns" to form the pixel array. Alternatively, each of the unit pixels is arranged in an irregular shape to form the pixel array.

[0063] Among them, each of the acquisition circuits includes at least one sensor circuit 100, analog-to-digital conversion circuit 200, digital storage circuit 300, digital comparison circuit 400, and digital output circuit 500.

[0064] Each of the sensor circuits 100 includes a photosensor, which is configured to generate a sensor signal VSIG based on an optical signal illuminating at least one pixel corresponding to the photosensor. The photosensor can generate the sensor signal VSIG continuously or periodically.

[0065] The analog-to-digital conversion circuit 200 is configured to generate a current digital signal based on the sensor signal VSIG.

[0066] The digital storage circuit 300 is configured to store a previous digital signal. Here, the previous digital signal is a digital signal generated based on a previous sensor signal VSIG. The previous sensor signal VSIG corresponds to a signal generated and stored in the digital storage circuit 300 during any previous sampling period. Alternatively, the previous digital signal can also be provided externally and stored. To ensure the timeliness of data storage and considering that the level of the digital signal does not degrade over time, by storing the previous digital signal, and the storage circuit of the previous digital signal can be physically implemented with a smaller volume and can be used more conveniently and efficiently, such as combination and comparison with other pixels or other signals.

[0067] The digital comparison circuit 400 is configured to compare the level of the current digital signal with the level of the previous digital signal and determine whether there is a changing level in the current digital signal. The digital output circuit 500 is configured to generate and output an event signal when there is a changing level in the current digital signal.

[0068] In another embodiment of the present invention, the analog-to-digital conversion circuit 200 is a single-slope analog-to-digital converter.

[0069] Among them, the analog-to-digital conversion circuit 200 uses different storage ramps and comparison ramps for comparison to achieve analog-to-digital conversion, and the comparison of the storage ramp and the comparison ramp generates a difference function.

[0070] Among them, in another embodiment of the present invention, such as Figure 1As shown, the analog-to-digital conversion circuit 200 includes a comparator, and the comparator is used to compare the sensor signal VSIG with a scanning signal. The conversion from the analog signal (VSIG) to the digital signal is performed through the comparator and the scanning signal applied to the pixel array. According to the switching point of the comparator output, the digital representation is copied to the digital storage circuit 300.

[0071] Among them, the scanning signal can be an analog scanning input signal, or a continuous analog scanning input signal. Each analog scanning input signal provides a different slope of the ramp signal, and the slope of the ramp signal is the slope of the stored ramp.

[0072] By providing different slopes of the ramp signal, the comparison between the analog scanning input signals with different slopes of the ramp signal and the sensor signal VSIG is realized, thereby improving the accuracy of converting the sensor signal VSIG into a digital signal.

[0073] In another embodiment of the present invention, when the stored ramp and the comparison ramp are compared, a difference value between the two is generated each time of comparison. The difference value is a voltage difference, and the voltage differences are summarized to form a difference function, and the difference function is used to characterize the difference between the stored ramp and the comparison ramp.

[0074] When the slope of the stored ramp is set to be constant, if the slope of the comparison ramp is also constant, there is a specific voltage difference between the stored ramp and the comparison ramp, and the specific voltage difference is constant. This situation is applicable when the slope of the stored ramp is changed. When the slope of the comparison ramp is constant or changes, when the slope changes, the voltages of both the stored ramp and the comparison ramp change. At this time, there is a specific voltage difference between the stored ramp and the comparison ramp, and the specific voltage difference changes with the voltage changes of the stored ramp and the comparison ramp.

[0075] In another embodiment of the present invention, the stored ramp is generated from the digital signal pre-stored in the digital storage circuit 300. The pre-stored digital signal is generated from the VSIG signal at the previous moment or in the previous time period, and is used to characterize the light intensity at the previous moment or in the previous time period.

[0076] Then, the stored ramp is compared with the comparison ramp generated from the current VSIG signal at the current moment, so as to generate a digital signal when the light intensity signal changes.

[0077] In this way, when the stored ramp is from the previous moment, the slope of the stored ramp is constant. When the stored ramp is from the previous time period, the slope of the stored ramp changes. In this way, based on the digital signal at the backend, the current analog-to-digital signal conversion is regulated, the conversion efficiency is improved, and at the same time, the generation of redundant events is reduced.

[0078] In another embodiment of the present invention, the storage ramp is preset and the ramp slope is changed in real time. The change of the ramp slope is based on the difference value generated by the comparison between the previous moment and the comparison ramp. By changing the ramp slope, the difference value between the changed storage ramp and the comparison ramp is reduced, thereby realizing the change of the difference function.

[0079] In another embodiment of the present invention, the storage ramp and the comparison ramp are used according to the state of the pixel. Wherein, the change of the pixel state reverses the definitions of the storage ramp and the comparison ramp. That is, with the change of the pixel state, the comparison ramp at the current moment will be defined as the storage ramp at the next moment, so as to realize the judgment of whether the light intensity changes between the current moment and the next moment.

[0080] If the digital code A is recorded during the storage ramp, the digital code B is read during the comparison ramp, and B≥A, it indicates that VSIG during the comparison is greater than VSIG. That is, through the difference between the ramps, the change of the VSIG signal between the current moment and the previous moment is characterized, and further the change of the light intensity of the external environment is characterized.

[0081] In another embodiment of the present invention, according to the state of the pixel, one storage ramp is compared with one or more comparison ramps.

[0082] Alternatively, multiple storage ramps are compared with one or more comparison ramps.

[0083] Wherein, comparing one storage ramp with one comparison ramp generates one difference function. When one storage ramp is compared with multiple comparison ramps, one difference function is generated respectively.

[0084] When multiple storage ramps are compared with one comparison ramp, one difference function is generated respectively.

[0085] When multiple storage ramps are compared with multiple comparison ramps, one difference function is generated respectively.

[0086] In another embodiment of the present invention, one storage ramp is compared with one or more comparison ramps, or multiple storage ramps are compared with one or more comparison ramps; wherein, comparing one storage ramp with one comparison ramp generates one difference function; and the comparison between the storage ramp and the comparison ramp is not affected by the pixel state.

[0087] In another embodiment of the present invention, during the ramp, a digital comparison is performed based on the storage ramp and the comparison ramp, and the digital comparison value generated by the digital comparison is not stored.

[0088] Among them, the ramp device stores the existence time of the storage ramp and the comparison ramp. By only comparing and not storing the comparison value temporarily, it suppresses the signal change caused by the local light intensity change and reduces the generation of invalid events.

[0089] In another embodiment of the present invention, after the ramp period, a digital comparison is performed based on the storage ramp and the comparison ramp, and the digital comparison value generated by the digital comparison is stored.

[0090] The acquisition circuit can detect the light intensity that changes within a period of time. If the digital comparison circuit 400 detects a difference between the previously stored previous digital signal and the subsequent digital signal, and this difference meets a preset given criterion, an event is generated, that is, an event signal is generated, and the fact of the change is reported as event information at the output device, such as an image sensor.

[0091] In another embodiment of the present invention, one acquisition circuit corresponds to at least two unit pixels, that is, one acquisition circuit can collect signals from at least two unit pixels. Preferably, at least two unit pixels are adjacent to each other.

[0092] In another embodiment of the present invention, as Figure 1 shown, the sensor circuit 100 generates the sensor signal VSIG in response to a control signal, and the sensor signal VSIG changes its correspondence with the light intensity based on the control of the control signal, and / or, the sensor circuit 100 generates the sensor signal VSIG in response to a control signal during operation, and the sensor signal VSIG changes its correspondence with the light intensity based on the control of the control signal.

[0093] Among them, the control signal acts on the entire pixel array or local unit pixels in the pixel array. The operation period is the usage period of the differential image sensor with digital pixel storage and synchronous sampling.

[0094] In another embodiment of the present invention, the sensor signal VSIG generated by the sensor circuit 100 has any one of the following correspondence relationships with the light intensity:

[0095] The sensor signal VSIG is linearly correlated with the light intensity, or the sensor signal VSIG is non-linearly correlated with the light intensity, where the non-linear correlation includes but is not limited to logarithmic change; or the sensor signal VSIG is in a combined relationship of non-linear and linear correlation with the light intensity; among them, the above correspondence relationships change in response to a control signal.

[0096] Among them, the sensor signal VSIG is linearly correlated with the light intensity, which means converting the optical illumination intensity signal into an analog electrical signal (VSIG) with a linear relationship. This conversion relationship is applicable to application environments with low light and little light change.

[0097] The sensor signal VSIG is non-linearly correlated with the light intensity, which means converting the optical illumination intensity signal into an analog electrical signal with a logarithmic relationship. This conversion relationship is advantageous in terms of the high dynamic range of the optical illumination signal, because the output analog signal will not saturate over a wide range of illumination intensities, and can more truly reflect the change of the optical illumination intensity.

[0098] The sensor signal VSIG is in a combined relationship of non-linear and linear correlation with the light intensity, which means converting the optical illumination intensity signal into an analog electrical signal with a combined logarithmic and linear relationship or a similar non-linear function relationship.

[0099] During actual operation, the specific conversion correspondence is selected or set according to actual requirements.

[0100] Specifically, a signal source, such as a signal generation circuit, can be separately set to provide the control signal. Based on actual use, the control signal is a signal that changes from time to time. The change of the control signal comes from the global light intensity or the actual operation during the operation. Based on the control signal, the conversion of the corresponding relationship between the sensor signal VSIG and the light intensity is realized to meet the data usage requirements in different scenarios.

[0101] In another embodiment of the present invention, as Figure 1 shown, the analog-to-digital conversion circuit 200 includes a comparator, and the comparator is used to compare the sensor signal VSIG with a scanning signal. The conversion from the analog signal (VSIG) to the digital signal is performed through the comparator and the scanning signal applied to the pixel array. According to the switching point of the comparator output, the digital representation is copied to the digital storage circuit 300.

[0102] Among them, the scanning signal can be an analog scanning input signal, or a continuous analog scanning input signal, where each analog scanning input signal provides a different slope of the ramp signal. By providing different slopes of the ramp signal, the comparison between the analog scanning input signals with different slopes of the ramp signal and the sensor signal VSIG is realized, thereby improving the accuracy of converting the sensor signal VSIG into a digital signal.

[0103] Among them, a single or continuous analog scanning input signal is used to provide at least one of the analog-to-digital conversion circuits 200. That is, one scanning signal can scan one or more analog-to-digital conversion circuits 200 in one acquisition circuit, or can scan one or more analog-to-digital conversion circuits 200 in multiple acquisition circuits simultaneously, thereby improving the analog-to-digital conversion efficiency and the data processing speed.

[0104] And / or, b) The analog-to-digital conversion circuit 200 is further configured to provide a digital code parallel to the scanning signal to at least one pixel, and the digital code stored in the digital storage circuit 300 is the digital code value output after being compared by a comparator. The digital code is preferably a Gray code, which makes full use of the fact that the Gray code does not need to be synchronized with the digital storage, and only one bit changes at a time, and the codes on either side of the Gray code are valid, and the Gray code has fewer edges, reducing the power consumption during the data processing.

[0105] In another embodiment of the present invention, the digital output circuit 500 is configured to generate and output an event signal when there is a changing level in the current digital signal and the changing level exceeds a preset specific threshold, where the specific threshold is set based on the fixed intensity change rate of the optical signal.

[0106] The optical signal is the signal emitted by an artificial light source. Based on different voltages of the power supply, the artificial light source usually has a regularly changing intensity. That is, due to the limitations of its own structure and control, the artificial light source on the market has a regularly changing intensity. For example, the fixed intensity change of an incandescent lamp is 14%, the fixed intensity change of a fluorescent lamp is 8%, and the fixed intensity change of an LED is 9%. If the specific threshold required for the digital output circuit 500 to generate and output an event signal is set to be less than the fixed intensity change of the artificial light source, such as 2%-3%, then the modulation of artificial lighting is very likely to cause a large number of redundant events.

[0107] Therefore, in order to suppress or eliminate the redundant events caused by the fixed intensity change of the artificial light source, it is necessary to set the specific threshold based on the fixed intensity change rate of the optical signal.

[0108] Specifically, when there is more than one optical signal, the specific threshold is greater than the fixed intensity change rate corresponding to the optical signal, thus reducing the generation of redundant events.

[0109] When there are multiple optical signals and each optical signal has a different fixed intensity change rate, the specific threshold is greater than the maximum fixed intensity change rate among the fixed intensity change rates of each optical signal, thereby ensuring that it is not affected by the fixed intensity change rates of all the optical signals themselves. This setting is applicable to the situation where the number of inherent optical signals in a known background environment is known, and it is necessary to shield the redundant events caused by all the inherent optical signals in the background.

[0110] When conducting relevant experiments, there are three inherent optical signals in the background environment of the experiment. In order to obtain the event change caused by the optical signals other than the three optical signals in the background environment, it is necessary to shield the redundant events caused by the three optical signals in the background environment during the experiment. Therefore, at this time, the specific threshold is set to be greater than the maximum fixed intensity change rate among the three fixed intensity change rates corresponding to the three optical signals in the background environment, which also meets the experimental requirements.

[0111] In this case, the fixed intensity change rate of the optical signals other than the three optical signals in the background environment is greater than the specific threshold. Generally, the specific threshold is set slightly greater than the maximum fixed intensity change rate among the three fixed intensity change rates corresponding to the three optical signals in the background environment, so as to ensure that the acquisition of events caused by the target optical signals is not affected while shielding the redundant events generated by the optical signals in the background.

[0112] When testing the differential image sensor with digital pixel storage and synchronous sampling,

[0113] In another embodiment of the present invention, the specific threshold can be set in advance or set in real time according to the change of the optical signal during operation.

[0114] In another embodiment of the present invention, the repetition rate of the scanning signal is preset and used to reduce or eliminate the redundant events generated due to the change of the light intensity of the optical signal illuminating the optical sensor. Specifically, the repetition rate of the scanning signal is the frequency at which the scanning signal is provided to the analog-to-digital conversion circuit 200.

[0115] Among them, the repetition rate of the scanning signal is set based on at least one preset repetition rate, and the preset repetition rate matches the modulation frequency of the optical signal irradiating the optical sensor.

[0116] It should be understood that the repetition rate of the scanning signal is selected from one or more preset repetition rates, and each preset repetition rate corresponds to the light source modulation of the optical signal irradiating the optical sensor. It should be understood that when the fixed intensity change rate of the optical signal itself is certain, the number of redundant events generated within a period of time is certain. And within this period of time, if the scanning signal continuously scans and the repetition rate is always higher than the modulation frequency of the optical signal irradiating the optical sensor, then all the redundant events generated within this event segment will be acquired. Therefore, in order to reduce the number of redundant events, it is necessary to adjust the repetition rate of the scanning signal.

[0117] When the modulation frequencies of the optical signals irradiating the optical sensor are multiple, such as 50HZ, 100HZ, and 200HZ respectively, that is, at 50HZ, 100HZ, and 200HZ, within one cycle, the numbers of intensity changes of the corresponding light intensities are 100 times, 200 times, and 400 times respectively, then the numbers of redundant events generated are 100, 200, and 400 respectively. At this time, in order to reduce the redundant events, the repetition rate of the scanning signal is set based on the repetition rates corresponding to 50HZ, 100HZ, and 200HZ. For example, it can be set that the repetition rate is less than the number of intensity changes of the light intensity. For example, the repetition rate of the scanning signal is set to scan 25 times within one cycle, and one redundant event is obtained for each scan, with a total of 25 redundant events obtained, thereby reducing the number of redundant events.

[0118] Preferably, the repetition rate of the scanning signal is a multiple of the modulation frequency of the optical signal irradiating the optical sensor.

[0119] In another embodiment of the present invention, the scanning signal is a linear ramp, or a non-linear ramp, and the non-linear ramp includes, but is not limited to, an exponential ramp. The scanning signal can be generated centrally. Considering that the light intensity mostly changes non-linearly, it is preferably to set the scanning signal as non-linear. Therefore, the scanning signal can be generated by the signal generating device on the device to enhance the sharing on the hardware circuit and save the volume required for the hardware circuit.

[0120] In another embodiment of the present invention, the scanning signal changes during operation, and / or the period of the scanning signal changes during operation, and / or the scanning signal is interrupted at a specific time during operation, and the specific time is set during operation or preset in advance. Among them, during operation, the scanning signal of the linear ramp or non-linear ramp can be changed to match different sensor signals VSIG to achieve real-time matching and comparison.

[0121] Specifically, the scanning signal is repeatedly scanned out during operation according to a preset scanning rule. During operation, the repetition of the scanning signal can be interrupted or extended as needed to change the repetition rate of the scanning.

[0122] In another embodiment of the present invention, as Figure 1 shown, the differential image sensor with digital pixel storage and synchronous sampling further includes at least one reference pixel, and the reference pixel is used to define the ramp signal. The reference pixel is preset or automatically set in real time during operation and is associated with the sensor signal VSIG. After obtaining a specific numerical representation or numerical range representation of a sensor signal VSIG, the reference pixel can be set, and the reference pixel is used to define the ramp signal to achieve more accurate and redundant-free comparison, so as to more efficiently promote subsequent data generation.

[0123] In another embodiment of the present invention, the reference pixel is disposed outside the pixel array.

[0124] In another embodiment of the present invention, as Figure 1 shown, the digital comparison circuit 400 compares with the level of the previous digital signal during the scanning of the analog signal; or, the digital comparison circuit 400 compares with the level of the previous digital signal after the scanning of the analog signal.

[0125] When the digital comparison circuit 400 compares with the level of the previous digital signal during the scanning of the analog signal, it is not necessary to store the A / D output value, and only the comparison result needs to be stored.

[0126] During the comparison after the scanning of the analog signal, an order, such as bitwise, can be used.

[0127] In another embodiment of the present invention, the digital comparison circuit 400 uses dynamic logic to compare levels, or the digital comparison circuit 400 uses static logic to compare levels.

[0128] Among them, when comparison is required in a low-power state, static logic is selected to compare levels. When high speed of comparison is required and the output result needs to be maintained for a long time, dynamic logic is selected to compare levels.

[0129] In another embodiment of the present invention, feedback from the result of the previous comparison is used to perform digital comparison, that is, the comparison between the next moment and the previous moment, which is beneficial to suppressing noise and reducing the generation of false events.

[0130] In another embodiment of the present invention, both the comparison of digital signals and the output of event signals need to have specific differences to be realized, thereby reducing the number of generated times and improving the resolution of time occurrence. Among them, the level of the signal, the operation mode, the level of adjacent pixels, the previous level of the pixel, the time generated by the pixel previously, and the events generated by adjacent pixels will all have the specific differences.

[0131] In another embodiment of the present invention, the digital output circuit 500 is used to output the output signal according to the comparison result of the digital comparison circuit 400 in adjacent pixels, or the digital output circuit 500 is used to generate an output signal according to a function in a fixed configuration of adjacent pixels or a function in a configuration that changes during operation, where the function includes but is not limited to an average function.

[0132] In another embodiment of the present invention, the digital output circuit 500 is used to generate an event signal based on the outputs of adjacent pixels, or, the output circuit is used to generate an event signal based on the levels stored in adjacent pixels, or, the output circuit is used to generate an event signal based on a plurality of pre-stored stored values.

[0133] In another embodiment of the present invention, as Figure 1 shown, the digital output circuit 500 is used to generate an event signal indicating the direction of a level change, and / or, the digital output circuit 500 is applied to generate an event signal indicating only one change direction of one changed level, and / or, the digital output circuit 500 is used to generate an event signal indicating the amplitude of the changed level, and / or, the digital output circuit 500 is used to generate the light intensity before and / or after the changed level changes.

[0134] In another embodiment of the present invention, the generation of an event signal of one acquisition circuit depends on the event signal of another adjacent acquisition circuit, thereby filtering events to reduce the number of false events. Alternatively, the generation of an event signal of one acquisition circuit depends on the digital levels stored in another adjacent acquisition circuit, thereby filtering events to provide higher sensitivity near the edge.

[0135] In another embodiment of the present invention, the generation of an event signal of one acquisition circuit depends on a plurality of digital signals stored in the digital storage circuit 300 in the acquisition circuit.

[0136] In another embodiment of the present invention, when the output of the event signal is a single event bit, such as an up event, it characterizes an increasing light intensity. When the output of the event signal is two event bits, such as up and down, the event output can indicate whether the light intensity or the pixel signal VSIG is greater than or less than the stored light intensity level or the stored VSIG. The event output also includes the change amplitude of the digital signal, and based on this change amplitude, the image can be perfectly reconstructed from the event information, thereby obtaining the currently detected image.

[0137] In another embodiment of the present invention, as Figure 1As shown, the digital storage circuit 300 is used to provide a stored previous digital signal at the output line of the pixel array, and / or, the digital storage circuit 300 is used to selectively provide a stored previous digital signal for a unit pixel having an event output on the output line of the pixel array, and / or, the digital storage circuit 300 is used to provide a stored previous digital signal at the output line using a time column line, and / or, the digital storage circuit 300 is used to write the stored previous digital signal into a unit pixel, and / or, the digital storage circuit 300 is used to write a data stream into the previous digital signal; the digital storage circuit 300 is used to write a data stream as an event stream into the stored previous digital signal.

[0138] In another embodiment of the present invention, a stored digital level can be written into the pixel arrangement as a reference for comparison. And a data stream can be written into the stored digital level to intentionally provide a time-related reference for event generation. Among them, the stored digital level is the previous digital signal.

[0139] In another embodiment, the data stream is generally generated by a sensor.

[0140] The output line provided by the digital storage circuit 300 is at least one and is provided to at least one digital output circuit 500 so that one digital output circuit 500 or multiple digital output circuits 500 can share.

[0141] The previous digital signal stored in the digital storage circuit 300 can be written by an external pixel array, or, the signal output at a previous moment can be written into the stored previous digital signal, thereby characterizing the change process of the signal. For example, when the digital signal output at time t1 is p, then at the next time t2, time t1 is the previous moment, and the output at time t1 is p, then p is the previous digital signal for time t2. This previous digital signal is used as a reference. At time t2, it is only output when compared with the reference, that is, when the signal changes compared with time t1, and then an event signal is output. That is, a signal is only output when it changes, reducing the generation of redundant data.

[0142] In another embodiment of the present invention, the digital storage value from the target area is selectively read, that is, through random access.

[0143] In another embodiment of the present invention, a sampling rate can also be set. The sampling rate is the frequency of collecting event signal generation. The sampling rate can be changed from time to time during operation based on an electrical configuration, such as a hardware circuit. Different sampling rates also determine different power consumption and event generation rates. When no event signal appears for a long time or during non-use time, a very low power consumption mode can be achieved by reducing the sampling rate, or, by completely interrupting the sampling for a period of time, thereby reducing power consumption.

[0144] In another embodiment of the present invention, the sensor circuit 100 receives feedback corresponding to the digital level stored in the digital storage circuit 300 or the digital levels of adjacent pixels stored in the digital storage circuit 300 from the digital storage circuit 300, and the sensor circuit 100 generates a sensor signal VSIG based on the feedback card, and / or, the analog-to-digital conversion circuit 200 receives feedback from the level of the previously stored digital signal or the level of the previously stored digital signal of an adjacent unit pixel, and adjusts the currently output digital signal according to the feedback. Through the setting of the feedback, the generated sensor signal VSIG is made more accurate.

[0145] In another embodiment of the present invention, the sensor circuit 100 is used to provide an analog representation of the sensor signal VSIG, and / or, the differential image sensor with digital pixel storage and synchronous sampling is composed of multiple semiconductor layers, and each of the semiconductor layers has different optimized functions.

[0146] Specifically, the semiconductor process type of each layer is optimized for the function of that layer. For example, the performance is improved, specifically such as improving heat dissipation. Or, following the requirements of device size and device cost, the silicon area of the semiconductor layer is reduced to meet the actual use requirements, as long as it can be ensured that the changed physical structure of the semiconductor layer can improve the accuracy and efficiency of the light intensity change, and reduce the electrical interference between the control signal line and the photoelectric sensor circuit 100.

[0147] The output mode of the event signal includes a continuous output mode and an intermittent output mode. The output mode of the event signal is switched between the continuous output mode and the intermittent output mode in response to an interrupt control signal, and the interrupt control signal is generated by the digital output circuit 500. By setting the continuous output mode, it is applicable to the situation where the event signal is generated at a high frequency, that is, a high-power consumption state. The intermittent output mode is applicable to a low-power consumption state, that is, when the frequency of the event signal generation is not high, or it can be understood that the event signal is not generated for a long time.

[0148] When in the intermittent output mode, the event signal is intermittently output at a preset specific time interval. That is, the event signal is first stored and lasts for a period of time, and after continuously storing for a period of time, the same output event signal is output. Among them, the specific time interval is set by itself, such as set to 5 minutes, 10 minutes or even half an hour, or 6 hours, etc. Taking 5 minutes as an example, the event signal is output once every 5 minutes.

[0149] In this way, by using the intermittent output and the characteristic that the digital signal is easy to store in the digital output circuit 500 and there is no data loss, it ensures low power consumption while ensuring the lossless output of the event.

[0150] When in the continuous output mode, the event signal is output immediately when a change in the current digital signal level is detected.

[0151] In another embodiment of the present invention, when the digital output circuit 500 detects a change in the current digital signal level within a first specific time period and the frequency of the level change does not exceed a preset frequency, the digital output circuit 500 controls the output mode of the event signal to be an intermittent output mode. The first specific time period is set as needed, such as set to 1 minute. The preset frequency is pre-set. When the frequency of the level change does not exceed the preset frequency, it indicates that the generation frequency of the event signal is low at this time. Therefore, at this time, it can be set to a low-power mode, that is, an intermittent output mode.

[0152] In another embodiment of the present invention, when the digital output circuit 500 detects a change in the current digital signal level within a first specific time period and the frequency of the level change exceeds the preset frequency, at this time, the generation frequency of the event signal is high, which is a high-power mode. At this time, the digital output circuit 500 controls the output mode of the event signal to be a continuous output mode.

[0153] In another embodiment of the present invention, when the digital storage circuit 300 does not obtain a digital signal generated based on the previous sensor signal VSIG within a second specific time period, it indicates that no new digital signal is generated within the second specific time period. Therefore, it indicates that the external environment has no change at this time, that is, no new event signal will be generated at this time. It can be determined that this is a low-power mode at this time. Therefore, a first instruction signal is generated and sent to the digital output circuit 500, and the digital output circuit 500 controls the output mode of the event signal to be an intermittent output mode according to the first instruction signal.

[0154] Further, when the digital storage circuit 300 obtains at least one digital signal generated based on the previous sensor signal VSIG within a second specific time period, it indicates that the external environment is continuously changing at this time. Therefore, a second instruction signal is generated and sent to the digital output circuit 500, and the digital output circuit 500 controls the output mode of the event signal to be a continuous output mode according to the second instruction signal.

[0155] In addition, the storage of digital signals does not degrade over time, and information can be retained indefinitely without losing information or requiring image frame refreshing. In some applications, such as monitoring, in different triggering environments, the image usually does not change for a long time. By setting the switching of the event output mode, the observation rate can be greatly reduced and power can be saved.

[0156] In another embodiment of the present invention, each of the acquisition circuits may be connected to a common or multiple common event output lines. A group of acquisition circuits may share any one of the digital output circuit 500, the digital comparison circuit 400, the digital storage circuit 300, and the analog-to-digital conversion circuit 200.

[0157] The foregoing are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A differential image sensor with digital pixel storage and synchronous sampling, characterized in that, it includes a pixel array and a plurality of acquisition circuits. The pixel array includes a plurality of unit pixels, and each of the acquisition circuits corresponds to at least one unit pixel; wherein, each of the acquisition circuits includes: at least one sensor circuit, and each of the sensor circuits includes an optical sensor, and the optical sensor is used to generate a sensor signal VSIG according to the optical signal illuminating at least one pixel corresponding to the optical sensor; at least one analog-to-digital conversion circuit, and the analog-to-digital conversion circuit is used to generate a current digital signal according to the sensor signal VSIG; at least one digital storage circuit, and the digital storage circuit is used to store a previous digital signal, wherein the previous digital signal is a digital signal generated according to a previous sensor signal VSIG; at least one digital comparison circuit, and the digital comparison circuit is used to compare the level of the current digital signal with the level of the previous digital signal and determine whether the current digital signal has a changing level; at least one digital output circuit, and the digital output circuit is used to generate and output an event signal when the current digital signal has a changing level and the changing level exceeds a preset threshold, wherein the threshold is set based on a fixed intensity change rate of the optical signal; when the optical signal is one, the threshold is greater than the fixed intensity change rate corresponding to the optical signal; when the optical signals are multiple and each of the optical signals has a different fixed intensity change rate, the threshold is greater than the maximum fixed intensity change rate among the fixed intensity change rates; the analog-to-digital conversion circuit includes a comparator, and the comparator is used to compare the sensor signal VSIG with any one of the following scanning signals: aa) an analog scanning input signal; and, bb) continuous analog scanning input signals, and each analog scanning input signal provides a different ramp signal slope; wherein, the repetition rate of the scanning signal is preset and used to reduce or eliminate redundant events caused by the change of the optical intensity of the optical signal illuminating the optical sensor.

2. The differential image sensor with digital pixel storage and synchronous sampling according to claim 1, characterized in that, the repetition rate of the scanning signal is set based on at least one preset repetition rate, and the preset repetition rate matches the modulation frequency of the optical signal irradiating the optical sensor.

3. The differential image sensor with digital pixel storage and synchronous sampling according to claim 1, characterized in that, the repetition rate of the scanning signal is a multiple of the modulation frequency of the optical signal irradiating the optical sensor.

4. The differential image sensor with digital pixel storage and synchronous sampling according to claim 2, characterized in that, the digital comparison circuit compares with the level of the previous digital signal during the scanning of the analog signal; or, the digital comparison circuit compares with the level of the previous digital signal after the scanning of the analog signal.

5. The differential image sensor with digital pixel storage and synchronous sampling according to claim 4, characterized in that, The digital output circuit is configured to output an output signal based on the comparison results in adjacent pixels by the digital comparison circuit, or, The digital output circuit is configured to generate an output signal based on a function in a fixed configuration of adjacent pixels or a function in a configuration that changes during operation, where the function includes, but is not limited to, an averaging function.

6. The differential image sensor with digital pixel storage and synchronous sampling according to claim 5, wherein, The digital output circuit is configured to generate an event signal based on the outputs of adjacent pixels, or, The digital output circuit is configured to generate an event signal based on the levels stored in adjacent pixels, or, The digital output circuit is configured to generate an event signal based on a plurality of pre-stored stored values.

7. The differential image sensor with digital pixel storage and synchronous sampling according to claim 6, wherein, The digital output circuit is configured to generate an event signal indicating the direction of the level change, and / or, The digital output circuit is applied to generate an event signal indicating only one change direction of one changed level, and / or, The digital output circuit is configured to generate an event signal indicating the amplitude of the changed level, and / or, The digital output circuit is configured to generate an indication of the light intensity before and / or after the level change.

8. The differential image sensor with digital pixel storage and synchronous sampling according to claim 7, wherein, The digital storage circuit is configured to provide a stored previous digital signal at the output lines of the pixel array, and / or, The digital storage circuit is configured to selectively provide a stored previous digital signal for unit pixels with event outputs at the output lines of the pixel array, and / or, The digital storage circuit is configured to provide a stored previous digital signal at the output lines using time column lines, and / or, The digital storage circuit is configured to write the stored previous digital signal to a unit pixel, and / or, The digital storage circuit is configured to write a data stream to the previous digital signal; The digital storage circuit is configured to write from a data stream as an event stream to the previously stored digital signal.

Images