Event vision sensors and associated systems, devices, and methods

Abstract

The disclosure relates to event vision sensors and associated systems, devices, and methods. In one embodiment, an event vision sensor includes an event vision pixel array and an event signal processor. The event signal processor is configured to identify a defective event visual pixel of the array based on a noise event initiation rate corresponding to the event visual pixel. The noise event initiation rate for each event visual pixel may be based on a measure of a probability that the event visual pixel detects a noise event over time. Each event visual pixel may include internal circuitry (e.g., a memory component, such as a latch) that may be used to disable the event visual pixel to detect an event or mask the output of the event visual pixel such that an event cannot be read out from the pixel when the event visual pixel is identified as defective.

Description

Technical Field

[0001] The present disclosure relates generally to event vision sensors. For example, several embodiments of the present invention relate to event vision sensors that employ defective pixel suppression, such as intra-pixel defective pixel suppression based on probabilistic determination of noise event initiation rates. Background Art

[0002] Image sensors have become ubiquitous and are now widely used in digital cameras, cell phones, video cameras, as well as in medical, automotive and other applications. As image sensors are integrated into a wider range of electronic devices, it is desirable to enhance their functionality, performance metrics, etc. in as many ways as possible (e.g., resolution, power consumption, dynamic range, etc.) through device architecture design and image acquisition processing.

[0003] A typical image sensor operates in response to image light from an external scene being incident on the image sensor. The image sensor includes an array of pixels having photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and generate image charges after absorbing the image light. The image charge generated by the pixel light can be measured as an analog output image signal on a column bit line that varies according to the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out from the column bit line as an analog image signal and converted into a digital value to provide information representing the external scene. Summary of the invention

[0004] In one aspect, the present disclosure provides an event vision sensor comprising: an array of event vision pixels arranged in rows and columns; and an event signal processor configured to identify defective event vision pixels corresponding to the array based at least in part on a noise event initiation rate, wherein the noise event initiation rate is a measurement based at least in part on a probability of the defective event vision pixels detecting a noise event over time.

[0005] On the other hand, the present disclosure provides a method comprising: identifying an event vision pixel as defective based at least in part on a noise event triggering rate of the event vision pixel corresponding to an event vision sensor, wherein the noise event triggering rate is a measurement based at least in part on the probability of the event vision pixel detecting a noise event over time; and in response to identifying the event vision pixel as defective, preventing the event vision pixel from outputting event data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive embodiments of the present invention are described below with reference to the following figures, wherein the same or similar reference numerals are used to refer to all the same or similar components unless otherwise specified.

[0007] Figure 1is a partial schematic block diagram of an event vision sensor and an illumination source configured in accordance with various embodiments of the present invention.

[0008] Figure 2 is a partial schematic block diagram of an event signal processor configured according to various embodiments of the present invention.

[0009] Figure 3 is a partially schematic block diagram illustrating an event vision pixel configured in accordance with various embodiments of the present invention.

[0010] Figure 4 is a partial schematic block diagram illustrating another event vision pixel configured in accordance with various embodiments of the present invention.

[0011] Figure 5 is a partial schematic block diagram illustrating yet another event vision pixel configured in accordance with various embodiments of the present invention.

[0012] Figure 6 is a partial schematic block diagram illustrating yet another event vision pixel configured in accordance with various embodiments of the present invention.

[0013] Figure 7 is a flow chart illustrating a method of identifying and disabling / masking defective event vision pixels of an event vision sensor according to various embodiments of the present invention.

[0014] Figure 8 is a flow chart illustrating a method of determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0015] Fig. 9 is a flow chart illustrating another method of determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0016] Fig.10 is a flow chart illustrating yet another method of determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0017] Those skilled in the art will appreciate that the elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to aid in understanding various aspects of the present invention. In addition, common but well-understood elements or methods that are useful or necessary in commercially feasible embodiments are generally not depicted in the figures or described in detail below so as not to unnecessarily obscure the description of various aspects of the present invention. DETAILED DESCRIPTION

[0018] The present disclosure relates to event vision sensors. For example, several embodiments of the present invention relate to event vision sensors that employ defective pixel suppression, such as intra-pixel defective pixel suppression. In the following description, specific details are set forth to provide a thorough understanding of aspects of the present invention. However, one skilled in the relevant art will recognize that the systems, devices, and techniques described herein may be practiced without one or more of the specific details set forth herein or using other methods, components, materials, etc.

[0019] Reference to an "example" or "embodiment" in this specification means that a particular feature, structure, or characteristic described in conjunction with the example or embodiment is included in at least one example or embodiment of the invention. Thus, the phrases "for example," "as an example," or "embodiment" used herein do not necessarily all refer to the same example or embodiment and are not necessarily limited to the particular example or embodiment being discussed. Furthermore, the features, structures, or characteristics of the invention described herein may be combined in any suitable manner to provide additional examples or embodiments of the invention.

[0020] For ease of description, spatially relative terms (e.g., "below," "below," "above," "below," "above," "up," "top," "bottom," "left," "right," "center," "middle," etc.) may be used herein to describe the relationship of one element or one feature relative to one or more other elements or features, as illustrated in the figures. It should be understood that the spatially relative terms are intended to cover different orientations of the device or system in use or operation in addition to the orientation depicted in the figures. For example, if the device or system illustrated in the figures is rotated, turned, or flipped about a horizontal axis, then the element or feature described as being "below" or "below" or "below" one or more other elements or features may be oriented to be "above" one or more other elements or features. Therefore, the exemplary terms "below" and "below" are non-limiting and may cover both above and below orientations. The device or system may be additionally or alternatively oriented in a manner other than that illustrated in the figures (e.g., rotated 90 degrees about a vertical axis or in other orientations) and the spatially relative descriptors used herein may be interpreted accordingly. In addition, it will also be understood that when an element is referred to as being between two other elements, it can be the only element between the two other elements, or one or more intervening elements may also be present.

[0021] In this specification, several technical terms are used. These terms should be taken with their ordinary meaning in the art, unless explicitly defined herein or otherwise clearly implied by the context of their use. It should be noted that component names and symbols are used interchangeably in this document (e.g., Si and silicon); however, both have the same meaning.

[0022] A. Overview

[0023] Active pixel sensors (e.g., CMOS imaging systems) typically employ an array of active pixels with a globally defined integration time. Thus, the active pixels in an active pixel sensor typically have the same integration time, and each pixel in the array is typically converted into a digital signal regardless of its content (e.g., whether the external scene captured by the pixel has changed since the last time the pixel was read out). In other words, the image data generated by the active pixels in, for example, a CMOS imager is read out in frames of known size, regardless of whether there are events in the external scene.

[0024] In contrast, when a pixel captures an external scene change (e.g., an event), an event vision sensor (e.g., an event-driven sensor or a dynamic vision sensor) reads out the pixel and / or converts the corresponding pixel signal into a digital signal. In other words, the pixels of the event vision sensor that do not detect an external scene change are not read out and / or the pixel signals corresponding to such pixels are not converted into digital signals. Therefore, each pixel of the event vision sensor can be independent of other pixels of the event vision sensor, and only the pixel that detects the external scene change needs to be read out and / or its corresponding pixel signal needs to be converted into a digital signal or recorded (thereby saving power). Thus, the event vision sensor does not need to record the entire conventional image and therefore does not need to capture and record all the highly redundant information of a normal image frame by frame. Therefore, the event vision sensor can be used to detect movement or motion in an external scene (e.g., rather than for capturing / reading out all frames of an image or video), while being able to (i) use a low data rate and (ii) achieve ultra-high frame rate or speed capabilities.

[0025] When an event vision pixel array is used to monitor an external scene in an event vision sensor, the event vision sensor rarely detects and records isolated events corresponding to legitimate activities in the external scene (e.g., events corresponding to a single isolated event vision pixel in the array). This isolated event often corresponds to noise and / or is caused by a defective event vision pixel in the array. Therefore, many event vision sensors use various techniques (e.g., encoding or compression techniques) to filter out noise event signals read out from event vision pixels. As a specific example, when the first event vision pixel of the array detects an event, some event vision sensors analyze a group of event vision pixels including the first event vision pixel. If the event vision sensor determines that a threshold number of event vision pixels in the group detects an event, then the event detected by the first event vision pixel is classified as corresponding to a legitimate activity in the external scene monitored by the event vision sensor. On the other hand, if the event vision sensor determines that less than a threshold number of event vision pixels in the group detect an event, then the event detected by the first event vision pixel is classified as a noise event and / or discarded. This technique is called coincidence detection.

[0026] Coincidence detection eliminates small statistics. In other words, coincidence detection cannot analyze or provide any insight into long-term trends (e.g., coincidence detection cannot provide any insight into whether a noise event is relatively rare or common for a particular event vision pixel to detect a noise event). Therefore, recidivists (e.g., defective pixels that often detect noise events) are not processed and continue to consume excess bandwidth of the event vision sensor to read out noise events. In addition, the small statistics approach of coincidence detection increases the likelihood that the event vision sensor will classify a legitimate event as a noise event or vice versa.

[0027] Furthermore, coincidence detection and many other conventional methods operate on signals read out from event vision pixels of the array. In other words, many conventional methods identify defective event vision pixels and / or filter out corresponding signals only after noise events are read out from the event vision pixels during normal operation of the event vision sensor. Therefore, event vision sensors employing these conventional methods typically utilize excess bandwidth of the event vision sensor during normal operation to first read out signals of defective event vision pixels (consuming excess bandwidth) before (a) the event vision pixels are identified as defective and / or (b) their corresponding signals are filtered out and / or discarded.

[0028] In view of the problems discussed above, the present invention provides various solutions for identifying defective event vision pixels and disabling or masking their outputs using in-pixel circuitry. For example, several embodiments of the present invention identify defective event vision pixels of an event vision sensor based on a probabilistic determination of a noise event initiation rate for each event vision pixel. More specifically, during one or more points in the life of the event vision sensor (e.g., during wafer testing, during power-up or initialization, after a predetermined amount of time, outside of normal operation, when the event vision sensor determines that the event initiation rate of one or more event vision pixels (corresponding to legitimate and / or noise events) exceeds a threshold, etc.), the event vision pixels may be exposed to DC illumination while the event vision sensor (e.g., an event signal processor of the event vision sensor) determines the noise event initiation rate for each event vision pixel. In some embodiments, the noise event initiation rate of the event vision pixel may be based on a measurement of the probability that the event vision pixel detects a noise event within a time interval of a set duration after the event vision pixel is reset. In these and other embodiments, the noise event initiation rate of the event vision pixel may be based on an observation of the probability that the event vision pixel detects a noise event within various time intervals of different durations after the event vision pixel is reset. Once the noise event initiation rate for an event vision pixel is obtained, the noise event initiation rate may be compared to one or more thresholds to identify whether the event vision pixel is defective (eg, unacceptably likely to record or unacceptably prone to recording noise events).

[0029] When an event vision pixel is identified as defective, the event vision sensor may program a memory component (e.g., latch, flip-flop, SRAM) contained within the event vision pixel to disable the event vision pixel (e.g., prevent the event vision pixel from detecting an event) or mask its output (e.g., prevent an event detected by the event vision pixel from being output from the event vision pixel). Additionally or alternatively, the address of the event vision pixel may be added to a lookup table maintained by the event vision sensor so that the event vision pixel may be disabled or masked whenever the event vision sensor is powered on or initialized.

[0030] In this manner, the present invention provides various solutions for identifying defective event vision pixels of an event vision sensor without the need to (a) read out noise events as well as legitimate events during normal operation and (b) discern whether an event should be classified as legitimate or noise. In addition, the present invention provides an intra-pixel defective pixel suppression solution that can be used to disable event vision pixels identified as defective and / or mask their output. Such defective event vision pixels can then be prevented from consuming excess bandwidth of the event vision sensor to read out noise events detected by the event vision pixels. Therefore, compared to an event vision sensor lacking a defective event vision pixel suppression solution and / or a corresponding intra-pixel circuit system, the present invention is expected to reduce the number of noise events read out to an event signal processor of an event vision sensor, increase the percentage of legitimate events in the events read out to the event signal processor, and / or utilize less power.

[0031] B. Selected embodiments of event vision sensors with defective pixel suppression and associated systems, devices, and methods

[0032] Figure 1 1 is a partially schematic block diagram of an event vision sensor 100 (also referred to herein as an "imager") configured in accordance with various embodiments of the present invention. As shown, the event vision sensor 100 includes an array 102 of event vision pixel circuits 110 ("event vision pixels 110"), row control circuitry 104, column control circuitry 106, an event signal processor 108, a transmitter 116, a lookup table 142 ("LUT 142"), control logic 144, and peripheral circuitry 146. Also shown is an illumination source 120 configured to project light 150 onto the event vision pixels 110 of the array 102. In some embodiments, the illumination source 120 may be a component of the event vision sensor 100 or an imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be external to the event vision sensor 100 and the imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be omitted.

[0033] Event vision pixels 110 are arranged in rows and columns in array 102 ( Figure 110 and 110, respectively. The event vision pixels 110 are configured to detect changes (e.g., events) in light received from an external scene. To this end, each event vision pixel 110 may include: (i) a photosensor (such as a photodiode) configured to photogenerate a charge or photocurrent in response to incident light received from an external scene; (ii) a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent generated by the photosensor into a voltage; and (iii) a filter amplifier coupled to the photocurrent / voltage converter to generate a filtered and amplified signal in response to the voltage received from the photocurrent / voltage converter. The event vision pixel 110 may further include a threshold comparison circuit or stage that generates and receives a handshake signal in response to an event detected asynchronously in incident light received from an external scene. Alternatively, the threshold comparison circuit may be included in circuitry peripheral to or external to the event vision pixel 110 of the array 102 (e.g., in the event signal processor 108), such as within a column readout circuitry. The photosensor, photocurrent / voltage converter, filter amplifier, and / or threshold comparison stage of the event vision pixel 110 are collectively referred to herein as event vision sensor (EVS) core pixel circuitry 130 .

[0034] like Figure 1 , the event vision pixels 110 configured in accordance with various embodiments of the present invention may each further include a programmable storage component 136 (illustrated and hereinafter referred to as a "latch 136") coupled to the EVS core pixel circuitry 130. The latch 136 of the event vision pixel 110 may be programmed using a program signal (prog), a row signal (row) corresponding to the event vision pixel 110, and a column signal (col) corresponding to the event vision pixel 110. More specifically, when an event vision pixel 110 of the array 102 is identified as a defective pixel (as discussed in more detail below), an address (row address and column address) corresponding to the location of the event vision pixel 110 in the array 102 may be stored in the LUT 142. The control logic 144 may then use the address stored in the LUT 142 to program (e.g., after the event vision sensor 100 is powered on or initialized) the latch 136 corresponding to the event vision pixel 110 to disable the event vision pixel 110 and / or mask its output so that the event detected by the event vision pixel 110 is not read out from the array 102. Figures 3 to 6 Various embodiments of event vision pixel 110 incorporating latches similar to latch 136 are discussed in detail.

[0035] Figure 1The row control circuitry 104 and the column control circuitry 106 are used to respectively control the rows and columns of event vision pixels 110 in the array 102. For example, the row control circuitry 104 and / or the column control circuitry 106 may be configured to reset specific event vision pixels 110 (e.g., individual ones or rows of event vision pixels 110) of the array 102 and / or read out the event vision pixels 110 (e.g., individual ones or rows of event vision pixels 110) from the array 102 (e.g., along corresponding column bit lines connected to the event vision pixels 110).

[0036] The event data read out from the event vision pixels 110 of the array 102 may be passed to the event signal processor 108 of the event vision sensor 100 for processing. The event data processed by the event signal processor 108 may be provided to a transmitter 116 to transmit the event data out of the event vision sensor 100, such as to a receiver (not shown) of a corresponding imaging system. Additionally or alternatively, all or a subset of the event data may be stored in a memory (e.g., before or after being provided to the event signal processor 108 and / or the transmitter 116).

[0037] The event signal processor 108 of the event vision sensor 100 may additionally be used to identify defective event vision pixels 110 of the array 102. For example, the illumination source 120 may be used to project light 150 (e.g., DC illumination) onto the event vision pixels 110 of the array 102. In some embodiments, the light 150 may have a constant intensity or brightness (or an intensity or brightness that remains substantially constant or changes slowly over time). Thus, when the illumination source 120 continuously projects the light 150 onto the event vision pixels 110 of the array 102, the event vision pixels 110 should not (unless defective or subject to noise) detect an event or detect an event at a frequency above a threshold. The event signal processor 108 may therefore utilize the illumination source 120 to identify defective or noisy event vision pixels 110 of the array 102 and thereafter disable such pixels 110 and / or mask their outputs. hereinafter referred to as Figures 7 to 10 A method of identifying defective event visual pixels 110 of array 102 is discussed in detail.

[0038] 1. Selected embodiments of event signal handlers

[0039] Figure 2 FIG. 2 is a schematic block diagram of a portion of an event signal processor 208 configured in accordance with various embodiments of the present invention. It should be appreciated that the event signal processor 208 illustrated may be Figure 11 or another example of an event vision pixel of the present invention. As shown, the event signal processor 208 includes a defective pixel removal block 270, which includes various stages or circuits that can be used to identify and disable / mask defective event vision pixels of the corresponding event vision sensor. For example, the defective pixel removal block 270 includes a comparator voltage regulator block 271, an event occurrence recording block 272, an event occurrence characterization block 275, and a defective pixel determination and recording block 278.

[0040] The comparator voltage regulator block 271 of the event signal processor 208 may be configured to regulate the voltage generated by the event visual pixel (eg Figure 1 The event of the event vision pixel 110) generates a comparator voltage threshold used by the comparator. For example, the comparator voltage regulator block 271 can be coupled to the peripheral circuit system (e.g. Figure 1 The peripheral circuitry 146 of the visual pixel sensor 100. Using the peripheral circuitry, the comparator voltage regulator block 271 can regulate the voltage generated by the event visual pixel comparator for detecting UP events (e.g., greater than the comparator threshold voltage V U The comparator threshold voltage V U Additionally or alternatively, the comparator voltage regulator block 271 may use peripheral circuitry to regulate the voltage generated by the event visual pixel used by the comparator to detect a DOWN event (eg, greater than a comparator threshold voltage V D The comparator threshold voltage V D .

[0041] As discussed in more detail below, the comparator voltage regulator block 271 of the event signal processor 208 may be used to turn off or disable an event generating comparator of an event vision pixel while another event generating comparator is used to determine whether the event vision pixel is defective. For example, the comparator voltage regulator block 271 may be used to increase the comparator voltage threshold V used by the down event generating comparator of the event vision pixel to D Set to -∞ (or some other large negative voltage value), and the event signal processor 208 uses the up event of the event visual pixel to generate a comparator to determine whether the event visual pixel is defective. D Setting to -∞ (or some other large negative voltage value) may prevent the down event generating comparator from registering an event. Additionally or alternatively, the comparator voltage regulator block 271 may be used to adjust the comparator voltage threshold V used by the up event generating comparator of the event visual pixel to -∞. U Set one or more thresholds (or make the comparator voltage threshold V USweeping across a set of one or more thresholds) to enable, for example, the event occurrence characterization block 275 of the event signal processor 208 to model the probability of detecting a noise event within one or more observation windows.

[0042] The event occurrence record block 272 of the event signal processor 208 is configured to record noise events detected by event visual pixels. For example, the event occurrence record block 272 may record (i) a timestamp corresponding to the time when the noise event was detected and / or (ii) the location (e.g., row address and column address) of the event visual pixel where the noise event was detected. Additionally or alternatively, the event occurrence record block 272 may record the polarity or change of the noise event (e.g., indicating whether the noise event is an UP noise event or a DOWN noise event).

[0043] The event occurrence characterization block 275 may perform various calculations and modeling of noise events detected by the event visual pixel. For example, the event occurrence characterization block 275 may include an event probability distribution building block 276, which is configured to measure, observe and / or model one or more probabilities that the event visual pixel detects a noise event (e.g., at a given comparator voltage threshold) within one or more given observation windows (e.g., with different durations) based at least in part on the noise events recorded by the event occurrence recording block 272. Additionally or alternatively, the event occurrence characterization block 275 may include a statistics calculation block 277 configured to calculate various statistics of the event visual pixel. Examples of statistics that may be calculated by the statistics calculation block 277 include the noise event initiation rate of the event visual pixel, the variance of the probability density function determined by the event probability distribution building block 276, higher-order moments, quantiles, variability of any moment relative to a reference event visual pixel, and / or statistics associated with a group of event visual pixels (e.g., a group of event visual pixels including the event visual pixel and / or one or more event visual pixels falling within a certain distance from the event visual pixel).

[0044] The defective pixel determination and recording block 278 of the event signal processor 208 is configured to identify defective event visual pixels based on (a) statistics output from the statistics calculation block 277 and (b) specified criteria or thresholds. For example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the calculated noise event initiation rate is greater than a threshold. As another example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the variance or other statistics (e.g., other high-order moments) calculated by the statistics calculation block 277 is greater than a threshold. The threshold used by the defective pixel determination and recording block 278 may be predefined (e.g., preset, predetermined). Additionally or alternatively, the threshold used by the defective pixel determination and recording block 278 may depend on statistics related to one or more other (e.g., adjacent) event visual pixels. For example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the noise event initiation rate is greater than the average event occurrence rate corresponding to one or more adjacent event visual pixels (e.g., by a specified amount).

[0045] When the defective pixel determination and recording block 278 determines that the event visual pixel is defective, the defective pixel determination and recording block 278 may store the address (e.g., row address and column address) corresponding to the event visual pixel in a lookup table (e.g., in Figure 1 Additionally or alternatively, the defective pixel determination and recording block 278 may be coupled to control logic (e.g., Figure 1 In these embodiments, the defective pixel determination and recording block 278 may communicate with and / or use the control logic to disable the defective event vision pixel or mask its output to prevent the defective event vision pixel from detecting an event or outputting an event from the event vision pixel, respectively. For example, the defective pixel determination and recording block 278 may instruct the control logic to register a memory component (e.g., a latch, such as a latch) of the defective event vision pixel via the row control circuitry and / or column control circuitry of the corresponding event vision sensor. Figure 1 The latch 136 of the programming is performed. Figures 3 to 6 Circuitry for disabling defect event visual pixels and / or masking their outputs is discussed in greater detail.

[0046] As shown, the event signal processor 208 may additionally include an auxiliary function block 279, which includes circuits or blocks for performing various auxiliary processing functions. For example, the auxiliary function block 279 may include a segment classifier block and / or a shape classifier block that can be used to classify the segmentation and shape of the event data read out from the array, respectively. As another example, the auxiliary function block 279 may include an optical flow estimation block, which can be used to identify pixel-by-pixel, shape-by-shape, or segment-by-segment motion over time and / or between consecutive readouts (e.g., using correlation-based, block-matching-based, feature-tracking-based, energy-based, and / or gradient-based optical flow estimation). In these and other embodiments, the auxiliary function block 279 may include a compression block to compress the event data read out from the array. Additionally or alternatively, the event signal processor 208 may include a set of one or more buffers (e.g., a set of row buffers) that are operably coupled to one or more blocks or circuits of the event signal processor 208 to perform various processing functions.

[0047] 2. Selected Embodiments of Event Vision Pixels

[0048] Figure 3 FIG. 1 is a partially schematic block diagram illustrating an event vision pixel 310 configured according to various embodiments of the present invention. It should be understood that the event vision pixel 310 illustrated may be a pixel included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 310 includes an image sensor configured to respond to incident light 350 (e.g., from an external scene and / or from an illumination source such as Figure 1 The photoelectric sensor 331 is coupled to a logarithmic amplifier 332, which is configured to convert the photocurrent generated by the photosensor 331 into a voltage. In various examples, the logarithmic amplifier 332 is configured to generate a voltage by transducing the instantaneous photocurrent received from the photosensor 331. A difference detection amplifier 333 is coupled to the logarithmic amplifier 332 to generate a filtered and amplified signal in response to a difference detected in the voltages received from the logarithmic amplifier 332. In one example, the difference detection amplifier 333 is configured to compare the instantaneous logarithmic intensity of the voltage output of the logarithmic amplifier 332 with a reference level based on a reset condition or a latest event.

[0049] The event generating comparator 334 is coupled to the difference detection amplifier 333 to compare the filtered and amplified signal received from the difference detection amplifier 333 with a threshold to asynchronously detect an event indicated in the incident light 350. In one example, the event generating comparator 334 is configured to discern whether the signal difference is large enough to trigger an event. In some embodiments, the event generating comparator 334 includes a first comparator (not shown) configured to detect whether the signal difference corresponds to an 'UP' event (e.g., a change in the intensity of light incident on the photosensor 331 from darker to brighter and greater than a comparator voltage threshold V U The event generating comparator 334 may further include a second comparator (not shown) configured to detect whether the signal difference corresponds to a 'DOWN' event (e.g., a change in the intensity of light incident on the photosensor 331 from brighter to darker and greater than a comparator voltage threshold V D ).

[0050] Figure 3 The event vision pixel 310 may further include a scanner and readout logic 335 ("readout logic 335"). In some embodiments, the readout logic 335 may include a latch that is (a) coupled to one or more outputs of the event generating comparator 334 and (b) configured to trigger when the event vision pixel 310 detects an event. Additionally or alternatively, the readout logic 335 may be coupled to a readout control circuit system (e.g., a control circuit) of a corresponding event vision sensor. Figure 1 The readout logic 335 may be configured to read out the event vision pixel 310 whenever the event vision pixel 310 detects an event. In this manner, the readout logic 335 may be configured to ensure that the event vision pixel 310 is read out only when the event vision pixel 310 detects an event. In other words, the readout logic 335 may be configured to ensure that the event vision pixel 310 is not read out when the event vision pixel 310 does not detect an event. When the event vision pixel 310 detects an event, the event may be read out from the event vision pixel 310 onto a column bit line (not shown) corresponding to the event vision pixel 310.

[0051] Therefore, it should be understood that including similar Figure 3 The event vision pixel 310 of the event vision pixel of the event vision sensor (eg Figure 1 The event vision sensor 100 does not need to record the entire conventional image and therefore does not need to capture and record all the highly redundant information of a normal image frame by frame. Specifically, in various embodiments, the event vision sensor can read and record unique events. For example, the event vision sensor can read and / or record the location where the event was detected (e.g. Figure 1The event vision sensor may be used to detect movement or motion in an external scene (e.g., rather than for capturing / reading out a full frame of an image or video), thereby enabling the use of low data rates and achieving ultra-high frame rate or speed capabilities in the event vision sensor of the present invention.

[0052] Continue to refer Figure 3 , the event vision pixel 310 further includes a programmable memory element 336 (illustrated and hereinafter referred to as "latch 336") and a logic gate 337. The latch 336 includes three inputs and one output. The logic gate 337 includes: (a) a first input coupled to the output of the latch 336; (b) one or more second inputs coupled to one or more outputs of the readout logic 335; and (c) one or more outputs coupled to the column bit lines and / or readout control circuitry (e.g., Figure 1 Logic gate 337 is in the row control circuit system 104 and / or column control circuit system 106. Figure 3 An AND gate is shown in FIG. 1 as an example, but may be another logic gate in other embodiments of the present invention.

[0053] Latch 336 is configured to receive a program signal, a row signal, and a column signal at its input. As discussed in more detail below, these signals can be used to program latch 336 and selectively assert a mask signal output from latch 336 and fed into logic gate 337. (also referred to herein as a "deactivation signal ”). The row and column signals input into latch 336 may correspond to the position of event visual pixel 310 in an array of such pixels.

[0054] The mask signal output from latch 336 The state of depends on the input of latch 336. More specifically, in the illustrated example, the mask signal It may be defaulted to not asserted (e.g., it may be in the first state or be "1"). Therefore, under normal operation, the output of the logic gate 337 may follow the output of the readout logic 335. For example, when an event occurs in an external scene monitored by an event vision sensor including an event vision pixel 310, the event may be indicated as a rapid or sudden change in intensity or brightness in the incident light 350 received by the photosensor 331 of the event vision pixel 310. In other words, if the external scene is static and no event occurs, the brightness of the incident light 350 remains substantially unchanged. Thus, the photocurrent generated by the photosensor 331 remains substantially constant and the event vision pixel 310 (unless defective) does not record the event or does not record the event frequently. However, if an event occurs in the external scene (e.g., movement, lighting change, reflection, excitation, etc.), the event is indicated by an asynchronous rapid or sudden change in the brightness of the incident light 350. The brightness change may be from darker to brighter or from brighter to darker. Thus, there is an asynchronous change or Δ in the photocurrent generated by the photosensor 331. The change in photocurrent, or Δ, is converted to a voltage by log amplifier 332, filtered and amplified by difference detection amplifier 333, and then detected with event generation comparator 334. The event may be latched in readout logic 335 until it is read out to the corresponding column bit line via logic gate 337. Additionally or alternatively, readout logic 335 may send a request (e.g., via logic gate 337) to the readout control circuitry to read out the event visual pixel 310. The event may then be read out (via logic gate 337) to the column bit line corresponding to the event visual pixel 310.

[0055] On the other hand, if the event visual pixel 310 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 336 may be used to program the latch 336 and assert the mask signal When the mask signal is asserted When (for example, when shielding the signal When the mask signal 100 is asserted, one or more outputs of the readout logic 335 may be masked via the logic gate 337. For example, when the mask signal 100 is asserted, When , the logic gate 337 (i) can prevent the read request output from the readout logic 335 from reaching the readout control circuit system corresponding to the event vision pixel 310 and / or (ii) can prevent the event detected by the event vision pixel 310 from being output to the column bit line corresponding to the event vision pixel 310.

[0056] When the event visual pixel 310 is identified as a defective pixel, Figure 3Shielding the output of the event vision pixel 310 in the manner shown in FIG. 1 provides several advantages. For example, events detected by the event vision pixel 310 that are most likely due to noise and / or defects in the event vision pixel 310 are not output from the event vision pixel 310 and therefore do not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 310. In addition, only the output of the readout logic 335 is shielded. In other words, the various analog stages of the event vision pixel 310 (such as the logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334, and the readout logic 335) remain powered on. Therefore, the DC bias of each of these analog stages remains enabled, and the event vision pixel 310 (when identified as defective) continues to consume the same relative amount of current as other event vision pixels of the array incorporating the event vision pixel 310 that are not identified as defective. Therefore, even though the output of one or more defective event vision pixels across the array may be shielded, the current consumed by the event vision pixels across the array remains approximately uniform. Therefore, a substantial uniformity of IR drop across the array may be maintained or observed.

[0057] Figure 4 FIG. 4 is a partially schematic block diagram illustrating another event vision pixel 410 configured according to various embodiments of the present invention. It should be understood that the event vision pixel 410 illustrated may be a pixel included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 410 includes an image sensor configured to respond to incident light 450 (e.g., from an external scene and / or from an illumination source such as a Figure 1 The photoelectric sensor 431 receives the light from the illumination source 120 and generates a photoelectric charge or a photocurrent. The event vision pixel 410 further includes a logarithmic amplifier 432, a difference detection amplifier 433, an event generation comparator 434, and a readout logic 435. The logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 can be roughly similar to Figure 3 The logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434 and the readout logic 435 is omitted here.

[0058] Event vision pixel 410 additionally includes a programmable memory element 436 (illustrated and hereinafter referred to as "latch 436"). Latch 436 is substantially similar to Figure 3 Therefore, for the sake of brevity, the detailed discussion of latch 436 is basically omitted here. However, unlike Figure 3 The latch 336, Figure 4The latch 436 is coupled to the gate of the transistor 438, which acts as a switch to disable the latch 436 based on the deactivation signal output from the latch 436. The state of the event vision pixel 410 logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 selectively couples the power supply voltage V DD As discussed above, the deactivation signal The default is not asserted (e.g., it may be in the first state or "1"). Therefore, under normal operation, the logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 may be coupled to the power supply voltage V via the transistor 438. DD And the power supply voltage V DD On the other hand, when the event visual pixel 410 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 436 may be used to program the latch 436 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the transistor 438 is deactivated, the logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 of the event vision pixel 410 are connected to the power supply voltage V DD The decoupling prevents the event vision pixel 410 from detecting an event.

[0059] When the event visual pixel 410 is identified as a defective pixel, Figure 4 Preventing the event vision pixel 410 from detecting an event in the manner shown in FIG. 4 provides several advantages. For example, (i) an event that is attributable to noise and / or defects in the event vision pixel 410 and (ii) that would otherwise be detected by the event vision pixel 410 is prevented from being generated or output from the event vision pixel 410 and thus does not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 410. In addition, when the event vision pixel 410 is identified as defective, the analog stage of the event vision pixel 410 is prevented from consuming current. Thus, the power supply voltage V DD 410 while saving power that would otherwise be consumed by the analog stages of event vision pixel 410.

[0060] That is, shutting off the DC bias of each of the analog stages of the event vision pixel 410 in this manner may result in non-uniform current consumption across the event vision pixels of the array that includes the event vision pixel 410. In other words, when several event vision pixels of the array are identified as defective, non-uniformity of IR voltage drop across the array may be observed. Having recognized this problem, only a subset of the analog stages of the event vision pixel (e.g., representing any combination of logarithmic amplifiers, difference detection amplifiers, event generation comparators, and / or readout logic that are not all analog stages) may be disabled in some embodiments of the present invention when the event vision pixel is identified as defective (e.g., so that uniformity of IR voltage drop across the array may be substantially maintained or observed).

[0061] As a specific example, see Figure 5 , which is a partially schematic block diagram illustrating yet another event vision pixel 510 configured according to various embodiments of the present invention. It should be understood that the illustrated event vision pixel 510 may be included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 510 includes an image sensor configured to respond to incident light 550 (e.g., from an external scene and / or from an illumination source such as Figure 1 The event vision pixel 510 further includes a logarithmic amplifier 532, a differential detection amplifier 533, an event generation comparator 534, and a readout logic 535. The logarithmic amplifier 532, the differential detection amplifier 533, the event generation comparator 534, and the readout logic 535 can be roughly similar to Figure 3 The logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 532, the difference detection amplifier 533, the event generation comparator 534 and the readout logic 535 is omitted here.

[0062] Event vision pixel 510 additionally includes a programmable memory element 536 (illustrated and hereinafter referred to as "latch 536"). Latch 536 is substantially similar to Figure 4 Therefore, for the sake of brevity, the detailed discussion of latch 536 is basically omitted here. However, unlike Figure 4 The latch 436, Figure 5 The latch 536 is coupled to the gate of the transistor 538, which acts as a switch to disable the latch 536 based on the deactivation signal output from the latch 536. The state selectively couples only the logarithmic amplifier 532 of the event vision pixel 510 (rather than all analog stages) to the power supply voltage V DD Therefore, in the deactivation signal In normal operation, where the default is not asserted (eg, in the first state or “1”), the logarithmic amplifier 532 may be coupled to a power supply voltage V DD And the power supply voltage V DD On the other hand, when the event visual pixel 510 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 536 may be used to program the latch 536 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the transistor 538 is deactivated to connect only the logarithmic amplifier 532 of the event vision pixel 510 to the power supply voltage V DD Decoupling the logarithmic amplifier 532 from the power supply voltage V DD The decoupling may interrupt the photocurrent path from the photosensor 531 to the differential sense amplifier 533 to prevent the event vision pixel 510 from detecting the event.

[0063] When the event visual pixel 510 is identified as a defective pixel, Figure 5 Preventing the event vision pixel 510 from detecting an event in the manner shown in provides several advantages. For example, events that are (i) attributable to noise and / or defects in the event vision pixel 510 and (ii) that would otherwise be detected by the event vision pixel 510 are prevented from being generated or output from the event vision pixel 510 and thus do not consume the available bandwidth of the event vision sensor incorporated into the event vision pixel 510. In addition, when the event vision pixel 510 is identified as defective, the logarithmic amplifier 532 of the event vision pixel 510 is prevented from consuming current. Thus, power that would otherwise be consumed by the logarithmic amplifier 532 of the event vision pixel 510 is saved. Furthermore, when the event vision pixel 510 is identified as defective, only a subset of the analog stages of the event vision pixel 510 (here, only the logarithmic amplifier 532) is disconnected from the power supply. In other words, at least some of the other analog stages of the event vision pixel 510 (e.g., here, the difference detection amplifier 533, the event generation comparator 534, and the readout logic 535) remain powered on. Thus, the DC bias of each of these other stages remains enabled and the event vision pixel 510 continues to consume an amount of current that is relatively similar to the amount of current consumed when the event vision pixel 510 is not identified as defective (less than the amount of current consumed due to the logarithmic amplifier 532 and the power supply voltage V DD Therefore, even if a defect event occurs across the array, the number of logarithmic amplifiers of the visual pixels and the power supply voltage V DD By turning off, the current consumed by the event visual pixels across the array can remain substantially uniform. Substantially uniform IR voltage drop across the array can then be substantially maintained or observed.

[0064] Other methods of disabling defective event visual pixels and / or masking their outputs are certainly possible and fall within the scope of the present invention. For example, a programmable memory component may be provided so that the event generating comparator may be based on a disable signal output from the programmable memory component. Thus, continuing with this example, when an event visual pixel is identified as defective, the memory component may be programmed to assert a disable signal The comparator is turned off and / or decoupled from the power supply voltage, thereby making the event vision pixel unable to detect the event.

[0065] As another example, a programmable memory component may be coupled to a differential detection amplifier of an event visual pixel to detect a disable signal output from the programmable memory component. To selectively enable the differential detection amplifier. As a specific example, refer to Figure 6 , which is a partially schematic block diagram illustrating yet another event vision pixel 610 configured according to various embodiments of the present invention. It should be understood that the illustrated event vision pixel 610 may be included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 610 includes an image sensor configured to respond to incident light 650 (e.g., from an external scene and / or from an illumination source such as Figure 1 The photoelectric sensor 631 receives the light from the illumination source 120 and generates a photoelectric charge or a photocurrent. The event vision pixel 610 further includes a logarithmic amplifier 632, a difference detection amplifier 633, an event generation comparator 634, and a readout logic 635. The logarithmic amplifier 632, the event generation comparator 634, and the readout logic 635 can be roughly similar to Figure 3 The logarithmic amplifier 332, the event generating comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 632, the event generating comparator 634 and the readout logic 635 is omitted here.

[0066] For clarity and understanding, Figure 6 Detailed description of the difference detection amplifier 633 is given in FIG. As shown, the difference detection amplifier 633 includes a first capacitor 622, a second capacitor 623, an amplifier 624, and a reset transistor 625. The amplifier 624 includes an input (e.g., an inverting input) and an output. The first capacitor 622 is coupled between the output of the logarithmic amplifier 632 and the input of the amplifier 624, and the second capacitor 623 is coupled between the input of the amplifier 624 and the output of the amplifier 624. The reset transistor 625 is coupled between the input of the amplifier 624 and the output of the amplifier 624.

[0067] The first capacitor 622, the second capacitor 623, the amplifier 624, and the reset transistor 625 form a filter amplifier that is configured to produce a filtered and amplified signal in response to a voltage output by the logarithmic amplifier 632 of the event vision pixel 610. More specifically, the filter amplifier includes a high pass filter configured to filter out lower frequency components from the voltage received by the logarithmic amplifier 632. Thus, the event vision pixel 610 can ignore slow or gradual changes in the photocurrent generated by the photosensor 631 in response to the incident light 650, and can instead detect rapid and sudden changes that occur in the photocurrent generated by the photosensor 631 in response to the incident light 650. Additional details regarding the difference detection circuit and associated event vision pixels are provided in U.S. Patent Application No. 17 / 875,244, which is incorporated herein by reference in its entirety.

[0068] As shown, the event vision pixel 610 also includes a programmable memory component 636 (illustrated and hereinafter referred to as a "latch 636"). The latch 636 is substantially similar to Figure 3 636. Therefore, for the sake of brevity, detailed discussion of latch 636 is basically omitted here. The output of latch 636 is coupled to the input of inverter 639, and the output of inverter 639 is coupled to the input of logic gate 637. Logic gate 637 further includes (a) an input configured to receive reset signal RST and (b) an output coupled to the gate of reset transistor 625. Logic gate 637 is Figure 6 An OR gate is illustrated as an example, but in other embodiments of the present invention, it may be another logic gate that may or may not include the inverter 639.

[0069] The reset transistor 625 of the event vision pixel 610 is arranged as a reset switch and is configured to selectively couple the input of the amplifier 624 to the output of the amplifier 624 based on the output of the logic gate 637. Under normal operation by default being not asserted (e.g., in a first state or "1"), the output of the logic gate 637 may follow the reset signal RST. Specifically, when the reset signal RST is asserted, the reset transistor 625 may couple the input of the amplifier 624 to the output of the amplifier 624 to automatically zero the amplifier 624. When the reset signal RST is not asserted, the reset transistor 625 may decouple the input of the amplifier 624 from the output of the amplifier 624.

[0070] On the other hand, when the event visual pixel 610 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 636 may be used to program the latch 636 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the reset transistor 625 transitions to the second state or "0", the reset transistor 625 may remain activated to couple the input of the amplifier 624 to the output of the amplifier 624. Keeping the amplifier 624 auto-zeroed in this manner may disable or prevent the event vision pixel 610 from detecting an event.

[0071] When the event visual pixel 610 is identified as a defective pixel, Figure 6 Preventing the event vision pixel 610 from detecting an event in the manner shown in provides several advantages. For example, events that are (i) attributable to noise and / or defects in the event vision pixel 610 and (ii) that would otherwise be detected by the event vision pixel 610 are prevented from being generated or output from the event vision pixel 610 and thus do not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 610. In addition, each of the analog stages of the event vision pixel 610 continues to consume static power. In other words, the DC bias of each of the analog stages remains enabled and the event vision pixel 610 continues to consume an amount of current that is relatively similar to the amount of current consumed when the event vision pixel 610 is not identified as defective. Therefore, even if several defective event vision pixels across the array are unable to detect an event, the current consumed by the event vision pixels across the array can remain substantially uniform. Then, a substantial uniformity of IR drop across the array can be substantially maintained or observed.

[0072] 3. Related methods

[0073] Figure 7 7 is a flow chart illustrating a method 780 of identifying and disabling / masking defective event vision pixels according to various embodiments of the present invention. The method 780 is illustrated as a set of steps or blocks 781 to 787. All or a subset of one or more of blocks 781 to 787 may be performed by an event vision sensor (e.g., Figure 1 100). For example, all or a subset of one or more of blocks 781-787 may be performed by: (i) an event signal processor, (ii) an event vision pixel (e.g., a programmable memory component of the event vision pixel), (iii) a row control circuit system, (iv) a column control circuit system, (v) control logic, (vi) a lookup table, and / or (vii) a peripheral circuit system. Additionally or alternatively, all or a subset of blocks 781-787 may be performed by an illumination source (e.g., Figure 1 In addition, any one or more of blocks 781 to 787 may be performed according to the above Figures 1 to 6 discussion to be carried out.

[0074] Figure 7Method 780 begins at box 781, which determines the noise event initiation rate of the event visual pixel. Determining the noise event initiation rate may include determining the noise event initiation rate of the event visual pixel with one or more specified comparator thresholds in at least one event generation comparator for the event visual pixel. In these and other embodiments, determining the noise event initiation rate may include using only one event generation comparator (e.g., an upward event generation comparator or a downward event generation comparator) of the event visual pixel to determine the noise event initiation rate. In these embodiments, other event generation comparators of the event visual pixel may be disabled and / or turned off. For example, assuming that the upward event generation comparator of the event visual pixel is used to determine the noise event initiation rate of the event visual pixel, then the downward event generation comparator of the event visual pixel may be turned off or the comparator threshold value V in the downward event generation comparator may be used. D V may be set to -∞ (or some other large negative voltage value). As another example, assuming that the downward event generation comparator of the event vision pixel is used to determine the noise event initiation rate of the event vision pixel, the upward event generation comparator of the event vision pixel may be turned off or used as the comparator threshold V in the upward event generation comparator. U may be set to ∞ (or some other large positive voltage value). Alternatively, determining the event initiation rate may include using multiple event generating comparators of the event vision pixel (e.g., both an up event generating comparator and a down event generating comparator) to determine the event initiation rate or simultaneously enabling multiple event generating comparators of the event vision pixel to detect noise events.

[0075] In some embodiments, the noise event triggering rate of an event visual pixel may be determined by measuring (e.g., observing) the probability of the event visual pixel detecting a noise event ("triggering") at various time intervals of different durations after the event visual pixel is reset. For example, assuming that: (a) the probability of detecting a noise event (P fire ) + the probability of not detecting a noise event As shown in Equation 1 below; (b) The probability of detecting a noise event within a time step Δt (P fire(Δt)) is proportional to the time step Δt, as shown in the following equation 2; and (c) the probability that the event visual pixel detects a noise event at any time point t is equal to all other time points t (for example, the random triggering process for the event visual pixel to detect the noise event is "white" or the probability that the event visual pixel detects a noise event within the time step Δt is an independent and identically distributed random variable for each time step Δt), thereby allowing the following equation 3, then the probability that the event visual pixel detects a noise event over time can be modeled in exponential form and using the following equation 9, where α is the expected noise event trigger rate of the event visual pixel. More specifically, α can represent the noise event trigger rate of a single noise event within the time step Δt and can be used to refer to the following Figures 8 to 10 Any of the methods described in detail can be used to determine or approximate the representation.

[0076] Equation 1: Equation 2: P fire (Δt) = α·Δt

[0077] Equation 3:

[0078]

[0079] Equation 4:

[0080]

[0081] In other words, Equation 9 indicates that the probability that an event vision pixel detects a noise event increases exponentially with time. In other words, the probability that an event vision pixel does not detect a noise event decreases exponentially with time.

[0082] Figure 8 8 is a flow chart illustrating a method 890 for determining a noise event initiation rate of an event vision pixel according to various embodiments of the invention. In particular, the method 890 can be used to determine the noise event initiation rate using a 'slope method'. As shown, the method 890 begins at block 891, which exposes the event vision pixel to constant illumination. Exposing the event vision pixel to constant illumination may include exposing the event vision pixel to DC illumination, such as DC illumination projected onto the event vision pixel using an illumination source. In other embodiments, exposing the event vision pixel to constant illumination may include shielding the event vision pixel from ambient illumination (e.g., in an external scene), such as by placing a cap over a lens of an imaging system incorporating the event vision pixel. The event vision pixel may be exposed to constant illumination for block 892 discussed below.

[0083] In block 892, the method 890 then determines (e.g., measures, observes, identifies, approximates, estimates) the probability that the event visual pixel detects the noise event over time. Determining the probability that the event visual pixel detects the noise event over time may include observing whether the event visual pixel detects the noise event within one or more time intervals starting from the event visual pixel reset (e.g., at reset, upon reset, after reset, after reset). The time intervals may each have a set (e.g., preset, predetermined, predefined) duration T. For example, the method 890 may include stepping through a set of time intervals having different durations T. More specifically, the method 890 may include observing whether the event visual pixel detects the noise event within a first time interval starting from the event visual pixel reset. The first time interval may have a first duration T. Thereafter, the method 890 may include observing whether the event visual pixel detects the noise event within a second time interval starting from the event visual pixel reset. The second time interval may have a second duration T different from the first duration T.

[0084] As a specific example, block 892 of method 890 at Figure 8 890 as having several sub-steps 892a to 892d. In box 892a, the duration of the time interval is set or adjusted. For example, the duration of the time interval may be set to the first duration in a set (e.g., predetermined, preselected, predefined) of durations. In box 892b, the event visual pixel is reset, thereby starting the time interval. In box 892c, an indication of whether the event visual pixel detected a noise event during the time interval is recorded. For example, when the event visual pixel detects a noise event during the time interval, the method 890 may include recording (e.g., at the end of the time interval or when the noise event is detected) an indication that the event visual pixel detected the noise event. Additionally or alternatively, the method 890 may include recording a timestamp indicating the time when the event visual pixel detected the noise event, the polarity of the noise event, and / or a location (e.g., a row address and / or a column address) corresponding to the event visual pixel. When the event visual pixel does not detect a noise event during the time interval, the method 890 may include recording (e.g., at the end of the time interval or after) an indication that the event visual pixel did not detect the noise event.

[0085] In box 892d, the method 890 then determines whether to collect additional data for modeling the probability that the event visual pixel detects a noise event over time. For example, box 892 may be repeated several times to collect sufficient data for determining (e.g., estimating, approximating, identifying) the probability that the event visual pixel detects a noise event over time. As a specific example, boxes 892a to 892c may be repeated for the first time in the same time interval with a duration T, so that the method 890 includes collecting sufficient data for determining (e.g., estimating, approximating, identifying) the probability that the event visual pixel detects a noise event in this time interval. The first number of times that boxes 892a to 892c are repeated in the same time interval may be preset (e.g., predetermined, predefined). In addition or alternatively, boxes 892a to 892c may be repeated for the second time in different time intervals with different durations T. For example, the method 890 may return from box 892d to box 892a and adjust the duration T of the time interval to another duration T (e.g., in a set of predetermined durations T used by the method 890 to measure the probability that the event visual pixel detects a noise event). The second number of repetitions of blocks 892a through 892c within different time intervals T having different durations T may be preset (eg, predetermined, predefined).

[0086] When method 890 determines to collect additional probability data (block 892d: yes), method 890 may return to block 892a or block 892b. In the illustrated example, method 890 returns to block 892a to adjust the time interval duration T to correspond to a different time interval. In other non-illustrated examples of method 890, method 890 may return from block 892d to block 892b to maintain the current duration T so that method 890 includes collecting additional probability data within the same time interval. On the other hand, when method 890 determines not to collect additional probability data for event visual pixels (block 892d: no), method 890 may proceed to block 893.

[0087] In block 893, method 890 then obtains a probability density function for the event visual pixel. Obtaining the probability density function for the event visual pixel may include obtaining the probability density function based at least in part on the probability that the event visual pixel detects a noise event over time determined in block 892. For example, obtaining the probability density function for the event visual pixel may include: (i) plotting a graph for the probability data measured for the event visual pixel in block 892; (ii) fitting a curve to the measured probability data; and / or (iii) finding the derivative of the probability data and / or the fitted curve with respect to time t. As discussed above, it may be assumed that the probability that the event visual pixel detects a noise event over time may take an exponential form and be modeled using equation 9 above. Therefore, the probability density function for the event visual pixel may be modeled using equation 10 below, which represents the derivative of equation 9 above with respect to time t.

[0088] Equation 10: In the above equation 10, α is the decay rate of the exponential and may represent the average trigger rate of event visual pixels (or the slope noise event trigger rate).

[0089] In block 894, method 890 then determines the slope noise event initiation rate for the event visual pixel. For example, the slope noise event initiation rate for the event visual pixel can be solved directly using the 'slope method'. More specifically, the slope noise event initiation rate can be solved directly as follows: (i) taking the natural logarithm of the probability density function for the event visual pixel determined in block 893 (as shown in Equation 11 below); and (ii) thereafter taking the derivative of the natural logarithm of the probability density function to isolate the slope noise event initiation rate α (as shown in Equation 12 below).

[0090] Equation 11: ln[p fire (t)]=ln[α]-α·t

[0091] Equation 12: In other words, the measurement of the probability of detecting a noise event for an event visual pixel in block 892 of method 890 may be used to determine a probability density function for the event visual pixel in block 893, which may then be used to directly solve for the slope noise event initiation rate α for the event visual pixel in block 894. The slope noise event initiation rate α calculated in block 894 may be determined as Figure 7 The noise event triggering rate of the event visual pixel of box 781 of method 780.

[0092] Fig. 9 900 is a flow chart illustrating another method 900 for determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention. In particular, method 900 can be used to determine a noise event initiation rate using a 'curve fitting method'. Method 900 begins at box 901, which exposes the event visual pixel to constant illumination. The event visual pixel can be exposed to constant illumination for box 902 discussed below. In box 902, method 900 then determines (e.g., measures, observes, identifies, approximates, estimates) the probability that the event visual pixel detects a noise event over time. In box 903, method 900 then obtains a probability density function of the event visual pixel. Boxes 901 to 903 of method 900 are substantially similar to those discussed in detail above. Figure 8 Therefore, for the sake of brevity, the blocks 891 to 893 of the method 890 are omitted here. Fig. 9 Detailed discussion of blocks 901 through 903 of method 900.

[0093] In block 904, method 900 then determines the slope noise event initiation rate of the event visual pixel. For example, rather than using the slope method to determine the slope noise event initiation rate (e.g., Figure 8 Instead of performing the above operation in block 894 of method 890 of FIG. 8 , method 900 may include fitting (in block 904) the curve to the probability density function obtained for the event visual pixel in block 903. Thereafter, the slope noise event initiation rate α may be determined from the fitted curve and may be determined as Figure 7 The noise event triggering rate of the event visual pixel of box 781 of method 780.

[0094] Fig.10 1010 is a flow chart illustrating yet another method 1010 for determining a noise event initiation rate for an event visual pixel according to various embodiments of the present invention. In particular, method 1010 can be used to determine a noise event initiation rate using a 'direct method'. Method 1010 begins at block 1011, which exposes the event visual pixel to constant illumination. The event visual pixel may be exposed to constant illumination for block 1012. In block 1012, method 1010 then determines (e.g., measures, observes, identifies, approximates, estimates) the probability that the event visual pixel detects a noise event over time. Blocks 1011 and 1012 of method 1010 are substantially similar to those discussed in detail above. Figure 8 Therefore, for the sake of brevity, the blocks 891 and 892 of the method 890 are omitted here. Fig.10 Detailed discussion of blocks 1011 and 1012 of method 1010.

[0095] In block 1013, the method 1010 then determines the noise event triggering rate of the event visual pixel. For example, the noise event triggering rate may be determined explicitly from the measured probability that the event visual pixel detects a noise event over time (e.g., using a 'direct method'). More specifically, as discussed above, it may be assumed that the probability that the event visual pixel detects a noise event over time may take an exponential form and may be modeled using Equation 9 above. Thus, the triggering rate α of the event visual pixel may be determined directly by isolating the triggering rate α, as shown in Equation 13 below.

[0096] Equation 13: In other words, Equation 13 may be used to directly determine the triggering rate α of event visual pixels within a time interval of a given duration based at least in part on the measurement of the probability of event visual pixels detecting a noise event over time made in block 1012. The triggering rate α may be determined as Figure 7 The noise event initiation rate of the event visual pixel of block 781 of method 780 of FIG. 780 may be particularly robust to event visual pixels that exhibit large changes in noise event initiation rate over time.

[0097] In the above Fig.10 In block 1012 of method 1010, it is assumed that for each measurement, the event visual pixel detects a single noise event during a time interval of a given duration or detects no noise event during a time interval of a given duration. In other embodiments of the present invention, the event visual pixel may detect any number of noise events during a time interval of a given duration. In these embodiments, method 1010 may be extended to (i) counting the number of noise events detected by the event visual pixel during the time interval and (ii) determining an average noise event count within one or more time intervals of a specified duration. Based on these measurements, method 1010 may use, for example, a maximum likelihood estimation technique to determine a trigger rate α that most likely explains the observed average noise event count measurement. This trigger rate α may be determined as Figure 7 The noise event triggering rate of the event visual pixel of box 781 of method 780.

[0098] Reference again Figure 7 Once the noise event initiation rate is determined in box 781, method 780 proceeds to box 782, which compares the noise event initiation rate to one or more thresholds. In some embodiments, the one or more thresholds include a predetermined initiation rate threshold. The predetermined initiation rate threshold may be set to a value at or above which the event visual pixel may be defective (e.g., unacceptably likely to record or unacceptably prone to recording noise events). Additionally or alternatively, the one or more thresholds may include or be based at least in part on an average noise event initiation rate of a group of event visual pixels that include or exclude the event visual pixel of interest. For example, in box 781, method 780 may obtain the noise event initiation rate of the event visual pixel of interest and the noise event initiation rate of each of one or more adjacent event visual pixels (e.g., event visual pixels within a threshold distance from the event visual pixel of interest). The method may then calculate an average of the noise event initiation rate and may compare the noise event initiation rate of the event visual pixel of interest with the average noise event initiation rate.

[0099] In block 783, method 780 then determines whether the noise event initiation rate exceeds one or more thresholds from block 782. For example, when one or more thresholds from block 782 include a predetermined initiation rate threshold, method 780 may proceed to block 784 when the noise event initiation rate from 781 exceeds the predetermined initiation rate threshold (block 783: yes). As another example, when one or more thresholds from block 782 include an average noise event initiation rate (e.g., of a set of event visual pixels adjacent to the event visual pixel of interest), method 780 may proceed to block 784 when the noise event initiation rate exceeds the average noise event initiation rate by more than a certain amount (block 783: yes). On the other hand, when method 780 determines that the noise event initiation rate from block 781 does not exceed one or more thresholds from block 782 (block 783: no), method 780 may proceed to block 786.

[0100] In block 784, the method 780 then identifies the event vision pixel as defective. Identifying the event vision pixel as defective may include recording an address (e.g., a row address and / or a column address) corresponding to the event vision pixel in a lookup table. As discussed above, the lookup table may be used to later (e.g., after powering on or initializing the event vision sensor) identify an address corresponding to the defective event vision pixel. The method 780 may additionally or alternatively instruct the control logic to disable the event vision pixel and / or mask its output in block 784.

[0101] In block 785, the method 780 then disables the event vision pixel and / or masks its output. In some embodiments, disabling the event vision pixel may include programming memory components of the event vision pixel (e.g., causing one or more stages of the event vision pixel to be turned off and / or decoupled from a power supply voltage, one or more event generating comparators of the event vision pixel to be turned off, a differential detection amplifier of the event vision pixel to remain auto-zeroed, and / or the event vision pixel to be otherwise prevented from detecting noise events). In these and other embodiments, shielding the event vision pixel may include programming memory components of the event vision pixel (e.g., causing noise events detected by the event vision pixel to not be output from the event vision pixel and / or causing read requests output by readout logic of the event vision pixel to be masked).

[0102] In block 786, method 780 determines whether there are additional event visual pixels to analyze as possibly defective. When method 780 determines that there are additional event visual pixels to analyze (block 786: yes), method 780 may return to block 781. Otherwise, when method 780 determines that there are no additional event visual pixels to analyze (block 786: no), method 780 may proceed to block 787 to terminate.

[0103] Although blocks 781 through 787 of method 780 are discussed and illustrated in a particular order, Figure 7 The method 780 described in is not limited thereto. In other embodiments, the method 780 may be performed in a different order. In these and other embodiments, any of the blocks 781 to 787 of the method 780 may be performed before, during and / or after any of the other blocks 781 to 787 of the method 780. For example, the method 780 may expose all or a subset of the event vision pixels of the array to constant illumination simultaneously and thereafter determine the event triggering rates of these event vision pixels at approximately the same time (e.g., sequentially or simultaneously). As another example, the method 780 may perform one or more of the blocks 781 to 786 for multiple event vision pixels simultaneously. Again, a person of ordinary skill in the relevant art will recognize that the method 780 described may be modified and still remain within these and other embodiments of the present invention. For example, in some embodiments, one or more blocks 781 to 787 of the method 780 may be omitted and / or repeated. As another example, block 781 may be a given comparator threshold voltage (e.g., a given comparator threshold voltage V for an upward event generating comparator for an event vision pixel). U and / or a given comparator threshold voltage V for the down event generating comparator of the event visual pixel D Additionally or alternatively, blocks 781 to 786 may be repeated for multiple comparator threshold voltages. For example, method 780 repeatedly performs block 781 while stepping or scanning through a particular group of comparator threshold voltages V U and / or V D The method 780 may then determine whether the event visual pixel is defective based on one or more noise event initiation rates determined for the event visual pixel at one or more comparator voltage thresholds.

[0104] C. in conclusion

[0105] The above detailed description of the embodiments of the present invention is not intended to be exhaustive or to limit the present invention to the precise form disclosed above. Although specific embodiments and examples of the present invention are described above for illustration, those skilled in the relevant art will recognize that various equivalent modifications may be within the scope of the present invention. For example, although the steps are presented above in a given order, alternative embodiments may perform the steps in a different order. In addition, the various embodiments described herein may also be combined to provide additional embodiments.

[0106] It can be understood from the above that the specific embodiments of the present invention have been described herein for illustration, but the well-known structures and functions are not shown or described in detail so as not to unnecessarily obscure the description of the embodiments of the present invention. When any material incorporated herein by reference conflicts with the present disclosure, the present disclosure controls. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. In addition, unless the term "or" is explicitly limited to only mean that a single item excludes other items in a list involving two or more items, the use of "or" in this list should be interpreted as including any single item in the (a) list, all items in the (b) list, or any combination of items in the (c) list. In addition, as used herein, the phrase "and / or" in "A and / or B" refers to only A, only B, and both A and B. In addition, the terms "include", "include", "have", and "contain" are used throughout to mean that at least the listed features are included to not exclude any more of the same features and / or other features of additional types. Furthermore, as used herein, the phrases "based on", "depending on", "due to", and "in response to" should not be interpreted as referring to a set of closed conditions. For example, an exemplary step described as "based on condition A" may be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "based at least in part on" or the phrase "based at least in part on." In addition, the terms "connected" and "coupled" may be used interchangeably herein and refer to both direct and indirect connections or couplings. For example, element A "connected" or "coupled" to element B may refer to (i) A being directly "connected" or "coupled" to B and / or (ii) A being indirectly "connected" or "coupled" to B, where the context permits.

[0107] It can also be understood from the above that various modifications can be made without departing from the present disclosure or the present invention. For example, it should be understood by those of ordinary skill in the art that the various components of the present invention can be further divided into subcomponents or can be combined and integrated with the various components and functions of the present invention. In addition, certain aspects of the technology described in the context of a particular embodiment may also be combined or eliminated in other embodiments. In addition, although the advantages associated with certain embodiments of the present invention have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments necessarily need to exhibit such advantages to fall within the scope of the present invention. Therefore, the present disclosure and associated technology may cover other embodiments that are not explicitly shown or described herein.

Claims

1. An event vision sensor, comprising: an array of event vision pixels arranged in rows and columns; and an event signal processor configured to identify defect event visual pixels corresponding to the array based at least in part on a noise event initiation rate, wherein the noise event initiation rate is a measure of a probability of detecting a noise event over time for the defect event visual pixels based at least in part on the defect event visual pixels.

2. The event vision sensor of claim 1 , wherein the measure of the probability comprises an indication of whether the defect event vision pixel detected a noise event during one or more time intervals having one or more durations each starting from a reset of the defect event vision pixel.

3. The event vision sensor according to claim 1, wherein: To identify the defect event visual pixels, the event signal processor is configured to determine a probability density function corresponding to the defect event visual pixels; and The probability density function is based at least in part on the measure of the probability.

4. The event vision sensor of claim 3, wherein in order to identify the defect event vision pixel, the event signal processor is further configured to determine the noise event initiation rate corresponding to the defect event vision pixel based at least in part on a derivative of the natural logarithm of the probability density function.

5. The event vision sensor of claim 3, wherein in order to identify the defect event vision pixels, the event signal processor is further configured to determine the noise event initiation rate corresponding to the defect event vision pixels based at least in part on a curve fit with the probability density function.

6. The event vision sensor of claim 1 , wherein to identify the defect event vision pixels, the event signal processor is configured to directly solve the noise event initiation rate corresponding to the defect event vision pixels using an exponential equation that models the measurement of the probability.

7. The event vision sensor according to claim 1, wherein the defective event vision pixel comprises a programmable memory component, which can be used to mask the output of the event vision pixel so that the event detected by the defective event vision pixel is not output from the defective event vision pixel to the event signal processor.

8. The event vision sensor of claim 1, wherein the defective event vision pixel comprises: a photosensor configured to generate a photocurrent in response to incident light; a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent-to-voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent-to-voltage converter; at least one event generating comparator coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light; and A programmable memory component can be used to disable the photocurrent / voltage converter, the difference detection circuit, the at least one event generating comparator, or any combination thereof so that the defective event vision pixel cannot detect an event.

9. The event vision sensor according to claim 1, further comprising: a lookup table configured to store addresses of defect event visual pixels of the array identified by the event signal processor; and control logic configured, based at least in part on the addresses stored in the lookup table, to program programmable memory components of the defective event vision pixels after powering on or initializing the event vision sensor such that the defective event vision pixels are disabled or such that an output of each of the defective event vision pixels is masked.

10. The event vision sensor of claim 1, wherein to identify the defective event vision pixels, the event signal processor is configured to compare the noise event initiation rate with a threshold. 11 . The event vision sensor of claim 10 , wherein the threshold value represents an average noise event initiation rate corresponding to one or more event vision pixels of the array adjacent to the defect event vision pixel.

12. A method comprising: identifying an event vision pixel of an event vision sensor as defective based at least in part on a noise event initiation rate for the event vision pixel, wherein the noise event initiation rate is a measure based at least in part on a probability of detecting a noise event for the event vision pixel over time; and In response to identifying the event vision pixel as defective, the event vision pixel is prevented from outputting event data.

13. The method of claim 12, further comprising capturing the measure of the probability, wherein capturing the measure of the probability comprises recording whether the event vision pixel detects a noise event during one or more time intervals after the event vision pixel is reset.

14. The method of claim 13, wherein capturing the measure of the probability further comprises: exposing the event visual pixel to constant illumination for the entire duration of the one or more time intervals; and The event visual pixel is observed to see whether the noise event is detected.

15. The method of claim 12, further comprising determining a probability density function corresponding to the event visual pixel based at least in part on the measure of the probability.

16. The method of claim 15, further comprising determining the noise event initiation rate based at least in part on a derivative of a natural logarithm of the probability density function.

17. The method of claim 15, further comprising determining the noise event initiation rate based at least in part on a curve fit to the probability density function.

18. The method of claim 12, further comprising directly solving for the noise event initiation rate based on an exponential equation that models the measure of the probability.

19. The method of claim 12, wherein identifying the event visual pixel as defective comprises (a) comparing the noise event initiation rate to a preset threshold and (b) determining that the noise event initiation rate exceeds the preset threshold.

20. The method of claim 12, wherein identifying the event vision pixel as defective comprises (a) determining an average noise event initiation rate based at least in part on noise event initiation rates associated with one or more event vision pixels adjacent to the event vision pixel in an array of the event vision sensor, (b) comparing the noise event initiation rate to the average noise event initiation rate, and (c) determining that the noise event initiation rate exceeds the average noise event initiation rate by greater than a threshold amount.

21. The method of claim 12, wherein identifying the event vision pixel as defective comprises identifying the event vision pixel as defective during wafer testing, during initialization of the event vision sensor, after determining that a predetermined amount of time has elapsed, or after determining that an event triggering rate of one or more event vision pixels of the event vision sensor exceeds a threshold.

Images