Event vision sensors with defective pixel suppression and associated systems, devices, and methods

Abstract

Event vision sensors with defective pixel suppression and associated systems, devices, and methods are disclosed herein. In one embodiment, an event vision sensor includes an event vision pixel array and an event signal processor. The event signal processor is configured to identify a defective event visual pixel of the array based on a noise event initiation rate corresponding to the event visual pixel. The noise event initiation rate for each event visual pixel may be based on an interval of arrival time or a noise event rate corresponding to the event visual pixel. Each event visual pixel may include internal circuitry (e.g., a memory component, such as a latch) that may be used to disable the event visual pixel to detect an event or mask the output of the event visual pixel such that an event cannot be read out from the event visual pixel when the event visual pixel is identified as defective.

Description

Technical Field

[0001] The present disclosure relates generally to event vision sensors. For example, several embodiments of the present invention relate to event vision sensors that employ defective pixel suppression, such as intra-pixel defective pixel suppression based on noise event initiation rate. Background Art

[0002] Image sensors have become ubiquitous and are now widely used in digital cameras, cell phones, video cameras, as well as in medical, automotive and other applications. As image sensors are integrated into a wider range of electronic devices, it is desirable to enhance their functionality, performance metrics, etc. in as many ways as possible (e.g., resolution, power consumption, dynamic range, etc.) through device architecture design and image acquisition processing.

[0003] A typical image sensor operates in response to image light from an external scene being incident on the image sensor. The image sensor includes an array of pixels having photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and generate image charges after absorbing the image light. The image charge generated by the pixel light can be measured as an analog output image signal on a column bit line that varies according to the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out from the column bit line as an analog image signal and converted into a digital value to provide information representing the external scene. Summary of the invention

[0004] One aspect of the present disclosure relates to an event vision sensor, comprising: an array of event vision pixels arranged in rows and columns; and an event signal processor configured to identify defective event vision pixels corresponding to the array based at least in part on a noise event initiation rate, wherein the noise event initiation rate is based at least in part on an arrival interval time corresponding to the defective event vision pixels, and wherein the arrival interval time is equal to an amount of time that elapses between a first time that the defective event vision pixel is reset and a second time that the defective event vision pixel detects a noise event.

[0005] Another aspect of the present disclosure relates to an event vision sensor comprising: an array of event vision pixels arranged in rows and columns, wherein each of the event vision pixels of the array comprises: a photosensor configured to generate a photocurrent in response to incident light; a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltages received from the photocurrent / voltage converter; at least one event generating comparator coupled to the difference detection circuit and configured to compare the difference detected from the difference detection current to the photocurrent; The invention relates to a method for detecting an event indicated in the incident light by using a photocurrent / voltage converter, a difference detection circuit, and a memory component that can be programmed to: disable a subset of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator so that the event vision pixel cannot detect an event, wherein the subset does not include each of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator; or mask the output of the event vision pixel; and an event signal processor that is configured to identify the defective event vision pixel corresponding to the array based at least in part on a noise event initiation rate of the defective event vision pixel.

[0006] Yet another aspect of the present disclosure relates to a method comprising: identifying an event vision pixel of an event vision sensor as defective based at least in part on a noise event initiation rate of the event vision pixel, wherein the noise event initiation rate is based at least in part on an arrival interval associated with the event vision pixel, and wherein the arrival interval time represents an amount of time that elapses between a reset of the event vision image and a subsequent detection of a noise event; and preventing the event vision pixel from outputting event data in response to identifying the event vision pixel as defective. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Non-limiting and non-exhaustive embodiments of the present invention are described below with reference to the following figures, wherein the same or similar reference numerals are used to refer to all the same or similar components unless otherwise specified.

[0008] Figure 1 is a partial schematic block diagram of an event vision sensor and an illumination source configured in accordance with various embodiments of the present invention.

[0009] Figure 2 is a partial schematic block diagram of an event signal processor configured according to various embodiments of the present invention.

[0010] Figure 3 is a partially schematic block diagram illustrating an event vision pixel configured in accordance with various embodiments of the present invention.

[0011] Figure 4is a partial schematic block diagram illustrating another event vision pixel configured in accordance with various embodiments of the present invention.

[0012] Figure 5 is a partial schematic block diagram illustrating yet another event vision pixel configured in accordance with various embodiments of the present invention.

[0013] Figure 6 is a partial schematic block diagram illustrating yet another event vision pixel configured in accordance with various embodiments of the present invention.

[0014] Figure 7 is a flow chart illustrating a method of identifying and disabling / masking defective event vision pixels of an event vision sensor according to various embodiments of the present invention.

[0015] Figure 8 is a flow chart illustrating a method of determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0016] Fig. 9 is a flow chart illustrating another method of determining a noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0017] Those skilled in the art will appreciate that the elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to aid in understanding various aspects of the present invention. In addition, common but well-understood elements or methods that are useful or necessary in commercially feasible embodiments are generally not depicted in the figures or described in detail below so as not to unnecessarily obscure the description of various aspects of the present invention. DETAILED DESCRIPTION

[0018] The present disclosure relates to event vision sensors. For example, several embodiments of the present invention relate to event vision sensors that employ defective pixel suppression, such as intra-pixel defective pixel suppression. In the following description, specific details are set forth to provide a thorough understanding of aspects of the present invention. However, one skilled in the relevant art will recognize that the systems, devices, and techniques described herein may be practiced without one or more of the specific details set forth herein or using other methods, components, materials, etc.

[0019] Reference to an "example" or "embodiment" in this specification means that a particular feature, structure, or characteristic described in conjunction with the example or embodiment is included in at least one example or embodiment of the invention. Thus, the phrases "for example," "as an example," or "embodiment" used herein do not necessarily all refer to the same example or embodiment and are not necessarily limited to the particular example or embodiment being discussed. Furthermore, the features, structures, or characteristics of the invention described herein may be combined in any suitable manner to provide additional examples or embodiments of the invention.

[0020] For ease of description, spatially relative terms (e.g., "below," "below," "above," "below," "above," "up," "top," "bottom," "left," "right," "center," "middle," etc.) may be used herein to describe the relationship of one element or one feature relative to one or more other elements or features, as illustrated in the figures. It should be understood that the spatially relative terms are intended to cover different orientations of the device or system in use or operation in addition to the orientation depicted in the figures. For example, if the device or system illustrated in the figures is rotated, turned, or flipped about a horizontal axis, then the element or feature described as being "below" or "below" or "below" one or more other elements or features may be oriented to be "above" one or more other elements or features. Therefore, the exemplary terms "below" and "below" are non-limiting and may cover both above and below orientations. The device or system may be additionally or alternatively oriented in a manner other than that illustrated in the figures (e.g., rotated 90 degrees about a vertical axis or in other orientations) and the spatially relative descriptors used herein may be interpreted accordingly. In addition, it will also be understood that when an element is referred to as being between two other elements, it can be the only element between the two other elements, or one or more intervening elements may also be present.

[0021] In this specification, several technical terms are used. These terms should be taken with their ordinary meaning in the art, unless explicitly defined herein or otherwise clearly implied by the context of their use. It should be noted that component names and symbols are used interchangeably in this document (e.g., Si and silicon); however, both have the same meaning.

[0022] A. Overview

[0023] Active pixel sensors (e.g., CMOS imaging systems) typically employ an array of active pixels with a globally defined integration time. Thus, the active pixels in an active pixel sensor typically have the same integration time, and each pixel in the array is typically converted into a digital signal regardless of its content (e.g., whether the external scene captured by the pixel has changed since the last time the pixel was read out). In other words, the image data generated by the active pixels in, for example, a CMOS imager is read out in frames of known size, regardless of whether there are events in the external scene.

[0024] In contrast, when a pixel captures an external scene change (e.g., an event), an event vision sensor (e.g., an event-driven sensor or a dynamic vision sensor) reads out the pixel and / or converts the corresponding pixel signal into a digital signal. In other words, the pixels of the event vision sensor that do not detect an external scene change are not read out and / or the pixel signals corresponding to such pixels are not converted into digital signals. Therefore, each pixel of the event vision sensor can be independent of other pixels of the event vision sensor, and only the pixel that detects the external scene change needs to be read out and / or its corresponding pixel signal needs to be converted into a digital signal or recorded (thereby saving power). Thus, the event vision sensor does not need to record the entire conventional image and therefore does not need to capture and record all the highly redundant information of a normal image frame by frame. Therefore, the event vision sensor can be used to detect movement or motion in an external scene (e.g., rather than for capturing / reading out all frames of an image or video), while being able to (i) use a low data rate and (ii) achieve ultra-high frame rate or speed capabilities.

[0025] When an event vision pixel array is used to monitor an external scene in an event vision sensor, the event vision sensor rarely detects and records isolated events corresponding to legitimate activities in the external scene (e.g., events corresponding to a single isolated event vision pixel in the array). This isolated event often corresponds to noise and / or is caused by a defective event vision pixel in the array. Therefore, many event vision sensors use various techniques (e.g., encoding or compression techniques) to filter out noise event signals read out from event vision pixels. As a specific example, when the first event vision pixel of the array detects an event, some event vision sensors analyze a group of event vision pixels including the first event vision pixel. If the event vision sensor determines that a threshold number of event vision pixels in the group detects an event, then the event detected by the first event vision pixel is classified as corresponding to a legitimate activity in the external scene monitored by the event vision sensor. On the other hand, if the event vision sensor determines that less than a threshold number of event vision pixels in the group detect an event, then the event detected by the first event vision pixel is classified as a noise event and / or discarded. This technique is called coincidence detection.

[0026] Coincidence detection eliminates small statistics. In other words, coincidence detection cannot analyze or provide any insight into long-term trends (e.g., coincidence detection cannot provide any insight into whether a noise event is relatively rare or common for a particular event vision pixel to detect a noise event). Therefore, recidivists (e.g., defective pixels that often detect noise events) are not processed and continue to consume excess bandwidth of the event vision sensor to read out noise events. In addition, the small statistics approach of coincidence detection increases the likelihood that the event vision sensor will classify a legitimate event as a noise event or vice versa.

[0027] Furthermore, coincidence detection and many other conventional methods operate on signals read out from event vision pixels of the array. In other words, many conventional methods identify defective event vision pixels and / or filter out corresponding signals only after noise events are read out from the event vision pixels during normal operation of the event vision sensor. Therefore, event vision sensors employing these conventional methods typically utilize excess bandwidth of the event vision sensor during normal operation to first read out signals of defective event vision pixels (consuming excess bandwidth) before (a) the event vision pixels are identified as defective and / or (b) their corresponding signals are filtered out and / or discarded.

[0028] In view of the problems discussed above, the present invention provides various solutions for identifying defective event vision pixels and disabling or masking their outputs using in-pixel circuitry. For example, several embodiments of the present invention identify defective event vision pixels of an event vision sensor based on a noise event initiation rate of each event vision pixel. More specifically, during one or more points in the life of the event vision sensor (e.g., during wafer testing, during power-up or initialization, after a predetermined amount of time, outside of normal operation, when the event vision sensor determines that the event (legitimate and / or noise) initiation rate of one or more event vision pixels exceeds a threshold, etc.), the event vision pixels may be exposed to DC illumination while the event vision sensor (e.g., an event signal processor of the event vision sensor) determines the noise event initiation rate of each event vision pixel. In some embodiments, the noise event initiation rate of the event vision pixel may be based on a mean inter-arrival time (e.g., the average time between the time when the event vision pixel is reset and the time when the event vision pixel subsequently detects a noise event). In these and other embodiments, the noise event initiation rate of the event vision pixel may be based on the number of noise events detected within a period of time. Once the initiation rate for an event visual pixel is obtained, the initiation rate may be compared to one or more thresholds to identify whether the event visual pixel is defective (eg, unacceptably likely to record or unacceptably prone to recording noise events).

[0029] When an event vision pixel is identified as defective, the event vision sensor may program a memory component (e.g., a latch) contained within the event vision pixel to disable the event vision pixel (e.g., prevent the event vision pixel from detecting an event) or mask its output (e.g., prevent an event detected by the event vision pixel from being output from the event vision pixel). Additionally or alternatively, the address of the event vision pixel may be added to a lookup table maintained by the event vision sensor so that the event vision pixel may be disabled or masked whenever the event vision sensor is powered on or initialized.

[0030] In this manner, the present invention provides various solutions for identifying defective event vision pixels of an event vision sensor without the need to (a) read out noise events as well as legitimate events during normal operation and (b) discern whether an event should be classified as legitimate or noise. In addition, the present invention provides an intra-pixel defective pixel suppression solution that can be used to disable event vision pixels identified as defective and / or mask their output. Such defective event vision pixels can then be prevented from consuming excess bandwidth of the event vision sensor to read out noise events detected by the event vision pixels. Therefore, compared to an event vision sensor lacking a defective event vision pixel suppression solution and / or a corresponding intra-pixel circuit system, the present invention is expected to reduce the number of noise events read out to an event signal processor of an event vision sensor, increase the percentage of legitimate events in the events read out to the event signal processor, and / or utilize less power.

[0031] B. Selected embodiments of event vision sensors with defective pixel suppression and associated systems, devices, and methods

[0032] Figure 1 1 is a partially schematic block diagram of an event vision sensor 100 (also referred to herein as an "imager") configured in accordance with various embodiments of the present invention. As shown, the event vision sensor 100 includes an array 102 of event vision pixel circuits 110 ("event vision pixels 110"), row control circuitry 104, column control circuitry 106, an event signal processor 108, a transmitter 116, a lookup table 142 ("LUT 142"), control logic 144, and peripheral circuitry 146. Also shown is an illumination source 120 configured to project light 150 onto the event vision pixels 110 of the array 102. In some embodiments, the illumination source 120 may be a component of the event vision sensor 100 or an imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be external to the event vision sensor 100 and the imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be omitted.

[0033] Event vision pixels 110 are arranged in rows and columns in array 102 ( Figure 110 and 110, respectively. The event vision pixels 110 are configured to detect changes (e.g., events) in light received from an external scene. To this end, each event vision pixel 110 may include: (i) a photosensor (such as a photodiode) configured to photogenerate a charge or photocurrent in response to incident light received from an external scene; (ii) a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent generated by the photosensor into a voltage; and (iii) a filter amplifier coupled to the photocurrent / voltage converter to generate a filtered and amplified signal in response to the voltage received from the photocurrent / voltage converter. The event vision pixel 110 may further include a threshold comparison circuit or stage that generates and receives a handshake signal in response to an event detected asynchronously in incident light received from an external scene. Alternatively, the threshold comparison circuit may be included in circuitry peripheral to or external to the event vision pixel 110 of the array 102 (e.g., in the event signal processor 108), such as within a column readout circuitry. The photosensor, photocurrent / voltage converter, filter amplifier, and / or threshold comparison stage of the event vision pixel 110 are collectively referred to herein as event vision sensor (EVS) core pixel circuitry 130 .

[0034] like Figure 1 , the event vision pixels 110 configured in accordance with various embodiments of the present invention may each further include a programmable storage component 136 (illustrated and hereinafter referred to as a "latch 136") coupled to the EVS core pixel circuitry 130. The latch 136 of the event vision pixel 110 may be programmed using a program signal (prog), a row signal (row) corresponding to the event vision pixel 110, and a column signal (col) corresponding to the event vision pixel 110. More specifically, when an event vision pixel 110 of the array 102 is identified as a defective pixel (as discussed in more detail below), an address (row address and column address) corresponding to the location of the event vision pixel 110 in the array 102 may be stored in the LUT 142. The control logic 144 may then use the address stored in the LUT 142 to program (e.g., after the event vision sensor 100 is powered on or initialized) the latch 136 corresponding to the event vision pixel 110 to disable the event vision pixel 110 and / or mask its output so that the event detected by the event vision pixel 110 is not read out from the array 102. Figures 3 to 6 Various embodiments of event vision pixel 110 incorporating latches similar to latch 136 are discussed in detail.

[0035] Figure 1The row control circuitry 104 and the column control circuitry 106 are used to respectively control the rows and columns of event vision pixels 110 in the array 102. For example, the row control circuitry 104 and / or the column control circuitry 106 may be configured to reset particular event vision pixels 110 (e.g., individual ones or rows of event vision pixels 110) of the array 102 and / or read out the event vision pixels 110 (e.g., individual ones or rows of event vision pixels 110) from the array 102 (e.g., along corresponding column bit lines connected to the event vision pixels 110).

[0036] The event data read out from the event vision pixels 110 of the array 102 may be passed to the event signal processor 108 of the event vision sensor 100 for processing. The event data processed by the event signal processor 108 may be provided to a transmitter 116 to transmit the event data out of the event vision sensor 100, such as to a receiver (not shown) of a corresponding imaging system. Additionally or alternatively, all or a subset of the event data may be stored in a memory (e.g., before or after being provided to the event signal processor 108 and / or the transmitter 116).

[0037] The event signal processor 108 of the event vision sensor 100 may additionally be used to identify defective event vision pixels 110 of the array 102. For example, the illumination source 120 may be used to project light 150 (e.g., DC illumination) onto the event vision pixels 110 of the array 102. In some embodiments, the light 150 may have a constant intensity or brightness (or an intensity or brightness that remains substantially constant or changes slowly over time). Thus, when the illumination source 120 continuously projects the light 150 onto the event vision pixels 110 of the array 102, the event vision pixels 110 should not (unless defective or subject to noise) detect an event or detect an event at a frequency above a threshold. The event signal processor 108 may therefore utilize the illumination source 120 to identify defective or noisy event vision pixels 110 of the array 102 and thereafter disable such pixels 110 and / or mask their outputs. hereinafter referred to as Figures 7 to 9 A method of identifying defective event visual pixels 110 of array 102 is discussed in detail.

[0038] 1. Selected embodiments of event signal handlers

[0039] Figure 2 FIG. 2 is a schematic block diagram of a portion of an event signal processor 208 configured in accordance with various embodiments of the present invention. It should be appreciated that the event signal processor 208 illustrated may be Figure 11 or another example of an event vision pixel of the present invention. As shown, the event signal processor 208 includes a defective pixel removal block 270, which includes various stages or circuits that can be used to identify and disable / mask defective event vision pixels of the corresponding event vision sensor. For example, the defective pixel removal block 270 includes a comparator voltage regulator block 271, an event occurrence recording block 272, an event occurrence characterization block 275, and a defective pixel determination and recording block 278.

[0040] The comparator voltage regulator block 271 of the event signal processor 208 may be configured to regulate the voltage generated by the event visual pixel (e.g. Figure 1 The event of the event vision pixel 110) generates a comparator voltage threshold used by the comparator. For example, the comparator voltage regulator block 271 can be coupled to the peripheral circuit system (e.g. Figure 1 The peripheral circuitry 146 of the visual pixel sensor 100. Using the peripheral circuitry, the comparator voltage regulator block 271 can regulate the voltage generated by the event visual pixel comparator for detecting UP events (e.g., greater than the comparator threshold voltage V U The comparator threshold voltage V U Additionally or alternatively, the comparator voltage regulator block 271 may use peripheral circuitry to regulate the voltage generated by the event visual pixel used by the comparator to detect a DOWN event (eg, greater than a comparator threshold voltage V D The comparator threshold voltage V D .

[0041] As discussed in more detail below, the comparator voltage regulator block 271 of the event signal processor 208 may be used to turn off or disable an event generating comparator of an event vision pixel while another event generating comparator is used to determine whether the event vision pixel is defective. For example, the comparator voltage regulator block 271 may be used to increase the comparator voltage threshold V used by the down event generating comparator of the event vision pixel to D Set to -∞ (or some other large negative voltage value), and the event signal processor 208 uses the up event of the event visual pixel to generate a comparator to determine whether the event visual pixel is defective. D Setting to -∞ (or some other large negative voltage value) may prevent the down event generating comparator from registering an event. Additionally or alternatively, the comparator voltage regulator block 271 may be used to adjust the comparator voltage threshold V used by the up event generating comparator of the event visual pixel to -∞. U Set one or more thresholds (or make the comparator voltage threshold V USweeping across a set of one or more thresholds) to enable, for example, the event occurrence characterization block 275 of the event signal processor 208 to model the probability of detecting a noise event within one or more observation windows.

[0042] The event occurrence record block 272 of the event signal processor 208 is configured to record noise events detected by event visual pixels. For example, the event occurrence record block 272 may record (i) a timestamp corresponding to the time when the noise event was detected and / or (ii) the location (e.g., row address and column address) of the event visual pixel where the noise event was detected. Additionally or alternatively, the event occurrence record block 272 may record the polarity or change of the noise event (e.g., indicating whether the noise event is an UP noise event or a DOWN noise event).

[0043] The event occurrence characterization block 275 may perform various calculations and modeling of noise events detected by the event visual pixel. For example, the event occurrence characterization block 275 may include an event probability distribution modeling block 276, which is configured to model one or more probabilities that the event visual pixel detects a noise event (e.g., at a given comparator voltage threshold) within one or more given observation windows (e.g., having different lengths) based at least in part on the noise events recorded by the event occurrence recording block 272. Additionally or alternatively, the event occurrence characterization block 275 may include a statistics calculation block 277 configured to calculate various statistics of the event visual pixel. Examples of statistics that may be calculated by the statistics calculation block 277 include the noise event initiation rate of the event visual pixel, the variance of the probability density function determined by the event probability distribution modeling block 276, higher-order moments, quantiles, variability of any moment relative to a reference event visual pixel, and / or statistics associated with a group of event visual pixels (e.g., a group of event visual pixels including the event visual pixel and / or one or more event visual pixels falling within a certain distance from the event visual pixel).

[0044] The defective pixel determination and recording block 278 of the event signal processor 208 is configured to identify defective event visual pixels based on (a) statistics output from the statistics calculation block 277 and (b) specified criteria or thresholds. For example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the calculated noise event initiation rate is greater than a threshold. As another example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the variance or other statistics calculated by the statistics calculation block 277 is greater than a threshold. The threshold used by the defective pixel determination and recording block 278 may be predefined (e.g., preset, predetermined). Additionally or alternatively, the threshold used by the defective pixel determination and recording block 278 may depend on statistics related to one or more other (e.g., adjacent) event visual pixels. For example, the defective pixel determination and recording block 278 may determine that the event visual pixel is defective when the noise event initiation rate is greater than the average event occurrence rate corresponding to one or more adjacent event visual pixels (e.g., by a specified amount).

[0045] When the defective pixel determination and recording block 278 determines that the event visual pixel is defective, the defective pixel determination and recording block 278 may store the address (e.g., row address and column address) corresponding to the event visual pixel in a lookup table (e.g., in Figure 1 Additionally or alternatively, the defective pixel determination and recording block 278 may be coupled to control logic (e.g., Figure 1 In these embodiments, the defective pixel determination and recording block 278 may communicate with and / or use the control logic to disable the defective event vision pixel or mask its output to prevent the defective event vision pixel from detecting an event or outputting an event from the event vision pixel, respectively. For example, the defective pixel determination and recording block 278 may instruct the control logic to register a memory component (e.g., a latch, such as a latch) of the defective event vision pixel via the row control circuitry and / or column control circuitry of the corresponding event vision sensor. Figure 1 The latch 136 of the programming is performed. Figures 3 to 6 Circuitry for disabling defect event visual pixels and / or masking their outputs is discussed in greater detail.

[0046] As shown, the event signal processor 208 may additionally include an auxiliary function block 279, which includes circuits or blocks for performing various auxiliary processing functions. For example, the auxiliary function block 279 may include a segment classifier block and / or a shape classifier block that can be used to classify the segmentation and shape of the event data read out from the array, respectively. As another example, the auxiliary function block 279 may include an optical flow estimation block, which can be used to identify pixel-by-pixel, shape-by-shape, or segment-by-segment motion over time and / or between consecutive readouts (e.g., using correlation-based, block-matching-based, feature-tracking-based, energy-based, and / or gradient-based optical flow estimation). In these and other embodiments, the auxiliary function block 279 may include a compression block to compress the event data read out from the array. Additionally or alternatively, the event signal processor 208 may include a set of one or more buffers (e.g., a set of row buffers) that are operably coupled to one or more blocks or circuits of the event signal processor 208 to perform various processing functions.

[0047] 2. Selected Embodiments of Event Vision Pixels

[0048] Figure 3 FIG. 1 is a partially schematic block diagram illustrating an event vision pixel 310 configured according to various embodiments of the present invention. It should be understood that the event vision pixel 310 illustrated may be a pixel included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 310 includes an image sensor configured to respond to incident light 350 (e.g., from an external scene and / or from an illumination source such as Figure 1 The photoelectric sensor 331 is coupled to a logarithmic amplifier 332, which is configured to convert the photocurrent generated by the photosensor 331 into a voltage. In various examples, the logarithmic amplifier 332 is configured to generate a voltage by transducing the instantaneous photocurrent received from the photosensor 331. A difference detection amplifier 333 is coupled to the logarithmic amplifier 332 to generate a filtered and amplified signal in response to a difference detected in the voltages received from the logarithmic amplifier 332. In one example, the difference detection amplifier 333 is configured to compare the instantaneous logarithmic intensity of the voltage output of the logarithmic amplifier 332 with a reference level based on a reset condition or a latest event.

[0049] The event generating comparator 334 is coupled to the difference detection amplifier 333 to compare the filtered and amplified signal received from the difference detection amplifier 333 with a threshold to asynchronously detect an event indicated in the incident light 350. In one example, the event generating comparator 334 is configured to discern whether the signal difference is large enough to trigger an event. In some embodiments, the event generating comparator 334 includes a first comparator (not shown) configured to detect whether the signal difference corresponds to an 'UP' event (e.g., a change in the intensity of light incident on the photosensor 331 from darker to brighter and greater than a comparator voltage threshold V U The event generating comparator 334 may further include a second comparator (not shown) configured to detect whether the signal difference corresponds to a 'DOWN' event (e.g., a change in the intensity of light incident on the photosensor 331 from brighter to darker and greater than a comparator voltage threshold V D ).

[0050] Figure 3 The event vision pixel 310 may further include a scanner and readout logic 335 ("readout logic 335"). In some embodiments, the readout logic 335 may include a latch that is (a) coupled to one or more outputs of the event generating comparator 334 and (b) configured to trigger when the event vision pixel 310 detects an event. Additionally or alternatively, the readout logic 335 may be coupled to a readout control circuit system (e.g., a control circuit) of a corresponding event vision sensor. Figure 1 The readout logic 335 may be configured to read out the event vision pixel 310 whenever the event vision pixel 310 detects an event. In this manner, the readout logic 335 may be configured to ensure that the event vision pixel 310 is read out only when the event vision pixel 310 detects an event. In other words, the readout logic 335 may be configured to ensure that the event vision pixel 310 is not read out when the event vision pixel 310 does not detect an event. When the event vision pixel 310 detects an event, the event may be read out from the event vision pixel 310 onto a column bit line (not shown) corresponding to the event vision pixel 310.

[0051] Therefore, it should be understood that including similar Figure 3 The event vision pixel 310 of the event vision pixel of the event vision sensor (eg Figure 1 The event vision sensor 100 does not need to record the entire conventional image and therefore does not need to capture and record all the highly redundant information of a normal image frame by frame. Specifically, in various embodiments, the event vision sensor can read and record unique events. For example, the event vision sensor can read and / or record the location where the event was detected (e.g. Figure 1The event vision sensor may be used to detect movement or motion in an external scene (e.g., rather than for capturing / reading out a full frame of an image or video), thereby enabling the use of low data rates and achieving ultra-high frame rate or speed capabilities in the event vision sensor of the present invention.

[0052] Continue to refer Figure 3 , the event vision pixel 310 further includes a programmable memory element 336 (illustrated and hereinafter referred to as "latch 336") and a logic gate 337. The latch 336 includes three inputs and one output. The logic gate 337 includes: (a) a first input coupled to the output of the latch 336; (b) one or more second inputs coupled to one or more outputs of the readout logic 335; and (c) one or more outputs coupled to the column bit lines and / or readout control circuitry (e.g., Figure 1 Logic gate 337 is in the row control circuit system 104 and / or column control circuit system 106. Figure 3 An AND gate is shown in FIG. 1 as an example, but may be another logic gate in other embodiments of the present invention.

[0053] Latch 336 is configured to receive a program signal, a row signal, and a column signal at its input. As discussed in more detail below, these signals can be used to program latch 336 and selectively assert a mask signal output from latch 336 and fed into logic gate 337. (also referred to herein as a "deactivation signal ”). The row and column signals input into latch 336 may correspond to the position of event visual pixel 310 in an array of such pixels.

[0054] The mask signal output from latch 336 The state of depends on the input of latch 336. More specifically, in the illustrated example, the mask signal It may be defaulted to not asserted (e.g., it may be in the first state or be "1"). Therefore, under normal operation, the output of the logic gate 337 may follow the output of the readout logic 335. For example, when an event occurs in an external scene monitored by an event vision sensor including an event vision pixel 310, the event may be indicated as a rapid or sudden change in intensity or brightness in the incident light 350 received by the photosensor 331 of the event vision pixel 310. In other words, if the external scene is static and no event occurs, the brightness of the incident light 350 remains substantially unchanged. Thus, the photocurrent generated by the photosensor 331 remains substantially constant and the event vision pixel 310 (unless defective) does not record the event or does not record the event frequently. However, if an event occurs in the external scene (e.g., movement, lighting change, reflection, excitation, etc.), the event is indicated by an asynchronous rapid or sudden change in the brightness of the incident light 350. The brightness change may be from darker to brighter or from brighter to darker. Thus, there is an asynchronous change or Δ in the photocurrent generated by the photosensor 331. The change in photocurrent, or Δ, is converted to a voltage by log amplifier 332, filtered and amplified by difference detection amplifier 333, and then detected with event generation comparator 334. The event may be latched in readout logic 335 until it is read out to the corresponding column bit line via logic gate 337. Additionally or alternatively, readout logic 335 may send a request (e.g., via logic gate 337) to the readout control circuitry to read out the event visual pixel 310. The event may then be read out (via logic gate 337) to the column bit line corresponding to the event visual pixel 310.

[0055] On the other hand, if the event visual pixel 310 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 336 may be used to program the latch 336 and assert the mask signal When the mask signal is asserted When (for example, when shielding the signal When the mask signal 100 is asserted, one or more outputs of the readout logic 335 may be masked via the logic gate 337. For example, when the mask signal 100 is asserted, When , the logic gate 337 (i) can prevent the read request output from the readout logic 335 from reaching the readout control circuit system corresponding to the event vision pixel 310 and / or (ii) can prevent the event detected by the event vision pixel 310 from being output to the column bit line corresponding to the event vision pixel 310.

[0056] When the event visual pixel 310 is identified as a defective pixel, Figure 3Shielding the output of the event vision pixel 310 in the manner shown in FIG. 1 provides several advantages. For example, events detected by the event vision pixel 310 that are most likely due to noise and / or defects in the event vision pixel 310 are not output from the event vision pixel 310 and therefore do not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 310. In addition, only the output of the readout logic 335 is shielded. In other words, the various analog stages of the event vision pixel 310 (such as the logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334, and the readout logic 335) remain powered on. Therefore, the DC bias of each of these analog stages remains enabled, and the event vision pixel 310 (when identified as defective) continues to consume the same relative amount of current as other event vision pixels of the array incorporating the event vision pixel 310 that are not identified as defective. Therefore, even though the output of one or more defective event vision pixels across the array may be shielded, the current consumed by the event vision pixels across the array remains approximately uniform. Therefore, a substantial uniformity of IR drop across the array may be maintained or observed.

[0057] Figure 4 FIG. 4 is a partially schematic block diagram illustrating another event vision pixel 410 configured according to various embodiments of the present invention. It should be understood that the event vision pixel 410 illustrated may be a pixel included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 410 includes an image sensor configured to respond to incident light 450 (e.g., from an external scene and / or from an illumination source such as a Figure 1 The photoelectric sensor 431 receives the light from the illumination source 120 and generates a photoelectric charge or a photocurrent. The event vision pixel 410 further includes a logarithmic amplifier 432, a difference detection amplifier 433, an event generation comparator 434, and a readout logic 435. The logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 can be roughly similar to Figure 3 The logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434 and the readout logic 435 is omitted here.

[0058] Event vision pixel 410 additionally includes a programmable memory element 436 (illustrated and hereinafter referred to as "latch 436"). Latch 436 is substantially similar to Figure 3 Therefore, for the sake of brevity, the detailed discussion of latch 436 is basically omitted here. However, unlike Figure 3 The latch 336, Figure 4The latch 436 is coupled to the gate of the transistor 438, which acts as a switch to disable the latch 436 based on the deactivation signal output from the latch 436. The state of the event vision pixel 410 logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 selectively couples the power supply voltage V DD As discussed above, the deactivation signal The default is not asserted (e.g., it may be in the first state or "1"). Therefore, under normal operation, the logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 may be coupled to the power supply voltage V via the transistor 438. DD And the power supply voltage V DD On the other hand, when the event visual pixel 410 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 436 may be used to program the latch 436 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the transistor 438 is deactivated, the logarithmic amplifier 432, the difference detection amplifier 433, the event generating comparator 434 and the readout logic 435 of the event vision pixel 410 are connected to the power supply voltage V DD The decoupling prevents the event vision pixel 410 from detecting an event.

[0059] When the event visual pixel 410 is identified as a defective pixel, Figure 4 Preventing the event vision pixel 410 from detecting an event in the manner shown in FIG. 4 provides several advantages. For example, (i) an event that is attributable to noise and / or defects in the event vision pixel 410 and (ii) that would otherwise be detected by the event vision pixel 410 is prevented from being generated or output from the event vision pixel 410 and thus does not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 410. In addition, when the event vision pixel 410 is identified as defective, the analog stage of the event vision pixel 410 is prevented from consuming current. Thus, the power supply voltage V DD 410 while saving power that would otherwise be consumed by the analog stages of event vision pixel 410.

[0060] That is, shutting off the DC bias of each of the analog stages of the event vision pixel 410 in this manner may result in non-uniform current consumption across the event vision pixels of the array that includes the event vision pixel 410. In other words, when a number of event vision pixels of the array are identified as defective, non-uniformity in IR voltage drop across the array may be observed. Having recognized this problem, only a subset of the analog stages of the event vision pixel (e.g., any combination of logarithmic amplifiers, difference detection amplifiers, event generation comparators, and / or readout logic representing less than all analog stages) may be disabled in some embodiments of the present invention when the event vision pixel is identified as defective (e.g., so that uniformity in IR voltage drop across the array may be substantially maintained or observed).

[0061] As a specific example, see Figure 5 , which is a partially schematic block diagram illustrating yet another event vision pixel 510 configured according to various embodiments of the present invention. It should be understood that the illustrated event vision pixel 510 may be included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 510 includes an image sensor configured to respond to incident light 550 (e.g., from an external scene and / or from an illumination source such as Figure 1 The event vision pixel 510 further includes a logarithmic amplifier 532, a differential detection amplifier 533, an event generation comparator 534, and a readout logic 535. The logarithmic amplifier 532, the differential detection amplifier 533, the event generation comparator 534, and the readout logic 535 can be roughly similar to Figure 3 The logarithmic amplifier 332, the difference detection amplifier 333, the event generation comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 532, the difference detection amplifier 533, the event generation comparator 534 and the readout logic 535 is omitted here.

[0062] Event vision pixel 510 additionally includes a programmable memory element 536 (illustrated and hereinafter referred to as "latch 536"). Latch 536 is substantially similar to Figure 4 Therefore, for the sake of brevity, the detailed discussion of latch 536 is basically omitted here. However, unlike Figure 4 The latch 436, Figure 5 The latch 536 is coupled to the gate of the transistor 538, which acts as a switch to disable the latch 536 based on the deactivation signal output from the latch 536. The state selectively couples only the logarithmic amplifier 532 of the event vision pixel 510 (rather than all analog stages) to the power supply voltage V DD Therefore, in the deactivation signal In normal operation, where the default is not asserted (eg, in the first state or “1”), the logarithmic amplifier 532 may be coupled to a power supply voltage V DD And the power supply voltage V DD On the other hand, when the event visual pixel 510 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 536 may be used to program the latch 536 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the transistor 538 is deactivated to connect only the logarithmic amplifier 532 of the event vision pixel 510 to the power supply voltage V DD Decoupling the logarithmic amplifier 532 from the power supply voltage V DD The decoupling may interrupt the photocurrent path from the photosensor 531 to the differential sense amplifier 533 to prevent the event vision pixel 510 from detecting the event.

[0063] When the event visual pixel 510 is identified as a defective pixel, Figure 5 Preventing the event vision pixel 510 from detecting an event in the manner shown in provides several advantages. For example, events that are (i) attributable to noise and / or defects in the event vision pixel 510 and (ii) that would otherwise be detected by the event vision pixel 510 are prevented from being generated or output from the event vision pixel 510 and thus do not consume the available bandwidth of the event vision sensor incorporated into the event vision pixel 510. In addition, when the event vision pixel 510 is identified as defective, the logarithmic amplifier 532 of the event vision pixel 510 is prevented from consuming current. Thus, power that would otherwise be consumed by the logarithmic amplifier 532 of the event vision pixel 510 is saved. Furthermore, when the event vision pixel 510 is identified as defective, only a subset of the analog stages of the event vision pixel 510 (here, only the logarithmic amplifier 532) is disconnected from the power supply. In other words, at least some of the other analog stages of the event vision pixel 510 (e.g., here, the difference detection amplifier 533, the event generation comparator 534, and the readout logic 535) remain powered on. Thus, the DC bias of each of these other stages remains enabled and the event vision pixel 510 continues to consume an amount of current that is relatively similar to the amount of current consumed when the event vision pixel 510 is not identified as defective (less than the amount of current consumed due to the logarithmic amplifier 532 and the power supply voltage V DD Therefore, even if a defect event occurs across the array, the number of logarithmic amplifiers of the visual pixels and the power supply voltage V DD By turning off, the current consumed by the event visual pixels across the array can remain substantially uniform. Substantially uniform IR voltage drop across the array can then be substantially maintained or observed.

[0064] Other methods of disabling defective event visual pixels and / or masking their outputs are certainly possible and fall within the scope of the present invention. For example, a programmable memory component may be provided so that the event generating comparator may be based on a disable signal output from the programmable memory component. Thus, continuing with this example, when an event visual pixel is identified as defective, the memory component may be programmed to assert a disable signal The comparator is turned off and / or decoupled from the power supply voltage, thereby making the event vision pixel unable to detect the event.

[0065] As another example, a programmable memory component may be coupled to a differential detection amplifier of an event visual pixel to detect a disable signal output from the programmable memory component. To selectively enable the differential detection amplifier. As a specific example, refer to Figure 6 , which is a partially schematic block diagram illustrating yet another event vision pixel 610 configured according to various embodiments of the present invention. It should be understood that the illustrated event vision pixel 610 may be included in Figure 1 102 or another example of an event vision pixel of the present invention. As shown, the event vision pixel 610 includes an image sensor configured to respond to incident light 650 (e.g., from an external scene and / or from an illumination source such as Figure 1 The photoelectric sensor 631 receives the light from the illumination source 120 and generates a photoelectric charge or a photocurrent. The event vision pixel 610 further includes a logarithmic amplifier 632, a difference detection amplifier 633, an event generation comparator 634, and a readout logic 635. The logarithmic amplifier 632, the event generation comparator 634, and the readout logic 635 can be roughly similar to Figure 3 The logarithmic amplifier 332, the event generating comparator 334 and the readout logic 335 are shown in FIG. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 632, the event generating comparator 634 and the readout logic 635 is omitted here.

[0066] For clarity and understanding, Figure 6 Detailed description of the difference detection amplifier 633 is given in FIG. As shown, the difference detection amplifier 633 includes a first capacitor 622, a second capacitor 623, an amplifier 624, and a reset transistor 625. The amplifier 624 includes an input (e.g., an inverting input) and an output. The first capacitor 622 is coupled between the output of the logarithmic amplifier 632 and the input of the amplifier 624, and the second capacitor 623 is coupled between the input of the amplifier 624 and the output of the amplifier 624. The reset transistor 625 is coupled between the input of the amplifier 624 and the output of the amplifier 624.

[0067] The first capacitor 622, the second capacitor 623, the amplifier 624, and the reset transistor 625 form a filter amplifier that is configured to produce a filtered and amplified signal in response to a voltage output by the logarithmic amplifier 632 of the event vision pixel 610. More specifically, the filter amplifier includes a high pass filter configured to filter out lower frequency components from the voltage received by the logarithmic amplifier 632. Thus, the event vision pixel 610 can ignore slow or gradual changes in the photocurrent generated by the photosensor 631 in response to the incident light 650, and can instead detect rapid and sudden changes that occur in the photocurrent generated by the photosensor 631 in response to the incident light 650. Additional details regarding the difference detection circuit and associated event vision pixels are provided in U.S. Patent Application No. 17 / 875,244, which is incorporated herein by reference in its entirety.

[0068] As shown, the event vision pixel 610 also includes a programmable memory component 636 (illustrated and hereinafter referred to as a "latch 636"). The latch 636 is substantially similar to Figure 3 636. Therefore, for the sake of brevity, detailed discussion of latch 636 is basically omitted here. The output of latch 636 is coupled to the input of inverter 639, and the output of inverter 639 is coupled to the input of logic gate 637. Logic gate 637 further includes (a) an input configured to receive reset signal RST and (b) an output coupled to the gate of reset transistor 625. Logic gate 637 is Figure 6 An OR gate is illustrated as an example, but in other embodiments of the present invention, it may be another logic gate that may or may not include the inverter 639.

[0069] The reset transistor 625 of the event vision pixel 610 is arranged as a reset switch and is configured to selectively couple the input of the amplifier 624 to the output of the amplifier 624 based on the output of the logic gate 637. Under normal operation by default being not asserted (e.g., in a first state or "1"), the output of the logic gate 637 may follow the reset signal RST. Specifically, when the reset signal RST is asserted, the reset transistor 625 may couple the input of the amplifier 624 to the output of the amplifier 624 to automatically zero the amplifier 624. When the reset signal RST is not asserted, the reset transistor 625 may decouple the input of the amplifier 624 from the output of the amplifier 624.

[0070] On the other hand, when the event visual pixel 610 is identified as a defective pixel, the program signal, the row signal, and the column signal input into the latch 636 may be used to program the latch 636 and assert the disable signal When asserted deactivate signal When (for example, when the deactivation signal When the reset transistor 625 transitions to the second state or "0", the reset transistor 625 may remain activated to couple the input of the amplifier 624 to the output of the amplifier 624. Keeping the amplifier 624 auto-zeroed in this manner may disable or prevent the event vision pixel 610 from detecting an event.

[0071] When the event visual pixel 610 is identified as a defective pixel, Figure 6 Preventing the event vision pixel 610 from detecting an event in the manner shown in provides several advantages. For example, events that are (i) attributable to noise and / or defects in the event vision pixel 610 and (ii) that would otherwise be detected by the event vision pixel 610 are prevented from being generated or output from the event vision pixel 610 and thus do not consume the available bandwidth of the event vision sensor incorporating the event vision pixel 610. In addition, each of the analog stages of the event vision pixel 610 continues to consume static power. In other words, the DC bias of each of the analog stages remains enabled and the event vision pixel 610 continues to consume an amount of current that is relatively similar to the amount of current consumed when the event vision pixel 610 is not identified as defective. Therefore, even if several defective event vision pixels across the array are unable to detect an event, the current consumed by the event vision pixels across the array can remain substantially uniform. Then, a substantial uniformity of IR drop across the array can be substantially maintained or observed.

[0072] 3. Related methods

[0073] Figure 7 7 is a flow chart illustrating a method 780 of identifying and disabling / masking defective event vision pixels according to various embodiments of the present invention. The method 780 is illustrated as a set of steps or blocks 781 to 787. All or a subset of one or more of blocks 781 to 787 may be performed by an event vision sensor (e.g., Figure 1 100). For example, all or a subset of one or more of blocks 781-787 may be performed by: (i) an event signal processor, (ii) an event vision pixel (e.g., a programmable memory component of the event vision pixel), (iii) a row control circuit system, (iv) a column control circuit system, (v) control logic, (vi) a lookup table, and / or (vii) a peripheral circuit system. Additionally or alternatively, all or a subset of blocks 781-787 may be performed by an illumination source (e.g., Figure 1 In addition, any one or more of blocks 781 to 787 may be performed according to the above Figures 1 to 6 discussion to be carried out.

[0074] Figure 7The method 780 of begins at block 781, which determines a noise event initiation rate for an event vision pixel. Determining the noise event initiation rate may include determining the noise event initiation rate for the event vision pixel with one or more specified comparator thresholds in at least one event generation comparator for the event vision pixel. In these and other embodiments, determining the noise event initiation rate may include determining the noise event initiation rate using only one event generation comparator (e.g., an upward event generation comparator or a downward event generation comparator) of the event vision pixel. In these embodiments, the other event generation comparators of the event vision pixel may be disabled and / or turned off. For example, assuming that the upward event generation comparator of the event vision pixel is used to determine the noise event initiation rate for the event vision pixel, then the downward event generation comparator of the event vision pixel may be turned off or the comparator threshold used in the downward event generation comparator may be set to -∞ (or some other large negative voltage value). As another example, assuming that a downward event generation comparator of an event vision pixel is used to determine a noise event initiation rate of the event vision pixel, the upward event generation comparator of the event vision pixel may be turned off or the comparator threshold used in the upward event generation comparator may be set to ∞ (or some other large positive voltage value). Alternatively, determining the event initiation rate may include using multiple event generation comparators of the event vision pixel (e.g., both an upward event generation comparator and a downward event generation comparator) to determine the event initiation rate or simultaneously enabling multiple event generation comparators of the event vision pixel to detect noise events.

[0075] Figure 8 890 is a flow chart illustrating a method 890 for determining a noise event initiation rate of an event vision pixel according to various embodiments of the invention. As shown, the method 890 begins at block 891, which exposes the event vision pixel to constant illumination. Exposing the event vision pixel to constant illumination may include exposing the event vision pixel to DC illumination, such as DC illumination projected onto the event vision pixel using an illumination source. In other embodiments, exposing the event vision pixel to constant illumination may include shielding the event vision pixel from ambient illumination (e.g., in an external scene), such as by placing a cap over a lens of an imaging system incorporating the event vision pixel. The event vision pixel may be exposed to constant illumination for each of blocks 892 to 897 discussed below.

[0076] The method 890 then proceeds to blocks 892 and 893, which respectively reset the event visual pixel and then detect the noise event. In some embodiments, resetting the event visual pixel may include recording the time when the event visual pixel is reset in block 892. In these and other embodiments, detecting the noise event may include recording the time when the event visual pixel detects the noise event in block 893.

[0077] In block 894, the method 890 then calculates an inter-arrival time corresponding to the event visual pixel. The inter-arrival time in block 894 may be calculated as the time elapsed between the time the event visual pixel is reset in block 892 and the time the event visual pixel detects a noise event in block 893. In some embodiments, the method 890 may limit the inter-arrival time attributed to the event visual pixel in block 894. For example, if the event visual pixel does not detect a noise event in block 893 within a threshold period of time after the time the event visual pixel is reset in block 892, the method 890 may proceed to block 894 and may set the inter-arrival time of the event visual pixel to be equal to the threshold period of time or another predetermined (e.g., maximum) inter-arrival time value.

[0078] The method 890 optionally proceeds to blocks 895 to 897, which calculate another inter-arrival time for the event visual pixel. More specifically, the method 890 may optionally then again reset the event visual pixel in block 895, then detect a noise event in block 896, and calculate another inter-arrival time corresponding to the event visual pixel in block 897 in a manner similar to block 894. For example, the inter-arrival time in block 897 may be calculated as the time elapsed between the time the event visual pixel is reset in block 895 and the time the event visual pixel detects the noise event in block 896.

[0079] The method 890 may optionally continue to calculate one or more additional inter-arrival times after block 897. The number of inter-arrival times calculated for the event visual pixels may be preset (e.g., predefined, predetermined) for each iteration of the routine 890. Additionally or alternatively, the number of inter-arrival times calculated for the event visual pixels may depend on the amount of time allocated to perform the method 890 and / or the frequency with which the event visual pixels detect noise events.

[0080] In block 898, method 890 then calculates an average inter-arrival time for the event visual pixels. The average inter-arrival time for the event visual pixels may depend on the inter-arrival time calculated for the event visual pixels (e.g., in blocks 894 and / or 897). The average inter-arrival time calculated in block 898 may be determined as Figure 7 The noise event triggering rate of the event visual pixel of box 781 of method 780.

[0081] Fig. 9900 is a flow chart illustrating another method 900 for determining a noise event initiation rate of an event vision pixel according to various embodiments of the invention. The method 900 begins at block 901, which exposes the event vision pixel to constant illumination. Exposing the event vision pixel to constant illumination may include exposing the event vision pixel to DC illumination, such as DC illumination projected onto the event vision pixel using an illumination source. In other embodiments, exposing the event vision pixel to constant illumination may include shielding the event vision pixel from ambient illumination (e.g., in an external scene), such as by placing a cap over a lens of an imaging system incorporating the event vision pixel. The event vision pixel may be exposed to constant illumination for each of blocks 902 to 905 discussed below.

[0082] In blocks 902 and 903, method 900 then resets the event visual pixel and detects a noise event, respectively. In some embodiments, resetting the event visual pixel in block 902 includes recording the time at which the event visual pixel is reset.

[0083] Shortly after detecting the noise event in block 903 (e.g., immediately), the method 900 then resets the event visual pixel again in block 904. Then in block 905, the method 900 then detects the noise event again. At this point, the method 900 may proceed to block 906. Alternatively, the method 900 may optionally repeat blocks 904 and 905. The number of times that blocks 904 and 905 are repeated for each iteration of the method 900 may be preset (e.g., predefined, predetermined). Additionally or alternatively, the number of times blocks 904 and 905 are repeated may depend on the amount of time allocated to perform the method 900 and / or the frequency with which the event visual pixel detects the noise event.

[0084] Similar to Figure 8 Method 890, Fig. 9 The method 900 may limit the amount of time that elapses between the time the event visual pixel is reset and the time the event visual pixel subsequently detects a noise event. For example, if the event visual pixel does not detect a noise event in block 903 within a threshold period of time after the time the event visual pixel is reset in block 902, the method 900 may record the noise event for the event visual pixel and proceed to block 904. If the event visual pixel does not detect a noise event within a threshold amount of time after the time the event visual pixel is reset in block 904, the method 900 may additionally or alternatively perform a similar process in block 905.

[0085] In block 906, method 900 then calculates a noise event initiation rate. In some embodiments, the noise event initiation rate may be equal to the noise event rate or the number of noise events detected by the event visual pixel within the cumulative amount of time required for the event visual pixel to detect these noise events. For example, assuming that the event visual pixel detects noise events in blocks 903 and 905, the noise event rate of the event visual pixel may be calculated as the number of noise events detected (two (2) noise events) divided by the amount of time that elapsed between the time the event visual pixel was reset in block 902 and the time the event visual pixel detected the noise event in block 905. The noise event rate calculated in block 906 of method 900 may be determined as Figure 7 The noise event triggering rate of the event visual pixel of box 781 of method 780.

[0086] Available Fig. 9 Method 900 in addition to or instead of Figure 8 Alternatively, method 890 may be used instead of method 900. Method 890 has the advantage of being particularly robust to pile-ups, whereas method 900 is susceptible to high event rate pile-ups. For example, referring to Fig. 9 In method 900 of box 906 of method 900, the noise event initiation rate calculated in box 906 may depend at least in part on the amount of time that elapses between the time when the event visual pixel detects the noise event in box 903 and the time when the event visual pixel is reset in box 904. For low event rates, this amount of elapsed time is insignificant and can be ignored. But for high event rates, this amount of elapsed time may represent a large percentage of the total amount of time that elapses between the time when the event visual pixel is reset in box 902 and the time when the event visual pixel detects the noise event in box 905. Thus, for high event rates, the amount of time that elapses between the time when the event visual pixel detects the noise event in box 903 and the time when the event visual pixel is reset in box 904 may have a significant effect on the noise event initiation rate calculated in box 906. This effect is called "pile-up." Fig. 9 Compared with the method 900, because Figure 8 The event triggering rate calculated in box 898 of method 890 depends only on the arrival interval time of the event visual pixels (e.g., the time elapsed between the reset of the event visual pixel and the subsequent detection of the noise event), so method 890 is not susceptible to pile-up.

[0087] Reference again Figure 7Once the noise event initiation rate is determined in box 781, method 780 proceeds to box 782, which compares the noise event initiation rate to one or more thresholds. In some embodiments, the one or more thresholds include a predetermined initiation rate threshold. The predetermined initiation rate threshold may be set to a value at or above which the event visual pixel may be defective (e.g., unacceptably likely to record or unacceptably prone to recording noise events). Additionally or alternatively, the one or more thresholds may include or be based at least in part on an average noise event initiation rate of a group of event visual pixels that include or exclude the event visual pixel of interest. For example, in box 781, method 780 may obtain the noise event initiation rate of the event visual pixel of interest and the noise event initiation rate of each of one or more adjacent event visual pixels (e.g., event visual pixels within a threshold distance from the event visual pixel of interest). The method may then calculate an average of the noise event initiation rate and may compare the noise event initiation rate of the event visual pixel of interest with the average noise event initiation rate.

[0088] In block 783, method 780 then determines whether the noise event initiation rate exceeds one or more thresholds from block 782. For example, when one or more thresholds from block 782 include a predetermined initiation rate threshold, method 780 may proceed to block 784 when the noise event initiation rate from 781 exceeds the predetermined initiation rate threshold (block 783: yes). As another example, when one or more thresholds from block 782 include an average noise event initiation rate (e.g., of a set of event visual pixels adjacent to the event visual pixel of interest), method 780 may proceed to block 784 when the noise event initiation rate exceeds the average noise event initiation rate by more than a certain amount (block 783: yes). On the other hand, when method 780 determines that the noise event initiation rate from block 781 does not exceed one or more thresholds from block 782 (block 783: no), method 780 may proceed to block 786.

[0089] In block 784, the method 780 then identifies the event vision pixel as defective. Identifying the event vision pixel as defective may include recording an address (e.g., a row address and / or a column address) corresponding to the event vision pixel in a lookup table. As discussed above, the lookup table may be used to later (e.g., after powering on or initializing the event vision sensor) identify an address corresponding to the defective event vision pixel. The method 780 may additionally or alternatively instruct the control logic to disable the event vision pixel and / or mask its output in block 784.

[0090] In block 785, the method 780 then disables the event vision pixel and / or masks its output. In some embodiments, disabling the event vision pixel may include programming memory components of the event vision pixel (e.g., causing one or more stages of the event vision pixel to be turned off and / or decoupled from a power supply voltage, one or more event generating comparators of the event vision pixel to be turned off, a differential detection amplifier of the event vision pixel to remain auto-zeroed, and / or the event vision pixel to be otherwise prevented from detecting noise events). In these and other embodiments, shielding the event vision pixel may include programming memory components of the event vision pixel (e.g., causing noise events detected by the event vision pixel to not be output from the event vision pixel and / or causing read requests output by readout logic of the event vision pixel to be masked).

[0091] In block 786, method 780 determines whether there are additional event visual pixels to analyze as possibly defective. When method 780 determines that there are additional event visual pixels to analyze (block 786: yes), method 780 may return to block 781. Otherwise, when method 780 determines that there are no additional event visual pixels to analyze (block 786: no), method 780 may proceed to block 787 to terminate.

[0092] Although blocks 781 through 787 of method 780 are discussed and illustrated in a particular order, Figure 7 The method 780 described in is not limited thereto. In other embodiments, the method 780 may be performed in a different order. In these and other embodiments, any of the blocks 781 to 787 of the method 780 may be performed before, during, and / or after any of the other blocks 781 to 787 of the method 780. For example, the method 780 may expose all or a subset of the event vision pixels of the array to constant illumination simultaneously and thereafter determine the event triggering rates of these event vision pixels at approximately the same time (e.g., sequentially or simultaneously). As another example, the method 780 may perform one or more of the blocks 781 to 786 for multiple event vision pixels simultaneously. Again, a person of ordinary skill in the relevant art will recognize that the method 780 described may be altered and still remain within these and other embodiments of the present invention. For example, in some embodiments, one or more blocks 781 to 787 of the method 780 may be omitted and / or repeated. As another example, block 781 may be performed for a given comparator threshold voltage, such as a given comparator threshold voltage for an up event generating comparator for an event vision pixel and / or a given comparator threshold voltage for a down event generating comparator for an event vision pixel. Additionally or alternatively, blocks 781 through 786 may be repeated for a plurality of comparator threshold voltages. For example, method 780 repeatedly performs block 781 while stepping or scanning through a particular set of comparator threshold voltages. Method 780 may then determine whether an event vision pixel is defective based on one or more noise event initiation rates determined for the event vision pixel at one or more comparator voltage thresholds.

[0093] C. in conclusion

[0094] The above detailed description of the embodiments of the present invention is not intended to be exhaustive or to limit the present invention to the precise form disclosed above. Although specific embodiments and examples of the present invention are described above for illustration, those skilled in the relevant art will recognize that various equivalent modifications may be within the scope of the present invention. For example, although the steps are presented above in a given order, alternative embodiments may perform the steps in a different order. In addition, the various embodiments described herein may also be combined to provide additional embodiments.

[0095] It can be understood from the above that the specific embodiments of the present invention have been described herein for illustration, but the well-known structures and functions are not shown or described in detail so as not to unnecessarily obscure the description of the embodiments of the present invention. When any material incorporated herein by reference conflicts with the present disclosure, the present disclosure controls. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. In addition, unless the term "or" is explicitly limited to only mean that a single item excludes other items in a list involving two or more items, the use of "or" in this list should be interpreted as including any single item in the (a) list, all items in the (b) list, or any combination of items in the (c) list. In addition, as used herein, the phrase "and / or" in "A and / or B" refers to only A, only B, and both A and B. In addition, the terms "include", "include", "have", and "contain" are used throughout to mean that at least the listed features are included to not exclude any more of the same features and / or other features of additional types. Furthermore, as used herein, the phrases "based on", "depending on", "due to", and "in response to" should not be interpreted as referring to a set of closed conditions. For example, an exemplary step described as "based on condition A" may be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "based at least in part on" or the phrase "based at least in part on." In addition, the terms "connected" and "coupled" may be used interchangeably herein and refer to both direct and indirect connections or couplings. For example, element A "connected" or "coupled" to element B may refer to (i) A being directly "connected" or "coupled" to B and / or (ii) A being indirectly "connected" or "coupled" to B, where the context permits.

[0096] It can also be understood from the above that various modifications can be made without departing from the present disclosure or the present invention. For example, it should be understood by those of ordinary skill in the art that the various components of the present invention can be further divided into subcomponents or can be combined and integrated with the various components and functions of the present invention. In addition, certain aspects of the technology described in the context of a particular embodiment may also be combined or eliminated in other embodiments. In addition, although the advantages associated with certain embodiments of the present invention have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments necessarily need to exhibit such advantages to fall within the scope of the present invention. Therefore, the present disclosure and associated technology may cover other embodiments that are not explicitly shown or described herein.

Claims

1. An event vision sensor, comprising: an array of event vision pixels arranged in rows and columns; and an event signal processor configured to identify a defect event visual pixel based at least in part on a noise event initiation rate corresponding to the defect event visual pixel of the array, wherein the noise event initiation rate is based at least in part on an arrival interval time corresponding to the defect event visual pixel, and wherein the arrival interval time is equal to an amount of time that elapses between a first time when the defect event visual pixel is reset and a second time when the defect event visual pixel detects a noise event.

2. The event vision sensor of claim 1, wherein the defective event vision pixel comprises a programmable memory component operable to render the defective event vision pixel unable to detect an event or to mask an output of the event vision pixel.

3. The event vision sensor according to claim 2, wherein the programmable memory component can be used to mask the output of the defective event vision pixel so that the event detected by the defective event vision pixel is not output from the defective event vision pixel to the event signal processor.

4. The event vision sensor according to claim 2, wherein: The defect event visual pixel comprises a photosensor configured to generate a photocurrent in response to incident light, a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent into a voltage, a difference detection circuit coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter, and at least one event generating comparator coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light; and The programmable memory component may be used to disable the photocurrent-to-voltage converter, the difference detection circuit, and the at least one event generating comparator such that the defective event vision pixel cannot detect an event.

5. The event vision sensor according to claim 2, wherein: The defect event visual pixel comprises a photosensor configured to generate a photocurrent in response to incident light, a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent into a voltage, a difference detection circuit coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter, and at least one event generating comparator coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light; The programmable memory component may be used to disable a subset of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator; and The subset excludes each of the photocurrent-to-voltage converter, the difference detection circuit, and the at least one event generating comparator.

6. The event vision sensor according to claim 5, wherein: the subset includes only the difference detection circuit from a group including the photocurrent-to-voltage converter, the difference detection circuit, and the at least one event generating comparator; The difference detection circuit includes an amplifier; and The programmable memory component may be used to couple the input of the amplifier to the output of the amplifier to keep the amplifier auto-zeroed so that the defect event vision pixel cannot detect an event.

7. The event vision sensor of claim 2, further comprising control logic configured to program the programmable memory component of the defective event vision pixel to disable the defective event vision pixel or mask the output of the defective event vision pixel based at least in part on the event signal processor identifying the defective event vision pixel.

8. The event vision sensor according to claim 1, further comprising: a lookup table configured to store addresses of defect event visual pixels of the array identified by the event signal processor; and control logic configured to program the programmable memory components of the defective event vision pixels after the event vision sensor is powered on or initialized based at least in part on the addresses stored in the lookup table so that the defective event vision pixels are disabled or so that an output of each of the defective event vision pixels is masked.

9. The event vision sensor according to claim 1, wherein: The noise event initiation rate is based at least in part on an average inter-arrival time of the defect event visual pixels; and To identify the defect event visual pixels, the event signal processor is configured to compare the noise event initiation rate with a threshold.

10. The event vision sensor of claim 9, wherein the threshold value represents an average noise event initiation rate corresponding to one or more event vision pixels of the array adjacent to the defect event vision pixel.

11. An event vision sensor, comprising: An array of event vision pixels arranged in rows and columns, wherein each of the event vision pixels of the array comprises a photosensor configured to generate a photocurrent in response to incident light, a photocurrent / voltage converter coupled to the photosensor to convert the photocurrent into a voltage, a difference detection circuit coupled to the photocurrent-to-voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent-to-voltage converter, at least one event generating comparator coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold value to detect an event indicated in the incident light, and A memory component that can be programmed to: disabling a subset of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator so that the event vision pixel cannot detect an event, wherein the subset does not include each of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator; or shielding the output of the event visual pixel; and An event signal processor is configured to identify defect event visual pixels corresponding to the defect event visual pixels of the array based at least in part on noise event initiation rates.

12. An event vision sensor according to claim 11, wherein each of the noise event initiation rates is based at least in part on an arrival interval time corresponding to a corresponding one of the defect event vision pixels, and wherein the arrival interval time represents an amount of time that elapses between a first time when the corresponding one of the defect event vision pixels is reset and a second time when the corresponding one of the defect event vision pixels detects a noise event.

13. An event vision sensor according to claim 11, wherein each of the noise event triggering rates is based at least in part on a noise event rate corresponding to a corresponding one of the defect event vision pixels, and wherein the noise event rate represents the number of noise events detected by the corresponding one of the defect event vision pixels divided by the cumulative amount of time required for the corresponding one of the defect event vision pixels to detect the noise event since reset.

14. The event vision sensor according to claim 11, further comprising a lookup table configured to store addresses of event visual pixels identified as defective by the event signal processor; and Control logic is configured to program a memory component of the defect event visual pixel based at least in part on the address stored to the lookup table.

15. The event vision sensor of claim 11, further comprising control logic configured to program a memory component of the defective event vision pixels identified by the event signal processor.

16. A method comprising: identifying an event vision pixel of an event vision sensor as defective based at least in part on a noise event initiation rate corresponding to the event vision pixel, wherein the noise event initiation rate is based at least in part on an inter-arrival time associated with the event vision pixel, and wherein the inter-arrival time represents an amount of time that elapses between a reset of the event vision pixel and a subsequent detection of a noise event; and The event vision pixel is prevented from outputting event data in response to identifying the event vision pixel as defective.

17. The method of claim 16, further comprising determining the noise event initiation rate corresponding to the defect event visual pixels, wherein determining the noise event initiation rate comprises: When the event visual pixel is exposed to DC illumination light, determining a first inter-arrival time associated with the event visual pixel, determining a second inter-arrival time associated with the event visual pixel; and An average inter-arrival time is determined based at least in part on the first inter-arrival time and the second inter-arrival time.

18. The method of claim 16, wherein identifying the event visual pixel as defective comprises (a) comparing the noise event initiation rate to a preset threshold and (b) determining that the noise event initiation rate exceeds the preset threshold.

19. The method of claim 16, wherein identifying the event vision pixel as defective comprises (a) determining an average noise event initiation rate based at least in part on noise event initiation rates associated with one or more event vision pixels adjacent to the event vision pixel in an array of the event vision sensor, (b) comparing the noise event initiation rate to the average noise event initiation rate, and (c) determining that the noise event initiation rate exceeds the average noise event initiation rate by more than a threshold amount.

20. The method of claim 16, wherein preventing the event vision pixel from outputting the event data comprises programming a memory component included within the event vision pixel such that (i) an output of the event vision pixel is masked or (ii) the event vision pixel is prevented from detecting an event.

21. The method of claim 20, wherein programming the memory component comprises programming the memory component such that a photocurrent / voltage converter of the event vision pixel, a difference detection circuit of the event vision pixel, at least one event generating comparator of the event vision pixel, or any combination thereof, is disabled.

Images