Event vision sensor with defective pixel suppression and related systems, devices, and methods

Abstract

Disclosed herein are event vision sensors with defective pixel mitigation and related systems, devices, and methods. In one embodiment, an event vision sensor includes an array of event vision pixels and an event signal processor. The event signal processor is configured to identify defective event vision pixels of the array based on a noise event initiation rate corresponding to the event vision pixels. The noise event initiation rate of each event vision pixel can be based on an inter-arrival time or a noise event rate corresponding to the event vision pixel. Each event vision pixel can include internal circuitry (e.g., a memory component such as a latch) that can be used to disable the event vision pixel from detecting events or mask the output of the event vision pixel such that events cannot be read out from the event vision pixel when the event vision pixel is identified as defective.

Description

Technical Field

[0001] This disclosure generally relates to event vision sensors. For example, several embodiments of the invention relate to event vision sensors employing defective pixel suppression, such as intra-pixel defective pixel suppression based on noise event initiation rate. Background Technology

[0002] Image sensors have become ubiquitous and are now widely used in digital cameras, cellular phones, webcams, and in medical, automotive, and other applications. As image sensors are integrated into a wider range of electronic devices, there is a desire to enhance their functionality and performance in as many ways as possible through device architecture design and image acquisition and processing, such as resolution, power consumption, and dynamic range.

[0003] A typical image sensor operates in response to image light from an external scene incident upon it. The image sensor comprises an array of pixels with photosensitive elements (such as photodiodes) that absorb a portion of the incident image light and generate an image charge upon absorption. The image charge generated by the pixel light can be measured as an analog output image signal on the bit lines that varies according to the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out as an analog image signal from the bit lines and converted into a digital value to provide information representing the external scene. Summary of the Invention

[0004] One aspect of this disclosure relates to an event vision sensor comprising: an event vision pixel array arranged in rows and columns; and an event signal processor configured to identify the defective event vision pixel at least in part based on a noise event initiation rate corresponding to the defective event vision pixel in the array, wherein the noise event initiation rate is at least in part based on an arrival interval time corresponding to the defective event vision pixel, and wherein the arrival interval time is equal to the amount of time elapsed between a first time at which the defective event vision pixel is reset and a second time at which the defective event vision pixel detects a noise event.

[0005] Another aspect of this disclosure relates to an event vision sensor comprising: an event vision pixel array arranged in rows and columns, wherein each of the event vision pixels in the array comprises: a photodetector configured to generate a photocurrent in response to incident light; a photocurrent / voltage converter coupled to the photodetector to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter; and at least one event generation comparator coupled to the difference detection circuit and configured to compare the event generated from the difference detection circuit. The signal received by the path and at least one threshold are used to detect events indicated in the incident light; and a memory component that can be programmed to: disable a subset of the photocurrent / voltage converter, the differential detection circuit, and the at least one generating comparator, such that the event visual pixel cannot detect events, wherein the subset does not include each of the photocurrent / voltage converter, the differential detection circuit, and the at least one event generating comparator; or mask the output of the event visual pixel; and an event signal processor configured to identify the defective event visual pixel at least in part based on the noise event initiation rate corresponding to the defective event visual pixel in the array.

[0006] Another aspect of this disclosure relates to a method comprising: identifying an event visual pixel as defective based at least in part on a noise event initiation rate corresponding to an event visual pixel of an event visual sensor, wherein the noise event initiation rate is at least in part based on an arrival interval time associated with the event visual pixel, and wherein the arrival interval time represents the amount of time elapsed between the event visual image being reset and the subsequent detection of a noise event; and preventing the event visual pixel from outputting event data in response to identifying the event visual pixel as defective. Attached Figure Description

[0007] The following description, with reference to the figures below, illustrates non-limiting and non-exhaustive embodiments of the invention, wherein the same or similar reference numerals are used to refer to all the same or similar components unless otherwise specified.

[0008] Figure 1 This is a partial schematic block diagram of an event vision sensor and a lighting source configured according to various embodiments of the present invention.

[0009] Figure 2 This is a partial schematic block diagram of an event signal processor configured according to various embodiments of the present invention.

[0010] Figure 3 This is a partial schematic block diagram illustrating the configuration of event visual pixels according to various embodiments of the present invention.

[0011] Figure 4This is a partial schematic block diagram illustrating the configuration of another event visual pixel according to various embodiments of the present invention.

[0012] Figure 5 This is a partial schematic block diagram illustrating another event visual pixel configured according to various embodiments of the present invention.

[0013] Figure 6 This is a partial schematic block diagram illustrating another event visual pixel configured according to various embodiments of the present invention.

[0014] Figure 7 This is a flowchart illustrating a method for identifying and disabling / masking defective event visual pixels of an event visual sensor according to various embodiments of the present invention.

[0015] Figure 8 This is a flowchart illustrating a method for determining the noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0016] Figure 9 This is a flowchart illustrating another method for determining the noise event initiation rate of an event visual pixel according to various embodiments of the present invention.

[0017] Those skilled in the art will understand that the elements in the figures are for illustrative purposes and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be enlarged relative to other elements to aid in understanding various aspects of the invention. Furthermore, common but readily understood elements or methods that are useful or necessary in commercially viable embodiments are generally not depicted in the figures or described in detail below to avoid unnecessarily obscuring the description of various aspects of the invention. Detailed Implementation

[0018] This disclosure relates to event vision sensors. For example, several embodiments of the invention relate to event vision sensors employing defective pixel suppression (such as intra-pixel defective pixel suppression). In the following description, specific details are set forth to provide a thorough understanding of aspects of the invention. However, those skilled in the art will recognize that the systems, apparatuses, and techniques described herein can be practiced without using one or more of the specific details set forth herein or using other methods, components, materials, etc.

[0019] The terms "example" or "embodiment" in this specification mean that a particular feature, structure, or characteristic described in connection with an example or embodiment is included in at least one example or embodiment of the invention. Therefore, the phrases "for example," "as an example," or "embodiment" as used herein do not necessarily refer to all the same example or embodiment and are not necessarily limited to the specific example or embodiment discussed. Furthermore, the features, structures, or characteristics of the invention described herein may be combined in any suitable manner to provide further examples or embodiments of the invention.

[0020] For ease of description, spatial relative terms (e.g., "below," "below," "above," "under," "above," "top," "bottom," "left," "right," "center," "middle," etc.) are used herein to describe the relationship of an element or feature relative to one or more other elements or features, as illustrated in the figures. It should be understood that, in addition to the orientations depicted in the figures, spatial relative terms are intended to cover different orientations of the device or system during use or operation. For example, if the device or system illustrated in the figures rotates, turns, or flips about a horizontal axis, then an element or feature described as "below," "below," or "under" one or more other elements or features may be oriented "above" one or more other elements or features. Therefore, the exemplary terms "below" and "under" are non-limiting and can cover both above and below orientations. Devices or systems may be oriented in ways other than those illustrated in the figures (e.g., rotated 90 degrees about a vertical axis or otherwise) and thus the spatial relative descriptive terms used herein shall be interpreted accordingly. In addition, it should be understood that when an element is referred to as being between two other elements, it may be the only element between the two other elements or there may be one or more intermediary elements.

[0021] Several technical terms are used in this specification. These terms should be used with respect to their general meaning in the field, unless expressly defined herein or the context in which they are used clearly implies otherwise. It should be noted that component names and symbols are used interchangeably in this document (e.g., Si and silicon); however, they have the same meaning.

[0022] A. SUMMARY

[0023] Active pixel sensors (such as CMOS imaging systems) typically employ an active pixel array with a globally defined integration time. Therefore, active pixels in an active pixel sensor generally have the same integration time, and each pixel in the array is typically converted into a digital signal regardless of its content (e.g., whether the external scene captured by the pixel has changed since the last readout). In other words, image data generated by active pixels in, for example, a CMOS imager, is read out in frames of known size, regardless of whether events occur in the external scene.

[0024] In contrast, when a pixel detects a change in the external scene (e.g., an event), an event vision sensor (e.g., an event-driven sensor or a dynamic vision sensor) reads out the pixel and / or converts the corresponding pixel signal into a digital signal. In other words, pixels of an event vision sensor that do not detect a change in the external scene are not read out and / or the pixel signal corresponding to such a pixel is not converted into a digital signal. Therefore, each pixel of an event vision sensor can operate independently of other pixels in the event vision sensor, and only the pixel that detects a change in the external scene needs to be read out and / or its corresponding pixel signal converted into a digital signal or recorded (thereby saving power). Consequently, an event vision sensor does not need to record the entire regular image and therefore does not need to capture and record all the highly redundant information of a normal image frame by frame. Therefore, an event vision sensor can be used to detect movement or motion in the external scene (e.g., rather than for capturing / reading out all frames of an image or video), while being able to (i) use low data rates and (ii) achieve ultra-high frame rates or speed capabilities.

[0025] When an event vision pixel array is used to monitor an external scene in an event vision sensor, the event vision sensor rarely detects and records isolated events corresponding to legitimate activities in the external scene (e.g., events corresponding to a single isolated event vision pixel in the array). These isolated events often correspond to noise and / or are caused by defective event vision pixels in the array. Therefore, many event vision sensors employ various techniques (e.g., encoding or compression techniques) to filter out noisy event signals read from the event vision pixels. As a specific example, when the first event vision pixel in the array detects an event, some event vision sensors analyze a set of event vision pixels containing the first event vision pixel. If the event vision sensor determines that a threshold number of event vision pixels in the set have detected an event, then the event detected by the first event vision pixel is classified as corresponding to legitimate activities in the external scene monitored by the event vision sensor. On the other hand, if the event vision sensor determines that fewer than the threshold number of event vision pixels in the set have detected an event, then the event detected by the first event vision pixel is classified as a noise event and / or discarded. This technique is called coincidence detection.

[0026] Conformity detection eliminates small statistics. In other words, conformity detection cannot analyze or provide any insight into long-term trends (e.g., it cannot provide any insight into whether the detection of noisy events by visual pixels for a particular event is relatively rare or common). Therefore, repeat offenders (e.g., defective pixels that frequently detect noisy events) are not processed and continue to consume the excess bandwidth of the event vision sensor to read out noisy events. Furthermore, the small statistics of conformity detection increase the likelihood that the event vision sensor will classify a legitimate event as a noisy event or vice versa.

[0027] Furthermore, conventional methods operate by reading signals from the event vision pixels of an array. In other words, many conventional methods only identify defective event vision pixels and / or filter out their corresponding signals after reading out noisy events from the event vision pixels during normal operation of the event vision sensor. Therefore, event vision sensors employing these conventional methods typically utilize the excess bandwidth of the event vision sensor during normal operation to read out the signals of defective event vision pixels before (a) an event vision pixel is identified as defective and / or (b) its corresponding signal is filtered out and / or discarded (consuming excess bandwidth).

[0028] In view of the problems discussed above, the present invention provides various solutions for identifying defective event visual pixels using in-pixel circuitry and disabling or masking their output. For example, several embodiments of the invention identify defective event visual pixels of an event vision sensor based on the noise event initiation rate of each event visual pixel. More specifically, during one or more points in the lifetime of the event vision sensor (e.g., during wafer testing, during power-on or initialization, after a predetermined amount of time, outside of normal operation, when the event vision sensor determines that the event (legal and / or noise) initiation rate of one or more event visual pixels exceeds a threshold, etc.), the event visual pixel may be exposed to DC illumination when the event vision sensor (e.g., the event signal processor of the event vision sensor) determines the noise event initiation rate of each event visual pixel. In some embodiments, the noise event initiation rate of the event visual pixel may be based on the average arrival interval time (e.g., the average time between the time the event visual pixel resets and the time the event visual pixel subsequently detects a noise event). In these and other embodiments, the noise event initiation rate of the event visual pixel may be based on the number of noise events detected within a time period. Once the trigger rate of the event visual pixels is obtained, the trigger rate can be compared with one or more thresholds to identify whether the event visual pixels are defective (e.g., unacceptably likely to record or unacceptably prone to recording noisy events).

[0029] When an event vision pixel is identified as defective, the event vision sensor can programmably disable the event vision pixel (e.g., prevent the event vision pixel from detecting an event) or mask its output (e.g., prevent the event detected by the event vision pixel from being output by the event vision pixel) using a memory component (e.g., a latch) contained within the event vision pixel. Alternatively, the address of the event vision pixel can be added to a lookup table maintained by the event vision sensor, so that the event vision pixel can be disabled or masked whenever the event vision sensor is powered on or initialized.

[0030] In this manner, the present invention provides various solutions for identifying defective event visual pixels in an event vision sensor, which eliminates the need to (a) read out noisy events and legitimate events during normal operation and (b) distinguish whether an event should be classified as legitimate or noise. Furthermore, the present invention provides an intra-pixel defective pixel suppression solution that can be used to deactivate event visual pixels identified as defective and / or mask their output. This prevents such defective event visual pixels from consuming excess bandwidth of the event vision sensor to read out noise events detected by the event visual pixels. Therefore, compared to event vision sensors lacking defective event visual pixel suppression solutions and / or corresponding intra-pixel circuitry, the present invention is expected to reduce the number of noisy events read out to the event signal processor of the event vision sensor, increase the percentage of legitimate events read out to the event signal processor, and / or utilize less power.

[0031] B. Selected embodiments of event vision sensors with defective pixel suppression and related systems, devices, and methods

[0032] Figure 1 This is a partial schematic block diagram of an event vision sensor 100 (also referred to herein as an "imager") configured according to various embodiments of the present invention. As shown, the event vision sensor 100 includes an array 102 of event vision pixel circuitry 110 ("event vision pixel 110"), a row control circuitry system 104, a column control circuitry system 106, an event signal processor 108, a transmitter 116, a lookup table 142 ("LUT 142"), control logic 144, and peripheral circuitry 146. An illumination source 120 configured to project light 150 onto the event vision pixels 110 of the array 102 is also shown. In some embodiments, the illumination source 120 may be a component of the event vision sensor 100 or an imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be external to the event vision sensor 100 and the imaging system incorporated into the event vision sensor 100. In other embodiments, the illumination source 120 may be omitted.

[0033] Event visual pixels 110 are arranged in rows and columns in array 102. Figure 1(Rows and columns not shown). As discussed in more detail below, event visual pixels 110 are configured to detect changes in light received from an external scene (e.g., events). For this purpose, each event visual pixel 110 may include: (i) a photoelectric sensor (such as a photodiode) configured to generate a photocharge or photocurrent in response to incident light received from an external scene; (ii) a photocurrent / voltage converter coupled to the photoelectric sensor to convert the photocurrent generated by the photoelectric sensor into a voltage; and (iii) a filter amplifier coupled to the photocurrent / voltage converter to generate a filtered and amplified signal in response to a voltage received from the photocurrent / voltage converter. Event visual pixels 110 may further include threshold comparison circuitry or stages that generate and receive handshake signals in response to asynchronously detected events in the incident light received from an external scene. Alternatively, the threshold comparison circuitry may be included in a circuitry system peripheral to or outside the event visual pixels 110 of array 102 (e.g., in event signal processor 108), such as within a column readout circuitry system. The photoelectric sensor, photocurrent / voltage converter, filter amplifier, and / or threshold comparator of the event vision pixel 110 are collectively referred to herein as the Event Vision Sensor (EVS) Core Pixel Circuit System 130.

[0034] like Figure 1 As shown, the event visual pixels 110 configured according to various embodiments of the present invention may each further include a programmable memory component 136 (described as and hereinafter referred to as "latch 136") coupled to the EVS core pixel circuitry system 130. The latch 136 of the event visual pixel 110 can be programmed using a program signal (prog), a row signal corresponding to the event visual pixel 110, and a column signal corresponding to the event visual pixel 110. More specifically, when an event visual pixel 110 of array 102 is identified as a defective pixel (discussed in more detail below), the address (row address and column address) corresponding to the position of the event visual pixel 110 in array 102 can be stored in LUT 142. Control logic 144 can then use the address stored in LUT 142 to program (e.g., after the event vision sensor 100 is powered on or initialized) the latch 136 corresponding to event vision pixel 110 to disable event vision pixel 110 and / or mask its output, so that events detected by event vision pixel 110 are not read from array 102. See below for reference. Figures 3 to 6 Various embodiments of the event visual pixel 110, which incorporates a latch similar to latch 136, are discussed in detail.

[0035] Figure 1The row control circuitry 104 and column control circuitry 106 are used to control the rows and columns of event visual pixels 110 in the array 102, respectively. For example, the row control circuitry 104 and / or column control circuitry 106 may be configured to reset and / or read out event visual pixels 110 (e.g., individual ones or rows of event visual pixels 110) from the array 102 (e.g., along the corresponding column bit lines connected to the event visual pixels 110).

[0036] Event data read from the event vision pixels 110 of array 102 can be passed to the event signal processor 108 of the event vision sensor 100 for processing. The event data processed by the event signal processor 108 can be provided to the transmitter 116 to transmit the event data from the event vision sensor 100, such as to a receiver (not shown) of a corresponding imaging system. Alternatively, all or a subset of the event data can be stored in memory (e.g., before or after being provided to the event signal processor 108 and / or the transmitter 116).

[0037] The event signal processor 108 of the event vision sensor 100 can also be used to identify defective event vision pixels 110 of the array 102. For example, an illumination source 120 can be used to project light 150 (e.g., DC illumination) onto the event vision pixels 110 of the array 102. In some embodiments, the light 150 may have a constant intensity or brightness (or remain substantially constant or change slowly over time). Therefore, when the illumination source 120 continuously projects light 150 onto the event vision pixels 110 of the array 102, the event vision pixels 110 should not (unless defective or subjected to noise) detect an event or detect an event at a frequency higher than a threshold. The event signal processor 108 can therefore utilize the illumination source 120 to identify defective or noisy event vision pixels 110 of the array 102 and subsequently deactivate such pixels 110 and / or mask their output. Reference below Figures 7 to 9 The method for identifying defective events in visual pixels 110 of array 102 is discussed in detail.

[0038] 1. Selected embodiments of event signal processors

[0039] Figure 2 This is a partial schematic block diagram of an event signal processor 208 configured according to various embodiments of the present invention. It should be understood that the described event signal processor 208 may be... Figure 1The event signal processor 108 is an example of another event visual pixel of the present invention. As shown, the event signal processor 208 includes a defective pixel removal block 270, which includes various stages or circuits for identifying and disabling / masking defective event visual pixels corresponding to the event visual sensor. For example, the defective pixel removal block 270 includes a comparator voltage regulator block 271, an event occurrence recording block 272, an event occurrence characterization block 275, and a defective pixel determination and recording block 278.

[0040] The comparator voltage regulator block 271 of the event signal processor 208 can be configured to adjust the voltage of the event visual pixels (e.g., ...). Figure 1 The event-generating comparator uses a comparator voltage threshold for the event visual pixel 110. For example, the comparator voltage regulator block 271 can be coupled to the peripheral circuitry of the corresponding event visual sensor (e.g., ...). Figure 1 The peripheral circuitry system 146 of the visual pixel sensor 100. Using the peripheral circuitry system, the comparator voltage regulator block 271 can adjust the event-generating comparator of the event visual pixel for detecting UP events (e.g., events greater than the comparator threshold voltage V). U The comparator threshold voltage V (the change in brightness of the light incident on the photoelectric sensor of the event vision pixel from darker to brighter). U Alternatively, the comparator voltage regulator block 271 may use an external circuit system to adjust the event-generating comparator of the event visual pixel used to detect DOWN events (e.g., events greater than the comparator threshold voltage V). D The comparator threshold voltage V (the change in brightness of the light incident on the photoelectric sensor of the event vision pixel from brighter to darker). D .

[0041] As discussed in more detail below, the comparator voltage regulator block 271 of the event signal processor 208 can be used to turn off or disable the event generating comparator of the event visual pixel, while another event generating comparator is used to determine whether the event visual pixel is defective. For example, the comparator voltage regulator block 271 can be used to adjust the comparator voltage threshold V used by the downward event generating comparator of the event visual pixel. D Set to -∞ (or some other large negative voltage value), and the event signal processor 208 uses the upward event generation comparator of the event visual pixel to determine whether the event visual pixel is defective. Set the comparator voltage threshold V... D Setting it to -∞ (or some other large negative voltage value) prevents downward events from generating comparator recording events. Alternatively, comparator voltage regulator block 271 can be used to adjust the comparator voltage threshold V used by the comparator for upward events generated by the event visual pixel. U Set one or more thresholds (or make the comparator voltage threshold V) UScanning through a set of one or more thresholds enables, for example, the event occurrence characterization block 275 of the event signal processor 208 to model the probability of detecting a noisy event within one or more observation windows.

[0042] The event occurrence recording block 272 of the event signal processor 208 is configured to record noise events detected by event visual pixels. For example, the event occurrence recording block 272 may record (i) a timestamp corresponding to the time when the noise event was detected and / or (ii) the location (e.g., row address and column address) of the event visual pixel that detected the noise event. Alternatively, the event occurrence recording block 272 may record the polarity or change of the noise event (e.g., indicating whether the noise event is an UP noise event or a DOWN noise event).

[0043] Event occurrence characterization block 275 can perform various calculations and modeling of noise events detected by event visual pixels. For example, event occurrence characterization block 275 may include event probability distribution building module 276, which is configured to model one or more probabilities of an event visual pixel detecting a noise event (e.g., at a given comparator voltage threshold) within one or more given observation windows (e.g., of different lengths) based at least in part on noise events recorded by event occurrence recording block 272. Alternatively, event occurrence characterization block 275 may include statistical calculation block 277 configured to calculate various statistics for event visual pixels. Examples of statistics that can be calculated by statistical calculation block 277 include the noise event initiation rate of event visual pixels, the variance of the probability density function determined by event probability distribution building module 276, higher-order moments, quantiles, variability of any moment relative to a reference event visual pixel, and / or statistics related to a set of event visual pixels (e.g., a set of event visual pixels containing event visual pixels and / or one or more event visual pixels falling within a certain distance of said event visual pixels).

[0044] The defective pixel determination and recording block 278 of the event signal processor 208 is configured to identify defective event visual pixels based on (a) statistics output from the statistical calculation block 277 and (b) specified criteria or thresholds. For example, the defective pixel determination and recording block 278 may determine that an event visual pixel is defective when the calculated noise event occurrence rate is greater than a threshold. As another example, the defective pixel determination and recording block 278 may determine that an event visual pixel is defective when the variance or other statistics calculated by the statistical calculation block 277 is greater than a threshold. The threshold used by the defective pixel determination and recording block 278 may be predefined (e.g., preset, predetermined). Alternatively or additionally, the threshold used by the defective pixel determination and recording block 278 may depend on statistics associated with one or more other (e.g., adjacent) event visual pixels. For example, the defective pixel determination and recording block 278 may determine that an event visual pixel is defective when the noise event occurrence rate is greater than the average event occurrence rate corresponding to one or more adjacent event visual pixels (e.g., reaching a specified amount).

[0045] When the defective pixel determination and recording block 278 determines that the event visual pixel is defective, the defective pixel determination and recording block 278 can store the address (e.g., row address and column address) corresponding to the event visual pixel in a lookup table (e.g., in...). Figure 1 In LUT 142). Alternatively or concurrently, the defect pixel determination and recording block 278 may be coupled to control logic (e.g., Figure 1 Control logic 144). In these embodiments, the defective pixel determination and recording block 278 may communicate with and / or use the control logic to disable the defective event visual pixel or mask its output to prevent the defective event visual pixel from detecting an event or outputting an event from the event visual pixel, respectively. For example, the defective pixel determination and recording block 278 may instruct the control logic to access the memory components (e.g., latches, such as) of the defective event visual pixel via the row control circuitry and / or column control circuitry of the corresponding event visual sensor. Figure 1 The latch 136 is programmed. See below for reference. Figures 3 to 6 The circuitry for disabling visual pixels and / or masking their output during defective events is discussed in more detail.

[0046] As shown, the event signal processor 208 may additionally include an auxiliary function block 279, which includes circuitry or blocks for performing various auxiliary processing functions. For example, the auxiliary function block 279 may include a segmentation classifier block and / or a shape classifier block for classifying segments and shapes of event data read from the array, respectively. As another example, the auxiliary function block 279 may include an optical flow estimation block for identifying pixel-wise, shape-wise, or segment-wise motion over time and / or between consecutive readouts (e.g., using correlation-based, block-matching-based, feature-tracking-based, energy-based, and / or gradient-based optical flow estimation). In these and other embodiments, the auxiliary function block 279 may include a compression block to compress the event data read from the array. Additionally or alternatively, the event signal processor 208 may include a set of one or more buffers (e.g., a set of row buffers) operatively coupled to one or more blocks or circuitry of the event signal processor 208 to perform various processing functions.

[0047] 2. Selected embodiments of event vision pixels

[0048] Figure 3 This is a partial schematic block diagram illustrating the configuration of an event visual pixel 310 according to various embodiments of the present invention. It should be understood that the illustrated event visual pixel 310 may be included in... Figure 1 An instance of one of the event visual pixels 110 in array 102 or another event visual pixel of the present invention. As shown, event visual pixel 310 includes elements configured to respond to incident light 350 (e.g., from an external scene and / or from a lighting source such as... Figure 1 A photoelectric sensor 331 receives light from an illumination source 120 and generates photocharge or photocurrent. The photoelectric sensor 331 is coupled to a logarithmic amplifier 332, which is configured to convert the photocurrent generated by the photoelectric sensor 331 into a voltage. In various embodiments, the logarithmic amplifier 332 is configured to generate a voltage by transducing the instantaneous photocurrent received from the photoelectric sensor 331. A difference detection amplifier 333 is coupled to the logarithmic amplifier 332 to generate a filtered and amplified signal in response to a difference detected in the voltage received from the logarithmic amplifier 332. In one embodiment, the difference detection amplifier 333 is configured to compare the instantaneous logarithmic intensity of the voltage output of the logarithmic amplifier 332 with a reference level based on a reset condition or a recent event.

[0049] An event generation comparator 334 is coupled to a difference detection amplifier 333 to compare a filtered and amplified signal received from the difference detection amplifier 333 with a threshold for asynchronous detection of an event indicated in the incident light 350. In one example, the event generation comparator 334 is configured to determine whether the signal difference is large enough to trigger an event. In some embodiments, the event generation comparator 334 includes a first comparator (not shown) configured to detect whether the signal difference corresponds to an 'UP' event (e.g., a change in the intensity of light incident on the photodetector 331 from dark to bright and is greater than a comparator voltage threshold V). U The event-generating comparator 334 may further include a second comparator (not shown) configured to detect whether the signal difference corresponds to a 'DOWN' event (e.g., a change in the intensity of light incident on the photodetector 331 from brighter to darker and greater than the comparator voltage threshold V). D ).

[0050] Figure 3 The event visual pixel 310 may further include a scanner and readout logic 335 (“readout logic 335”). In some embodiments, the readout logic 335 may include a latch that is (a) coupled to one or more outputs of the event generation comparator 334 and (b) configured to trigger when the event visual pixel 310 detects an event. Alternatively or additionally, the readout logic 335 may be coupled to a readout control circuitry system corresponding to the event visual sensor (e.g., Figure 1 The row control circuitry 104 and / or column control circuitry 106 are used to request the readout of event visual pixel 310 whenever event visual pixel 310 detects an event. In this way, readout logic 335 can be used to ensure that event visual pixel 310 is readout only when event visual pixel 310 detects an event. In other words, readout logic 335 can be used to ensure that event visual pixel 310 is not readout when event visual pixel 310 does not detect an event. When event visual pixel 310 detects an event, the event can be read from event visual pixel 310 onto the column bit line (not shown) corresponding to event visual pixel 310.

[0051] Therefore, it should be understood that including similar Figure 3 The event visual pixel 310 of the event visual pixel of the event visual sensor (e.g. Figure 1 The event vision sensor 100 does not need to record the entire regular image and therefore does not need to capture and record all the highly redundant information of the normal image frame by frame. Specifically, in various embodiments, the event vision sensor can read and record unique events. For example, the event vision sensor can read and / or record the location where an event is detected (e.g., Figure 1The event vision sensor can be configured to detect movement or motion in an external scene (e.g., rather than for capturing / reading out all frames of an image or video), thereby enabling the use of low data rates and achieving ultra-high frame rates or speed capabilities in the event vision sensor of the present invention. This includes the xy coordinates of the corresponding event vision pixels in array 102, the change in the photocurrent of the event (e.g., brighter or darker), the polarity of the event's photocurrent, and / or the timing of the occurrence or detection of the event. In other words, the event vision sensor can be used to detect movement or motion in an external scene (e.g., rather than for capturing / reading out all frames of an image or video), thereby enabling the use of low data rates and achieving ultra-high frame rates or speed capabilities in the event vision sensor of the present invention.

[0052] Continue to refer to Figure 3 The event visual pixel 310 further includes a programmable memory component 336 (described as and hereinafter referred to as "latch 336") and a logic gate 337. The latch 336 includes three inputs and one output. The logic gate 337 includes: (a) a first input coupled to the output of the latch 336; (b) one or more second inputs coupled to one or more outputs of the readout logic 335; and (c) one or more outputs coupled to the column bit lines and / or readout control circuitry system corresponding to the event visual pixel (e.g., ...). Figure 1 The row control circuit system 104 and / or column control circuit system 106). Logic gate 337 in Figure 3 The example shown is an AND gate, but it can be another logic gate in other embodiments of the invention.

[0053] Latch 336 is configured to receive program signals, row signals, and column signals at its inputs. These signals, discussed in more detail below, can be used to program latch 336 and selectively assert masking signals output from latch 336 and fed into logic gate 337. (Also referred to in this article as "disable signal") The row and column signals input into latch 336 can correspond to the position of event visual pixel 310 in the array of such pixels.

[0054] The shielding signal output from latch 336 The state depends on the input to latch 336. More specifically, in the illustrated example, the masking signal... This can be assumed to be unassertified (e.g., it can be in the first state or "1"). Therefore, under normal operation, the output of logic gate 337 can follow the output of readout logic 335. For example, when an event occurs in an external scene monitored by an event vision sensor including event vision pixel 310, the event can be indicated by a rapid or sudden change in intensity or brightness in the incident light 350 received by the photoelectric sensor 331 of event vision pixel 310. In other words, if the external scene is static and no event occurs, the brightness of the incident light 350 remains substantially constant. Therefore, the photocurrent generated by the photoelectric sensor 331 remains substantially constant and the event vision pixel 310 (unless defective) does not record events or records events frequently. However, if an event occurs in the external scene (e.g., movement, lighting change, reflection, excitation, etc.), the event is indicated by an asynchronous rapid or sudden change in the brightness of the incident light 350. The brightness change can be from darker to brighter or from brighter to darker. Therefore, there is an asynchronous change or Δ in the photocurrent generated by the photoelectric sensor 331. The change in photocurrent, or Δ, is converted into a voltage by logarithmic amplifier 332, filtered and amplified by differential detection amplifier 333, and then detected by event generation comparator 334. The event can be latched in readout logic 335 until it is read out to the corresponding column bit line via logic gate 337. Alternatively, readout logic 335 can send a request (e.g., via logic gate 337) to the readout control circuitry to read out the event visual pixel 310. The event can then be read out (via logic gate 337) to the column bit line corresponding to the event visual pixel 310.

[0055] On the other hand, if the event visual pixel 310 is identified as a defective pixel, then the program signal, row signal, and column signal input to the latch 336 can be used to program the latch 336 and assert the mask signal. When asserting the signal blocking When (for example, when the signal is blocked) When transitioning to the second state or "0", one or more outputs of logic 335 can be read through logic gate 337. For example, when the assertion mask signal is used... At the same time, logic gate 337(i) can prevent a read request output from read logic 335 from reaching the read control circuit system corresponding to the event visual pixel 310 and / or (ii) can prevent an event detected by the event visual pixel 310 from being output to the column bit line corresponding to the event visual pixel 310.

[0056] When event visual pixel 310 is identified as a defective pixel, according to Figure 3The method shown in the diagram for shielding the output of event vision pixel 310 offers several advantages. For example, events detected by event vision pixel 310 that are most likely attributable to noise and / or defects in event vision pixel 310 are not output from event vision pixel 310 and therefore do not consume the available bandwidth of the event vision sensor incorporated into event vision pixel 310. Additionally, only the output of readout logic 335 is shielded. In other words, the various analog stages of event vision pixel 310 (e.g., logarithmic amplifier 332, differential detection amplifier 333, event generation comparator 334, and readout logic 335) remain powered on. Therefore, the DC bias of each of these analog stages remains enabled, and event vision pixel 310 (when identified as defective) continues to consume the same relative current as other event vision pixels incorporated into the array that are not identified as defective. Therefore, even though the output of one or more defective event vision pixels across the array can be shielded, the current consumed by event vision pixels across the array remains substantially uniform. Thus, substantially uniform IR drop across the array can be maintained or observed.

[0057] Figure 4 This is a partial schematic block diagram illustrating another event visual pixel 410 configured according to various embodiments of the present invention. It should be understood that the illustrated event visual pixel 410 may be included in... Figure 1 An instance of one of the event visual pixels 110 in array 102 or another event visual pixel of the present invention. As shown, event visual pixel 410 includes elements configured to respond to incident light 450 (e.g., from an external scene and / or from a lighting source such as... Figure 1 The illumination source 120 receives the photoelectric sensor 431, which generates photocharge or photocurrent. The event visual pixel 410 further includes a logarithmic amplifier 432, a difference detection amplifier 433, an event generation comparator 434, and readout logic 435. The logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 can be roughly similar to... Figure 3 The system includes a logarithmic amplifier 332, a difference detection amplifier 333, an event generation comparator 334, and a readout logic 335. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 is omitted here.

[0058] The event visual pixel 410 also includes a programmable memory component 436 (described as and referred to hereinafter as "latch 436"). Latch 436 is generally similar to... Figure 3 The latch 336. Therefore, for the sake of brevity, a detailed discussion of latch 436 is largely omitted here. However, unlike... Figure 3 latch 336, Figure 4The latch 436 is coupled to the gate of the transistor 438, which acts as a switch to respond to a deactivation signal output from the latch 436. The state selectively couples the logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 of the event visual pixel 410 to the power supply voltage V. DD As discussed above, disabling signals It can be assumed to be unasserted (e.g., it can be in the first state or "1"). Therefore, under normal operation, the logarithmic amplifier 432, the difference detection amplifier 433, the event generation comparator 434, and the readout logic 435 can be coupled to the power supply voltage V via transistor 438. DD And the voltage V is supplied by the power supply. DD Power supply. On the other hand, when the event visual pixel 410 is identified as a defective pixel, the program signal, row signal, and column signal input to the latch 436 can be used to program the latch 436 and assert a deactivation signal. When asserting the deactivation signal When (e.g., when the deactivation signal) When transitioning to the second state (or "0"), the activation transistor 438 can be deactivated to enable the logarithmic amplifier 432, differential detection amplifier 433, event generation comparator 434, and readout logic 435 of the event visual pixel 410 to be connected to the power supply voltage V. DD Decoupling prevents visual pixel 410 from detecting events.

[0059] When event visual pixel 410 is identified as a defective pixel, according to Figure 4 The method shown in the diagram to prevent event detection by event vision pixel 410 offers several advantages. For example, (i) noise and / or defects attributable to event vision pixel 410 and (ii) events that would otherwise be detected by event vision pixel 410 are prevented from being generated or output from event vision pixel 410 and thus do not consume the available bandwidth of the event vision sensor incorporated into event vision pixel 410. Additionally, when event vision pixel 410 is identified as defective, the analog stage of event vision pixel 410 is prevented from consuming current. Therefore, current consumption corresponding to the power supply voltage V is avoided. DD This reduces excessive coupling on the power supply line and saves power that would otherwise be consumed by the analog stage of the event vision pixel 410.

[0060] That is, cutting off the DC bias of each of the analog stages of the event visual pixel 410 in this manner can result in uneven current consumption across the array of event visual pixels containing the event visual pixel 410. In other words, when several event visual pixels in the array are identified as defective, uneven IR drop across the array can be observed. Recognizing this problem, in some embodiments of the invention, when an event visual pixel is identified as defective, only a subset of the analog stages of the event visual pixel (e.g., any combination of logarithmic amplifiers, differential detection amplifiers, event generation comparators, and / or readout logic less than all analog stages) can be disabled (e.g., so that the uniformity of IR drop across the array can be substantially maintained or observed).

[0061] As a specific example, refer to Figure 5 This is a partial schematic block diagram illustrating another event visual pixel 510 configured according to various embodiments of the present invention. It should be understood that the illustrated event visual pixel 510 may be included in... Figure 1 An instance of one of the event visual pixels 110 in array 102 or another event visual pixel of the present invention. As shown, event visual pixel 510 includes elements configured to respond to incident light 550 (e.g., from an external scene and / or from a lighting source such as... Figure 1 The illumination source 120 receives the photoelectric sensor 531, which generates photocharge or photocurrent. The event visual pixel 510 further includes a logarithmic amplifier 532, a difference detection amplifier 533, an event generation comparator 534, and readout logic 535. The logarithmic amplifier 532, the difference detection amplifier 533, the event generation comparator 534, and the readout logic 535 can be roughly similar to... Figure 3 The system includes a logarithmic amplifier 532, a difference detection amplifier 533, an event generation comparator 534, and a readout logic 535. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 532, the difference detection amplifier 533, the event generation comparator 534, and the readout logic 535 is omitted here.

[0062] The event visual pixel 510 also includes a programmable memory component 536 (described as and referred to hereinafter as "latch 536"). Latch 536 is generally similar to... Figure 4 The latch 436. Therefore, for the sake of brevity, a detailed discussion of latch 536 is largely omitted here. However, unlike... Figure 4 The latch 436, Figure 5 The latch 536 is coupled to the gate of the transistor 538, which acts as a switch to respond to a deactivation signal output from the latch 536. The state selectively couples only the logarithmic amplifier 532 of the event visual pixel 510 (rather than all analog stages) to the power supply voltage V. DD Therefore, in the deactivation signal Under normal operation, which is assumed to be unasserted (e.g., in the first state or "1"), the logarithmic amplifier 532 can be coupled to the power supply voltage V via transistor 538. DD And the voltage V is supplied by the power supply. DD Power supply. On the other hand, when the event visual pixel 510 is identified as a defective pixel, the program signal, row signal, and column signal input to the latch 536 can be used to program the latch 536 and assert a deactivation signal. When asserting the deactivation signal When (e.g., when the deactivation signal) When transitioning to the second state (or "0"), the activation transistor 538 can be deactivated to only enable the logarithmic amplifier 532 of the event visual pixel 510 to operate at the power supply voltage V. DD Decoupling. This disconnects the logarithmic amplifier 532 from the power supply voltage V. DD Decoupling can interrupt the photocurrent path from photoelectric sensor 531 to differential detection amplifier 533 to prevent event detection by event vision pixel 510.

[0063] When event visual pixel 510 is identified as a defective pixel, according to... Figure 5 The method shown in the diagram to prevent the event visual pixel 510 from detecting events provides several advantages. For example, (i) noise and / or defects attributable to the event visual pixel 510 and (ii) events that would otherwise be detected by the event visual pixel 510 are prevented from being generated or output from the event visual pixel 510 and thus do not consume the available bandwidth of the event visual sensor incorporated into the event visual pixel 510. Additionally, when the event visual pixel 510 is identified as defective, the logarithmic amplifier 532 of the event visual pixel 510 is prevented from consuming current. Thus, power that would otherwise be consumed by the logarithmic amplifier 532 of the event visual pixel 510 is saved. Furthermore, when the event visual pixel 510 is identified as defective, only a subset of the analog stages of the event visual pixel 510 (here, only the logarithmic amplifier 532) is disconnected from the power supply. In other words, at least some of the other analog stages of the event visual pixel 510 (e.g., here, the difference detection amplifier 533, the event generation comparator 534, and the readout logic 535) remain powered on. Therefore, the DC bias of each of these other stages remains enabled and the event vision pixel 510 continues to consume a current amount relatively similar to that consumed when the event vision pixel 510 is not identified as defective (less than that attributable to the logarithmic amplifier 532 and the power supply voltage V). DD (The current consumption is reduced due to disconnection). Therefore, even if the defect event visual pixels across the array are affected by the power supply voltage V, the several logarithmic amplifiers will still function. DD When disconnected, the current consumed by the event vision pixels across the array remains approximately uniform. Consequently, the approximate uniformity of the IR voltage drop across the array can be maintained or observed.

[0064] Other methods for disabling defective event visual pixels and / or masking their output are of course feasible and fall within the scope of this invention. For example, a programmable memory component can be configured such that an event generation comparator can be based on a deactivation signal output from the programmable memory component. An event-generating comparator is coupled to the event visual pixel in a way that enables or disables it. Therefore, continuing this example, when an event visual pixel is identified as defective, the memory component can be programmed to assert a deactivation signal. Furthermore, the comparator is turned off and / or decoupled from the power supply voltage. This prevents the event-visual pixels from detecting the event.

[0065] As another example, a programmable memory component can be coupled to a differential detection amplifier of an event vision pixel to detect a deactivation signal output from the programmable memory component. To selectively enable the differential detection amplifier. For a specific example, refer to... Figure 6 This is a partial schematic block diagram illustrating another event visual pixel 610 configured according to various embodiments of the present invention. It should be understood that the illustrated event visual pixel 610 may be included in... Figure 1 An instance of one of the event visual pixels 110 in array 102 or another event visual pixel of the present invention. As shown, event visual pixel 610 includes elements configured to respond to incident light 650 (e.g., from an external scene and / or from a lighting source such as... Figure 1 The illumination source 120 receives the photoelectric sensor 631, which generates photocharge or photocurrent. The event vision pixel 610 further includes a logarithmic amplifier 632, a difference detection amplifier 633, an event generation comparator 634, and readout logic 635. The logarithmic amplifier 632, the event generation comparator 634, and the readout logic 635 can be roughly similar to... Figure 3 The logarithmic amplifier 632, the event generation comparator 634, and the readout logic 635 are included. Therefore, for the sake of brevity, a detailed discussion of the logarithmic amplifier 632, the event generation comparator 634, and the readout logic 635 is omitted here.

[0066] For clarity and understanding, Figure 6 The differential detection amplifier 633 is described in detail below. As shown, the differential detection amplifier 633 includes a first capacitor 622, a second capacitor 623, an amplifier 624, and a reset transistor 625. The amplifier 624 includes an input (e.g., an inverting input) and an output. The first capacitor 622 is coupled between the output of the logarithmic amplifier 632 and the input of the amplifier 624, and the second capacitor 623 is coupled between the input of the amplifier 624 and the output of the amplifier 624. The reset transistor 625 is coupled between the input of the amplifier 624 and the output of the amplifier 624.

[0067] The first capacitor 622, the second capacitor 623, the amplifier 624, and the reset transistor 625 form a filter amplifier configured to generate a filtered and amplified signal in response to the voltage output from the logarithmic amplifier 632 of the event vision pixel 610. More specifically, the filter amplifier includes a high-pass filter configured to filter out lower frequency components from the voltage received from the logarithmic amplifier 632. Therefore, the event vision pixel 610 can ignore slow or gradual changes in the photocurrent generated by the photosensitive sensor 631 in response to incident light 650, and can instead detect rapid and sudden changes in the photocurrent generated by the photosensitive sensor 631 in response to incident light 650. Additional details regarding the differential detection circuitry and the associated event vision pixel are provided in U.S. Patent Application No. 17 / 875,244, which is incorporated herein by reference in its entirety.

[0068] As shown, the event visual pixel 610 also includes a programmable memory component 636 (described as and referred to hereinafter as "latch 636"). Latch 636 is generally similar to... Figure 3 The latch 636 is used. Therefore, for the sake of brevity, a detailed discussion of the latch 636 is largely omitted here. The output of the latch 636 is coupled to the input of the inverter 639, and the output of the inverter 639 is coupled to the input of the logic gate 637. The logic gate 637 further includes (a) an input configured to receive the reset signal RST and (b) an output coupled to the gate of the reset transistor 625. The logic gate 637 in Figure 6 The description uses an OR gate as an example, but in other embodiments of the present invention, it may be another logic gate that may or may not include an inverter 639.

[0069] The reset transistor 625 of the event visual pixel 610 is arranged as a reset switch and configured to selectively couple the input of amplifier 624 to the output of amplifier 624 based on the output of logic gate 637. Therefore, the deactivation signal output from latch 636 is... Under normal operation, with the default condition of no assertion (e.g., in the first state or "1"), the output of logic gate 637 may follow the reset signal RST. Specifically, when the reset signal RST is asserted, reset transistor 625 may couple the input of amplifier 624 to the output of amplifier 624 to automatically return amplifier 624 to zero. When the reset signal RST is not asserted, reset transistor 625 may decouple the input of amplifier 624 from the output of amplifier 624.

[0070] On the other hand, when the event visual pixel 610 is identified as a defective pixel, the program signal, row signal, and column signal input to the latch 636 can be used to program the latch 636 and assert a deactivation signal. When asserting the deactivation signal When (e.g., when the deactivation signal) When transitioning to the second state (or "0"), the reset transistor 625 remains active to couple the input of amplifier 624 to the output of amplifier 624. This automatic zeroing of amplifier 624 disables or prevents event detection by the event visual pixel 610.

[0071] When event visual pixel 610 is identified as a defective pixel, according to Figure 6 The method shown in the diagram to prevent event detection by event vision pixel 610 offers several advantages. For example, (i) noise and / or defects attributable to event vision pixel 610 and (ii) events that would otherwise be detected by event vision pixel 610 are prevented from being generated or output from event vision pixel 610 and thus do not consume the available bandwidth of the event vision sensor incorporated into event vision pixel 610. Furthermore, each of the analog stages of event vision pixel 610 continues to consume quiescent power. In other words, the DC bias of each of the analog stages remains enabled and event vision pixel 610 continues to consume a current amount relatively similar to that consumed when event vision pixel 610 is not identified as defective. Therefore, even if several defective event vision pixels across the array fail to detect the event, the current consumed by the event vision pixels across the array can remain substantially uniform. Consequently, substantially uniform IR voltage drop across the array can be maintained or observed.

[0072] 3. Related methods

[0073] Figure 7 This is a flowchart illustrating a method 780 for identifying and disabling / masking defective event visual pixels according to various embodiments of the present invention. Method 780 is described as a set of steps or blocks 781 to 787. All or a subset of one or more of blocks 781 to 787 may be generated by an event visual sensor (e.g., Figure 1 The event vision sensor 100) performs various functions. For example, all or a subset of one or more of the blocks 781 to 787 may be performed by: (i) an event signal processor, (ii) an event vision pixel (e.g., a programmable memory component for the event vision pixel), (iii) a row control circuit system, (iv) a column control circuit system, (v) control logic, (vi) a lookup table, and / or (vii) a peripheral circuit system. Alternatively or additionally, all or a subset of the blocks 781 to 787 may be performed by an illumination source (such as...) Figure 1 The lighting source 120) is executed. Furthermore, any or more of boxes 781 to 787 may be performed according to the above. Figures 1 to 6 The discussion will be carried out.

[0074] Figure 7Method 780 begins at block 781, which determines the noise event triggering rate of an event visual pixel. Determining the noise event triggering rate may include determining the noise event triggering rate of the event visual pixel using one or more specified comparator thresholds in at least one event generating comparator for the event visual pixel. In these and other embodiments, determining the noise event triggering rate may include using only one event generating comparator (e.g., an upward event generating comparator or a downward event generating comparator) of the event visual pixel to determine the noise event triggering rate. In these embodiments, other event generating comparators of the event visual pixel may be disabled and / or turned off. For example, assuming that the upward event generating comparator of the event visual pixel is used to determine the noise event triggering rate of the event visual pixel, then the downward event generating comparator of the event visual pixel may be turned off or the comparator threshold in the downward event generating comparator may be set to -∞ (or some other large negative voltage value). As another example, assuming the downward event generation comparator of the event visual pixel is used to determine the noise event triggering rate of the event visual pixel, then the upward event generation comparator of the event visual pixel can be turned off, or the comparator threshold in the upward event generation comparator can be set to ∞ (or some other large positive voltage value). Alternatively, determining the event triggering rate may involve using multiple event generation comparators of the event visual pixel (e.g., both the upward and downward event generation comparators) to determine the event triggering rate, or simultaneously enabling multiple event generation comparators of the event visual pixel to detect noise events.

[0075] Figure 8 This is a flowchart illustrating a method 890 for determining the noise event priming rate of an event visual pixel according to various embodiments of the present invention. As shown, method 890 begins at block 891, which exposes the event visual pixel to constant illumination. Exposing the event visual pixel to constant illumination may include exposing the event visual pixel to DC illumination, such as DC illumination projected onto the event visual pixel using a light source. In other embodiments, exposing the event visual pixel to constant illumination may include protecting the event visual pixel from ambient lighting (e.g., in an external scene), such as by placing a cap over a lens of an imaging system incorporated into the event visual pixel. The event visual pixel may be exposed to constant illumination for each of blocks 892 to 897 discussed below.

[0076] Method 890 then proceeds to frames 892 and 893, which respectively reset the event visual pixel and subsequently detect a noise event. In some embodiments, resetting the event visual pixel may include recording the time of resetting the event visual pixel in frame 892. Detecting a noise event may include recording the time when the event visual pixel detects a noise event in frame 893.

[0077] In block 894, method 890 then calculates the arrival interval time corresponding to the event visual pixel. The arrival interval time in block 894 can be calculated as the time elapsed between the time when the event visual pixel is reset in block 892 and the time when the event visual pixel detects a noise event in block 893. In some embodiments, method 890 can limit the arrival interval time in block 894 attributed to the event visual pixel. For example, if no noise event is detected in block 893 within a threshold period after the time when the event visual pixel is reset in block 892, then method 890 can proceed to block 894 and the arrival interval time of the event visual pixel can be set to be equal to the threshold period or another predetermined (e.g., maximum) arrival interval time value.

[0078] Method 890 optionally continues with blocks 895 to 897, which calculate another arrival interval time for the event visual pixel. More specifically, method 890 may optionally then reset the event visual pixel again in block 895, subsequently detect a noise event in block 896, and calculate another arrival interval time corresponding to the event visual pixel in block 897 in a manner similar to block 894. For example, the arrival interval time in block 897 may be calculated as the elapsed time between the time the event visual pixel is reset in block 895 and the time the event visual pixel detects a noise event in block 896.

[0079] Method 890 may optionally follow block 897 by calculating one or more additional arrival intervals. For each iteration of routine 890, the number of arrival intervals calculated for the event visual pixel may be preset (e.g., predefined, predetermined). Alternatively, the number of arrival intervals calculated for the event visual pixel may depend on the amount of time allocated to perform method 890 and / or the frequency at which the event visual pixel detects noise events.

[0080] In box 898, method 890 then calculates the average arrival interval time of the event visual pixels. The average arrival interval time of the event visual pixels may depend on the arrival interval time calculated for the event visual pixels (e.g., in boxes 894 and / or 897). The average arrival interval time calculated in box 898 may be determined as... Figure 7 Method 780, frame 781, event visual pixel noise event triggering rate.

[0081] Figure 9This is a flowchart illustrating another method 900 for determining the noise event priming rate of an event visual pixel according to various embodiments of the present invention. Method 900 begins at block 901, which exposes the event visual pixel to constant illumination. Exposing the event visual pixel to constant illumination may include exposing the event visual pixel to DC illumination, such as DC illumination projected onto the event visual pixel using a light source. In other embodiments, exposing the event visual pixel to constant illumination may include protecting the event visual pixel from ambient lighting (e.g., in an external scene), such as by placing a cap over a lens of an imaging system incorporated into the event visual pixel. The event visual pixel may be exposed to constant illumination for each of blocks 902 through 905 discussed below.

[0082] In blocks 902 and 903, method 900 then resets the event visual pixel and detects a noise event, respectively. In some embodiments, resetting the event visual pixel in block 902 includes recording the time at which the event visual pixel is reset.

[0083] Shortly after detecting a noise event in block 903 (e.g., immediately), method 900 then resets the event visual pixel again in block 904. Then, in block 905, method 900 detects the noise event again. At this point, method 900 can proceed to block 906. Alternatively, method 900 may optionally repeat blocks 904 and 905. The number of times blocks 904 and 905 are repeated in each iteration of method 900 can be preset (e.g., predefined, predetermined). Additionally or alternatively, the number of times blocks 904 and 905 are repeated may depend on the amount of time allocated to perform method 900 and / or the frequency at which the event visual pixel detects a noise event.

[0084] Similar to Figure 8 Method 890, Figure 9 Method 900 may be limited to the amount of time elapsed between the time when the event visual pixel is reset and the time when the event visual pixel subsequently detects a noise event. For example, if no noise event is detected in box 903 within a threshold time period after the time when the event visual pixel is reset in box 902, then method 900 may record the noise event of the event visual pixel and proceed to box 904. If no noise event is detected in the event visual pixel within a threshold time period after the time when the event visual pixel is reset in box 904, then method 900 may additionally or alternatively perform a similar process in box 905.

[0085] In block 906, method 900 then calculates the noise event initiation rate. In some embodiments, the noise event initiation rate may be equal to the noise event rate or the number of noise events detected by the event visual pixel within the cumulative amount of time required for the event visual pixel to detect these noise events. For example, assuming the event visual pixel detects noise events in blocks 903 and 905, then the noise event rate of the event visual pixel may be calculated as the number of detected noise events (two (2) noise events) divided by the amount of time elapsed between the time the event visual pixel is reset in block 902 and the time the event visual pixel detects the noise event in block 905. The noise event rate calculated in block 906 of method 900 may be determined as Figure 7 Method 780, frame 781, event visual pixel noise event triggering rate.

[0086] Usable Figure 9 Method 900 plus or substitute Figure 8 Method 890 determines the event triggering rate of the event visual pixel. Alternatively, method 890 can be used instead of method 900. Method 890 has the advantage of being particularly robust to stacking, while method 900 is susceptible to high event rate stacking. For example, refer to Figure 9 In method 900, the noise event initiation rate calculated in block 906 can depend at least in part on the amount of time elapsed between the time when the event visual pixel detects a noise event in block 903 and the time when the event visual pixel resets in block 904. For low event rates, this elapsed time is negligible. However, for high event rates, this elapsed time can represent a large percentage of the total amount of time elapsed between the time when the event visual pixel resets in block 902 and the time when the event visual pixel detects a noise event in block 905. Therefore, for high event rates, the amount of time elapsed between the time when the event visual pixel detects a noise event in block 903 and the time when the event visual pixel resets in block 904 can have a significant impact on the noise event initiation rate calculated in block 906. This impact is referred to as "stacking". Figure 9 Compared to method 900, because... Figure 8 The event triggering rate calculated in block 898 of method 890 depends only on the arrival interval of the event visual pixel (e.g., the elapsed time between the event visual pixel reset and the subsequent detection of a noise event), so method 890 is not easily affected by stacking.

[0087] Refer again Figure 7Once the noise event initiation rate is determined in block 781, method 780 proceeds to block 782, which compares the noise event initiation rate with one or more thresholds. In some embodiments, the one or more thresholds include a predetermined initiation rate threshold. The predetermined initiation rate threshold may be set to a value at or above which the event visual pixel may be defective (e.g., unacceptably likely to record or unacceptably prone to recording noise events). Alternatively or additionally, the one or more thresholds may include or be at least partially based on the average noise event initiation rate of a set of event visual pixels that include or exclude the event visual pixel of interest. For example, in block 781, method 780 may obtain the noise event initiation rate of the event visual pixel of interest and the noise event initiation rate of each of one or more adjacent event visual pixels (e.g., event visual pixels within a threshold distance from the event visual pixel of interest). The method may then calculate an average of the noise event initiation rates and compare the noise event initiation rate of the event visual pixel of interest with the average noise event initiation rate.

[0088] In block 783, method 780 then determines whether the noise event priming rate exceeds one or more thresholds from block 782. For example, when one or more thresholds from block 782 include a predetermined priming rate threshold, method 780 may proceed to block 784 if the noise event priming rate from block 781 exceeds the predetermined priming rate threshold (block 783: Yes). As another example, when one or more thresholds from block 782 include an average noise event priming rate (e.g., a set of event visual pixels adjacent to the event visual pixel of interest), method 780 may proceed to block 784 if the noise event priming rate exceeds the average noise event priming rate by a certain amount (block 783: Yes). On the other hand, when method 780 determines that the noise event priming rate from block 781 does not exceed one or more thresholds from block 782 (block 783: No), method 780 may proceed to block 786.

[0089] In block 784, method 780 then identifies the event visual pixel as defective. Identifying an event visual pixel as defective may involve recording the address corresponding to the event visual pixel (e.g., row address and / or column address) in a lookup table. As discussed above, the lookup table can be used later (e.g., after powering on or initializing the event visual sensor) to identify the address corresponding to the defective event visual pixel. Method 780 may additionally or alternatively instruct control logic to disable the event visual pixel and / or mask its output in block 784.

[0090] In block 785, method 780 then disables the event visual pixel and / or masks its output. In some embodiments, disabling the event visual pixel may include programming the memory components of the event visual pixel (e.g., causing one or more stages of the event visual pixel to be turned off and / or decoupled from the power supply voltage, turning off one or more event generating comparators of the event visual pixel, keeping the differential detection amplifier of the event visual pixel auto-zero, and / or otherwise preventing the event visual pixel from detecting noise events). In these and other embodiments, masking the event visual pixel may include programming the memory components of the event visual pixel (e.g., causing noise events detected by the event visual pixel not to be output from the event visual pixel and / or masking read requests output by the readout logic of the event visual pixel).

[0091] In box 786, method 780 determines whether there are any additional event visual pixels to analyze that may be defective. When method 780 determines that there are additional event visual pixels to analyze (box 786: Yes), method 780 may return to box 781. Otherwise, when method 780 determines that there are no additional event visual pixels to analyze (box 786: No), method 780 may proceed to box 787 and terminate.

[0092] Although boxes 781 to 787 of method 780 are discussed and explained in a specific order, Figure 7 The method 780 described herein is not limited thereto. In other embodiments, method 780 may be performed in a different order. In these and other embodiments, any of blocks 781 to 787 of method 780 may be performed before, during, and / or after any of the other blocks 781 to 787 of method 780. For example, method 780 may expose all or a subset of event visual pixels of an array to constant illumination simultaneously and then determine the event priming rate of these event visual pixels substantially simultaneously (e.g., sequentially or simultaneously). As another example, method 780 may perform one or more of blocks 781 to 786 simultaneously on a plurality of event visual pixels. Furthermore, those skilled in the art will recognize that the described method 780 may be modified and remains within these and other embodiments of the invention. For example, in some embodiments, one or more blocks 781 to 787 of method 780 may be omitted and / or repeated. As another example, block 781 can be performed given a comparator threshold voltage (e.g., a given comparator threshold voltage for the upward event generation comparator for the event visual pixel and / or a given comparator threshold voltage for the downward event generation comparator for the event visual pixel). Alternatively or additionally, blocks 781 to 786 can be repeated for multiple comparator threshold voltages. For example, method 780 repeats block 781 while stepping through or scanning through a specific set of comparator threshold voltages. Method 780 can then determine whether the event visual pixel is defective based on one or more noise event initiation rates determined for the event visual pixel at one or more comparator voltage thresholds.

[0093] C. CONCLUSION

[0094] The above detailed description of embodiments of the present invention is not intended to be exhaustive or to limit the invention to the precise forms disclosed above. Although specific embodiments and examples of the invention have been described above for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the invention. For example, although the steps are presented in a given order above, alternative embodiments may perform the steps in a different order. Furthermore, the various embodiments described herein may be combined to provide other embodiments.

[0095] As can be understood from the foregoing, specific embodiments of the invention have been described herein for illustrative purposes, but well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention. This disclosure takes control when any material incorporated herein by reference conflicts with this disclosure. Singular or plural terms may be included, respectively, where the context permits. Furthermore, unless the use of the term “or” in a list referring to two or more items is explicitly limited to meaning only that a single item excludes other items, its use in this list should be interpreted as including (a) any single item in the list, (b) all items in the list, or (c) any combination of items in the list. Additionally, as used herein, the phrase “and / or” in “A and / or B” refers to only A, only B, and both A and B. Furthermore, the terms “comprising,” “including,” “having,” and “containing” are used throughout to mean at least including the listed features without excluding any more of the same features and / or other features of additional types. Moreover, as used herein, the phrases “based on,” “depending on,” “due to,” and “in response to” should not be construed as referring to a set of closing conditions. For example, without departing from the scope of this disclosure, an exemplary step described as “based on condition A” may be based on both condition A and condition B. In other words, as used herein, the phrase “based on” should be interpreted in the same manner as the phrase “at least partially based on” or the phrase “at least partially based on.” Furthermore, the terms “connected” and “coupled” are used interchangeably herein and refer to both direct and indirect connections or couplings. For example, where the context permits, “connected” or “coupled” to element B may mean (i) A is directly “connected” or directly “coupled” to B and / or (ii) A is indirectly “connected” or indirectly “coupled” to B.

[0096] It will also be understood from the foregoing that various modifications can be made without departing from this disclosure or the present invention. For example, those skilled in the art will understand that the various components of the present invention can be further divided into sub-components or the various components and functions of the present invention can be combined and integrated. Furthermore, certain aspects of the technology described in the context of specific embodiments may be combined or eliminated in other embodiments. Moreover, although advantages associated with certain embodiments of the present invention have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments necessarily need to exhibit such advantages to fall within the scope of the present invention. Therefore, this disclosure and related technologies may cover other embodiments not explicitly shown or described herein.

Claims

1. An event vision sensor, comprising: An event visual pixel array, arranged in rows and columns; and An event signal processor is configured to identify the defective event visual pixel based at least in part on a noise event initiation rate corresponding to the defective event visual pixel in the array, wherein the noise event initiation rate is based at least in part on an arrival interval time corresponding to the defective event visual pixel, and wherein the arrival interval time is equal to the amount of time elapsed between a first time when the defective event visual pixel is reset and a second time when the defective event visual pixel detects a noise event.

2. The event vision sensor of claim 1, wherein the defective event vision pixel includes a programmable memory component that can be used to prevent the defective event vision pixel from detecting events or to mask the output of the defective event vision pixel.

3. The event vision sensor of claim 2, wherein the programmable memory component can be used to mask the output of the defective event vision pixel such that an event detected by the defective event vision pixel is not output from the defective event vision pixel to the event signal processor.

4. The event vision sensor according to claim 2, wherein: The defect event visual pixels include A photoelectric sensor configured to generate a photocurrent in response to incident light. A photocurrent / voltage converter, coupled to the photosensor, to convert the photocurrent into a voltage. A difference detection circuit, coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter, and At least one event generating comparator is coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light; and The programmable memory component can be used to disable the photocurrent / voltage converter, the differential detection circuit, and the at least one event generating comparator, so that the defect event visual pixel cannot detect the event.

5. The event vision sensor according to claim 2, wherein: The defect event visual pixels include A photoelectric sensor configured to generate a photocurrent in response to incident light. A photocurrent / voltage converter, coupled to the photosensor, to convert the photocurrent into a voltage. A difference detection circuit, coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter, and At least one event generating comparator is coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light; The programmable memory component can be used to disable a subset of the photocurrent / voltage converter, the differential detection circuit, and the at least one event generating comparator; and The subset does not include each of the photocurrent / voltage converter, the differential detection circuit, and the at least one event generating comparator.

6. The event vision sensor according to claim 5, wherein: The subset includes only the difference detection circuit from the group comprising the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator; The difference detection circuit includes an amplifier; and The programmable memory component can be used to couple the input of the amplifier to the output of the amplifier so that the amplifier remains automatically zero, making the defect event visual pixel unable to detect the event.

7. The event vision sensor of claim 2, further comprising control logic configured to program the programmable memory component of the defective event vision pixel to disable the defective event vision pixel or mask the output of the defective event vision pixel, based at least in part on the event signal processor identifying the defective event vision pixel.

8. The event vision sensor according to claim 1, further comprising: A lookup table configured to store the addresses of visual pixels of defective events in the array, as identified by the event signal processor; and Control logic configured to program the programmable memory components of the defective event visual pixels after the event visual sensor is powered on or initialized, based at least in part on the address stored in the lookup table, such that the defective event visual pixels are deactivated or the output of each of the defective event visual pixels is masked.

9. The event vision sensor according to claim 1, wherein: The noise event initiation rate is based at least in part on the average arrival interval of the visual pixels of the defective event; and In order to identify the visual pixels of the defective event, the event signal processor is configured to compare the noise event initiation rate with a threshold.

10. The event vision sensor of claim 9, wherein the threshold represents the average noise event initiation rate corresponding to one or more event vision pixels adjacent to the defective event vision pixel in the array.

11. An event vision sensor, comprising: An event visual pixel array arranged in rows and columns, wherein each of the event visual pixels in the array comprises A photoelectric sensor configured to generate a photocurrent in response to incident light. A photocurrent / voltage converter, coupled to the photosensor, to convert the photocurrent into a voltage. A difference detection circuit, coupled to the photocurrent / voltage converter and configured to generate a signal in response to a difference detected in the voltage received from the photocurrent / voltage converter. At least one event generating comparator is coupled to the difference detection circuit and configured to compare the signal received from the difference detection circuit with at least one threshold to detect an event indicated in the incident light. Memory components that can be programmed to: A subset of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator of the event visual pixel is disabled, such that the event visual pixel cannot detect events, wherein the subset does not include each of the photocurrent / voltage converter, the difference detection circuit, and the at least one event generating comparator of the event visual pixel; or The output of the visual pixels of the event is masked; and An event signal processor is configured to identify the defective event visual pixel at least in part based on the noise event initiation rate corresponding to the defective event visual pixel in the array.

12. The event vision sensor of claim 11, wherein each of the noise event initiation rates is at least partially based on the arrival interval time of a corresponding entity in the defective event vision pixel, and wherein the arrival interval time represents the amount of time elapsed between a first time at which the corresponding entity in the defective event vision pixel resets and a second time at which the corresponding entity in the defective event vision pixel detects a noise event.

13. The event vision sensor of claim 11, wherein each of the noise event initiation rates is at least partially based on the noise event rate corresponding to the corresponding entity in the defective event vision pixel, and wherein the noise event rate represents the number of noise events detected by the corresponding entity in the defective event vision pixel divided by the cumulative amount of time required for the corresponding entity in the defective event vision pixel to detect the noise event since a reset.

14. The event vision sensor of claim 11, further comprising: A lookup table configured to store the addresses of visual pixels of events identified as defective by the event signal processor; and Control logic configured to program the memory components of the visual pixels of the defective event based at least in part on the address stored in the lookup table.

15. The event vision sensor of claim 11, further comprising control logic configured to program a memory component of the defective event vision pixel identified by the event signal processor.

16. A method comprising: The event visual pixel is identified as defective based at least in part on the noise event triggering rate of the event visual pixel corresponding to the event visual sensor, wherein the noise event triggering rate is at least in part based on the arrival interval time associated with the event visual pixel, and wherein the arrival interval time represents the amount of time elapsed between the event visual pixel being reset and the subsequent detection of a noise event. and In response to identifying the event visual pixel as defective, the event visual pixel is prevented from outputting event data.

17. The method of claim 16, further comprising determining the noise event initiation rate corresponding to the visual pixel of the defect event, wherein determining the noise event initiation rate comprises: When the event visual pixel is exposed to DC illumination light Determine the first arrival interval time associated with the visual pixel of the event. Determine the second arrival interval time associated with the visual pixel of the event; and The average arrival interval time is determined at least in part based on the first arrival interval time and the second arrival interval time.

18. The method of claim 16, wherein identifying the event visual pixel as defective comprises (a) comparing the noise event initiation rate with a preset threshold and (b) determining that the noise event initiation rate exceeds the preset threshold.

19. The method of claim 16, wherein identifying the event visual pixel as defective comprises (a) determining an average noise event triggering rate based at least in part on a noise event triggering rate associated with one or more event visual pixels adjacent to the event visual pixel in the array of the event visual sensor, (b) comparing the noise event triggering rate with the average noise event triggering rate, and (c) determining that the noise event triggering rate exceeds the average noise event triggering rate by a greater than a threshold amount.

20. The method of claim 16, wherein preventing the event visual pixel from outputting the event data comprises programming a memory component contained within the event visual pixel such that (i) the output of the event visual pixel is masked or (ii) the event visual pixel is prevented from detecting an event.

21. The method of claim 20, wherein programming the memory component comprises programming the memory component such that the photocurrent / voltage converter of the event visual pixel, the difference detection circuit of the event visual pixel, at least one event generating comparator of the event visual pixel, or any combination thereof, is deactivated.

Images