Event visual sensor with event data compression, including event visual sensor with event data compression in pixels, and related systems, apparatuses, and methods

Abstract

Disclosed herein are event vision sensors utilizing event data compression, including event vision sensors utilizing pixel-level event data compression, and related systems, devices, and methods. In one embodiment, an event vision sensor includes a plurality of event vision pixels. Each event vision pixel of the plurality of event vision pixels is configured to generate event data based on events indicated in incident light received from an external scene, and includes a compression circuit configured to compress the event data before the event data is read out from the event vision pixel. Each compression circuit can include a temporal aggregation circuit configured to track a number of the events detected by the corresponding event vision pixel within a defined timing window. The compressed event data can be read out from the event vision sensor and used to generate a pseudo-frame. The event vision sensor can optionally perform frame-level compression on the pseudo-frame.

Description

Technical Field

[0001] This disclosure generally relates to event vision sensors. For example, several embodiments of the present technology relate to event vision sensors employing on-chip event data compression (e.g., in-pixel event data compression). Background Technology

[0002] Image sensors have become ubiquitous and are now widely used in digital cameras, cellular phones, security cameras, and medical, automotive, and other applications. As image sensors are integrated into a wider range of electronic devices, it is expected that their functionality, performance, and so on can be enhanced in as many ways as possible (e.g., resolution, power consumption, dynamic range, etc.) through both device architecture design and image acquisition and processing.

[0003] A typical image sensor operates in response to image light incident on it from an external scene. The image sensor comprises an array of pixels with photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and immediately generate an image charge upon absorption. The image charge generated by the pixel light can be measured as an analog output image signal on the column lines, which varies as a function of the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out as an analog image signal from the column lines and converted into a digital value to provide information representing the external scene. Summary of the Invention

[0004] One aspect of this disclosure relates to an event vision pixel comprising: a photoelectric sensor configured to generate a photocurrent in response to incident light received from an external scene; a photocurrent-to-voltage converter coupled to the photoelectric sensor to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to a difference detected over time in the voltage received from the photocurrent-to-voltage converter; an event generation comparator coupled to the difference detection circuit to detect an event occurring within the external scene based on a comparison of the filtered and amplified signal with one or more thresholds; and a time aggregation circuit coupled to the event generation comparator to track the aggregated number of events detected by the event generation comparator within a defined timing window.

[0005] Another aspect of this disclosure relates to an event vision sensor comprising: a plurality of event vision pixels, wherein: each of the plurality of event vision pixels is configured to generate event data based at least in part on an event indicated in incident light received from an external scene, and each of the plurality of event vision pixels includes compression circuitry configured to compress the event data before the event data is read out from the event vision pixel.

[0006] Another aspect of this disclosure relates to a method of operating an event vision sensor comprising one or more event vision pixels, the method comprising: for each of the one or more event vision pixels, detecting an event indicated in incident light received from an external scene within a timing window, and tracking the number of events detected within the timing window. Attached Figure Description

[0007] The following description, with reference to the figures, outlines non-limiting and non-exhaustive embodiments of the present technology, wherein similar or analogous element symbols are used throughout to refer to similar or analogous components unless otherwise specified.

[0008] Figure 1 This is a partial schematic block diagram of an event vision sensor configured according to various embodiments of the present technology.

[0009] Figure 2 This is a partial schematic block diagram illustrating the configuration of event visual pixels according to various embodiments of the present technology.

[0010] Figure 3 This is a partial schematic block diagram of a time aggregation circuit configured according to various embodiments of the present technology.

[0011] Figure 4 This is a diagram illustrating a method for compressing event data according to various embodiments of the present technology.

[0012] Figure 5 This is a partial schematic block diagram of another time aggregation circuit configured according to various embodiments of the present technology.

[0013] Figure 6 This is a diagram illustrating various embodiments of the present technology. Figure 5 The signal diagram shows the method of operating the time aggregation circuit.

[0014] Figure 7 This is a partial schematic block diagram of another time aggregation circuit configured according to various embodiments of the present technology.

[0015] Figure 8 This is a diagram illustrating various embodiments of the present technology. Figure 7The signal diagram shows the method of operating the time aggregation circuit.

[0016] Figure 9 This is a partial schematic block diagram of another time aggregation circuit configured according to various embodiments of the present technology.

[0017] Figure 10 This is a diagram illustrating another method of event data compression according to various embodiments of the present technology.

[0018] Figure 11 This is a partial schematic block diagram of an event signal processor configured according to various embodiments of the present technology.

[0019] Figure 12 This is a flowchart illustrating a method for operating an event vision sensor according to various embodiments of the present technology.

[0020] Those skilled in the art will understand that the elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be enlarged relative to other elements to aid in understanding various aspects of the art. Furthermore, common but well-known elements or methods that are useful or necessary in commercially feasible embodiments are generally not depicted in the figures or described in detail below to avoid unnecessarily obscuring the description of various aspects of the art. Detailed Implementation

[0021] This disclosure relates to event vision sensors. For example, several embodiments of the technology relate to event vision sensors employing on-chip (e.g., in-pixel) and / or frame-level event data compression. Specific details are set forth in the following description to provide a thorough understanding of various aspects of the technology. However, those skilled in the art will recognize that the systems, apparatuses, and techniques described herein can be practiced without the one or more of the specific details set forth herein, or using other methods, components, materials, etc.

[0022] The references to "example" or "embodiment" in this specification mean that a particular feature, structure, or characteristic described in connection with the example or embodiment is included in at least one example or embodiment of the present technology. Therefore, the phrases "for example," "as an example," or "embodiment" as used herein do not necessarily all refer to the same example or embodiment, and are not necessarily limited to the specific example or embodiment discussed. Furthermore, the features, structures, or characteristics of the present technology described herein can be combined in any suitable manner to provide further examples or embodiments of the present technology.

[0023] For ease of description, spatial relative terms (e.g., “below,” “under,” “above,” “below,” “above,” “upper,” “top,” “bottom,” “left,” “right,” “center,” “middle,” etc.) are used herein to describe the relationship of an element or feature relative to one or more other elements or features as illustrated in the figures. It should be understood that, in addition to the orientations described in the figures, the spatial relative terms are intended to cover different orientations of the device or system during use or operation. For example, if the device or system illustrated in the figures is rotated, turned, or flipped about a horizontal axis, then an element or feature described as “below,” “under,” or “below” one or more other elements or features may be oriented “above” one or more other elements or features. Therefore, the exemplary terms “below” and “below” are non-limiting and may encompass both above and below orientations. Alternatively, the device or system may be oriented in other ways illustrated in the figures (e.g., rotated 90 degrees about a vertical axis, or otherwise), and the spatial relative descriptors used herein shall be interpreted accordingly. In addition, it should be understood that when an element is referred to as being "between" two other elements, it can be the only element between the two other elements, or there may be one or more intermediate elements.

[0024] Several terms are used throughout this specification. These terms will be given their ordinary meaning in the field of their respective domains, unless otherwise specifically defined herein or the context in which they are used will clearly imply otherwise. It should be noted that in this document, component names and symbols are used interchangeably (e.g., Si and silicon); however, they have the same meaning.

[0025] A. Overview

[0026] Active pixel sensors (e.g., CMOS imaging systems) typically employ an array of active pixels with globally defined integration times. Therefore, active pixels in an active pixel sensor typically have the same integration time, and each pixel in the array is typically converted into a digital signal regardless of its content (e.g., regardless of whether the external scene captured by the pixel has changed since the last readout). In other words, image data generated by active pixels in, for example, a CMOS imager, is read out in frames of a known size regardless of whether events occur in the external scene.

[0027] In contrast, when a pixel captures a change in the external scene (e.g., an event), the event vision sensor (e.g., an event-driven sensor or a dynamic vision sensor) reads out the pixel and / or converts the corresponding pixel signal into a digital signal. In other words, pixels of the event vision sensor that do not detect a change in the external scene are not read out, and / or the pixel signals corresponding to such pixels are not converted into digital signals. Therefore, each pixel of the event vision sensor can operate independently of the other pixels of the event vision sensor, and only the pixels that detect a change in the external scene need to be read out, and / or their corresponding pixel signals need to be converted into digital signals or recorded (thus saving power). In this way, the event vision sensor does not need to record the entire regular image, and therefore does not have the burden of capturing and recording all the highly redundant information of a normal image frame by frame. As a result, the event vision sensor can be used to detect movement or motion in the external scene (e.g., as opposed to capturing / reading out an entire frame of an image or video) while enabling (i) the use of low data rates and (ii) the achievement of ultra-high frame rates or speed capabilities.

[0028] Even so, when an event vision sensor records events detected in an external scene, it typically records the location where each event was detected (e.g., the xy coordinates of the event vision pixel in an array of such pixels), the polarity of the photocurrent change for that event (e.g., brighter or darker), and / or the timing of when the event occurred or was detected. Therefore, for a megapixel array of event vision pixels, the event vision sensor typically records approximately 25 bits of information for each individual event detected by the megapixel array's event vision pixels (e.g., ten bits for recording the x-position of the corresponding event vision pixel in the megapixel array, ten bits for recording the y-position of the corresponding event vision pixel, one bit for recording the polarity change corresponding to the event, and four bits for recording a timestamp indicating when the event was detected). When events are sparse and only a few events are detected immediately or within a short timeframe by the array's event vision pixels, recording 25 bits of information for each individual event is certainly more efficient than recording an entire frame of an image or video in which only a few event vision pixels in the array have detected the event. However, compared to pixel sensors, when a large number of event vision pixels detect an event immediately or several events are detected within a short period of time, recording 25 bits of information for each individual event quickly reduces the relative efficiency of event vision sensors.

[0029] Furthermore, event vision sensors rarely detect and register isolated events corresponding to legitimate activities in the external scene monitored by the event vision sensor. More often, that isolated event corresponds to noise. Instead, activities in the external scene typically contain some spatial or temporal redundancy. For example, when an individual moves across the external scene monitored by the event vision sensor, the same individual is detected in events registered by several different groups of event vision pixels across the array of such pixels in the event vision sensor. Therefore, spatial redundancy often exists in the information captured by the event vision sensor when an individual moves across the external scene. As another example, when the event vision sensor monitors an external scene containing a flashing yellow traffic light, the same group of event vision pixels in the event vision sensor will detect the event regardless of when the yellow light changes (e.g., from on to off and / or from off to on). In other words, the same group of event vision pixels will detect multiple instances of the same activity over time. Thus, some temporal redundancy exists in the information captured by the event vision pixels when the traffic light changes within a short time frame.

[0030] Given the problems discussed above, this technology provides an on-chip compression solution for reducing the amount of information used to record events detected by an event vision sensor. For example, several embodiments of this technology relate to an event vision sensor comprising event vision pixels with time aggregation circuitry capable of performing pixel-level compression of event information (e.g., time-dependent aggregation). As a specific example, several event vision pixels of this technology include counters configured to count the number of events detected by the corresponding event vision pixel within a defined time period (e.g., the number of UP events, the number of DOWN events, or the difference between the number of UP events and the number of DOWN events). The counts maintained by the event vision pixels can then be read out and combined to form one or more pseudo-frames corresponding to the defined time period. In some embodiments, the event vision sensor may then optionally perform frame-level compression on the pseudo-frames (e.g., to further reduce the amount of information used to record events detected by the event vision pixels within the defined time period). Therefore, instead of reading events from event vision pixels and recording them individually, the event vision sensor configured according to various embodiments of the present technology is configured to: (i) encode events detected by the same event vision pixel and occurring within the same timing window into a group using in-pixel compression circuitry; and (ii) read out the count corresponding to the group as event data that can be used to form pseudo-frames. As a result, the event vision sensor configured according to various embodiments of the present technology can read and record event data from event vision pixels using fewer bits than event vision sensors lacking on-chip compression solutions typically do.

[0031] In other words, this technology provides an on-chip compression solution (e.g., pixel-level compression and / or frame-level compression) for reducing the amount of information required to record events. Consequently, compared to event vision sensors lacking an on-chip compression solution, the amount of memory used to record such events is reduced, which can result in longer recording times for a fixed amount of memory, as a larger amount of event data can be stored in this type of memory before it becomes full. Furthermore, compared to event vision sensors lacking an on-chip compression solution, and / or to imaging systems that perform compression outside the chip (e.g., outside the event vision sensor of the imaging system), this on-chip compression solution provided by this technology promises to reduce latency, enabling the use of even lower data rates and utilizing less power.

[0032] B. The selected practical application of event visual sensors and related systems, devices and methods utilizing event data compression. Regulations

[0033] Figure 1 This is a partial schematic block diagram of an event vision sensor 100 configured according to various embodiments of the present technology. As shown, the event vision sensor 100 includes an array 102 of event vision pixels 110, a row control circuitry system 104, a column control circuitry system 106, an event signal processor 108, and a transmitter 116. In some embodiments, the event vision sensor 100 may additionally include a memory 112. As an example, the memory 112 is located in... Figure 1 The memory is shown as FIFO buffer 112. In other embodiments, memory 112 may be another suitable type of buffer or memory, and / or may be omitted or located at other locations in the event vision sensor 100 (e.g., within the event signal processor 108 or transmitter 116, or after transmitter 116).

[0034] Array 102 includes multiple event vision pixel circuits 110 arranged in rows and columns. Figure 1(Rows and columns not shown). As discussed in more detail below, event visual pixels 110 are configured to capture changes (e.g., events) in the external scene. To this end, each event visual pixel 110 may include: (i) a photoelectric sensor, such as a photodiode, configured to generate a photocharge or photocurrent in response to incident light received from the external scene; (ii) a photocurrent-to-voltage converter coupled to the photoelectric sensor to convert the photocurrent generated by the photoelectric sensor into a voltage; and (iii) a filter amplifier coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to a voltage received from the photocurrent-to-voltage converter. The event visual pixel may further include a threshold comparison circuit or stage to generate and receive a handshake signal in response to an event asynchronously detected in the incident light received from the external scene. Alternatively, the threshold comparison circuit may be included in a peripheral or external circuitry of the event visual pixels of array 102 (e.g., in event signal processor 108), such as within a column readout circuitry. The event vision pixel configured according to this technology may further include a time aggregation circuit or stage, which helps to encode events that occur or are detected within a specified time window together, thereby compressing the total amount of information required to record the events detected by the event vision pixel.

[0035] Figure 1 The row control circuitry 104 and column control circuitry 106 are used to control the rows and columns of event visual pixels 110 in the array 102, respectively. For example, the row control circuitry 104 and / or column control circuitry 106 may be configured to reset specific (e.g., individual rows) event visual pixels 110 of the array 102, and / or read out (e.g., individual rows) event visual pixels 110 from the array 102 (e.g., along the corresponding column bit lines connected to the event visual pixels 110).

[0036] Event data read from the event visual pixels 110 of array 102 can be transmitted to the event signal processor 108 of the event visual sensor 100 for processing. As discussed in more detail below, the processing performed by the event signal processor 108 may optionally include compression of pseudoframes of the event data read from the event visual pixels 110 of array 102. The event data processed by the event signal processor 108 can be provided to the transmitter 116 for transmitting the event data out of the event visual sensor 100, for example, to a receiver (not shown) of a corresponding imaging system. Alternatively, all or a subset of the event data can be stored in memory 112 (e.g., before or after being provided to the event signal processor 108 and / or the transmitter 116).

[0037] Figure 2This is a partial schematic block diagram illustrating examples of event visual pixel 210 configured according to various embodiments of the present technology. It should be understood that the illustrated event visual pixel circuitry 210 may be included within... Figure 1 This is an example of one of the event vision pixel circuits 110 in array 102, or another example of an event vision pixel circuit of the present technology. As shown, event vision pixel 210 includes a photodiode 231 configured to generate photocharge or photocurrent in response to incident light 250 received from an external scene. Photodiode 231 is coupled to a logarithmic amplifier 232, which is configured to convert the photocurrent generated by photodiode 231 into a voltage. In various examples, logarithmic amplifier 232 is configured to generate a voltage by transducing the instantaneous photocurrent received from photodiode 231. Difference detection amplifier 233 is coupled to logarithmic amplifier 232 to generate a filtered and amplified signal in response to a difference detected in the voltage received from logarithmic amplifier 232. In one example, difference detection amplifier 233 is configured to compare the instantaneous logarithmic intensity of the voltage output of logarithmic amplifier 232 with a reference level based on a reset condition or a last event.

[0038] An event generation comparator 234 is coupled to a difference detection amplifier 233 to compare a filtered and amplified signal received from the difference detection amplifier 233 with a threshold, thereby asynchronously detecting an event occurring in the external scene in response to incident light 250. In one example, the event generation comparator 234 is configured to determine whether the signal difference is significant enough to trigger an event. In the illustrated embodiment, the event generation comparator 234 includes a first comparator 234a configured to detect whether the signal difference corresponds to an "UP" event (e.g., a change in light intensity incident on photodiode 231 from darker to brighter and greater than a threshold). The event generation comparator 234 further includes a second comparator 234b configured to detect whether the signal difference corresponds to a "DOWN" event (e.g., a change in light intensity incident on photodiode 231 from brighter to darker and greater than a threshold).

[0039] Figure 2 The event visual pixel 210 further includes a time aggregation circuit or stage 235 (also referred to herein as "pixel compression circuit" or "compression circuit") coupled to the output of the event generation comparator 234. See below for reference. Figures 3 to 10As discussed in more detail, the time aggregation circuit 235 may include one or more counters configured to count events (UP events and / or DOWN events) detected by the event generation comparator 234 during a defined timing window. At the end of the timing window, the count maintained by the one or more counters may be read as event data from the event visual pixel 210 onto column or bit line 238. The event data may include: (a) a count representing the number of UP events detected by the corresponding event visual pixel during the timing window; (b) a count representing the number of DOWN events detected by the corresponding event visual pixel during the timing window; and / or (c) a count representing the difference between the number of UP events detected by the corresponding event visual pixel during the timing window and the number of DOWN events detected by the corresponding event visual pixel during the timing window.

[0040] See below for reference Figure 4 and 10As discussed in more detail, the time aggregation circuit 235 facilitates the encoding of multiple events occurring within the same timing window together, rather than encoding them individually. For example, in the case of a megapixel array, instead of individually reading out the events detected by the event visual pixel 210 and recording each of those events individually using approximately 25 bits of information (e.g., ten bits for recording the x-position of the event visual pixel 210 in the megapixel array, ten bits for recording the y-position of the event visual pixel 210 in the megapixel array, one bit for recording the polarity change corresponding to each individual event, and four bits for recording the timestamp indicating when the event visual pixel 210 detected each individual event), the events detected by the event visual pixel 210 during the same timing window can be read out and recorded as a group using a smaller number of bits. More specifically, at the end or after a given timing window, counts representing the following can be read from event visual pixel 210 as event data: (i) the number of UP events detected by event visual pixel 210 during the timing window; (ii) the number of DOWN events detected by event visual pixel 210; and / or (iii) the difference between the number of UP events detected during the timing window and the number of DOWN events detected during the timing window. The event data can then be recorded using: (a) a single instance of ten bits reflecting the x-position of event visual pixel 210 in the megapixel array; (b) a single instance of ten bits reflecting the y-position of event visual pixel 210 in the megapixel array; (c) a specified number of bits (e.g., two to four bits) reflecting the count value; and (d) a specified number of bits (e.g., four bits) identifying the corresponding timing window (e.g., via a timestamp reflecting the start or end of the timing window, a timing window sequence identifier, frame count, etc.). In other words, the time aggregation circuit 235 compresses the amount of information needed to record the events detected by the event visual pixel 210 by using a single x-position data entry, a single y-position data entry, and a single timestamp data entry for all events detected by the event visual pixel 210 within the same timing window.

[0041] Refer again Figure 2 The event visual pixel 210 optionally includes a scanner and readout logic 236 coupled to the output of the event generating comparator 234 via logic gate 237. In the illustrated embodiment, logic gate 237 is an "OR" gate and is configured to trigger a latch in the scanner and readout logic 236 regardless of when the event generating comparator 234 detects an event (e.g., an UP event or a DOWN event). Therefore, triggering the latch indicates that the event visual pixel 210 has detected at least one event during a given timing window. Although... Figure 2Not shown in the diagram, but the scanner and readout logic 236 can be coupled to the row control circuitry of the corresponding event visual sensor (e.g., Figure 1 The row control circuitry system 104 can be used to request the readout of the event visual pixel 210 regardless of when the latch is triggered. In this way, the scanner and readout logic 236 and logic gate 237 can be used to ensure that the event visual pixel 210 is readout only if the event visual pixel 210 has detected at least one event within a given timing window. In other words, the scanner and readout logic 236 and logic gate 237 can be used to ensure that if the count maintained by the event visual pixel 210 indicates that the event visual pixel 210 has not detected at least one event during the timing window, those counts are not readout from the event visual pixel 210.

[0042] In other embodiments, the scanner and readout logic 236 of the event visual pixel 210 may include a first latch coupled to the output of the first comparator 234a and a second latch, different from the first latch and coupled to the output of the second comparator 234b. In these embodiments, logic gate 237 may be omitted. The first latch may be triggered when the event visual pixel 210 detects at least one UP event during a given timing window, and the second latch may be triggered when the event visual pixel 210 detects at least one DOWN event during a given timing window. Including separate latches for UP and DOWN events in the scanner and readout logic 236 provides greater granularity or control over the readout of different event counts maintained by the time aggregation circuit 235. For example, using a separate latch can help read out the UP event count only when the event visual pixel 210 detects at least one UP event during a given timing window (e.g., regardless of whether the event visual pixel 210 detects a DOWN event during a given timing window), and read out the DOWN event count only when the event visual pixel 210 detects at least one DOWN event during a given timing window (e.g., regardless of whether the event visual pixel 210 detects an UP event during a given timing window).

[0043] For clarity and understanding, the following will be provided Figure 2The following is a brief overview of the operation of the event visual pixel 210 illustrated in the diagram. The event visual pixel 210 functions such that when an event occurs in the external scene, that event is indicated by a rapid or sudden change in intensity or brightness in the incident light 250 received by the photodiode 231. In other words, if the external scene is static and no event occurs, the brightness of the incident light 250 remains substantially unchanged. Thus, the photocurrent generated by the photodiode 231 remains substantially constant. However, if an event occurs in the external scene (e.g., movement, change in illumination, albedo, emissivity, etc.), then the event is indicated by an asynchronous rapid or sudden change in the brightness of the incident light 250. The change in brightness can be from darker to brighter or from brighter to darker. Therefore, there is an asynchronous change, or δ, in the photocurrent generated by the photodiode 231. The change or δ in the photocurrent is converted into a voltage by the logarithmic amplifier 232, filtered and amplified by the differential detection amplifier 233, detected by the event generation comparator 234, and counted by the time aggregation circuit 235. When an event is the first event to occur within a given timing window, the event can trigger the scanner and the latch in the readout logic 236 via logic gate 237, thereby notifying the row control circuitry (not shown) that the event visual pixel 210 has detected at least one event within the given timing window and requesting the readout of the event visual pixel 210 (e.g., at or after the end of the timing window). This process can be repeated for each event occurring within the given timing window.

[0044] At or after the end of a given timing window, and assuming that event visual pixel 210 has detected at least one event within the given timing window (e.g., as indicated by a triggering of a latch in the scanner and readout logic 236), event data can be read from event visual pixel 210 sequentially or simultaneously, along with event data generated by other event visual pixels in the array containing event visual pixel 210. Event data detected by the event visual pixels of the array can be read from the array asynchronously and / or in an order different from the order in which the event data was generated. Digital timestamps, sequence identifiers, frame counts, etc., associated with each timing window help ensure that events detected by different event visual pixels but corresponding to the same timing window are related / correlated with each other (e.g., for storage or reconstruction). On the other hand, when event visual pixel 210 does not detect at least one event within a given timing window (e.g., as indicated by an untriggered latch in the scanner and readout logic 236), event data is not read from event visual pixel 210 within the given timing window.

[0045] Therefore, it should be understood that circuitry containing event-visual pixels (e.g., Figure 2 The event vision pixel circuit 210) of the event vision sensor (e.g., Figure 1The event vision sensor 110 does not need to record the entire normal image, and therefore is not burdened by having to capture and record all the highly redundant information of the normal image frame-to-frame. Instead, in various embodiments, the event vision sensor may only record events and / or event counts. For example, Figure 1 The event vision sensor 100 can record the location where an event is detected (e.g., Figure 1 The event vision sensor 100 can be used to detect movement or motion in an external scene (e.g., as opposed to capturing / reading out entire frames of images or videos) and can facilitate group-based encoding of events, thereby compressing the total amount of information required to record events detected by the event vision pixels 110 of the array 102, enabling the use of lower data rates and enabling ultra-high frame rates or speed capabilities. (The xy coordinates of the event vision pixels 110 in the array 102), the count of UP and / or DOWN events detected during a given timing window (or the difference between such counts), and / or timing or identifiers corresponding to the given timing window. In other words, the event vision sensor 100 can be used to detect movement or motion in an external scene (e.g., as opposed to capturing / reading out entire frames of images or videos) and can facilitate group-based encoding of events, thereby compressing the total amount of information required to record events detected by the event vision pixels 110 of the array 102, enabling the use of lower data rates and enabling ultra-high frame rates or speed capabilities.

[0046] Figure 3 This is a partial schematic block diagram of a time aggregation circuit 335 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 335 may be included in... Figure 2 This is an example of a time aggregation circuit 235 in the event visual pixel 210, or another example of a time aggregation circuit of the present technology. As shown, the time aggregation circuit 335 includes a first counter 335a and a second counter 335b. In some embodiments, the first counter 335a is a first ripple counter, and / or the second counter 335b is a second ripple counter. The first counter 335a is configured to: (a) receive a signal UP from the event generation comparator of the corresponding event visual pixel, indicating when the corresponding event visual pixel detects an UP event; and (b) count the number of times the signal UP is asserted during a timing window. Similarly, the second counter 335b is configured to: (a) receive a signal DOWN from the event generation comparator of the corresponding event visual pixel, indicating when the corresponding event visual pixel detects a DOWN event; and (b) count the number of times the signal DOWN is asserted during a timing window.

[0047] For clarity and understanding, the following references are provided. Figure 2 and 3The operation of the time aggregation circuit 335 will be discussed below. A first counter 335a may be configured to receive the output (signal UP) of the first comparator 234a in the event generation comparator 234 of the event visual pixel 210, and a second counter 335b may be configured to receive the output (signal DOWN) of the second comparator 234b in the event generation comparator 234. Continuing this example, the first counter 335a and the second counter 335b may be reset at or before the start of a timing window. Whenever the first comparator 234a detects an UP event, the first comparator 234a may assert or pulse the signal UP. The first counter 335a may then update (e.g., increment) a counter value representing the number of times the first comparator 234a asserts or pulses the signal UP during the timing window. The counter value maintained by the first counter 335a is referred to herein as the UP event count. Similarly, whenever the second comparator 234b detects a DOWN event, it can assert or pulse the DOWN signal. The second counter 335b can then update (e.g., increment) a counter value representing the number of times the second comparator 234b asserts or pulses the DOWN signal during a timing window. The counter value maintained by the second counter 335b is referred to herein as the DOWN event count.

[0048] Assuming that event visual pixel 210 detects at least one event (UP or DOWN) during the timing window, the latch in the scanner and readout logic 236 of event visual pixel 210 will be triggered via logic gate 237, thereby signaling the corresponding event visual sensor (e.g., the corresponding row control circuitry) to read out event visual pixel 210. Therefore, at or after the end of the timing window, the event visual sensor can read out the UP event count and the DOWN event count onto column line 338. The UP event count and the DOWN event count can be read out from event visual pixel 210 in any order relative to each other and / or relative to other event counts maintained by other event visual pixels of the event visual sensor. On the other hand, when event visual pixel 210 does not detect at least one event (UP or DOWN) during the timing window, the latch in the scanner and readout logic 236 will not be triggered via logic gate 237. Thus, the event count maintained by the first counter 335a and the second counter 335b within the corresponding timing window will not be read onto the column line 338 at or after the timing window ends. Regardless of whether the event visual pixel 210 detects at least one event during the timing window, the event count maintained by the first counter 335a and the second counter 335b can be reset at or before the start of the next timing window.

[0049] Figure 4This is a diagram illustrating an overview of event data compression that can be performed using a time aggregation circuit of event visual pixels configured according to various embodiments of the present technology. Figure 4 A plot 440 includes UP and DOWN events detected by the event visual pixel during a timing window 441 between time t1 and time t2. More specifically, plot 440 illustrates that the event visual pixel detected eight (8) UP events and ten (10) DOWN events during the timing window 441 in the order shown. As discussed above, a first counter of the event visual pixel (e.g., Figure 3 The first counter 335a) counts the UP events, and a second counter (e.g., Figure 3 The second counter 335b) counts the DOWN events. The UP event count maintained by the first counter and the DOWN event count maintained by the second counter can be read from the event visual pixel at or after the timing window 441 (e.g., at or after time t2). Note that the UP event count and DOWN event count only reflect the number of UP events and / or DOWN events detected by the event visual pixel during the timing window 441, respectively. In other words, the count only reflects the aggregate number of UP events and DOWN events detected by the event visual pixel within the timing window 441 between time t1 and time t2, respectively. Therefore, during the timing window 441, the counter values ​​maintained by the counters do not record, track, or otherwise reflect the accurate timing of each individual event detected by the event visual pixel. Therefore, this pixel-wise compression of event data performed by the event visual pixel can be considered a lossy form of compression in which the timing information of individual events is lost.

[0050] In some embodiments, the UP event count and / or DOWN event count read from the event visual pixel can be combined with other event counts read from other event visual pixels of the event visual sensor to form a pseudo-frame. For example, such as Figure 4 As shown, the UP event count corresponding to the timing window 441 of drawing 440 can be combined with other UP event counts read from other event visual pixels within the same timing window 441 to form a first pseudo-frame 450a. Similarly, the DOWN event count corresponding to the timing window 441 of drawing 440 can be combined with other DOWN event counts read from other event visual pixels within the same timing window 441 to form a second pseudo-frame 450b different from the first pseudo-frame 450a. Alternatively, both the UP event count and the DOWN event count corresponding to the timing window 441 of drawing 440 can be combined with other UP event counts and DOWN event counts read from other event visual pixels within the same timing window 441 in a single pseudo-frame. In this case, refer to the following... Figure 11 As discussed in more detail, the event vision sensor may optionally perform frame-level compression on the first pseudo-frame 450a and / or the second pseudo-frame 450b.

[0051] Frames containing event counts read from the event visual pixels of the array within different timing windows are referred to herein as "pseudo-frames" because the event data contained in each pseudo-frame represents the number of events detected by the event visual pixels within the defined timing window. This event data is proportional to changes in contrast or information in the external scene. On the other hand, image data frames are conventionally associated with exposure time, and the image data contained in such frames represents the amount of photons / electrons generated by the light incident on the photoelectric sensor of each pixel during the exposure time.

[0052] Refer again Figure 4 Figure 445 illustrates the reconstruction of event data from pseudoframes 450a and 450b. More specifically, during reconstruction, event data corresponding to UP events detected by the event visual pixels during timing window 441 can be reconstructed using the first pseudoframe 450a, and event data corresponding to DOWN events detected by the event visual pixels during timing window 441 can be reconstructed using the second pseudoframe 450b. Because timing information about when each of the individual events occurred precisely during timing window 441 is lost during pixel compression of the event data, assumptions are made during reconstruction about when each of the events occurred within timing window 441. In the illustrated example, it is assumed that the event visual pixels detect eight UP events at uniform intervals across timing window 441, and ten DOWN events at uniform intervals across timing window 441. Therefore, during reconstruction, the eight UP events can be evenly distributed across timing window 441, and the ten DOWN events can be evenly distributed across timing window 441.

[0053] Other assumptions regarding the distribution of UP and / or DOWN events across the time window are of course possible and within the scope of this technique. For example, it can be assumed that the event visual pixel detects the UP and / or DOWN events exponentially across the timing window 441. Furthermore, although the same assumption is made for both UP and DOWN events across the time window 441 in the illustrated example, different assumptions can be made for the UP and DOWN events during reconstruction. Alternatively, multiple assumptions can be made for the UP and DOWN events during reconstruction. For example, it can be assumed that the DOWN event is detected by the event visual pixel at uniform intervals during the first half of the timing window, and then by the event visual pixel exponentially during the second half of the timing window. Furthermore, for the same pseudoframe, the assumptions made during the reconstruction of event data corresponding to one event visual pixel can, but need not, be the same as the assumptions made during the reconstruction of event data corresponding to another event visual pixel. Furthermore, for different pseudoframes corresponding to the same visual pixel of the same event (e.g., different UP event pseudoframes or different DOWN event pseudoframes), the assumptions made during the reconstruction of event data from those different pseudoframes can, but do not need to, be the same across different pseudoframes.

[0054] Figure 5 This is a partial schematic block diagram of another time aggregation circuit 535 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 535 may be included in... Figure 2 This is an example of a time aggregation circuit 235 in the event visual pixel 210, or another example of a time aggregation circuit of the present technology. As shown, the time aggregation circuit 535 includes a first counter 535a (e.g., a first analog counter) and a second counter 535b (e.g., a second analog counter). The first counter 535a includes a capacitor 563a having a capacitance C1. The top plate of the capacitor 563a generates a comparator (e.g., based on the corresponding event from the corresponding event visual pixel) Figure 2 The signal UP received by the first comparator 234a) is selectively coupled to the current source 561a via switch 532a (e.g., a transistor). The top plate of capacitor 563a is further (a) selectively coupled to the reference voltage V via switch 564a (e.g., a transistor) based on the reset signal RES. ref (b) is coupled to the input of buffer or amplifier 565a. The base plate of capacitor 563a is grounded. Additionally, the output of amplifier 565a is based on the UP event readout signal R. U It is selectively coupled to column line 538 via switch 566a (e.g., transistor).

[0055] Similarly, the second counter 535b includes a capacitor 563b with capacitance C2. The top plate of capacitor 563b generates a comparator based on the corresponding event from the corresponding event visual pixel (e.g., Figure 2 The signal DOWN received by the second comparator 234b is selectively coupled to the current source 561b via a switch 532b (e.g., a transistor). The top plate of the capacitor 563b is further (a) selectively coupled to the reference voltage V based on the reset signal RES via a switch 564b (e.g., a transistor). ref (b) is coupled to the input of buffer or amplifier 565b. The base plate of capacitor 563b is grounded. Additionally, the output of amplifier 565b is based on the DOWN event readout signal R. D It is selectively coupled to column line 538 via switch 566b (e.g., transistor).

[0056] Figure 6 This is a diagram illustrating various embodiments of the present technology. Figure 5 An exemplary signal diagram 668 illustrates the method of operating the time aggregation circuit 535. (See also...) Figure 5 and 6 The reset signal RES is asserted at time t0 to selectively couple (a) the top plate of capacitor 563a of the first counter 535a and the top plate of capacitor 563b of the second counter 535b to (b) the reference voltage V. ref The top plates of capacitors 563a and 563b are coupled to the reference voltage V. ref This will reset the voltages on capacitors 563a and 563b to known values. Although not shown, this corresponds to one or more latches in the scanner and readout logic of the visual pixel of the event (e.g., referenced above). Figure 2 The latch in the scanner and readout logic 236 of the event visual pixel 210 can be reset at time t0 or at time t1 or at another time before.

[0057] Then, the reset signal RES is in Figure 6 The assertion is lifted at time t1, as shown in the figure, thereby removing the top plates of capacitors 563a and 563b from the reference voltage V. ref Decoupling is performed, and the start of the timing window between time t1 and time t2 is marked. During this period, the first counter 535a and the second counter 535b count the number of UP events and DOWN events detected by the corresponding event visual pixel, respectively. In the illustrated example, during the timing window between time t1 and time t2, the corresponding event visual pixel first detects one UP event and then detects two DOWN events. When the corresponding event visual pixel detects an UP event, the event visual pixel (e.g., the event generation comparator of the event visual pixel, such as...)... Figure 2 The first comparator 234a) delivers a pulsed signal UP. Subsequently, the top plate of capacitor 563a of the first counter 535a is coupled to current source 561a for the duration of the pulse, thereby reducing the voltage at capacitor 563a by an amount proportional to the duration of the pulse. In some embodiments, one or more latches in the scanner and readout logic (e.g., scanner and readout logic 236) of the corresponding event visual pixel can be triggered after the UP event is detected to indicate that the corresponding event visual pixel has detected at least one event during a timing window, and thereby requesting the readout of the corresponding event visual pixel at or after the end of the timing window.

[0058] Similarly, when the corresponding event visual pixel detects Figure 6 The diagram illustrates the first DOWN event, during which the event visual pixel (e.g., the event generation comparator of the event visual pixel, e.g.) Figure 2 The second comparator 234b delivers a pulsed signal DOWN. Subsequently, the top plate of capacitor 563b of the second counter 535b is coupled to current source 561b for the duration of the pulse, thereby reducing the voltage at capacitor 563b by an amount proportional to the pulse duration. This process occurs when the event visual pixel detects... Figure 6 The second DOWN event, as illustrated in the diagram, is repeated, thereby further reducing the voltage at capacitor 563b.

[0059] As discussed above, the current timed window ends at time t2. Figure 6 As shown in, the signal R U At time t3, an assertion is made (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of amplifier 565a of the first counter 535a to column line 538. Then, a first voltage value (representing the aggregated number of UP events detected by the event visual pixels during the timing window) is read onto column line 538. Then, signal R... U The assertion is de-asserted at time t4 to decouple the output of amplifier 565a from column line 538. Afterwards, signal R... D At time t5, an assertion is made (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of amplifier 565b of the second counter 535b to column line 538. Subsequently, a second voltage value (representing the aggregated number of DOWN events detected by the event visual pixels during the timing window) is read onto column line 538. Then, signal R... D The assertion is de-asserted at time t6 to decouple the output of amplifier 565b from column line 538. The reset signal RES can then be asserted at time t7, at the expected start of the next timing window.

[0060] Despite Figure 6 In the embodiment illustrated in the diagram, the first voltage value from the first counter 535a is read before the second voltage value from the second counter 535b. However, in other embodiments of the art, the second voltage value may be read before the first voltage value. Alternatively, the first counter 535a may be coupled to a column line different from that of the second counter 535b. In these embodiments, the first voltage value may be read before, after, or simultaneously with the second voltage value.

[0061] There may be a mismatch between the capacitance C1 of capacitor 563a and the capacitance C2 of capacitor 563b. Alternatively, the current I supplied to the top plate of capacitor 563a by current source 561a may be (i) a (ii) The current I supplied to the top plate of capacitor 563b by current source 561b b There may be a mismatch between them. Such a mismatch can affect the voltage values ​​read from the first counter 535a and / or the second counter 535b. Therefore, in some embodiments, the event vision sensor may perform calibration techniques (e.g., at the event signal processor) on the voltage values ​​read from the first counter 535a and / or the second counter 535b to address (a) the mismatch between capacitors C1 and C2 and / or (b) the current I. a with I b The mismatch between them.

[0062] Figure 7 This is a partial schematic block diagram of another time aggregation circuit 735 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 735 may be included in... Figure 2 This is an example of a time aggregation circuit 235 in the event visual pixel 210, or another example of a time aggregation circuit of the present technology. As shown, the time aggregation circuit 735 includes a first counter 735a (e.g., a first analog counter) and a second counter 735b (e.g., a second analog counter). The first counter 735a includes (i) a first capacitor 771a having a capacitance C1 and (ii) a second capacitor 772a having a capacitance C2. In some embodiments, the second capacitance C2 may be greater than the first capacitance C1. In other embodiments, the second capacitance C2 may be equal to or less than the first capacitance C1.

[0063] The top plate of the second capacitor 772a is selectively coupled to the reference voltage V via a switch 775a (e.g., a transistor) based on the reset signal RES. RESThe top plate of the second capacitor 772a is further (a) coupled to the input of the buffer or amplifier 776a, and (b) based on a comparator generated from the corresponding event (e.g., Figure 2 The signal UP received by the first comparator 234a is selectively coupled to the top plate of the first capacitor 771a via a switch 774a (e.g., a transistor). The top plate of the first capacitor 771a is based on the signal V. br Further selectively coupled to the reference voltage V via switch 773a (e.g., transistor). R In some embodiments, the reference voltage V R This is equivalent to a voltage less than the reference voltage V. RES The equivalent voltage value. In other embodiments, the reference voltage V R This is equivalent to a voltage greater than or equal to the reference voltage V. RES The equivalent voltage value. The base plates of the first capacitor 771a and the second capacitor 772a are grounded. Additionally, the output of amplifier 776a is based on the UP event readout signal R. U It is selectively coupled to the column line 738 via a switch 777a (e.g., a transistor).

[0064] Similarly, the second counter 735b includes (i) a first capacitor 771b having capacitance C1 and (ii) a second capacitor 772b having capacitance C2. In some embodiments, the second capacitor C2 may be greater than the first capacitor C1. In other embodiments, the second capacitor C2 may be equal to or less than the first capacitor C1. The top plate of the second capacitor 772b is selectively coupled to a reference voltage V via a switch 775b (e.g., a transistor) based on a reset signal RES. RES The top plate of the second capacitor 772b is further (a) coupled to the input of the buffer or amplifier 776b, and (b) based on a comparator generated from the corresponding event (e.g., Figure 2 The signal DOWN received by the second comparator 234b is selectively coupled to the top plate of the first capacitor 771b via a switch 774b (e.g., a transistor). The top plate of the first capacitor 771b is further coupled based on the signal V. br Selectively coupled to the reference voltage V via switch 773b (e.g., transistor). R Signal V br It can be selectively coupled to the top plate of the first capacitor 771a used to connect the first counter 735a to the reference voltage V. R signal V br The signals are either the same or different. The base plates of the first capacitor 771b and the second capacitor 772b of the second counter 735b are grounded. Additionally, the output of the amplifier 776b is based on the DOWN event readout signal R. DIt is selectively coupled to column line 738 via switch 777b (e.g., transistor).

[0065] Figure 8 This is a diagram illustrating various embodiments of the present technology. Figure 7 An exemplary signal diagram illustrating the method of operating the time aggregation circuit 735. (See also...) Figure 7 and 8 The reset signal RES is asserted at time t0 to selectively couple (a) the top plate of the second capacitor 772a of the first counter 735a and the top plate of the second capacitor 772b of the second counter 735b to (b) the reference voltage V. RES Additionally, signal V br It is asserted at time t0 to selectively couple (a) the top plate of the first capacitor 771a of the first counter 735a and the top plate of the first capacitor 771b of the second counter 735b to (b) the reference voltage V. R Despite Figure 8 The example shows that both are asserted at time t0, but in other embodiments, signal V... br Assertions can be made before or after the reset signal RES. Although not shown, this corresponds to one or more latches in the scanner and readout logic of the visual pixel of the event (e.g., referenced above). Figure 2 The latch in the scanner and readout logic 236 of the event visual pixel 210 can be reset at time t0 or at time t1 or at another time before.

[0066] Then, the reset signal RES and signal V br exist Figure 8 The assertion is lifted at time t1 as shown in the figure, thereby (i) removing the top plates of capacitors 771a and 771b from the reference voltage V. R Decoupling, (ii) removing the top plates of capacitors 772a and 772b from the reference voltage V RES Decoupling, and (iii) marking the start of the timing window between time t1 and time t2, during which the first counter 735a and the second counter 735b count the number of UP events and DOWN events detected by the corresponding event visual pixel, respectively. In the illustrated example, during the timing window between time t1 and time t2, the corresponding event visual pixel first detects one UP event and then detects two DOWN events. When the corresponding event visual pixel detects an UP event, the event visual pixel (e.g., the event generation comparator of the event visual pixel, such as...)... Figure 2The first comparator 234a) delivers a pulse signal UP. Subsequently, the top plate of the first capacitor 771a of the first counter 735a is coupled to the top plate of the second capacitor 772a via switch 774a during the pulse duration. When the top plates of the first capacitor 771a and the second capacitor 772a are coupled together, charge is shared between capacitors 771a and 772a, and the voltage at the second capacitor 772a immediately before the switch 774a is activated drops at least partially based on the reference voltage V at the first capacitor 771a. R The quantity. Then, the signal UP is deasserted to decouple the top plates of capacitors 771a and 772a from each other. Afterwards, the signal V... br It is asserted to activate switch 773a and restore the voltage at the first capacitor 771a to the reference voltage V. R After that, signal V br The assertion is lifted. In some embodiments, one or more latches in the scanner and readout logic (e.g., scanner and readout logic 236) of the corresponding event visual pixel may be triggered after an UP event is detected to indicate that the corresponding event visual pixel has detected at least one event during the timing window, and thereby request the readout of the corresponding event visual pixel at or after the end of the timing window.

[0067] Although not shown, the time aggregation circuitry 735 and / or the event visual pixel 210 may include features for controlling the signal V. br A timing circuit system that determines when an assertion occurs after a signal UP or DOWN has been asserted. For example, the timing aggregation circuit 735 and / or the event visual pixel 210 may include logic gates (e.g., "OR" logic gates) coupled to the output of the event generating comparator 234. Regardless of when the event generating comparator 234 detects an event, the output of the logic gates can be used to assert the signal V. br The delay element can be coupled to the output of the logic gate and configured to introduce a delay on the output, such that switches 773a and / or 773b are not activated until the signal UP or the signal DOWN has been pulsed to activate switches 774a or 774b respectively, thereby coupling the top plates of capacitors 771 and 772 of the corresponding counter 735 together. In other words, the delay element can be configured to cause the signal V br It was not asserted at the same time as the signal UP or signal DOWN.

[0068] When the corresponding event visual pixel is detected Figure 8 The diagram illustrates the first DOWN event, during which the event visual pixel (e.g., the event generation comparator of the event visual pixel, e.g.) Figure 2The second comparator 234b delivers a pulsed signal DOWN, thereby activating switch 774b and coupling the top plate of the first capacitor 771b of the second counter 735b to the top plate of the second capacitor 772b. Subsequently, (i) charge is shared between capacitors 771b and 772b, and (ii) the voltage at the second capacitor 772b immediately before the switch 774b is activated drops at least partially based on the reference voltage V at the first capacitor 771b. R The quantity. Then, the signal DOWN is deasserted to decouple the top plates of capacitors 771b and 772b. Afterwards, the signal V... br It is asserted to activate switch 773b and restore the voltage at the first capacitor 771b to the reference voltage V. R After that, signal V br The assertion is lifted. This process occurs when the event is detected by the visual pixel. Figure 8 The second DOWN event, as illustrated in the diagram, is repeated, thereby further reducing the voltage at the second capacitor 772b.

[0069] As discussed above, the current timed window ends at time t2. Figure 8 As shown in, the signal R U At time t3, an assertion is made (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of amplifier 776a of the first counter 735a to column line 738. Then, a first voltage value (representing the aggregated number of UP events detected by the event visual pixels during the timing window) is read onto column line 738. Then, signal R... U The assertion is de-asserted at time t4 to decouple the output of amplifier 776a from column line 738. Afterwards, signal R... D At time t5, an assertion is made (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of amplifier 776b of the second counter 735b to column line 738. Subsequently, a second voltage value (representing the aggregated number of DOWN events detected by the event visual pixels during the timing window) is read onto column line 738. Then, signal R... D The assertion is de-asserted at time t6 to decouple the output of amplifier 776b from column line 738. Then, the reset signal RES and signal V... br It can be asserted at time t7 when the next timing window is expected to begin.

[0070] Despite Figure 8In the embodiment illustrated in the diagram, the first voltage value from the first counter 735a is read before the second voltage value from the second counter 735b. However, in other embodiments of the art, the second voltage value may be read before the first voltage value. Alternatively, the first counter 735a may be coupled to a column line different from that of the second counter 735b. In these embodiments, the first voltage value may be read before, after, or simultaneously with the second voltage value.

[0071] Note that, compared to Figure 5 Comparing the first voltage value and the second voltage value read from the first counter 535a and the second counter 535b, from Figure 7 The first and second voltages read from the first counter 735a and the second counter 735b do not depend on the absolute current or absolute capacitance of the current source or capacitor, respectively. Instead, the voltage value read from each of the counters 735a and 735b depends on the ratio of the capacitances of the corresponding capacitors within the corresponding counter 735a or 735b. Therefore, no calibration technique is needed to address any mismatch between the capacitances of the capacitors in the counters 735a and 735b or between the current sources. However, when an UP event is detected during a given timing window, the voltage drop across the second capacitor 772a may become smaller for each consecutive UP event detected during that given timing window. Similarly, when a DOWN event is detected during a given timing window, the voltage drop across the second capacitor 772b may become smaller for each consecutive DOWN event detected during that given timing window. Therefore, the event vision sensor can perform calibration techniques (e.g., at the event signal processor) to address this nonlinearity.

[0072] Further attention should be paid to, Figure 5 Time aggregation circuit 535 and Figure 7 The time aggregation circuit 735 is similar to Figure 3 The time aggregation circuit 335 includes, for example, each circuit comprising: (i) a first counter for counting the number of UP events detected by the corresponding event visual pixel during a timing window; and (ii) a second counter for counting the number of DOWN events detected by the corresponding event visual pixel during the timing window. Therefore, an event vision sensor having event visual pixels employing time aggregation circuit 535 and / or time aggregation circuit 735 can utilize the above reference. Figure 4 The event data compression procedure is discussed in detail.

[0073] Figure 9 This is a partial schematic block diagram of another time aggregation circuit 935 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 935 may be included in... Figure 2 This is an example of a time aggregation circuit 235 in the event visual pixel 210, or another example of a time aggregation circuit of the present technology. As shown, the time aggregation circuit 935 includes an increment / decrement counter 939. The increment / decrement counter 939 includes two inputs. The first input is configured to generate a comparator (e.g., from the corresponding event) Figure 2 The first comparator 234a) receives the UP signal. The second input is configured to generate a comparator (e.g., from the corresponding event) Figure 2 The second comparator 234b) receives the DOWN signal. The increment / decrement counter 939 further includes an output coupled to the column line 938.

[0074] In operation, the increment / decrement counter 939 is configured to track events detected by the corresponding event visual pixel during a given timing window. Unlike other counters discussed above, the increment / decrement counter 939 does not necessarily output the total number of UP events or DOWN events detected by the corresponding event visual pixel during the given timing window. Instead, the increment / decrement counter 939 outputs the difference between (i) the number of UP events detected by the corresponding event visual pixel during the timing window and (ii) the number of DOWN events detected by the corresponding event visual pixel.

[0075] To clarify and understand the purpose, consider Figure 10 The illustrations depict various embodiments of the present technology. Figure 9 An overview of the event data compression performed by the time aggregation circuit 935. Figure 10 A plot 1041 contains UP and DOWN events detected by the event visual pixels within a timing window 1041 between time t1 and time t2. Plot 1040 is similar to... Figure 4 The diagram 440. For example, diagram 1040 illustrates that the corresponding event visual pixels detected eight (8) UP events and ten (10) DOWN events in the order shown during the timing window 1041. It can be used... Figure 9 The time aggregation circuit 935 uses an increment / decrement counter 939 to track these events. The increment / decrement counter 939 can output a value (e.g., a single value) representing the difference between the number of UP events detected and the number of DOWN events during the timing window 1041, rather than outputting two values ​​indicating the absolute number of UP events or DOWN events detected by the event visual pixel during the timing window 1041. Therefore, in the illustrated example, a value indicating that the event visual pixel detected two more DOWN events than UP events during the timing window 1041 will be output to column line 938.

[0076] Note that the value output from counter 939 to column line 938 merely reflects the difference between the number of UP events and the number of DOWN events detected by the event visual pixel during timing window 1041. In other words, the value output to column line 939 does not reflect the precise timing at which each individual event was detected by the event visual pixel during timing window 1041, nor does it necessarily indicate the absolute number of UP or DOWN events detected during timing window 1041. Therefore, this pixel-level compression of event data performed by the event visual pixel can be considered a lossy form of compression in which the event data and the timing information of individual events are lost.

[0077] Similar to the reference above Figure 4 The event data discussed, read out onto column line 938, can be combined with other values ​​read from other event vision pixels of the event vision sensor to form a pseudo-frame. For example, such as Figure 10 As shown, the value read from the increment / decrement counter 939 onto the column line 938 within the timing window 1041 can be combined with other values ​​read from other event visual pixels within the same timing window 1041 to form a pseudo-frame 1050. This will then be further explained in the following references. Figure 11 As discussed in more detail, the event vision sensor may optionally perform compression on pseudoframe 1050.

[0078] Frames showing the difference in event counts read from the event visual pixels of the array within different timing windows are also referred to herein as "pseudo-frames" because the event data contained in each such pseudo-frame represents the difference between the number of UP events detected by the event visual pixels during a given timing window and the number of DOWN events, the difference being proportional to changes in contrast or information in the external scene. In contrast, as discussed above, image data frames are conventionally related to exposure time, and the image data represents the amount of photons / electrons generated by the light incident on the photoelectric sensor of each pixel during the exposure time.

[0079] Figure 10The diagram further includes an illustration of a plot 1045 reconstructing the event data based on pseudoframe 1050. More specifically, the event data corresponding to plot 1040 and the event visual pixels within timing window 1041 can be reconstructed using the relevant event data 1055 stored in pseudoframe 1050. As discussed above, the absolute number of UP events detected by the event visual pixels during timing window 1041, the absolute number of DOWN events detected by the event visual pixels during timing window 1041, and / or the timing information regarding when each of the individual events occurred during timing window 1041 may be lost during pixel compression of the event data. Therefore, assumptions can be made during reconstruction regarding the number of events detected by the event visual pixels during timing window 1041 and regarding when each individual event occurred within timing window 1041. In the illustrated example, it is assumed that the event visual pixels detected two DOWN events and zero UP events during timing window 1041 based on the difference output from the increment / decrement counter 939 above to the column line 938. It is further assumed that the visual pixels of the events detect two DOWN events at uniform intervals across the timing window 1041. Therefore, during reconstruction, the two DOWN events can be uniformly distributed across the timing window 1041, as shown in Figure 1045.

[0080] Other assumptions regarding (i) the number of UP or DOWN events detected by the event visual pixel during timing window 1041 and / or (ii) the distribution of UP and / or DOWN events across the timing window are of course possible and within the scope of this technique. Alternatively, different assumptions may be made for UP and DOWN events, and / or multiple assumptions may be made for UP or DOWN events during reconstruction, such that the distribution of UP and / or DOWN events can vary across the time window. Furthermore, for the same pseudoframe, the assumptions made during the reconstruction of event data corresponding to one event visual pixel may, but need not, be the same as the assumptions made during the reconstruction of event data corresponding to another event visual pixel. Furthermore, for different pseudoframes (e.g., pseudoframes corresponding to different timing windows), the assumptions made during the reconstruction of event data corresponding to event visual pixels may, but need not, be the same across different pseudoframes.

[0081] Figure 11 This is a partial schematic block diagram of an event signal processor 1108 configured according to various embodiments of the present technology. It should be understood that the illustrated event signal processor 1108 may be... Figure 1This is an example of an event signal processor 108 of an event vision sensor 100, or another example of an event signal processor of the present technology. As shown, the event signal processor 1108 includes a defective pixel removal block 1182, a compression block 1183, and an auxiliary function block 1186. The defective pixel removal block 1182 can be used to remove pixels corresponding to an array (e.g., Figure 1 The array 102) contains pixel signals of visual pixels representing defective events. The auxiliary function block 1186 may include one or more other stages or circuits for performing various auxiliary processing functions.

[0082] As discussed above, an event vision sensor configured according to this technology can form pseudoframes using event data read from a temporal aggregation circuit or level of event vision pixels in an array. These pseudoframes can typically contain event data representing shapes or objects, meaning that, in addition to or as an alternative to pixel-level compression performed by the temporal aggregation circuit of the event vision pixels, a compression algorithm (e.g., wavelet compression) can be applied to the pseudoframes to compress the event data. Therefore, the compression block 1183 of the event signal processor 1108 can be used to compress event data at the pseudoframe level. In some embodiments, the compression block 1183 can utilize a lossy compression algorithm to compress the pseudoframes. Examples of lossy compression algorithms that can be used include Joint Picture Experts Group (JPEG) compression, Advanced Video Coding (AVC) compression (also known as H.264 compression), and High Efficiency Video Coding (HEVC) compression (also known as H.265 compression). In these and other embodiments, the compression block 1183 of the event signal processor 1108 can utilize a lossless compression algorithm, such as Moving Picture Experts Group (MPEG) compression.

[0083] although Figure 11 Not shown, but the event signal processor 1108 may additionally include other circuitry or modules for performing specific processing functions. For example, the event signal processor 1108 may include a segmentation classifier block and / or a shape classifier block for classifying segments and shapes of event data read from the array, respectively. As another example, the event signal processor 1108 may include an optical flow estimation block for identifying pixel-by-pixel, shape-by-shape, or segment-by-segment motion over time and / or between consecutive readouts (e.g., using correlation-based, block-matching-based, feature-tracking-based, energy-based, and / or gradient-based optical flow estimation). Alternatively, the event signal processor 1108 may include one or more buffers (e.g., a set of row buffers) operatively coupled to one or more blocks or circuitry of the event signal processor 1108 to perform various processing functions.

[0084] Figure 12 This is an illustration of various embodiments of the present technology of an event visual sensor (e.g., Figure 1The flowchart illustrates a method 1290 for operating an event vision sensor 100. Method 1290 is illustrated as a set of steps or blocks 1291 to 1297. All or a subset of one or more of blocks 1291 to 1297 may be performed by various components of the event vision sensor. For example, all or a subset of one or more of blocks 1291 to 1297 may be performed by: (i) individual event vision pixels (e.g., one or more event generation comparators 234 and / or event vision pixel timing aggregation circuitry or stages); (ii) an event vision pixel array; (iii) a row control circuitry system; (iv) a column control circuitry system; (v) an event signal processor; (vi) a memory and / or (vii) a transmitter. Furthermore, any one or more of blocks 1291 to 1297 may be performed according to the above... Figures 1 to 11 The argument is to be implemented.

[0085] Figure 12 Method 1290 begins detecting events occurring within a timed window at box 1291. Detecting events occurring within a timed window may include detecting one or more UP events and / or DOWN events that occur within the timed window and / or correspond to activities in the external scene.

[0086] At block 1292, method 1290 continues to compress event data corresponding to a timing window. In some embodiments, compressing the generated event data may include compressing the event data generated by at least one event visual pixel of the event visual sensor and / or before reading the event data from at least one event visual pixel. In these and other embodiments, compressing the event data may include compressing the event data using a temporal aggregation level (e.g., corresponding to the event visual pixel). Using a temporal aggregation level to compress the event data may include compressing the event data using one or more counters. In some embodiments, compressing the event data includes tracking events detected during a given timing window or a time period. For example, compressing the event data may include tracking or counting: (i) the absolute number of UP events detected by the event visual pixel (e.g., within the timing window); and / or (ii) the absolute number of DOWN events detected by the event visual pixel (e.g., within the timing window). In these and other embodiments, compressing the event data may include tracking or calculating the difference between (a) the number of UP events detected by the event visual pixels (e.g., within a timing window) and (b) the number of DOWN events detected by the event visual pixels (e.g., within a timing window).

[0087] At block 1293, method 1290 continues to read compressed event data from the event visual pixel. In some embodiments, reading the compressed event data may include reading the absolute number of UP events detected by the event visual pixel (e.g., within a timing window) and / or reading the absolute number of DOWN events detected by the event visual pixel (e.g., within a timing window). In these and other embodiments, reading the compressed event data may include reading the difference between (a) the number of UP events detected by the event visual pixel (e.g., within a timing window) and (b) the number of DOWN events detected by the event visual pixel (e.g., within a timing window).

[0088] In some embodiments, reading compressed event data may involve reading the compressed event data from the event visual pixel only if the event visual pixel has detected at least one event during a given timing window. For example, compressed event data (UP event count, DOWN event count, or the difference between the UP event count and the DOWN event count) corresponding to a given timing window may be read from the event visual pixel only if the event visual pixel has detected at least one event (UP event or DOWN event) during a given timing window. As another example, the UP event count may be read from the event visual pixel as compressed event data only if the event visual pixel has detected at least one UP event during the corresponding given timing window. Alternatively, the DOWN event count may be read from the event visual pixel only if the event visual pixel has detected at least one DOWN event during the corresponding given timing window.

[0089] At block 1294, method 1290 continues to generate a pseudo-frame using compressed event data read from one or more event visual pixels. Generating a pseudo-frame may involve combining or relating compressed event data read from a first event visual pixel within a given timing window with compressed event data read from at least one other event visual pixel within the same timing window. In some embodiments, generating a pseudo-frame may include: (i) generating a first pseudo-frame using first compressed event data read from a first set of event visual pixels, the first compressed event data representing the number of UP events detected by the first set of event visual pixels within the given timing window; and / or (ii) generating a second pseudo-frame using second compressed event data read from a second set of event visual pixels, the second compressed event data representing the number of DOWN events detected by the second set of event visual pixels within the given timing window. The first set of event visual pixels may comprise a group of event visual pixels that are the same as or different from the second set of event visual pixels.

[0090] At block 1295, method 1290 continues to perform compression on the one or more pseudo-frames generated at block 1294. In some embodiments, performing compression on the one or more pseudo-frames may include performing lossy compression on the one or more pseudo-frames. For example, performing compression on the one or more pseudo-frames may include performing JPEG, H264, and / or H265 on the one or more pseudo-frames. In these and other embodiments, performing compression on the one or more pseudo-frames may include performing lossless compression on the one or more pseudo-frames. For example, performing compression on the one or more pseudo-frames may include performing MPEG on the one or more pseudo-frames.

[0091] At block 1296, method 1290 continues to store compressed event data and / or pseudoframes. Storing event data and / or pseudoframes may include storing the event data and / or pseudoframes into memory, such as into a buffer of the event vision sensor.

[0092] At box 1297, method 1290 continues to transmit event data and / or pseudoframes. Transmitting event data and / or pseudoframes may include transmitting the event data and / or pseudoframes out of the event vision sensor and / or to a receiver in the corresponding imaging system.

[0093] Although blocks 1291 to 1297 of method 1290 are discussed and illustrated in a specific order, Figure 12The method 1290 illustrated herein is not limited thereto. In other embodiments, method 1290 may be performed in a different order. In these and other embodiments, any of blocks 1291 to 1297 of method 1290 may be performed before, during, and / or after any of the other blocks 1291 to 1297 of method 1290. For example, block 1295 may be performed concurrently with the generation of the pseudo-frame at block 1294. As another example, block 1296 may be performed before or concurrently with the execution of blocks 1294 and / or 1295. As yet another example, blocks 1291, 1292, and / or 1293 may be performed concurrently with the execution of one or more of blocks 1294 to 1297. Furthermore, those skilled in the art will recognize that the illustrated method 1290 may be modified while remaining within these and other embodiments of the present technology. For example, in some embodiments, one or more boxes 1291 to 1297 of method 1290 may be omitted and / or repeated. As a specific example, in some embodiments (e.g., in embodiments where the event vision sensor performs frame-level compression on pseudo-frames at box 1295 but not pixel-level compression on events detected at pixel level), box 1292 may be omitted. As another specific example, in some embodiments (e.g., in embodiments where the event vision sensor performs pixel-level compression on events detected at pixel level but not frame-level compression on pseudo-frames), box 1295 may be omitted. As yet another specific example, in some embodiments (e.g., in embodiments where at least some of the event data read from event vision pixels and / or used to form one or more pseudo-frames is discarded instead of being stored in memory and / or transmitted out of the corresponding event vision sensor), boxes 1296 and / or 1297 may be omitted.

[0094] As discussed above, this technique facilitates compression of event data at the pixel level and / or at the pseudo-frame level. Consequently, compared to the amount of data used to record events in an event vision sensor that does not perform such compression, the amount of data stored in the memory (e.g., buffer) of an event vision sensor configured according to various embodiments of this technique to record events detected by the event vision pixels of the event vision sensor can be reduced. Consequently, compared to other event vision sensors, the number of events that an event vision sensor using this technique can store in a fixed amount of memory can be increased. In some cases, this technique can therefore facilitate longer recording times before the fixed amount of memory becomes full.

[0095] One application of this technology includes high-speed video capture. For example, an active pixel sensor can be used to capture image data (e.g., video) of an external scene, while an event vision sensor can be used to capture event data corresponding to activities occurring within the external scene. The event data captured and compressed by the event vision sensor can be decompressed and synthesized with the image data captured using the active pixel sensor. Thus, this technology can be used to detect and / or record high-speed components of activities occurring in an external scene, which may be useful, for example, when capturing slow-motion video.

[0096] Although several embodiments of the present technology have been discussed above in the context of counting and compressing events at individual pixel levels, the present technology is not limited thereto. For example, event vision pixels in an array of such pixels included in an event vision sensor may be organized into blocks or groups. In some embodiments, one or more time aggregation circuits may be shared among a group of event vision pixels. Continuing this example, one or more time aggregation circuits may include one or more counters configured to count: (a) UP events detected by the event vision pixels of the group; (b) DOWN events detected by the event vision pixels of the group; and / or (c) the difference between the number of UP events detected by the event vision pixels of the group and the number of DOWN events detected by the event vision pixels of the group. One or more time aggregation circuits may be located within one or more event vision pixels of the group, or may be peripheral circuitry located outside the event vision pixels. Event data generated by one or more time aggregation circuits may be read out and combined with event data corresponding to one or more other groups of event vision pixels to form one or more pseudo-frames. Then, the event vision sensor can optionally perform frame-level compression on such pseudo-frames.

[0097] In other embodiments of this technology, the event vision pixel may include time aggregation circuitry that counts only the number of UP events and / or DOWN events detected by the event vision pixel within a given timing window. At or after the end of the given timing window, the UP event count and / or DOWN event count may be output to peripheral circuitry to, for example, calculate one or more differences between the number of UP events detected by the event vision pixel within the given timing window and the number of DOWN events detected by the event vision pixel within the given timing window. These differences may then be read and used to generate one or more pseudo-frames, on which the event vision sensor may optionally perform frame-level compression.

[0098] C. in conclusion

[0099] The above detailed description of embodiments of this technology is not intended to be exhaustive or to limit the technology to the precise forms disclosed above. While specific embodiments and examples of this technology have been described above for illustrative purposes, those skilled in the art will recognize that various equivalent modifications can be made within the scope of this technology. For example, although the steps are presented in a given order above, alternative embodiments may perform the steps in a different order. Furthermore, the various embodiments described herein may be combined to provide further embodiments.

[0100] Based on the foregoing, it is understood that, for illustrative purposes, specific embodiments of the present technology have been described herein, but well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of embodiments of the present technology. In the event of any conflict between any material incorporated herein by reference and this disclosure, this disclosure shall prevail. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, unless the word “or” is expressly limited to meaning only a single item excluding other items when referring to a list of two or more items, its use in this list shall be interpreted as including (a) any single item in the list, (b) all items in the list, or (c) any combination of items in the list. Additionally, as used herein, “and / or” in the phrase “A and / or B” refers to A alone, B alone, or both A and B. Furthermore, the terms “comprising,” “including,” “having,” and “with” are used throughout to mean at least the listed features, such that no further number of the same features and / or additional types of features are excluded. Furthermore, as used herein, the phrases “based on,” “depending on,” “as a result of,” and “in response to” should not be construed as references to a closed set of conditions. For example, an exemplary step described as “based on condition A” could be based on both condition A and condition B without departing from the scope of the invention. In other words, as used herein, the phrase “based on” should not be interpreted in the same way as the phrase “at least partially based on.” Additionally, the terms “connected” and “coupled” are used interchangeably herein and refer to both direct and indirect connection or coupling. For example, where the context permits, element A being “connected” or “coupled” to element B could mean (i) directly “connected” or directly “coupled” to B and / or (ii) indirectly “connected” or indirectly “coupled” to B.

[0101] Based on the foregoing, it will also be understood that various modifications can be made without departing from this disclosure or technology. For example, those skilled in the art will understand that various components of this technology can be further divided into sub-components, or various components and functions of this technology can be combined and integrated. Additionally, certain aspects of this technology described in the context of a particular embodiment can be combined or eliminated in other embodiments. Furthermore, although advantages associated with certain embodiments of this technology have been described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments necessarily need to exhibit such advantages to fall within the scope of this invention. Therefore, this invention and related technologies may encompass other embodiments not explicitly shown or described herein.

Claims

1. An event visual pixel, comprising: A photoelectric sensor configured to generate a photocurrent in response to incident light received from an external scene; A photocurrent-to-voltage converter coupled to the photosensor to convert the photocurrent into a voltage; A difference detection circuit, coupled to the photocurrent-to-voltage converter, generates a filtered and amplified signal in response to a difference detected over time in the voltage received from the photocurrent-to-voltage converter. An event generation comparator, coupled to the difference detection circuit, detects events occurring within the external scene based on a comparison of the filtered and amplified signal with one or more thresholds, wherein the events include UP events and DOWN events, wherein each of the UP events corresponds to a change in the incident light from darker to brighter, and each of the DOWN events corresponds to a change in the incident light from brighter to darker. and A time aggregation circuit, coupled to the event generation comparator, tracks the aggregated number of events detected by the event generation comparator within a specified timing window, and reads out the number of events detected by the event visual pixel within the specified timing window at or after the end of the specified timing window.

2. The event visual pixel of claim 1, wherein the time aggregation circuit includes at least one counter configured to count the number of aggregated events detected by the event generating comparator within the defined timing window.

3. The event visual pixel according to claim 2, wherein the at least one counter comprises: A first capacitor having a top plate and a grounded bottom plate, wherein the top plate of the first capacitor is selectively coupled to a first reference voltage via a first switch and based on a first reset signal; and A second capacitor having a top plate and a grounded bottom plate, wherein the top plate of the second capacitor is: (a) selectively coupled to the top plate of the first capacitor via a second switch and based on an event signal received from the event generating comparator; (b) selectively coupled to a second reference voltage via a third switch and based on a second reset signal; and (c)(i) selectively coupled to a column line via an amplifier and a fourth switch and (ii) based on a readout signal.

4. The event visual pixel according to claim 2, wherein the at least one counter comprises: A first counter is configured to track the aggregate number of UP events detected by the event generating comparator within the specified timing window; and A second counter is configured to track the aggregate number of DOWN events detected by the event generating comparator within the specified timing window.

5. The event visual pixel of claim 1, further comprising readout logic coupled to the output of the event generating comparator, wherein the readout logic includes a latch that is triggered when the event generating comparator detects at least one event within the specified timing window.

6. An event vision sensor, comprising: Multiple event visual pixels, in: Each of the plurality of event visual pixels is configured to generate event data based at least in part on an event indicated in incident light received from an external scene, wherein the event includes UP events and DOWN events, wherein each of the UP events corresponds to a change in the incident light from darker to brighter, and each of the DOWN events corresponds to a change in the incident light from brighter to darker. Each of the plurality of event visual pixels includes compression circuitry configured to compress the event data before it is read out from the event visual pixel.

7. The event vision sensor according to claim 6, wherein: Each compression circuit includes a time aggregation circuit; and In order to compress the event data, the time aggregation circuit is configured to track the number of events detected by the corresponding event visual pixels within a specified timing window.

8. The event vision sensor of claim 7, wherein each time aggregation circuit includes at least one counter configured to count the number of events detected by the corresponding event vision pixel within the defined timing window.

9. The event vision sensor according to claim 7, wherein: Each time aggregation circuit includes an increment / decrement counter configured to track the difference between (a) the number of UP events detected by the corresponding event visual pixel within the specified timing window and (b) the number of DOWN events detected by the corresponding event visual pixel within the specified timing window.

10. The event vision sensor of claim 7, wherein each of the plurality of event vision pixels further includes readout logic, the readout logic including at least one latch, wherein the at least one latch is triggered when the corresponding event vision pixel detects at least one event within the specified timing window.

11. The event vision sensor of claim 6, wherein the event vision sensor is configured to form pseudo-frames based at least in part on event data readouts from one or more of the plurality of event vision pixels.

12. The event vision sensor of claim 11, further comprising an event signal processor configured to perform a compression algorithm on the pseudoframe.

13. The event vision sensor of claim 12, wherein the event signal processor is configured to perform a lossy compression algorithm on the pseudo-frame.

14. The event vision sensor of claim 13, wherein the lossy compression algorithm comprises Joint Image Experts Group compression, Advanced Video Coding Compression, High Efficiency Video Coding Compression, or any combination thereof.

15. A method of operating an event vision sensor comprising one or more event vision pixels, the method comprising: For each of the one or more event visual pixels— Detect events indicated in incident light received from an external scene within a timing window, wherein the events include UP events and DOWN events, wherein each of the UP events corresponds to a change in the incident light from darker to brighter, and each of the DOWN events corresponds to a change in the incident light from brighter to darker. Track the number of events detected within the timed window; and At or after the end of the timing window, the number of events detected by at least one of the one or more event visual pixels within the timing window is read out.

16. The method of claim 15, wherein: The number of events detected within the timed window includes: (a) the number of UP events detected within the timed window; and (b) the number of DOWN events detected within the timed window.

17. The method of claim 15, wherein: The number of events detected within the timed window includes the difference between (a) the number of UP events detected within the timed window and (b) the number of DOWN events detected within the timed window.

18. The method of claim 15, further comprising: For each of the individual event visual pixels in the one or more event visual pixels, the number of events detected within the timing window is read out only if the individual event visual pixel detects at least one event during the timing window.

19. The method of claim 15, further comprising forming a pseudo-frame corresponding to the timing window based at least in part on the number of events detected within the timing window by all or a subset of the one or more event visual pixels.

20. The method of claim 19, wherein: The pseudo-frame is the first pseudo-frame, and the subset is the first subset; Forming the first pseudo-frame includes forming the first pseudo-frame based at least in part on the number of UP events detected within the timing window by all or a subset of the one or more event visual pixels. The method further includes forming a second pseudo-frame corresponding to the timing window based at least in part on the number of DOWN events detected within the timing window by all or a second subset of the one or more event visual pixels.

21. The method of claim 19, further comprising performing frame-level compression on the pseudo-frame.

22. The method of claim 15, wherein tracking the number of events detected within the timing window comprises: The counter is reset at or before the start of the timing window; and The counter is used to count the number of events.

Images