Event vision sensors utilizing event data compression, including event vision sensors utilizing event data compression in pixels, and associated systems, devices, and methods

By using time aggregation circuit in event vision sensors to aggregate and compress event data, the problem of large amount of information in event vision sensors is solved, and more efficient data storage and transmission is achieved.

CN119094912AActive Publication Date: 2024-12-06OMNIVISION TECHNOLOGIES INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410593396.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-05
Filing Date
2024-05-14
Publication Date
2024-12-06
Estimated Expiration
2044-05-14

AI Technical Summary

Technical Problem

When an event vision sensor detects a large number of events, the amount of information recorded in each event is large, resulting in reduced efficiency. It is often detected that isolated events are noise, with spatial or temporal redundancy, affecting data storage and transmission efficiency.

Method used

The on-chip time aggregation circuit is used to compress the event data, and the events detected in the timing window are aggregated through a counter to form a pseudo-frame, and further compress it at the frame level to reduce the amount of recorded information.

Benefits of technology

The amount of event information is reduced, the recording time of memory is extended, the waiting time for data transmission and processing is reduced, and the data rate and power utilization efficiency are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094912A_ABST
    Figure CN119094912A_ABST
Patent Text Reader

Abstract

Disclosed herein are event vision sensors utilizing event data compression, including event vision sensors utilizing event data compression in pixels, and associated systems, devices, and methods. In one embodiment, an event vision sensor includes a plurality of event vision pixels. Each event visual pixel of the plurality of event visual pixels is configured to generate event data based on an event indicated in incident light received from an external scene, and includes a compression circuit configured to compress the event data before the event data is read out from the event visual pixel. Each compression circuit may include a time aggregation circuit configured to track a number of the events detected by the corresponding event visual pixel within a specified timing window. The compressed event data may be read out from the event vision sensor and used to generate a pseudo frame. The event vision sensor may optionally perform frame level compression on the pseudo frame.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to event vision sensors. For example, several embodiments of the present technology relate to event vision sensors that employ on-chip event data compression (eg, in-pixel event data compression). Background Art

[0002] Image sensors have become ubiquitous and are now widely used in digital cameras, cellular phones, security cameras, and medical, automotive, and other applications. As image sensors are integrated into a wider range of electronic devices, it is desirable to enhance their functionality, performance metrics, and the like in as many ways as possible (e.g., resolution, power consumption, dynamic range, etc.) both through device architecture design and image acquisition processing.

[0003] A typical image sensor operates in response to image light incident on the image sensor from an external scene. The image sensor includes an array of pixels having photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and generate an image charge upon absorption of the image light. The image charge generated by the pixel light can be measured as an analog output image signal on a column bit line that varies as a function of the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out from the column bit line as an analog image signal and converted into a digital value to provide information representing the external scene. Summary of the invention

[0004] One aspect of the present disclosure relates to an event vision pixel, comprising: a photosensor configured to generate a photocurrent in response to incident light received from an external scene; a photocurrent-to-voltage converter coupled to the photosensor to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to a difference detected over time in the voltage received from the photocurrent-to-voltage converter; an event generating comparator coupled to the difference detection circuit to detect events occurring within the external scene based on a comparison of the filtered and amplified signal with one or more thresholds; and a time aggregation circuit coupled to the event generating comparator to track an aggregated number of events detected by the event generating comparator within a specified timing window.

[0005] Another aspect of the present disclosure relates to an event vision sensor, comprising: a plurality of event vision pixels, wherein: each of the plurality of event vision pixels is configured to generate event data based at least in part on an event indicated in incident light received from an external scene, and each of the plurality of event vision pixels includes a compression circuit configured to compress the event data before the event data is read out from the event vision pixel.

[0006] Yet another aspect of the present disclosure relates to a method of operating an event vision sensor comprising one or more event vision pixels, the method comprising: for each individual event vision pixel of the one or more event vision pixels - detecting an event indicated in incident light received from an external scene within a timing window, and tracking the number of the events detected within the timing window. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Non-limiting and non-exhaustive embodiments of the present technology are described below with reference to the following figures, wherein like or similar reference numerals are used throughout to refer to like or similar components unless otherwise specified.

[0008] Figure 1 is a partial schematic block diagram of an event vision sensor configured in accordance with various embodiments of the present technology.

[0009] Figure 2 is a partial schematic block diagram illustrating an event vision pixel configured in accordance with various embodiments of the present technology.

[0010] Figure 3 is a partial schematic block diagram of a temporal aggregation circuit configured in accordance with various embodiments of the present technology.

[0011] Figure 4 is a diagram illustrating a method of event data compression according to various embodiments of the present technology.

[0012] Figure 5 is a partial schematic block diagram of another time aggregation circuit configured in accordance with various embodiments of the present technology.

[0013] Figure 6 is a diagram illustrating various embodiments according to the present technology Figure 5 A signal diagram of a method of operating a temporal aggregation circuit.

[0014] Figure 7 is a partial schematic block diagram of still another time aggregation circuit configured in accordance with various embodiments of the present technology.

[0015] Figure 8 is a diagram illustrating various embodiments according to the present technology Figure 7A signal diagram of a method of operating a temporal aggregation circuit.

[0016] Fig. 9 is a partial schematic block diagram of yet another time aggregation circuit configured in accordance with various embodiments of the present technology.

[0017] Fig.10 is a diagram illustrating another method of event data compression in accordance with various embodiments of the present technology.

[0018] Fig.11 is a partial schematic block diagram of an event signal processor configured according to various embodiments of the present technology.

[0019] Fig.12 is a flow chart illustrating a method of operating an event vision sensor in accordance with various embodiments of the present technology.

[0020] Those skilled in the art will appreciate that the elements in the various figures are illustrated for simplicity and clarity purposes and are not necessarily drawn to scale. For example, the dimensions of some elements in the various figures may be exaggerated relative to other elements to aid in understanding various aspects of the technology. In addition, common but well-known elements or methods that are useful or necessary in commercially feasible embodiments are generally not depicted in the drawings or described in detail below to avoid unnecessarily obscuring the description of various aspects of the technology. DETAILED DESCRIPTION

[0021] The present disclosure relates to event vision sensors. For example, several embodiments of the present technology relate to event vision sensors that employ on-chip (e.g., in-pixel) and / or frame-level event data compression. In the following description, specific details are set forth to provide a thorough understanding of various aspects of the present technology. However, one skilled in the art will recognize that the systems, devices, and techniques described herein may be practiced without one or more of the specific details set forth herein, or with other methods, components, materials, etc.

[0022] References in this specification to an "example" or "embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the example or embodiment is included in at least one example or embodiment of the present technology. Thus, the phrases "for example," "as an example," or "embodiment" used herein do not necessarily all refer to the same example or embodiment, and are not necessarily limited to the particular example or embodiment discussed. Furthermore, the features, structures, or characteristics of the present technology described herein may be combined in any suitable manner to provide further examples or embodiments of the present technology.

[0023] For ease of description, spatially relative terms (e.g., "below," "below," "above," "below," "above," "upper," "top," "bottom," "left," "right," "center," "middle," etc.) may be used herein to describe the relationship of an element or a feature relative to one or more other elements or features as illustrated in the figures. It should be understood that, in addition to the orientations described in the figures, spatially relative terms are intended to cover different orientations of the device or system when in use or operation. For example, if the device or system illustrated in the figures is rotated, turned, or flipped around a horizontal axis, an element or feature described as "below" or "below" or "below" one or more other elements or features may be oriented "above" one or more other elements or features. Therefore, the exemplary terms "below" and "below" are non-limiting and may encompass both the orientation of above and below. Additionally or alternatively, the device or system may be oriented in other ways as illustrated in the figures (e.g., rotated 90 degrees around a vertical axis, or in other orientations), and the spatially relative descriptors used herein are interpreted accordingly. In addition, it will also be understood that when an element is referred to as being “between” two other elements, it can be the only element between the two other elements, or one or more intervening elements may also be present.

[0024] Several terms are used throughout this specification. These terms will assume their ordinary meaning in the art to which they belong, unless otherwise specifically defined herein or the context in which they are used would clearly imply otherwise. It should be noted that in this document, element names and symbols are used interchangeably (e.g., Si and silicon); however, both have the same meaning.

[0025] A. Overview

[0026] Active pixel sensors (e.g., CMOS imaging systems) typically employ an array of active pixels with globally defined integration times. Thus, the active pixels in an active pixel sensor typically have the same integration time, and each pixel in the array is typically converted into a digital signal regardless of its content (e.g., regardless of whether the external scene captured by the pixel has changed since the last time the pixel was read out). In other words, the image data generated by the active pixels in, for example, a CMOS imager is read out in frames of known size regardless of whether there are events in the external scene.

[0027] In contrast, when a pixel captures a change (e.g., an event) in an external scene, an event vision sensor (e.g., an event-driven sensor or a dynamic vision sensor) reads out the pixel and / or converts the corresponding pixel signal into a digital signal. In other words, pixels of the event vision sensor that do not detect a change in the external scene are not read out, and / or the pixel signals corresponding to such pixels are not converted into digital signals. Therefore, each pixel of the event vision sensor can be independent of other pixels of the event vision sensor, and only the pixel that detects a change in the external scene needs to be read out, and / or its corresponding pixel signal needs to be converted into a digital signal or recorded (thereby saving power). In this way, the event vision sensor does not need to record the entire conventional image, and therefore does not have the burden of capturing and recording all the highly redundant information of a normal image frame by frame. As a result, the event vision sensor can be used to detect movement or motion in an external scene while enabling (i) the use of low data rates and (ii) achieving ultra-high frame rate or speed capabilities (e.g., as opposed to being used to capture / read out an entire frame of an image or video).

[0028] Even so, when an event vision sensor records events detected in an external scene, the event vision sensor typically records the location where each event was detected (e.g., the xy coordinates of the event vision pixel in an array of such pixels), the polarity of the photocurrent change for that event (e.g., brighter or darker), and / or the timing corresponding to when the event occurred or was detected. Therefore, for event vision pixels of a megapixel array, the event vision sensor typically records about 25 bits of information for each individual event detected by the event vision pixels of the megapixel array (e.g., ten bits of information for recording the x position of the corresponding event vision pixel of the megapixel array, ten bits of information for recording the y position of the corresponding event vision pixel, one bit of information for recording the polarity of the change corresponding to the event, and four bits of information for recording a timestamp indicating when that event was detected). When events are sparse and only a few events are detected by the event vision pixels of the array at once or within a short period of time, recording 25 bits of information for each individual event is certainly more efficient than recording an entire frame of an image or video in which only a few event vision pixels of the array have detected the event. However, recording 25 bits of information for each individual event can quickly reduce the relative efficiency of an event vision sensor compared to an active pixel sensor when a large number of event vision pixels detect events at once or detect several events within a short period of time.

[0029] In addition, the event vision sensor rarely detects and registers isolated events corresponding to legitimate activities in the external scene monitored by the event vision sensor. More commonly, that isolated event corresponds to noise. Instead, the activities in the external scene usually include some spatial or temporal redundancy. For example, when an individual moves across an external scene monitored by the event vision sensor, the same individual is detected in the events registered by several different event vision pixel groups across the array of such pixels in the event vision sensor. Therefore, when an individual moves across an external scene, there is often spatial redundancy in the information captured by the event vision sensor. As another example, when the event vision sensor monitors an external scene including a flashing yellow traffic light, no matter when the yellow light changes (e.g., from on to off and / or from off to on), the same group of event vision pixels of the event vision sensor will detect the event. In other words, the same group of event vision pixels will detect multiple instances of the same activity over time. In this way, when the traffic light changes in a short amount of time, there is some temporal redundancy in the information captured by the event vision pixels.

[0030] Given the problems discussed above, the present technology provides an on-chip compression solution that can be used to reduce the amount of information used to record events detected by an event vision sensor. For example, several embodiments of the present technology relate to an event vision sensor that includes an event vision pixel with a time aggregation circuit that is capable of performing pixel-in-pixel compression of event information (e.g., time-dependent aggregation). As a specific example, several event vision pixels of the present technology include a counter that is configured to count the number of events (e.g., the number of UP events, the number of DOWN events, or the difference between the number of UP events and the number of DOWN events) detected by the corresponding event vision pixel within a specified time period. Subsequently, the counts maintained by the event vision pixel can be read out and combined to form one or more pseudo-frames corresponding to the specified time period. In some embodiments, the event vision sensor can then optionally perform frame-level compression on the pseudo-frame (e.g., to further reduce the amount of information used to record events detected by the event vision pixel within the specified time period). Therefore, instead of reading out events from event vision pixels and recording them individually, an event vision sensor configured according to various embodiments of the present technology is configured to: (i) encode events detected by the same event vision pixel and occurring within the same timing window as a group using in-pixel compression circuitry; and (ii) read out counts corresponding to the group as event data that can be used to form a pseudo frame. As a result, an event vision sensor configured according to various embodiments of the present technology can read out event data from event vision pixels and record the event data using a smaller number of bits than is typically used by event vision sensors that lack an on-chip compression solution.

[0031] In other words, the present technology provides an on-chip compression solution (e.g., an in-pixel compression solution and / or a frame-level compression solution) for reducing the amount of information required to record an event. In turn, the amount of memory used to record such events is reduced compared to an event vision sensor that lacks an on-chip compression solution, which can result in longer recording times for a fixed amount of memory because a larger amount of event data can be stored to such memory before the memory becomes full. Furthermore, such an on-chip compression solution provided by the present technology is expected to reduce latency, enable the use of even lower data rates, and utilize less power compared to event vision sensors that lack an on-chip compression solution, and / or compared to imaging systems that perform compression off-chip (e.g., external to the imaging system's event vision sensor).

[0032] B. Selected embodiments of event vision sensors and associated systems, devices, and methods utilizing event data compression Examples

[0033] Figure 1 1 is a partial schematic block diagram of an event vision sensor 100 configured in accordance with various embodiments of the present technology. As shown, the event vision sensor 100 includes an array 102 of event vision pixels 110, a row control circuit system 104, a column control circuit system 106, an event signal processor 108, and a transmitter 116. In some embodiments, the event vision sensor 100 may additionally include a memory 112. As an example, the memory 112 is stored in a plurality of memory cells. Figure 1 1 is shown as a FIFO buffer 112. In other embodiments, the memory 112 may be another suitable type of buffer or memory, and / or may be omitted or located elsewhere in the event vision sensor 100 (e.g., within the event signal processor 108 or the transmitter 116, or after the transmitter 116).

[0034] The array 102 includes a plurality of event vision pixel circuits 110 arranged in rows and columns ( Figure 1100). As discussed in more detail below, the event vision pixel 110 is configured to capture changes (e.g., events) in an external scene. To this end, each event vision pixel 110 may include: (i) a photosensor, such as a photodiode, configured to photogenerate charge or photocurrent in response to incident light received from the external scene; (ii) a photocurrent-to-voltage converter coupled to the photosensor to convert the photocurrent generated by the photosensor into a voltage; and (iii) a filter amplifier coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to the voltage received from the photocurrent-to-voltage converter. The event vision pixel may further include a threshold comparison circuit or stage to generate and receive a handshake signal in response to an event asynchronously detected in incident light received from the external scene. Alternatively, the threshold comparison circuit may be included in circuitry peripheral to or external to the event vision pixel of the array 102 (e.g., in the event signal processor 108), such as within the column readout circuitry. Event vision pixels configured in accordance with the present technology may further include temporal aggregation circuitry or stages that facilitate encoding together events that occur or are detected within a specified time window, thereby compressing the total amount of information required to record the events detected by the event vision pixel.

[0035] Figure 1 The row control circuitry 104 and the column control circuitry 106 are used to respectively control the rows and columns of event vision pixels 110 in the array 102. For example, the row control circuitry 104 and / or the column control circuitry 106 may be configured to reset specific (e.g., individual one or more rows) event vision pixels 110 of the array 102 and / or read out (e.g., individual one or more rows) event vision pixels 110 from the array 102 (e.g., along corresponding column bit lines connected to the event vision pixels 110).

[0036] The event data read out from the event vision pixels 110 of the array 102 may be transmitted to the event signal processor 108 of the event vision sensor 100 for processing. As discussed in more detail below, the processing performed by the event signal processor 108 may optionally include compressing pseudo frames of the event data read out from the event vision pixels 110 of the array 102. The event data processed by the event signal processor 108 may be provided to a transmitter 116 for transmitting the event data out of the event vision sensor 100, such as to a receiver (not shown) of a corresponding imaging system. Additionally or alternatively, all or a subset of the event data may be stored in the memory 112 (e.g., before or after being provided to the event signal processor 108 and / or the transmitter 116).

[0037] Figure 22 is a partially schematic block diagram illustrating an example of an event vision pixel 210 configured according to various embodiments of the present technology. It should be understood that the illustrated event vision pixel circuit 210 may be included in Figure 1 1 , or an example of another event vision pixel circuit of the present technology. As shown, the event vision pixel 210 includes a photodiode 231 configured to photogenerate charge or photocurrent in response to incident light 250 received from an external scene. The photodiode 231 is coupled to a logarithmic amplifier 232, which is configured to convert the photocurrent generated by the photodiode 231 into a voltage. In various examples, the logarithmic amplifier 232 is configured to generate a voltage by transducing the instantaneous photocurrent received from the photodiode 231. A difference detection amplifier 233 is coupled to the logarithmic amplifier 232 to generate a filtered and amplified signal in response to a difference detected in the voltage received from the logarithmic amplifier 232. In one example, the difference detection amplifier 233 is configured to compare the instantaneous logarithmic intensity of the voltage output of the logarithmic amplifier 232 with a reference level based on a reset condition or a last event.

[0038] The event generating comparator 234 is coupled to the difference detection amplifier 233 to compare the filtered and amplified signal received from the difference detection amplifier 233 with a threshold value, thereby asynchronously detecting an event occurring in the external scene in response to the incident light 250. In one example, the event generating comparator 234 is configured to discern whether the signal difference is significant enough to trigger an event. In the illustrated embodiment, the event generating comparator 234 includes a first comparator 234a, which is configured to detect whether the signal difference corresponds to an "UP" event (e.g., a change in the light intensity incident on the photodiode 231 from darker to brighter and greater than a threshold value). The event generating comparator 234 further includes a second comparator 234b, which is configured to detect whether the signal difference corresponds to a "DOWN" event (e.g., a change in the light intensity incident on the photodiode 231 from brighter to darker and greater than a threshold value).

[0039] Figure 2 The event visual pixel 210 further includes a temporal aggregation circuit or stage 235 (also referred to herein as a "pixel compression circuit" or "compression circuit") coupled to the output of the event generating comparator 234. Figures 3 to 10As discussed in greater detail, the temporal aggregation circuit 235 may include one or more counters configured to count events (UP events and / or DOWN events) detected by the event generating comparator 234 during a specified timing window. At the end of the timing window, the counts maintained by the one or more counters may be read out from the event vision pixel 210 onto the column or bit line 238 as event data. The event data may include: (a) a count representing the number of UP events detected by the corresponding event vision pixel during the timing window; (b) a count representing the number of DOWN events detected by the corresponding event vision pixel during the timing window; and / or (c) a count representing the difference between the number of UP events detected by the corresponding event vision pixel during the timing window and the number of DOWN events detected by the corresponding event vision pixel during the timing window.

[0040] As referenced below Figure 4 and 10As discussed in more detail, the temporal aggregation circuit 235 facilitates encoding several events occurring within the same timing window together, rather than individually. For example, in the case of a megapixel array, rather than individually reading out the events detected by the event vision pixels 210 and individually recording each of those events using approximately 25 bits of information (e.g., ten bits for recording the x-position of the event vision pixel 210 in the megapixel array, ten bits for recording the y-position of the event vision pixel 210 in the megapixel array, one bit for recording the polarity of the change corresponding to each individual event, and four bits for recording a timestamp indicating when the event vision pixel 210 detected each individual event), the events detected by the event vision pixels 210 during the same timing window may be read out and recorded as a group using a smaller number of bits. More specifically, at or after the end of a given timing window, counts representing the following may be read out from the event vision pixel 210 as event data: (i) the number of UP events detected by the event vision pixel 210 during the timing window; (ii) the number of DOWN events detected by the event vision pixel 210; and / or (iii) the difference between the number of UP events detected during the timing window and the number of DOWN events detected during the timing window. Event data may then be recorded using: (a) a single instance of ten bits reflecting the x-position of the event vision pixel 210 in the megapixel array; (b) a single instance of ten bits reflecting the y-position of the event vision pixel 210 in the megapixel array; (c) a specified number of bits (e.g., two to four bits) reflecting the count value; and (d) a specified number of bits (e.g., four bits) identifying the corresponding timing window (e.g., via a timestamp reflecting the start or end of the timing window, a timing window sequence identifier, a frame count number, etc.). In other words, the temporal aggregation circuit 235 compresses the amount of information required to record events detected by the event vision pixel 210 by using a single x-position data entry, a single y-position data entry, and a single timestamp data entry for all events detected by the event vision pixel 210 within the same timing window.

[0041] Reference again Figure 2 , the event vision pixel 210 optionally includes a scanner and readout logic 236 coupled to the output of the event generating comparator 234 via a logic gate 237. In the illustrated embodiment, the logic gate 237 is an "OR" gate and is configured to trigger a latch in the scanner and readout logic 236 regardless of when the event generating comparator 234 detects an event (e.g., an UP event or a DOWN event). Thus, the triggering of the latch indicates that the event vision pixel 210 has detected at least one event during a given timing window. Although Figure 22, but the scanner and readout logic 236 may be coupled to the row control circuitry (e.g., Figure 1 23) and can be used to request readout of the event vision pixel 210 regardless of when the latch is triggered. In this way, the scanner and readout logic 236 and the logic gate 237 can be utilized to ensure that the event vision pixel 210 is read out only when the event vision pixel 210 has detected at least one event within a given timing window. In other words, the scanner and readout logic 236 and the logic gate 237 can be utilized to ensure that when the counts maintained by the event vision pixel 210 indicate that the event vision pixel 210 has not detected at least one event during the timing window, those counts are not read out from the event vision pixel 210.

[0042] In other embodiments, the scanner and readout logic 236 of the event vision pixel 210 may include a first latch coupled to the output of the first comparator 234a and a second latch different from the first latch and coupled to the output of the second comparator 234b. In these embodiments, the logic gate 237 may be omitted. The first latch may be triggered when the event vision pixel 210 detects at least one UP event during a given timing window, and the second latch may be triggered when the event vision pixel 210 detects at least one DOWN event during a given timing window. Including separate latches for UP events and DOWN events in the scanner and readout logic 236 may provide greater granularity or control over the readout of different event counts maintained by the time aggregation circuit 235. For example, using separate latches may facilitate reading out an UP event count only when the event vision pixel 210 detects at least one UP event during a given timing window (e.g., regardless of whether the event vision pixel 210 detects a DOWN event during the given timing window), and reading out a DOWN event count only when the event vision pixel 210 detects at least one DOWN event during a given timing window (e.g., regardless of whether the event vision pixel 210 detects an UP event during the given timing window).

[0043] For the sake of clarity and understanding, Figure 22 is a brief overview of the operation of the event vision pixel 210 illustrated in FIG. The function of the event vision pixel 210 is such that when an event occurs in the external scene, that event is indicated as a rapid or sudden change in intensity or brightness in the incident light 250 received by the photodiode 231. In other words, if the external scene is static and no event occurs, the brightness of the incident light 250 remains substantially unchanged. In this way, the photocurrent generated by the photodiode 231 remains substantially constant. However, if an event occurs in the external scene (e.g., movement, lighting change, albedo, emittance, etc.), then an asynchronous rapid or sudden change in the brightness of the incident light 250 is used to indicate that event. The change in brightness can be from darker to brighter or from brighter to darker. Therefore, there is an asynchronous change or δ in the photocurrent generated by the photodiode 231. The change or δ in the photocurrent is converted to a voltage by the logarithmic amplifier 232, filtered and amplified by the difference detection amplifier 233, detected by the event generation comparator 234, and counted by the time aggregation circuit 235. When the event is the first event to occur within a given timing window, the event may trigger a latch in the scanner and readout logic 236 via logic gate 237, thereby notifying the row control circuitry (not shown) that the event vision pixel 210 has detected at least one event within the given timing window and requesting readout (e.g., at or after the end of the timing window) of the event vision pixel 210. The above process may be repeated for each event that occurs within a given timing window.

[0044] At or after the end of a given timing window, and assuming that the event vision pixel 210 has detected at least one event within the given timing window (e.g., as indicated by the triggering of a latch in the scanner and readout logic 236), the event data may be read out from the event vision pixel 210 in sequence or simultaneously with the event data generated by other event vision pixels of the array including the event vision pixel 210. The event data detected by the event vision pixels of the array may be read out from the array asynchronously and / or in an order different from the order in which the event data was generated. The digital timestamp, sequence identifier, frame count number, etc. associated with each timing window may help ensure that events detected by different event vision pixels but corresponding to the same timing window are related / associated with each other (e.g., for storage or reconstruction). On the other hand, when the event vision pixel 210 does not detect at least one event within the given timing window (e.g., as indicated by an untriggered latch in the scanner and readout logic 236), the event data is not read out from the event vision pixel 210 within the given timing window.

[0045] Therefore, it should be understood that event vision pixel circuits (e.g., Figure 2 The event vision pixel circuit 210) of the event vision sensor (e.g., Figure 1The event vision sensor 110 does not need to record the entire conventional image, and therefore does not have the burden of having to capture and record all the highly redundant information of a normal image frame by frame. Instead, in various embodiments, the event vision sensor may only record events and / or event counts. For example, Figure 1 The visual sensor 100 may record the location where the event is detected (e.g., Figure 1 The event vision sensor 100 may include the xy coordinates of the event vision pixels 110 in the array 102 of FIG. 1 ), the count of UP and / or DOWN events detected during a given timing window (or the difference between such counts), and / or the timing or identifier corresponding to a given timing window. In other words, the event vision sensor 100 may be used to detect movement or motion in an external scene (e.g., as opposed to being used to capture / read out an entire frame of an image or video), and may facilitate encoding events in groups, thereby compressing the total amount of information required to record events detected by the event vision pixels 110 of the array 102, thereby enabling the use of lower data rates and enabling ultra-high frame rate or speed capabilities.

[0046] Figure 3 FIG. 3 is a partial schematic block diagram of a time aggregation circuit 335 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 335 may be included in Figure 2 , or an example of a time aggregation circuit 235 in an event vision pixel 210 of the present technology. As shown, the time aggregation circuit 335 includes a first counter 335a and a second counter 335b. In some embodiments, the first counter 335a is a first ripple counter, and / or the second counter 335b is a second ripple counter. The first counter 335a is configured to: (a) receive a signal UP from an event generation comparator of a corresponding event vision pixel indicating when the corresponding event vision pixel detects an UP event; and (b) count the number of times the signal UP is asserted during a timing window. Similarly, the second counter 335b is configured to: (a) receive a signal DOWN from an event generation comparator of a corresponding event vision pixel indicating when the corresponding event vision pixel detects a DOWN event; and (b) count the number of times the signal DOWN is asserted during a timing window.

[0047] For the purpose of clarity and understanding, reference will now be made to Figure 2 and 310. The operation of the time aggregation circuit 335 is discussed below. The first counter 335a may be configured to receive the output (signal UP) of the first comparator 234a in the event generating comparator 234 in the event visual pixel 210, and the second counter 335b may be configured to receive the output (signal DOWN) of the second comparator 234b in the event generating comparator 234. Continuing with this example, the first counter 335a and the second counter 335b may be reset at or before the start of the timing window. Whenever the first comparator 234a detects an UP event, the first comparator 234a may assert or pulse the signal UP. Then, the first counter 335a may update (e.g., increment) a counter value representing the number of times the first comparator 234a asserts or pulses the signal UP during the timing window. The counter value maintained by the first counter 335a is referred to herein as an UP event count. Similarly, whenever the second comparator 234b detects a DOWN event using the second comparator 234b, the second comparator 234b may assert or pulse the signal DOWN. Then, the second counter 335b may update (e.g., increment) a counter value representing the number of times the second comparator 234b asserts or pulses the signal DOWN during the timing window. The counter value maintained by the second counter 335b is referred to herein as a DOWN event count.

[0048] Assuming that the event vision pixel 210 detects at least one event (UP or DOWN) during the timing window, the latch in the scanner and readout logic 236 of the event vision pixel 210 will be triggered via the logic gate 237, thereby signaling to the corresponding event vision sensor (e.g., the corresponding row control circuitry) that the event vision pixel 210 should be read out. Therefore, at or after the end of the timing window, the event vision sensor can read out the UP event count and the DOWN event count onto the column bit line 338. The UP event count and the DOWN event count can be read out from the event vision pixel 210 in any order relative to each other and / or relative to other event counts maintained by other event vision pixels of the event vision sensor. On the other hand, when the event vision pixel 210 does not detect at least one event (UP or DOWN) during the timing window, the latch in the scanner and readout logic 236 will not be triggered via the logic gate 237. Thus, the event counts maintained by the first counter 335a and the second counter 335b within the corresponding timing window will not be read out to the column bit line 338 at or after the end of the timing window. Regardless of whether the event vision pixel 210 detects at least one event during the timing window, the event counts maintained by the first counter 335a and the second counter 335b may be reset at or before the beginning of the next timing window.

[0049] Figure 4is a diagram illustrating an overview of event data compression that may be performed using a temporal aggregation circuit of event vision pixels configured in accordance with various embodiments of the present technology. Figure 4 A plot 440 includes UP events and DOWN events detected by an event vision pixel during a timing window 441 between time t1 and time t2. More specifically, the plot 440 illustrates that the event vision pixel detects eight (8) UP events and ten (10) DOWN events during the timing window 441 and in the order shown. As discussed above, a first counter of the event vision pixel (e.g., Figure 3 The UP events may be counted using a first counter 335a of FIG. 1 and a second counter (eg, Figure 3 The second counter 335b) counts the DOWN events. The UP event count maintained by the first counter and the DOWN event count maintained by the second counter can be read out from the event vision pixel at the end of the timing window 441 or after (for example, at or after time t2). Note that the UP event count and the DOWN event count only reflect the number of UP events and / or DOWN events detected by the event vision pixel during the timing window 441, respectively. In other words, the counts only reflect the aggregate number of UP events and DOWN events detected by the event vision pixel within the timing window 441 between time t1 and time t2, respectively. Therefore, during the timing window 441, the counter value maintained by the counter does not record, track or otherwise reflect the exact timing of each individual event detected by the event vision pixel. Therefore, this in-pixel compression of event data performed by the event vision pixel can be considered to be a lossy form of compression in which the temporal dependency information of individual events is lost.

[0050] In some embodiments, the UP event count and / or DOWN event count read out from the event vision pixel may be combined with other event counts read out from other event vision pixels of the event vision sensor to form a pseudo frame. Figure 4 As shown in , the UP event count of the timing window 441 corresponding to the drawing 440 can be combined with other UP event counts read from other event visual pixels within the same timing window 441 to form a first pseudo frame 450a. Similarly, the DOWN event count of the timing window 441 corresponding to the drawing 440 can be combined with other DOWN event counts read from other event visual pixels within the same timing window 441 to form a second pseudo frame 450b that is different from the first pseudo frame 450a. Another option is that both the UP event count and the DOWN event count of the timing window 441 corresponding to the drawing 440 can be combined in a single pseudo frame together with other UP event counts and DOWN event counts read from other event visual pixels within the same timing window 441. At this time, as described below with reference to Fig.11 As discussed in greater detail, the event vision sensor may optionally perform frame level compression on the first pseudo frame 450a and / or the second pseudo frame 450b.

[0051] Frames of event counts read out from the event vision pixels of the array within different timing windows are referred to herein as "pseudo-frames" because the event data contained in each pseudo-frame represents the number of events detected by the event vision pixels within the specified timing window. Such event data is proportional to the contrast change or information change in the external scene. On the other hand, by convention, image data frames are associated with an exposure time, and the image data contained in such frames represents the amount of photons / electrons generated by the light incident on the photosensor of each pixel during the exposure time.

[0052] Reference again Figure 4 , a plot 445 is shown to illustrate reconstruction of event data from pseudo frames 450a and 450b. More specifically, during reconstruction, event data corresponding to UP events detected by the event visual pixel during the timing window 441 can be reconstructed using the first pseudo frame 450a, and event data corresponding to DOWN events detected by the event visual pixel during the timing window 441 can be reconstructed using the second pseudo frame 450b. Because the time-dependent information about when exactly each of the individual events occurred during the timing window 441 is lost during compression in the pixels of the event data, assumptions are made during reconstruction about when each of the events occurred within the timing window 441. In the illustrated example, it is assumed that the event visual pixel detects eight UP events at uniform intervals across the timing window 441, and the event visual pixel detects ten DOWN events at uniform intervals across the timing window 441. Therefore, during reconstruction, eight UP events can be evenly distributed across the timing window 441, and ten DOWN events can be evenly distributed across the timing window 441.

[0053] Other assumptions about the distribution of UP events and / or DOWN events across the time window are certainly possible and within the scope of the present technology. For example, it may be assumed that the event visual pixel detects UP events and / or DOWN events exponentially across the timing window 441. In addition, although the same assumption is made for both UP events and DOWN events across the time window 441 in the illustrated example, different assumptions may be made for UP events and DOWN events during reconstruction. Additionally or alternatively, multiple assumptions may be made for UP events and DOWN events during reconstruction. For example, it may be assumed that DOWN events are detected at uniform intervals by the event visual pixel during the first half of the timing window, and then the DOWN events are detected exponentially by the event visual pixel during the second half of the timing window. In addition, for the same pseudo-frame, the assumptions made during the reconstruction of the event data corresponding to one event visual pixel may be, but need not be, the same as the assumptions made during the reconstruction of the event data corresponding to another event visual pixel. Furthermore, for different pseudo frames corresponding to the same event visual pixel (eg, different UP event pseudo frames or different DOWN event pseudo frames), the assumptions made during reconstruction of event data from those different pseudo frames may, but need not, be the same across different pseudo frames.

[0054] Figure 5 is a partial schematic block diagram of another time aggregation circuit 535 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 535 may be included in Figure 2 2 is an example of a temporal aggregation circuit 235 in an event visual pixel 210 of the present invention, or an example of another temporal aggregation circuit of the present technology. As shown, the temporal aggregation circuit 535 includes a first counter 535a (e.g., a first analog counter) and a second counter 535b (e.g., a second analog counter). The first counter 535a includes a capacitor 563a with a capacitance C1. The top plate of the capacitor 563a generates a comparator (e.g., Figure 2 The signal UP received by the first comparator 234a) is selectively coupled to the current source 561a via the switch 532a (e.g., a transistor). The top plate of the capacitor 563a is further (a) selectively coupled to the reference voltage V via the switch 564a (e.g., a transistor) based on the reset signal RES. ref , and (b) is coupled to the input of a buffer or amplifier 565a. The bottom plate of capacitor 563a is grounded. In addition, the output of amplifier 565a reads out a signal R based on an UP event. U Selectively coupled to column bit line 538 via switch 566a (eg, transistor).

[0055] Similarly, the second counter 535b includes a capacitor 563b having a capacitance C2. The top plate of the capacitor 563b generates a comparator (e.g., Figure 2 The signal DOWN received by the second comparator 234b) is selectively coupled to the current source 561b via the switch 532b (e.g., a transistor). The top plate of the capacitor 563b is further (a) selectively coupled to the reference voltage V via the switch 564b (e.g., a transistor) based on the reset signal RES. ref , and (b) is coupled to the input of a buffer or amplifier 565b. The bottom plate of capacitor 563b is grounded. In addition, the output of amplifier 565b reads out a signal R based on a DOWN event. D Selectively coupled to column bit line 538 via switch 566b (eg, transistor).

[0056] Figure 6 is a diagram illustrating various embodiments according to the present technology Figure 5 An exemplary signal diagram 668 of a method of operating the temporal aggregation circuit 535 is shown. Figure 5 and 6 , the reset signal RES is asserted at time t0 to selectively couple (a) the top plate of the capacitor 563a of the first counter 535a and the top plate of the capacitor 563b of the second counter 535b to (b) the reference voltage V ref The top plates of capacitors 563a and 563b are coupled to a reference voltage V ref The voltages on capacitors 563a and 563b are reset to known values. Although not shown, one or more latches in the scanner and readout logic corresponding to the event visual pixel (e.g., as described above with reference to Figure 2 The discussed event vision pixel 210 scanner and latches in the readout logic 236 may be reset at time t0 or at another timing at or before time t1.

[0057] Then, the reset signal RES is Figure 6 is deasserted at time t1 shown in FIG. 1 , thereby pulling the top plates of capacitors 563a and 563b from the reference voltage V ref The first counter 535a and the second counter 535b are decoupled and mark the beginning of the timing window between time t1 and time t2, during which the first counter 535a and the second counter 535b respectively count the number of UP events and DOWN events detected by the corresponding event vision pixel. In the illustrated example, during the timing window between time t1 and time t2, the corresponding event vision pixel first detects one UP event and then detects two DOWN events. When the corresponding event vision pixel detects an UP event, the event vision pixel (e.g., the event vision pixel's event generating comparator, e.g., Figure 2 The first comparator 234a of the first counter 535a delivers the signal UP in a pulsed manner. In turn, the top plate of the capacitor 563a of the first counter 535a is coupled to the current source 561a for the duration of the pulse, thereby reducing the voltage at the capacitor 563a by an amount proportional to the duration of the pulse. In some embodiments, one or more latches in the scanner and readout logic (e.g., scanner and readout logic 236) of the corresponding event vision pixel may be triggered after detecting the UP event to indicate that the corresponding event vision pixel has detected at least one event during the timing window, and thereby requesting that the corresponding event vision pixel be read out at or after the end of the timing window.

[0058] Similarly, when the corresponding event visual pixel detects Figure 6 At the first DOWN event illustrated in FIG. , the event visual pixel (eg, the event visual pixel) generates a comparator, such as Figure 2 The second comparator 234b of the second counter 535b delivers the signal DOWN in a pulsed manner. Then, the top plate of the capacitor 563b of the second counter 535b is coupled to the current source 561b for the duration of the pulse, thereby reducing the voltage at the capacitor 563b by an amount proportional to the duration of the pulse. This process is performed when the event visual pixel detects Figure 6 , is repeated when the second DOWN event illustrated in FIG. 5 is performed, thereby further reducing the voltage at capacitor 563b.

[0059] As discussed above, the current timing window ends at time t2. Figure 6 As shown in, the signal R U At time t3, the first counter 535a is asserted (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of the amplifier 565a of the first counter 535a to the column bit line 538. Subsequently, the first voltage value (representing the aggregate number of UP events detected by the event vision pixel during the timing window) is read out onto the column bit line 538. Then, the signal R U is deasserted at time t4 to decouple the output of amplifier 565a from column bit line 538. Thereafter, signal R D At time t5, the signal R is asserted (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of the amplifier 565b of the second counter 535b to the column bit line 538. Subsequently, the second voltage value (representing the aggregate number of DOWN events detected by the event vision pixel during the timing window) is read out onto the column bit line 538. Then, the signal R D 565b is de-asserted at time t6 to decouple the output of amplifier 565b from column bit line 538. Reset signal RES may then be asserted at time t7 in anticipation of the start of the next timing window.

[0060] Despite Figure 6 In the embodiment illustrated in , the first voltage value from the first counter 535a is read out before the second voltage value from the second counter 535b, but in other embodiments of the present technology, the second voltage value may be read out before the first voltage value. Additionally or alternatively, the first counter 535a may be coupled to a different column bit line than the second counter 535b. In these embodiments, the first voltage value may be read out before, after, or at the same time as the second voltage value is read out.

[0061] There may be a mismatch between the capacitance C1 of capacitor 563a and the capacitance C2 of capacitor 563b. Additionally or alternatively, in (i) the current I provided by current source 561a to the top plate of capacitor 563a a and (ii) the current I supplied to the top plate of capacitor 563b by current source 561b b There may be a mismatch between the capacitors C1 and C2 and / or the current I . Such a mismatch may affect the voltage values ​​read from the first counter 535a and / or the second counter 535b. Therefore, in some embodiments, the event vision sensor may perform a calibration technique (e.g., at the event signal processor) on the voltage values ​​read from the first counter 535a and / or the second counter 535b to account for (a) the mismatch between the capacitors C1 and C2 and / or (b) the current I a with I b The mismatch between .

[0062] Figure 7 7 is a partial schematic block diagram of another time aggregation circuit 735 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 735 can be included in Figure 2 2 is an example of a temporal aggregation circuit 235 in an event visual pixel 210 of the present invention, or an example of another temporal aggregation circuit of the present technology. As shown, the temporal aggregation circuit 735 includes a first counter 735a (e.g., a first analog counter) and a second counter 735b (e.g., a second analog counter). The first counter 735a includes (i) a first capacitor 771a having a capacitance C1 and (ii) a second capacitor 772a having a capacitance C2. In some embodiments, the second capacitance C2 may be greater than the first capacitance C1. In other embodiments, the second capacitance C2 may be equal to or less than the first capacitance C1.

[0063] The top plate of the second capacitor 772a is selectively coupled to a reference voltage V via a switch 775a (eg, a transistor) based on a reset signal RES. RESThe top plate of the second capacitor 772a is further (a) coupled to the input of a buffer or amplifier 776a and (b) coupled to a comparator (eg, Figure 2 The signal UP received by the first comparator 234a) is selectively coupled to the top plate of the first capacitor 771a via a switch 774a (eg, a transistor). The top plate of the first capacitor 771a is selectively coupled to the top plate of the first capacitor 771a based on the signal V br The voltage is further selectively coupled to a reference voltage V via a switch 773a (eg, a transistor). R In some embodiments, the reference voltage V R It is equivalent to less than the reference voltage V RES In other embodiments, the reference voltage V R Can be equivalent to greater than or equal to the reference voltage V RES The bottom plates of the first capacitor 771a and the second capacitor 772a are grounded. In addition, the output of the amplifier 776a is based on the UP event read signal R U Selectively coupled to column bit line 738 via switch 777a (eg, transistor).

[0064] Similarly, the second counter 735b includes (i) a first capacitor 771b having a capacitance C1 and (ii) a second capacitor 772b having a capacitance C2. In some embodiments, the second capacitance C2 may be greater than the first capacitance C1. In other embodiments, the second capacitance C2 may be equal to or less than the first capacitance C1. The top plate of the second capacitor 772b is selectively coupled to a reference voltage V via a switch 775b (e.g., a transistor) based on a reset signal RES. RES The top plate of the second capacitor 772b is further (a) coupled to the input of a buffer or amplifier 776b and (b) coupled to a comparator (e.g., Figure 2 The signal DOWN received by the second comparator 234b) is selectively coupled to the top plate of the first capacitor 771b via a switch 774b (eg, a transistor). The top plate of the first capacitor 771b is further coupled based on the signal V br is selectively coupled to a reference voltage V via a switch 773b (eg, a transistor). R Signal V br may be used to selectively couple the top plate of the first capacitor 771a of the first counter 735a to a reference voltage V R The signal V br The first capacitor 771b and the second capacitor 772b of the second counter 735b are connected to the ground. In addition, the output of the amplifier 776b is based on the DOWN event read signal R DSelectively coupled to column bit line 738 via switch 777b (eg, transistor).

[0065] Figure 8 is a diagram illustrating various embodiments according to the present technology Figure 7 An exemplary signal diagram of a method of operating the time aggregation circuit 735 is shown in FIG. Figure 7 and 8 , the reset signal RES is asserted at time t0 to selectively couple (a) the top plate of the second capacitor 772a of the first counter 735a and the top plate of the second capacitor 772b of the second counter 735b to (b) the reference voltage V RES In addition, the signal V br is asserted at time t0 to selectively couple (a) the top plate of the first capacitor 771a of the first counter 735a and the top plate of the first capacitor 771b of the second counter 735b to (b) the reference voltage V R Although Figure 8 , but in other embodiments, signal V br It may be asserted before or after the reset signal RES is asserted. Although not shown, one or more latches in the scanner and readout logic corresponding to the event visual pixel (e.g., as described above with reference to Figure 2 The discussed event vision pixel 210 scanner and latches in the readout logic 236 may be reset at time t0 or at another timing at or before time t1.

[0066] Then, the reset signal RES and the signal V br exist Figure 8 is deasserted at time t1 shown in FIG. 1 , thereby (i) pulling the top plates of capacitors 771a and 771b from the reference voltage V R decoupling, (ii) the top plates of capacitors 772a and 772b are connected from the reference voltage V RES decoupled, and (iii) marking the start of a timing window between time t1 and time t2, during which the first counter 735a and the second counter 735b count the number of UP events and DOWN events detected by the corresponding event vision pixel, respectively. In the illustrated example, during the timing window between time t1 and time t2, the corresponding event vision pixel first detects one UP event and then detects two DOWN events. When the corresponding event vision pixel detects an UP event, the event vision pixel (e.g., the event vision pixel's event generating comparator, e.g., Figure 2The first comparator 234a of the first counter 735a delivers the signal UP in a pulsed manner. Then, the top plate of the first capacitor 771a of the first counter 735a is coupled to the top plate of the second capacitor 772a via the switch 774a for the duration of the pulse. When the top plates of the first capacitor 771a and the second capacitor 772a are coupled together, the charge is shared among the capacitors 771a and 772a, and the voltage at the second capacitor 772a drops immediately before the switch 774a is activated based at least in part on the reference voltage V at the first capacitor 771a. R Then, the signal UP is deasserted to decouple the top plates of capacitors 771a and 772a from each other. Thereafter, the signal V br is asserted to activate switch 773a and restore the voltage at the first capacitor 771a to the reference voltage V R After that, the signal V br In some embodiments, one or more latches in the scanner and readout logic (e.g., scanner and readout logic 236) of the corresponding event vision pixel may be triggered after detecting the UP event to indicate that the corresponding event vision pixel has detected at least one event during the timing window and thereby requesting that the corresponding event vision pixel be read out at or after the end of the timing window.

[0067] Although not shown, the temporal aggregation circuit 735 and / or the event vision pixel 210 may include a circuit for controlling the br The timing circuitry for determining when the signal V is asserted after the signal UP or the signal DOWN is asserted. For example, the temporal aggregation circuit 735 and / or the event vision pixel 210 may include a logic gate (e.g., an “OR” logic gate) coupled to the output of the event generating comparator 234. Regardless of when the event generating comparator 234 detects an event, the output of the logic gate may be used to assert the signal V br The delay element may be coupled to the output of the logic gate and configured to introduce a delay at the output such that switches 773a and / or 773b are not activated until after the signal UP or the signal DOWN has been pulsed to activate switch 774a or switch 774b, respectively, thereby coupling together the top plates of capacitors 771 and 772 of the corresponding counter 735. In other words, the delay element may be configured such that the signal V br Not asserted at the same time that signal UP or signal DOWN is asserted.

[0068] When the corresponding event visual pixel detects Figure 8 At the first DOWN event illustrated in FIG. , the event visual pixel (eg, the event visual pixel) generates a comparator, such as Figure 2The second comparator 234b of the second counter 735b) pulses the signal DOWN, thereby activating the switch 774b and coupling the top plate of the first capacitor 771b of the second counter 735b to the top plate of the second capacitor 772b. Subsequently, (i) the charge is shared between the capacitors 771b and 772b, and (ii) the voltage at the second capacitor 772b drops immediately before the switch 774b is activated based at least in part on the reference voltage V at the first capacitor 771b. R Then, the signal DOWN is deasserted to decouple the top plates of capacitors 771b and 772b. Thereafter, the signal V br is asserted to activate switch 773b and restore the voltage at the first capacitor 771b to the reference voltage V R After that, the signal V br is deasserted. This process occurs when an event visual pixel detects Figure 8 The second DOWN event illustrated in FIG. 7 is repeated, thereby further reducing the voltage at the second capacitor 772 b.

[0069] As discussed above, the current timing window ends at time t2. Figure 8 As shown in, the signal R U At time t3, the first counter 735a is asserted (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of the amplifier 776a of the first counter 735a to the column bit line 738. Subsequently, the first voltage value (representing the aggregate number of UP events detected by the event vision pixel during the timing window) is read out onto the column bit line 738. Then, the signal R U is deasserted at time t4 to decouple the output of amplifier 776a from column bit line 738. Thereafter, signal R D At time t5, the signal R is asserted (e.g., based at least in part on the triggering of one or more latches in the scanner and readout logic) to couple the output of the amplifier 776b of the second counter 735b to the column bit line 738. Subsequently, the second voltage value (representing the aggregate number of DOWN events detected by the event vision pixel during the timing window) is read out onto the column bit line 738. Then, the signal R D At time t6, it is deasserted to decouple the output of amplifier 776b from column bit line 738. Then, reset signal RES and signal V br May be asserted at time t7 in anticipation of the start of the next timing window.

[0070] Despite Figure 8In the embodiment illustrated in , the first voltage value from the first counter 735a is read out before the second voltage value from the second counter 735b, but in other embodiments of the present technology, the second voltage value may be read out before the first voltage value. Additionally or alternatively, the first counter 735a may be coupled to a different column bit line than the second counter 735b. In these embodiments, the first voltage value may be read out before, after, or at the same time as the second voltage value is read out.

[0071] Note that Figure 5 The first voltage value and the second voltage value read by the first counter 535a and the second counter 535b are compared. Figure 7 The first voltage and the second voltage read out by the first counter 735a and the second counter 735b do not depend on the absolute current or absolute capacitance of the current source or capacitor, respectively. Instead, the voltage value read out from each of the counters 735a and 735b depends on the ratio of the capacitance of the corresponding capacitor in the corresponding one of the counters 735a or 735b. Therefore, no calibration technique is required to solve any mismatch between the capacitances of the capacitors in the counters 735a and 735b or between the current sources. However, when an UP event is detected during a given timing window, the voltage drop occurring on the second capacitor 772a can become smaller for each continuous UP event detected during that given timing window. Similarly, when a DOWN event is detected during a given timing window, the voltage drop occurring on the second capacitor 772b can become smaller for each continuous DOWN event detected during that given timing window. Therefore, the event vision sensor can perform a calibration technique (e.g., at the event signal processor) to solve this nonlinearity.

[0072] Further note, Figure 5 The time aggregation circuit 535 and Figure 7 The time aggregation circuit 735 is similar to Figure 3 The time aggregation circuit 335 of the embodiment of the present invention is used because each circuit includes: (i) a first counter for counting the number of UP events detected by the corresponding event vision pixel during the timing window; and (ii) a second counter for counting the number of DOWN events detected by the corresponding event vision pixel during the timing window. Therefore, an event vision sensor having event vision pixels using the time aggregation circuit 535 and / or the time aggregation circuit 735 can utilize the above reference Figure 4 The event data compression procedure discussed in detail.

[0073] Fig. 9 9 is a partial schematic block diagram of another time aggregation circuit 935 configured according to various embodiments of the present technology. It should be understood that the illustrated time aggregation circuit 935 can be included in Figure 2 210, or another example of a temporal aggregation circuit of the present technology. As shown, the temporal aggregation circuit 935 includes an up / down counter 939. The up / down counter 939 includes two inputs. The first input is configured to generate a comparator (e.g., Figure 2 The first comparator 234a of the embodiment receives the signal UP. The second input is configured to generate a comparator (eg Figure 2 The second comparator 234b) receives the signal DOWN. The up / down counter 939 further includes an output coupled to the column bit line 938.

[0074] In operation, the up / down counter 939 is configured to track the events detected by the corresponding event visual pixel during a given timing window. Compared to the other counters discussed above, the up / down counter 939 does not necessarily output the total number of UP events or the total number of DOWN events detected by the corresponding event visual pixel during a given timing window. Instead, the up / down counter 939 outputs the difference between (i) the number of UP events detected by the corresponding event visual pixel during the timing window and (ii) the number of DOWN events detected by the corresponding event visual pixel.

[0075] For the purpose of clarity and understanding, consider Fig.10 , which illustrates how various embodiments of the present technology can be used Fig. 9 An overview of the event data compression performed by the temporal aggregation circuit 935. Fig.10 Plot 1041 includes UP events and DOWN events detected by an event visual pixel within a timing window 1041 between time t1 and time t2. Plot 1040 is similar to Figure 4 For example, plot 1040 illustrates that eight (8) UP events and ten (10) DOWN events are detected during timing window 1041 and in the order shown for the corresponding event visual pixel. Fig. 9 The up / down counter 939 of the temporal aggregation circuit 935 of the embodiment of the present invention is used to track these events. The up / down counter 939 may output a value (e.g., a single value) representing the difference between the number of UP events and the number of DOWN events detected during the timing window 1041, rather than outputting two values ​​indicating the absolute number of UP events or DOWN events detected by the event vision pixel during the timing window 1041, respectively. Thus, in the illustrated example, the up / down counter 939 may output a value indicating that two more DOWN events than UP events are detected by the event vision pixel during the timing window 1041 to the column bit line 938.

[0076] Note that the value output by the counter 939 onto the column bit line 938 merely reflects the difference between the number of UP events and the number of DOWN events detected by the event vision pixel during the timing window 1041. In other words, the value output onto the column bit line 939 does not reflect the exact timing of when each individual event was detected by the event vision pixel during the timing window 1041, nor does the value necessarily indicate the absolute number of UP events or DOWN events detected during the timing window 1041. Therefore, this in-pixel compression of event data performed by the event vision pixel can be considered a form of lossy compression in which the event data as well as the temporal dependency information of the individual events is lost.

[0077] Similar to the above reference Figure 4 For the event data discussed, the value read out onto column bit line 938 may be combined with other values ​​read out from other event vision pixels of the event vision sensor to form a pseudo frame. Fig.10 As shown in FIG. 1 , the value read out from the up / down counter 939 onto the column bit line 938 within the timing window 1041 can be combined with other values ​​read out from other event visual pixels within the same timing window 1041 to form a pseudo frame 1050. Thereafter, as described below with reference to Fig.11 As discussed in more detail, the event vision sensor may optionally perform compression on the pseudo frame 1050 .

[0078] Frames of event count differences read out from the event vision pixels of the array within different timing windows are also referred to herein as "pseudo-frames" because the event data contained in each such pseudo-frame represents the difference between the number of UP events and the number of DOWN events detected by the event vision pixels during a given timing window, the difference being proportional to the contrast change or information change in the external scene. In contrast, as discussed above, by convention, frames of image data are associated with an exposure time, and the image data represents the amount of photons / electrons generated by the light incident on the photosensor of each pixel during the exposure time.

[0079] Fig.10Further included is a drawing 1045 illustrating reconstruction of event data based on pseudo frame 1050. More specifically, event data corresponding to drawing 1040 and event visual pixels within timing window 1041 may be reconstructed using associated event data 1055 stored in pseudo frame 1050. As discussed above, the absolute number of UP events detected by the event visual pixel during timing window 1041, the absolute number of DOWN events detected by the event visual pixel during timing window 1041, and / or dependency information about when exactly each of the individual events occurred during timing window 1041 may be lost during compression in the pixels of the event data. Thus, assumptions may be made during reconstruction about the number of events detected by the event visual pixel during timing window 1041 and about when each individual event occurred within timing window 1041. In the illustrated example, it is assumed that the event visual pixel detected two DOWN events and zero UP events during timing window 1041 based on the difference output onto column bit line 938 by up / down counter 939 above. It is further assumed that the event visual pixel detects two DOWN events at uniform intervals across the timing window 1041. Therefore, during reconstruction, the two DOWN events may be evenly distributed across the timing window 1041, as shown in plot 1045.

[0080] Other assumptions about (i) the number of UP or DOWN events detected by the event visual pixel during the timing window 1041 and / or (ii) the distribution of UP events and / or DOWN events across the timing window are certainly possible and within the scope of the present technology. In addition or as an alternative, different assumptions may be made for UP events and DOWN events, and / or multiple assumptions may be made for UP events or DOWN events during reconstruction, so that the distribution of UP events and / or DOWN events may vary across the time window. In addition, for the same pseudo-frame, the assumptions made during the reconstruction of event data corresponding to one event visual pixel may, but need not, be the same as the assumptions made during the reconstruction of event data corresponding to another event visual pixel. In addition, for different pseudo-frames (e.g., pseudo-frames corresponding to different timing windows), the assumptions made during the reconstruction of event data corresponding to the event visual pixel may, but need not, be the same across different pseudo-frames.

[0081] Fig.11 is a partial schematic block diagram of an event signal processor 1108 configured according to various embodiments of the present technology. It should be understood that the illustrated event signal processor 1108 can be Figure 1100, or an example of another event signal processor of the present technology. As shown, the event signal processor 1108 includes a defective pixel removal block 1182, a compression block 1183, and an auxiliary function block 1186. The defective pixel removal block 1182 can be used to remove the defective pixel corresponding to the array (e.g., Figure 1 The auxiliary function block 1186 may include one or more other stages or circuits for performing various auxiliary processing functions.

[0082] As discussed above, an event vision sensor configured according to the present technology can use event data read out from the time aggregation circuit or level of the event vision pixels in the array to form a pseudo frame. These pseudo frames may generally include event data that appear as shapes or objects, which means that in addition to or as an alternative to the compression in pixels performed by the time aggregation circuit of the event vision pixels, a compression algorithm (e.g., a wavelet compression algorithm) can be applied to the pseudo frame to compress the event data. Therefore, the compression block 1183 of the event signal processor 1108 can be used to compress the event data at the pseudo frame level. In some embodiments, the compression block 1183 can compress the pseudo frame using a lossy compression algorithm. Examples of lossy compression algorithms that can be used include Joint Photographic Experts Group (JPEG) compression, Advanced Video Coding (AVC) compression (also known as H264 compression), and High Efficiency Video Coding (HEVC) compression (also known as H265 compression). In these and other embodiments, the compression block 1183 of the event signal processor 1108 can utilize a lossless compression algorithm such as Moving Picture Experts Group (MPEG) compression.

[0083] although Fig.11 1108, but the event signal processor 1108 may additionally include other circuits or modules for performing specific processing functions. For example, the event signal processor 1108 may include a segment classifier block and / or a shape classifier block that can be used to classify the segment and shape of the event data read out from the array, respectively. As another example, the event signal processor 1108 may include an optical flow estimation block that can be used to identify pixel-by-pixel, shape-by-shape, or segment-by-segment motion over time and / or between consecutive readouts (e.g., using correlation-based, block-matching-based, feature-tracking-based, energy-based, and / or gradient-based optical flow estimation). Additionally or alternatively, the event signal processor 1108 may include a set of one or more buffers (e.g., a set of row buffers) that can be operably coupled to one or more of the blocks or circuits of the event signal processor 1108 to perform various processing functions.

[0084] Fig.12 is a diagram illustrating an event vision sensor (e.g., Figure 1Flowchart of method 1290 for operating an event vision sensor 100). Method 1290 is illustrated as a set of steps or boxes 1291 to 1297. All or a subset of one or more of boxes 1291 to 1297 may be performed by various components of the event vision sensor. For example, all or a subset of one or more of boxes 1291 to 1297 may be performed by: (i) individual event vision pixels (e.g., one or more event generating comparators 234 and / or temporal aggregation circuits or stages of event vision pixels); (ii) event vision pixel arrays; (iii) row control circuit systems; (iv) column control circuit systems; (v) event signal processors; (vi) memory and / or (vii) transmitters. In addition, any one or more of boxes 1291 to 1297 may be performed according to the above Figures 1 to 11 to be implemented by the discussion.

[0085] Fig.12 The method 1290 begins detecting events occurring within a timing window at block 1291. Detecting events occurring within a timing window may include detecting one or more UP events and / or DOWN events occurring within the timing window and / or corresponding to activities in an external scene.

[0086] At box 1292, method 1290 continues to compress event data corresponding to the timing window. In some embodiments, compressing the generated event data may include compressing the event data generated by at least one event vision pixel of the event vision sensor and / or before reading the event data from at least one event vision pixel. In these and other embodiments, compressing the event data may include compressing the event data using a temporal aggregation level (e.g., corresponding to the event vision pixel). Compressing the event data using the temporal aggregation level may include compressing the event data using one or more counters. In some embodiments, compressing the event data includes tracking events detected during a given timing window or within a time period. For example, compressing the event data may include tracking or counting the following: (i) the absolute number of UP events detected by the event vision pixel (e.g., within the timing window); and / or (ii) the absolute number of DOWN events detected by the event vision pixel (e.g., within the timing window). In these and other embodiments, compressing event data may include tracking or calculating the difference between (a) the number of UP events detected by the event vision pixels (e.g., within a timing window) and (b) the number of DOWN events detected by the event vision pixels (e.g., within a timing window).

[0087] At block 1293, the method 1290 continues by reading out the compressed event data from the event vision pixel. In some embodiments, reading out the compressed event data may include reading out the absolute number of UP events detected by the event vision pixel (e.g., within a timing window) and / or reading out the absolute number of DOWN events detected by the event vision pixel (e.g., within a timing window). In these and other embodiments, reading out the compressed event data may include reading out the difference between (a) the number of UP events detected by the event vision pixel (e.g., within a timing window) and (b) the number of DOWN events detected by the event vision pixel (e.g., within a timing window).

[0088] In some embodiments, reading out the compressed event data may include reading out the compressed event data from the event vision pixel only when the event vision pixel has detected at least one event during a given timing window. For example, the compressed event data (UP event count, DOWN event count, or the difference between the UP event count and the DOWN event count) corresponding to the given timing window may be read out from the event vision pixel only when the event vision pixel has detected at least one event (UP event or DOWN event) during the given timing window. As another example, the UP event count may be read out from the event vision pixel as compressed event data only when the event vision pixel has detected at least one UP event during the corresponding given timing window. Additionally or alternatively, the DOWN event count may be read out from the event vision pixel as compressed event data only when the event vision pixel has detected at least one DOWN event during the corresponding given timing window.

[0089] At box 1294, method 1290 continues to generate a pseudo frame using compressed event data read from one or more event visual pixels. Generating a pseudo frame may include combining or correlating compressed event data read from a first event visual pixel within a given timing window with compressed event data read from at least one other event visual pixel within a given timing window. In some embodiments, generating a pseudo frame may include: (i) generating a first pseudo frame using first compressed event data read from a first group of event visual pixels, the first compressed event data representing the number of UP events detected by the first group of event visual pixels within a given timing window; and / or (ii) generating a second pseudo frame using second compressed event data read from a second group of event visual pixels, the second compressed event data representing the number of DOWN events detected by the second group of event visual pixels within a given timing window. The first group of event visual pixels may include a group of event visual pixels that is the same as or different from the second group of event visual pixels.

[0090] At block 1295, the method 1290 continues to perform compression on the one or more pseudo frames generated at block 1294. In some embodiments, performing compression on the one or more pseudo frames may include performing lossy compression on the one or more pseudo frames. For example, performing compression on the one or more pseudo frames may include performing JPEG, H264, and / or H265 on the one or more pseudo frames. In these and other embodiments, performing compression on the one or more pseudo frames may include performing lossless compression on the one or more pseudo frames. For example, performing compression on the one or more pseudo frames may include performing MPEG on the one or more pseudo frames.

[0091] The method 1290 continues with storing the compressed event data and / or pseudo frames at block 1296. Storing the event data and / or pseudo frames may include storing the event data and / or pseudo frames to a memory, such as to a buffer of an event vision sensor.

[0092] At block 1297, method 1290 continues with transmitting the event data and / or pseudo frame. Transmitting the event data and / or pseudo frame may include transmitting the event data and / or pseudo frame out of the event vision sensor and / or to a receiver of a corresponding imaging system.

[0093] Although blocks 1291 through 1297 of method 1290 are discussed and illustrated in a particular order, Fig.12The method 1290 illustrated in the figure is not limited thereto. In other embodiments, the method 1290 can be executed in different orders. In these and other embodiments, any one of the blocks 1291 to 1297 of the method 1290 can be executed before, during and / or after any one of the other blocks 1291 to 1297 of the method 1290. For example, block 1295 can be executed while a pseudo frame is generated at block 1294. As another example, block 1296 can be executed before or while blocks 1294 and / or 1295 are executed. As still another example, blocks 1291, 1292 and / or 1293 can be executed while executing one or more of blocks 1294 to 1297. In addition, those skilled in the art will recognize that the illustrated method 1290 can be changed and still remain within these and other embodiments of the present technology. For example, in some embodiments, one or more blocks 1291-1297 of method 1290 may be omitted and / or repeated. As a specific example, in some embodiments (e.g., in embodiments where the event vision sensor performs frame-level compression on pseudo frames but does not perform in-pixel compression on events detected at the pixel level at block 1295), block 1292 may be omitted. As another specific example, in some embodiments (e.g., in embodiments where the event vision sensor performs in-pixel compression on events detected at the pixel level but does not perform frame-level compression on pseudo frames), block 1295 may be omitted. As still another specific example, in some embodiments (e.g., in embodiments where at least some of the event data read out from the event vision pixels and / or used to form one or more pseudo frames is discarded rather than stored to memory and / or transmitted out of the corresponding event vision sensor), block 1296 and / or block 1297 may be omitted.

[0094] As discussed above, the present technology facilitates compressing event data in pixels and / or at a pseudo-frame level. As a result, the amount of data stored to a memory (e.g., a buffer) of an event vision sensor configured according to various embodiments of the present technology to record events detected by event vision pixels of the event vision sensor can be reduced compared to the amount of data used to record events in an event vision sensor that does not perform such compression. As a result, the number of events that can be stored to a fixed amount of memory using an event vision sensor of the present technology can be increased compared to other event vision sensors. In some cases, the present technology can therefore facilitate longer recording times before the fixed amount of memory becomes full.

[0095] One application of the present technology includes high-speed video capture. For example, an active pixel sensor can be used to capture image data (e.g., video) of an external scene, while an event vision sensor can be used to capture event data corresponding to activities occurring within the external scene. Event data captured and compressed by the event vision sensor can be decompressed and synthesized with image data captured using the active pixel sensor. In this way, the present technology can be used to detect and / or record high-speed components of activities occurring in an external scene, which can be useful, for example, when capturing slow-motion video.

[0096] Although several embodiments of the present technology are discussed above in the context of counting and compressing events at the individual pixel level, the present technology is not limited thereto. For example, event vision pixels in an array of such pixels included in an event vision sensor may be organized into blocks or groups. In some embodiments, one or more time aggregation circuits may be shared among a group of event vision pixels. Continuing with this example, one or more time aggregation circuits may include one or more counters configured to count the following: (a) UP events detected by event vision pixels of the group; (b) DOWN events detected by event vision pixels of the group; and / or (c) the difference between the number of UP events detected by event vision pixels of the group and the number of DOWN events detected by event vision pixels of the group. One or more time aggregation circuits may be located within one or more event vision pixels of the group, or may be a peripheral circuit located outside the event vision pixels. Event data generated by one or more time aggregation circuits may be read out and combined with event data corresponding to one or more other event vision pixel groups to form one or more pseudo frames. The event vision sensor may then optionally perform frame level compression on such pseudo frames.

[0097] In still other embodiments of the present technology, the event vision pixel may include a time aggregation circuit that counts only the number of UP events and / or DOWN events detected by the event vision pixel within a given timing window. At or after the end of the given timing window, the UP event count and / or DOWN event count may be output to the peripheral circuit system to, for example, calculate one or more differences between the number of UP events detected by the event vision pixel within the given timing window and the number of DOWN events detected by the event vision pixel within the given timing window. The one or more differences may then be read and used to generate one or more pseudo frames, and the event vision sensor may optionally perform frame level compression on the one or more pseudo frames.

[0098] C. in conclusion

[0099] The above detailed description of the embodiments of the present technology is not intended to be exhaustive or to limit the present technology to the precise form disclosed above. Although specific embodiments and examples of the present technology are described above for illustrative purposes, various equivalent modifications may be made within the scope of the present technology as will be appreciated by those skilled in the art. For example, although the steps are presented above in a given order, alternative embodiments may perform the steps in a different order. In addition, the various embodiments described herein may be combined to provide further embodiments.

[0100] According to the foregoing, it can be understood that, for illustrative purposes, a specific embodiment of the present technology has been described herein, but well-known structures and functions have not yet been shown or described in detail to avoid unnecessary ambiguity in the description of the embodiments of the present technology. To the extent that any material incorporated herein by reference conflicts with the present disclosure, the present disclosure shall prevail. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. In addition, unless the word "or" is explicitly limited to a single item that only means to exclude other items when referring to a list of two or more items, the use of "or" in this list will be interpreted as including any combination of any single item in the (a) list, all items in the (b) list, or items in the (c) list. In addition, as used herein, the phrase "and / or" in "A and / or B" refers to A alone, B alone, and both A and B. In addition, the terms "include", "include", "have (having)" and "have (with)" are used throughout to mean at least including the listed features, so that any more number of the same features and / or features of additional types are not excluded. In addition, as used herein, the phrases "based on," "depending on," "as a result of," and "in response to" should not be interpreted as references to a closed set of conditions. For example, an exemplary step described as "based on condition A" can be based on both condition A and condition B without departing from the scope of the present invention. In other words, as used herein, the phrase "based on" should not be interpreted in the same manner as the phrase "based at least in part on." In addition, the terms "connected" and "coupled" are used interchangeably herein and refer to both direct and indirect connections or couplings. For example, where the context permits, element A "connected" or "coupled" to element B may refer to (i) being directly "connected" or directly "coupled" to B and / or (ii) being indirectly "connected" or indirectly "coupled" to B.

[0101] Based on the foregoing, it will also be understood that various modifications can be made without departing from the present disclosure or technology. For example, it will be understood by those skilled in the art that the various components of the present technology can be further divided into subcomponents, or the various components and functions of the present technology can be combined and integrated. In addition, certain aspects of the present technology described in the context of a particular embodiment can be combined or eliminated in other embodiments. In addition, although the advantages associated with certain embodiments of the present technology have been described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments are necessary to exhibit such advantages to fall within the scope of the present invention. Therefore, the present invention and associated technology may encompass other embodiments that are not explicitly shown or described herein.

Claims

1. An event visual pixel, comprising: a photosensor configured to generate a photocurrent in response to incident light received from an external scene; a photocurrent-to-voltage converter coupled to the photosensor to convert the photocurrent into a voltage; a difference detection circuit coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to a difference detected over time in the voltage received from the photocurrent-to-voltage converter; an event generating comparator coupled to the difference detection circuit to detect an event occurring within the external scene based on a comparison of the filtered and amplified signal to one or more thresholds; and A time aggregation circuit is coupled to the event generation comparator to track an aggregate number of events detected by the event generation comparator within a specified timing window.

2. An event vision pixel according to claim 1, wherein the temporal aggregation circuit comprises at least one counter configured to count the aggregated number of events detected by the event generating comparator within the specified timing window.

3. The event visual pixel of claim 2, wherein the at least one counter comprises: a first capacitor having a top plate and a grounded bottom plate, wherein the top plate of the first capacitor is selectively coupled to a first reference voltage via a first switch and based on a first reset signal; and A second capacitor having a top plate and a grounded bottom plate, wherein the top plate of the second capacitor: (a) is selectively coupled to the top plate of the first capacitor via a second switch and based on an event signal received from the event generating comparator; (b) is selectively coupled to a second reference voltage via a third switch and based on a second reset signal; and (c) is selectively coupled to a column bit line (i) via an amplifier and a fourth switch and (ii) based on a readout signal.

4. The event vision pixel of claim 2, wherein the at least one counter comprises: a first counter configured to track an aggregate number of UP events detected by the event generating comparator within the specified timing window, each of the UP events corresponding to a change in the incident light from darker to brighter; and A second counter is configured to track an aggregate number of DOWN events detected by the event generating comparator within the specified timing window, each of the DOWN events corresponding to a change of the incident light from brighter to darker.

5. The event vision pixel of claim 1 , further comprising a readout logic coupled to the output of the event generating comparator, wherein the readout logic comprises a latch that is triggered when the event generating comparator detects at least one event within the specified timing window.

6. An event vision sensor, comprising: Multiple event visual pixels, in: Each event vision pixel of the plurality of event vision pixels is configured to generate event data based at least in part on an event indicated in incident light received from an external scene, Each event vision pixel of the plurality of event vision pixels includes compression circuitry configured to compress the event data before the event data is read out from the event vision pixel.

7. The event vision sensor according to claim 6, wherein: Each compression circuit includes a time aggregation circuit; and To compress the event data, the temporal aggregation circuit is configured to track the number of events detected by the corresponding event visual pixels within a specified timing window.

8. The event vision sensor of claim 7, wherein each temporal aggregation circuit comprises at least one counter configured to count the number of the events detected by the corresponding event vision pixel within the specified timing window.

9. The event vision sensor according to claim 7, wherein: Each temporal aggregation circuit includes an up / down counter configured to track a difference between (a) a number of UP events detected by a corresponding said event vision pixel within said specified timing window and (b) a number of DOWN events detected by a corresponding said event vision pixel within said specified timing window; Each of the UP events corresponds to a change in the incident light from darker to brighter; and Each of the DOWN events corresponds to a change of the incident light from brighter to darker.

10. The event vision sensor of claim 7, wherein each of the plurality of event vision pixels further comprises a readout logic, the readout logic comprising at least one latch, the at least one latch being triggered when the corresponding event vision pixel detects at least one event within the specified timing window.

11. The event vision sensor of claim 6, wherein the event vision sensor is configured to form a pseudo frame based at least in part on event data readout of one or more event vision pixels of the plurality of event vision pixels. 12 . The event vision sensor of claim 11 , further comprising an event signal processor configured to perform a compression algorithm on the pseudo frame.

13. The event vision sensor of claim 12, wherein the event signal processor is configured to perform a lossy compression algorithm on the pseudo frame.

14. The event vision sensor of claim 13, wherein the lossy compression algorithm comprises Joint Photographic Experts Group compression, Advanced Video Coding compression (also known as H264 compression), High Efficiency Video Coding compression (also known as H265 compression), or any combination thereof.

15. A method of operating an event vision sensor comprising one or more event vision pixels, the method comprising: For each individual event visual pixel of the one or more event visual pixels— detecting an event indicated by incident light received from an external scene within a timing window, and The number of said events detected within said timing window is tracked.

16. The method of claim 15, wherein: Tracking the number of events detected within the timing window includes: (a) tracking the number of UP events detected within the timing window; and (b) tracking the number of DOWN events detected within the timing window; Each of the UP events corresponds to a change in the incident light from darker to brighter; and Each of the DOWN events corresponds to a change of the incident light from brighter to darker.

17. The method of claim 15, wherein: Tracking the number of events detected within the timing window includes tracking a difference between (a) a number of UP events detected within the timing window and (b) a number of DOWN events detected within the timing window; Each of the UP events corresponds to a change in the incident light from darker to brighter; and Each of the DOWN events corresponds to a change of the incident light from brighter to darker.

18. The method of claim 15, further comprising: At or after the end of the timing window, the number of the events detected by at least one of the one or more event vision pixels within the timing window is read out.

19. The method of claim 15, further comprising: For each of the individual event vision pixels of the one or more event vision pixels, the number of the events detected within the timing window is read out only if the individual event vision pixel detects at least one event during the timing window.

20. The method of claim 15, further comprising forming a pseudo frame corresponding to the timing window based at least in part on the number of events detected within the timing window by all or a subset of the one or more event visual pixels.

21. The method of claim 20, wherein: The pseudo frame is a first pseudo frame, and the subset is a first subset; forming the first pseudo frame comprises forming the first pseudo frame based at least in part on the number of UP events detected within the timing window by all or the first subset of the one or more event visual pixels; Each of the UP events corresponds to a change of the incident light from darker to brighter; The method further includes forming a second pseudo frame corresponding to the timing window based at least in part on the number of DOWN events detected within the timing window by all or a second subset of the one or more event visual pixels; and Each of the DOWN events corresponds to a change of the incident light from brighter to darker.

22. The method of claim 20, further comprising performing frame level compression on the pseudo frame.

23. The method of claim 15, wherein tracking the number of the events detected within the timing window comprises: resetting the counter at or before the start of the timing window; and The number of the events is counted using the counter.

Citation Information

Patent Citations

  • Vision sensor, image processing device including the vision sensor, and operating method of the vision sensor

    CN112153309A

  • Object recognition method and device, chip and electronic equipment

    CN113408671A

  • Customer clustering using integer programming

    US20220284457A1

  • Image sensor, data processing device, and image sensor system

    WO2023058670A1