Readout of events from an array of event-based image sensors and pixel circuits - Patents.com

Abstract

The circuit includes a selection circuit electrically coupled to pixels of the line of the event visual sensor. The selection circuit is configured to receive an activation signal from active pixels of the line, generate an acknowledge signal in response to receiving the activation signal, and send the acknowledge signal to the pixels of the line. The circuit also includes a control circuit electrically coupled to the pixel of the line that is the furthest away from the selection circuit. The control circuit is configured to receive the acknowledge signal from the selection circuit and generate a process read signal in response to receiving the acknowledge signal. The circuit also includes an interface circuit electrically coupled to the pixels of the line, the interface circuit configured to cause a reset of the selection circuit, the control circuit, and the pixels of the line after receiving the process read signal.

Description

[Technical Field]

[0001] This application claims priority to U.S. Provisional Patent Application No. 62 / 976,868, filed February 14, 2020, the contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates generally to the field of event-based vision sensing, and to systems and methods for processing events. More particularly, but not exclusively, the present disclosure relates to event array readout control for event-based image sensing. The sensors and techniques disclosed herein can be used in a variety of applications and vision systems, such as security systems, autonomous vehicles, and other systems that benefit from fast and efficient motion or event detection. [Background technology]

[0003] Current image sensors use multiple pixels that make up a semiconductor charge-coupled device (CCD), complementary metal-oxide semiconductor (CMOS) sensor, n-type metal-oxide semiconductor (NMOS) sensor, or other sensor to capture a digital image of a scene. However, traditional image sensors are slow to detect sudden motion because each frame is captured as a full image of the scene. Furthermore, such image sensors generate large amounts of data, exponentially increasing the amount of post-processing required to sift, for example, motion information from the captured image.

[0004] Many existing technologies do not require the vast amount of detail provided by image sensors that capture the full image. For example, a security system or other similar system may only be interested in motion data and not in portions of the image that are motionless. In another example, an autonomous vehicle needs to process captured data quickly and efficiently to make decisions comparable to human perception times (typically on the order of one second or less). Such efficiency is limited when large amounts of data must be discarded (e.g., by post-processing) to obtain the portions of the captured data that are relevant to the situation. Summary of the Invention

[0005] An embodiment of the present disclosure provides a circuit for sampling data from pixels of an event visual sensor. The circuit may include a selection circuit, a control circuit, and an interface circuit. The selection circuit may be electrically coupled to pixels of a line of the event visual sensor. The selection circuit may be configured to receive an activation signal from active pixels of the line, generate an acknowledge signal in response to receiving the activation signal, and transmit the acknowledge signal to the pixels of the line, where each pixel of the line is configured to generate a request read signal when the pixel is active upon receiving the acknowledge signal. The control circuit may be electrically coupled to a pixel of the line that is farthest from the selection circuit. The control circuit may be configured to receive the acknowledge signal from the selection circuit and generate a process read signal in response to receiving the acknowledge signal. The interface circuit may be electrically coupled to the pixels of the line and configured to reset the selection circuit, the control circuit, and the pixels of the line after receiving the process read signal.

[0006] An embodiment of the present disclosure also provides a circuit for sampling data from pixels of an event visual sensor. The circuit may include a line selection circuit, an interface circuit, and a timer circuit. The line selection circuit may be electrically coupled to pixels of a line of the event visual sensor. The line selection circuit may be configured to receive an activation signal from active pixels of the line, generate an acknowledge signal and a start signal in response to receiving the activation signal, and send the acknowledge signal to the pixels of the line, where each pixel of the line is configured to generate a request read signal when the pixel is active upon receiving the acknowledge signal. The interface circuit may be electrically coupled to the pixels of the line. The interface circuit may be configured to receive a data signal from the pixel that generated the request read signal in response to receiving the request read signal. The timer circuit may be electrically coupled to the interface circuit and the line selection circuit. The timer circuit may be configured to receive a start signal from the line selection circuit and to reset the line selection circuit and the pixels of the line if a data signal is not received by the interface circuit after a time interval from a timestamp at which the start signal was received exceeds a threshold time.

[0007] An embodiment of the present disclosure further provides a pixel circuit for use in an image sensor. The pixel circuit may include a comparator and a first latch circuit. The comparator may be configured to generate a first request read signal when an input signal of the comparator matches a first condition, the input signal being generated from the photosensitive element in response to the intensity of light incident on the photosensitive element. The first latch circuit may include a first set input, a first reset input, and a first output, the first reset input and the first output being electrically coupled to the interface circuit, and the first set input being electrically coupled to the comparator. The first latch circuit may be configured to receive the first request read signal from the comparator via the first set input, output the first request read signal to the interface circuit, lock the first set input to not receive a signal, and reset the first set input to receive a new signal in response to receiving an acknowledge signal from the interface circuit via the first reset input.

[0008] The accompanying drawings, which constitute a part of this specification, illustrate several embodiments and, together with the description, serve to explain the principles and features of the disclosed embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a schematic diagram of an example architecture of a pixel of an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of an example architecture of a readout interface for pixels of an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram of an exemplary circuit for an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a schematic diagram of another exemplary circuit for an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a schematic diagram of an example architecture of another readout interface for pixels of an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a schematic diagram of yet another exemplary circuit for an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram of an example architecture of the protocol control circuit in FIG. 6, according to an embodiment of the present disclosure. [Figure 8] FIG. 2 is a schematic diagram of a signal timing diagram for a readout circuit of an event-based vision sensor, according to an embodiment of the present disclosure. [Figure 9] FIG. 1 is a schematic diagram of an exemplary architecture of a chip-top circuit design for an event-based vision sensor, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010]

[0003] Embodiments disclosed herein relate to systems and methods for visual sensing, including asynchronous, time-based sensing. The disclosed embodiments also relate to event array readout architectures and controls for event-based visual sensing. Advantageously, exemplary embodiments can provide fast and efficient sensing. Embodiments of the present disclosure can be implemented and used in a variety of applications and vision systems, such as security systems, autonomous vehicles, and other systems that benefit from fast and efficient motion or event detection. While embodiments of the present disclosure are generally described with reference to vision systems, it will be understood that such systems can be part of a camera, LIDAR, or other sensor system.

[0011] In current image or video processing systems, an image sensor may acquire and process visual information to reconstruct an image. The image or video acquisition and processing system may include an array of optical sensors, each of which may acquire visual information to reconstruct an image representation of a visual scene. This process may be repeated at a predetermined pace.

[0012] Event-based (EB) vision sensors detect temporal contrast exceeding a preset relative threshold for each pixel to track the temporal evolution of relative light changes (contrast detection, CD) and define sampling points for frame-free pixel-level measurements of absolute intensity (exposure metering, EM). EB sensors are gaining popularity in high-speed, low-power machine vision applications thanks to their temporal precision in recorded data, reduced post-processing costs due to suppressed temporal redundancy, and wide dynamic range operation within a scene. Temporal contrast (CD) information can be encoded in the form of "events," which are data packets containing the pixel's X and Y coordinates, a timestamp, and contrast polarity. To maximize benefit from individual pixels' ability to sample visual information with high temporal precision, early timestamps and high readout throughput can be important to preserve the timing of events.

[0013] In some embodiments, for real-time artificial vision (also called "computer vision" or "machine vision"), a vision acquisition and processing system may be configured to acquire and process only data representing changes in current visual information relative to previously acquired visual information. Such sensors or vision systems may not generate frames of images. Such vision sensors may include, for example, temporal contrast (TC) sensors, contrast detection (CD) sensors, or dynamic vision sensors (DVS). Such sensors are referred to in this disclosure as "event-based vision sensors."

[0014] For example, TC sensors do not record images frame by frame like current imaging systems. Instead, each pixel in a TC sensor can determine the time derivative of the light it senses. In some embodiments, the pixel can optionally perform further processing on the time derivative. When the time derivative exceeds a preset threshold, the pixel can generate an "event" by outputting a signal. With a short latency, the pixel can further transmit data related to the event. In some embodiments, the transmitted data can include the location (e.g., x and y coordinates) of the pixel located within the TC sensor (e.g., having a two-dimensional pixel matrix). In some embodiments, the transmitted data can include a sign bit (e.g., positive or negative sign) representing the polarity of the temporal evolution of the light intensity sensed by the pixel. In some embodiments, the transmitted data can include a timestamp of the occurrence of the event. In some embodiments, the transmitted data of a pixel can include a flow of (x, y, s) values, where x and y represent the coordinates of the pixel and s represents the polarity. The value of s may represent a relative change in light intensity detected by the pixel, where the value of s may represent the magnitude of the change and the sign of s may represent the direction of the change (e.g., increase or decrease). Typically, the pixel circuit of a TC sensor may operate asynchronously, i.e., the pixel circuit of a TC sensor is typically not quantized (e.g., not clocked) with respect to a time base.

[0015] Event-based vision sensors (e.g., TC sensors) can use address-event-representation readout (AER) for data readout. In some embodiments, the AER can use a handshake protocol (e.g., a synchronous or asynchronous handshake protocol) between the pixels and the readout periphery of the event-based vision sensor. In some embodiments, the AER can be located after the readout system. In some embodiments, the AER can be located between the event-based vision sensor chip and an external data receiver.

[0016] In some embodiments, the readout cycle of an event can include two separate handshake cycles. For example, a pixel can send a request signal to the row-wise readout periphery for row selection. Additionally or optionally, after receiving a row acknowledge signal (e.g., indicating successful row selection), the pixel can send a request signal in the column direction of the readout periphery. The readout cycle for outputting an event by a pixel can end when the pixel receives a column acknowledge signal (e.g., indicating successful column selection). In some embodiments, after receiving the row acknowledge signal, the readout cycle may end (i.e., the pixel does not send a request signal in the column direction of the readout periphery or wait for the column acknowledge signal).

[0017] In this disclosure, "row" and "column" refer to two different dimensions of the readout periphery, and the nomenclature is arbitrary. Both dimensions of the readout periphery can be interchanged without changing the functionality of the readout periphery. Hereinafter, for ease of explanation and to avoid ambiguity, the first and second dimensions accessed by a pixel will be referred to as "row" and "column," respectively, and will be designated by the dimension designators "Y" and "X," respectively.

[0018] In some cases, two or more pixels may simultaneously access the row-wise readout periphery to select a row for readout. In those cases, an arbiter circuit can be used to prioritize the requests. In some embodiments, the arbiter can be implemented as asynchronous logic circuitry or synchronous (e.g., clocked) logic circuitry that uses a handshake approach for communication. In some embodiments, a simple scanner can be used to control the sequence of active row selections.

[0019] For event vision sensors (e.g., TC sensors) with readouts that implement a handshake protocol (e.g., a synchronous or asynchronous handshake protocol), there is still a risk (referred to as the "readout lock risk" or "readout lock problem") that the readout may be blocked and stall the handshake protocol if a request in the readout system is not acknowledged. For example, this can occur when a pixel detects a temporal contrast event and generates a corresponding row request (reqY) signal, but before the row acknowledge (ackY) signal is generated by the readout periphery, the temporal contrast event disappears (e.g., the voltage signal associated with the pixel's optical input exceeding the detection threshold falls below the detection threshold, causing the comparator to switch back to an inactive state). In such a case, the pixel cannot generate a column request (reqX) signal (because the temporal contrast event disappears), and the readout system freezes in the middle of the event readout sequence (i.e., the readout system waits until the pixel completes the request signal generation protocol). A new event may occur at one of the pixels in the same row, halting the readout of all pixels in the event-based vision sensor for an indefinite period before the system can be unlocked. This undesirable behavior can cause the device to temporarily stop functioning and result in data loss.

[0020] According to one aspect of the present disclosure, a readout system for an event-based vision sensor is described that addresses the above risks and drawbacks. The event-based vision sensor may have a plurality of pixels. As used herein, "pixel" refers to the smallest element of the sensor that converts light into an electrical signal. Also, as disclosed herein, pixels may be provided in an array of any size and shape suitable for the sensing system.

[0021] 1 is a schematic diagram of an example architecture of a pixel 100 of an event-based vision sensor, according to an embodiment of the present disclosure. In some embodiments, the pixel may include a light-sensitive element 102. For example, the light-sensitive element 102 may include a photodiode (e.g., a pn junction or a PIN structure) or any other element configured to convert light 104 into an electrical signal. The light-sensitive element 102 generates a current I that is proportional to the intensity of the light 104 impinging on the light-sensitive element 102. ph may be generated.

[0022] In some embodiments, each pixel may further include an optical signal converter (not shown in FIG. 1 ) connected to the photosensitive element 102. The optical signal converter may be configured to provide a current signal at a first output that is linearly proportional to the intensity of light impinging on the photosensitive element 102 and a voltage signal at a second output that is logarithmic to the intensity of light 104 impinging on the photosensitive element 102. For example, the optical signal converter may include a plurality of transistors (not shown in FIG. 1 ), such as metal-oxide-semiconductor (MOS) transistors, complementary metal-oxide-semiconductor (CMOS) transistors, or any other three-terminal circuit element configured to amplify or switch an electronic signal.

[0023] In some embodiments, each pixel may further include a detector (not shown in FIG. 1 ). The detector may be configured to generate a trigger signal autonomously and independently of the detectors of other pixels when the detector's signal, which is proportional to the voltage signal at the second output of the optical signal converter, exceeds a threshold. For example, the detector may include one or more voltage comparators and one or more capacitors arranged to reset upon receiving a control signal and to transmit a trigger signal when the threshold is exceeded. Various forms of capacitors may be used in the present disclosure. For example, the capacitor may be a discrete device including two parallel (or substantially parallel) plates, optionally with a dielectric therebetween, or may be a parasitic capacitance present at a circuit node (e.g., due to semiconductor implementation of other circuit elements). Other forms of capacitors are also possible, such as metal-oxide-semiconductor (MOS) capacitors, metal-insulator-metal (MiM) capacitors, metal fringe capacitors, trench capacitors, or the like.

[0024] As a further example, as shown in FIG. 1 , pixel 100 may include a photosensitive element 102 (e.g., a partially pinned photodiode), a subthreshold MOS-based logarithmic photocurrent-to-voltage converter 106 (shown in FIG. 1 as “log I / V converter 106”), an asynchronous delta modulation or “level crossing” sampler 108 (shown in FIG. 1 as “ADM 108”), a voltage comparator 110 (e.g., for both polarities), and logic with ADM control (shown in FIG. 1 as “CTRL ADM "), and interface and status logic to the readout peripheral (shown in FIG. 1 as "ISL112").

[0025] 2 is a schematic diagram of an example architecture of a readout interface 200 for a pixel of an event-based vision sensor, according to an embodiment of the present disclosure. Readout interface 200 can be used to implement the interface and state logic or ISL 112 of FIG. 1. In some embodiments, each pixel of the event-based vision sensor can include a readout interface similar to readout interface 200.

[0026] The readout interface 200 can mitigate the risk of readout lock by adding at least one memory element (e.g., a latch) to every pixel. The memory element can immediately "lock" each temporal contrast event detected by the pixel. This lock can only be released by a corresponding acknowledge signal received by the same pixel. This lock cannot be released by the same pixel detecting further changes in the optical signal incident thereon, ensuring the completion of the request generation protocol for the same pixel.

[0027] As shown in the exemplary embodiment of FIG. 2, readout interface 200 includes two input latches 202 and 204. In some embodiments, input latches 202 and 204 can reduce power when comparator switching (e.g., switching of voltage comparator 110 in FIG. 1) is slow. In some embodiments, input latches 202 and 204 can further prevent or reduce ringing. In some embodiments, input latches 202 and 204 are associated with gated latches 206 and 208, respectively. Gated latches 206 and 208 can prevent slow request generation or event loss. In some embodiments, input latches 202 and 204 of readout interface 200 can store only pixels with events.

[0028] Each of the input latches 202 and 204 may include a SET input (designated as “S”) and a RESET input (designated as “R”). The first input latch 202 may have its SET input electrically coupled to a first output of a comparator (e.g., voltage comparator 110 of FIG. 1) that detects a positive polarity of temporal contrast (designated as “CON” in FIGS. 1-2). The second input latch 204 may have its SET input electrically coupled to a second output of a comparator (e.g., voltage comparator 110 of FIG. 1) that detects a negative polarity of temporal contrast (designated as “COFF” in FIGS. 1-2). When a temporal contrast is detected and one of the comparator outputs (e.g., CON or COFF) is activated, the corresponding latch (e.g., first input latch 202 or second input latch 204) may change its state. When CON is activated, the first input latch 202 may send a row request signal (represented as "reqY ON" in FIG. 2) indicating positive polarity through its output (represented as "Q") for row selection. When COFF is activated, the second input latch 204 may send a row request signal (represented as "reqY OFF" in FIG. 2) indicating negative polarity through its output (represented as "Q") for row selection. After successful row selection, the row selection circuit (e.g., the row selection circuit 302 in FIG. 3) may send an acknowledge signal (represented as "ackY" in FIG. 2) to the readout interface 200 (e.g., to both input latches 202 and 204). After receiving ackY, the readout interface 200 may be triggered to send a locked event to the input latches 202 and 204. For example, when CON is activated and reqY ON is sent, the first input latch 202 can send its locked event (represented as "reqX ON" in FIGS. 1-2). As another example, when COFF is activated and reqY OFF is sent, the second input latch 204 can send its locked event (represented as "reqX OFF" in FIGS. 1-2).When a read is in progress (e.g., when at least one of the input latches 202 or 204 is transmitting its locked event), the read interface 200 may be triggered by ackY to activate signals disabling both input latches 202 and 204 via their enable inputs (represented as "E") to disable the CON and COFF paths to prevent event loss. After completing the read, the read interface 200 then asserts a control signal (represented as "CTRL" in FIGS. 1-2) to the RESET inputs (represented as "R") of the two input latches 202 and 204 to reset both input latches. ADM ”). Changing CON or COFF has no effect on the resetting of input latches 202 and 204.

[0029] FIG. 3 is a schematic diagram of an exemplary circuit 300 for an event-based vision sensor according to one embodiment of the present disclosure. The circuit 300 includes an array of pixels, whose coordinates are represented by (0,0), (1,0), ... (M,N). Each of the pixels (e.g., the pixel at coordinate (m,n)) may send a row request signal (represented as "reqY" in FIG. 3) to a row selection circuit 302 electrically coupled to the pixel and receive a row selection acknowledge signal (represented as "ackY" in FIG. 3) from the row selection circuit 302. After receiving ackY, the pixel may further activate a column request signal (represented as "reqX" in FIG. 3) to an interface cell to activate data readout. "ON" and "OFF" following reqX may represent the polarity of the pixel's output data. The data output by the pixels of the selected row (“reqX[0:2M-1]”) to the interface cells 306 (e.g., a total of 2M interface cells labeled [0:2M-1]) can be further output by the interface cells 306 to an output format circuit 308. A row address encoder 304 (referred to as a “Y address encoder 304”) is electrically coupled to the pixels and can extract and output coordinates of the pixels of the selected row (represented as “addrY[0:log2(N)-1]” in FIG. 3 ) to the output format circuit 308. The output format circuit 308 can combine reqX[0:2M-1] and addrY[0:log2(N)-1] and output them as formatted data 310. A read controller 312 can control the output format circuit 308, the interface cells 306, and the row select circuit 302.

[0030] In some embodiments, to mitigate the readout lock problem, each pixel of circuit 300 may be pixel 100 and its ISL may include readout interface 200.

[0031] In some embodiments, to reduce additional components inside each pixel, an external "watchdog" timer circuit (not shown in FIG. 3) can be used in the event-based vision sensor to mitigate the readout lock issue. This timer circuit can unlock the readout system after a preset time of inactivity after receiving the last ackY signal from the row select circuit 302, regardless of whether reqX was received by the interface cell 306. In some embodiments, the preset time may be programmable.

[0032] 4 is a schematic diagram of another exemplary circuit 400 for an event-based visual sensor, in accordance with one embodiment of the present disclosure. In comparison to circuit 300, read controller 402 of circuit 400 includes a timer circuit (e.g., an external "watchdog" timer circuit as described above).

[0033] In some embodiments, when a pixel (e.g., pixel (m, n)) in a row (e.g., row n) of the circuit 400 detects a temporal contrast event, the pixel can send a row request signal (reqY) to the row select circuit 302. Then, if the row is successfully selected by the row select circuit 302 in response to receiving reqY, the row select circuit 302 can send an acknowledge signal ackY back to some or all of the pixels in the active row (e.g., row n). The row select circuit 302 can also generate a “rowreq” signal and send it to the readout controller 402. The “rowreq” signal can start a timer circuit. The y-address encoder 304 can construct row addresses for the active rows and output them as addrY[0:log2(N)−1].

[0034] After receiving ackY, all pixels that detected an event for the active row (the pixel that triggered sending reqY, if still activated, and any pixels that detected a new event for the active row after sending reqY and before receiving ackY) can send reqX to interface cell 306 to remove their reqY signal. Thus, the common row request (reqY) signal line for the active row can be made inactive.

[0035] The interface cell 306 can sample reqX[0:2M−1] into its memory element and notify the read controller 402 with a load signal (denoted as “load” in FIG. 4). After receiving the load signal, the read controller 402 can send a “rowack” signal to the row select circuit 302 to reset the row select circuit 302. The row select circuit 302 can then release the ackY for the active row. Deactivating the common ackY signal line can cause the active pixels of the active row to remove their activated reqX signal, which ends the read cycle. A new row can now be selected by the row select circuit 302 in response to receiving another reqY.

[0036] In some embodiments, when a read lock occurs and the read system freezes, the circuit 400 can be unlocked by a rowack signal controlled by a timer circuit in the read controller 402 as follows: After the read controller 402 receives the rowreq signal, if no reqX signal is sent to the interface cell 306 after a preset time (e.g., there are no events to read), and therefore the read controller 402 does not receive a load signal, the read controller 402 can activate to send a rowack signal to the row select circuit 302 to reset the row select circuit 302. After the reset, the row select circuit 302 can release the ackY of the active row. A new row can then be selected by the row select circuit in response to receiving another reqY.

[0037] In some embodiments, the preset time can be programmed to allow all active pixels in a row to complete sending their reqX uninterrupted, regardless of the number and distribution of active pixels in the active row.

[0038] 5 is a schematic diagram of an example architecture of another readout interface 500 for a pixel of an event-based vision sensor, according to one embodiment of the present disclosure. Compared to FIG. 2, readout interface 500 is similar to readout interface 200, but does not include any input latches. In some embodiments, the pixel of circuit 400 of FIG. 4 can be pixel 100, and its ISL can include readout interface 200 or 500. By doing so, circuit 400 can alleviate the readout lock problem without adding significant additional components to the circuit.

[0039] In FIG. 5, the read interface 500 includes two switches 502 and 504 for each of the comparator outputs (i.e., for each of the CON and COFF paths). For each of the CON and COFF paths, the first switch 502 is connected to the output of the comparator (e.g., the voltage comparator 110 in FIG. 1), and the second switch 504 is connected to the ackY path. When CON (or COFF) is activated, the first switch 502 can be connected to send reqY ON (or reqY OFF) for row selection. After successful row selection, the row selection circuit (e.g., the row selection circuit 302 in FIGS. 3-4) can send ackY to the read interface 500. After receiving ackY, the read interface 500 can be triggered to send an event. For example, when CON is activated and reqY ON is sent, the read interface 500 can send reqX ON. As another example, when COFF is activated and reqY OFF is sent, read interface 500 can send reqX OFF. When a read is in progress (e.g., an event is being sent), read interface 500 can be triggered by ackY to disconnect switches 502 and 504, disabling the CON and COFF paths to prevent event loss. After completing the read, read interface 500 can then use CTRL ADM can be triggered (for example, by grounding) to activate reqX ON and reset reqX OFF.

[0040] In some embodiments, to mitigate the read lock problem, a protocol control circuit can be added to the end of each row of pixels in the circuit 300 of the FIG. 3 embodiment. FIG. 6 is a schematic diagram of an exemplary circuit 600 for an event-based vision sensor in accordance with an embodiment of the present disclosure. The circuit 600 is similar to the circuit 300, but further includes a protocol control circuit (represented as “PCC 0,” “PCC 1,” … “PCC N”). In some embodiments, the pixels of the circuit 600 may be the pixels 100, whose ISLs may include the read interface 200 or 500. By doing so, the circuit 600 can mitigate the read lock problem without adding significant additional components to the circuit. FIG. 7 is a schematic diagram of an exemplary architecture of a protocol control circuit 700 for use in FIG. 6 in accordance with an embodiment of the present disclosure. The protocol control circuit 700 can be triggered to generate a column request signal (represented as “reqX_t” in FIG. 7 ) when the row to which it is electrically coupled is selected and when it receives an ackY. The protocol control circuit 700 can mimic the output of a pixel interface so that it has the same electrical characteristics as a pixel.

[0041] 6, in some embodiments, a protocol control circuit can be added on the opposite side of the row select circuit 302. In some embodiments, the protocol control circuit can be electrically coupled to the readout system in the same way as the pixels in the row. Unlike the pixels, which can only generate reqX when they are active after receiving ackY, the protocol control circuit can always generate and send a protocol column request signal (represented as "reqX_t" in FIG. 6) to the interface cell 306 in response to receiving ackY from the row select circuit 302.

[0042] In some embodiments, as shown in FIG. 6 , the protocol control circuit can be positioned such that it is a longer distance to the row select circuit 302 than any of the pixels in the same row. By doing so, any signal generated by the row select circuit 302 arrives at the protocol control circuit last (i.e., all pixels in the same row receive the signal before the protocol control circuit). Then, when the protocol control circuit receives ackY, activation of reqX_t occurs. Because the protocol control circuit is positioned at the end of the row (i.e., it has the longest distance to the row select circuit 302 compared to the pixels in the selected row), the protocol control circuit receives ackY immediately after the last pixel in the row, and all reqX (e.g., reqX ON or reqX OFF) from active pixels can be sent for readout before reqX_t is sent. Because reqX_t can always be generated even when there are no active pixels sending reqX, it can always trigger the continuation of the readout process, thereby preventing readout lock issues. Furthermore, the design of circuit 600 allows reqX_t to be generated with minimal delay when there are no active pixels sending reqX. In this way, if a read lock problem occurs, the time required to unlock the read process can be automatically minimized, minimizing the risk of data loss and unnecessary delays. Circuit 600 can be robust against device mismatches, process variations, temperature variations, or any other operating conditions.

[0043] In some embodiments, when interface cell 306 receives reqX_t, it can send a load signal to read controller 602 (e.g., similar to read controller 312 of FIG. 3) to indicate that it did not receive a reqX signal from a pixel in the active row. After receiving the load signal, read controller 602 can send a rowack signal to row select circuit 302. After receiving the rowack signal, row select circuit 302 can be reset and release ackY for the active row. A new row can then be selected by row select circuit 302 in response to receiving another reqY.

[0044] In some embodiments, circuit 600 can additionally include a timer circuit (e.g., the timer circuit of circuit 400 of FIG. 4) in readout controller 602 (e.g., similar to readout controller 402 of FIG. 4). In some embodiments, the pixel of circuit 600 can be pixel 100, and its ISL can include readout interface 200 or 500. In some embodiments, circuit 600 can further include a timer circuit of readout controller 602, and the pixel of circuit 600 can be pixel 100, and its ISL can include readout interface 200 or 500. By doing so, the readout lock problem can be avoided by at least two alternative independent mechanisms. Such a redundant design can further reduce the possibility of the readout lock problem occurring.

[0045] 8 is a schematic diagram of a signal timing diagram of a readout circuit of an event-based vision sensor according to an embodiment of the present disclosure. In FIG. 8, the timelines of different signals are aligned, and dashed lines with arrows indicate trigger relationships between the signals. FIG. 8 shows two handshake cycles of circuit 600, the first of which begins with the activation of reqY and ends with the activation of Ack, and the second of which begins with the activation of Ack and ends with the deactivation of Ack.

[0046] As shown in Figure 8, when a pixel in a row detects a temporal contrast event, that pixel may activate a reqY signal. When a row is successfully selected by a row select circuit (e.g., row select circuit 302 in Figure 6) in response to reqY, the row select circuit may activate ackY for the active row.

[0047] The activation of ackY may further activate three signals: reqX, which is generated by the active pixel of the active row (e.g., pixel (1,1) in FIG. 6), reqX_t, which is generated by the protocol control circuit (e.g., PCC1) of the active row after all pixels in the active row have received ackY, and rowreq, which is generated by the row select circuit. For an active row, all active pixels after receiving ackY may generate the reqX signal as described above.

[0048] A read controller (e.g., read controller 602 in FIG. 6) can receive a load signal derived from the reqX_t signal generated by the protocol control sequence and activate a Req signal to an output format circuit (e.g., output format circuit 308 in FIG. 6). The output format circuit can receive reqX[0:2M-1] from an interface cell (e.g., interface cell 306 in FIG. 6) and addrY[0:log2(N)-1] from a y-address encoder (e.g., y-address encoder 304 in FIG. 6). After completing reception of reqX[0:2M-1] and addrY[0:log2(N)-1], the output format circuit can output the data and activate Ack to the read controller.

[0049] In some embodiments, if reqX[0:2M-1] is not received by the interface cell, the interface cell may eventually receive reqX_t sent by the protocol control circuit and further activate a load signal (not shown in FIG. 8) to the read controller. Upon receiving the load signal, the read controller may activate Req to the output format circuit to continue the protocol even though the reqX signal is not active. The output format circuit then suppresses reading of empty rows.

[0050] Upon receiving Ack from the output format circuit, the read controller can activate rowack to the row select circuit. The row select circuit can then deactivate two signals, rowreq and ackY. Deactivating ackY can trigger the deactivation of reqX, reqX_t, rowack, Req, and Ack. Deactivating rowreq can indicate to the read controller that ackY has been deactivated, which can further deactivate rowack.

[0051] 9 is a schematic diagram of an exemplary architecture of a chip-top circuit 900 for an event-based vision sensor according to an embodiment of the present disclosure. The chip-top circuit 900 includes a pixel array 902. As an example, the pixel array 902 can have a surface aspect ratio of 16:9 (e.g., 1280 x 720 pixels). The chip-top circuit 900 can also include an asynchronous readout circuit 904 (a "Y arbiter" or "row selector") for selective readout (e.g., readout of rows of pixels), an X latch 906 for storing (e.g., temporarily storing) pixel data from the selected readout (e.g., row of pixels selected by the asynchronous readout circuit 906), and a vector readout and timestamp circuit 908 for receiving data from the X latch 906 and synchronizing, packetizing, and timestamp the received data (e.g., by attaching a digital timestamp to the data packets). Additionally, chip-top circuitry 900 may include an event signal processing (ESP) circuit 910 for receiving data from vector readout and timestamp circuitry 908 and for filtering, formatting, and preprocessing the received data, and a digital interface 912 for receiving data from ESP circuitry 910 for sending the data off-chip. By way of example, digital interface 912 may be implemented as a serial interface, a parallel interface, a low-voltage differential signaling (LVDS) interface, a mobile industry processor interface (MIPI), or another suitable type of interface. In one embodiment, digital interface 912 is implemented as a parallel interface and has a suitable width, such as 16 bits wide.

[0052] 9 , the chip-top circuitry 900 may further include an on-chip power management circuit 914 for high-level integration of components, a configurable ROI circuit 916 for cropping or sub-sampling configuration, a digital data interface 918 (e.g., a serial peripheral interface, also known as “SPI”) for receiving setup data sent to the chip-top circuitry 900 during startup, a register map 920 for storing the setup data received by the digital data interface 918, a bias generator 922 (e.g., a digital-to-analog converter, also known as “DAC”), a bias generator control circuit 924 for controlling analog circuitry in the bias generator 922 (e.g., via SPI commands) based on the register setting data stored in the register map 920, and a state machine 926 for controlling the readout protocol by sending and receiving signals to and from the asynchronous readout circuit 904, the X latch 906, and the vector readout and timestamp circuit 908. In some embodiments, the ESP circuitry 910 may include a look-up table (LUT)-based address filter (not shown in FIG. 9 ) for filtering out selected events (e.g., from defective pixels). Consistent with embodiments of the present disclosure, the chip-top circuit 900 may additionally or alternatively include other components than the example shown and described in connection with FIG.

[0053] The above description has been presented for purposes of illustration. It is not exhaustive and is not limited to the precise forms and embodiments disclosed. Modifications and adaptations of the embodiments will be apparent from consideration of the specification and practice of the disclosed embodiments. For example, while the described implementations include circuits and hardware, systems and methods consistent with the present disclosure may be implemented by any suitable combination of hardware, firmware, and / or software. Further, while certain components are described as being coupled together, such components may also be integrated with each other or distributed in any suitable manner.

[0054] Furthermore, while illustrative embodiments have been described herein, the scope includes any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., combinations of aspects across various embodiments disclosed herein), adaptations, and / or variations based on this disclosure. Claim elements should be interpreted broadly based on the language used in the claims and not limited to the examples described herein and during prosecution of the application, which examples should be interpreted as non-exclusive. Furthermore, the steps of the disclosed methods can be modified in any manner, including rearranging steps and / or inserting or deleting steps.

[0055] The features and advantages of the present disclosure will be apparent from the detailed specification, and therefore, the appended claims are intended to cover all systems and methods that fall within the true spirit and scope of the present disclosure. As used herein, the indefinite articles "a" and "an" mean "one or more." Similarly, the use of plural terms does not necessarily imply a plurality unless the context clearly dictates otherwise. Words such as "and" or "or" mean "and / or" unless otherwise specified. Moreover, because numerous variations and modifications will readily occur from consideration of this disclosure, it is not desired to limit the disclosure to the exact construction and operation shown and described, and, therefore, resort may be made to all suitable modifications and equivalents that are within the scope of the present disclosure.

[0056] Other embodiments will be apparent from consideration of the specification and practice of the embodiments disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosed embodiments being indicated by the following claims.

Claims

1. A sampling circuit for sampling data from pixels of an event vision sensor having rows and columns of pixels, the sampling circuit comprising: a row selection circuit electrically coupled to a row of pixels of the event vision sensor, the row selection circuit comprising: receiving an activation signal (reqY) from an active pixel of said row; generating an acknowledge signal (ackY) in response to receiving the activation signal (reqY); sending the acknowledge signal (ackY) to pixels of the row, each pixel being configured to generate a request read signal (reqX) when the pixel is active upon receiving the acknowledge signal (ackY), and output the request read signal (reqX) to an interface circuit; a row selection circuit configured to: a plurality of protocol control circuits, each protocol control circuit electrically coupled to a pixel and located at the end of a pixel row, such that the protocol control circuit receives the acknowledge signal (ackY) after all pixels in the row have received the acknowledge signal (ackY), the pixel having a distance to the row selection circuit, the distance being longer than any of the pixels in the respective row; receiving the acknowledge signal (ackY) from the row selection circuit; generating a process read signal (reqX_t) in response to receiving the acknowledge signal (ackY); a protocol control circuit configured to: Equipped with a sampling circuit electrically coupled to the row of pixels and configured to enable the row selection circuit, the protocol control circuit, and the row of pixels after receiving the process read signal (reqX_t) on the row;

2. the interface circuit is further configured to generate a load signal for output to a read controller in response to receiving the process read signal (reqX_t); 2. The sampling circuit of claim 1, wherein the readout controller is electrically coupled to the interface circuit and the row select circuit, and is configured to reset the row select circuit, the protocol control circuit, and the pixels of the row in response to receiving the load signal from the interface circuit.

3. A sampling circuit as described in claim 1 or 2, wherein when a pixel of a row line receives the acknowledge signal (ackY), if the temporal contrast determined by the pixel exceeds a threshold, the request read signal (reqX) is generated by the pixel.

4. 4. The sampling circuit of claim 3, wherein the request read signal (reqX) comprises a first signal (reqX ON) indicating a positive change in the temporal contrast and a second signal (reqX OFF) indicating a negative change in the temporal contrast.

5. 5. The sampling circuit of claim 1, wherein the interface circuit is further configured to, in response to receiving the request read signal (reqX), receive data signals from all pixels that generated the request read signal (reqX).

6. an address encoding circuit electrically coupled to the row of pixels and the row selection circuit, the address encoding circuit configured to generate address signals (addrY) for all pixels that generated the request read signal (reqX) in response to receiving the acknowledge signal (ackY) from the row selection circuit; The sampling circuit of claim 5 further comprising:

7. an output format circuit electrically coupled to the interface circuit and the address encoding circuit, the output format circuit comprising: receiving the data signal from the interface circuit and the address signal (addrY) from the address encoding circuit; The data signal and the address signal (addrY) are used to generate output data. an output format circuit configured to The sampling circuit of claim 6 further comprising:

8. a timer circuit electrically coupled to the interface circuit and the row select circuit, the timer circuit comprising: receiving a start signal (rowreq) generated by the row selection circuit in response to receiving the activation signal (reqY); resetting the row selection circuit, the protocol control circuit, and the pixels of the row if no data signal is received by the interface circuit after a time interval from a timestamp at which the start signal (rowreq) is received exceeds a threshold time; A timer circuit configured to The sampling circuit according to any one of claims 5 to 7, further comprising:

9. The sampling circuit of claim 8, wherein the interface circuit is further configured to generate a load signal for output to a read controller in response to receiving the process read signal (reqX_t), the read controller comprising the timer circuit.

10. A sampling circuit described in any one of claims 1 to 9, wherein the activation signal (reqY) is generated by the active pixel when the temporal contrast determined by the active pixel exceeds a threshold.

11. Each pixel is a comparator configured to generate a first request read signal when an input signal of the comparator meets a first condition, the input signal being generated from the photosensitive element in response to a temporal contrast in luminance of light incident on the photosensitive element, the temporal contrast exceeding a threshold; a first latch circuit comprising a first set input, a first reset input, and a first output, the first reset input and the first output being electrically coupled to the interface circuit, and the first set input being electrically coupled to the comparator; receiving the first request read signal from the comparator via the first set input; outputting the first read request signal to the interface circuit; locking the first set input to receive no signal; resetting the first set input to receive a new signal in response to receiving the acknowledge signal (ackY) from the interface circuit via the first reset input; a first latch circuit configured as follows: The sampling circuit according to any one of claims 1 to 10, comprising:

12. 12. The sampling circuit of claim 11, wherein the comparator is further configured to generate a second request read signal when the comparator input signal meets a second condition.

Images