Row selector for event readout

Abstract

A method for operating an event-based image sensor comprising a plurality of pixel circuits (1), each pixel circuit being configured to detect a change in illumination and thus activate a row request, where a pixel array comprises a readout circuit (14), where:-at the beginning of a readout cycle, the readout circuit (14) is configured to detect a change in illumination and thus activate a row request; the row requests activated at the start are sampled as activated sampled row requests,-the activated sampled row requests are sequentially selected by the organizer, and in each row with the selected sampled activated row request, each pixel circuit with the activated row request is read and reset, and then the other row is selected,-the activated sampled row requests are sampled as activated sampled row requests,-the activated sampled row requests are sequentially selected by the organizer, and in each row with the selected sampled activated row request, each pixel circuit with the activated row request is read and reset. The organizer selects the sample activation row requests in a sequence organized by a predetermined rule based on the respective row order of the rows having the sample activation row requests until all of the sample activation row requests have been selected, where the rows are associated with the row order, and the organizer selects the sample activation row requests in accordance with the sequence organized by the predetermined rule based on the respective row order of the rows having the sample activation row requests.

Description

Technical Field

[0001] The present invention relates to a method for operating an event-based image sensor (also referred to as a neuromorphic sensor). Background Art

[0002] In a conventional camera, the device records photo after photo. In an event-based image sensor, there are no photos. Like a camera, the integrated circuit contains a matrix of light sensors. However, in a conventional camera, each light sensor samples at a fixed frequency, while in a temporal contrast sensor, the pixels are not sampled: each pixel calculates the change in the intensity of the light it senses, optionally performs some processing on this intensity change, and when the calculated quantity exceeds a defined level or threshold, the pixel generates an "analog pixel event" or APE.

[0003] The APEs must be processed to transmit information about the light intensity change that occurred at the pixel where the APE originated. Readout is performed to transmit the APEs in the form of digital events to a processing pipeline where various processing operations such as filtering, formatting, packing, interfacing, etc. are performed. Some processing elements, such as filters, and especially spatio-temporal filters and anti-flicker, require access to data related to pixels previously stored in memory. Many memory accesses are involved, which means significant power consumption, especially since the memory is typically SRAM (static random access memory). SRAM has low idle power consumption but high operating power consumption. Thus, the power consumption of SRAM is directly related to the frequency at which it is accessed.

[0004] To achieve a good rendering of the image dynamics, the sensor is configured such that even a tiny light change may trigger an APE in at least one pixel. Additionally, the always desired higher resolution means more pixels generating events. Thus, many events must be processed per second, involving many memory accesses. Consequently, the sensor experiences a large amount of power consumption.

[0005] Therefore, there is a need for an event-based image sensor that has reduced power consumption without sacrificing resolution or sensitivity. Summary of the Invention

[0006] The present invention relates to a method for operating an event-based image sensor that includes a plurality of pixel circuits arranged in rows and columns to form a pixel array, each pixel circuit being arranged at a location defined by an address, wherein each pixel circuit is configured to detect a change in illumination in a photosensor signal derived from a photocurrent generated by light incident on a photosensitive element of the pixel circuit and thereby activate a row request, and wherein the pixel array includes a readout circuit coupled to the pixel circuits via row lines, the readout circuit being configured to receive the row requests from the pixel circuits and interact with the pixel circuits to read and reset the pixel lines.

[0007] During a readout period, multiple pixel circuits are read and reset row by row by a readout circuit, and:

[0008] - The readout period is initiated by a row request sensed on at least one row line. In response to the row request sensed on at least one row line and a sampling request synchronized with a clock signal, the row request activated at the start of the readout period is sampled as an activated sampled row request.

[0009] - The activated sampled row requests are selected by an organizer. In each row having a selected sampled activated row request, each pixel circuit having an activated row request is read and reset. Subsequently, another row having a sampled activated row request is selected until all sampled activated row requests are selected.

[0010] Wherein, the row is associated with a row order, and the organizer installs a sequence organized by a predetermined rule and selects the sampled activated row requests based on the corresponding positions of the rows having the sampled activated row requests in the row order. Wherein, the sequence in which the activated sampled row requests are selected is independent of the order in which the row requests are activated.

[0011] Other preferred but non-limiting aspects of the present invention are as follows, which are either separate or technically feasible combinations:

[0012] - The predetermined rule for organizing the sequence includes a descending or ascending order of rows in the pixel array;

[0013] - During the readout period, since the empty arbiter signal indicates that the organizer is not empty, newly activated row requests are not sampled as activated sampled row requests. When no row request is being processed, the organizer is empty;

[0014] - The readout circuit includes an input logic coupled to the row lines. The readout circuit is configured to sense a row request on at least one row line and send a row request indicator to a state machine to indicate that at least one row line has a row request, thereby initiating the readout period;

[0015] - The row request activated at the start of the readout is converted by the input logic of the readout circuit into a derived row request, and then the derived row request is sampled as an activated sampled row request. Wherein, the conversion of the row request to the derived row request is conditional on selecting the row associated with the row request by a selection signal;

[0016] - The state machine is configured to send a sampling request to an interface, and the interface is configured to sample the row request on the row line in response to the row request, and the sampling request is synchronized with the clock signal;

[0017] - The sampling request is generated by the state machine in response to a row request indicator and a read confirmation from the pixel circuits of the column interface indicating readiness to read a new row of pixels, and / or is generated periodically during the readout of the pixel circuits of a row;

[0018] - The sampling request is generated by the state machine synchronized with a frame rate defined by an external signal.

[0019] The present invention also relates to a method for operating an event-based image sensor, the image sensor including a plurality of pixel circuits arranged in rows and columns to form a pixel array, each pixel circuit having a state and being arranged at a position defined by an address, wherein each pixel circuit is configured to detect a change in light illumination and thus change its state to have an active event, wherein the pixel array includes a readout circuit coupled to the pixel circuits, the readout circuit including an event data formatter configured to receive the state of the pixel circuits and the addresses of the pixel circuits having active events, wherein the event data formatter is configured to encode the state of the pixel circuits into a frame packet, the frame packet including a series of data words describing the state of each pixel circuit in the pixel array,

[0020] wherein the frame packet includes a first type of data word quantifying adjacent pixel circuits without active events and a second type of data word reflecting the occurrence of active events in a set of adjacent pixel circuits.

[0021] Other preferred but non-limiting aspects of the present invention are as follows, which are either separate or in technically feasible combinations:

[0022] - Each data word includes a header and a payload, the header containing a code depending on the type of data word, and the payload containing: in the first type of data word, a count of adjacent groups of adjacent pixel circuits without active events, and in the second type of data word, a series of bits indicating whether an active event has occurred in each pixel circuit of a set of adjacent pixel circuits, wherein the bit arrangement reflects the arrangement of the pixel circuits in the set of adjacent pixel circuits;

[0023] - An event is characterized by a polarity reflecting the direction of change in light illumination, and the second type of data word includes: a first subtype reflecting the occurrence of active events of a first polarity in a set of adjacent pixel circuits, and a second subtype reflecting the occurrence of active events of a second polarity in a set of adjacent pixel circuits;

[0024] - At least one first type of data word in the frame packet is determined based on the address of the pixel circuit having an active event;

[0025] - The event data formatter receives the state of each pixel circuit in a set of adjacent pixel circuits and performs a test to detect whether no event has occurred in the set of adjacent pixel circuits;

[0026] - The groups of adjacent pixel circuits are equal-sized subdivisions of the rows of the pixel array. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Other aspects, objects, and advantages of the present invention will become more apparent after reading the following detailed description of the preferred embodiments of the present invention given as non - limiting examples and referring to the drawings, in which:

[0028] - Figure 1 Shows a simplified diagram of a pixel array of an event - based image sensor according to a possible embodiment of the present invention;

[0029] - Figure 2 Shows a simplified diagram of a pixel structure of a pixel array of an event - based image sensor according to a possible embodiment of the present invention;

[0030] - Figure 3 Shows a simplified diagram of the interaction between components of a row selector according to a possible embodiment of the present invention;

[0031] - Figure 4 Shows an example of a state diagram of a state machine;

[0032] - Figure 5 Shows a timing diagram that shows how various signals change in a simplified example of operation involving only three rows;

[0033] - Figure 6 Shows a schematic example of a row - level circuit of an input logic;

[0034] - Figure 7 Shows a schematic example of a row - level interface circuit;

[0035] - Figure 8 Shows a simplified schematic diagram of an example of an arbitration tree for row selection;

[0036] - Figure 9 Shows an Figure 8 arbitrator unit circuit in an arbitration tree having multiplexed four request inputs and four acknowledgement outputs;

[0037] - Figure 10 Shows a possible embodiment of a compressed event packet data format using an 18 - bit word including a 2 - bit header and a 16 - bit payload;

[0038] - Figure 11 Is an example of a sensor matrix that shows the generation of a compressed frame in a global synchronous operation mode during pixel array readout. DETAILED DESCRIPTION

[0039] In the present disclosure, "row" and "column" refer to two different dimensions of the readout periphery, and their naming is arbitrary. Both of the two dimensions of the readout periphery can be interchanged without changing the function of the readout periphery. For the sake of convenience of explanation and without causing ambiguity, hereinafter, the first dimension and the second dimension accessed by the pixel are respectively referred to as "row" and "column", which are represented by the dimension identifiers "Y" and "X", respectively.

[0040] Figure 1 A simplified diagram of an event-based image sensor according to a possible embodiment is shown. The event-based image sensor includes a plurality of pixel circuits forming a pixel array, or more simply, pixel 1. For clarity, Figure 1 A limited number of pixels 1 are shown. It should be understood that the pixel array includes a large number of pixels 1, such as having a size exceeding 256x256 pixels, and for example, a resolution of 1280x720 pixels.

[0041] As Figure 2 shown, each pixel 1 includes a photosensor circuit 10 and a change detector 12. The photosensor circuit 10 is configured to transfer a photosensor signal derived from a photocurrent generated by light irradiating a photosensitive element of the photosensor 10. The change detector 12 is configured to detect a change in the photosensor signal V pr . A band-pass filter may be provided between the photosensor circuit 10 and the change detector 12 to filter the photosensor signal V pr .

[0042] The photosensitive element of the photosensor 10 is typically a photodiode and converts incident light into a photocurrent determined by the exposure of the photosensitive element. The photosensor signal V pr is typically logarithmically related to the photocurrent. Typically, the photosensor signal V pr can be approximated as

[0043] V pr = k1 ln(I pr ) + k2

[0044] where I pr is the intensity of the photocurrent, and k1 and k2 are constant factors. The instantaneous voltage value of the photosensor signal V pr is logarithmically related to the instantaneous intensity of the photocurrent, and thus the measurement of the photosensor signal V pr allows the exposure level of the photosensitive element of the photosensor circuit 10 to be obtained.

[0045] The change detector 12 is configured to detect a change in the photosensor signal V pr . The change detector 12 continuously monitors the change in the photosensor signal V pr and issues a detection signal each time a change is detected. The detection signal identifies the photosensor signal Vpr the fraction exceeding the adjustable voltage threshold increases or decreases. Thus, the photosensor signal V pr variation is converted into an analog pixel event or APE in the pixel detection signal.

[0046] The event is characterized as being at least polar (indifferently designated as high / low, positive / negative or ON / OFF), reflecting the direction of change of the photosensor signal V caused by the APE. pr More precisely, when the photosensor signal V pr increases, i.e., exceeds the threshold of the previous value, the change detector 12 generates a high-polarity event, and when the photosensor signal V pr decreases, i.e., is below the threshold relative to the previous value, the change detector 12 generates a low-polarity event. A comparator is typically used to compare the instantaneous value of the photosensor signal V pr with a previous value used as a reference. The event is characterized not only by its polarity but also by the pixel address, which consists of the row coordinate and column coordinate (x, y respectively) of the pixel 1 that generated the event. The event is also generated or sent at a specific time and can also be characterized by the occurrence time, which can be encoded in a timestamp associated with the event. A change detector 12 is described in patents US 7,728,269 and US 8,780,240, which can be used to detect changes in the light intensity received by a pixel, for example, in the case of a dynamic vision sensor (DVS).

[0047] The pixel array includes a readout circuit 14 coupled to the pixel circuit via row lines 3. The readout circuit 14 typically includes several different modules, such as a row selector 2 coupled to the pixel circuit via row lines 3 or a line interface 4 (or X interface) coupled to the pixel circuit 1 via column lines 5. The line interface 4 receives data such as column requests from the read pixel circuit 1 according to the light change, for example, an ON column request for indicating a positive light change (i.e., an increase) and / or an OFF column request for indicating a negative light change (i.e., a decrease). The readout circuit 14 is configured to receive row requests from the pixel circuit 1 and interact with the pixel circuit 1 to read and reset the pixel circuit 1.

[0048] The readout circuit 14 may further include an event data formatter 6 or EDF, which is configured to receive column data from the line interface 4 and row addresses from the row selector 2. The event data formatter 6 formats the received data for transmission over a bus for further processing. For example, the event data formatter 6 encodes the received data into packets. The event data formatter 6 may also associate a timestamp with the received data, such as the time corresponding to the selected row or the received row address, or the time to read an entire frame, or the time corresponding to the time associated with an external readout signal, or more precisely an externally driven row request sampling signal. The timestamp may be generated by the event data formatter 6 or a dedicated timestamp generator.

[0049] More specifically, the readout circuit 14 is configured to sample row requests on the row lines and send the sampled row requests to an organizer of the readout circuit 14. During a readout cycle, multiple pixel circuits 1 are read and reset row by row by the readout circuit 14. The readout cycle is initiated by a row request sensed on at least one row line, and in response to sensing the row request on at least one row line and a sampling request synchronized with a clock signal, the row request activated at the start of the readout cycle is sampled as an activated sampled row request. The readout cycle is initiated by a state machine in response to sensing a row request on at least one row line. The sampling of the activated row request is caused by the state machine sending a sampling request synchronized with the clock signal in response to sensing the row request on at least one row line.

[0050] As long as the sampled row request is not acknowledged, the organizer determines which row to select in the rows having unacknowledged sampled row requests based on a predetermined rule, and the pixel circuits 1 of the selected row having a row request are read such that the row request of the selected row is acknowledged.

[0051] In a preferred embodiment, the organizer is an arbitration tree that is a selector based on a tree structure. Other types of organizers, such as scanners, may be used to perform the selection of active or locked rows in the interface. However, the arbitration tree is particularly advantageous for fast row selection, especially when not many rows are active.

[0052] In the following description, the organizer is an arbitration tree, and the signals related to the organizer are labeled as arbiter signals. For example, the empty organizer signal signaling that the organizer is empty is an empty arbiter signal. All features and signals related to the arbitration tree apply to the organizer.

[0053] Row selection does not follow a simple scan routine of scanning all rows, but rather uses certain pixel array signals to determine the active rows to be served and the optimal sequence of readout operations. The system is controlled by a state machine and benefits from the sparsity of pixel activation by allowing, for example, skipping inactive portions of the array during a readout cycle. The inherent temporal accuracy of event pixel data can be fully preserved.

[0054] Reference Figure 3 、 Figure 4 and Figure 5 , an operation example of the event-based image sensor will now be described. Figure 3 is a simplified diagram of the interaction between components of the row selector of the readout circuit. Figure 4 is the state diagram of the state machine 24. The transition conditions are represented by letters, and the actions are represented by Roman numerals. Figure 5 is an example of a waveform diagram of the row selector operation, where an example of three rows is shown. In this figure, the low level of the waveform corresponds to 0 or the deactivated state, while the high level of the waveform corresponds to the activated state or 1. This convention is not restrictive, and other conventions can be used, such as where the low level corresponds to the activated state or 1. In this example, the sensor is initially idle, and Figure 5 the row address y_address in

[0055] Initially, the state machine 24 is in the idle state S0, where the state machine 24 waits for the generation of the row request req_y. To this end, the state machine 24 can test the row request indicator req_y_or, whose activated state indicates that at least one row request signal req_y is activated. If the row request indicator req_y_or is deactivated (i.e., req_y_or = 0), there is no active row request signal req_y, and the state machine remains in the idle state S0. The row request indicator req_y_or is managed, for example, by the input logic circuit 22, which receives all the row requests req_y and can generate the row request indicator req_y_or from the row requests req_y through a simple OR logic gate. Since no row request is being processed, the arbiter 28 is empty, and thus the empty arbiter signal arb_empty is activated (i.e., arb_empty = 1).

[0056] When the photosensor signal V is detected by the change detector 12 of the pixel circuit 1 prWhen there is a light change in, the row request req_y is generated by the pixel circuit 1 on the row line 3 coupled to the readout circuit 14. Let's note the row request req_y[n] of row n. At Figure 5 In the example of, at the first moment T1, the light change is detected by the pixels of the third row, and the pixel circuit 1 sets the row request req_y[2] of the third row to 1 on the row line 3 coupled to the pixel circuit 1. It should be noted that the time T1 does not correspond to the rising edge or the falling edge of the clock signal clk, because the change detection and the row request req_y are event-driven and thus out of sync with the clock signal clk.

[0057] In the row selector 2 of the readout circuit 14, the input logic circuit 22 coupled to the row line 3 senses that the row request req_y is activated on at least one row line 3 and thus activates the row request indicator req_y_or. The activation state of the row request indicator req_y_or indicates that at least one row line 3 has a row request req_y that is activated and sensed by the state machine 24. As Figure 5 shown, the row request indicator req_y_or is set to 1 at time T1 while activating the row request req_y[2] of the third row.

[0058] Shortly after the first moment T1, the light change is detected by the pixels of the second row, and the pixel circuit 1 sets the row request of the second row req_y[1] to 1. Even though the input logic circuit 22 detects the activation of the second row request req_y[1], the row request indicator req_y_or does not change, having been activated in the high state (i.e., 1).

[0059] In addition to activating the row request indicator req_y_or, the input logic circuit 22 can also activate the derived row request signal req_y_d[n] for each corresponding activated row request req_y[n], and the row request signal req_y_d[n] is conveyed to the interface circuit 26. The condition for activating the row request signal req_y_d[n] in response to the corresponding row request signal req_y[n] is that the empty arbiter signal arb_empty is activated (i.e., arb_empty = 1). More specifically, as long as the empty arbiter signal arb_empty is activated, the states of all row requests, whether activated or deactivated, are reflected. During the readout period, since the empty arbiter signal indicates that the arbitration tree is not empty, newly activated row requests are not sampled into the activated sampled row requests. As Figure 5As shown, the exported row request signals req_y_d[2] of the third row request req_y[2] and req_y_d[1] of the second row request req_y[1] are activated simultaneously with their respective row requests req_y[2] and req_y[1]. The exported row request signals req_y_d[0:N-1] allow selection of the row requests to be processed during the readout period. If, for example, a new row request, such as req_y[0], appears after the activation of the empty arbiter signal arb_empty, the readout period is not affected.

[0060] At time T2, synchronized with the clock signal clk (more precisely, at the rising edge of the clock signal clk), in response to the active state of the row request indicator req_y_or, the state machine 24 of the readout circuit 14 transitions from the idle state S0 to the first state S1 and activates the sampling request clk_req for the interface circuit 26. The sampling request clk_req is synchronized with the clock signal clk, i.e., synchronized with the rising or falling edge of the clock signal clk, to allow proper processing of the components of the readout circuit.

[0061] Preferably, since the asynchronous row request indicator req_y_or should be synchronized with the state machine clock to avoid metastability, it is possible to have a one-clock-cycle delay between the activation of the row request indicator req_y_or by the state machine 24 and the activation of the sampling request clk_req. This is Figure 5 the case where, after the activation of the row request indicator req_y_or, i.e., at time T2, the sampling request clk_req is activated at the second rising edge of the clock signal clk instead of at the first rising edge after time T1.

[0062] As described above, the activation of the sampling request can be fully conditional on the reception of the row request indicator req_y_or and can thus be generated at the next moment of the clock signal clk. This method can be defined as a fully event-driven mode, in which, whenever the pixel circuit 1 detects a change in the illumination in its field of view, only the row / pixel circuit 1 is requested to be read out or confirmed on demand in a defined order. This event-driven method guarantees the minimum time between the detection of the illumination change by the pixel circuit 1 and its readout, especially for low to medium levels of scene activity. In this method, the readout timing is independent of any external signal (such as the frame rate).

[0063] By adjusting the activation of the sampling request clk_req to an external signal, the independence of the readout from the external signal can be alleviated. All row request signals req_y of the pixel array are sampled simultaneously by the external signal at a certain fixed or variable rate. This method can be defined as the global synchronization mode. For the event-driven mode, only the requested rows / pixels are acknowledged and read out in a defined order. In this global synchronization method, event data with a certain frame rate is generated, which is beneficial for use with frame-based processing methods / algorithms.

[0064] It is also possible to sample the row request signals req_y of the array row by row in a defined order at a certain fixed row rate and frame rate by an external signal, and again adjust the sampling request clk_req to an external signal, to adapt to standard, frame-based communication protocols and processing methods that cannot store a complete frame. If there are active pixel circuits 1 in a row (i.e., with an active row request req_y), they are read out and acknowledged, otherwise the readout circuit waits according to the row rate until the next row is accessed. This method can be defined as the rolling synchronization mode. This method can ensure the continuous flow of data. Preferably, in order to benefit from the sparsity of light events most of the time, an external signal defining a frame rate that is faster than the time required to read out all pixels can be used, because most of them usually do not need to be read out. This allows for the implementation of a frame rate much faster than that of traditional image sensors.

[0065] In response to the activation of the sampling request clk_req by the state machine 24, the interface 26 samples the row requests req_y[0:N - 1] of all rows (i.e., the row request req_y[n] of each row n) as the sampled row requests req_y_arb[0:N - 1]. As shown, the sampled row requests can be derived from the row requests req_y_d[0:N - 1] provided to the interface 26 by the input logic circuit 22, and these row requests are the same as the row requests req_y_[0:N - 1] as long as the empty arbiter signal arb_empty is activated. In Figure 5 the example, at time T2, the interface 26 samples the derived row requests req_y_d in three rows: req_y_d[0] = 0, req_y_d[1] = 1, req_y_d[2] = 1. In Figure 7 the exemplary circuit of the row-level interface 26, a storage element 42 such as a D flip-flop receives the row request req_y[n] of row n, which is sampled as the sampled row request req_y_arb[n] by the sampling request clk_req.

[0066] The N sampled row requests req_y_arb[0:N-1] of all rows are available for the arbitration tree 28 of the readout circuit 1. The readout circuit 1 determines, via the arbitration tree 28, which row to select among the rows having unacknowledged sampled row requests req_y_arb[0:N-1] based on a predetermined rule. The rows are associated with a row order, which means that a position in the row order is associated with each row. Thus, each row can be identified and positioned relative to other rows in the row order. Typically, the row order simply consists of sequentially numbering the rows from the top to the bottom of the array (e.g., row 1 is above row 2, row 2 is above row 3, etc.), or conversely, sequentially numbering the rows from the bottom to the top of the array. Other types of row orders can be chosen. Based on the corresponding positions of the row requests having sampled active rows in the row order, the row order is traversed according to the predetermined rule.

[0067] The arbitration tree 28 sequentially selects the active sampled row requests req_y_arb, and in each row having the selected sampled active row request ack_y_arb, each pixel circuit having an active row request is read and reset before another row having a sampled active row request is selected, until all sampled active row requests req_y_arb have been selected. The predetermined rule is, for example, an ascending order (i.e., 0, 1, 2, etc.) or a descending order (i.e., N, N-1, N-2, etc.) for traversing the row order, or it can be another rule for traversing the row order. For example, a rule having non-consecutive rows in the row order can be used. The rows can also be organized into groups for consecutive traversal. For example, the first group of non-consecutive rows can be traversed before the second group of non-consecutive rows is traversed. For example, an interleaved sequence can be used, where the even rows (e.g., 0, 2, 4, 6, etc.) are considered before the odd rows (e.g., 1, 3, 5, 7, etc.), or vice versa. Such an interleaved sequence will allow a trade-off between the temporal accuracy and the spatial accuracy of the sensor field of view.

[0068] Preferably, the sequence in which the active sampled row requests are selected is independent of the temporal order of the active row requests. Preferably, the same predetermined rule is used in each readout cycle, but it can be changed between two readout cycles. The predetermined rule is stored in the arbitration tree 28 as the rule based on which the rows to be processed are selected. The rule can be stored as the selection order of the rows, i.e., a list of rows specified by their positions in the row order and organized according to the predetermined rule.

[0069] Preferably, the rule organizes the traversal of the entire row order. In other words, if all rows have sampled active row requests, the application of the rule will result in the organizer 28 selecting all rows one by one.

[0070] In the depicted example, the sampled row request signals are designated as req_y_arb because they are arbiter inputs, but they carry the same information or status as the derived row requests req_y_d[0:N-1], i.e., the status of the row requests req_y[0:N-1] before the arbiter empty signal arb_empty is deactivated. The arbitration tree 28 selects a row n in rows 0 to N-1 based on a predetermined order, where the sampled row requests req_y[0:n-1] are active (i.e., req_y_arb[n]=1). The arbitration tree 28 activates the acknowledged row request ack_y_arb[n] of the selected row n (ack_y_arb[n]=1), and keeps the acknowledged row requests ack_y_arb of the N-1 rows other than the selected row n (ack_y_arb[i≠n]=0) deactivated.

[0071] In Figure 5 the example, at time T2, the arbitration tree 28 must select a row between the second row and the third row because their respective sampled row requests req_y_arb[1;2] are active. The first sampled row request req_y_arb[0] is not active (i.e., req_y_arb[0]=0), and thus the first row cannot be selected. In this non-limiting example, the arbitration tree 28 selects rows based on ascending order, which means the second row is selected before the third row, even though the second row request req_y[1] is slightly activated after the third row request req_y[2]. Therefore, the arbitration tree 28 activates the acknowledged row request ack_y_arb[1] of the second row (ack_y_arb[1]=1), and keeps the acknowledged row request ack_y_arb[0] of the first row (ack_y_arb[0]=0) and the acknowledged row request ack_y_arb[2] of the third row (ack_y_arb[2]=0) inactive to indicate that the corresponding rows are not selected at this time.

[0072] Figure 8Shows a simplified schematic diagram of an example of an arbitration tree 28 for row selection. In this non-limiting example, the arbitration tree 28 includes 4:1 arbiter units, i.e., multiplexing four rows. It is also possible not to multiplex, or to have another multiplexing factor. The arbiter units are organized in levels, with the higher levels connected to the lower levels. The outgoing tree row request req_y_tree_out and the outgoing multiplexed acknowledge request ack_y_tree_out are connections to the next level in the arbitration tree 28. The sampled row request req_y_arb is propagated through the arbitration tree 28 to the top, and the resulting acknowledge signal is conveyed back as the acknowledge row request ack_y_arb to the root of the arbitration tree 28. Additionally, at each stage, input signals (lsb, msb) for the y-address encoder are generated. The output of the last arbiter unit is also used as an indicator that the arbiter is empty (req_y_arb_or = 0) or not empty (req_y_arb_or = 1).

[0073] Figure 9 Shows Figure 8 A possible embodiment of the circuit 90 of the 4:1 arbiter unit in the arbitration tree 28, which circuit has four multiplexed request inputs and four acknowledge outputs. In the depicted example, the rows are the first four rows numbered 0 to 3, but of course other row numbers can also be processed. The circuit 90 includes a request path 91, an acknowledge path 92, and an address path 93 to generate input signals for a y-address encoder that generates a y_address signal.

[0074] In the request path 91, the req arb logic circuit 91b receives the incoming multiplexed tree row requests req_y_tree_In[0:3] on the input terminal 91a and outputs the outgoing tree row request req_y_tree_out to a higher level in the arbitration tree 28 on the output terminal 91c. Preferably, the req arb logic circuit 91b performs a logical "or" function on the incoming multiplexed tree row requests req_y_tree_in[0:3], which means that if one of the incoming multiplexed tree row requests req_y_tree_in[0:3] is activated (i.e., 1) to propagate the input signal to a higher level in the arbitration tree 28, then the outgoing tree row request req_y_tree_out must be activated (i.e., 1).

[0075] For example, a possible logic function to implement this function could be:

[0076] req_y_tree_out = req_y_tree_in[3] v req_y_tree_in[2] v req_y_tree_in[1] v req_y_tree_in[0].

[0077] Where "v" represents the "OR" function.

[0078] In the acknowledgment path 92, the ack arb logic circuit 92b receives an incoming tree acknowledgment request ack_y_tree_in from a higher level at the input terminal 92a and outputs an outgoing multiplexed acknowledgment request ack_y_tree_out[0:3] to a lower level in the arbiter tree 28 at the output terminal 92b. The outgoing multiplexed acknowledgment request ack_y_tree_out[0:3] must be activated (i.e., 1) to transmit the corresponding acknowledgment signal to a lower level in the arbiter tree 28. Therefore, if the incoming tree acknowledgment request ack_y_tree is activated (i.e., 1), the outgoing multiplexed acknowledgment request ack_y_tree_out[n] must be activated only for the first signal n in the bus of the activated req_y_tree_in[0:n:3]. The input terminal 91a in the row request path 91a is connected to the ack arb logic 92b to provide the incoming multiplexed tree row request req_y_tree_in[n]. For example, a possible logic function to implement this function can be:

[0079] ack_y_tree_out[0] = req_y_tree_in[0] ^ ack_y_tree_in

[0080] ack_y_tree_out[1] = (req_y_tree_in[1] ^ ack_y_tree_in) ^!req_y_tree_in[0]

[0081] ack_y_tree_out[2] = (req_y_tree_in[2] ^ ack_y_tree_in) ^!req_y_tree_in[1] ^!req_y_tree_in[0]

[0082] ack_y_tree_out[3] = (req_y_tree_in[3] ^ ack_y_tree_in) ^!req_y_tree_in[2] ^!req_y_tree_in[1] ^!req_y_tree_in[0]

[0083] Where ^ represents the AND function and! represents logical negation.

[0084] The input signals (lsb, msb) of the y address encoder are generated by the y arb logic 93a and issued at the output terminal 93b. The input signals lsb, msb for the y address encoder are generated, for example, by the following equations:

[0085] lsb =!ack_y_tree_out[1] ^!ack_y_tree_out[0]

[0086] msb =!ack_y_tree_out[3] ^!ack_y_tree_out[2]

[0087] The y encoder generates a digital word of width log2(N), which encodes the address of the row selected for confirmation. Each bit n of the address consists of the OR connection of the output signals lsb[n] or msb[n] of the corresponding arbiter units of all rows:

[0088] - y_address[0] = OR(lsb[0][0:N / 4 - 1])

[0089] - y_address[1] = OR(msb[0][0:N / 4 - 1]))

[0090] - y_address[2] = OR(lsb[1][0:N / 16 - 1])) and so on.

[0091] Since at least one sampled row request is confirmed by the arbitration tree 28, the arbitration tree 28 activates the arbiter indicator req_arb_or (i.e., req_arb_or = 1) to indicate that the arbitration tree 28 is sensing at least one active sampled row request req_y_arb[n].

[0092] While selecting the row to be read, the arbitration tree 28 encodes the address of the selected row, i.e., the address of the row that activates the confirmed row request ack_y_arb[n], and transmits the encoded address in the address signal y_address[0; ceil(log2(N)) - 1], as described above. In this example, the address is represented in binary numbers, so the length of the address is log2(N). In Figure 5 the example, the second row is selected among three rows, and the address signal y_address is the encoded address d1 for the second row and is, for example, 0001. Other encodings can be used. The address signal y_address is sent to the event data formatter 6 responsible for forming the data packet and sending it to the bus. Typically, the address signal y_address is encoded into the header of the data packet containing the event vector read from that row.

[0093] In the next step, at time T3, in response to the arbiter indicator req_arb_or being activated and synchronized with the clock signal clk to synchronize this step with the clock signal, the state machine 24 transitions from the first state S1 to the second state S2 and activates the clock confirmation clk_ack (i.e., Figure 5where clk_ack = 1) so that interface 26 processes the acknowledgement row request ack_y_arb. In response to the clock acknowledgement clk_ack, interface 26 samples all the acknowledgement row requests ack_y_arb. The acknowledgement row request ack_y_arb is applied to the sensor array as the acknowledgement row signal ack_y. In this example, the acknowledgement row signal ack_y[1] of the second row is activated because the second row is the selected row, while the acknowledgement row signals ack_y[0;2] of the other rows remain deactivated. More precisely, as Figure 7 shown, in an example of an embodiment of the interface 26 row-level circuit, the selected acknowledgement row request ack_y_arb[n] is sampled by the clock acknowledgement clk_ack in a storage element 40 such as a D flip-flop. As a result, the output of the storage element is the acknowledgement row signal ack_y[n] sent to the sensor array. The sampled acknowledgement row signal ack_y_d[n] is similar to the acknowledgement row signal ak_y[n], but in Figure 7 it is referred to as the acknowledgement row signal ack_y_d[n] to indicate that it is sampled by the clock acknowledgement signal clk_ack. For simplicity, the sampled acknowledgement row signal ack_y_d[n] can be referred to as the acknowledgement row signal ack_y_d[n].

[0094] The state machine 24 also activates the row validity signal row_valid (i.e., row_valid = 1) to indicate that the address signal y_address is valid. In response to the arbiter indicator req_arb_or being activated, the state machine 24 also deactivates the empty arbiter signal arb_empty (i.e., arb_empty = 0) to indicate that the arbiter 28 is no longer empty. The empty arbiter signal arb_empty is sensed by the input logic 22. Since the empty arbiter signal arb_empty is deactivated, the input logic 22 no longer converts the incoming new row request req_y[n] into a derived row request req_y_d[n] that can be processed in this read cycle.

[0095] Figure 6 shows a schematic example of the input logic 22 row-level circuit. There are as many such circuits as there are rows in the pixel array. The input logic 22 activates or deactivates the derived row request req_y_d[n], or holds the derived row request req_y_d[n], based on the row request req_y[n], the state of the arbiter 28 indicated by the empty arbiter signal arb_empty, and the row state indicated by the sampled row request req_y_arb[n] from the interface 26. The select signal sel[n] is used to deselect a particular row that is not being read (sel[n] = 0). In addition, the derived row request signals req_y_d[0:N-1] of all rows are OR-connected to generate the row request indicator req_y_or at the block level.

[0096] The first NAND gate 30 receives the row request req_y[n] of the row, the select signal sel[n] of the row, and the empty arbiter signal arb_empty. As described above, the first NAND gate 30 is used to activate the derived row request req_y_d[n]. Once the empty arbiter signal arb_empty is deactivated, the first NAND gate 30 no longer activates the corresponding derived row request req_y_d[n]. The second NAND gate 32 receives the row request req_y[n] of the row, the select signal sel[n] for the row, the negation of the empty arbiter signal arb_empty, and the sampled row request req_y_arb[n] for the row. The second NAND gate 32 is used to maintain the state of the derived row request req_y_d[n] after the empty arbiter signal arb_empty is deactivated. The outputs of the first NAND gate 30 and the second NAND gate 32 are inputs to a third NAND gate 34, which outputs the derived row request req_y_d[n] for the row.

[0097] In response to the acknowledged row signal ack_y[n], all active pixel circuits 1 (i.e., those having unacknowledged row requests) in the selected row (i.e., row 2 in this example) send a data request or a column request req_x. Preferably, there are two types of column requests: an ON column request req_x_on for indicating a positive light change (i.e., an increase) and an OFF column request req_x_off for indicating a negative light change (i.e., a decrease). For example, a pixel circuit 1 that has experienced a positive light change will activate the ON column request req_x_on (i.e., req_x_on = 1), while a pixel circuit 1 that has experienced a negative light change will activate the OFF column request req_x_off (i.e., req_x_off = 1). If a pixel circuit 1 has experienced both an increase and a decrease in light since the last sample, there may be two column requests for the same pixel circuit 1. The column interface 4 samples the column requests req_x[M] for M columns from the selected row n. The column requests sampled by the column interface 4 are then sent to formatted event data, where the column requests are associated with the corresponding row addresses, e.g., in a data packet.

[0098] When the pixel circuit 1 is read, it is also reset, which means that the column requests req_x are deactivated (req_x_on = 0 and / or req_x_off = 0), and the row request is released (req_y[n] = 0). It should be noted that the reading and subsequent release of the row request req_y[n] are asynchronous and may take longer than one clock cycle of the clock signal clk, and may vary for different pixel rows. In Figure 5In the example, the time to read the second row is longer than that to read the third row, and the release of the row request req_y[1] for the second row takes approximately 2.25 clock cycles, while the release of the row request req_y[2] for the third row takes approximately 0.75 clock cycles.

[0099] After sampling the column request req_x[M] and once ready to read the next selected row, the column interface 4 confirms that the circuit pixel 1 of the selected row n has been read by activating the read confirmation send_ack_y (i.e., send_ack_y = 1) within a given duration (here during one clock cycle). At time T4, the activated send_ack_y causes the state machine 24 to transition from the second state S2 to the third state S3. By doing so, the state machine 24 issues a sampling request clk_req and deactivates the row validity signal row_valid (i.e., row_valid = 0) to indicate that no new row address y_address is provided.

[0100] After resetting the pixel circuit 1 in the second row, the row request req_y[1] for the second row is then released. It should be noted that the deactivation of the row request req_y[n] depends on the time required by the column interface 4 to read the pixel circuit and is preferably not regulated by any clock signal clk but is asynchronous. In Figure 5 the example, the deactivation of the row request req_y[1] for the second row occurs between the rising edge of the clock signal clk at time T4 and the next rising edge of the clock signal clk. At the same time, the derived row request req_y_d[1] for the second row returns to the deactivated state (i.e., req_y_d[1] = 0).

[0101] In the third state S3, the state machine 24 preferably generates a sampling request clk_req periodically at each clock signal clk. The sampling request clk_req is received by the interface 26, which samples all the row requests req_y[0:N-1] in the incoming sampled row requests req_y_arb[0:N-1] to update the sampled row requests req_y_arb[0:N-1] in order to detect the release of the previously saved row request req_y[n]. In fact, after the readout of the selected row is completed, the row request req_y[n] in the selected row is now deactivated, and the sampled row request req_y_arb[n] of the selected row sent by the interface 26 to the arbiter 28 is now deactivated. However, the other sampled row requests req_y_arb remain unchanged. In Figure 5 the example, the row request req_y[1] for the second row being 0 causes the sampled row request req_y_arb[1] for the second row to become 0, but the row request req_y[2] for the third row remains 1.

[0102] Interface 26 includes a combinatorial logic block that indicates the situation where during a request confirmation period before the release of the confirmation line signal ack_y[n], the sampled row request req_y_arb[n] has been deactivated while the sampled confirmation line signal ack_y_d[n] remains active. In Figure 7 In an exemplary circuit of interface 26 at the row level, NOR gate 44 receives the sampled confirmation line signal ack_y_d[n] and the negation of the sampled row request req_y_arb[n] as inputs and outputs a row difference indicator ack1reqarb0[n] for that row. The signals ack1reqarb0[0:N-1] for all rows are OR-connected to generate a difference indicator signal ack1reqarb0_or at the array level. In our example, the row difference indicator ack1reqarb0[1] for the second row is activated, and thus the difference indicator ack1reqarb0_or is activated (i.e., ack1reqarb0_or = 1). As long as the difference indicator ack1reqarb0_or remains active, state machine 24 remains in the third state S3.

[0103] Since the sampled row request req_y_arb[n] for the selected row sent by interface 26 to arbiter 28 is now deactivated, arbiter 28 deactivates the confirmation line request ack_y_arb[n] for the previously selected row and checks if there are any unsampled row requests req_y_arb[n] that have not been confirmed. If there is at least one sampled row request req_y_arb still active on interface 26, arbiter 28 will select the next active row line based on a predetermined order. For example, the next selected active row is selected in ascending or descending order. A new confirmation line request ack_y_arb[n] is activated. In Figure 5 In the example of, the confirmation line request ack_y_arb[1] for the second row is deactivated (i.e., ack_y_arb[1] = 0). Since the sampled row request req_y_arb[2] for the third row is still active and is the next in the predetermined order, the confirmation line request ack_y_arb[2] for the third row is activated (i.e., ack_y_arb[2] = 1).

[0104] As previously mentioned, while selecting a row, arbiter tree 28 encodes the address of the selected row, i.e., the address of the row for which the confirmation line request ack_y_arb[n] is active, and transmits the encoded address in the address signal y_address[0; ceil(log2(N))-1]. In Figure 5 In the example of, the third row is now selected and the address signal y_address becomes the encoded address d2 for the third row.

[0105] Next, at time T5, the difference indicator ack1reqarb0_or is activated, and the arbiter indicator req_arb_or still being activated (i.e., req_arb_or = 1) causes the state machine 24 to transition from the third state S3 to the second state S2. In response to the state machine 24 sensing an outstanding row request due to the arbiter indicator req_arb_or, the state machine 24 activates the clock acknowledgement clk_ack which is used as a sampling signal (i.e., clk_ack = 1).

[0106] After sensing the activation of the clock acknowledgement clk_ack, the interface 26 samples all the acknowledgement row requests ack_y_arb. The acknowledgement row requests ack_y_arb are applied to the sensor array as the acknowledgement row signal ack_y. Thus, the change in the selected row is acknowledged in the acknowledgement row signal ack_y. The acknowledgement row signal ak_y is released by the previous selected row and is assigned to the newly selected row. In Figure 5 the example of, only the acknowledgement row request ack_y_arb[2] of the third row is acknowledged (ack_y_arb[2] = 1) at time T5, while the other rows have outstanding row requests (ack_y_arb[0] = 0, ack_y_arg[0] = 1). Thus, the acknowledgement row signal ack_y[1] of the second row is deactivated, while the acknowledgement row signal ack_y[2] of the third row is activated.

[0107] The row validity signal row_valid is also activated by the state machine 24 (i.e., row_valid = 1) to indicate that the address signal y_address is valid. Since the activation state of the acknowledgement row signal ack_y[n] of the newly selected row now matches the activation state of the sampled row request req_y_arb[n] of the same newly selected row, the interface 26 deactivates the difference indicator ack1reqarb0_or (i.e., ack1reqarb0_or = 0).

[0108] As described above, in response to the acknowledgement row signal ack_y[n], all the active (i.e., having outstanding row requests) pixel circuits 1 in the selected row (i.e., row n) send data requests or column requests req_x, such as ON column request req_x_on and / or OFF column request req_x_off. The column interface 4 samples the column requests req_x[M] for M columns originating from the selected row n. During the readout process of the pixel circuit 1, the row request req_y of the pixel circuit is reset. After each active pixel circuit 1 on the selected row line 3 has been read, the row request req_y[n] on the row line 3 becomes 0, and thus the derived row request req_y_d[n] also becomes 0.

[0109] In Figure 5In the example, after all the active pixel circuits 1 in the third row have been read, the row request req_y[2] of the third row is deactivated (i.e., req_y[2]=0). Once the row request req_y is read, it is released by the third row. As mentioned before, the timing of such operations depends on the readout and can be asynchronous with respect to the clock signal. At the same time, the derived row request req_y_d[2] returns to the deactivated state (i.e., req_y_d[2]=0).

[0110] Due to the release of the row request req_y[2] of the third row, no row request req_y in the three-row line 3 remains active, and the row request indicator req_y_or returns to the deactivated state (req_y_or = 0). Of course, if any row request req_y[N] is still active, the row request indicator req_y_or will still be active, and the row request req_y of another row will be processed as described above.

[0111] After sampling the column request req_x[M] and once ready to read the next selected row, the column interface 4 confirms that the circuit pixels 1 of the selected row n have been read by activating the read confirmation send_ack_y (i.e., send_ack_y = 1) for a given duration (here, during one clock cycle).

[0112] At time T6, the activated send_ack_y causes the state machine 24 to transition from the second state S2 to the third state S3. By doing so, the state machine 24 issues a sampling request clk_req and deactivates the row validity signal row_valid (i.e., row_valid = 0) to indicate that no new row address y_address is provided. Optionally, the row address signal y_address can return to a default address, such as Figure 5 d0 in the example.

[0113] In response to the sampling request signal clk_req, the interface 26 samples the derived row request req_y_d[N] as the sampled row request req_y_arb[N] through the interface 26. The activation state of the derived row request req_y_d[N] is transmitted to the sampled row request req_y_arb[N]. In Figure 5 the example, the derived row request req_y_d[2] of the third row has returned to the deactivated state, and the sampled row request req_y_arb[2] of the third row sent by the interface 26 to the arbiter 28 becomes deactivated, i.e., returns to 0.

[0114] Since the acknowledgement row signal ack_y[2] of the third row is still active while the sampled row request req_y_arb[2] of the same selected row is deactivated, the difference indicator ack1reqarb0_or is activated by interface 26 to indicate to the state machine 24 that the acknowledgement row signal ack_y[n] is held by a row without an active sampled row request req_y_arb[n].

[0115] The arbiter 28 deactivates the acknowledgement row request ack_y_arb[n] of the previously selected row and checks if there are still unacknowledged sampled row requests req_y_arb[n]. If so, the process returns to the operation described previously. If not, the arbiter 28 will not select a new sensor array row. In Figure 5 the example of, the acknowledgement row request ack_y_arb[2] of the third row is deactivated and there are no other unacknowledged sampled row requests req_y_arb[n] (all sampled row requests req_y_arb[n] are deactivated). In this case, the arbiter tree 28 deactivates the arbiter indicator req_arb_or (i.e., req_arb_or = 0) to indicate that the arbiter tree 27 is now empty.

[0116] The difference indicator ack1reqarb0_or is activated (ack1reqarb0_or = 1) and the arbiter indicator req_arb_or is deactivated (req_arb_or = 0), indicating that all row requests involved in the readout process have been processed. In this example, there are no longer any active row requests, as indicated by the deactivation of the row request indicator req_y_or (i.e., req_y_or = 0). Therefore, the state machine 24 transitions from the third state S3 to the idle state S0, waiting for a new row request to appear to start a new readout cycle. The state machine 24 activates the clock acknowledgement clk_ack (clk_ack = 1) and the empty arbiter signal arb_empty (i.e., arb_empty = 1) to indicate that the arbiter 28 is empty and can process new incoming row requests.

[0117] If there are active row requests waiting to be processed in the new cycle, the row request indicator req_y_or will be activated (i.e., req_y_or = 1) and the state machine will transition from the third state S3 to the first state S1. In addition to activating the clock acknowledgement clk_ack (clk_ack = 1) and the empty arbiter signal arb_empty (i.e., arb_empty = 1), the state machine 24 will also issue a sampling request clk_req so that the sampling of the row request req_y[N] can be processed. A new readout cycle will start.

[0118] It should be noted that if the state machine 24 is in the first state S1, and although there is a previous sampling request clk_req, the arbitration tree 28 still senses that there is no active sampling row request req_y_arb[n], and thus the arbiter indicator req_arb_or is deactivated (i.e., req_arb_or = 0), the state machine 24 will transition to the idle state S0 and deactivate the empty arbiter signal arb_empty (i.e., arb_empty = 0). This ensures that the row selector is not blocked, for example, in the case where there is only one active row request req_y[n] and it is removed before it is sampled.

[0119] The activation of the clock acknowledgement clk_ack (clk_ack = 1) causes the interface 26 to sample the acknowledgement row request ack_y_arb[n] and output the derived acknowledgement row signal ack_y[n]. In Figure 5 the example of, the sampled acknowledgement row signal ack_y[2] of the third row thus returns to the deactivated state (i.e., ack_y[2] = 0). Since the state of the acknowledgement row signal ack_y[n] now matches the state of the sampling row request req_y_arb[n], the interface 26 deactivates the difference indicator ack1reqarb0_or (i.e., ack1reqarb0_or = 0).

[0120] As mentioned before, by adjusting the activation of the sampling request clk_req to an external signal, the independence of the readout from the external signal can be alleviated. All row request signals req_y of the pixel array are sampled simultaneously by an external signal at a certain fixed or variable rate. This method can be defined as the global synchronization mode. It is also possible to sample the row request signals req_y of the array row by row in a defined order at a certain fixed row rate and frame rate by adjusting the sampling request clk_req to an external signal again, to accommodate standards, frame-based communication protocols, and processing methods that cannot store a complete frame. If there are active pixel circuits 1 in a row (i.e., having an active row request req_y), they are read out and acknowledged, otherwise the readout circuit waits according to the row rate until the next row is accessed. This method can be defined as the rolling synchronization mode.

[0121] Using an external trigger signal in global synchronization mode and rolling synchronization mode allows binary frames to be generated in a straightforward manner. A frame is a description of the state of each pixel circuit in a pixel array simultaneously. The binary frame is similar to the frame in a standard image sensor, with each pixel containing one of four possible values: no event, high-polarity event or on event (positive contrast), low-polarity event or off event (negative contrast), or on event and off event (both contrasts). In a binary coding scheme, each pixel circuit 1 requires 2 bits. In global synchronization mode, when all the pixels in a frame are sampled simultaneously, the same timestamp is assigned to them, while in rolling synchronization operation mode, all the pixels in a row have the same timestamp. The value of the timestamp can be derived (externally) from the frame rate (and line rate). The instantaneous frame rate can be inherently conveyed using the available frame rate encoding mechanism of an established data transfer method, such as the blanking signal in the Mobile Industry Processor Interface MIPI protocol.

[0122] Especially in the global synchronization operation mode with a high sampling rate for low to medium range scene activities, the binary frame contains only a few events, while each pixel circuit 1 still needs to transmit 2 bits. This either requires a permanently high transmission bandwidth (and power consumption) or limits the maximum possible frame rate.

[0123] A solution to this problem is to use a frame representation in a compressed event packet data format, which allows for dynamic encoding during the readout of the sensor array. The readout circuit 14 includes an event data formatter 6, which is configured to receive the state of the pixel circuit 1 and the address of the pixel circuit 1 with an active event (i.e., the address signal y_address). The event data formatter 6 forms data words of the frame packet and sends them to the bus.

[0124] The frame packet in the compressed event packet data format (i.e., the entire image) consists of a series of data words that describe the state of each pixel circuit 1 in the sensor array. The state of the pixel circuit 1 can be an empty state when the pixel circuit 1 does not generate an active event and no light change is detected, or it can be a state where an active event is generated or activated if the pixel circuit detects a light change. In the above operation method, the state of the pixel circuit 1 is directly given by the data request or column request req_x retrieved by the column interface 4 when reading the pixel circuit 1 and is provided to the event data formatter 6.

[0125] In the compressed event packet data format, the frame packet includes a first type of data word that quantifies adjacent pixel circuits with no active events, and a second type of data word that reflects the occurrence of active events in a group of adjacent pixel circuits. Each data word includes a header and a payload. The header contains a code that depends on the type of data word, i.e., it defines the nature of the data in the payload. The payload contains:

[0126] - In the first type of data word, the count of adjacent groups of adjacent pixel circuits 1 with no active events, and

[0127] - In the second type of data word, a series of bits indicates whether an active event has occurred in each pixel circuit 1 of a group of adjacent pixel circuits 1, and the bit arrangement reflects the arrangement of the pixel circuits in the group of adjacent pixel circuits 1.

[0128] More precisely, the event is characterized by a polarity that reflects the direction (increase or decrease) of the light intensity change, and the second type of data word includes:

[0129] - A first subtype that reflects the occurrence of active events of a first polarity in a group of adjacent pixel circuits, and

[0130] - A second subtype that reflects the occurrence of active events of a second polarity in a group of adjacent pixel circuits.

[0131] For example, the first polarity is an increase in light intensity, i.e., for ON events, and the second polarity is a decrease in light intensity, i.e., for OFF events.

[0132] The group of adjacent pixel circuits 1 is an equally sized subdivision of a row of the pixel array. For example, for an array with a row of 1280 pixel circuits 1, a group of adjacent pixel circuits 1 can consist of 16 consecutive pixel circuits, resulting in 80 groups per row of pixel circuits 1, or of 20 consecutive pixel circuits 1, resulting in 64 groups per row, or of 32 consecutive pixel circuits 1, resulting in 40 groups per row, and so on.

[0133] The states of the pixel circuits 1 in a group of adjacent pixel circuits 1 form a bit vector. The vector contains the same number of bits as the group of pixel circuits, and each bit describes the state of the corresponding pixel circuit 1 with the same arrangement: the first bit corresponds to the first pixel circuit 1 of the group of adjacent pixel circuits 1, the second bit corresponds to the second pixel circuit 1 of the group of adjacent pixel circuits 1, and so on. For example, for an array with a row of 1280 pixel circuits 1 divided into groups of 16 consecutive pixel circuits 1, the payload has a length of 16 bits, resulting in 80 vectors per row. If the row is divided into groups of 20 consecutive pixel circuits 1, the payload has a length of 20 bits, resulting in 64 vectors per row, and so on.

[0134] In the first type of data word, the payload contains the number of empty vectors (between the vectors containing events). In the second type of data word, the payload contains vectors including ON events or vectors including OFF events, and the data type of the payload is encoded in the header. Figure 10 A possible embodiment of the compressed event packet data format using an 18-bit data word including a 2-bit header and a 16-bit payload is shown.

[0135] In this example, the first code in the header (e.g., 00) indicates that the payload contains the number of empty vectors and thus belongs to the first type of data word. The number of empty vectors is, for example, binary-encoded and can range from 1 to 2 16 empty vectors. The second code in the header (e.g., 01) indicates that the payload contains vectors of ON events and thus belongs to the first subtype of the second type of data word. The third code in the header (e.g., 10) indicates that the payload contains vectors of OFF events and thus belongs to the first subtype of the second type of data word. In this example, a 16-bit payload is used, which means that the active events of 16 pixel circuits 1 in a row can be encoded in the payload. As an example of encoding, a bit "1" at position i in the j-th k-bit vector of a row (i, j, k are natural integers, where k is the width of the payload vector) may mean that an event has occurred at the ((j - 1)*k + i)-th pixel circuit 1 in the row. The header indicates whether the detected event is an ON event or an OFF event. Typically, "0" indicates no active event.

[0136] Of course, the size of the encoding or the header or the payload can be adjusted. Preferably, the length of the payload is selected such that the length of the row in terms of the number of pixel circuits 1 is a multiple of the length of the payload in bits, as described above. Typically, the size of the group of pixel circuits 1 and thus the length of the vector is a power of 2. The size of the group of pixel circuits 1 and thus the length of the vector should not be too high, because if a large number of events occur simultaneously, it is similar to processing each row, while if the events are few, it provides a small bandwidth reduction. If the size of the group of pixel circuits 1 and thus the length of the vector is too low, the header will become a significant overhead. Preferably, the size of the group of pixel circuits 1 and thus the length of the vector is between 8 and 128, and preferably between 8 and 64.

[0137] The following uses Figure 11 the example of a sensor matrix 100 with 1280 x 720 pixels shown below to explain the generation of a compressed frame in the global synchronization operation mode during pixel array readout. The sensor matrix 100 has a group of 16 pixel circuits and a readout vector size of 16 pixels (which corresponds to 80 vectors per row). In this example, there are five active pixel circuits 1 with active events (i.e., with an active row request signal req_y):

[0138] - Four pixel circuits 1 with ON events (represented by plus signs): the first pixel circuit 101 at (1, 4), the second pixel circuit 102 at (1269, 8), the third pixel circuit 103 at (2, 717), and the fourth pixel circuit 104 at (8, 717);

[0139] - A pixel circuit 1 with an OFF event (represented by a minus sign): the fifth pixel circuit 105 at (1276, 8).

[0140] Since there is a correspondence between the groups of pixel circuits 1 and the vectors, the vectors are also used hereinafter to refer to the corresponding groups of pixel circuits 1. Some groups or vectors of pixel circuits 1 that divide rows are shown with a shaded pattern. During pixel array readout, the first row selected is row 5 (i.e., row number 4), which contains an ON event in the first vector 111 of the fifth row, corresponding to the first pixel circuit 101. All the other 1280 / 16*4 = 320 vectors before the first vector 111 in row 5 are empty (e.g., filled with 0 due to no event detected), which can be directly derived from the address signal y_address sent by an organizer such as the arbiter 28. Preferably, at least one first type data word in the frame packet is determined according to the address of the pixel circuit 1 with an active event, as shown in this example.

[0141] The following vectors in row 5 are empty. This can be detected from the addresses of the following pixel circuits 1 with active events. Alternatively, when the event data formatter 6 receives the states of each pixel circuit in a group of adjacent pixel circuits, the event data formatter 6 can perform a test to detect whether no event occurs in a group of adjacent pixel circuits 1. This can be easily detected by applying an OR connection between all column values of the vectors during row readout. For this purpose, the event data formatter 6 can include such an OR connection, typically composed of logic gates, to detect the emptiness of the vectors. Then the row request for row 5 is confirmed.

[0142] The next row to be selected is row 9 (i.e., row 8). In this row 9, the last (for example) vector 112 contains the second pixel circuit 102 with an ON event and the fifth pixel 102 with an OFF event. Therefore, the vector 112 for the ON event will contain '1' only at the 5th position for the ON event, and the vector 122 for the OFF event will contain '1' at the 13th position. The other vectors in row 9 are empty.

[0143] After the row request for row 9 is confirmed, row 718 (i.e., row 717) is selected, for which the first vector 113 contains two pixels with ON events. The first vector 113 for the ON event will contain '1' at the 3rd position and the 9th position. After reading row 718, the organizer recognizes that there are no more active sampling row requests and deactivates the arbiter indicator req_arb. Therefore, it can be determined that the following vectors are empty, and the corresponding number of empty vectors will be inserted at the end of the frame. A new readout process for the next frame can be started.

[0144] The data words generated for the example are:

[0145] 00 0000000101000000: 320 empty vectors

[0146] 01 0100000000000000: ON event vector

[0147] 00 0000000110001110: 398 empty vectors

[0148] 01 0000100000000000: ON event vector

[0149] 10 0000000000001000: OFF event vector

[0150] 00 1101110101000000: 56640 empty vectors

[0151] 01 0010000010000000: ON event vector

[0152] 00 0000000011101111: 239 empty vectors

[0153] In this example, using the compressed event packet data format, only 144 bits are required to encode the information for the entire frame, instead of 1280 * 720 * 2 bit = 1843200 bits, and the compression ratio is approximately 10000. For frames capturing higher scene activities, this factor will of course decrease, but even in the worst - case scenario, i.e., the highly unlikely case where each vector contains both ON events and OFF events, the overhead due to the data word header is only 12.5%.

[0154] This method allows the frame rate to be automatically maximized according to the read - out and transfer times of the pixel data. Alternatively, a fixed frame rate can also be used for sensor operation. In this case, the pixel array is read out periodically at defined time points.

[0155] Although the invention has been described with respect to certain preferred embodiments, it is obvious that the invention is in no way limited thereto, and the invention includes all technical equivalents of the described devices and their combinations. In particular, it is obvious to those skilled in the art that various changes and modifications can be made without departing from the scope of the invention as defined in the appended claims.

Claims

1. A method for operating an event-based image sensor, the image sensor comprising a plurality of pixel circuits (1) arranged in rows and columns to form a pixel array, each pixel circuit (1) being arranged at a location defined by an address, wherein, Each pixel circuit (1) is configured to detect a change in illumination in a photosensor signal derived from a photocurrent generated by light incident on a photosensitive element of the pixel circuit (1), and thereby activate a row request, wherein the pixel array includes a readout circuit coupled to the pixel circuit via a row line, the readout circuit being configured to receive the row request from the pixel circuit and interact with the pixel circuit to read and reset the pixel circuit. During a readout period, a plurality of pixel circuits (1) are read and reset row by row by the readout circuit, and: - The readout period is initiated by sensing a row request (req_y) on at least one row line, and in response to sensing the row request (req_y) on at least one row line and a sampling request synchronized with a clock signal, the row request (req_y) activated at the start of the readout period is sampled as an activated sampled row request (req_y_arb). - The activated sampled row request (req_y_arb) is selected by an organizer (28), and in each row having the selected sampled activated row request (ack_y_arb), each pixel circuit having an activated row request is read and reset, and then another row having a sampled activated row request is selected until all the sampled activated row requests (req_y_arb) have been selected. The rows are associated with a row order, and the organizer (28) selects the sampled activated row requests according to a sequence organized by a predetermined rule and based on the respective positions of the rows having the sampled activated row requests in the row order, wherein the sequence in which the activated sampled row requests are selected is independent of the order in which the row requests are activated.

2. The method according to claim 1, wherein The predetermined rule for organizing the sequence includes a descending or ascending order of the rows.

3. The method according to any one of claims 1 to 2, wherein During the readout period, since an empty arbiter signal indicates that the organizer is not empty, newly activated row requests are not sampled as activated sampled row requests, and when no row request is being processed, the organizer is empty.

4. The method according to any one of claims 1 to 3, wherein The readout circuit includes an input logic (22) coupled to the row line, the readout circuit being configured to sense a row request (req_y) on at least one row line and send a row request indicator (req_y_or) to a state machine to indicate that at least one row line has a row request, thereby initiating the readout period.

5. The method according to any one of claims 1 to 4, wherein The row request activated at the start of the readout is converted by the input logic (22) of the readout circuit into a derived row request, and then the derived row request is sampled as an activated sampled row request, wherein the conversion of the row request (req_y) to the derived row request (req_y_d) is conditional on selecting a row associated with the row request (rq_y) by a selection signal.

6. The method according to any one of claims 1 to 5, wherein The state machine (24) is configured to send a sampling request (clk_req) to an interface (26), the interface (26) being configured to sample the row request on the row line in response to the row request, the sampling request being synchronized with a clock signal.

7. The method according to the preceding claim, wherein, The sampling request (clk_req) is generated by the state machine (24) in response to the row request indicator and a read confirmation (send_ack_y) of the pixel circuit indicating that the column interface is ready to read a new row, and / or is generated periodically during the readout of the pixel circuit of the row.

8. The method according to any one of the preceding two claims, wherein, The sampling request is generated by the state machine in synchronization with a frame rate defined by an external signal.

Images