Image frames are generated using data from the event camera.

Abstract

A method for generating an image frame from event packets received from an event camera includes: forming a block buffer, the size of which is configured to accumulate event information for a subset of image blocks, the block buffer having an associated block table that determines a mapping between each block of an image frame and the image frame, wherein event information is accumulated in the block buffer for each block of the image frame. For each event packet: identifying an image block corresponding to a pixel position of the event packet; adding event information to the block buffer in response to the block buffer storing information about another event corresponding to the image block; and adding an image block to the block buffer in response to the block buffer not storing information about another event corresponding to the image block and in response to the block buffer being able to accumulate event information for at least one or more blocks.

Description

Technical Field

[0001] The present invention relates to a method and module for forming at least one image frame based on event camera information. Background Technology

[0002] An event camera is a camera with sensors where the pixels of the sensors respond independently to changes in measurements. For example, each pixel can store a reference brightness and continuously compare the reference brightness with the measured brightness. If the difference between the measured brightness and the reference brightness in a pixel exceeds a predetermined threshold, the pixel resets its reference brightness, and an event is said to have occurred. Events cause event packets to be generated and sent by the camera. The content of the event packet is defined by the type and / or configuration of the event camera. The content includes the address of the pixel that detected the event and a timestamp that typically corresponds to the time the event occurred. The content may include the polarity of the detected change (indicating increase or decrease), the current measurement value, or the detected change itself. In operation, the event camera outputs an asynchronous stream of event packets triggered by changes in the scene imaged by the event camera.

[0003] Posch, C., Serrano-Gotarredona, T., Linares-Barranco, B., & Delbruck, T., “Retinomorphic event-based vision sensors: bioinspired cameras with spiking output”, Proceedings of the IEEE, 102(10), 1470-1484, (2014), European Patent No. EP3440833, PCT Application WO2019 / 145516 and PCT Application WO2019 / 180033 from Prophesee.

[0004] When using an event camera, it is still desirable to reconstruct a texture (or spatial) image using a set of events accumulated over a given time period. For example, when imaging a face, the reconstructed texture image can be used to determine characteristics such as eye gaze or eye state (e.g., blinking or open eyes), as described in PCT application WO2019 / 145578 (reference number: FN-630-PCT). (It should also be understood that spectral image information can be generated regardless of where spatial image information is available.)

[0005] Two neural network (NN)-based event camera reconstruction methods are E2VID and Firenet, discussed in "Fast image reconstruction with an event camera" by Scheeerlinck, C., Rebecq, H., Gehrig, D., Barnes, N., Mahony, R., and Scaramuzza, D. (2020, IEEE Winter Conference on Applications of Computer Vision (pp. 156-163)).

[0006] EP3,279,769 discloses an event signal processing method for an event-based sensor, the method comprising: receiving an activation signal from a sensor array to indicate the sensing of an event; increasing the cumulative number of events in a sensing area of ​​the sensor array corresponding to the activation signal; decreasing the cumulative number of events in each sensing area based on parameters; and identifying sensing areas in the sensing area where the cumulative number of events exceeds a threshold as flashing areas. Summary of the Invention

[0007] This invention is defined by the independent claims. Other optional features are defined in the dependent claims.

[0008] In embodiments of the present invention, compared to known methods, the amount of memory required to process event packets from event cameras is reduced by using a block buffer whose size is set to store only information of a subset of the event camera's pixels.

[0009] Based on the nature of event cameras, it can be assumed that event packets are generated only by pixels corresponding to moving or changed objects. The remaining pixels in the event camera's sensor do not generate event packets and can be considered as being associated with non-moving portions of the imaging scene. This invention prevents memory from being unnecessarily allocated for imaging non-moving portions of the scene.

[0010] In some embodiments, a preliminary decoder is used to count the received event packets or extract timestamps from the received event packets. The number of received event packets or the timing of the received event packets determined by the preliminary decoder can then be used to trigger the writing of one or more image frames.

[0011] In some embodiments, additional filtering is performed on the block buffer to ensure that areas of event camera sensors that generate few events are removed from the block buffer. This allows the size of the block buffer to be minimized and helps avoid storing blocks that consist only of noise in the block buffer. Attached Figure Description

[0012] Embodiments of the invention will now be described by way of example with reference to the accompanying drawings, in which:

[0013] Figure 1 An example of an event camera installed in a vehicle to monitor the driver is shown.

[0014] Figure 2 An example of an image frame generated from an event camera pointing at a moving face is shown, which has been divided into sixteen blocks;

[0015] Figure 3 A module for processing event packets from an event camera is schematically illustrated according to an embodiment of the present invention;

[0016] Figure 4 The steps performed in processing an event packet from an event camera according to an embodiment of the present invention are shown; and

[0017] Figure 5 This is a flowchart corresponding to a method for processing event packets from an event camera according to an embodiment of the present invention. Detailed Implementation

[0018] Embodiments of the present invention will now be described with reference to the accompanying drawings.

[0019] See now Figure 1 The illustration depicts an exemplary application in which an embodiment of the invention can be used. A driver 32 is shown inside the passenger compartment of a vehicle 31. The driver 32's head is monitored by a driver monitoring system (DMS). The DMS includes a user-facing event camera 33 configured such that the user 32's head is within the field of view 34 of the user-facing event camera 33. The user is the driver 32 of the vehicle 31, and therefore the user-facing event camera 33 is a driver-facing event camera. However, in the DMS system, additionally or alternatively, the event camera may also capture images of one or more passengers within the field of view.

[0020] Embodiments of the invention are particularly advantageous when at least a portion of the scene imaged by the event camera remains unchanged. Therefore, an event camera 33 pointing invariably towards the interior of vehicle 31 is a very suitable application. However, the invention is not limited to vehicle applications and can be used in any situation where an event camera can be employed.

[0021] The sensitivity of the event camera used in embodiments of the invention need not be limited to any particular wavelength range, but will most generally be sensitive to near-infrared (NIR) and / or visible (RGB) light. The event camera will typically be in the form of a camera module, which includes a housing for a lens and a sensor, the lens being used to focus light onto the sensor. The camera module may also have electronics for powering the sensor and being able to communicate with it. The camera module may also include electronics for processing the acquired images. This processing may be low-level image signal processing, such as gain control, exposure control, color balancing, noise reduction, etc., and / or it may involve more powerful processing, for example, for computer vision.

[0022] Embodiments of the present invention can be implemented in a module located between an event camera sensor and a system bus (e.g., an Advanced Scalable Interface (AXI) bus) to write image frame information into system memory via the bus, but this requires minimal intermediate memory. However, the event sensor can provide event information (in parallel) directly to main memory or to other modules.

[0023] Figure 2 An image frame 11 of the type provided by such a module connected to an event camera 33, which is pointed at a scene containing a face 12, is shown according to an embodiment of the present invention. The image frame 11 consists of pixels such as pixels 13. Black pixels represent zero events, i.e., no changes were detected, while white pixels represent one or more detected events, i.e., detected changes.

[0024] The image is divided into blocks, each block corresponding to at least one pixel. Figure 2 The sixteen blocks T1-T16 shown are arranged in a regular rectangular checkerboard pattern. Image 11 is X pixels wide × Y pixels high, therefore there are N = X * Y pixels in the image. The size of the face 12 imaged within the scene is such that many image blocks (T1, T5, T9) are black and do not involve any movement at any given time. However, the face 12 at the center of the imaged scene has moved, thereby generating event packets, especially at the moving edge features of the face 12.

[0025] It can export event information generated across one or more event periods. Figure 2 The generated image frames have an event period, which is the smallest amount of time that the event camera can resolve, with each event period assigned a unique timestamp. In the case of deriving image frames from event information generated across more than one event period, the information at each pixel within the image frame can include a net polarity for one or more events, which may be recorded for pixels over multiple event periods.

[0026] Since event packets can be generated from any pixel that detects a change, in known methods, the total memory allocated to a block is set to record changes from each pixel. Therefore, for each pixel of the event camera, within the memory allocated to the block, there will be a segment of memory where event information can be accumulated. The aim of embodiments of the present invention is to reduce this memory requirement.

[0027] Figure 3 A schematic diagram of an exemplary module 60 configured to generate image frame information from event camera information is shown according to an embodiment of the present invention. In this embodiment, the generated image frame information may include channels of tensor type employed in Scheerlink et al. and U.S. Patent Applications Nos. 16 / 904,122 and 16 / 941,799 cited above. Each tensor includes one or more image frames, wherein each image frame accumulates information from one or more event periods.

[0028] from Figure 3 Starting from the left, the preliminary decoder 62 receives event packet stream 61 from the event camera. The received event packets include pixel identifiers and timestamps for event period and event information. The preliminary decoder can count the received event packets starting from a given timestamp. The number of event packets and / or timestamps extracted by the preliminary decoder 62 can provide a stopping condition that determines when the tensor's image frame information should be written to main memory (DDR) via the bus, as explained below.

[0029] Note that image information can be written to main memory in blocks via the bus, thus reducing the bus overhead of writing event information from the event camera to main memory on a block-by-block basis.

[0030] When a downstream module or application (not shown) uses multi-frame tensors, each image frame can be written to the FIFO portion of main memory, where the latest image frame replaces the oldest frame. Alternatively, if event information for multiple image frames is to be accumulated within module 60, the storage required for each frame will be relative to... Figure 3 The single frame example shown is effectively copied.

[0031] In any case, the stopping condition can be common to all image frames for which event information has been accumulated in module 60, common to a subset of image frames, or specific to each image frame.

[0032] In this embodiment, event packets processed by the preliminary decoder 62 are fed into an event buffer 63 to absorb fluctuations in the event packet frequency. In some embodiments, the event buffer 63 is a FIFO buffer. If an excessive number of event packets and / or an excessively high rate of event packets are detected, an event buffer overflow can be recorded. Event packets in the buffer, or events subsequently received within a predetermined time, can be discarded to allow the system to continue operating despite the buffer overflow.

[0033] Other techniques can be used to avoid this overflow or to avoid the use of buffer 63, such as providing a parallel path for receiving and accumulating event packet information.

[0034] The event packets in event buffer 63 are then processed by event decoder 64. Event decoder 64 extracts information from each processed event packet. For each event packet, the extracted information includes the x column and y row of the event from the pixel identifier, the polarity of the event, and the timestamp.

[0035] The x and y values ​​extracted by the event decoder are processed in two ways. First, the x and y values ​​are processed by the intra-block positioning module 65a to find the corresponding intra-block location. This can be achieved without requiring additional information. For example, if an image 100 pixels wide × 100 pixels high is divided into a rectangular grid of 10-pixel squares, then calculating "x modulo 10" and "y modulo 10" will determine the intra-block location of any given event.

[0036] Secondly, the x and y values ​​are processed by the block identification module 65b to identify the image ( Figure 2 The blocks (T1…T16) in the block identification module 65b include the locations from which events are generated. The output of the block identification module 65b is the identified block. The identified block is then processed in the block table management module 66, which communicates with the block table 67 stored in the local memory of module 60.

[0037] Block table 67 includes one or more records, each including a block number 67a and a memory address 67b in block buffer 68. In each record, the memory address 67b specifies the offset within block buffer 68 relative to the memory allocated to the given block number 67a. Since the offset specifies the starting position of the block in memory, all offsets are separated by at least the memory required to store one block.

[0038] The block buffer 68 includes memory allocated to a predetermined number of blocks 68a. In at least one operating mode, the predetermined number of blocks 68a is less than the total number of blocks required to form a complete image frame.

[0039] Each image frame will have its own block buffer 68 and block table 67, where a timestamp from the event decoder 64 is used to determine the image frame and associated block buffer for the received event packet.

[0040] It should be noted that in this example, which shows only one block buffer 68 and block table 67, the buffer information needs to be written to main memory before it can be refreshed with event information from the next image frame. However, when there is more than one block buffer, a first subset of the buffer can accumulate event packet information while different subsets of the block buffer are written to main memory, and the operations performed on the subsets are then switched.

[0041] The block table management module 66 queries the records of the block table 67 to see if any record 67a has a block number that matches the block identified by the incoming event. If a record matching the identified block is found, the memory address of the matching record is returned to the block management module 66.

[0042] If no record matches the identified block and if there is space in the block buffer 68, the block table management module 66 adds the new record for the identified block to the block table 67 and allocates memory in the block buffer 68 for the identified block. The starting address of the memory allocated in the block buffer 68 is also recorded in the block table 67 as the memory address of the new record.

[0043] The block formatter 70 combines the block offset from the matching or new record in the block table 67 with the x and y pixel offsets within the block. The block offset determines the start of the allocated memory for the block in the block buffer 68, and the offset within the block determines where in the allocated memory the event information should be directed.

[0044] Therefore, the block formatter 70 determines the memory address in the block buffer for the processed event packet. Note that if the block buffer 68 is to accumulate information for more than one event cycle in a given image frame, event information from more than one event may have already accumulated at a given location in the block buffer 68. In this case, the block formatter 70 needs to first read the value at that address and then add that value to the current value for the most recent event before writing the result back to its location in the block buffer 68. (This accumulation can, of course, be done in a single operation using dedicated hardware.)

[0045] In such an embodiment, it may also be useful to maintain a count of the number of events for each pixel using the accumulated value within block buffer 68.

[0046] The counts from the preliminary decoder 62 are monitored, and an image frame is considered complete once the necessary number of event packets for the image frame have been received, or once the timestamps extracted from the received event packets reveal that the time has exceeded the range of the image frame. (If it can be assumed that events 61 are provided sequentially from the event camera, then a first timestamp with a number indicating the event period following the timestamp of the previous event period indicates that events from the previous event period will no longer be received. If this is the last event period—the events of which will be accumulated in the image frame—then the frame can be written to main memory.)

[0047] For each complete image frame, the mapping write module 69 can be used to process the data in the block buffer 68. The mapping write module 69 receives block data from the block buffer 68 and uses information from the block table 67 to infer the mapping between the block data and the image frame. Since the block buffer 68 does not contain all the data required to generate a complete image, the mapping write module 69 fills in the remaining portion of the image. The filled values ​​can be blank, null, or, particularly, values ​​extrapolated or interpolated from the data in the block buffer, especially in areas near the edges of blocks already stored in the block buffer. The resulting image frame information is transmitted to main memory DDR via the AXI bus for recording or use by other downstream modules or applications.

[0048] Note that if the block buffer also includes a count of the number of events for each pixel, this information can be written into the image frame (possibly as an additional channel). Such information can be used downstream, for example, as described in PCT application No. PCT / EP2021 / 058973 (reference number: FN-675-PCT) filed April 6, 2021, to improve image reconstruction based on event camera information.

[0049] As mentioned above, during normal operation, event packets related to blocks in block buffer 68 that have not been allocated memory can be received, resulting in an increase in the number of blocks whose information is stored in block buffer 68. However, this is not possible if the block buffer for an image frame is full, for example, if all memory in the buffer has been allocated to other blocks. In this case, there are several possible actions that the system may take depending on the configuration, including either ignoring event information for all blocks in the buffer for the image frame that have not yet been stored in the block buffer, or attempting to flush blocks from the block buffer that indicate that they may not have accumulated enough information for a given image frame.

[0050] In some embodiments, additional constraints may exist regarding whether to add blocks to the block buffer. For example, a downstream application or module may signal across the system bus to module controller 72 that one or more regions of the image are irrelevant. Thus, as explained in U.S. Patent Applications Nos. 16 / 904,122 and 16 / 941,799 cited above, one application of the multi-frame tensors provided by module 60 to memory is for a downstream neural network to identify objects of interest within the camera's field of view—in this example, this could include the coordinates of a box used to define face 12. These coordinates can be fed back to controller 72 by the downstream neural network so that if an event packet relates to an identified block outside that box, the block is marked as one of the irrelevant blocks. Then, even if space exists in block buffer 68, table management module 66 will not allocate the identified block, but will simply discard the event packet. In this way, the system can further reduce the required memory.

[0051] Note that, in addition to having a lower temporal resolution than possible by accumulating event information for more than one event period in the image frame, the block buffer 68 can also store event information at a lower spatial resolution than the spatial resolution of the image frame information used to write to main memory DDR. Therefore, using, for example, 2×2 downsampling, event information from four pixels of the event camera can be written to a common location in the block buffer. Then, when the image frame information is written to main memory DDR, this information can be upsampled as needed by the mapping write module 69.

[0052] In the exemplary application described above, module 60 can generate image frame information, wherein each image frame accumulates event information for a fixed number of one or more event periods.

[0053] Module 60 may continue to do so until a downstream module or application signals to controller 72 that, in addition to or as an alternative to the image frame information it has already generated, it should provide an image frame in which event information is accumulated, possibly only for a limited region of interest within the image, until the preliminary decoder 62 has counted a given number of events. Once that count has been reached, mapping write module 69 may write the image frame to main memory 69. The texture image may then be reconstructed from the event information using the information accumulated in that image frame, possibly only from the limited region of interest, as described in U.S. Patent Applications Nos. 16 / 904,122 and 16 / 941,799 cited above, and techniques similar to those described by Scheelink et al.; or indeed for any other application including, for example, blink detection, where the region of interest may only include the eye region of face 12.

[0054] At any stage, module 60 can then be requested to switch back to using only the first operating mode to generate frames, where each image frame accumulates event information for a fixed number of one or more event cycles.

[0055] In another possible operating mode, the module includes corresponding block buffers and block tables for multiple image frames, and specifically, the block formatter 70 can downsample the information being written to the block buffer 68, generating downsampled image frames that occur over a large number of event periods or according to a threshold number of events, particularly in the background portion of the image (e.g., in...). Figure 2 Events occurring outside the facial region 12 are used to accumulate event information. When such an image frame is written to main memory DDR, it can be used by downstream modules or applications to reconstruct a textured background image of the scene imaged by camera 33. In PCT application PCT / EP2020 / 073928 (reference number: FN-654-PCT), where camera 33 includes an actuator that can be triggered to cause a change in the light field incident on the surface of the event sensor, thereby generating a set of events based on pixels distributed on the surface of the event sensor, the technique disclosed in PCT application PCT / EP2020 / 073928 can also be used to enable the event sensor to generate the required number of events to produce an image frame with the required information, thereby generating such a textured image. In this operating mode, when using downsampling, block table 67 and associated buffer 80 can include information for all blocks of the image, but at lower resolutions, this may involve excessive memory.

[0056] Figure 4Shows the steps of a method that can be executed in module 60 for processing an event packet stream from an event camera according to an embodiment of the present invention. In a first step 40, a block buffer 68 is established for an image frame. Next, an event packet is received 41, and the image block of the event packet is identified 42. Then the block buffer is checked 43 to see if memory has been allocated in the block buffer 68 for the identified block. If so, the information from the event packet is added 45 to the block buffer 68. In step 44, if memory has not been allocated to the block buffer for the identified block, then an action including any of the following is performed: allocate 44a memory in the block buffer 68 for the identified block and add this offset to the block table; if the block buffer / table is full, flush (not shown) the least used blocks from the block buffer / table to allow memory to be allocated in the block buffer 68 and the block table 67 for the identified block; or if the block buffer / table is full, ignore the event packet information (and possibly report an overflow 44b). Once the stop condition for the image frame has been met, the block buffer 68 is written out 46 to form an image frame.

[0057] Figure 5 A flowchart is provided that illustrates the operation of the method. As Figure 2 Demonstrated, any image of an event camera can be divided into N' blocks. Each block corresponds to a portion of the image and thus corresponds to one or more pixels, so N'<N. In the present invention, a block buffer 68 is formed 40, which includes memory only for a subset of the N' blocks. The block buffer 68 is associated with a block table 67 that includes records, such that for each block stored in the block buffer 68, there is a record in the block table 67. Each record relates to the corresponding block and includes the number of the corresponding block and the memory address that has been allocated for the corresponding block. The memory address can be an offset from a known memory address or an absolute memory address. The size of the block buffer can be fixed or adjustable in use.

[0058] Receiving 41 an event packet means receiving an event packet from an event camera, which is a data packet of a known predetermined format. The predetermined format varies according to the specific configuration of the event camera. For some embodiments of the present invention, each event packet is an indication of the pixels where an event has been detected. Since the position of the pixels is known, this indication can be used to determine the position of each detected event. In some embodiments, the event packet may also include a timestamp corresponding to the time when the event was detected and / or event information, and the event information may include at least one of the following: the polarity of the event, a measure of brightness, and a measure of the brightness difference.

[0059] When the received event packet determines the location, that location can be used to identify the block corresponding to that location (42). As an example, consider... Figure 2 The image contains a first event packet indicating the first pixel (first row, first column) and a second event packet indicating the last pixel (last row, last column). The first event packet corresponds to the first block T1, and the second event packet corresponds to the sixteenth block T16.

[0060] Once the block corresponding to the event packet has been identified, block buffer 68 needs to be checked to see if memory in block buffer 68 has been allocated to the identified block. This check is performed by searching the records in block table 67 to see if any record matches, i.e., includes the identified block.

[0061] If a record matching the identified block is found in block table 67, the event information from the event packet is added to block buffer 68. To add the event, the memory address of the identified block is determined from the matching record. The event's location can then be analyzed to determine its position within the identified block, i.e., its position within the block. The memory address of the identified block is then combined with its position within the block to determine the address in block buffer 68 where the information related to the event packet should be recorded. The determined address in the block buffer is then read (if the image frame accumulates information for more than one event period) and the event information is added to the existing value or incremented / decremented according to the polarity of the event information.

[0062] If no record is found in the block table 67 containing the identified block, but there is remaining space in the block buffer 68 (i.e., not all block buffers 68 have been allocated), then the block buffers can be adjusted to allocate memory to the identified block. Allocating memory for the identified block involves increasing the memory allocated to the block buffer 68 for an image frame to overwrite another block, and adjusting the block table 67 by adding a new record. The new record includes the identified block and the memory address defining the newly allocated memory in the block buffer.

[0063] If no record is found in the block table containing the identified block, and there is no remaining space in the block buffer, a block buffer overflow condition occurs. Under a block buffer overflow condition, at least one action may be performed depending on the system configuration. In one configuration, the system marks the block buffer as overflowed. Marking the block buffer as overflowed avoids misleading other systems that might access downstream data. The system can then terminate processing. Alternatively, the system may continue by discarding further received event packets until the block buffer is empty, or by immediately emptying the block buffer. Emptying the block buffer may include forming one or more images from the currently recorded information or discarding the information in the block buffer without forming any images. Discarding information during an overflow condition ensures that the system is always operational, even if some information is lost.

[0064] In some configurations, the thresholding process processes blocks in the block buffer to remove one or more blocks that contain less useful information than other blocks, thus freeing up space in the block buffer. The thresholding process aggregates information from each block in the block buffer. For each block, if the aggregated value is less than a threshold, the block is removed from the block buffer, and the record of the removed block is removed from block table 67. In this way, one or more blocks with few events can be removed. This thresholding process helps avoid wasting memory on blocks included in the block buffer solely due to sparse noise-generating events.

[0065] Each time a block buffer condition overflow occurs, the thresholding process can be repeated with a higher threshold. In some embodiments, the higher threshold is the previous threshold multiplied by α, where α is the superunit number. By adjusting α, the increase in noise can be suppressed.

[0066] Event information from event packets accumulates in the block buffer until at least one stopping condition is met. One possible stopping condition is the processing of a set number of event packets for a given number of image frames. Another possible stopping condition is the detection of an event packet whose timestamp falls outside the range of timestamps accumulated for the image frames.

[0067] Once at least one stopping condition has been processed, block buffer 46 will be written out and the block buffer and associated block table will be cleared. In other words, once all the necessary information has been accumulated in the block buffer, one or more image frames can be written out using the accumulated event information in the block buffer and the mapping in the block table.

[0068] In some of the variations described above, a simple filtering mechanism can be used that ignores blocks by not allocating block buffer memory to blocks affected by a small number of events. As an example of a filtering mechanism, adding new blocks to the block buffer can be prevented until a set number of event packets with a corresponding position to the new block have been recorded.

Claims

1. A method for generating image frames from a series of event packets (61) received from an event camera (33), the method being performed in a module (60), the module (60) comprising: An interface configured to receive event packets (61) from the event camera and an interface (69) for writing image frame information to memory (DDR) across a system bus, each event packet including event information indicating the x and y pixel positions of the event camera, an indication of a detected change in the intensity of incident light at the x and y pixel positions, and an event period in which the event occurs, the image frame being defined by a block array, each block corresponding to multiple pixel positions, the method comprising: A block buffer (68) is formed in the memory, the size of which is set to accumulate event information for a subset (68a) of image blocks, the block buffer having an associated block table (67) that determines the mapping between each block of the image frame and the image frame, wherein event information for each block of the image frame is accumulated in the block buffer; For each of the series of event packets received from the event camera: Identify the image block corresponding to the pixel position of the event packet; In response to the block buffer storing information about at least one other event corresponding to the image block, the event information is added to the block buffer; In response to the block buffer not storing information about at least one other event corresponding to the image block and in response to the block buffer being capable of accumulating event information for at least one or more blocks, the image block is added to the subset of blocks, and event information for the subset of blocks is accumulated in the block buffer; and In response to a stop condition, the image frame is formed by mapping the information in the block buffer to the image frame using the block table.

2. The method according to claim 1, further comprising: In response to the block buffer not storing information about at least one other event corresponding to the image block and the block buffer being unable to accumulate event information for at least one or more blocks, an indication is made that the block buffer overflows.

3. The method according to claim 2, comprising: In response to a buffer overflow indication, the information in the event packet is discarded.

4. The method according to claim 2, comprising: In response to a buffer overflow indication, information for at least one block is refreshed from the block buffer.

5. The method of claim 4, wherein the refresh comprises: For each block in the block buffer where event information has accumulated: The information for the block is aggregated into an aggregated value; as well as If the aggregate value is less than the threshold, then the block is refreshed from the block buffer.

6. The method according to claim 1, further comprising: The number of events having an event period corresponding to one or more event periods is counted, wherein for the one or more event periods, event information is accumulated for image frames. The stopping condition includes the count exceeding a threshold.

7. The method of claim 1, wherein the stopping condition includes: Receive event packets for an event period that occurs after an event period range, wherein event information is accumulated for image frames for the event period range.

8. The method according to claim 1, comprising: For each of the multiple image frames for which image information will be accumulated, a block buffer and an associated block table are provided, and the image frame information is written to the system memory across the system bus.

9. The method according to claim 8 further comprises: While writing image frame information for another image frame, event information for one image frame is accumulated.

10. The method of claim 8, wherein the stopping condition for one image frame is different from the stopping condition for another frame.

11. The method according to claim 1, comprising: Discard event information corresponding to image blocks outside the region of interest of the image frame.

12. The method according to claim 1, comprising: Event information is accumulated in the block buffer at a spatial resolution less than or equal to the spatial resolution of the formed image frame.

13. The method of claim 1, wherein adding the event information to the block buffer comprises: Read the current accumulated value for the pixel position from the block buffer; The indication of the detected change in the intensity of the incident light at the pixel location is added to the current accumulated value; And write the sum back to the location for the pixel in the block buffer.

14. A module comprising the interface configured to receive the event from the event camera and the interface for writing image frame information to a memory across a system bus, the module being configured to perform the method of claim 1.

15. A vehicle comprising the module of claim 14, the module being operatively coupled across the system bus to an event camera and the memory.

Images