Event sensor and method for generating a signal stream containing event data

Abstract

The present invention relates to an event sensor comprising a pixel array (10) configured to generate a signal stream containing event data in response to light incident on the pixel array (10). The event sensor comprises: for each pixel of the pixel array (10), a photovoltaic converter (1) and an electronic converter (2) connected to the photovoltaic converter (1), the photovoltaic converter (1) and the electronic converter (2) configured to generate and store a digital current pixel value dependent on the intensity of light incident on the photovoltaic converter (1); for each pixel of the pixel array (10), a corresponding multi-bit digital storage (31) configured to store a previous pixel value; and a readout processor (4) connected to the electronic converter (2) and the multi-bit digital storage (31) configured to generate a pixel event value of the event data based on a pixel subtraction result of subtracting the previous pixel value from the current pixel value. The present invention also relates to a method for generating a signal stream containing event data in response to light incident on the pixel array (10).

Description

Technical Field

[0001] The present invention relates to an event sensor and a method for generating a signal stream including event data.

Background Art

[0002] Conventional image sensors record a scene by capturing a series of consecutive frames in the form of digital intensity values measured for each pixel per frame and then transferring all these digital intensity values to a receiving computer for processing. The main drawbacks of this approach include limited temporal resolution, high latency, and high data redundancy.

[0003] In some machine vision tasks, such as tracking, localizing, and detecting moving objects, it is desirable for machine vision algorithms to receive image sensor inputs with high temporal resolution and minimal latency. For such machine vision tasks, it is also not desirable to receive and process large amounts of redundant or irrelevant information. Thus, such machine vision tasks require smart image sensors with high temporal resolution, low latency, and low data redundancy. This requirement is not met by conventional image sensors.

[0004] An alternative type of image sensor has been proposed in U.S. Patent No. 7,728,269B2, which encodes the temporal visual contrast of the scene captured by its photosensors. By encoding the temporal contrast, the temporal redundancy of the output data of the image sensor is almost removed, thereby creating activity-driven sparse data in the form of ON / OFF events. This means that each event consists of a coded ON / OFF and pixel coordinates. Such an event-based temporal contrast sensor concept provides a unique combination of advantages not offered by conventional image sensors: high dynamic range, high temporal resolution, low latency, and low data redundancy. Therefore, the concept of event-based sensors has been adopted in a number of alternative or improved designs and is also the basis of the present invention.

[0005] The design proposed in U.S. Patent No. 7,728,269B2 utilizes the subthreshold MOSFET behavior for logarithmic current-voltage conversion to achieve a high dynamic range. Since the temporal contrast is measured instead of the absolute intensity, this design is not affected by the influence of pixel-to-pixel offset mismatches in logarithmic current-voltage conversion. However, this design is still troubled by pixel-to-pixel gain mismatches in logarithmic current-voltage conversion as well as in the change detection stage. In addition, this design uses a storage capacitor to store past light intensity-dependent signals, which is prone to various leakages that cause inaccurate output events.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] The object of the present invention is to propose an event sensor and an accompanying imaging method with improved output accuracy. In addition, the proposed design must have the potential to enable compensation for pixel-to-pixel offset mismatches and / or pixel-to-pixel gain mismatches.

Means for Solving the Problems

[0008] This object is achieved by the present invention by providing an event sensor having the features of claim 1 and a method having the features of claim 15. Further advantageous embodiments of the present invention are the subject matter of the dependent claims.

[0009] According to the present invention, an event sensor comprises a pixel array and is configured to produce a signal stream containing event data in response to light incident on the pixel array. In particular, the pixel array may comprise a rectangular array of W×H pixels, where W is referred to as the number of widths indicating the number of pixels along a first direction, and H is referred to as the number of heights indicating the number of pixels along a second direction perpendicular to the first direction. The pixel array may also be said to have H pixels in W columns and W pixels in H rows.

[0010] Each pixel of the pixel array comprises a photoelectric converter and an electronic converter connected to the photoelectric converter. The photoelectric converter and the electronic converter are configured to produce and store a digital current pixel value that depends on the intensity of the light incident on the photoelectric converter.

[0011] In a relaxed context, the term "light" typically refers to the visible range of electromagnetic (EM) wavelengths from 400 - 700 nm. However, EM radiation within the ultraviolet (UV) and near-infrared wavelengths can still cause a certain degree of the photovoltaic effect in semiconductors, particularly silicon. The detectable wavelength range of a photovoltaic converter can be further extended when newer types of materials other than silicon are used as photovoltaic materials. Therefore, the term "EM radiation" covers more applicable cases of inventions than the term "light" in a relaxed context. However, in a scientific context, and thus also in this context, "light" can refer to the wavelength range between and including UV and infrared in advantageous embodiments, while also referring to EM radiation of any wavelength. Therefore, in the following, the expressions "light" and "EM radiation" may be used interchangeably with each other.

[0012] The event sensor further comprises corresponding multi-bit digital storage configured to store the previous pixel value for each pixel of the pixel array. The expressions "current pixel value" or "present pixel value" refer to the latest pixel value obtained from a pair of a photovoltaic converter and an electronic converter, while the expressions "previous pixel value" or "past pixel value" refer to the pixel value obtained temporally prior to the current or present pixel value, particularly during a previous readout process.

[0013] Furthermore, the event sensor comprises a readout processor configured to generate a pixel event value for a corresponding pixel. And the generated pixel event value is part of the event data that constitutes a signal stream created by the event sensor. For this purpose, the readout processor is connected to the electronic converter and its corresponding multi-bit digital storage. The readout processor generates a pixel event value based on a pixel subtraction result obtained by subtracting the previous pixel value from the current pixel value. In particular, the pixel event value is derived from comparing the pixel subtraction result with a threshold value. This means that when the pixel subtraction result exceeds or surpasses the threshold value, it is determined that the pixel event value is either a specific increment value, or simply a binary on / off or "1" / "0" bit value. Alternatively, and more advantageously, the pixel event value is determined to be the pixel subtraction result, or a value dependent on the pixel subtraction result, when this pixel subtraction result exceeds the threshold value. As further discussed below, the pixel subtraction result can be scaled by a gain correction factor to further correct for inter-pixel gain mismatch.

[0014] The readout processor generating a pixel event value based on the pixel subtraction result particularly means that the pixel subtraction result can determine whether a pixel event value is generated in the first place. Thus, for example, when the pixel subtraction result does not exceed a specific threshold value, there are cases where the readout processor does not generate a pixel event value.

[0015] According to an advantageous embodiment, the readout processor is configured to overwrite the stored previous pixel value with the current pixel value each time a pixel event value is generated. This particularly means that the stored previous pixel value is not overwritten unless the readout processor generates a new pixel event value. The advantages of this approach are discussed in the following paragraphs:

[0016] Conventional prior art motion detection techniques compare two consecutive frames of an image captured from two adjacent points in time. Thus, only relatively fast motions, or relatively fast changes in the intensity of light impinging on the pixels, can produce significant differences between consecutive frames of an image that are distinguishable from intrinsic temporal noise. Conventional motion detection techniques can be used at a reduced frame rate to enable the detection of relatively slow motions, but doing so impedes the detection of relatively fast motions.

[0017] In contrast, in the case of threshold-dependent pixel event generation, by overwriting the previous pixel value with the current pixel value only when a pixel event is generated, the readout processor can ensure that relatively slow changes in the intensity of light impinging on the pixel are ultimately detected. In other words, the multi-bit digital storage can maintain the same previous pixel value as long as the change in the intensity of light impinging on the pixel does not exceed an amount defined by the change detection threshold. Thus, the readout processor can compare the current pixel value from the current point in time and the previous pixel value from a past point in time that is relatively distant from the current point in time so that a relatively slow change in the intensity of light impinging on the pixel is detected when the current pixel value exceeds the change detection threshold and is different from the previous pixel value.

[0018] In addition, advantageously, the readout processor decides individually for each pixel whether to overwrite the previous pixel value with the current pixel value. Thus, the multi-bit digital storage of different pixels can store previous pixel values from different past points in time, enabling the readout processor to detect a wide range of change speeds in the intensity of light impinging on these pixels, for example, from relatively slow to relatively fast changes.

[0019] The optoelectronic converter converts incident light into a corresponding analog electrical signal, and the electronic converter further converts these analog signals into digital signals with the help of an analog-to-digital converter. The optoelectronic converter is a converter that performs photon-electron conversion. The optoelectronic converter can be a photodiode, particularly a partially or fully pinned photodiode (PPD). It can be configured to generate a current called a photocurrent that depends, particularly linearly or proportionally, on the intensity of the light impinging on the optoelectronic converter. In contrast, the electronic converter is so named because it simply performs the conversion between different electronic signals or electronic-electron conversion.

[0020] The electronic converter can include an electronic signal converter that can be a current-current converter, a current-voltage converter, a voltage-current converter, or a voltage-voltage converter. The output of the electronic signal converter can depend linearly, logarithmically, or according to some other function on its input signal. The function can be a compression function such as a logarithmic function to enable compressing a much wider range of inputs (e.g., corresponding to a six-digit light intensity variation) into a relatively small range of electronic signals (e.g., a voltage swing of 500 mV). As an example, the optoelectronic converter performs photon-electron conversion to generate a photocurrent. This photocurrent is converted into an analog voltage signal in a logarithmic current-voltage converter that is part of the electronic converter. Then, this analog voltage signal that depends logarithmically on the incident light intensity is converted into a digital signal or value using an analog-to-digital converter that is also part of the electronic converter, as described below.

[0021] Therefore, advantageously, the electronic converter includes an analog-to-digital converter and is configured to create the current pixel value that depends logarithmically on the intensity of the light incident on the optoelectronic converter.

[0022] One advantageous concept for an analog-to-digital converter (ADC) is the so-called single-slope ADC concept. For this purpose, the ADC comprises a converter memory which is a multi-bit memory and is configured to continuously compare an analog input signal with a reference signal, receive a digital count and a reference signal synchronized with the digital count, and store the value of the digital count in the converter memory when the reference signal becomes equal to or crosses the analog input signal. In particular, the reference signal can be a slope with a constant gradient, while the digital count counts linearly or logarithmically synchronized with the slope of the reference signal. In the case of a linear count, the ADC creates a digital value linearly dependent on the analog input signal, while in the case of a logarithmic count, the generated digital value is logarithmically dependent on the analog input signal. Other combinations of the digital count and the reference signal can be used to obtain an output value linearly or logarithmically dependent on the analog input, such as an exponentially varying reference signal and a digitally counting linearly, to obtain logarithmic analog-to-digital conversion. The digital count can in particular be in Gray code format.

[0023] For this purpose, the analog input signal provided to the ADC, in particular by an electronic signal converter, depends on the intensity of the light incident on the photoelectric converter. Thus, the ADC ensures that the converter memory is filled with multi-bit digital values that depend, in particular linearly or logarithmically, on the intensity of the light impinging on the corresponding pixel or photoelectric converter.

[0024] As further explained above, the electronic converter may comprise an electronic signal converter, such as a current-voltage converter, that converts an electronic analog signal (e.g., a current) generated by a photovoltaic converter into a different electronic analog signal (e.g., a voltage). Advantageously, the current-voltage converter is a logarithmic converter and the analog-digital converter is a linear converter, or the current-voltage converter is a linear converter and the analog-digital converter is a logarithmic converter. As described above, these two possible embodiments ensure that the digital current pixel value produced and stored, particularly stored in the converter memory, is logarithmically dependent on the intensity of the light incident on the pixel or the photovoltaic converter. The logarithmic dependence has the advantage of a higher dynamic range, but alternatively, it is possible to have a different, i.e., non-logarithmic relationship, e.g., a linear dependence, between the light intensity and the digital current pixel value produced and stored.

[0025] As described above, the pixel array consists of pixel columns of width number (W) and pixel rows of height number (H). Since each pixel also has a corresponding multi-bit digital storage for storing the previous pixel value and possibly other parameters for said pixel, the event sensor preferably also comprises a pixel parameter memory consisting of a W×H multi-bit digital storage. This is optional, but the multi-bit digital storage of the pixel parameter memory need not be arranged in an array like the pixel array. Furthermore, the readout processor comprises W processing blocks, each of said processing blocks being configured to process one of the W pixel columns. In other words, any processing said to be performed by the readout processor on the individual current pixel value of a pixel or the parameter value of the multi-bit digital storage or the like can actually be performed by the corresponding processing block.

[0026] When the readout processor consists of exactly W processing blocks, each processing block can be dedicated to processing the H pixels within one column of the pixel array. In this case, the readout processor consists of a row of W processing blocks.

[0027] Alternatively, the read processor may have a number of processing blocks that exceeds W, particularly a multiple of W times the number of processing blocks, for example, perhaps two times, three times, or more (M times) of W processing blocks arranged in M rows of W columns, where M is significantly smaller than H. In this alternative case, the H pixels in each column may be divided into M groups, and each group is processed by one of the M processing blocks dedicated to the column of pixels. In other words, every Mth processing block may be dedicated to processing a column of H pixels. Since all processing blocks can process simultaneously, the pixels in M rows can be processed simultaneously.

[0028] As yet another alternative, the read processor may have a number of processing blocks that is less than W, particularly a number obtained by dividing W by an integer, for example, a number obtained by dividing W by 2, a number obtained by dividing W by 3, or perhaps a number obtained by dividing W by D processing blocks arranged in a row of a column, where D is significantly smaller than W. In this alternative case, each processing block may be dedicated to processing D×H pixels within an array of D columns of pixels.

[0029] Advantageously, the multi-bit digital storage is further configured to store a change detection threshold. At this time, the read processor may be configured to generate and / or output the pixel event value on the condition that the current pixel value is greater than the change detection threshold and different from the previous pixel value. This means that the pixel event value is generated only when the intensity of the light impinging on the corresponding pixel changes by more than a specific amount defined by the change detection threshold. The multi-bit digital storage may be further configured to store a past event timestamp for the corresponding pixel. Here, the read processor may be configured to generate and / or output the pixel event value on the condition that the past event timestamp is older than a predetermined time interval. The past event timestamp is a value representing the previous time when the pixel event value was generated by the corresponding pixel. By referring to this parameter, it is possible to avoid two or more pixel event values being generated for the same pixel within a very short period of time. Advantageously, both the change detection threshold and the past event timestamp are considered when determining whether a new pixel event value should be generated. Instead of one change detection threshold, the multi-bit digital storage may be configured to store two separate change detection thresholds, namely a positive change detection threshold and a negative change detection threshold.

[0030] The predetermined time interval values described above may be stored in the corresponding multi-bit storage of each individual pixel. However, the predetermined time interval does not have to be pixel-specific, i.e., one such value can be applied to all pixels of the pixel array. Therefore, it may be advantageous to store the predetermined time interval value in a single separate memory accessed by all processing blocks of the read processor.

[0031] According to an advantageous embodiment, the read processor is configured such that generating the pixel event value is one of two or more processing options determined by a processing option parameter. In other words, the processing option parameter can determine whether the pixel event value is generated, or whether another processing option, such as generating logarithmic intensity or entering a calibration mode, is continued. The expressions "processing option" and "processing mode" can be used interchangeably with each other. Furthermore, generating the pixel event value can be regarded as processing the corresponding pixel in a temporal contrast mode.

[0032] In an advantageous embodiment, the read processor is configured to process the first pixel of the pixel array or all the pixels within the first pixel group of the pixel array according to a first processing option, and to process the second pixel of the pixel array or all the pixels within the second pixel group of the pixel array according to a second processing option. In other words, the processing option for the first subset of pixels is configured for one option, while the processing option for the second subset of pixels is configured for a different option. In this way, the pixel array is segmented and different processing modes are applied to different segments. Three or more segments can be generated, for example, by configuring the read processor to process the third pixel or all the pixels within the third pixel group according to a third processing option.

[0033] There may be one processing option parameter for all pixels, or there may be one processing option parameter specialized for a group of pixels. In an extreme case, each individual pixel may have its own dedicated processing option parameter. In the latter embodiment, the multi-bit digital storage corresponding to the pixel is configured to further store the processing option parameter for the corresponding pixel, and the read processor is configured to process the current pixel value according to the processing option parameter for the corresponding pixel. In other words, the read processor processes each pixel according to the processing option parameter value of the pixel. In this way, one or more first groups of pixels may be assigned a first processing option parameter value, while one or more second groups of pixels may be assigned a second processing option parameter value, and so on. As an example detailed further below, at a particular time, a first group of pixels may be designated for time contrast event detection, and a second group of pixels may be designated for logarithmic intensity frame generation. In other words, it is possible to divide the pixel array into two or more subsets of pixels, and the pixels of each subset are specialized for subset-specific processing options. For example, a first subset of pixels may be configured for pixel event value generation, while a second subset of pixels is configured for logarithmic intensity generation.

[0034] Advantageously, the read processor may be configured to generate an intensity value in the pixel such that one of the processing options depends on the intensity of the light incident on the photoelectric converter of the pixel. In particular, this processing option may be to generate a logarithmic intensity in the pixel that is a value that depends logarithmically on the intensity of the light incident on the photoelectric converter. This means that instead of perhaps generating a pixel event value for this particular pixel, the read processor generates an intensity or logarithmic intensity value. This option may also be referred to as processing the corresponding pixel in logarithmic intensity mode.

[0035] In one preferred embodiment, the readout processor is configured such that the processing options include a calibration mode, and the current pixel value is output unprocessed by the readout processor. During a calibration process that occurs when one pixel, one or more pixel groups, or all pixels of the pixel array are in the calibration mode, some of the other parameters within the multi-bit digital storage of the corresponding pixel(s) can be calculated and changed for optimal functioning of the event sensor. In particular, the change detection threshold, the gain correction factor, and / or the offset compensation value can be acquired or reacquired during such a calibration process. For this purpose, the readout processor sends the current pixel value to an external device without processing it. During the calibration process, the pixel array or a region thereof can be exposed to a known light source, and the unprocessed current pixel values can be utilized by an external device to calculate various parameters for each pixel. The external device can be a processor that is not part of the event sensor as described herein, but it can be a processing device that is part of the device in which the event sensor is incorporated, or even a processing device that is placed on the same chip and / or within the same housing as the event sensor.

[0036] Advantageously, the readout processor is configured such that the processing options include simultaneously generating a logarithmic intensity and a pixel event value at the pixel. As described above, the logarithmic intensity is a value that depends logarithmically on the intensity of the light incident on the photoelectric converter, while the pixel event value is based on a pixel subtraction result that subtracts the previous pixel value from the current pixel value. In other words, both the pixel event value and the logarithmic intensity at the same pixel are created by the readout processor. This processing option can be seen as a hybrid mode between a temporal contrast mode and a logarithmic intensity mode.

[0037] In a preferred embodiment, the multi-bit digital storage is further configured to store an offset compensation value, and the read processor is configured to consider the offset compensation value when generating the logarithmic intensity. In particular, the offset compensation value can be subtracted from the current pixel value to derive an offset-compensated current pixel value, and can be regarded as the basis of the intensity or logarithmic intensity of the pixel.

[0038] In a further advantageous embodiment, the multi-bit digital storage is further configured to store a gain correction factor. Here, the read processor is configured to consider this gain correction factor when generating the pixel event value and / or the logarithmic intensity. In particular, the pixel subtraction result or the offset-compensated current pixel value is multiplied by the gain correction factor to obtain the pixel event value or the logarithmic intensity.

[0039] When the multi-bit digital storage stores both the offset compensation value and the gain correction factor, Logarithmic intensity = (Current pixel value - Offset compensation value) * Gain correction factor, And / or Pixel event value = (Current pixel value - Past pixel value) * Gain correction factor That is. As described above, the pixel event value is calculated on the condition that the pixel subtraction result exceeds the change detection threshold. This calculation is not performed when the pixel subtraction result does not exceed the change detection threshold.

[0040] The pixel parameter memory, which is a collection of multi-bit digital storages each corresponding to a pixel of the pixel array, can thus be configured to store, for each pixel, processing option parameters, previous pixel values, change detection thresholds, gain correction factors, past event timestamps, and / or offset compensation values. Each of these parameters has been described above. The pixel parameter memory can be accessible through an external connection such that it can be directly accessed, i.e., read from or written to, by an external source. The external source is, in particular, an input source external to the event sensor. In particular, the parameters in the pixel parameter memory can be written or changed during a calibration process / procedure or calibration mode. Some pixel parameters, in particular the change detection threshold, can be adjusted or updated during the operation of the event sensor, in particular during the process of generating pixel event values and / or logarithmic intensities. Such an update procedure requires reading the old change detection threshold from each individual multi-bit storage of the pixel parameter memory, calculating the new change detection threshold, and writing the new change detection threshold back into the corresponding multi-bit storage.

[0041] According to an advantageous embodiment, the photoelectric converter and the electronic converter of each pixel together occupy a common physical region or volume within the pixel array. On the other hand, the readout processor and the pixel parameter memory can be placed in different regions or volumes remote from the pixel array. In this specification, the expression "volume" refers to the space or three-dimensional (3D) region within a semiconductor device.

[0042] When pixels are arranged within a rectangular pixel array of W columns and H rows, each column or multiple columns of pixels may share one or more common processing blocks of a readout processor. Thus, it may be referred to as a column-parallel readout processor configured to process one row, multiple rows, or a part of a row of pixels in the pixel array in parallel. The corresponding processing block(s) of the column(s) may be placed on the event sensor die at the end of the column(s), and may be connected to each pixel of the column(s) using a shared bus hereinafter referred to as the ADC bus. For example, when a pixel outputs a 12-bit current pixel value, the ADC bus may be 12 bits wide. In particular, the ADC bus may be connected to the output of an analog-to-digital converter and accesses the converter memory of each pixel. Preferably, when the ADC is implemented using digital counts, the digital counts may be supplied to the ADCs of each column via the same ADC bus. In this case, the ADC bus may be utilized for both the purpose of supplying digital counts and the purpose of reading out the current pixel value.

[0043] Similarly, the multi-bit digital storage of pixel parameter memories can also be arranged in an array of W-column and H-row multi-bit digital storage, or they can be configured in some other way to be accessed by a read processor one row, multiple rows, or a part of a row at a time. And here too, each read processing block can be connected to one column, multiple columns, or a part of a column of the multi-bit digital storage. Thus, one column, multiple columns, or a part of a column of the multi-bit digital storage can also share a single shared parameter bus, for example, a 75-bit parameter bus if the total size of all the parameters of one multi-bit digital storage is 75 bits. In practice, a parameter bus for transporting all the parameters of one multi-bit digital storage at a time, such as a 75-bit parameter bus, may be too wide. In this case, the parameter bus can have a smaller width, such as a 38-bit or narrower bus, which then requires two or more clocks to transfer all the parameters of one multi-bit digital storage.

[0044] As an alternative to the implementation of reading the current pixel value via the ADC bus, one or more columns of pixels can be read by corresponding processing block(s) via a shift register system. During the read, the converter memories within each column can be connected to each other to form a chain of shift registers. The current pixel value stored in the converter memory is then moved one pixel at a time towards the corresponding processing block(s) until all the current pixel values of the column are read by the said processing block(s). This is a destructive read process where the stored current pixel value in the converter memory is deleted or overwritten. A separate chain of shift registers may also be required to implement such a shift register system for reading and writing to the pixel parameter memory. Therefore, it is more efficient to read from and write to the pixel parameter memory using the parameter bus described above.

[0045] During the readout, the readout controller selects which row(s) or part of a row of the pixel array can access the shared ADC bus and which (corresponding) row(s) or part of a row of the pixel parameter memory can access the shared parameter bus.

[0046] It is possible and advantageous to fabricate the event sensor on a single die such that the pixel array, the pixel parameter memory, and the processing block are all physically located on the same die. However, in modern technology, it may be more practical to fabricate this event sensor using a two-die stacked structure where the pixel array is fabricated on the sensor die and the pixel parameter memory and the processing block are fabricated on the logic die, in which case the sensor die is stacked and bonded on top of the logic die. In future embodiments, it may be advantageous to utilize a three-die stacked structure: the first die carries an array of photovoltaic converters connected to or paired with an electronic signal converter, or an array of photovoltaic converters connected to or paired with a part of the electronic signal converter; the second die carries an array of ADCs, or an array of parts of the electronic signal converter connected to or paired with the ADC; and the third die holds the pixel parameter memory and the processing block.

[0047] According to a further aspect of the present invention, a method for creating a signal stream including event data in response to light incident on a pixel array is proposed. The method comprises the following steps: - creating and storing, by an electronic converter connected to the photovoltaic converter, a digital current pixel value depending on the intensity of light incident on the photovoltaic converter of each pixel of the pixel array; and - providing, for each pixel of the pixel array, the previous pixel value stored in the multi-bit digital storage corresponding to the pixel; - A step of generating a pixel event value of the event data based on a pixel subtraction result obtained by subtracting the previous pixel value from the current pixel value by a readout processor connected to the electronic converter and the multi-bit digital storage. including.

[0048] Any features and advantages described herein in relation to the event sensor may also be similarly applied to the method.

[0049] Some examples of embodiments of the present invention will be described in more detail in the following description with reference to the accompanying schematic diagrams.

Brief Description of Drawings

[0050]

Figure 1

Figure 2

Figure 3a

Figure 3b

Figure 3c

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8a

Figure 8b

Figure 8c

[0051] FIG. 1 shows a block diagram of a pixel according to one preferred embodiment. Incident light is detected here by a photoelectric converter 1 which is a photodiode 1. The photodiode generates a current I, a photocurrent, which depends on the intensity of the light incident on the active region of the photodiode 1. A current-voltage converter 21 converts the photocurrent I into a voltage V. This conversion can be a linear conversion where the voltage linearly depends on or is proportional to the photocurrent. Alternatively, it can be a logarithmic conversion where the voltage logarithmically depends on the photocurrent. Other conversion functions may be possible but are less practical.

[0052] The current-voltage converter 21 and an analog-digital converter (ADC) 23 together form an electronic converter 2, which, together with the photodiode 1, creates a digital current pixel value. The ADC 23 can be a linear converter, and its output digital value linearly depends on its input analog value. Alternatively, the ADC 23 can be a logarithmic converter, and its output digital value logarithmically depends on its input analog value. Advantageously, since the digital current pixel value generated by the combination of the current-voltage converter 21 and the ADC 23 logarithmically depends on the incident light or the photocurrent, the current-voltage converter 21 that performs a linear conversion can be combined with the ADC 23 that performs a logarithmic conversion. Conversely, the current-voltage converter 21 that performs a logarithmic conversion can be combined with the ADC 23 that performs a linear conversion to achieve a similar result.

[0053] The ADC23 shown here is a single-slope analog-to-digital converter, and its function will be described in more detail with reference to Figure 2. The central part of the single-slope analog-to-digital converter 23 is the converter memory 25. When the analog-to-digital conversion is completed, the output digital value, that is, the current pixel value, is stored in the converter memory 25. The digital count, which is a multi-bit digital value that is being counted up or down, is supplied to the converter memory 25 via the path indicated as the arrow labeled "W". Further, a reference signal, which is a ramp with a positive or negative slope, is supplied to the comparator 24. The other input of the comparator 24 is connected to the output of the current-to-voltage converter 21 in order to obtain the input analog value to be converted, which is voltage V here. When the ramping reference signal crosses voltage V, the comparator 24 generates a signal, which is used as a latch signal for the converter memory 25. Upon receiving the latch signal, the converter memory 25 stores the current value of the digital count supplied to it. The value stored in the converter memory 25 represents the current pixel value and can be read out later via the path indicated by the arrow labeled "R". Whether the converter memory 25 is in the write "W" mode or the read "R" mode is determined by the R / W selection signal supplied to the converter memory 25.

[0054] Whether the ADC23 implements a linear conversion or a logarithmic conversion can be determined by the relationship between the reference signal and the digital count. For example, an exponential ramping reference signal combined with a digitally count that increases or decreases linearly can enable a logarithmic conversion. However, advantageously, the reference signal is a linear ramping signal with a single slope, that is, a constant slope, but linear or logarithmic conversion can be achieved by a digitally count that increases or decreases linearly or logarithmically, respectively.

[0055] The reference signal is a global reference voltage generated peripherally. The digital count is a global bit pattern from a peripheral counter supplied through a shared digital bus. The reference signal voltage begins to sweep from a start voltage level to an end voltage level. At the same time, the peripheral counter begins to count. Depending on whether it is linear counting or logarithmic counting, the bit pattern value of the digital count increases or decreases linearly or logarithmically with time. When a latch signal is generated during the transition of two consecutive counts, the bit pattern of the digital count is preferably in Gray code format to avoid storing a false bit pattern value. The converter memory 25 can be based on either an SRAM circuit or a DRAM circuit.

[0056] The ADC23 resolution should be determined after considering the target contrast sensitivity threshold and the dynamic range of the event sensor, as well as the desired gain correction allowance and offset tolerance for coping with pixel-to-pixel mismatches. The purpose of the gain correction allowance is mainly to correct the pixel-to-pixel gain mismatch introduced by the current-voltage converter 21 later. The purpose of the offset tolerance is to accept the pixel-to-pixel offset mismatch in the output of the current-voltage converter 21, in other words, to ensure that the input analog value of the ADC23, which is the voltage V, falls within the input range supported by the ADC23. Summarizing these considerations, the resolution of the ADC23 is: [Number] can be determined as.

[0057] For example, to achieve a 15% contrast sensitivity threshold over a 120 dB dynamic range while enabling 4-bit gain correction and 1-bit offset tolerance, the ADC23 must have at least a 12-bit resolution.

[0058] Implementations of logarithmic current-voltage converters according to three preferred embodiments are shown in FIGS. 3a-3c. By utilizing the subthreshold behavior of either an N-type MOSFET (NMOS) or a P-type MOSFET (PMOS), high dynamic range and low current-voltage conversion delay are achieved. In the circuit shown in FIG. 3a, the inverting amplifier 26 is connected between the source and gate of the NMOS. In the circuit according to FIG. 3b, the inverting amplifier 26 is connected between the source and drain of the PMOS. In both circuits, the generated voltage V is supplied at the output of the inverting amplifier 26. In the circuit of FIG. 3c, the photodiode 1 used is a surface-mount photodiode. It is connected in series to two NMOSs. The buffer 27 is connected to its input to the source / drain connection between the two NMOSs and supplies the voltage V at its output. In FIG. 3a, a bias voltage is supplied to the inverting amplifier 26. In FIG. 3b, a bias voltage is supplied to the inverting amplifier 26 and another perhaps separate bias voltage is supplied to the gate of the PMOS. In FIG. 3c, a bias voltage is supplied to the buffer 27 and another perhaps separate bias voltage is supplied to the gate of the lower NMOS. The drain of the NMOS in FIG. 3a as well as the drains and gates of the upper NMOS in FIG. 3c are connected to the supply voltage(s) of the circuit.

[0059] FIG. 4 shows a block diagram of an event sensor according to a preferred embodiment. The central elements of the event sensor are a pixel array 10 with W columns × H rows of pixels, a pixel parameter memory 3 with W columns × H rows of multi-bit digital storage 31, and a readout processor 4 with W column parallel processing blocks 41. Thus, there is a one-to-one mapping between each pixel of the pixel array 10 and the corresponding multi-bit digital storage 31 of the pixel parameter memory 3. Additionally, a bias generator 51 provides pixel bias voltage(s) to the pixel array 10 to bias the current-voltage converter 21 shown in FIGS. 3a-3c. The ADC controller 52 generates a ramp / reference signal and linear or logarithmic digital counts in Gray code format and provides them to the pixel array 10 for the in-pixel ADC 23.

[0060] The read processor 4 reads the current pixel value from the pixel array 10 and reads / writes pixel parameters to / from the pixel parameter memory 3. The pixel parameter memory 3 is preferably SRAM-based. The read and write operations among the pixel array 10, the read processor 4, and the pixel parameter memory 3 are adjusted by the read controller 55. The pixel parameter memory 3 can be loaded from an external non-volatile storage 62 (e.g., flash memory) and configured by an external computer 61 via a pixel parameter configurator 56.

[0061] Each multi-bit digital storage 31 of the pixel parameter memory 3 stores the pixel parameters of its corresponding pixel. The pixel parameters include processing option parameters, previous pixel values, one or two (separate positive and negative) change detection thresholds (plural possible), gain correction coefficients, and additionally, past event timestamps, as well as offset compensation values or offset compensators.

[0062] The processing options are pixel-specific and determine how the processing block 41 operates on the corresponding pixels. The pixels have five processing options or modes determined by the processing option parameters: (1) temporal contrast detection, (2) temporal contrast detection and measurement, (3) logarithmic intensity measurement, (4) temporal contrast detection and logarithmic intensity measurement, and (5) calibration. In the temporal contrast detection mode and the temporal contrast detection and measurement mode, pixel event values can be generated at the corresponding pixels (depending on the temporal contrast). In the logarithmic intensity measurement mode, the logarithmic intensity at the corresponding pixels is generated. The temporal contrast detection and logarithmic intensity measurement mode is a hybrid mode in which both pixel event values and logarithmic intensities can be generated at the corresponding pixels (depending on the temporal contrast). In the calibration mode, the event sensor performs measurements under predetermined environmental parameters such as temperature and during exposure of the pixel array to radiation having predetermined parameters such as intensity and wavelength, and as a result, some or all of the pixel parameters of the pixels are calibrated or partially calibrated by obtaining or adjusting them.

[0063] The change detection threshold(s), gain correction factor, and offset compensation value are all pixel-specific to enable pixel-to-pixel gain and offset mismatch correction and compensation. Those values can be obtained through a one-time calibration procedure.

[0064] Each multi-bit digital storage 31 of the pixel parameter memory 3 stores all of the above-described pixel parameters, and each parameter can be assumed to be 12 bits, except for the 3-bit processing option parameter. Thus, each multi-bit digital storage 31 will have a total of 75 bits. In this case, the silicon area of the entire pixel parameter memory 3 supplying a 1-megapixel pixel array is less than 10 mm when using state-of-the-art 28 nm SRAM technology. 2 less.

[0065] The event sensor can switch between several different operating modes by configuring the processing options of the pixels. The event sensor has two basic operating modes: the temporal contrast event mode and the logarithmic intensity frame mode, which are described in more detail below with reference to FIGS. 5 and 6. In the temporal contrast event mode, the row address encoder 53, the column address encoder 54, and the readout processor 4 all contribute to the event sensor output. In the logarithmic intensity frame mode, only the readout processor 4 contributes to the event sensor output. In addition to these two basic operating modes, the event sensor can also operate in various temporal contrast and logarithmic intensity hybrid modes. Finally, the event sensor has a calibration mode.

[0066] Temporal contrast event mode The temporal contrast event mode operation begins with the pixel-parallel conversion of the current pixel value of the analog voltage V dependent on the light intensity, and the storage of the current pixel value in the converter memory of the in-pixel ADC by the synchronous global shutter method. The current pixel values of the entire pixel array are collectively called a frame.

[0067] Next, the readout processor reads the current pixel values row by row from the pixel array. At the same time, when each row of the pixel array is being read out, the readout processor also reads the corresponding pixel parameters of the pixels in the same row from the pixel parameter memory. In other words, the current pixel values and the corresponding pixel parameters of the pixels in the same row are received simultaneously by the readout processor. The information transfer between the pixel array, the pixel parameter memory, and the readout processor is regulated by the readout controller.

[0068] In the time contrast event mode, the processing options for all pixels are configured for (1) time contrast detection, or (2) time contrast detection and measurement. Therefore, each processing block performs the following processing steps to determine whether to output a time contrast event from the corresponding pixel, as shown in the flowchart in Figure 5: 1. Receive the current pixel value and the corresponding pixel parameters of the same pixel 501. 2. Convert the current pixel value from gray code to binary format 502. 3. Calculate the signed difference between the current pixel value and the previous pixel value 503. 4. Determine whether the signed difference exceeds the positive or negative change detection threshold of the corresponding pixel, and optionally, whether the past event timestamp meets the condition (e.g., older than 1 ms with respect to the current timestamp) 504: a. If YES: i. Request a time contrast event output 505. ii. If the processing option is (2) time contrast detection and measurement: Calculate the magnitude of the gain-corrected time contrast by multiplying the signed difference by the gain correction factor of the corresponding pixel 506. iii. Update the corresponding pixel parameters, including overwriting the previous pixel value with the current pixel value and optionally overwriting the past event timestamp with the current timestamp 507. b. If NO: Do nothing. 5. End of processing 508.

[0069] When the read processor finishes processing a line of the frame, those processing blocks with a request for time contrast event output communicate their corresponding information as the time contrast event output of the event sensor, preferably via a high-speed token-based communication system such as that described in European Patent Application Publication No. 3561685A1. Each time contrast event also includes the pixel address obtained from the row and column encoders, the code of the time contrast, the current timestamp (possibly shared by all time contrast events from the same frame), and optionally, the magnitude of the gain-corrected time contrast. The read processor then proceeds to read and process the next line of the frame. After the read processor finishes reading and processing the entire frame, the event sensor may repeat the above-described operating steps for the next frame.

[0070] All information from the pixel array and pixel parameter memory required by the read processor is digital and has a high-speed access time on the order of 10 ns. The processing blocks mainly perform only addition / subtraction and multiplication operations, which can be achieved by combinational logic. Thus, pipelined per-line reading and processing of one frame containing 1000 lines introduces only an additional delay on the order of 10 μs.

[0071] Logarithmic intensity frame mode Logarithmic intensity mode or logarithmic intensity frame mode operation also starts with capturing a frame of the current pixel values and storing the frame in the converter memory of the in-pixel ADC by the synchronous global shutter method.

[0072] The read processor then reads the current pixel values and corresponding pixel parameters of the pixels in the same row in a per-line manner, which is adjusted by the read controller. Up to this point, the logarithmic intensity frame mode operation is the same as the time contrast event mode. The main difference between the logarithmic intensity frame mode and the time contrast event mode occurs in the processing block.

[0073] In the logarithmic intensity frame mode, all pixel processing options are configured for (3) logarithmic intensity measurement. Therefore, each processing block performs the following processing steps to correct for pixel-to-pixel gain and offset mismatches within the current pixel value, as shown in the flowchart in FIG. 6: 1. Receive the current pixel value and corresponding pixel parameters for the same pixel 601. 2. Convert the current pixel value from gray code to binary format 602. 3. Calculate the offset-compensated pixel value by subtracting the offset compensator for the corresponding pixel from the current pixel value 603. 4. Calculate the final offset-compensated and gain-corrected pixel value by multiplying the offset-compensated pixel value by the gain correction factor for the corresponding pixel 604. 5. End of processing 605.

[0074] When the processing blocks finish processing a row of the frame, their final offset-compensated and gain-corrected pixel values are communicated as the logarithmic intensity frame output of the event sensor, reflecting the logarithmic intensity measurements of their corresponding pixels. Preferably, the high-speed token-based communication system used in the time contrast event mode can also function as the event sensor output in the logarithmic intensity frame mode, while all processing blocks continuously communicate the logarithmic intensity measurements of their corresponding pixels, and the pixel address information can be discarded. Alternatively, the event sensor output in the logarithmic intensity frame mode can employ a standard shift register-based communication system. The readout processor then proceeds to read and process the next row of the frame. After the readout processor has finished reading and processing the entire frame, the event sensor can repeat the above-described operational steps for the next frame.

[0075] Time Contrast and Logarithmic Intensity Hybrid Mode Variant 1 The time contrast and logarithmic intensity hybrid mode variant 1 is substantially the same as the time contrast event mode, except for some processing steps performed by the processing block.

[0076] In the time contrast and logarithmic intensity hybrid mode variant 1, the processing options for all pixels are configured for (4) time contrast detection and logarithmic intensity measurement. Therefore, as shown in the flowchart in FIG. 7, each processing block performs the following processing steps to determine whether to output a time contrast event and a logarithmic intensity measurement from the corresponding pixel: 1. Receive the current pixel value and the corresponding pixel parameters of the same pixel 701. 2. Convert the current pixel value from gray code to binary format 702. 3. Calculate the signed difference between the current pixel value and the previous pixel value 703. 4. Determine whether the signed difference exceeds the positive or negative change detection threshold of the corresponding pixel, and optionally, whether the past event timestamp meets the condition (e.g., older than 1 ms with respect to the current timestamp) 704: a. If YES: i. Request a time contrast event output 705. ii. Calculate the offset-compensated pixel value by subtracting the offset compensator of the corresponding pixel from the current pixel value 706. iii. Calculate the final offset-compensated and gain-corrected pixel value by multiplying the offset-compensated pixel value by the gain correction factor of the corresponding pixel 707. iv. Update the corresponding pixel parameters, including overwriting the previous pixel value with the current pixel value and optionally overwriting the past event timestamp with the current timestamp 708. b. If NO: Do nothing. 5. End of processing 709.

[0077] When the read processor finishes processing a line of the frame, those processing blocks with a request for time contrast event output communicate their corresponding information as the time contrast event and logarithmic intensity hybrid output of the event sensor, preferably via the same high-speed token-based communication system used in the time contrast event mode. Each time contrast event then includes the pixel address obtained from the row and column address encoders, optionally the code for the time contrast, the current timestamp (possibly shared by all time contrast events from the same frame), and the offset-compensated and gain-corrected pixel value reflecting the logarithmic intensity measurement of the corresponding pixel. The read processor then proceeds to read and process the next line of the frame. After the read processor finishes reading and processing the entire frame, the event sensor may repeat the above-described operational steps for the next frame.

[0078] Other time contrast and logarithmic intensity hybrid mode variants Since the processing options are pixel-specific, the processing options for one subset of pixels are configured for one option, while the processing options for another subset of pixels are configured for a different option. This approach is discussed in more detail below with reference to FIGS. 8a-8c, which schematically show the pixel array as rectangular regions divided into various subsets of pixels.

[0079] In the first example shown in FIG. 8a, the processing options for the first subset 81 of pixels in the center of the pixel array are configured for (3) logarithmic intensity measurement, while the processing options for the second subset 82 of pixels corresponding to the remainder of the pixels in the pixel array are configured for (2) time contrast detection and measurement.

[0080] In a second example shown in FIG. 8b, the processing options for a third subset 83 of the pixels scattered within the pixel array can be configured for (3) logarithmic intensity measurement, while the processing options for a fourth subset 84 corresponding to the remainder of the pixels within the pixel array are configured for (2) temporal contrast detection and measurement.

[0081] In a third example shown in FIG. 8c, the processing options for a fifth subset 85 of the pixels at the center of the pixel array can be configured for (3) logarithmic intensity measurement, while the processing options for a sixth subset 86 of the pixels scattered within the pixel array are configured for (4) temporal contrast detection and logarithmic intensity measurement, and the processing options for a seventh subset 87 corresponding to the remainder of the pixels within the pixel array are configured for (2) temporal contrast detection and measurement.

[0082] There can be more different ways to configure the processing options of the pixel array using different combinations of pixel subsets and their processing options. In various such temporal contrast and logarithmic intensity hybrid modes, the event sensor output is preferably supported by the same high-speed token-based communication system used in the temporal contrast event mode. The processing options of the pixel array are configured by an external computer. Thus, the external computer can decode the output of an event sensor in various such hybrid modes based on the known processing option configuration of the pixel array.

[0083] Calibration mode Pixel-specific change detection threshold(s), gain correction factor, and offset compensation value can be obtained using a one-time calibration procedure. In the calibration mode, the processing options for all pixels are configured as calibration options. Thus, the readout processor communicates the current pixel values as the calibration mode output of the event sensor without any processing via the same communication scheme used in the logarithmic intensity frame mode.

[0084] To obtain per-pixel change detection thresholds (plural possible) and gain correction factors, all pixels are exposed to a uniform light source, once at a low illumination level and at another time at a high illumination level. The event sensor captures a first frame called a low frame at the low illumination level and outputs a first frame of current pixel values called low pixel values. The event sensor also captures a second frame called a high frame at the high illumination level and outputs a second frame of current pixel values called high pixel values. These two steps are repeated several times (e.g., 10 times, 100 times, or more) without a specific order of occurrence. As a result, the event sensor has created for each pixel several low pixel values (e.g., 10, 100, or more low pixel values) and several high pixel values (e.g., 10, 100, or more high pixel values).

[0085] To minimize the effect of temporal noise in the event sensor, the temporal average low pixel value and the temporal average high pixel value are calculated for each pixel as follows:

Number

Number

[0086] Next, the pixel value difference is calculated for each pixel as: (Pixel's) Pixel value difference = (This pixel's) Temporal average high pixel value - (This pixel's) Temporal average low pixel value is calculated as.

[0087] Also, the array average pixel value difference is:

Number

[0088] Therefore, the per-pixel gain correction factor is:

Number

[0089] The change detection threshold for each pixel is: [Number] can be calculated as

[0090] For example, if the low illumination level is 0.1 lux, the high illumination level is 100 k lux, and the array average pixel value difference is 2000, the gain correction factor for a pixel with a pixel value difference of 1800 must be approximately 1.1. If the target contrast sensitivity threshold is 15%, the change detection threshold for this pixel must be approximately 18.

[0091] To further obtain a pixel-specific offset compensator, the event sensor captures a third frame in the dark, which is called a dark frame and outputs a third frame of the current pixel value called the dark pixel value. This step is repeated several times (e.g., 10 times, 100 times, or more). As a result, the event sensor has created several dark pixel values (e.g., 10, 100, or more dark pixel values) for each pixel.

[0092] To minimize the effect of temporal noise in the event sensor, the temporal average dark pixel value is calculated as its offset compensator for each pixel: [Number]

[0093] The dark pixel value varies with temperature and can also vary with the bias setting of the pixel front-end electronic signal converter, here a current-voltage converter. Therefore, it is noteworthy that the offset compensation value calibration result obtained at one specific operating temperature and using one specific pixel bias setting is most effective at approximately the same operating temperature and using approximately the same pixel bias setting. Thus, it is recommended to calibrate the event sensor against the offset compensator using one (or several) of the most expected operating temperature(s) and one (or several) of the most likely pixel bias setting(s) for a given use case.

[0094] During the calibration procedure, pixel-specific change detection threshold(s), gain correction factor, and offset compensation value are calculated by an external computer, then written into an external non-volatile storage, and loaded into the pixel parameter memory via the pixel parameter configurator at startup for subsequent event sensor operation.

[0095] In-operation Pixel Parameter Adjustment During event sensor operation, i.e., all other operating modes other than the calibration mode for the event sensor, some of the pixel parameters may need to be adjusted by an external computer. The following are some examples:

[0096] To switch the operating mode of the event sensor, the processing options of the pixel array need to be reconfigured. The new processing option configuration is determined by an external computer and written into the pixel parameter memory as well as the external non-volatile storage via the pixel parameter configurator.

[0097] While the event sensor operates in logarithmic intensity frame mode or time contrast and logarithmic intensity hybrid mode, if the pixel bias setting and / or the operating temperature change, a new set of offset compensation values for the pixel array corresponding to the new pixel bias setting and / or the new operating temperature is required by the event sensor to achieve optimal output accuracy. Preferably, several sets of offset compensation values for the pixel array are obtained through calibration in several expected pixel bias settings and operating temperature ranges and stored in an external non-volatile storage. The set corresponding to the new bias setting and / or the new operating temperature range is selected by an external computer and written into the pixel parameter memory via a pixel parameter configurator.

[0098] When the event sensor operates in time contrast event mode or time contrast and logarithmic intensity hybrid mode, it is sometimes desirable to adjust the contrast sensitivity threshold. To do so, the pixel-specific change detection threshold(s) in the pixel parameter memory are updated via a pixel parameter configurator. The new change detection threshold for each pixel is: (New change detection threshold of the pixel) = (Old change detection threshold of this pixel) · log (1+古いコントラスト感度閾値) (1 + New contrast sensitivity threshold) and is calculated as such.

[0099] For example, if the old contrast sensitivity threshold is 15%, the old change detection threshold of the pixel is 18, and the new contrast sensitivity threshold to be achieved is 30%, the new change detection threshold of this pixel should be 34.

[0100] During the adjustment of the pixel-specific change detection threshold(s), the old change detection threshold(s) of each pixel are sent to an external computer by a pixel parameter configurator. For each old change detection threshold, the external computer calculates the corresponding new change detection threshold based on the old and new contrast sensitivity threshold targets. The resulting new change detection threshold(s) for each pixel are then written back into the pixel parameter memory as well as the external non-volatile storage via the pixel parameter configurator.

[0101] Reference numerals: 10 Pixel array 1 Photoelectric converter, photodiode, PPD 2 Electronic converter 21 Electronic signal converter, current-voltage converter 23 Analog-digital converter 24 Comparator 25 Converter memory 26 Inverting amplifier 27 Buffer 3 Pixel parameter memory 31 Has multi-bit digital storage 4 Readout processor 41 With processing block 51 Bias generator 52 ADC controller 53 Row address encoder 54 Column address encoder 55 Readout controller 56 Pixel parameter configurator 61 External computer 62 External non-volatile storage

Claims

1. An event sensor comprising a pixel array (10), configured to create a signal stream containing event data in response to light incident on the pixel array (10), - a photoelectric converter (1) for each pixel of the pixel array (10) and an electronic converter (2) connected to the photoelectric converter (1), wherein the photoelectric converter (1) and the electronic converter (2) are configured to create and store a digital current pixel value that depends on the intensity of the light incident on the photoelectric converter (1), the photoelectric converter (1) and the electronic converter (2); - a corresponding multi-bit digital storage (31) for each pixel of the pixel array (10), configured to store pixel parameters including the previous pixel value; - a readout processor (4) comprising a plurality of processing blocks, connected to the electronic converter (2) and the multi-bit digital storage (31), and configured to generate a pixel event value of the event data based on a pixel subtraction result obtained by subtracting the previous pixel value from the digital current pixel value comprising The event sensor is configured to support a time contrast event mode that starts with pixel-parallel conversion of an analog voltage dependent on light intensity to a digital current pixel value and storing the digital current pixel value in the converter memory of the electronic converter (2) by means of a synchronous global shutter method. Subsequently, the readout processor reads out the digital current pixel value row by row from the pixel array and simultaneously the corresponding pixel parameters of the same row of pixels from the memory, and the information transfer between the pixel array, the memory and the readout processor is adjusted by a readout controller. Each processing block performs the following processing steps to determine whether a time contrast event should be output from the corresponding pixel: - receiving (501) the digital current pixel value and the corresponding pixel parameters of the same pixel; - calculating (503) the signed difference between the digital current pixel value and the previous pixel value; - determining (504) whether the signed difference exceeds the positive or negative change detection threshold of the corresponding pixel. If yes, requesting a time contrast event output (505) to generate the pixel event value and updating the corresponding pixel parameters, including overwriting the previous pixel value with the digital current pixel value. If no, no such action is taken An event sensor configured to perform **Claim 2** The event sensor according to claim 1, wherein the reading processor (4) is configured to overwrite the stored previous pixel value with the current pixel value each time a pixel event value is generated. **Claim 3** The event sensor according to claim 1, wherein the electronic converter (2) includes an analog-to-digital converter (23) and is configured to create the current pixel value that depends logarithmically on the intensity of light incident on the photovoltaic converter (1). **Claim 4** The event sensor according to claim 3, wherein the electronic converter (2) includes a current-to-voltage converter (21), the current-to-voltage converter (21) is a logarithmic converter, and the analog-to-digital converter (23) is a linear converter, or the current-to-voltage converter (21) is a linear converter and the analog-to-digital converter (23) is a logarithmic converter. **Claim 5** The pixel array (10) consists of a pixel parameter memory comprising a certain number of pixel columns in width, a certain number of pixel rows in height, and one multi-bit digital storage (31) for each of the width number × height number of pixels. The reading processor (4) includes the width number of processing blocks (41), and each of the processing blocks (41) is configured to process one of the pixel columns, or The reading processor (4) includes a number of processing blocks (41) obtained by multiplying the width number by an integer, and each of the processing blocks (41) is configured to process a subset of one of the pixel columns, or The event sensor according to any one of claims 1 to 4, wherein the reading processor (4) includes a number of processing blocks (41) obtained by dividing the width number by an integer, and each of the processing blocks (41) is configured to process a multiple of the pixel columns. **Claim 6** The multi-bit digital storage (31) is configured to further store a change detection threshold value and / or a past event time stamp for a corresponding pixel, and the read processor (4) is configured to generate and / or output the pixel event value on the condition that the current pixel value is greater than the change detection threshold value and different from the previous pixel value, and / or on the condition that the past event time stamp is older than a predetermined time interval. The event sensor according to any one of claims 1 to 5.

7. The read processor (4) is configured such that generating the pixel event value is one of two or more processing options determined by a processing option parameter. The event sensor according to any one of claims 1 to 6.

8. The read processor (4) is configured to process a first pixel of the pixel array (10) or all pixels within a first pixel group of the pixel array (10) according to a first processing option, and to process a second pixel of the pixel array (10) or all pixels within a second pixel group of the pixel array (10) according to a second processing option. The event sensor according to claim 7.

9. The multi-bit digital storage (31) is configured to further store a processing option parameter for a corresponding pixel, and the read processor (4) is configured to process the current pixel value according to the processing option parameter for the corresponding pixel. The event sensor according to any one of claims 1 to 8.

10. The read processor (4) is configured such that the two or more processing options include generation of a logarithmic intensity in the pixel, which is a value that depends logarithmically on the intensity of light incident on the photoelectric converter (1). The event sensor according to any one of claims 7 to 9.

11. The read processor (4) is configured such that the two or more processing options include a calibration mode, and the current pixel value is output unprocessed by the read processor (4). The event sensor according to any one of claims 7 to 10.

12. The read processor (4) simultaneously generates the pixel event value in the pixel based on the logarithmic intensity in the pixel, where the two or more processing options are values that depend logarithmically on the intensity of the light incident on the photovoltaic converter (1), and the pixel subtraction result obtained by subtracting the previous pixel value from the current pixel value. The event sensor according to any one of claims 7 to 11, characterized in that it is configured to include this.

13. The multi-bit digital storage (31) is configured to further store an offset compensation value, and the read processor (4) is configured to consider the offset compensation value when generating the logarithmic intensity. The event sensor according to claim 10 or 12, characterized in that it is like this.

14. The multi-bit digital storage (31) is configured to further store a gain correction coefficient, and the read processor (4) is configured to consider the gain correction coefficient when generating the pixel event value and / or the logarithmic intensity. The event sensor according to claim 10, 12 or 13, characterized in that it is like this.

15. The photovoltaic converter (1) and the electronic converter (2) together occupy a common physical region or volume within the pixel array (10). The event sensor according to any one of claims 1 to 14, characterized in that it is like this.

16. The read processor (4) is configured to process one or more rows or a part of one row of the pixels of the pixel array (10) in parallel. The event sensor according to any one of claims 1 to 15, characterized in that it is like this.

17. A method for creating a signal stream including event data in response to light incident on a pixel array (10), comprising the following steps: - Generating and storing a digital current pixel value that depends on the intensity of the light incident on the photovoltaic converter (1) of each pixel of the pixel array (10) by an electronic converter (2) connected to the photovoltaic converter (1); - Providing pixel parameters including the previous pixel value stored in the multi-bit digital storage (31) corresponding to the pixel for each pixel of the pixel array (10); - A step of generating the pixel event value of the event data based on the pixel subtraction result obtained by subtracting the previous pixel value from the digital current pixel value by a readout processor (4) having a plurality of processing blocks and connected to the electronic converter (2) and the multi-bit digital storage (31). Including The method supports a time contrast event mode that starts with pixel-parallel conversion of an analog voltage dependent on light intensity into a digital current pixel value and storing the digital current pixel value in the converter memory of an electronic converter (2) by means of a synchronous global shutter method. Then, a readout processor reads out the digital current pixel value row by row from the pixel array and simultaneously the corresponding pixel parameters of the same row of pixels from the memory. The information transfer between the pixel array, the memory and the readout processor is adjusted by a readout controller. Each processing block performs the following processing steps to determine whether to output a time contrast event from the corresponding pixel: - Receiving (501) the digital current pixel value and the corresponding pixel parameters of the same pixel - Calculating (503) the signed difference between the digital current pixel value and the previous pixel value - Determining (504) whether the signed difference exceeds the positive or negative change detection threshold of the corresponding pixel. If yes, requesting a time contrast event output (505) to generate the pixel event value and updating (507) the corresponding pixel parameters, including overwriting the previous pixel value with the digital current pixel value. If no, not performing such an action A method configured to perform.

Images