Event sensor and method for generating a signal stream comprising event data

Abstract

The present invention relates to an event sensor comprising a pixel array (10) and configured to generate a signal stream comprising event data in response to light incident on the pixel array (10). The event sensor comprises: a photoelectric converter (1) for each pixel of the pixel array (10) and an electronic converter (2) connected to the photoelectric converter (1), wherein the photoelectric converter (1) and the electronic converter (2) are configured to generate and store a digital current pixel value, the current pixel value depending on the intensity of light incident on the photoelectric converter (1); a corresponding multi-bit digital memory (31) for each pixel in the pixel array (10), which is configured to store a previous pixel value; and a readout processor (4) connected to the electronic converter (2) and the multi-bit digital memory (31) and configured to generate a pixel event value of the event data based on a pixel subtraction result of subtracting the previous pixel value from the current pixel value. The present invention also relates to a method for generating a signal stream comprising event data in response to light incident on the pixel array (10).

Description

Technical Field

[0001] The present invention relates to an event sensor and a method for generating a signal stream comprising event data. Background Art

[0002] Traditional image sensors record a scene by capturing a sequence of consecutive frames, measuring digital intensity values ​​for each pixel in each frame, and then transmitting all of these digital intensity values ​​to a receiving computer for processing. The main drawbacks of this approach include limited temporal resolution, high latency, and high data redundancy.

[0003] In some machine vision tasks, such as tracking, localizing, and detecting moving objects, it is desirable for the machine vision algorithm to receive image sensor input with high temporal resolution and minimal latency. For these machine vision tasks, it is also undesirable to receive and process a large amount of redundant or irrelevant information. Therefore, these machine vision tasks require intelligent image sensors with high temporal resolution, low latency, and low data redundancy. Traditional image sensors cannot meet these requirements.

[0004] In US Pat. No. 7,728,269 B2, an alternative type of image sensor is proposed that encodes the temporal visual contrast of the scene captured by its photosensor. By encoding the temporal contrast, the temporal redundancy of the image sensor's output data is nearly eliminated, resulting in activity-driven sparse data in the format of ON / OFF events. This means that each event consists of the symbol ON / OFF and the pixel coordinates. This event-based temporal contrast sensor concept offers a unique combination of advantages not found in conventional image sensors: high dynamic range, high temporal resolution, low latency, and low data redundancy. Consequently, the concept of event-based sensors has been adopted in many alternative or improved designs and is also the basis of the present invention.

[0005] The design proposed in US Pat. No. 7,728,269 B2 exploits subthreshold MOSFET behavior for logarithmic current-to-voltage conversion to achieve high dynamic range. Because temporal contrast is measured rather than absolute intensity, this design is immune to inter-pixel offset mismatches in the logarithmic current-to-voltage conversion. However, this design still suffers from inter-pixel gain mismatches in both the logarithmic current-to-voltage conversion and the change detection stages. Furthermore, this design uses a storage capacitor to store past light intensity-dependent signals, which are susceptible to various leakage events, leading to inaccurate output events. Summary of the Invention

[0006] The object of the present invention is to propose an event sensor and an accompanying imaging method with improved output accuracy. Furthermore, the proposed design should have the potential to allow compensation of inter-pixel offset mismatch and / or inter-pixel gain mismatch.

[0007] According to the present invention, an event sensor includes a pixel array and is configured to generate a signal stream including event data in response to light incident on the pixel array. In particular, the pixel array can include a rectangular array of W×H pixels, where W will be referred to as the width number, which refers to the number of pixels along a first direction, and H will be referred to as the height number, which refers to the number of pixels along a second direction perpendicular to the first direction. The pixel array can also be referred to as having W columns of H pixels and H rows of W pixels.

[0008] Each pixel in the pixel array includes a photoelectric converter and an electronic converter connected to the photoelectric converter. The electronic converter is configured to generate and store a digital current pixel value, the current pixel value depending on the intensity of light incident on the photoelectric converter.

[0009] While the term "light" in informal contexts typically refers to the visible range (400-700 nm) of electromagnetic (EM) wavelengths, EM radiation in the ultraviolet (UV) and near-infrared wavelengths can still induce a photovoltaic effect in semiconductors, particularly silicon, to a certain extent. Using more exotic materials besides silicon as photovoltaic materials can further expand the detectable wavelength range of photovoltaic converters. Therefore, the term "EM radiation" encompasses more applications of the present invention than the informal term "light." However, in a scientific context, and therefore in this context, "light" can refer to EM radiation of any wavelength, and in advantageous embodiments, "light" refers to the wavelength range between and including the UV and infrared. Therefore, hereinafter, the terms "light" and "EM radiation" may be used interchangeably.

[0010] The event sensor further includes providing a corresponding multi-bit digital memory for each pixel of the pixel array, which is configured to store a previous pixel value. The expression "current pixel value" or "present pixel value" refers to the most recent pixel value obtained from the pair of photoelectric converter and electronic converter, while the expression "previous pixel value" or "past pixel value" refers to a pixel value obtained temporarily before the current or current pixel value (particularly during a previous readout process).

[0011] In addition, the event sensor includes a readout processor configured to generate a pixel event value for the corresponding pixel. The generated pixel event value will then be part of the event data, which constitutes the signal stream generated by the event sensor. For this purpose, the readout processor is connected to the electronic converter and its corresponding multi-bit digital memory. The readout processor generates the pixel event value based on a pixel subtraction result of subtracting the previous pixel value from the current pixel value. In particular, the pixel event value is obtained by comparing the pixel subtraction result with a threshold value. This means that once the pixel subtraction result exceeds or exceeds the threshold value, the pixel event value is either determined as a certain incremental value or simply a binary on / off or "1" / "0" bit value. Alternatively, more advantageously, if this pixel subtraction result exceeds the threshold value, the pixel event value is determined as the pixel subtraction result or a value dependent on the pixel subtraction result. As will be discussed further below, the pixel subtraction result can be further scaled by a gain correction factor to correct for inter-pixel gain mismatch.

[0012] The readout processor generates a pixel event value based on the pixel subtraction result, and in particular, the pixel subtraction result can determine whether a pixel event value is generated. Therefore, there are cases where the readout processor does not generate a pixel event value (for example, if the pixel subtraction result does not exceed a certain threshold).

[0013] According to an advantageous embodiment, the readout processor is configured to overwrite the stored previous pixel value with the current pixel value whenever a pixel event value has been generated. This means in particular that the stored previous pixel value will not be overwritten as long as the readout processor does not generate a new pixel event value. The advantages of this approach are discussed in the following paragraphs:

[0014] Conventional, prior art motion detection techniques compare two consecutive image frames captured at two adjacent points in time. Therefore, only relatively rapid motion, or relatively rapid changes in the intensity of light striking a pixel, can produce noticeable differences between consecutive image frames and distinguish these differences from inherent temporal noise. While conventional motion detection techniques can detect relatively slow motion by reducing the frame rate, doing so hinders the detection of relatively fast motion.

[0015] In contrast, in the case of threshold-dependent pixel event generation, the readout processor ensures that relatively slow changes in the intensity of light impinging on the pixel are ultimately detected by overwriting the previous pixel value with the current pixel value only when the pixel event is generated. In other words, the multi-bit digital memory can retain the same previous pixel value as long as the intensity of the light impinging on the pixel does not change by more than the amount defined by the change detection threshold. Thus, the readout processor can compare the current pixel value from the current point in time with the previous pixel value from a point in the past that is relatively distant from the current point in time, so that relatively slow changes in the intensity of the light impinging on the pixel can be detected as soon as the current pixel value differs from the previous pixel value by more than the change detection threshold.

[0016] Furthermore, the readout processor advantageously decides for each pixel individually whether to overwrite a previous pixel value with a current pixel value. Thus, the multi-bit digital memories of different pixels can store previous pixel values ​​from different past points in time, allowing the readout processor to detect a wide range of rates of change of the light intensity impinging on those pixels, e.g., from relatively slow changes to relatively rapid changes.

[0017] Photoelectric converters convert incident light into corresponding analog electrical signals, which electronic converters further convert into digital signals with the help of analog-to-digital converters. A photovoltaic converter is a converter that performs photoelectric conversion. The photoelectric converter can be a photodiode, in particular a partially or fully pinned photodiode (PPD). It can be configured to generate a current, known as a photocurrent, that depends, in particular linearly depends on or is proportional to, the intensity of light impinging on the photoelectric converter. In contrast, electronic converters are so named because they only perform conversions between different electronic signals, or electron-to-electron conversions.

[0018] The electronic converter may comprise an electronic signal converter, which may be a current-to-current converter, a current-to-voltage converter, a voltage-to-current converter or a voltage-to-voltage converter. The output of the electronic signal converter may depend on its input signal linearly, logarithmically or according to some other function. The function may in particular be a compression function, such as a logarithmic function, so as to allow a wider range of inputs (e.g. corresponding to a 60-fold change in light intensity) to be compressed into a relatively small range of electronic signals (e.g. a voltage swing of 500 mV). For example, a photoelectric converter performs photon-to-electron conversion, thereby generating a photocurrent. A logarithmic current-to-voltage converter converts this photocurrent into an analog voltage signal, which is part of the electronic converter. This analog voltage signal, which is logarithmically related to the incident light intensity, is then converted into a digital signal or numerical value using an analog-to-digital converter, which is also part of the electronic converter, as will be described below.

[0019] Advantageously, therefore, the electronic converter comprises an analog-to-digital converter and is configured to generate said current pixel value, said current pixel value being logarithmically related to the intensity of light incident on said photoelectric converter.

[0020] One advantageous concept for an analog-to-digital converter (ADC) is the so-called single-slope ADC concept. To this end, the ADC includes a converter memory, which is a multi-bit memory and is configured to receive a digital count and a reference signal synchronized with the digital count, to continuously compare an analog input signal with the reference signal, and to store the digital count value in the converter memory as soon as the reference signal is equal to or exceeds the analog input signal. In particular, the reference signal can have a slope with a constant gradient, while the digital count is counted linearly or logarithmically in synchronization with the slope of the reference signal. In the case of linear counting, the ADC will generate a digital value that is linearly related to the analog input signal, while in the case of logarithmic counting, the generated digital value is logarithmically related to the analog input signal. Other combinations of digital count and reference signal can produce output values ​​linearly or logarithmically based on the analog input (e.g., an exponentially varying reference signal and a linearly counted digital count) to obtain a logarithmic-to-analog conversion. The digital count can, in particular, be in Gray code format.

[0021] For this purpose, the analog input signal supplied to the ADC (in particular, the analog input signal supplied to the ADC by an electronic signal converter) depends on the intensity of the light incident on the photoelectric converter. The ADC thus ensures that the converter memory will be filled with a multi-bit digital value that depends, in particular, linearly or logarithmically on the intensity of the light incident on the corresponding pixel or photoelectric converter.

[0022] As further explained above, the electronic converter may comprise an electronic signal converter, such as a current-to-voltage converter, which converts an electronic analog signal (e.g., a current) generated by the photovoltaic converter into a different electronic analog signal (e.g., a voltage). Advantageously, the current-to-voltage converter is a logarithmic converter, while the analog-to-digital converter is a linear converter, or the current-to-voltage converter is a linear converter, while the analog-to-digital converter is a logarithmic converter. As previously described, these two possible embodiments will ensure that the digital current pixel value generated and stored (in particular, the digital current pixel value stored in the converter memory) has a logarithmic relationship to the light intensity incident on the pixel or photoelectric converter. Although a logarithmic relationship has the advantage of a higher dynamic range, alternatively, there may be a difference between the light intensity and the digital current pixel value generated and stored, i.e., there is a non-logarithmic relationship, such as a linear relationship.

[0023] As previously mentioned, the pixel array is composed of a width (W) number of pixel columns and a height (H) number of pixel rows. Since each pixel also has a corresponding multi-bit digital memory for storing previous pixel values ​​and possible other parameters of the pixel, the event sensor preferably also includes a pixel parameter memory composed of W×H multi-bit digital memories. Although this is an option, the multi-bit digital memories of the pixel parameter memory do not have to be arranged in an array like the pixel array. In addition, the readout processor includes W processing blocks, each of which is configured to process one of the W pixel columns. In other words, any processing performed by the readout processor on a single current pixel value of a pixel or a parameter value of a multi-bit digital memory can actually be performed by the corresponding processing block.

[0024] When the readout processor consists of exactly W processing blocks, each processing block can be dedicated to processing H pixels in one column of the pixel array. In this case, the readout processor consists of a row of W processing blocks.

[0025] Alternatively, the readout processor may have more than W processing blocks, in particular, a multiple of W processing blocks, such as 2, 3, or more (M times) W processing blocks, which may be arranged in M ​​rows of W columns, where M is significantly smaller than H. In this alternative, the H pixels in each column can be divided into M groups, each group being processed by one of the M processing blocks dedicated to that column of pixels. In other words, each of the M processing blocks can be dedicated to processing a column of H pixels. Because all processing blocks can process simultaneously, M rows of pixels can be processed simultaneously.

[0026] As another alternative, the readout processor may have fewer than W processing blocks, in particular W divided by an integer number of processing blocks, such as W divided by 2, W divided by 3, or W divided by D processing blocks, which may be arranged in a row of W divided by D columns, where D is significantly smaller than W. In this alternative, each processing block may be dedicated to processing D×H pixels in D columns of the pixel array.

[0027] Advantageously, the multi-bit digital memory is configured to further store a change detection threshold. The readout processor can then be configured to generate and / or output the pixel event value if the current pixel value differs from the previous pixel value by more than the change detection threshold. This means that a pixel event value is only generated when the intensity of light incident on the corresponding pixel changes by more than a specific amount defined by the change detection threshold. The multi-bit digital memory can also be configured to store a past event timestamp for the corresponding pixel. Here, the readout processor can be configured to generate and / or output the pixel event value if the past event timestamp is older than a predetermined time interval. The past event timestamp is a value that represents the time at which the corresponding pixel previously generated a pixel event value. By referring to this parameter, it is possible to avoid generating two or more pixel event values ​​for the same pixel within a very short period of time. Advantageously, both the change detection threshold and the past event timestamp are taken into account when determining whether to generate a new pixel event value. Rather than storing a single change detection threshold, the multi-bit digital memory can be configured to store two separate change detection thresholds (i.e., a positive change detection threshold and a negative change detection threshold).

[0028] The predetermined time interval value described above can be stored in a corresponding multi-bit memory for each individual pixel. However, the predetermined time interval need not be pixel-specific, i.e., one such value can apply to all pixels of the pixel array. Therefore, it may be advantageous to store the predetermined time interval value in a single, separate memory that is accessible to all processing blocks of the readout processor.

[0029] According to an advantageous embodiment, the readout processor is configured such that generating the pixel event value is one of two or more processing options determined by a processing option parameter. In other words, the processing option parameter can determine whether to generate the pixel event value or whether to pursue another processing option, such as generating a logarithmic intensity or entering a calibration mode. The expressions "processing option" and "processing mode" can be used interchangeably. Furthermore, generating a pixel event value can be considered as processing the corresponding pixel in a temporal contrast mode.

[0030] In an advantageous embodiment, the readout processor is configured to process a first pixel or all pixels in a first group of pixels of the pixel array according to a first processing option, and to process a second pixel or all pixels in a second group of pixels of the pixel array according to a second processing option. In other words, the processing option for the first subset of pixels is configured as one option, while the processing option for the second subset of pixels is configured as a different option. In this way, the pixel array can be partitioned, with different processing modes applied to different partitions. More than two partitions can be generated by configuring the readout processor to process a third pixel or all pixels in a third group of pixels according to a third processing option, and so on.

[0031] There may be one processing option parameter for all pixels, or one processing option parameter specific to a group of pixels. In the extreme case, each individual pixel may have its own dedicated processing option parameter. In the latter embodiment, the multi-bit digital memory corresponding to a pixel is configured to further store the processing option parameter for the corresponding pixel, wherein the readout processor is configured to process the current pixel value according to the processing option parameter for the corresponding pixel. In other words, the readout processor processes each pixel according to the processing option parameter value for that pixel. Thus, one or more pixels in a first group may be assigned a first processing option parameter value, while one or more pixels in a second group may be assigned a second processing option parameter value, and so on. As an example, which will be described in further detail, at a given time, a first group of pixels may be assigned for temporal contrast event detection, while a second group of pixels may be assigned for log-intensity frame generation. In other words, the pixel array may be divided into two or more subsets of pixels, with the pixels in each subset being dedicated to a subset-specific processing option. For example, a first subset of pixels may be configured for pixel event value generation, while a second subset of pixels may be configured for log-intensity frame generation.

[0032] Advantageously, the readout processor can be configured such that one of the processing options is to generate an intensity value at the pixel, the intensity value being dependent on the intensity of light incident on the photoelectric converter of the pixel. In particular, this processing option can be to generate a logarithmic intensity at the pixel, the logarithmic intensity being a value that is logarithmically related to the intensity of light incident on the photoelectric converter. This means that, rather than possibly generating a pixel event value for this particular pixel, the readout processor will generate an intensity or logarithmic intensity value. This option can also be referred to as processing the corresponding pixel in a logarithmic intensity mode.

[0033] According to a preferred embodiment, the readout processor is configured such that the processing options include a calibration mode, in which the current pixel value is output by the readout processor without processing. During the calibration process, while a pixel, one or more groups of pixels, or all pixels of the pixel array are in calibration mode, other parameters in the multi-bit digital memory of the corresponding pixel can be calculated and changed to optimize the function of the event sensor. In particular, a change detection threshold, a gain correction factor, and / or an offset compensation value can be obtained or retrieved during this calibration process. To this end, the readout processor sends the current pixel value to an external device without processing it. During the calibration process, the pixel array or a region thereof can be exposed to a known light source, and the external device can then use the unprocessed current pixel value to calculate various parameters for each pixel. While the external device can be a processor that is not part of the event sensor described herein, it can also be a processing device that is part of a device in which the event sensor is built, or even a processing device that is placed on the same chip and / or in the same housing as the event sensor.

[0034] Advantageously, the readout processor is configured such that a processing option includes simultaneously generating a logarithmic intensity and a pixel event value at the pixel. As previously described, the logarithmic intensity is a value that is logarithmically related to the intensity of light incident on the photoelectric converter, while the pixel event value is a pixel subtraction result based on subtracting the previous pixel value from the current pixel value. In other words, both the pixel event value and the logarithmic intensity of the same pixel are generated by the readout processor. This processing option can be considered a hybrid between a temporal contrast mode and a logarithmic intensity mode.

[0035] In a preferred embodiment, the multi-bit digital memory is further configured to store an offset compensation value, wherein the readout processor is configured to take the offset compensation value into account when generating the log-intensity. In particular, the offset compensation value can be subtracted from the current pixel value to derive an offset-compensated current pixel value, which can then be used as the basis for the intensity or log-intensity of the pixel.

[0036] In a further advantageous embodiment, the multi-bit digital memory is configured to further store a gain correction factor. Here, the readout processor is configured to take this gain correction factor into account when generating the pixel event value and / or the logarithmic intensity. In particular, the pixel subtraction result or the offset-compensated current pixel value is multiplied by the gain correction factor to obtain the pixel event value or the logarithmic intensity.

[0037] When the multi-bit digital memory stores the offset compensation value and the gain correction factor, then

[0038] Logarithmic intensity = (current pixel value - offset compensation value) * gain correction coefficient,

[0039] and / or

[0040] Pixel event value = (current pixel value - past pixel value) * gain correction coefficient.

[0041] As mentioned above, the pixel event value is calculated if the pixel subtraction result exceeds the change detection threshold. If the pixel subtraction result does not exceed the change detection threshold, this calculation is not performed.

[0042] Each pixel parameter memory, consisting of a multi-bit digital memory, corresponds to a pixel of the pixel array and can be configured to store processing option parameters, previous pixel values, change detection thresholds, gain correction factors, past event timestamps and / or offset compensation values ​​for each pixel. Each of these parameters has been described above. The pixel parameter memory can be accessed via an external connection so that it can be directly accessed (i.e., read from or written to) by an external source. The external source is, in particular, an input source external to the event sensor. In particular, the parameters in the pixel parameter memory can be written to or changed during a calibration process / procedure or calibration mode. During operation of the event sensor, in particular during the process of generating pixel event values ​​and / or logarithmic intensities, some pixel parameters (in particular, the change detection threshold) can be adjusted or updated. This update procedure requires reading the old change detection threshold from each individual multi-bit memory in the pixel parameter memory, calculating the new change detection threshold, and writing the new change detection threshold back to the corresponding multi-bit memory.

[0043] According to an advantageous embodiment, the photoelectric converter and the electronic converter of each pixel cooperatively occupy a common physical area or volume in the pixel array. On the other hand, the readout processor and the pixel parameter memory can be placed in a different area or volume away from the pixel array. Here, the expression "volume" refers to a space or a three-dimensional (3D) area in the semiconductor device.

[0044] If the pixels are arranged in a rectangular pixel array of W columns and H rows, each column or multiple columns of pixels can share one or more common processing blocks of the readout processor. Therefore, the readout processor can be referred to as a column-parallel readout processor and is configured to process one row, multiple rows or a portion of a row of pixels of the pixel array in parallel. The processing blocks of the corresponding columns can be placed on the event sensor die at the end of the column and can be connected to each pixel of the column via a shared bus, which is referred to as the ADC bus below. For example, in the case where the pixel outputs a 12-bit current pixel value, the ADC bus can be 12 bits wide. In particular, the ADC bus can be connected to the output of the analog-to-digital converter to access the converter memory of each pixel. Preferably, if the ADC is implemented using a digital count, the digital count can be provided to the ADC of each column via the same ADC bus. In this case, the ADC bus can be used both to provide a digital count and to read out the current pixel value.

[0045] Similarly, the multi-bit digital memories of the pixel parameter memory can also be arranged into an array of W columns and H rows of multi-bit digital memories, or they can be configured in some other way so that one row, multiple rows, or a portion of a row can be accessed by the readout processor at a time. Then, each readout processing block here can also be connected to a column, multiple columns, or a portion of a column of the multi-bit digital memory. Therefore, if the total size of all parameters of a multi-bit digital memory is 75 bits, a column, multiple columns, or a portion of a column of the multi-bit digital memory can also share a common parameter bus, such as a 75-bit parameter bus. In fact, a parameter bus (such as a 75-bit parameter bus) that transmits all parameters of a multi-bit digital memory at once may be too wide. In this case, the parameter bus can have a smaller width, such as a 38-bit or narrower bus, and then two or more clocks are required to transmit all parameters of a multi-bit digital memory.

[0046] As an alternative to implementing the readout of the current pixel values ​​via the ADC bus, one or more columns of pixels can be read out by the corresponding processing blocks via a shift register system. During readout, the converter memories within each column can be connected to each other to form a shift register chain. The current pixel values ​​stored in the converter memories will then be moved to the corresponding processing blocks one pixel at a time until all current pixel values ​​of a column have been read by the processing blocks. This is a destructive readout process, and the current pixel values ​​stored in the converter memories will be deleted or overwritten. In addition, in order to implement such a shift register system for reading and writing pixel parameter memories, a separate shift register chain may be required. Therefore, it is more efficient to read from and write to the pixel parameter memories via the above-mentioned parameter bus.

[0047] During readout, the readout controller selects which row or portion of a row of the pixel array has access to the shared ADC bus and which (corresponding) row or portion of a row of the pixel parameter memory has access to the shared parameter bus.

[0048] It is possible and advantageous to manufacture the event sensor on a single die, which allows the pixel array, pixel parameter memory, and processing block to be physically arranged on the same die. However, with today's technology, it may be more practical to manufacture this event sensor using a 2-die stack structure, where the pixel array is manufactured on the sensor die and the pixel parameter memory and processing block are manufactured on the logic die, where the sensor die is stacked and combined on top of the logic die. In future embodiments, it may be advantageous to utilize a 3-die stack structure: the first die carries an array of photoelectric converters connected to or paired with electronic signal converters, or an array of photoelectric converters connected to or paired with a portion of electronic signal converters; the second die carries an array of ADCs, or an array of a portion of electronic signal converters connected to or paired with an ADC; and the third die carries the pixel parameter memory and processing block.

[0049] According to another aspect of the present invention, a method for generating a signal stream comprising event data in response to light incident on a pixel array is provided. The method comprises the following steps:

[0050] - generating and storing a digital current pixel value, said digital current pixel value being dependent on the intensity of light incident on the photoelectric converter of each pixel of said pixel array by an electronic converter connected to said photoelectric converter;

[0051] - providing a previous pixel value for each pixel of the pixel array, the previous pixel value being stored in a multi-bit digital memory corresponding to the pixel; and

[0052] - generating a pixel event value for said event data based on a pixel subtraction result of subtracting said previous pixel value from said current pixel value by a readout processor connected to said electronic converter and said multi-bit digital memory.

[0053] Any features and advantages described herein with respect to event sensors may similarly apply to the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Some examples of embodiments of the present invention will be explained in more detail in the following description with reference to the accompanying drawings, in which:

[0055] Figure 1 shows a block diagram of a pixel according to one embodiment;

[0056] Figure 2 Shown in Figure 1 A block diagram of a single slope analog-to-digital converter used in a pixel;

[0057] Figures 3a to 3c Three different circuit diagrams of a logarithmic current-to-voltage converter according to three different embodiments are shown;

[0058] Figure 4 shows a block diagram of an event sensor according to a preferred embodiment;

[0059] Figure 5 a flow chart showing the steps performed by a readout processor in a temporal contrast event mode according to a preferred embodiment;

[0060] Figure 6 a flow chart showing the steps performed by a readout processor in logarithmic intensity mode according to a preferred embodiment;

[0061] Figure 7 a flow chart showing the steps performed by a readout processor in temporal contrast and log intensity blending modes according to a preferred embodiment;

[0062] Figures 8a to 8c Three different possible ways of partitioning the pixel array and applying different processing modes to different partitions are schematically shown. DETAILED DESCRIPTION

[0063] Figure 1 A block diagram of a pixel according to a preferred embodiment is shown. Incident light is detected by a photoelectric converter 1, which is a photodiode 1 in this case. The photodiode generates a current I, i.e., a photocurrent, which depends on the intensity of the light incident on the active area of ​​the photodiode 1. A current-to-voltage converter 21 converts the photocurrent I into a voltage V. This conversion can be a linear conversion, in which the voltage is linearly related or proportional to the photocurrent. Alternatively, the conversion can be a logarithmic conversion, in which the voltage is logarithmically related to the photocurrent. Other conversion functions may be possible but are less practical.

[0064] The current-to-voltage converter 21 and the analog-to-digital converter (ADC) 23 cooperate to form an electronic converter 2 that cooperates with the photodiode 1 to produce a digital current pixel value. The ADC 23 can be a linear converter, whose output digital value is linearly related to its input analog value. Alternatively, the ADC 23 can be a logarithmic converter, wherein its output digital value is logarithmically related to its input analog value. Advantageously, since the digital current pixel value produced by the combination of the current-to-voltage converter 21 and the ADC 23 is logarithmically related to the incident light or photocurrent, the linearly converted current-to-voltage converter 21 can be combined with the logarithmically converted ADC 23. Conversely, the logarithmically converted current-to-voltage converter 21 can be combined with the linearly converted ADC 23 to obtain similar results.

[0065] The ADC 23 shown here is a single slope analog to digital converter, and its function will be referred to as Figure 2Described in more detail. The central part of the single-slope analog-to-digital converter 23 is the converter memory 25. Once the analog-to-digital conversion is completed, its output digital value (i.e., the current pixel value) is stored in the converter memory 25. The digital count is a multi-bit digital value that is being counted up or down, which is provided to the converter memory 25 via the path indicated by the arrow marked "W". In addition, a reference signal that is a ramp with a positive or negative slope is provided to the comparator 24. The other input of the comparator 24 is connected to the output of the current-to-voltage converter 21 to obtain its input analog value to be converted (here, the voltage V). Once the ramp reference signal exceeds the voltage V, the comparator 24 generates a signal that serves as a latch signal for the converter memory 25. Once the latch signal is received, the converter memory 25 stores the current value of the digital count provided to it. The value stored in the converter memory 25 represents the current pixel value and can be read out later via the path indicated by the arrow with the letter "R". Whether the converter memory 25 is in the write “W” mode or the read “R” mode is determined by the R / W selection signal supplied to the converter memory 25 .

[0066] Whether ADC 23 implements linear or logarithmic conversion can be determined by the relationship between the reference signal and the digital count. For example, an exponentially ramped reference signal combined with a linearly increasing or decreasing digital count can allow for logarithmic conversion. However, advantageously, the reference signal is a single-slope (i.e., constant-slope) linear ramp signal, and linear or logarithmic conversion can be achieved by linearly or logarithmically increasing or decreasing the digital count, respectively.

[0067] The reference signal is a global reference voltage generated by the peripherals. The digital count is a global bit pattern provided by the peripheral counters via a shared digital bus. The reference signal voltage sweeps from a starting voltage level to an ending voltage level. Simultaneously, the peripheral counters begin counting. Depending on whether the counting is linear or logarithmic, the digital count bit pattern value increases or decreases linearly or logarithmically over time. To avoid storing spurious bit pattern values ​​when a latch signal is generated between two consecutive count conversions, the digital count bit pattern is preferably in Gray code format. Converter memory 25 can be based on SRAM or DRAM circuitry.

[0068] The resolution of ADC 23 should be determined after considering the target contrast sensitivity threshold and the dynamic range of the event sensor, as well as the desired gain correction margin and offset margin to handle inter-pixel mismatch. The purpose of the gain correction margin is to later correct for the inter-pixel gain mismatch introduced primarily by the current-to-voltage converter 21. The purpose of the offset margin is to accommodate the inter-pixel offset mismatch of the output of the current-to-voltage converter 21, in other words, to ensure that the input analog value (i.e., voltage V) of ADC 23 falls within the input range supported by ADC 23. Taking these considerations into account, the resolution of ADC 23 can be determined as:

[0069]

[0070] For example, to achieve a contrast sensitivity threshold of 15% within a dynamic range of 120 dB while allowing for a 4-bit gain correction tolerance and a 1-bit offset tolerance, the ADC 23 should have at least 12-bit resolution.

[0071] Figures 3a to 3c The implementation of the logarithmic current-voltage converter according to three preferred embodiments is shown. By utilizing the subthreshold characteristics of N-type MOSFET (NMOS) or P-type MOSFET (PMOS), high dynamic range and low current-voltage conversion delay can be achieved. Figure 3a In the circuit shown, the inverting amplifier 26 is connected between the source and gate of the NMOS. Figure 3b In the circuit of FIG. 1 , the inverting amplifier 26 is connected between the source and drain of the PMOS. In both circuits, the generated voltage V is supplied to the output of the inverting amplifier 26. Figure 3c In the circuit, the photodiode 1 used is a pinned photodiode. It is connected in series with two NMOS transistors. The input of the buffer 27 is connected to the source / drain connection between the two NMOS transistors and a voltage V is provided at its output. Figure 3a In , the bias voltage is provided to the inverting amplifier 26. Figure 3b In , a bias voltage is provided to the inverting amplifier 26, and another possibly separate bias voltage is provided to the gate of the PMOS. Figure 3c In the embodiment, a bias voltage is provided to the buffer 27 and another, possibly separate, bias voltage is provided to the gate of the bottom NMOS. Figure 3a The drain of the NMOS and Figure 3c The drain and gate of the top NMOS are connected to the supply voltage of the circuit.

[0072] Figure 4FIG1 shows a block diagram of an event sensor according to a preferred embodiment. The central elements of the event sensor are a pixel array 10 having W columns×H rows of pixels, a pixel parameter memory 3 having W columns×H rows of multi-bit digital memories 31, and a readout processor 4 having W columns of parallel processing blocks 41. Thus, there is a one-to-one mapping between each pixel of the pixel array 10 and the corresponding multi-bit digital memory 31 of the pixel parameter memory 3. In addition, a bias generator 51 provides a pixel bias voltage to the pixel array 10 for biasing the current-to-voltage converter 21, as shown in FIG1 . Figures 3a to 3c The ADC controller 52 generates a ramp / reference signal and a linear or logarithmic digital count in Gray code format and provides them to the pixel array 10 for the in-pixel ADC 23 .

[0073] The readout processor 4 reads the current pixel value from the pixel array 10 and reads / writes pixel parameters from / to the pixel parameter memory 3. The pixel parameter memory 3 is preferably SRAM-based. Read and write operations between the pixel array 10, the readout processor 4, and the pixel parameter memory 3 are coordinated by a readout controller 55. The pixel parameter memory 3 can be loaded from an external non-volatile memory 62 (e.g., flash memory) and configured by an external computer 61 through a pixel parameter configurator 56.

[0074] Each multi-bit digital memory 31 of the pixel parameter memory 3 stores pixel parameters of its corresponding pixel. The pixel parameters include processing option parameters, previous pixel values, one or two (separate positive and negative) change detection thresholds, a gain correction factor, and may also include a past event timestamp and an offset compensation value or offset compensator.

[0075] The processing options are pixel specific and determine how the processing block 41 operates on the corresponding pixel. There are five processing options or processing modes for a pixel determined by the processing option parameters: (1) temporal contrast detection, (2) temporal contrast detection and measurement, (3) logarithmic intensity measurement, (4) temporal contrast detection and logarithmic intensity measurement, and (5) calibration. In the temporal contrast detection mode and the temporal contrast detection and measurement mode, a pixel event value can be generated at the corresponding pixel (depending on the temporal contrast). In the logarithmic intensity measurement mode, a logarithmic intensity is generated at the corresponding pixel. The temporal contrast detection and logarithmic intensity measurement mode is a hybrid mode in which both a pixel event value and a logarithmic intensity can be generated at the corresponding pixel (depending on the temporal contrast). In the calibration mode, the event sensor is calibrated or partially calibrated by taking measurements under predetermined environmental parameters (e.g., temperature) and during exposure of the pixel array to radiation having predetermined parameters (e.g., intensity, wavelength, etc.), and thereby obtaining or adjusting pixel parameters for some or all pixels.

[0076] The change detection threshold, gain correction factor, and offset compensation value are all pixel-specific to allow inter-pixel gain and offset mismatch correction and compensation. Their values ​​can be obtained through a one-time calibration procedure.

[0077] It can be assumed that each multi-bit digital memory 31 of the pixel parameter memory 3 stores all the above-mentioned pixel parameters, and each parameter is 12 bits, except for the processing option parameter which is 3 bits. Therefore, each multi-bit digital memory 31 has a total of 75 bits. In this case, when using the most advanced 28nm SRAM technology, the silicon area of ​​the entire pixel parameter memory 3 serving a 1 million pixel array is less than 10mm 2 .

[0078] The event sensor can be switched between several different operating modes by configuring the pixel processing options. The event sensor has two basic operating modes: time contrast event mode and logarithmic intensity frame mode, which will be discussed below. Figure 5 and Figure 6 A more detailed description is given below. In the time contrast event mode, the row address encoder 53, the column address encoder 54, and the readout processor 4 all contribute to the output of the event sensor. In the log intensity frame mode, only the readout processor 4 contributes to the output of the event sensor. In addition to these two basic operating modes, the event sensor can also operate in various mixed time contrast and log intensity modes. Finally, the event sensor has a calibration mode.

[0079] Temporal Contrast Event Pattern

[0080] The operation of the time contrast event mode begins by converting the light intensity-related analog voltage Vpixel in parallel to the current pixel value and storing it in the converter memory of the in-pixel ADC in a synchronized global shutter manner. The current pixel value of the entire pixel array is collectively called a frame.

[0081] The readout processor then reads the current pixel values ​​from the pixel array row by row. Simultaneously, as it reads each row of the pixel array, it also reads the corresponding pixel parameters for the pixels in that row from the pixel parameter memory. In other words, the readout processor simultaneously receives both the current pixel values ​​and the corresponding pixel parameters for the pixels in that row. The readout controller coordinates the transfer of information between the pixel array, the pixel parameter memory, and the readout processor.

[0082] In the temporal contrast event mode, the processing options for all pixels are configured as (1) temporal contrast detection, or (2) temporal contrast detection and measurement. Therefore, each processing block performs the following processing steps to decide whether to output a temporal contrast event from the corresponding pixel, such as Figure 5 As shown in the flowchart:

[0083] 1.501 receives the current pixel value and corresponding pixel parameters of the same pixel.

[0084] 2.502 Convert the current pixel value from Gray code to binary format.

[0085] 3.503 Calculate the signed difference between the current pixel value and the previous pixel value.

[0086] 4.504 Determine whether the sign difference exceeds the positive or negative change detection threshold for the corresponding pixel, and optionally, whether the past event timestamp satisfies the condition (e.g., 1 ms earlier relative to the current timestamp):

[0087] a. If yes:

[0088] i.505 requests time contrast event output.

[0089] ii.506 If the processing option is (2) Temporal Contrast Detection and Measurement: Calculate the gain-corrected temporal contrast magnitude by multiplying the signature difference by the gain correction factor for the corresponding pixel.

[0090] iii.507 updates the corresponding pixel parameters, including overwriting the previous pixel value with the current pixel value, and optionally overwriting the past event timestamp with the current timestamp.

[0091] b. If no: do nothing.

[0092] 5.508 Processing completed.

[0093] When the readout processor completes processing for a row of frames, the processing blocks that requested the temporal contrast event output will preferably transmit the corresponding information as the temporal contrast event output of the event sensor via a token-based high-speed communication system such as that described in EP3561685A1. Each temporal contrast event contains the pixel address obtained from the row address encoder and the column address encoder, the sign of the temporal contrast, the current timestamp (which may be shared by all temporal contrast events from the same frame), and optionally also the gain-corrected temporal contrast value. The readout processor then continues to read and process the next row of frames. After the readout processor completes reading and processing an entire frame, the event sensor can repeat the above steps on the next frame.

[0094] The information from the pixel array and pixel parameter memory required by the readout processor is digital and has a fast access time of approximately 10ns. The processing block primarily performs only addition / subtraction and multiplication operations, which can be implemented using combinational logic. Therefore, pipelined, row-by-row reading and processing of a frame containing 1000 rows only adds approximately 10μs of latency.

[0095] Logarithmic intensity frame mode

[0096] Operation in log intensity mode or log intensity frame mode also begins by capturing a frame of current pixel values ​​and storing the frame in the converter memory of the in-pixel ADC in a synchronized global shutter manner.

[0097] The readout processor, coordinated by the readout controller, then reads the current pixel values ​​and corresponding pixel parameters for each row of pixels on a row-by-row basis. Up to this point, the operation of log-intensity frame mode is identical to that of temporal contrast event mode. The main difference between log-intensity frame mode and temporal contrast event mode occurs in the processing blocks.

[0098] In log intensity frame mode, the processing options for all pixels are configured as (3) log intensity measurement. Therefore, each processing block performs the following processing steps to correct the inter-pixel gain and offset mismatch in the current pixel value, as follows: Figure 6 As shown in the flowchart:

[0099] 1.601 receives the current pixel value and corresponding pixel parameters of the same pixel.

[0100] 2.602 Convert the current pixel value from Gray code to binary format.

[0101] 3.603 Calculate an offset-compensated pixel value by subtracting the offset compensator of the corresponding pixel from the current pixel value.

[0102] 4.604 Calculate the final offset-compensated pixel value and gain-corrected pixel value by multiplying the offset-compensated pixel value by the gain correction factor of the corresponding pixel.

[0103] 5.605 Processing completed.

[0104] When the processing blocks have completed processing a row of frames, their final offset-compensated pixel values ​​and gain-corrected pixel values ​​reflect the logarithmic intensity measurements of their corresponding pixels, which are transmitted as the logarithmic intensity frame output of the event sensor. Preferably, the token-based high-speed communication system used in the time-contrast event mode can also serve the event sensor output in the logarithmic intensity frame mode, where all processing blocks transmit the logarithmic intensity measurements of their corresponding pixels in turn, and the pixel address information can be discarded. Alternatively, the event sensor output in the logarithmic intensity frame mode can use a standard shift register-based communication system. The readout processor then continues to read and process the next row of frames. After the readout processor has completed reading and processing the entire frame, the event sensor can repeat the above operation steps on the next frame.

[0105] Temporal Contrast and Log Intensity blending mode variant 1

[0106] The Temporal Contrast and Log-Intensity Mixed Mode variant 1 is almost identical to the Temporal Contrast Event Mode, except for a few processing steps performed by the processing block.

[0107] In the temporal contrast and log intensity mixed mode variant 1, the processing options for all pixels are configured as (4) temporal contrast detection and log intensity measurement. Therefore, if Figure 7 As shown in the flowchart in , each processing block performs the following processing steps to decide whether to output a temporal contrast event and the logarithmic intensity measurement from the corresponding pixel:

[0108] 1.701 receives the current pixel value and corresponding pixel parameters of the same pixel.

[0109] 2.702 Convert the current pixel value from Gray code to binary format.

[0110] 3.703 Calculate the signed difference between the current pixel value and the previous pixel value.

[0111] 4.704 Determine whether the sign difference exceeds the positive or negative change detection threshold for the corresponding pixel, and optionally,

[0112] Whether the event timestamp meets the conditions (for example, 1ms earlier than the current timestamp):

[0113] a. If yes:

[0114] i.705 requests time contrast event output.

[0115] ii.706 calculates an offset-compensated pixel value by subtracting the offset compensator of the corresponding pixel from the current pixel value.

[0116] iii. 707 Calculate final offset-compensated pixel values ​​and gain-corrected pixel values ​​by multiplying the offset-compensated pixel values ​​by the gain correction factors of the corresponding pixels.

[0117] iv.708 updates the corresponding pixel parameters, including overwriting the previous pixel value with the current pixel value, and optionally overwriting the past event timestamp with the current timestamp.

[0118] b. If no: do nothing.

[0119] 5.709 Processing completed.

[0120] When the readout processor completes processing for a row of frames, those processing blocks that requested the temporal contrast event output will preferably transmit the corresponding information as the temporal contrast and log-intensity mixed output of the event sensor via the same token-based high-speed communication system used in the temporal contrast event mode. Each temporal contrast event then contains the pixel address obtained from the row address encoder and column address encoder, the sign of the optional temporal contrast, the current timestamp (which may be shared by all temporal contrast events from the same frame), and the offset-compensated pixel value and gain-corrected pixel value reflecting the log-intensity measurement of the corresponding pixel. The readout processor then continues to read and process the next row of frames. After the readout processor completes reading and processing an entire frame, the event sensor can repeat the above steps on the next frame.

[0121] Other temporal contrast and log intensity blend mode variants

[0122] Because processing options are pixel-specific, a subset of pixels can have processing options configured with one option, while another subset of pixels can have processing options configured with a different option. Figures 8a to 8c Discussing this approach in more detail, Figures 8a to 8c The pixel array is schematically shown as a rectangular area divided into various subsets of pixels.

[0123] exist Figure 8a In the first example shown, the processing options for a first subset of pixels 81 at the center of the pixel array can be configured as (3) logarithmic intensity measurement, while the processing options for a second subset of pixels 82 corresponding to the remaining pixels in the pixel array are configured as (2) temporal contrast detection and measurement.

[0124] exist Figure 8b In the second example shown, the processing options for a third subset of pixels 83 interspersed in the pixel array may be configured as (3) logarithmic intensity measurement, while the processing options for a fourth subset 84 corresponding to the remaining pixels in the pixel array may be configured as (2) temporal contrast detection and measurement.

[0125] exist Figure 8c In the third example shown, the processing options for the fifth subset of pixels 85 located at the center of the pixel array can be configured as (3) logarithmic intensity measurement, while the processing options for the sixth subset of pixels 86 scattered in the pixel array are configured as (4) temporal contrast detection and logarithmic intensity measurement, and the processing options for the seventh subset 87 corresponding to the remaining pixels in the pixel array are configured as (2) temporal contrast detection and measurement.

[0126] Using different combinations of pixel subsets and their processing options, the processing options for the pixel array can be configured in many different ways. In various such time-contrast and log-intensity hybrid modes, the event sensor output is preferably supported by the same token-based high-speed communication system used in the time-contrast event mode. The processing options for the pixel array are configured by an external computer. Thus, the external computer can decode the event sensor output in various such hybrid modes based on the known processing option configuration for the pixel array.

[0127] Calibration Mode

[0128] Pixel-specific change detection thresholds, gain correction factors, and offset compensation values ​​can be obtained through a one-time calibration procedure. In calibration mode, all pixel processing options are configured as calibration options. Therefore, the readout processor transmits the current pixel value as the calibration mode output of the event sensor without any processing, using the same communication scheme used in logarithmic intensity frame mode.

[0129] To obtain pixel-specific change detection thresholds and gain correction factors, all pixels are exposed to a uniform light source (once at a low illumination level and once at a high illumination level). The event sensor captures a first frame at a low illumination level, referred to as a low frame, and outputs a first frame of current pixel values, referred to as low pixel values. The event sensor also captures a second frame at a high illumination level, referred to as a high frame, and outputs a second frame of current pixel values, referred to as high pixel values. These two steps are repeated multiple times (e.g., 10, 100, or more times) in no particular time sequence. As a result, the event sensor generates multiple low pixel values ​​(e.g., 10, 100, or more low pixel values) and multiple high pixel values ​​(e.g., 10, 100, or more high pixel values) for each pixel.

[0130] To minimize the effect of temporal noise in the event sensor, the temporal average low pixel value and the temporal average high pixel value are calculated for each pixel:

[0131]

[0132]

[0133] Then, the pixel value difference is calculated for each pixel as:

[0134] Pixel value difference (of pixels) = time average high pixel value (of pixels) - time average low pixel value (of pixels)

[0135] And the average pixel value difference of the array is calculated as:

[0136]

[0137] Therefore, the gain correction coefficient for each pixel can be calculated as:

[0138]

[0139] The change detection threshold for each pixel can be calculated as:

[0140]

[0141] For example, if the low illumination level is 0.1 lux, the high illumination level is 100k lux, and the array average pixel value difference is 2000, then the gain correction factor for a pixel with a pixel value difference of 1800 should be approximately 1.1. If the target contrast sensitivity threshold is 15%, then the change detection threshold for this pixel should be approximately 18.

[0142] To further obtain a pixel-specific offset compensator, the event sensor captures a third frame in darkness, referred to as a dark frame, and outputs a third frame of current pixel values, referred to as dark pixel values. This step is repeated multiple times (e.g., 10, 100, or more times). As a result, the event sensor generates multiple dark pixel values ​​for each pixel (e.g., 10, 100, or more dark pixel values).

[0143] To minimize the effect of temporal noise in the event sensor, the temporal average dark pixel value is calculated for each pixel as its offset compensator:

[0144]

[0145] It is worth noting that because dark pixel values ​​vary with temperature and may also vary with the bias settings of the pixel front-end electronic signal converter (here, the current-to-voltage converter), the offset compensation value calibration results obtained at a specific operating temperature and using a specific pixel bias setting are most valid when used at approximately the same operating temperature and using approximately the same pixel bias setting. Therefore, it is recommended to calibrate the event sensor of the offset compensator at the most expected operating temperature (or temperatures) and using the most likely pixel bias setting (or temperatures).

[0146] During the calibration procedure, pixel-specific change detection thresholds, gain correction factors, and offset compensation values ​​are calculated by an external computer, then written to external non-volatile memory and loaded into the pixel parameter memory via the pixel parameter configurator at power-up for subsequent event sensor operations.

[0147] Pixel parameter adjustments during operation

[0148] During event sensor operation, i.e., during all other operating modes of the event sensor except for its calibration mode, there are cases where some pixel parameters need to be adjusted by an external computer. Here are a few examples:

[0149] To switch the operating mode of the event sensor, the processing options of the pixel array need to be reconfigured. The new processing option configuration is determined by an external computer and written to the pixel parameter memory and external non-volatile memory through the pixel parameter configurator.

[0150] If the pixel bias setting and / or operating temperature change while the event sensor is operating in log intensity frame mode or temporal contrast and log intensity mixed mode, a new set of offset compensation values ​​for the pixel array (corresponding to the new pixel bias setting and / or new operating temperature) is required in order for the event sensor to achieve optimal output accuracy. Preferably, multiple sets of offset compensation values ​​for the pixel array are obtained by calibration under several expected pixel bias settings and operating temperature ranges and stored in an external non-volatile memory. The settings corresponding to the new bias setting and / or new operating temperature range are selected by an external computer and written to the pixel parameter memory by a pixel parameter configurator.

[0151] When the event sensor is operating in time contrast event mode or mixed time contrast and log intensity mode, it is sometimes necessary to adjust the contrast sensitivity threshold. To do this, the pixel-specific change detection threshold in the pixel parameter memory is updated via the pixel parameter configurator. The new change detection threshold for each pixel is calculated as:

[0152]

[0153] For example, if the old contrast sensitivity threshold was 15%, the old change detection threshold for a pixel was 18, and the new contrast sensitivity threshold to be achieved is 30%, then the new change detection threshold for this pixel should be 34.

[0154] During pixel-specific change detection threshold adjustment, the pixel parameter configurator sends the old change detection threshold for each pixel to the external computer. For each old change detection threshold, the external computer calculates a corresponding new change detection threshold based on the old contrast sensitivity threshold target and the new contrast sensitivity threshold target. The new change detection threshold for each pixel is then written back to the pixel parameter memory and external non-volatile memory via the pixel parameter configurator.

[0155] Reference numerals:

[0156] 10-pixel array

[0157] 1 Photoelectric converter, photodiode, PPD 2 Electronic converter 21 Electronic signal converter, current-voltage converter 23 Analog-to-digital converter

[0158] 24 Comparators

[0159] 25 Converter Memory

[0160] 26 Inverting Amplifier

[0161] 27 Buffer

[0162] 3 pixel parameter memories, which have

[0163] 31-bit digital memory

[0164] 4 readout processors, which have

[0165] 41 Processing Block

[0166] 51 Bias Generator

[0167] 52 ADC controller

[0168] 53-line address encoder

[0169] 54 column address encoder

[0170] 55 Readout Controller

[0171] 56 Pixel Parameter Configurator

[0172] 61 External computer 62 External non-volatile memory

Claims

1. An event sensor comprising a pixel array (10) and configured to generate a signal stream comprising event data responsive to light incident on the pixel array (10), the event sensor comprising: a photoelectric converter (1) for each pixel of the pixel array (10) and an electronic converter (2) connected to the photoelectric converter (1), wherein the electronic converter (2) is configured to generate and store a digital current pixel value, the digital current pixel value being dependent on the intensity of light incident on the photoelectric converter (1); a respective multi-bit digital memory (31) for each pixel of the pixel array (10), configured to store respective pixel parameters including a previous pixel value; and a readout processor (4) comprising a plurality of processing blocks connected to the electronic converter (2) and the multi-bit digital memory (31), configured to generate a pixel event value of the event data based on a pixel subtraction result of subtracting the previous pixel value from the current pixel value of the digital data, The event sensor is configured to support a time contrast event mode, wherein the operation of the time contrast event mode begins by performing pixel-parallel conversion of analog voltages related to light intensity into digital current pixel values, and storing the digital current pixel values ​​in a converter memory of an electronic converter (2) in a synchronized global shutter manner, and then a readout processor (4) reads the digital current pixel values ​​from the converter memory row by row, and at the same time, the readout processor (4) also reads corresponding pixel parameters of pixels in the same row from a multi-bit digital memory (31), The information transmission between the converter memory, the multi-bit digital memory (31) and the readout processor (4) is coordinated by the readout controller. Each processing block is configured to perform the following processing steps to determine whether to output a temporal contrast event from a corresponding pixel: Receive the current pixel value of the number of the same pixel and the corresponding pixel parameters; Calculate the signed difference between the current pixel value and the previous pixel value of the number; determining whether the sign difference exceeds the positive or negative change detection threshold for the corresponding pixel, If so, a temporal contrast event output is requested and corresponding pixel parameters are updated, said updating of the corresponding pixel parameters including overwriting the previous pixel value with the digital current pixel value; if not, no such operation is performed.

2. The event sensor according to claim 1, wherein: The readout processor (4) is configured to overwrite a stored previous pixel value with the digital current pixel value each time a pixel event value has been generated.

3. The event sensor according to claim 1, wherein: The electronic converter (2) comprises an analog-to-digital converter (23) and is configured to generate a digital current pixel value, wherein the digital current pixel value is logarithmically related to the intensity of light incident on the photoelectric converter (1).

4. The event sensor according to claim 3, wherein: The electronic converter (2) comprises a current-voltage converter (21), wherein the current-voltage converter (21) is a logarithmic converter and the analog-to-digital converter (23) is a linear converter, or the current-voltage converter (21) is a linear converter and the analog-to-digital converter (23) is a logarithmic converter.

5. The event sensor according to claim 1, wherein The pixel array (10) is composed of a width of pixel columns and a height of pixel rows, and the pixel parameter memory is composed of a plurality of multi-bit digital memories (31) for each pixel in the width multiplied by the height. wherein the readout processor (4) comprises a number of processing blocks (41) of the width, each of the processing blocks (41) being configured to process one of the pixel columns, or wherein the readout processor (4) comprises an integer number of processing blocks (41) of the width, each of the processing blocks (41) being configured to process a subset of one of the pixel columns, or The readout processor (4) includes processing blocks (41) whose number of widths is divided by an integer, and each of the processing blocks (41) is configured to process a plurality of the pixel columns.

6. The event sensor according to claim 1, wherein The multi-bit digital memory (31) is configured to further store a change detection threshold and / or a past event timestamp of the corresponding pixel, wherein the readout processor (4) is configured to generate and / or output the pixel event value when the current pixel value of the digital pixel differs from the previous pixel value by more than the change detection threshold and / or when the past event timestamp is earlier than a predetermined time interval.

7. The event sensor according to claim 1, wherein: The readout processor (4) is configured such that generating the pixel event value is one of two or more processing options determined by a processing option parameter.

8. The event sensor according to claim 7, characterized in that The readout processor (4) is configured to process a first pixel or all pixels in a first group of pixels of the pixel array (10) according to a first processing option, and to process a second pixel or all pixels in a second group of pixels of the pixel array (10) according to a second processing option.

9. The event sensor according to claim 7, wherein: The multi-bit digital memory (31) is configured to further store processing option parameters corresponding to the pixel, wherein the readout processor (4) is configured to process the current pixel value of the digital according to the processing option parameters corresponding to the pixel.

10. The event sensor according to any one of claims 7 to 9, characterized in that The readout processor (4) is configured such that the two or more processing options include generating a logarithmic intensity at the pixel, the logarithmic intensity being a value that is logarithmically related to the intensity of light incident on the photoelectric converter (1).

11. The event sensor according to any one of claims 7 to 9, characterized in that The readout processor (4) is configured such that the two or more processing options include a calibration mode in which the digital current pixel value is output by the readout processor (4) without processing.

12. The event sensor according to any one of claims 7 to 9, characterized in that The readout processor (4) is configured so that the two or more processing options include simultaneously generating a logarithmic intensity at the pixel and a pixel event value at the pixel based on a pixel subtraction result of subtracting the previous pixel value from the digital current pixel value, wherein the logarithmic intensity is a value that is logarithmically related to the intensity of light incident on the photoelectric converter (1).

13. The event sensor according to claim 10, wherein: The multi-bit digital memory (31) is configured to further store an offset compensation value, wherein the readout processor (4) is configured to take the offset compensation value into account when generating the logarithmic intensity.

14. The event sensor according to claim 12, wherein: The multi-bit digital memory (31) is configured to further store a gain correction factor, wherein the readout processor (4) is configured to take the gain correction factor into account when generating the pixel event value and / or the logarithmic intensity.

15. The event sensor according to claim 1, wherein The photoelectric converter (1) and the electronic converter (2) cooperatively occupy a common physical area or volume in the pixel array (10).

16. The event sensor according to claim 1, wherein The readout processor (4) is configured to process pixels of one or more rows or a portion of a row of the pixel array (10) in parallel.

17. A method for generating a signal stream comprising event data responsive to light incident on a pixel array (10), comprising the steps of: An electronic converter (2) connected to the photoelectric converter (1) generates and stores a digital current pixel value, the digital current pixel value depending on the intensity of light incident on the photoelectric converter (1) of each pixel of the pixel array (10); providing, for each pixel of the pixel array (10), corresponding pixel parameters including a previous pixel value stored in a multi-bit digital memory (31) corresponding to the pixel; generating a pixel event value for the event data based on a pixel subtraction result of subtracting the previous pixel value from the digital current pixel value by a readout processor (4), wherein the readout processor (4) includes a plurality of processing blocks connected to the electronic converter (2) and the multi-bit digital memory (31); and The analog-to-digital converter (23) of the electronic converter (2) converts the analog voltage related to the light intensity into a digital current pixel value in parallel, and stores the digital current pixel value in the converter memory of the electronic converter (2) in a synchronized global shutter manner. Then, the readout processor (4) reads the digital current pixel value from the converter memory row by row. At the same time, the readout processor (4) also reads the corresponding pixel parameters of the pixels in the same row from the multi-bit digital memory (31). The information transmission between the converter memory, the multi-bit digital memory (31) and the readout processor (4) is coordinated by the readout controller. The method further includes causing each processing block to perform the following processing steps to determine whether to output a temporal contrast event from a corresponding pixel: Receive the current pixel value of the number of the same pixel and the corresponding pixel parameters; Calculate the signed difference between the current pixel value and the previous pixel value of the number; determining whether the sign difference exceeds the positive or negative change detection threshold for the corresponding pixel, If so, a temporal contrast event output is requested and corresponding pixel parameters are updated, said updating of the corresponding pixel parameters including overwriting the previous pixel value with the digital current pixel value; if not, no such operation is performed.

Images