Sensitive dynamic vision sensor (DVS) and method for vision sensing

Abstract

Sensitive Dynamic Vision Sensor comprising multiple physical pixels, each comprising: a photoreceptor module configured to amplify and duplicate a voltage representing the detected light; a contrast computation module configured to reset the voltage and to launch events; a first and second pixel memory bits, connected to neighbor pixels to share its voltage; and a switch swr configured to disable the contrast computation module; being the physical pixels arranged as macro pixels, each comprising: a master pixel configured not to share voltage with other pixels and to launch events, and the rest of pixels configured to share its voltage with the master pixel, which launches an event when the combined voltage is over a LR-temporal contrast, and a controller configured to set the rest of pixels: in a LR-region, switch swr is activated, or in a HR-region, switch swr is deactivated, launching events when the voltage is over a HR-temporal contrast.

Description

[0001]SENSITIVE DYNAMIC VISION SENSOR (DVS) AND METHOD FOR VISION SENSING OBJECT OF THE INVENTIONThe invention is related to the field of vision sensing, more particularly to DVScameras.The object of the invention is a DVS sensor which allows to perform foveation bycomputing temporal contrast on pixels individually and on macro pixels. Also, allowsto compute different temporal contrasts for low-resolution and for high-resolution, simultaneously. Another object of the invention is a method for vision sensing which uses the DVS sensor defined. BACKGROUND ARTDynamic Vision Sensors (DVS) are a novel type of vision sensors that operatesimilarly to a biological retina. A biological retina does not produce sequences offrames or images. A biological retina is continuously and dynamically sending nervousspikes to the virtual cortex, transmitting a flow of spiking events that represent thedynamically changing visual scene. A cell (or pixel) in the retina only sends a spikewhen there is meaningful information to transmit, like a change of light. A DVS pixelwill send out its (x,y) coordinate when it detects a relative change of light above agiven “temporal contrast” level.Dynamic Vision Sensors (DVS) are radically different vision sensors with respect to imagers. In a DVS there are no images. Consequently, foveation techniques used in imagers cannot be directly used or mapped to DVS cameras. In a DVS, pixels whose light change by a relative amount, called temporal contrast, send out off chip their (x,y) coordinate. Output is sparse and represents the continuous dynamic scene change. On the other hand, imagers produce images every fixed time step. Each pixel is represented by its incident light or luminance. Imagers generate full images every time step.Fig. 1 represents the operation of conventional frame-based cameras. Every timeTframe a new frame or image is provided. Each image is generated by having eachpixel in the sensor integrate (accumulate) light (this is, the photo current delivered bya photo diode) during a given “exposure time”. This is typically done by usingcapacitors. Fig. 1 illustrates the images captured by such a camera observing arotating white disk with a black dot close to its border. If the disk rotates slowly, eachframe will capture a clear black dot at different positions. However, if the disk rotatesfaster so that during the camera’s “exposure time” the black dot has moved asignificant distance, the frames will capture the dots with a blurred tail.Fig. 2 illustrates a simplified concept of a conventional imager. First of all, an imagergenerates a full luminance image every given time interval called Tframe. One way ofdoing this is by using a pixel circuit similar to the one shown in Fig. 2(b). Each pixelcontains a capacitor Cinteg (physical or parasitic) that integrates photo current Iphbetween the instant it is reset by signal “reset” and the instant exposure stops byde-activating signal “exposure”. Depending on the amount of charge accumulatedonto capacitor Cinteg during this exposure interval, a given luminance dependencevoltage VC will be hold at capacitor Cinteg. This voltage is transformed into a highcurrent by activating row signal “read” and setting sufficiently low column voltage Vcol.This way, each pixel in a row provides an output current Iout that depends on thetemporarily hold voltage VC. Consequently, all VC voltages in a row can be read outin parallel. As explained, imagers read out full images every Tframe, thus producing aconstant and maximum flow of information representing plain pixel luminance,independently of scene dynamics.In a DVS, pixels do not accumulate light (or photo current) during an exposure time.Photo current is monitored continuously and when it has changed by a given relativeamount, called “temporal contrast”, the pixel will produce a new spike. This spike issent off-chip with sub- microsecond delay by giving the (x,y) coordinate of the pixelwithin the pixel array and sign bit that specifies whether light has increased ordecreased by a given percent.Foveation is the technique by which part of the visual field is captured at highresolution, while the rest is captured at low resolution.Foveation in imagers is done by combining pixels into macro-pixels and averaging the luminance of their individual pixels. For this, all pixels need to sense their individual luminance. For the selected fovea regions, which remain in high resolution (HR), every pixel keeps its individual luminance level, which is communicated off chip for this frame. For the rest of the image outside any fovea region, pixels are grouped in macro-pixels by averaging the luminance of its individual pixels, and this reduced information is communicated off chip for this frame. Consequently, in imagers, first a full high-resolution image is captured inside the chip, which is somehow processed to communicate off chip with different resolutions depending on the selected fovea regions.In foveated imagers, pixel circuit does not require any modification with respect to anon-foveated imager implementation. Instead of that, pixel grouping and averaging isperformed by a peripheral circuitry, and a reduced information is sent off-chip.In DVS camera, there are no images. No full high-resolution luminance image is captured and processed in the chip. In a DVS, pixels have to be combined within the pixel array to compute temporal contrast either on individual pixels or on combined macro-pixels. DESCRIPTION OF THE INVENTION The invention relates to a sensitive Dynamic Vision Sensor (sDVS) configured to generate events on detecting a relative amount of light variation. The sensitive DVS sensor of the invention allows to compute temporal contrast on grouped macro-pixels by interconnecting specific nodes of a specific photoreceptor circuit within a DVS pixel. The sensitive Dynamic Vision Sensor (sDVS) of the invention is configured to generate events when there is a pre-set relative amount of light variation. Each event comprises pixel coordinates and a polarity (ON, OFF). The sensitive Dynamic Vision Sensor (sDVS) of the invention also allows to define fovea regions, regions with high-resolution, and regions with low-resolution. The sensitive Dynamic Vision Sensor (sDVS) of the invention comprises a plurality of physical pixels, and each physical pixel comprises: -a photoreceptor module (p) comprising:o a photoreceptor stage comprising a transistor ^^^ and configured totransform a received light into a voltage ^^^; oa trans-impedance preamplification stage connected to thephotoreceptor and comprising one or more stacked diodes configured to amplify the received voltage, and one or more transistors configured to provide the voltage ^^^to the stacked diodes and to duplicate said voltage ^^^; -a contrast computation module (c) comprising:o a buffer stage connected to the trans-impedance preamplificationstage; oan amplification stage connected to the buffer stage, comprising twocapacitors (C1 and C2) and configured to amplify the received voltage by a relationship C1 / C2 between the capacitance value of the two capacitors and to reset the voltage to VREF when the amplified voltage Vdiff reaches an upper temporal contrast or a lower temporal contrast; and oa differentiation stage connected to the amplification stage andconfigured to launch an ON event when the voltage Vdiff reaches the upper temporal contrast and an OFF event when the voltage Vdiff reaches a lower temporal contrast; In this case, a LR upper temporal contrast and a LR lower temporal contrast are defined for LR regions and a HR upper temporal contrast and a HR lower temporal contrast are defined for HR regions. In the sensitive Dynamic Vision Sensor (sDVS) of the invention the physical pixels are arranged as macro pixels. For adding control elements to the pixels, each pixel also comprises: -a first pixel memory bit, connected to a first neighbor pixel and configured toreceive and share the duplicated pre-amplified voltage from the transistor of the trans-impedance preamplification stage; -a second pixel memory bit connected to a second neighbor pixel andconfigured receive and share the duplicated pre-amplified voltage from the transistor of the trans-impedance preamplification stage; and -a switch swr configured to reset continuously the capacitor C2 of the contrastcomputation module, disabling said contrast computation module for not launching events; Also, in each macro pixel it is defined: -a master pixel configured to not share information with other pixels bydeactivating the first and second pixel memory bits and configured to launch events by deactivating the switch swr, and -the rest of pixels.The rest of pixels in the macro pixel are configured to share its pre-amplified voltage from the trans-impedance preamplification stage with the master pixel by activating the first and / or second pixel memory bits such that the contrast computation module of the master pixel receives a summed voltage from the rest of pixels of the macropixel and when the summed voltage reaches the LR upper temporal contrast or theLR lower temporal contrast an event is launched. Thus, the rest of pixels share its pre-amplified voltage between them, adding in each step the voltage detected by its photoreceptor stage. By sharing the rest of pixels its pre-amplified voltage between them the pixels close to the master pixel would provide to said master pixel the summed voltage from all the rest of pixels of the macro pixel. The sensitive Dynamic Vision Sensor (sDVS) of the invention further comprises a controller configured to define the state of one or more of the rest of pixels to be: -in a LR region, wherein the switch to reset is activated, disabling the contrastcomputation module for not launching events; or -in a HR region, wherein the switch to reset is deactivated, enabling thecontrast computation module to launch events when the voltage received from the trans-impedance preamplification stage reaches the HR upper temporal contrast or the HR lower temporal contrast. The temporal contrast for LR regions and the temporal contrast for HR regions could be set to be the same.The photoreceptor stage of the pixels also comprises an inverting voltage amplifier.Also, the one or more transistors of the trans-impedance preamplification stage aretwo transistors (Mlocaland Mshare) connected to the first and second pixel memory bits and to the contrast computation module (c) and configured to duplicate the voltageVph. Preferably, said transistors of the trans-impedance preamplification stageconform a Serrano current mirror In such a case, the sensitive Dynamic Vision Sensor (sDVS) of the invention couldalso comprise a switch swla and a switch swlb connected to the transistor (Mlocal) andconfigured to activate or deactivate the first and second pixel memory bits. The sensitive DVS sensor of the invention could further comprising a switch comml connected to the switch swr and configured to activate or deactivate the contrast computation module (c). Preferably, the photoreceptor converts the incident radiation firstly into a continuousphoto-current (^^^(^)) and then to a logarithmic voltage representation as:The sensitive DVS sensor of the invention could also comprise an asynchronousdigital communication circuit connected to an external circuitry and configured to sendpixel events out of the sensitive DVS sensor using an Address Event Representation(AER) format.Moreover, the sensitive DVS sensor could further comprise a Row Arbiter and a RowBuffer, connected to the asynchronous digital communication circuit of the pixels, and an external communication module, connected to the Row Arbiter and the Row Buffer.In said case, the pixel event is sent by:- when at least one event is launched in a pixel, sending a row-wise Rqst signalto the Row Arbiter, which is configured to determine the y-coordinate of saidrow and to reply by activating an Ack signal,- sending all active pixels in the row a polarity signal (p+ or p-) to the RowBuffer, which is configured to store x-coordinates, and -sending the x-coordinates, the polarity and the y-coordinate to the externalcommunication module, which is configured to send them out of the sensitive DVS sensor.The sensitive DVS sensor of the invention is configured to send out address eventswith micro to milli second delays.Moreover, the sensitive DVS sensor of the invention could further comprise aClassifier module and two output channels, a LR channel, providing absolutecoordinates of LR macro-pixels (x, y, s), and a HR channel, providing relativecoordinates within the HR region (x-xL, y-yL, s). In this case, the Classifier modulecomprises HR region limit coordinates and is configured to determine if an Address-Event (x,y,s) is in the HR region or LR region. In some implementations, the two output channels could be merged and, in this case,the Address-Event (x,y,s) further comprises a parameter ‘r’ to distinguish whether theevent belongs to the LR region or HR region.The invention also relates to a method for vision sensing which comprises the stepsof: -sensing light and converting it to voltage Vph by using thephotoreceptor stage; -pre-amplifying the received voltage by using the trans-impedancepreamplification stage; -duplicating the pre-amplified voltage by using the trans-impedancepreamplification stage; -if the pixel is not a master pixel, the switch swla is OFF and the switch swlb isON, sending one of the duplicated voltages to the first and second pixel memory bits; -if the pixel is a master pixel, the switch swla is ON and the switch swlb is OFF,receiving a signal from the first and second pixel memory bits of one ormore neighbor pixels generating a combined voltage, representing the sum of photocurrents from the rest of pixels of the macro pixel;- if the pixel is not a master pixel, amplifying the other duplicated voltageby using the one or more stacked diodes of the trans-impedance preamplification stage; -if the pixel is a master pixel, amplifying the combined voltage from the restof pixels of the macro pixel; -sending the amplified voltage to the buffer stage if the pixel is a masterpixel, the switch swris set permanently ON, or if the pixel is not a master pixel and switch swris ON, otherwise, if the switch swris OFF, resetting the amplified voltage; -sending the amplified voltage to the amplification stage of the contrastcomputation module (c); -amplifying the voltage by ^^ / ^^;- resetting the voltage to ^^^^ whenever the voltage Vdiff reaches an upperthreshold V+or a lower threshold V-; and -launching a sending of event by the differentiation stage: an ON eventwhen the voltage Vdiff reaches an upper threshold V+and an OFF event when the voltage Vdiff reaches a lower threshold V-;The method for vision sensing defined could comprise two versions of the global biassignals VREF: (^^^^^^ , ^ ^^^^ , ^ ^^^^ ) and (^^^^^^ , ^ ^^^^ , ^ ^^^^ ). Thus, if the signal comingout of the first or second pixel memory bits (T OR R) is high, the HR version biases(^^^^^^ , ^ ^^^^ , ^ ^^^^ ) is selected for the comparators in the contrast computation circuitand, if the signal coming out of the first or second pixel memory bits (T OR R) is low,the LR version biases (^ ^^^^^^ , ^^^^ , ^^^^^) is selected. The method of the invention allows to compute different temporal contrast levels for the overall low-resolution scene and the selected high-resolution fovea areas. Additionally, the disclosed method allows to simultaneously compute the low- resolution temporal contrast of the full low-resolution visual scene and the temporal contrast of selected high-resolution fovea regions. DESCRIPTION OF THE DRAWINGSFigure 1 shows an illustration of conventional video frame-based camera capturing arotating disk with a dot, wherein each frame needs an exposure time, such that when disk rotation is slow, the frames capture a still dot, and when the dot changes position during the exposure time, blurring appears in the captured frames.Figure 2 shows a (a) generic imager pixel array and periphery, (b) a possible physicalpixel diagram of a generic imager.Figure 3 illustrates events generation by an individual pixel, such that every time thevoltage varies over a pre-defined threshold, a spike (or event) is generated. If thevariation is upwards, an ON event is produced, otherwise an OFF event is produced.Figure 4 shows a slow and fast motion of rotating black dot. Black filled points in (x,y,t)represent OFF events in the corresponding pixels, and white filled points in (x,y,t) represent ON events in the corresponding pixels. In the fast motion case, the sameevents are produced but at a faster speed without loss of information nor blurring.Figure 5 shows a block diagram of a DVS pixel.Figure 6 shows an example of two types of DVS photoreceptor circuit.Figure 7 shows an example of a change detector circuit (CD).Figure 8 shows an example of the operation of a CD circuit.Figure 9 shows an example of overall physical architecture of a DVS sensor.Figure 10 shows an example of a sensitive DVS pixel circuit, with a stacked-diodespre-amplification stage.Figure 11 shows a (a) DVS pixel array in Low-Resolution (LR) mode, where physicalpixels are grouped into macro pixels, each macro-pixel comprising 3x3 physical pixels. (b) When configured as LR macro-pixel, the ‘p’ part of all physical pixels is combined, and all ‘c’ parts are disabled, except one of them (in this example, the top right one), which computes the contrast of the full LR macro-pixel. (c) When configuring a region in High-Resolution (HR), the macro-pixels in this region are un- grouped and each physical pixel uses its own ‘c’ part to compute contrast of its own ‘p’ part.Fig. 12 shows a modified pixel circuit to allow interconnection to neighboring topand / or right pixel for macro-pixel grouping, while disabling the contrast computation part ‘c’.Figure 13 shows an illustration of peripheral control modules to handle AER eventssent from the pixels.Figure 14 shows a modified pixel according to the invention which allowssimultaneous event generation of LR and HR events within foveas.Figure 15 shows an illustration of the motion of a fovea in an arrangement of pixels.Figure 16 shows a block diagram of a pixel according to the present invention. PREFERRED EMBODIMENTS OF THE INVENTIONThe present invention relates to a DVS sensor configured to send events when thereis a variation in light intensity greater than a threshold. The DVS sensor of theinvention allows to simultaneously compute a low-resolution temporal contrast of thefull low-resolution visual scene and a high-resolution temporal contrast of selectedhigh-resolution fovea regions. As explained before, the DVS sensor of the invention is not an imager, thus, the term“imager” in the context of the invention is not used for referring to a DVS. Also, theoutput produced by a DVS sensor is not an “image”.A typical DVS pixel is composed of:1) photoreceptor stage that converts continuously incident light into a continuousphoto current Iph(t) which is then converted into a continuous logarithmicvoltage representation ^^^(^) = ^^^^^(^^^(^)).2) a change detector stage that generates an output event when haschanged by a given amount |Vθ|. If the event has been produced by anincrease in photo current, it is an ON event. If it has been produced by adecrease in photo current, it is an OFF event.3) And an asynchronous digital communication circuit that interacts with circuitryat the periphery of the pixel array to send pixel events out of the chip usingan Address Event Representation (AER) format. This way, the output of aDVS camera is a flow of address events (x,y,p) with the pixel coordinate andpolarity (ON or OFF) of the pixels that have noticed a change of light abovea temporal contrast (LR upper temporal contrast, LR lower temporal contrast,HR upper temporal contrast or HR lower temporal contrast) since a previousevent. Address events are sent off chip with micro to milli second delays sincethey were detected by the pixel, depending on ambient light.Fig. 3 illustrates this concept for one single DVS pixel. A conventional frame-basedcamera would sample each pixel every time Tframe and accumulate the photo currentduring an “exposure time”. If the light on the pixel stays constant, the pixel woulddeliver the same information every Tframe, consuming communication energycontinuously, and making subsequent image processing systems consume energy forevery frame. However, a DVS camera only sends off-chip information of the changes.The pixel in Fig. 3 sends out a positive event when with VA, Vθ > 0, or sends out a negative event when where C+ and C - are a positive and negative temporal contrast, respectively.Fig.4 illustrates the events captured by a DVS sensor observing a rotating white diskwith a black dot close to its border. The DVS sensor produces a flow of output signedaddress events (x,y,p) with sub-microsecond time resolution. In Fig. 3 a negativeevent is represented by a filled black dot in (x,y,t) position. This means that pixel (x,y)experienced the transition from the white background to the black rotating dot at timet. Similarly, an unfilled dot in Fig. 4 indicates that pixel (x,y) experienced thetransition from the dark dot to the white background at time t.Therefore, DVS sensors only send information for the pixels whose luminance haschanged beyond a given temporal contrast. This way, a DVS sensor provides justthe information of moving edges and textures, thus significantly reducing informationflow, as well as computational burden on subsequent vision processing andcomputing stages. Furthermore, output address events become available at sub-microsecond delays, opening new avenues for very high-speed dynamic sceneanalyses. On the contrary, imagers need to wait a time Tframe to first expose all pixelsa given exposure time and to second transmit the full frame off-chip.As explained, DVS concept is radically different from said conventional imagers. ADVS sensor includes fairly complex pixels, as opposed to conventional imagers. ADVS pixel consist typically of three parts, as shown in Fig. 5, photoreceptor (Ph R),change detector (CD), and digital communication (Dig. comm.) circuits:1. A photoreceptor module (Ph R) that transforms instantaneous and continuouslight (or photo current of its photo diode) Iph(t) into a continuous voltageproportional to its logarithm: Some photoreceptor circuits used in DVS pixels are shown in Fig.6. Fig.6(a)shows an example of photoreceptor circuit, while Fig. 6(b) shows a sensitiveDVS (sDVS) photoreceptor circuit.The sDVS (sensitive-DVS) is a variation of the DVS pixel, which adds anintermediate pre-amplification stage based on a stacked-diode trans- impedance amplification, which introduced negligible-mismatch amplification, yielding to a 10x contrast sensitivity improvement, while requiring smaller pixel area. 2. A change-detector (CD) circuit, that activates a digital “ON” or “OFF” signalwhenever voltage Vph has increased or decreased by Vθ, respectively, asillustrated in Fig. 3. A typical change-detector (CD) circuit is shown in Fig. 7,using idealized differential amplifiers. In practice, these differential amplifiersare implemented with highly simplified circuits.The operation of the change-detector (CD) circuit is illustrated in Fig. 8. Theinput signal Vph, shown in Fig.3, is amplified and inverted by a gain ^ ^= −^^. However, this amplified version Vdiff is reset to VREF whenever it reaches eitherVREF-ACVθ or VREF+ACVθ, activating either signals ON or OFF, respectively,which triggers the Reset signal, discharging capacitor C2. This way, when Vphis increasing (-ACVph is decreasing), ON events are generated whenever Vphincreases by an amount Vθ, or OFF events are generated whenever Vphdecreases by an amount Vθ.3. The asynchronous communication digital circuit is known in the art. Basically,it is in charge of communicating with a peripheral row arbiter, shown in Fig.9. The asynchronous communication digital circuit sends a row-wise Rqstsignal to the row arbiter whenever the ON or OFF comparators are activated.Thus, the row arbiter knows the y-coordinate of the Rgst signal. Then, the arbiterreplies by activating a corresponding Ack signal. When the Ack signal isreceived, all pixels that are active in the row will send up to the Row Buffereither a p+ or p- active signal, as shown in Fig. 9, depending on whethercomparator ON or OFF has been activated.The p+ or p- active signals are latched in the buffer, storing the x-coordinates withthe polarity p of the active pixels. These signed x-coordinates, together with the arbitery-row-coordinate will be sequenced off-chip by a circuit “Ext. com.The DVS sensor of the invention is also configured to perform a foveation technique.This technique allows to capture part of the visual field at high resolution, while therest of the visual field is captured at low resolution. For example, human eyes havethis property, which allows to significantly reduce information flow and dataprocessing requirements within the visual cortex.In a DVS camera the sensing process is radically different from frame-based imagers.There are never images nor frames. Consequently, it is not possible to average theluminance information of neighboring pixels within a macro-pixel. Furthermore, in aDVS camera, the output information is not the luminance, but the relative change ofluminance. Thus, for generating LR macro-pixels, it would be necessary to computethe relative luminance change for the light impinging in all HR pixels with the LRmacro-pixel.One possible solution could be to externally collect the address events whoseaddresses fall within a macro-pixel. However, this would not reduce the eventcommunication flow with respect to a full HR version. Additionally, accumulating therelative change spikes of all individual HR pixels, does not represent the relativeluminance change of one equivalent macro-pixel. For example, in a case wherein allHR pixels within a LR macro-pixel are subject to exactly the same change ofluminance, each HR pixel sends out n events. In this case, the full macro-pixel isalso subject to the same relative change of luminance and thus should also send outonly n events for the full macro-pixel. Furthermore, this solution will not allow to usedifferent temporal contrast settings for the HR individual pixels than for the LR macro-pixels.One solution to perform pixel grouping in a DVS camera is to short-circuit the outputof all the HR pixels photodiodes within a macro-pixel, and use the rest ofphotoreceptor, change detector, and asynchronous digital communication circuitryof only one master HR pixel within the macro-pixel. This way, the master pixel wouldcompute relative changes of the sum of photo diode currents of all HR pixelswithin the macro-pixel. However, directly shorting neighboring photo diodes hasseveral problems. On one hand, DVS pixels are large, which would lead to long linesinterconnecting the photo diodes, introducing significant parasitics for the photocurrents, which can be as small as femto amperes, thus penalizing speed responseof photoreceptor circuits. On the other hand, adding switches interconnectingneighboring photo diodes will add important parasitic leakage currents, which maybecome comparable to the actual photo diode currents, specially under low lightconditions.Another possibility could be, for example, to add the voltages ^^^(^) = ^^^^^(^^^(^))of neighboring pixels. This could be done, for example, by shorting node Vx in Fig.7among neighboring pixels. However, this would compute ∑^ log (^^^^) = log (∏^ ^^^^ )and not log (∑^ ^^^^) , which is what is needed for a macro-pixel. Additionally, this wouldintroduce important parasitic leakage currents to the highly leakage-sensitive node Vx. Another possibility could be to add the voltages in a sensitive DVS sensor. In this case, the output voltage of the photoreceptor module is amplified by a mismatch-lessfactor N as ^^^(^) = ^^^^^^(^^^(^)), being N the number of stacked diode-connected NMOS transistors. By shorting node Vph(in Fig. 6b) among neighboring photoreceptors, the resulting voltage is given by In this way, it is possible to compute relative light changes on grouped pixels into onemacro-pixel with equivalent photo current . Furthermore, shorting node Vphamong neighboring pixels has the advantage that this node is driven by an amplified version of each pixel photo current AIIphi, where gain AItypically ranges over several orders of magnitude. This will compensate the impact of leakage currents and long line interconnection parasitic. Thus, for performing foveation in DVS sensors, the photoreceptor circuits ofneighboring pixels are interconnected instead of interconnecting directly thephotodiode nodes.Fig. 10 shows an exemplary embodiment of a pixel schematics comprising one 4-transistor stacked-diode trans-impedance pre-amplification stage. Thispreamplification stage multiplies by N, wherein N is number of stacked diodes, theprevious voltage amplification path gain, given by a capacitive ratio C1 / C2.Since the DVS contrast sensitivity is inversely proportional to this voltage path gain,such sensitivity is improved by the mismatch-less integer N. It is possible to alsocascade more of such pre-amplification stages to further increase the gain, whichallows to rely on smaller capacitor ratios C1 / C2, thus reducing pixel area.This trans-impedance-based preamplification stage in Fig. 10 has a differentphotoreceptor stage than the presented in Fig. 6(a).The DVS sensor comprises a pMOS transistor Mp2 for providing an input current tothe trans-impedance amplification stage. Transistor pair Mp1-Mp2, together with theinverting voltage amplifier biased by Ibo, constitute an active input current mirror(Serrano Current Mirror) whose current amplification factor AI is exponentiallycontrolled by a voltage difference VGA-VG, as shown in Fig. 10.This way, the very small photocurrent Iph sensed by the photodiode can be easilyamplified a few orders of magnitude before feeding the trans-impedancepreamplification stage. The current amplification factor AI suffers from inter-pixelmismatch, but this mismatch does not affect the DVS contrast computation.The operation of the pixel is as follows:Photodiode current Iph(t) is amplified by the Serrano current mirror to ^^ ^ ^^^(^) atthe output of Mp2. This current crosses the diode-connected NMOS transistor stack.The voltage drop at each diode-connected transistor is proportional to the log of thiscurrent plus some offset voltage. Consequently, preamplifier output voltage vo(t) isproportional to ^ ^ ^^^(^^ ^ ^^^(^)) plus some other offset voltage. Since the log ofproducts can be decomposed into sum of logs, vo(t) is proportional to ^ ^ ^^^(^^^(^)),and the AI dependence can be incorporated into the offset term.The buffer introduces a gain approximately equal to ‘1’, so that vob(t) is equal to^ ^ ^^^(^^^(^)) plus some other constant offset voltage. The amplification anddifferentiation (also called “change detector” CD) stage amplifies vob(t) by ^^ / ^^ andresets it to ^^^^ whenever digital signal ^^^^^^ is activated. This way, ^^^^^(^) isproportional to ^^ / ^^ ^ ^ ^ ^^^(^^^(^)) and is reset to ^^^^ whenever ^^^^^(^)reaches either an upper threshold ^^(controlled by ^^) or a lower one ^^(controlled by ^^). ^Consequently, two consecutive resets, one at ti-1 and the next at ti, satisfy ∆^^^^^ = the upper thresholdwas reached, or ∆^ ^ ^^^^^ = −|^ − ^^^^| = ∆^ if the lower threshold wasreached. In the former case, the pixel generates a positive ON event, while in thelatter a negative OFF one.Since ^^^^^(^) = ^^ / ^^ ^ ^ ^ ^^^(^^^(^)) plus some offset term where simplified notation ^^ = ^^^(^^) is used. Depending on whether ∆^^ / ^is positiveor negative, the equation can be rewritten as ^^+ and ^^are the positive and negative temporal contrast, respectively, which are controlled by ^^^ and ^ ^^ , respectively, as well as ^^^^. ^^+ and ^^can be tuned tobe equal, in which case both are a temporal contrast C of the DVS.The DVS pixel in Fig.10 can be separated into two parts:a) The part comprised by the “photoreceptor” and “preamplification” stages inFig. 11, called ‘p’, andb) the part comprised by the “buffer”, the “amplification and differentiation”,and “comparator” stages in Fig. 11, which conforms the contrastcomputation part, called ‘c’.Fig. 11 illustrates conceptually how these ‘c’ and ‘p’ parts are configured to groupDVS pixels into macro-pixels for operation in Low-Resolution (LR), and how they areconfigured for standard High-Resolution (HR).By default, the full array is set to LR and physical pixels are grouped into macro-pixels.When grouped into macro-pixels (see Fig.11b) the ‘p’ parts of all physical pixels arecombined, so that vo of the macro-pixel (let’s call it vog) is proportional to^ ^ ^^^(∑ ^^^^(^)), while all ‘c’ parts are disable, except one.This single ‘c’ part will receive the combined vog of the macro-pixel and computethe contrast on it.Fig. 12 shows a modified pixel version (let’s call it EF-pixel, where EF stands for‘Electronic Foveation’) with respect to the one in Fig.10, that includes disabling of the‘c’ part and sharing node vo with top and / or right neighbors. For this, the pixel includestwo memory bits: ‘T’ (connect to top pixel if set) and ‘R’ (connect to right pixel if set).If either of the two memory bits is set to 1, the ‘c’ part is disabled by constantlyresetting capacitor C2 and thus constantly freezing the contrast computation part. Forthe macro-pixel in Fig. 11b the top-right pixel would be set to T=R=0 and its ‘c’ partwill handle the contrast computation of the macro-pixel. For the other pixels in thetop row, their setting would be T=0, R=1. For the other pixels in the right column,T=1, R=0. And for the rest T=R=1.As soon as a macro-pixel is selected to be part of a foveated region (or ‘Region ofInterest’, ROI) all their pixels are set quickly to High-Resolution (HR) mode by settingT=R=0 for all of them. Since the fovea region will be, in general, moving over thevisual field, the DVS sensor of the invention could comprise a controller that keepssetting and unsetting the ‘T’ and ‘R’ bits of the pixels within the macro-pixels that goin and out of foveas.A possible option is illustrated in Fig. 13. The ‘Controller’ is in charge of setting / un-setting T / R switches as pixels are grouped / ungrouped as macro-pixels. The controllerreceives continuously updates of the fovea through coordinate parameter (xL, yL, xH,yH). The ‘AER Event Read-Out Periphery’ circuitry extracts the active pixels in thetraditional way a DVS would operate. For each pixel with an active ‘c’ part thatproduces an event, this circuitry would provide its absolute coordinate and eventpolarity as an Address-Event (x,y,s). This absolute-coordinate event is sent to a“Classifier” module, which also knows the presently active fovea limit coordinates, andsends out the event through two output channels:- The LR channel, providing the absolute coordinates of LR macro-pixels (x, y,s). -The HR channel, providing relative coordinates within the fovea (x-xL, y-yL, s).Optionally these two channels could be merged into a single one and add an extraparameter to the event ‘r’ to distinguish whether the event belongs to the LR full arrayor the fovea.For the case of simultaneously active multiple foveas, both the ‘Controller’ and‘Classifier’ need to be capable of handling multiple foveas, the Classifier would needto have several HR output channels or add the extra tag ‘r’ to distinguish among them,and the optional ‘Merger’ could be either removed, adapted, or have multiple of them.All this will depend on how the events will be sent off-chip and how many AER outputchannels are feasible.In the foveation method described so far, the pixels within a macro-pixel region(whether active or not) either send events individually (if in HR mode) or only the top-right pixel sends events (if in LR mode). This produces two flows of events:- One for the HR foveated regions- And one for the full LR array but excluding all HR regions.Thus, this implementation has the drawback that relative light changes can only becomputed on either grouped macro-pixels (LR), or un-grouped individual pixels (HR),but not on both simultaneously. Once a fovea HR region is established, the LR macro-pixel relative variation for that region is lost and not available anymore. This poses agreat challenge to the post-DVS computing system observing the LR visual scene andin charge of identifying and tracking regions of interest to be set in HR. This could bea potential problem for quickly and efficiently tracking foveas, once they are identifiedand activated.To overcome this potential problem, the EF-pixel circuitry could be modified asshown in Fig. 12. In Fig. 12, the LR macro-pixel is always sending out the LR eventinformation, whether or not it has been set for HR within a fovea. This implies that fora physical pixel, its photocurrent is duplicated.This way, if this pixel belongs to an active fovea, one of the replicas can be used forits local pixel-level HR contrast computation and the other replica can be used forsharing at the macro-pixel level and compute LR contrast. Fig. 14 shows anexemplary embodiment of this implementation. The original Serrano current-mirroroutput transistor Mp2 in Fig. 12 is duplicated into transistors Mshare and Mlocal. Thiswill add some mismatch, but this has no implications since, as explained, theSerrano current mirror output gain AI has no impact on the contrast computation.In the modified EF-pixel, bits ‘T’ and ‘R’ are set initially according to the desired Low-Resolution and are never changed (unless one wants to change the low resolutionof the overall full scene). This way, the amplified photocurrent replica provided bytransistor Mshare is always sent to the top and / or right pixel within the pre-definedmacro-pixels to always compute the LR contrast.In this implementation, the DVS sensor comprises an additional memory bit ‘comm’which changes dynamically depending on whether the macro-pixel enters / leaves afovea. The operation of the master pixel within a macro-pixel differs from the rest,resulting:a) Operation of pixels which are not the master pixel within a macro-pixelFor these pixels one or both of bits ‘T’ and ‘R’ are always set to ‘1’. Therefore, ‘swla’is OFF and ‘swlb’ is ON, so that the local pixel photocurrent signal (from Mlocal) isalways sent to the local contrast computing circuit. If the additional bit memorycomm=‘0’, then this pixel belongs to a macro-pixel that does not belong to any fovea,and thus should be silent. Therefore switch ‘swr’ is ON and the ‘c’ part is alwaysreset and cannot compute contrast nor send out events. On the other hand, ifcomm=‘1’ it means the pixel belongs to a fovea and should generate HR events.Therefore, ‘swr’ is OFF and the pixel is computing contrast using the local pixel-onlyphotocurrent coming from Mlocal.b) Operation of the master pixels within a macro-pixelFor these pixels it is always ‘T=R=0’, set initially. Also, for these pixels one has to setalways comm=‘1’, and leave it unchanged. Consequently, these pixels are never tobe updated by the peripheral circuitry. Under these conditions, ‘swla’ is always ON and‘swlb’ is always OFF. Consequently, the ‘c’ part computes contrast from the collectedphotocurrent of all pixels within its macro-pixel and the reset switch ‘swr’ is OFF.Unfortunately, master pixels do not participate in the HR mode of an active fovea,being silent for the HR event flow. This is however a mild problem, as the fovea hassufficient information from the majority of pixels. Additionally, in practice, the fovearegion is continuously moving and thus these silent pixels change place within thefovea, so that their transient inactivity reflects as a type of noise within the fovea.This is illustrated in Fig. 15, where the fovea region is moved by HR precision (notLR). On the other hand, if fovea regions move at LR, the silent HR pixels inside thefovea would stay at fixed positions always, as in Fig.15a.Optionally, for the EF-pixel in Fig.14, it is possible to revert back to the case in whichthe fovea regions do not send LR events, but all send HR events. There are two ways:a) In first way, one just needs to set one or both of the ‘T’ / ‘R’ bits of the top-right pixel(within a macro-pixel) to ‘1’. In this case, this master pixel would also use the localphotocurrent only coming out of Mlocal for contrast computation. However, now theMshare replica is routed to neighbor pixels vog nodes. In this case, one can route allMshare replicas within a fovea to the most master pixel of the full fovea and make onlythis one silent within the complete fovea by setting its com bit ‘0’. This way, allpixels within the fovea are sending now HR events, except for only the top rightpixel of the full fovea. In this approach, when the fovea moves, one needs to changethe settings of the macro-pixels master pixel that enter / leave a fovea, as well as thecom bit of all pixels that enter / leave a fovea.b) In a second way, one can set both of the ‘T’ / ‘R’ bits of all the pixels within a foveato ‘0’. In this case, all pixels in a fovea use only their local photocurrent coming out ofMlocal for contrast computation. Also, for all pixels within the fovea, their com bit is setto ‘1’. This way, all pixels within the fovea are sending now HR events. In thisapproach, one needs to change the settings for all pixels that enter / leave a fovea.In case the LR mode resolution can be decided and kept fixed before chip fabrication,it is possible to make a more compact pixel by hardwiring it. In this case, memory bits‘T’ and ‘R’ would not be required, as well as the switches they control. The layout ofthe physical pixels would vary slightly, depending on the required values for the hard-wired ‘T’ and ‘R’.When processing the full LR visual field or a selected HR region, it is very likelythey may require different contrast threshold settings.In this case, as shown in Fig. 14, the HR pixel array could have two versions of theglobal bias signals VREF, and (^^^^^^ , ^ ^^^^ , ^ ^^^^ ). T^^hen, the signalcoming out of the OR gate (T OR R), if high, should select the HR version biases(^^^^^^ , ^ ^^^^ , ^ ^^^^ ) for the comparators in the contrast computation circuit.Otherwise, the LR version biases (^^^^^^ , ^ ^^^^ , ^ ^^^^ ) should be used. These contrastselectors are always set and do not change when moving the fovea.The DVS sensor defined could be implemented as a full pixel circuit, as shown inFig. 16. In the top part are shown the three sub-circuits discussed so far: thephotoreceptor, the CD, and the asynchronous digital communication circuit. Thebottom part shows the memory block sub-circuit, which stores the local bit values Rl,Tl, and signal comml, which enables event communication for this pixel.Therefore, if it is disabled (^^^^ ^^^ ^^^ is high), reset is permanently active (^^^^^ ^^^ is low),preventing any communication by the digital communication block.

Claims

CLAIMS1. A sensitive Dynamic Vision Sensor (sDVS) configured to generate events ondetecting a relative amount of light variation, comprising each event: pixelcoordinates and a polarity (ON, OFF), and configured to define fovea regions withhigh-resolution and regions with low-resolution, and comprising a plurality ofphysical pixels each one comprising: -a photoreceptor module (p) comprising:o a photoreceptor stage comprising a transistor ^^^ and configured totransform a received light into a voltage ^^^, wherein the photoreceptor stage also comprises an inverting voltage amplifier; oa trans-impedance preamplification stage connected to thephotoreceptor and comprising one or more stacked diodes configuredto amplify the received voltage, and one or more transistors configuredto provide the voltage ^^^ to the stacked diodes and to duplicate saidvoltage ^^^, wherein the one or more transistors of the trans- impedance preamplification stage are two transistors (Mlocal and Mshare) connected to the first and second pixel memory bits and to the contrast computation module (c) and configured to duplicate the voltage Vph; -a contrast computation module (c) comprising:o a buffer stage connected to the trans-impedance preamplificationstage; oan amplification stage connected to the buffer stage, comprising twocapacitors (C1 and C2) and configured to amplify the received voltage by a relationship C1 / C2 between the capacitance value of the twocapacitors and to reset the voltage to VREF when the amplified voltage Vdiff reaches an upper temporal contrast or a lower temporal contrast; and oa differentiation stage connected to the amplification stage andconfigured to launch an ON event when the voltage Vdiffreaches the upper temporal contrast and an OFF event when the voltage Vdiffreaches a lower temporal contrast; wherein a LR upper temporal contrast and a LR lower temporal contrast are defined for LR regions and a HR upper temporal contrast and a HR lowertemporal contrast are defined for HR regions, and wherein the physical pixels are arranged as macro pixels, each pixel further comprising:- a first pixel memory bit, connected to a first neighbor pixel and configured toreceive and share the duplicated pre-amplified voltage from the transistor of the trans-impedance preamplification stage;- a second pixel memory bit connected to a second neighbor pixel andconfigured receive and share the duplicated pre-amplified voltage from the transistor of the trans-impedance preamplification stage; and- a switch swr configured to reset continuously the capacitor C2 of the contrastcomputation module, disabling said contrast computation module for not launching events;and wherein each macro pixel comprises:- a master pixel configured not to share its voltage with other pixels bydeactivating the first and second pixel memory bits and configured to launch events by deactivating the switch swr, and- the rest of pixels; andwherein the rest of pixels in the macro pixel are configured to share its pre- amplified voltage from the trans-impedance preamplification stage with the master pixel by activating the first and / or second pixel memory bits such that thecontrast computation module of the master pixel receives a combined voltagerepresenting the sum of photocurrents from the rest of pixels of the macro pixeland when the combined voltage reaches the LR upper temporal contrast or theLR lower temporal contrast an event is launched, and wherein the sensitive Dynamic Vision Sensor (sDVS) further comprises acontroller configured to define the state of one or more of the rest of pixels to be:- in a LR region, wherein the switch to reset is activated, disabling the contrastcomputation module for not launching events; or- in a HR region, wherein the switch to reset is deactivated, enabling thecontrast computation module to launch events when the voltage receivedfrom the trans-impedance preamplification stage reaches the HR upper temporal contrast or the HR lower temporal contrast.

2. The sensitive DVS sensor according to claim 1, further comprising a switch swlaand a switch swlb connected to the transistor (Mlocal) and configured to activate or deactivate the first and second pixel memory bits.

3. The sensitive DVS sensor according to claim 1, further comprising a switch commlconnected to the switch swr and configured to activate or deactivate the contrast computation module (c).

4. The sensitive DVS sensor according to claim 1, wherein the photoreceptorconverts incident radiation into a continuous photo-current and then to alogarithmic voltage representation as:

5. The sensitive DVS sensor according to claim 1, wherein the transistors of thetrans-impedance preamplification stage conform an active input current mirror.

6. The sensitive DVS sensor according to claim 1, further comprising anasynchronous digital communication circuit connected to an external circuitry andconfigured to send pixel events out of the sensitive DVS sensor using anAddress Event Representation (AER) format.

7. The sensitive DVS sensor according to claim 6, further comprising a Row Arbiterand a Row Buffer, connected to the asynchronous digital communication circuit of the pixels, and an external communication module, connected to the Row Arbiter and the Row Buffer, and wherein the pixel event is sent by:- when at least one event is launched in a pixel, sending a row-wise Rqst signalto the Row Arbiter, which is configured to determine the y-coordinate of saidrow and to reply by activating an Ack signal,- sending all active pixels in the row a polarity signal (p+ or p-) to the RowBuffer, which is configured to store x-coordinates, and- sending the x-coordinates, the polarity and the y-coordinate to the externalcommunication module, which is configured to send them out of the sensitive DVS sensor.

8. The sensitive DVS sensor according to claim 6, wherein address events are sentout of the sensitive DVS sensor with micro to milli second delays.

9. The sensitive DVS sensor according to claim 6, further comprising a Classifiermodule and two output channels, a LR channel, providing absolute coordinates ofLR macro-pixels (x, y, s), and a HR channel, providing relative coordinates withinthe HR region (x-xL, y-yL, s), and wherein the Classifier module comprises HRregion limit coordinates and is configured to determine if an Address-Event (x,y,s)is in the HR region or LR region.

10. The sensitive DVS sensor according to claim 9, wherein the two output channelsare merged and the Address-Event (x,y,s) further comprises a parameter ‘r’ todistinguish whether the event belongs to the LR region or HR region.

11. A method for vision sensing comprising the steps of:- sensing light and converting it to voltage Vph by using thephotoreceptor stage; -pre-amplifying the received voltage by using the trans-impedancepreamplification stage; -duplicating the pre-amplified voltage by using the trans-impedancepreamplification stage; -if the pixel is not a master pixel, the switch swla is OFF and the switch swlb isON, sending one of the duplicated voltages to the first and second pixel memory bits; -if the pixel is a master pixel, the switch swla is ON and the switch swlb is OFF,receiving a signal from the first and second pixel memory bits of one ormore neighbor pixels representing a summed voltage from the rest of pixelsof the macro pixel; -if the pixel is not a master pixel, amplifying the other duplicated voltageby using the one or more stacked diodes of the trans-impedance preamplification stage; -if the pixel is a master pixel, amplifying the summed voltage from the rest ofpixels of the macro pixel;- sending the amplified voltage to the buffer stage if the pixel is a masterpixel, the switch swr is set permanently ON, or if the pixel is not a master pixel and switch swr is ON, otherwise, if the switch swr is OFF, resetting the amplified voltage; -sending the amplified voltage to the amplification stage of the contrastcomputation module (c); -amplifying the voltage by ^^ / ^^;- resetting the voltage to ^^^^ whenever the voltage Vdiff reaches an upperthreshold V+ or a lower threshold V-; and- launching a sending of event by the differentiation stage: an ON eventwhen the voltage Vdiffreaches an upper threshold V+and an OFF event when the voltage Vdiffreaches a lower threshold V-;12. The method for vision sensing according to claim 11, wherein two versions of theglobal bias signals VREF are provided: (^^^^^^ , ^ ^^^^ , ^ ^^^^ ) and (^^^^^^ ^^ , ^ ^^^^ , ^^^^^), such that if the signal coming out of the first or second pixel memory bits (T ORR) is high, the HR version biases (^ , ^ ^ ^^^^^^ ^^^ , ^^^^ ) is selected for thecomparators in the contrast computation circuit and, if the signal coming out ofthe first or second pixel memory bits (T OR R) is low, the LR version biases(^^^^^^ , ^ ^^^^ , ^ ^^^^ ) is selected.

Images