Processing device, sensor device and method for operating a processing device

Abstract

A processing device (100) for converting a series of asynchronously generated data tuples (D) into a feature vector (V) is disclosed, which processing device (100) is configured to receive a number of data tuples (D), generate for each data tuple (D) an intermediate vector (I), generate the feature vector (V) based on all intermediate vectors (I), and update the feature vector (V) based on the reception of additional data tuples. Here, when a new data tuple is received, an intermediate vector is generated for said new data tuple and a new feature vector is generated based on the previous feature vector (V) and the intermediate vector of the new data tuple, and for each entry of the feature vector (V) it is monitored whether a change of value occurs, and if for one of the entries of the feature vector (V) no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.

Description

PROCESSING DEVICE, SENSOR DEVICE AND METHOD FOR OPERATING A PROCESSING DEVICEFIELD OF THE INVENTIONThe present technology relates to a processing device, a sensor device, and a method for operating a processing device, in particular, to a processing device, a sensor device and a method for operating a processing device that allows an improved processing of event data.BACKGROUNDPresently, sensor data obtained in dynamic / event vision sensors, DVS / EVS, are further processed to give estimates on the observed scenes. This is often done by using algorithms that need a synchronized input such as e.g. machine learning algorithms that are trained to fulfill certain tasks in order to generate the desired estimates. Here, often the main advantage of DVS / EVS data is lost, i.e. the simultaneously high temporal resolution and spatial sparsity (implying a low latency), due to the need to fit the asynchronously generated DVS / EVS data into a synchronously operating framework.Improved processing devices, sensor devices and methods for operating these processing and sensor devices are desirable that mitigate this problem.SUMMARY OF INVENTIONTo this end, a processing device is provided for converting a series of asynchronously generated data tuples into a feature vector, the processing device being configured to receive a number of data tuples, to generate for each data tuple an intermediate vector, to generate the feature vector based on all intermediate vectors, and to update the feature vector based on the reception of additional data tuples. Here, when a new data tuple is received, an intermediate vector is generated for said new data tuple and a new feature vector is generated based on the previous feature vector and the intermediate vector of the new data tuple, and for each entry of the feature vector it is monitored whether a change of value occurs, and if for one of the entries of the feature vector no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.Further, a sensor device is provided that comprises the above processing device and a vision sensor that comprises a pixel array having a plurality of pixels each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, based on which electrical signals event data are generated, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel above an event detection threshold. Here, the vision sensor is configured to generate one data tuple for each event, the data tuple indicating at least the time of event generation and the coordinates of the event on the pixel array.Further, a method is provided that is executed by a processing device for converting a series ofasynchronously generated data tuples into a feature vector, the method comprising: receiving a number n of data tuples; generating for each data tuple an intermediate vector; generating the feature vector based on all intermediate vectors; and updating the feature vector based on the reception of additional data tuples. Here, when a new data tuple is received, an intermediate vector is generated for said new data tuple and a new feature vector is generated based on the previous feature vector and the intermediate vector of the new data tuple, and for each entry of the feature vector it is monitored whether a change of value occurs, and if for one of the entries of the feature vector no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.Further, in the above method, a vision sensor that comprises a pixel array having a plurality of pixels each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, performs forming event data based on the generated electrical signals, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel above an event detection threshold; and generating one data tuple for each event, the data tuple indicating at least the time of event generation and the coordinates of the event on the pixel array.In the above, data tuples that can be asynchronously generated, preferably by an DVS / EVS, are transformed into a vector representation that can be used as input of further processing steps, like e.g. as input for artificial intelligence models such as neural networks or transformer networks / algorithms, i.e. attention-based vector sequence processing algorithms. Here, the vector is updated after receipt of new data tuples, which allows for an asynchronous update, if the new data tuples are generated or received asynchronously. Moreover, the influence of older data tuples, i.e. data tuples that have not been generated or received recently is reduced by reducing vector values over time that did not change for a predetermined time. In this manner the vector representation can be asynchronously kept in an up-to-date state, while it can on the other hand be output to or requested by a further processing stage synchronously / periodically or on demand. This allows to transfer the simultaneously present high temporal resolution and spatial sparsity of e.g. event data to a vector representation that can be used for in principle known further processing steps. This allows reducing the latency and increasing the time resolution of this processing.BRIEF DESCRIPTION OF DRAWINGSFig. 1 is a schematic diagram of a sensor device.Fig. 2 is a schematic block diagram of a sensor section.Fig. 3 is a schematic block diagram of a pixel array section.Fig. 4 is a schematic circuit diagram of a pixel block.Fig. 5 is a schematic block diagram illustrating of an event detecting section.Fig. 6 is a schematic circuit diagram of a current-voltage converting section.Fig. 7 is a schematic circuit diagram of a subtraction section and a quantization section.Fig. 8 is a schematic diagram of a frame data generation method based on event data.Fig. 9 is a schematic block diagram of another quantization section.Fig. 10 is a schematic diagram of another event detecting section.Fig. 11 is a schematic block diagram of another pixel array section.Fig. 12 is a schematic circuit diagram of another pixel block.Fig. 13 is a schematic block diagram of a scan-type sensor device.Fig. 14A to 14C are a schematic block diagram of a processing device.Fig. 15 is a schematic block diagram of a sensor device.Fig. 16 is a schematic illustration of counting in a processing device.Fig. 17A and 17B are schematic illustrations of feature vector generation.Figs. 18A and 18B are schematic illustrations of the use of multi-layered perceptrons in a processing device.Fig. 19 is a schematic illustration of an area-wise generation of feature vectors.Fig. 20 is another schematic illustration of an area-wise generation of feature vectors.Fig. 21 is a schematic block diagram of a sensor system.Fig. 22 is a schematic illustration of a chip design of a processing device.Fig. 23 is a schematic process flow of a method for operating a processing device.Fig. 24 is a schematic block diagram of a vehicle control system.Fig. 25 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section.Fig. 26A and 26B are schematic illustrations of a mobile device and a head mounted display comprising a sensor device.DETAILED DESCRIPTIONThe present disclosure is directed to mitigating problems related to processing of asynchronously generated data like the data of event based / dynamic vision sensors, EVS / DVS. Although the present disclosure can in principle be applied to any tuples of asynchronously generated data, the below detailed description will mainly focus on the example of EVS / DVS data, in order to simplify the description and also in order to cover an important application example.First, a possible implementation of a EVS / DVS will be described. This is of course purely exemplary. It is to be understood that EVSs / DVSs could also be implemented differently.Fig. 1 is a diagram illustrating a configuration example of a sensor device 10, which is in the example of Fig. 1 constituted by a sensor chip.The sensor device 10 is a single-chip semiconductor chip and includes a sensor die (substrate) 11, which serves as a plurality of dies (substrates), and a logic die 12 that are stacked. Note that, the sensor device 10 can also include only a single die or three or more stacked dies.In the sensor device 10 of Fig. 1, the sensor die 11 includes (a circuit serving as) a sensor section 21, and the logic die 12 includes a logic section 22. Note that, the sensor section 21 can be partly formed on the logic die 12. Further, the logic section 22 can be partly fonued on the sensor die 11.The sensor section 21 includes pixels configured to perform photoelectric conversion on incident light to generate electrical signals, and generates event data indicating the occurrence of events that are changes in the electrical signal of the pixels. The sensor section 21 supplies the event data to the logic section 22. That is, the sensor section 21 perfomrs imaging of perfonning, in the pixels, photoelectric conversion on incident light to generate electrical signals, similarly to a synchronous image sensor, for example. The sensor section 21, however, generates event data indicating the occurrence of events that are changes in the electrical signal of the pixels instead of generating image data in a frame format (frame data). The sensor section 21 outputs, to the logic section 22, the event data obtained by the imaging.Here, the synchronous image sensor is an image sensor configured to perform imaging in synchronization with a vertical synchronization signal and output frame data that is image data in a frame format. The sensor section 21 can be regarded as asynchronous (an asynchronous image sensor) in contrast to the synchronous image sensor, since the sensor section 21 does not operate in synchronization with a vertical synchronization signal when outputting event data. In particular, the sensor section 21 can output event data with a temporal precision of 10'6s.Note that, the sensor section 21 may generate and output, other than event data, frame data, similarly toRECTIFIED SHEET (RULE 91) ISA / EPthe synchronous image sensor. In addition, the sensor section 21 can output, together with event data, electrical signals of pixels in which events have occurred, as pixel signals that are pixel values of the pixels in frame data.The logic section 22 controls the sensor section 21 as needed. Further, the logic section 22 performs various types of data processing, such as data processing of generating frame data on the basis of event data from the sensor section 21 and image processing on frame data from the sensor section 21 or frame data generated on the basis of the event data from the sensor section 21, and outputs data processing results obtained by performing the various types of data processing on the event data and the frame data. The logic section 22 may implement the functions of a control unit as described below.Fig. 2 is a block diagram illustrating a configuration example of the sensor section 21 of Fig. 1.The sensor section 21 includes a pixel array section 31, a driving section 32, an arbiter 33, an AD (Analog to Digital) conversion section 34, and an output section 35.The pixel array section 31 includes a plurality of pixels 51 (Fig. 3) arrayed in a two-dimensional lattice pattern. The pixel array section 31 detects, in a case where a change larger than a predetermined threshold (including a change equal to or larger than the threshold as needed) has occurred in (a voltage corresponding to) a photocurrent that is an electrical signal generated by photoelectric conversion in the pixel 51, the change in the photocurrent as an event. In a case of detecting an event, the pixel array section 31 outputs, to the arbiter 33, a request for requesting the output of event data indicating the occurrence of the event. Then, in a case of receiving a response indicating event data output permission from the arbiter 33, the pixel array section 31 outputs the event data to the driving section 32 and the output section 35. In addition, the pixel array section 31 may output an electrical signal of the pixel 51 in which the event has been detected to the AD conversion section 34.The driving section 32 supplies control signals to the pixel array section 31 to drive the pixel array section 31. For example, the driving section 32 drives the pixel 51 regarding which the pixel array section 31 has output event data, so that the pixel 51 in question supplies (outputs) a pixel signal to the AD conversion section 34.The arbiter 33 arbitrates the requests for requesting the output of event data from the pixel array section 31, and returns responses indicating event data output permission or prohibition to the pixel array section 31.The AD conversion section 34 includes, for example, a single-slope ADC (AD converter) (not illustrated) in each column of pixel blocks 41 (Fig. 3) described later, for example. The AD conversion section 34 performs, with the ADC in each column, AD conversion on pixel signals of the pixels 51 of the pixel blocks 41 in the column, and supplies the resultant to the output section 35. Note that, the AD conversion section 34 can perform CDS (Correlated Double Sampling) together with pixel signal AD conversion.The output section 35 performs necessary processing on the pixel signals from the AD conversion section 34 and the event data from the pixel array section 31 and supplies the resultant to the logic section 22 (Fig. 1).Here, a change in the photocurrent generated in the pixel 51 can be recognized as a change in the amount of light entering the pixel 51, so that it can also be said that an event is a change in light amount (a change in light amount larger than the threshold) in the pixel 51.Event data indicating the occurrence of an event at least includes location information (coordinates or the like) indicating the location of a pixel block in which a change in light amount, which is the event, has occurred. Besides, the event data can also include the polarity (positive or negative) of the change in light amount.With regard to the series of event data that is output from the pixel array section 31 at timings at which events have occurred, it can be said that, as long as the event data interval is the same as the event occurrence interval, the event data implicitly includes time point information indicating (relative) time points at which the events have occurred. However, for example, when the event data is stored in a memory and the event data interval is no longer the same as the event occurrence interval, the time point information implicitly included in the event data is lost. Thus, the output section 35 includes, in event data, time point information indicating (relative) time points at which events have occurred, such as timestamps, before the event data interval is changed from the event occurrence interval. The processing of including time point information in event data can be performed in any block other than the output section 35 as long as the processing is performed before time point information implicitly included in event data is lost.Fig. 3 is a block diagram illustrating a configuration example of the pixel array section 31 of Fig. 2.The pixel array section 31 includes the plurality of pixel blocks 41. The pixel block 41 includes the IxJ pixels 51 that are one or more pixels arrayed in I rows and J columns (I and J are integers), an event detecting section 52, and a pixel signal generating section 53. The one or more pixels 51 in the pixel block 41 share the event detecting section 52 and the pixel signal generating section 53. Further, in each column of the pixel blocks 41, a VSL (Vertical Signal Line) for connecting the pixel blocks 41 to the ADC of the AD conversion section 34 is wired.The pixel 51 receives light incident from an object and performs photoelectric conversion to generate a photocurrent serving as an electrical signal. The pixel 51 supplies the photocurrent to the event detecting section 52 under the control of the driving section 32.The event detecting section 52 detects, as an event, a change larger than the predetermined threshold in photocurrent from each of the pixels 51, under the control of the driving section 32. In a case of detecting an event, the event detecting section 52 supplies, to the arbiter 33 (Fig. 2), a request for requesting the output of event data indicating the occurrence of the event. Then, when receiving aresponse indicating event data output permission to the request from the arbiter 33, the event detecting section 52 outputs the event data to the driving section 32 and the output section 35.The pixel signal generating section 53 generates, in the case where the event detecting section 52 has detected an event, a voltage corresponding to a photocurrent from the pixel 51 as a pixel signal, and supplies the voltage to the AD conversion section 34 through the VSL, under the control of the driving section 32.Here, detecting a change larger than the predetermined threshold in photocurrent as an event can also be recognized as detecting, as an event, absence of change larger than the predetermined threshold in photocurrent. The pixel signal generating section 53 can generate a pixel signal in the case where absence of change larger than the predetermined threshold in photocurrent has been detected as an event as well as in the case where a change larger than the predetermined threshold in photocurrent has been detected as an event.Fig. 4 is a circuit diagram illustrating a configuration example of the pixel block 41.The pixel block 41 includes, as described with reference to Fig. 3, the pixels 51, the event detecting section 52, and the pixel signal generating section 53.The pixel 51 includes a photoelectric conversion element 61 and transfer transistors 62 and 63.The photoelectric conversion element 61 includes, for example, a PD (Photodiode). The photoelectric conversion element 61 receives incident light and performs photoelectric conversion to generate charges.The transfer transistor 62 includes, for example, an N (Negative)-type MOS (Metal-Oxide- Semiconductor) FET (Field Effect Transistor). The transfer transistor 62 of the n-th pixel 51 of the IxJ pixels 51 in the pixel block 41 is turned on or off in response to a control signal OFGn supplied from the driving section 32 (Fig. 2). When the transfer transistor 62 is turned on, charges generated in the photoelectric conversion element 61 are transferred (supplied) to the event detecting section 52, as a photocurrent.The transfer transistor 63 includes, for example, an N-type MOSFET. The transfer transistor 63 of the n- th pixel 51 of the IxJ pixels 51 in the pixel block 41 is turned on or off in response to a control signal TRGn supplied from the driving section 32. When the transfer transistor 63 is turned on, charges generated in the photoelectric conversion element 61 are transferred to an FD 74 of the pixel signal generating section 53.The IxJ pixels 51 in the pixel block 41 are connected to the event detecting section 52 of the pixel block 41 through nodes 60. Thus, photocurrents generated in (the photoelectric conversion elements 61 of) the pixels 51 are supplied to the event detecting section 52 through the nodes 60. As a result, the event detecting section 52 receives the sum of photocurrents from all the pixels 51 in the pixel block 41. Thus,the event detecting section 52 detects, as an event, a change in sum of photocurrents supplied from the IxJ pixels 51 in the pixel block 41.The pixel signal generating section 53 includes a reset transistor 71, an amplification transistor 72, a selection transistor 73, and the FD (Floating Diffusion) 74.The reset transistor 71, the amplification transistor 72, and the selection transistor 73 include, for example, N-type MOSFETs.The reset transistor 71 is turned on or off in response to a control signal RST supplied from the driving section 32 (Fig. 2). When the reset transistor 71 is turned on, the FD 74 is connected to a power supply VDD, and charges accumulated in the FD 74 are thus discharged to the power supply VDD. With this, the FD 74 is reset.The amplification transistor 72 has a gate connected to the FD 74, a drain connected to the power supply VDD, and a source connected to the VSL through the selection transistor 73. The amplification transistor 72 is a source follower and outputs a voltage (electrical signal) corresponding to the voltage of the FD 74 supplied to the gate to the VSL through the selection transistor 73.The selection transistor 73 is turned on or off in response to a control signal SEL supplied from the driving section 32. When the selection transistor 73 is turned on, a voltage corresponding to the voltage of the FD 74 from the amplification transistor 72 is output to the VSL.The FD 74 accumulates charges transferred from the photoelectric conversion elements 61 of the pixels 51 through the transfer transistors 63, and converts the charges to voltages.With regard to the pixels 51 and the pixel signal generating section 53, which are configured as described above, the driving section 32 turns on the transfer transistors 62 with control signals OFGn, so that the transfer transistors 62 supply, to the event detecting section 52, photocurrents based on charges generated in the photoelectric conversion elements 61 of the pixels 51. With this, the event detecting section 52 receives a current that is the sum of the photocurrents from all the pixels 51 in the pixel block 41, which might also be only a single pixel.When the event detecting section 52 detects, as an event, a change in photocurrent (sum of photocurrents) in the pixel block 41, the driving section 32 turns off the transfer transistors 62 of all the pixels 51 in the pixel block 41, to thereby stop the supply of the photocurrents to the event detecting section 52. Then, the driving section 32 sequentially turns on, with the control signals TRGn, the transfer transistors 63 of the pixels 51 in the pixel block 41 in which the event has been detected, so that the transfer transistors 63 transfers charges generated in the photoelectric conversion elements 61 to the FD 74. The FD 74 accumulates the charges transferred from (the photoelectric conversion elements 61 of) the pixels 51. Voltages corresponding to the charges accumulated in the FD 74 are output to the VSL, as pixel signals of the pixels 51, through the amplification transistor 72 and the selection transistor 73.As described above, in the sensor section 21 (Fig. 2), only pixel signals of the pixels 51 in the pixel block 41 in which an event has been detected are sequentially output to the VSL. The pixel signals output to the VSL are supplied to the AD conversion section 34 to be subjected to AD conversion.Here, in the pixels 51 in the pixel block 41, the transfer transistors 63 can be turned on not sequentially but simultaneously. In this case, the sum of pixel signals of all the pixels 51 in the pixel block 41 can be output.In the pixel array section 31 of Fig. 3, the pixel block 41 includes one or more pixels 51, and the one or more pixels 51 share the event detecting section 52 and the pixel signal generating section 53. Thus, in the case where the pixel block 41 includes a plurality of pixels 51, the numbers of the event detecting sections 52 and the pixel signal generating sections 53 can be reduced as compared to a case where the event detecting section 52 and the pixel signal generating section 53 are provided for each of the pixels 51, with the result that the scale of the pixel array section 31 can be reduced.Note that, in the case where the pixel block 41 includes a plurality of pixels 51, the event detecting section 52 can be provided for each of the pixels 51. In the case where the plurality of pixels 51 in the pixel block 41 share the event detecting section 52, events are detected in units of the pixel blocks 41. In the case where the event detecting section 52 is provided for each of the pixels 51, however, events can be detected in units of the pixels 51.Yet, even in the case where the plurality of pixels 51 in the pixel block 41 share the single event detecting section 52, events can be detected in units of the pixels 51 when the transfer transistors 62 of the plurality of pixels 51 are temporarily turned on in a time-division manner.Further, in a case where there is no need to output pixel signals, the pixel block 41 can be formed without the pixel signal generating section 53. In the case where the pixel block 41 is formed without the pixel signal generating section 53, the sensor section 21 can be formed without the AD conversion section 34 and the transfer transistors 63. In this case, the scale of the sensor section 21 can be reduced. The sensor will then output the address of the pixel (block) in which the event occurred, if necessary with a time stamp.Fig. 5 is a block diagram illustrating a configuration example of the event detecting section 52 of Fig. 3.The event detecting section 52 includes a current-voltage converting section 81, a buffer 82, a subtraction section 83, a quantization section 84, and a transfer section 85.The current-voltage converting section 81 converts (a sum of) photocurrents from the pixels 51 to voltages corresponding to the logarithms of the photocurrents (hereinafter also referred to as a "photovoltage") and supplies the voltages to the buffer 82.The buffer 82 buffers photovoltages from the current-voltage converting section 81 and supplies the resultant to the subtraction section 83.The subtraction section 83 calculates, at a timing instructed by a row driving signal that is a control signal from the driving section 32, a difference between the current photovoltage and a photovoltage at a timing slightly shifted from the current time, and supplies a difference signal corresponding to the difference to the quantization section 84.The quantization section 84 quantizes difference signals from the subtraction section 83 to digital signals and supplies the quantized values of the difference signals to the transfer section 85 as event data.The transfer section 85 transfers (outputs), on the basis of event data from the quantization section 84, the event data to the output section 35. That is, the transfer section 85 supplies a request for requesting the output of the event data to the arbiter 33. Then, when receiving a response indicating event data output permission to the request from the arbiter 33, the transfer section 85 outputs the event data to the output section 35.Fig. 6 is a circuit diagram illustrating a configuration example of the current-voltage converting section 81 of Fig. 5.The current-voltage converting section 81 includes transistors 91 to 93. As the transistors 91 and 93, for example, N-type MOSFETs can be employed. As the transistor 92, for example, a P-type MOSFET can be employed.The transistor 91 has a source connected to the gate of the transistor 93, and a photocurrent is supplied from the pixel 51 to the connecting point between the source of the transistor 91 and the gate of the transistor 93. The transistor 91 has a drain connected to the power supply VDD and a gate connected to the drain of the transistor 93.The transistor 92 has a source connected to the power supply VDD and a drain connected to the connecting point between the gate of the transistor 91 and the drain of the transistor 93. A predetermined bias voltage Vbias is applied to the gate of the transistor 92. With the bias voltage Vbias, the transistor 92 is turned on or off, and the operation of the current-voltage converting section 81 is turned on or off depending on whether the transistor 92 is turned on or off.The source of the transistor 93 is grounded.In the current-voltage converting section 81, the transistor 91 has the drain connected on the power supply VDD side. The source of the transistor 91 is connected to the pixels 51 (Fig. 4), so that photocurrents based on charges generated in the photoelectric conversion elements 61 of the pixels 51 flow through the transistor 91 (from the drain to the source). The transistor 91 operates in a subthreshold region, and at the gate of the transistor 91, photovoltages corresponding to the logarithms of the photocurrents flowingthrough the transistor 91 are generated. As described above, in the current-voltage converting section 81, the transistor 91 converts photocurrents from the pixels 51 to photovoltages corresponding to the logarithms of the photocurrents.In the current-voltage converting section 81, the transistor 91 has the gate connected to the connecting point between the drain of the transistor 92 and the drain of the transistor 93, and the photovoltages are output from the connecting point in question.Fig. 7 is a circuit diagram illustrating configuration examples of the subtraction section 83 and the quantization section 84 of Fig. 5.The subtraction section 83 includes a capacitor 101, an operational amplifier 102, a capacitor 103, and a switch 104. The quantization section 84 includes a comparator 111.The capacitor 101 has one end connected to the output terminal of the buffer 82 (Fig. 5) and the other end connected to the input terminal (inverting input terminal) of the operational amplifier 102. Thus, photovoltages are input to the input terminal of the operational amplifier 102 through the capacitor 101.The operational amplifier 102 has an output terminal connected to the non-inverting input terminal (+) of the comparator 111.The capacitor 103 has one end connected to the input terminal of the operational amplifier 102 and the other end connected to the output terminal of the operational amplifier 102.The switch 104 is connected to the capacitor 103 to switch the connections between the ends of the capacitor 103. The switch 104 is turned on or off in response to a row driving signal that is a control signal from the driving section 32, to thereby switch the connections between the ends of the capacitor 103.A photovoltage on the buffer 82 (Fig. 5) side of the capacitor 101 when the switch 104 is on is denoted by Vinit, and the capacitance (electrostatic capacitance) of the capacitor 101 is denoted by Cl. The input terminal of the operational amplifier 102 serves as a virtual ground terminal, and a charge Qinit that is accumulated in the capacitor 101 in the case where the switch 104 is on is expressed by Expression (1).Qinit = Cl x Vinit (1)Further, in the case where the switch 104 is on, the connection between the ends of the capacitor 103 is cut (short-circuited), so that no charge is accumulated in the capacitor 103.When a photovoltage on the buffer 82 (Fig. 5) side of the capacitor 101 in the case where the switch 104 has thereafter been turned off is denoted by Vafter, a charge Qafter that is accumulated in the capacitor 101 in the case where the switch 104 is off is expressed by Expression (2).Qafter = Cl x Vafter (2)When the capacitance of the capacitor 103 is denoted by C2 and the output voltage of the operational amplifier 102 is denoted by Vout, a charge Q2 that is accumulated in the capacitor 103 is expressed by Expression (3).Q2 = -C2 x Vout (3)Since the total amount of charges in the capacitors 101 and 103 does not change before and after the switch 104 is turned off, Expression (4) is established.Qinit = Qafter + Q2 (4)When Expression (1) to Expression (3) are substituted for Expression (4), Expression (5) is obtained.Vout = -(C1 / C2) x (Vafter - Vinit) (5)With Expression (5), the subtraction section 83 subtracts the photovoltage Vinit from the photovoltage Vafter, that is, calculates the difference signal (Vout) corresponding to a difference Vafter - Vinit between the photovoltages Vafter and Vinit. With Expression (5), the subtraction gain of the subtraction section 83 is C1 / C2. Since the maximum gain is normally desired, Cl is preferably set to a large value and C2 is preferably set to a small value. Meanwhile, when C2 is too small, kTC noise increases, resulting in a risk of deteriorated noise characteristics. Thus, the capacitance C2 can only be reduced in a range that achieves acceptable noise. Further, since the pixel blocks 41 each have installed therein the event detecting section 52 including the subtraction section 83, the capacitances Cl and C2 have space constraints. In consideration of these matters, the values of the capacitances Cl and C2 are determined.The comparator 111 compares a difference signal from the subtraction section 83 with a predetermined threshold (voltage) Vth (>0) applied to the inverting input terminal (-), thereby quantizing the difference signal. The comparator 111 outputs the quantized value obtained by the quantization to the transfer section 85 as event data.For example, in a case where a difference signal is larger than the threshold Vth, the comparator 111 outputs an H (High) level indicating 1, as event data indicating the occurrence of an event. In a case where a difference signal is not larger than the threshold Vth, the comparator 111 outputs an L (Low) level indicating 0, as event data indicating that no event has occurred.The transfer section 85 supplies a request to the arbiter 33 in a case where it is confirmed on the basis of event data from the quantization section 84 that a change in light amount that is an event has occurred, that is, in the case where the difference signal (Vout) is larger than the threshold Vth. When receiving a response indicating event data output permission, the transfer section 85 outputs the event data indicatingthe occurrence of the event (for example, H level) to the output section 35.The output section 35 includes, in event data from the transfer section 85, location / address information regarding (the pixel block 41 including) the pixel 51 in which an event indicated by the event data has occurred and time point information indicating a time point at which the event has occurred, and further, as needed, the polarity of a change in light amount that is the event, i.e. whether the intensity did increase or decrease. The output section 35 outputs the event data.As the data format of event data including location information regarding the pixel 51 in which an event has occurred, time point information indicating a time point at which the event has occurred, and the polarity of a change in light amount that is the event, for example, the data format called "AER (Address Event Representation)" can be employed.Note that, a gain A of the entire event detecting section 52 is expressed by the following expression where the gain of the current-voltage converting section 81 is denoted by CGiogand the gain of the buffer 82 is 1.A = CGiogC 1 / C2 (ZiPhoto_n) (6)Here, iPhoto_n denotes a photocurrent of the n-th pixel 51 of the IxJ pixels 51 in the pixel block 41. In Expression (6), E denotes the summation of n that takes integers ranging from 1 to IxJ.Note that, the pixel 51 can receive any light as incident light with an optical fdter through which predetermined light passes, such as a color fdter. For example, in a case where the pixel 51 receives visible light as incident light, event data indicates the occurrence of changes in pixel value in images including visible objects. Further, for example, in a case where the pixel 51 receives, as incident light, infrared light, millimeter waves, or the like for ranging, event data indicates the occurrence of changes in distances to objects. In addition, for example, in a case where the pixel 51 receives infrared light for temperature measurement, as incident light, event data indicates the occurrence of changes in temperature of objects. In the present embodiment, the pixel 51 is assumed to receive visible light as incident light.Fig. 8 is a diagram illustrating an example of a frame data generation method based on event data.The logic section 22 sets a frame interval and a frame width on the basis of an externally input command, for example. Here, the frame interval represents the interval of frames of frame data that is generated on the basis of event data. The frame width represents the time width of event data that is used for generating frame data on a single frame. A frame interval and a frame width that are set by the logic section 22 are also referred to as a "set frame interval" and a "set frame width," respectively.The logic section 22 generates, on the basis of the set frame interval, the set frame width, and event data from the sensor section 21, frame data that is image data in a frame format, to thereby convert the event data to the frame data.That is, the logic section 22 generates, in each set frame interval, frame data on the basis of event data in the set frame width from the beginning of the set frame interval.Here, it is assumed that event data includes time point information ti indicating a time point at which an event has occurred (hereinafter also referred to as an "event time point") and coordinates (x, y) serving as location information regarding (the pixel block 41 including) the pixel 51 in which the event has occurred (hereinafter also referred to as an "event location").In Fig. 8, in a three-dimensional space (time and space) with the x axis, the y axis, and the time axis t, points representing event data are plotted on the basis of the event time point t and the event location (coordinates) (x, y) included in the event data.That is, when a location (x, y, t) on the three-dimensional space indicated by the event time point t and the event location (x, y) included in event data is regarded as the space-time location of an event, in Fig. 8, the points representing the event data are plotted on the space-time locations (x, y, t) of the events.The logic section 22 starts to generate frame data on the basis of event data by using, as a generation start time point at which frame data generation starts, a predetermined time point, for example, a time point at which frame data generation is externally instructed or a time point at which the sensor device 10 is powered on.Here, cuboids each having the set frame width in the direction of the time axis t in the set frame intervals, which appear from the generation start time point, are referred to as a "frame volume." The size of the frame volume in the x-axis direction or the y-axis direction is equal to the number of the pixel blocks 41 or the pixels 51 in the x-axis direction or the y-axis direction, for example.The logic section 22 generates, in each set frame interval, frame data on a single frame on the basis of event data in the frame volume having the set frame width from the beginning of the set frame interval.Frame data can be generated by, for example, setting white to a pixel (pixel value) in a frame at the event location (x, y) included in event data and setting a predetermined color such as gray to pixels at other locations in the frame.Besides, in a case where event data includes the polarity of a change in light amount that is an event, frame data can be generated in consideration of the polarity included in the event data. For example, white can be set to pixels in the case a positive polarity, while black can be set to pixels in the case of a negative polarity.In addition, in the case where pixel signals of the pixels 51 are also output when event data is output as described with reference to Fig. 3 and Fig. 4, frame data can be generated on the basis of the event data by using the pixel signals of the pixels 51. That is, frame data can be generated by setting, in a frame, a pixel at the event location (x, y) (in a block corresponding to the pixel block 41) included in event data toa pixel signal of the pixel 51 at the location (x, y) and setting a predetermined color such as gray to pixels at other locations.Note that, in the frame volume, there are a plurality of pieces of event data that are different in the event time point t but the same in the event location (x, y) in some cases. In this case, for example, event data at the latest or oldest event time point t can be prioritized. Further, in the case where event data includes polarities, the polarities of a plurality of pieces of event data that are different in the event time point t but the same in the event location (x, y) can be added together, and a pixel value based on the added value obtained by the addition can be set to a pixel at the event location (x, y).Here, in a case where the frame width and the frame interval are the same, the frame volumes are adjacent to each other without any gap. Further, in a case where the frame interval is larger than the frame width, the frame volumes are arranged with gaps. In a case where the frame width is larger than the frame interval, the frame volumes are arranged to be partly overlapped with each other.Fig. 9 is a block diagram illustrating another configuration example of the quantization section 84 of Fig. 5.Note that, in Fig. 9, parts corresponding to those in the case of Fig. 7 are denoted by the same reference signs, and the description thereof is omitted as appropriate below.In Fig. 9, the quantization section 84 includes comparators 111 and 112 and an output section 113.Thus, the quantization section 84 of Fig. 9 is similar to the case of Fig. 7 in including the comparator 111. However, the quantization section 84 of Fig. 9 is different from the case of Fig. 7 in newly including the comparator 112 and the output section 113.The event detecting section 52 (Fig. 5) including the quantization section 84 of Fig. 9 detects, in addition to events, the polarities of changes in light amount that are events.In the quantization section 84 of Fig. 9, the comparator 111 outputs, in the case where a difference signal is larger than the threshold Vth, the H level indicating 1, as event data indicating the occurrence of an event having the positive polarity. The comparator 111 outputs, in the case where a difference signal is not larger than the threshold Vth, the L level indicating 0, as event data indicating that no event having the positive polarity has occurred.Further, in the quantization section 84 of Fig. 9, a threshold Vth' (<Vth) is supplied to the non-inverting input terminal (+) of the comparator 112, and difference signals are supplied to the inverting input terminal (-) of the comparator 112 from the subtraction section 83. Here, for the sake of simple description, it is assumed that the threshold Vth' is equal to -Vth, for example, which needs however not to be the case.The comparator 112 compares a difference signal from the subtraction section 83 with the threshold Vth' applied to the inverting input terminal (-), thereby quantizing the difference signal. The comparator 112 outputs, as event data, the quantized value obtained by the quantization.For example, in a case where a difference signal is smaller than the threshold Vth' (the absolute value of the difference signal having a negative value is larger than the threshold Vth), the comparator 112 outputs the H level indicating 1, as event data indicating the occurrence of an event having the negative polarity. Further, in a case where a difference signal is not smaller than the threshold Vth' (the absolute value of the difference signal having a negative value is not larger than the threshold Vth), the comparator 112 outputs the L level indicating 0, as event data indicating that no event having the negative polarity has occurred.The output section 113 outputs, on the basis of event data output from the comparators 111 and 112, event data indicating the occurrence of an event having the positive polarity, event data indicating the occurrence of an event having the negative polarity, or event data indicating that no event has occurred to the transfer section 85.For example, the output section 113 outputs, in a case where event data from the comparator 111 is the H level indicating 1, +V volts indicating +1, as event data indicating the occurrence of an event having the positive polarity, to the transfer section 85. Further, the output section 113 outputs, in a case where event data from the comparator 112 is the H level indicating 1, -V volts indicating -1, as event data indicating the occurrence of an event having the negative polarity, to the transfer section 85. In addition, the output section 113 outputs, in a case where each event data from the comparators 111 and 112 is the L level indicating 0, 0 volts (GND level) indicating 0, as event data indicating that no event has occurred, to the transfer section 85.The transfer section 85 supplies a request to the arbiter 33 in the case where it is confirmed on the basis of event data from the output section 113 of the quantization section 84 that a change in light amount that is an event having the positive polarity or the negative polarity has occurred. After receiving a response indicating event data output permission, the transfer section 85 outputs event data indicating the occurrence of the event having the positive polarity or the negative polarity (+V volts indicating 1 or -V volts indicating -1) to the output section 35.Preferably, the quantization section 84 has a configuration as illustrated in Fig. 9.Fig. 10 is a diagram illustrating another configuration example of the event detecting section 52.In Fig. 10, the event detecting section 52 includes a subtractor 430, a quantizer 440, a memory 451, and a controller 452. The subtractor 430 and the quantizer 440 correspond to the subtraction section 83 and the quantization section 84, respectively.Note that, in Fig. 10, the event detecting section 52 further includes blocks corresponding to the current-voltage converting section 81 and the buffer 82, but the illustrations of the blocks are omitted in Fig. 10.The subtractor 430 includes a capacitor 431, an operational amplifier 432, a capacitor 433, and a switch 434. The capacitor 431, the operational amplifier 432, the capacitor 433, and the switch 434 correspond to the capacitor 101, the operational amplifier 102, the capacitor 103, and the switch 104, respectively.The quantizer 440 includes a comparator 441. The comparator 441 corresponds to the comparator 111.The comparator 441 compares a voltage signal (difference signal) from the subtractor 430 with the predetermined threshold voltage Vth applied to the inverting input terminal (-). The comparator 441 outputs a signal indicating the comparison result, as a detection signal (quantized value).The voltage signal from the subtractor 430 may be input to the input terminal (-) of the comparator 441, and the predetermined threshold voltage Vth may be input to the input terminal (+) of the comparator 441.The controller 452 supplies the predetermined threshold voltage Vth applied to the inverting input terminal (-) of the comparator 441. The threshold voltage Vth which is supplied may be changed in a time-division manner. For example, the controller 452 supplies a threshold voltage Vthl corresponding to ON events (for example, positive changes in photocurrent) and a threshold voltage Vth2 corresponding to OFF events (for example, negative changes in photocurrent) at different timings to allow the single comparator to detect a plurality of types of address events (events).The memory 451 accumulates output from the comparator 441 on the basis of Sample signals supplied from the controller 452. The memory 451 may be a sampling circuit, such as a switch, plastic, or capacitor, or a digital memory circuit, such as a latch or flip-flop. For example, the memory 451 may hold, in a period in which the threshold voltage Vth2 corresponding to OFF events is supplied to the inverting input terminal (-) of the comparator 441, the result of comparison by the comparator 441 using the threshold voltage Vthl corresponding to ON events. Note that, the memory 451 may be omitted, may be provided inside the pixel (pixel block 41), or may be provided outside the pixel.Fig. 11 is a block diagram illustrating another configuration example of the pixel array section 31 of Fig. 2.Note that, in Fig. 11, parts corresponding to those in the case of Fig. 3 are denoted by the same reference signs, and the description thereof is omitted as appropriate below.In Fig. 11, the pixel array section 31 includes the plurality of pixel blocks 41. The pixel block 41 includes the lx J pixels 51 that are one or more pixels and the event detecting section 52.Thus, the pixel array section 31 of Fig. 11 is similar to the case of Fig. 3 in that the pixel array section 31 includes the plurality of pixel blocks 41 and that the pixel block 41 includes one or more pixels 51 and the event detecting section 52. However, the pixel array section 31 of Fig. 11 is different from the case of Fig.3 in that the pixel block 41 does not include the pixel signal generating section 53.As described above, in the pixel array section 31 of Fig. 11, the pixel block 41 does not include the pixel signal generating section 53, so that the sensor section 21 (Fig. 2) can be formed without the AD conversion section 34.Fig. 12 is a circuit diagram illustrating a configuration example of the pixel block 41 of Fig. 11.As described with reference to Fig. 11, the pixel block 41 includes the pixels 51 and the event detecting section 52, but does not include the pixel signal generating section 53.In this case, the pixel 51 can only include the photoelectric conversion element 61 without the transfer transistors 62 and 63.Note that, in the case where the pixel 51 has the configuration illustrated in Fig. 12, the event detecting section 52 can output a voltage corresponding to a photocurrent from the pixel 51, as a pixel signal.Fig. 13 is a block diagram illustrating a configuration example of a scan type imaging device which may be used as an EVS.As illustrated in Fig. 13, an imaging device 510 includes a pixel array section 521, a driving section 522, a signal processing section 525, a read-out region selecting section 527, and an optional signal generating section 528.The pixel array section 521 includes a plurality of pixels 530. The plurality of pixels 530 each output an output signal in response to a selection signal from the read-out region selecting section 527. The plurality of pixels 530 can each include an in-pixel quantizer as illustrated in Fig. 10, for example. The plurality of pixels 530 outputs output signals corresponding to the amounts of change in light intensity. The plurality of pixels 530 may be two-dimensionally disposed in a matrix as illustrated in Fig. 13.The driving section 522 drives the plurality of pixels 530, so that the pixels 530 output pixel signals generated in the pixels 530 to the signal processing section 525 through an output line 514. Note that, the driving section 522 and the signal processing section 525 are circuit sections for acquiring grayscale information.The read-out region selecting section 527 selects some of the plurality of pixels 530 included in the pixel array section 521. For example, the read-out region selecting section 527 selects one or a plurality of rows included in the two-dimensional matrix structure corresponding to the pixel array section 521. The readout region selecting section 527 sequentially selects one or a plurality of rows on the basis of a cycle set in advance, e.g. based on a rolling shutter. Further, the read-out region selecting section 527 may determine a selection region on the basis of requests from the pixels 530 in the pixel array section 521.The optional signal generating section 528 may generate, on the basis of output signals of the pixels 530 selected by the read-out region selecting section 527, event signals corresponding to active pixels in which events have been detected of the selected pixels 530. The events mean an event that the intensity of light changes. The active pixels mean the pixel 530 in which the amount of change in light intensity corresponding to an output signal exceeds or falls below a threshold set in advance. For example, the signal generating section 528 compares output signals from the pixels 530 with a reference signal, and detects, as an active pixel, a pixel that outputs an output signal larger or smaller than the reference signal. The signal generating section 528 generates an event signal (event data) corresponding to the active pixel.The signal generating section 528 can include, for example, a column selecting circuit configured to arbitrate signals input to the signal generating section 528. Further, the signal generating section 528 can output not only information regarding active pixels in which events have been detected, but also information regarding non-active pixels in which no event has been detected.The signal generating section 528 outputs, through an output line 515, address information and timestamp information (for example, (X, Y, T)) regarding the active pixels in which the events have been detected. However, the data that is output from the signal generating section 528 may not only be the address information and the timestamp information, but also information in a frame format (for example, (0, 0, 1, o, -)).In the following description reference will mainly be made to sensor devices of the EVS type as described above in order to ease the description and to cover an important application example. However, the principles explained below apply just as well to any asynchronously generated data. In the following it will be described how an asynchronous data stream can be converted into data suitable for synchronous / periodic and / or on-demand processing without sacrificing the high temporal resolution and the low latency of asynchronous data generation.To this end, Figs. 14A to 14C illustrate in a simplified and schematic manner the generation of a feature vector V from a number of data tuples D that may have been generated asynchronously. Here, each data tuple is a set of ordered entries, in particular numbers, that indicate measurement results, internal states of a machine or an apparatus, or the like. The length of the data tuples is comparatively short, i.e. data tuples D have usually maximal 5, 10, or 20 entries. The state that is represented by all the data tuples D that are processed is to be mapped to a feature vector V, i.e. a series of numbers that shall represent the current state of the data tuples in further processing steps. To this end, the length of the feature vector V, i.e. the number of its entries, is larger than the length of each data tuple D. For example, the feature vector V can have more than 500 or 1,000 entries, e.g. 1024 entries. This ensures that the vector space of the feature vector V is large enough to allow a representation of all (or most) of the states indicated by the data tuples D.The feature vector(s) V are used in further processing steps. The feature vector(s) may in particular be used as input to a machine learning algorithm that operates on specific input states to solve particular tasks. The machine learning algorithm may for example be a neural network or a transformernetwork / algorithm. The task to be solved may for example be image or gesture recognition, image classification, a contextual search, an error analysis, a translation problem, image or text generation, or the like.The processing device 100 that is used for the conversion of the series of asynchronously generated data tuples D into the feature vector V may be any device that is capable to carry out the functions described below. In particular, the processing device 100 may be any chip, processor or computer that can carry out these functions. The processing device 100 may be implemented as software, as hardware or a mixture of hardware and software. The processing device 100 may be a CPU, a GPU, an ASIC, an FPGA or a combination of such components.As schematically illustrated in Fig. 14A, the processing device 100 receives a number of data tuples D that may be generated outside the processing device 100 or by the processing device 100 itself. The data tuples D may be generated at different times, i.e. asynchronously. In the context of Fig. 14A it is assumed that a specific number of data tuples D has already been generated before the conversion process starts. Asynchronous update after initial conversion will be described below with respect to Figs. 14B and 14C. However, it should be noted that the specific number of data tuples D may be rather small, e.g. below 5 or 10, or may even be one. Moreover, as described further below the generation of the feature vector V may only start if a sufficient number of data tuples D has already been received.The processing device 100 generates an intermediate vector I for each data tuple D. The intermediate vector I can be considered to indicate the state represented by the single data tuple D. Thus, also the intermediate vectors I have more entries than the data tuples D. The intermediate vectors I may be shorter or of the same length than the feature vector V. The feature vector V is then generated by taking all intermediate vectors I that are present into account. Thus the feature vector V represents the state formed by the single states of all data tuples D / intermediate vectors I.The above-described generation of the feature vector V is in its own not suitable to mitigate the abovedescribed problems, i.e. the loss of the advantages of high temporal resolution and low latency in the conversion of asynchronously generated data to data suitable for a synchronous further processing. In fact, if feature vectors V are generated only after a specific number of data tuples D have been generated / received or after a predetermined time interval, the processing will be dictated by the generation rate and / or the time cycle provided by the predetermined time interval. This will lead to a loss of temporal resolution. Moreover, if a particular number of data tuples need to be gathered during all instances of the processing the computational effort, and hence the energy consumption, will become high, which equals an increased latency.To avoid these drawbacks the feature vector is not calculated anew from time to time based on all available data tuples, but it is updated based on the reception of additional data tuples D’ as explained below with respect to Figs. 14B and 14C. This ensures that whenever new information is available in the form of at least one additional data tuple D’, this information is directly forwarded into an update of the feature vector V to generate a new feature vector V’. Thus, the feature vector update can be provided withthe same temporal resolution as the generation of data tuples. Further, since each new data tuple D’ is treated directly after its reception, the computational effort and the energy consumption (less area, less memory) is kept low, which implies a low latency.The update consists here of two parallel approaches. On the one hand, as stated above, it has to be ensured that new information is transferred directly and with low latency to the feature vector V. On the other hand, it has to be ensured that old information does not prevail and thus block the incorporation of new information, since in the end one is most interested in the current state represented by the data tuples D that have been received most recently. Thus, information carried over from older data tuples D must be removed from the feature vector V.The first part of the update, i.e. the incorporation of new data tuples D’ is illustrated in Fig. 14B. Here, a new data tuple D’ is received and an additional intermediate vector I’ is generated for said new data tuple D’. Then, a new feature vector V’ is generated based on the previous feature vector V and the additional intermediate vector I’ of the new data tuple D’. As represented by the hatched portions of V’ in Fig. 14B, the new feature vector V’ will contain some updated entries (hatched), while most of its entries remain unchanged. This corresponds to the fact that a single additional data tuple D’ will not turn over the entire state represented by all data tuples D processed so far. Accordingly, it can also be expected that most values of the new feature vector V’ remain the same during the update, while only some are updated.Thus, when a new data tuple D’ is received the new feature vector V’ is generated by recursively updating the previous feature vector V. This allows direct incorporation of additional information in a computationally non-complex manner. Here, it should be noted that although a direct processing of new data tuples D’ is preferably in view of temporal resolution and latency, it might be preferable for the reduction of computational load if several new data tuples D’ are gathered before updating the feature vector V. Thus, the number of new data tuples D’ may be monitored and the update can be triggered based on a condition on these new data tuples D’, e.g. on their number. Alternatively, the update could be performed periodically. Here, the update frequency should be sufficiently high to maintain a temporal resolution and latency that is as close as possible to that of the data tuple generation. For example, updates could be performed after 1, 2, 4, or 10 ms.In order to make room for the new information, it is monitored for each entry of the feature vector V whether a change of value occurs. If for one of the entries of the feature vector V no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time. This is illustrated in Fig. 14C. Here, two entries of the feature vector V are indicated that were not updated during the predetermined time. Then, the values of these entries will be reduced with increasing time in new feature vectors V’ as indicated by the hatched blocks on the right-hand side of Fig. 14C.This allows to remove information from the feature vector V that is by itself too predominant to be overwritten by newly arriving data tuples D. After the predetermined time, which defines basically how “current” the information in the feature vector has to be by defining a time window of interest, nonupdated values within the feature vector will be decreased in order to reduce their significance. Thepredetermined time may e.g. be in the same range as the aforementioned update frequency, i.e. in the millisecond range. It could, however, also be longer. The decrease in value will lead eventually to an overwriting of these decreased values in an updating process that is caused by the reception of additional data tuples D’. In this manner, old information is “decayed”, which can be seen as a prerequisite for the above described asynchronous and low latency updating process.Thus, the feature vector V provided by the processing device 100 is renewed as asynchronous and as temporally resolved as the generation / reception of the data tuples D. Thus, neither resolution nor speed is lost in the conversion process. Accordingly, processing steps that are based on the feature vector V can always operate on an up-to-date state.The above-described algorithm works in principle for all data tuples D. Practically important is here the example where the data tuples refer to events generated in an event vision sensor.To this end, as illustrated in Fig. 15, a sensor device 10 may be provided that comprise a processing device 100 as described above and a vision sensor 1010. The vision sensor 1010 and the processing device 100 may be arranged at different chips / sensor dies. As illustrated in Fig. 15, the vision sensor 1010 may be located on the sensor die 11 and the processing unit 100 may be located on the logic die 12.The vision sensor 1010 comprises a pixel array 1011 having a plurality of pixels 51 each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, based on which electrical signals event data are generated, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel 51 above an event detection threshold. Thus, the vision sensor 1010 may constitute an EVS as described above with respect to Figs. 1 to 13. The pixel array 1011 may be formed solely of event detection pixels or may be a hybrid sensor array comprising a mixture of event detection pixels and pixels that generate an electrical signal that indicates the absolute intensity of the received light. Also, the pixels 51 of the pixel array 1010 may have the ability to generate both, event data and intensity data, as was described above with respect to Figs. 3 and 4.The vision sensor 1010 is configured to generate one data tuple D for each event, the data tuple D indicating at least the time of event generation and the coordinates of the event on the pixel array 1011.Thus, every event occurring in the pixel array 1011 gives rise to a data tuple D and an intermediate vector I, which represents a state of the single event. To this end, the time t at which the event was generated is included in the data tuple, as well as the coordinates x and y of the respective pixel 51 in the pixel array 1011. Moreover, if the vision sensor 1010 is configured to determine the occurrence of a positive polarity event when the intensity change of the received light indicates an increase in intensity and a negative polarity event when the intensity change of the received light indicates a decrease in intensity, then also the polarity p of the respective event may be included in the data tuple. This is indicated in Fig. 15 by the data tuple D = (t, x, y, p).Here, it should be noted that although the feature vector V may in principle by generated based on allintermediate vectors I of all the events generated on the pixel array 1011, it is preferable to separate the pixel array 1011 into different areas and to generated feature vectors V for each of these areas. This allows to represent the imaging state captured via the events more precisely. A detailed description of this setup will be described below.In any case, the application of the above-described algorithm to events allows the conversion of asynchronously generated event data into feature vector(s) that are typically used for the analysis of synchronous, frame-based images. This allows to combine the advantages of event-based vision sensors, i.e. high temporal resolution and low latency, with the powerful image classification and analysis algorithms that are known for frame-based imaging. Accordingly, the above-described algorithm allows to provide highly reliable image classification and analysis tools, like e.g. face / object recognition, gesture recognition, image classification, image segmentation, or the like, with a high temporal resolution and a low latency. Moreover, the high temporal resolution allows also improving the precision and reliability of such tools.This application can be further improved by using a cyclic representation of time instead of the linearly increasing (natural) time. In fact, such a cyclic representation of time allows improving the applicability of artificial intelligence models to the generation of intermediate vectors I from data tuples D, which would be difficult to achieve when using a monotonically, linearly and infinitely increasing time coordinate. Using a cyclic time makes in particular a separation of time into different time blocks (of finite length) superfluous that would otherwise be necessary.In particular, as also represented by the dashed data tuple D in Fig. 15, each data tuple D may indicate time based on two values tx and ty of a sinusoidal representation of time according to the equations tx = a cos (cot) + q>, ty = a sin (cot) + cp, with a, co, and cp being parameters and t being (natural / real) time. However, also several “time” values may be used, e.g. by making the parameters a, co, and cp dependent on time and / or the position of the respective event on the pixel array 1011.When using an event vision sensor for the generation of data tuples, also the criteria when to reduce values of the feature vector V can be based on the generation of events.For example, the processing device 100 may count for each entry of the feature vector V the number of events Ne that occurred since the value of the respective entry did change the last time. This is illustrated in Fig. 16, where a (simplified) feature vector V with nine entries is shown. The processing unit 100 keeps one counter for each entry. After each occurrence of an event (either across the entire pixel array 1011 or within the area of the pixel array 1011 that contributed to the generation of the feature vector V) or after a predetermined time period (e.g. in the ms range), the counter corresponding to each entry of the feature vector V that was not updated is increased by one. Those counters (Ne2 and Ne5 in Fig. 16) whose corresponding entry was updated are reset to zero. Thus, each counter will hold a (probably different) value, indicating how long (either in real time or measured based on the event occurrence rate) no update did occur.If the value of Ne becomes larger than a predetermined update threshold N, the value of the respective entry will be decreased by over time. For example, the value will be decreased at each occurrence of an event, or after a predetermined time period (e.g. in the ms range). Here, the value A may be fixed, leading to a constant exponential decrease. However, A may also depend on how long after the transgression of the threshold the respective value is not updated and may increase with time. For example A = (Ne - N) / N may hold. Then, the exponential decrease will become faster with time. An exponential decrease is preferable, since this will not lead to negative numbers. However, also any other decrease, e.g. a linear decrease, is possible, as long as the decrease is stopped at zero.In this manner, the influence of older events (or event configurations) can be reduced over time. Moreover, the update threshold N allows setting a range of current events that are of interest and therefore not decreased. Of course, it is also possible to decrease all entries of the feature vector at all times. Also in this manner it is ensured that the events that were detected the latest have the most influence. In this case, it is not necessary to count the time between updates of an entry of the feature vector V. This simplifies the setup of the processing device 100.A particular simple manner of generating feature vectors V from intermediate vectors I and of updating feature vectors V is to use intermediate vectors I and feature vectors V that have the same length.Then, the processing device 100 can generate the feature vector V by taking for each entry of the feature vector V the maximum value of the corresponding entries of all intermediate vectors I, i.e. the processing unit applies a channel-wise max operation to the intermediate vectors.This is illustrated in Fig. 17A in a simplified manner. Here, four intermediate vectors I and one feature vector V are shown. The hatched entries of the intermediate vectors I represent the highest values in the respective entries (or channels) of the intermediate vectors I. Only these highest values are transferred to the feature vector V. As can be seen, the first intermediate vector I did not contribute anything to the feature vector V, while the third intermediate vector I contributes most. This reflects the importance of the respective event states represented by the different intermediate vectors I.The feature vector V can therefore be generated in a rather simple manner from the intermediate vectors I.In the generation of the new feature vector V’ each entry of the new feature vector V’ is the maximum value of the corresponding entries of the previous feature vector V and the intermediate vector I’ that was generated for the new data tuple D’.Thus, as shown in Fig. 17B only those values of the previous feature vector V are updated in the new feature vector V’ for which the additional intermediate vector F has a higher value (hatched entries in Fig. 17B). Accordingly, only those channels are updated for which the additional intermediate vector F carries larger values, which can be interpreted as more relevant information.In this regard it also has to be noted that applying the channel-wise max operation during update after arrival of new data tuples is particularly advantageous in view of the successive decrease of values of entries that were not updated previously. By decreasing these non-updated values with time they must eventually become smaller than corresponding values in the additional intermediate vectors. This ensures that information carried by events that are no longer of interest is reliably removed from the feature vector V. Thus, further processing that is based on the feature vector V provides always up-to-date results.Preferably, the processing device 100 is configured to generate the intermediate vectors I by using an artificial intelligence model. This model can be trained to extract relevant information for the task to which the feature vector serves as input. For example, if also this task is carried out by an artificial intelligence algorithm, as is nowadays common for image processing tasks like feature or gesture recognition, both networks can be trained together, e.g. by scoring the correctness of the final task result and thereby deducing a measure for the quality of the feature vector(s) V and the intermediate vectors I on which the feature vector(s) V are based. As this is in principle known to a skilled person, further description will be omitted here.The artificial intelligence model used to generate intermediate vectors I may be a neural network and in particular a plurality of multi-layer perceptrons, MLPs, that receives the data tuples D as input and provides the intermediate vectors I as output. An exemplary illustration of such a plurality of MLPs is given in Fig. 18A. Here it is shown that the data tuples D are fed through several MLPs until the intermediate vectors I are generated. In order to ensure that all data tuples D are treated the same, the MLPs are shared between the data tuples D, i.e. each data tuple D passes through the same series of MLPs. In particular, the weights of the MLPs will be the same for each data tuple D.The principle setup of an MLP is known to a skilled person. As exemplary illustrated in Fig. 18B an MLP has an input layer to which the entries of the data tuples D are fed, for example the values (tx, ty, x, y, p) of an event that have been described above. Each MLP has a series of hidden layers. In the example of Fig. 18B two hidden layers LI and L2 are shown. The hidden layers have a number of nodes that is predetermined and chosen to fit the task the MLP has to fulfill. The output layer provides the intermediate vector I corresponding to the input data tuple D, if only a single MLP is used. If several MLPs are used, the output serves as input for the next MLP. The nodes of each layer are connected with each node of the following layer but are not otherwise connected.By using a series of shared MLPs meaningful intermediate vectors I can be generated for the incoming event data tuples D. From these, feature vectors V can be generated that fit the task for which the feature vectors V serve as input. The generation of intermediate vectors can for example be carried out according to the principles laid down by Qi et al. in “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation” (2 December 2016, Computer Science 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)) that is incorporated for reference in this regard. However, every other artificial intelligence algorithm can be used in principle, as long as the resulting intermediate vectors I and feature vectors V can be trained to fit to the corresponding task for which they are used as input. For example, instead of the one-dimensional convolution provided by PointNet one could also use eventdriven two-dimensional convolutions or other algorithms.Above, description has been given for the generation of a single feature vector V for an event stream and the corresponding data tuples D. Although one could in principle deal with all events detected by the vision sensor 1010 by a single vector V, it is preferable to deal with events coming from different regions of the pixel array 1011 separately.To this end, the vision sensor 1010 and / or the processing device 100 is configured to divide the pixel array 1011 into different, preferably non-overlapping, areas A. The processing device 100 generates and updates then one feature vector V per area A only based on data tuples D that are generated for events that occurred at a pixel 51 within said area A. That is, a set of feature vectors V is generated for the events that occurred, where the maximum number of feature vectors V is equal to the number of areas A.This is schematically illustrated in Fig. 19, where a separation of the pixel array 1011 into nine areas A is shown together with the events of positive (black dots) and negative polarity (grey dots) that occurred during a certain time period. Of course, the number of nine areas A is purely exemplary and not limiting. Areas A might be formed by any number of pixels 51, like e.g. 2, 10, 8x8, or 100 pixels. Moreover, areas A might have arbitrary shapes. They may be rectangular or quadratic but may also be line shaped or comer shaped, e.g. to support detection of lines or edges in specific orientations that are expected to produce series of events in according orientations. The areas A may also overlap, e.g. when line shaped areas for vertical, horizontal, and 45° angled lines are used. Also the number and type of division may change with time. For the sake of simplicity the following discussion will focus on rectangular, nonoverlapping areas A of pixels 51.The events of each of these areas A lead to event data tuples D. The above-described conversion of these data tuples D into intermediate vectors I is then performed for each area A solely based on the events that occurred in the respective area A. The intermediate vectors I generated for a specific area A represent therefore only the state of the events that occurred in this area A. Consequently, the resulting feature vector V is a representation of the state of the area A, i.e. of the scene imaged to area A. Updating of the feature vector V is then done based on the new events that occurred in the corresponding area A. Decreasing feature vector values could be based on counting the number of events occurring in the respective area, on counting all the events that are occurring, or on an intermediate solution like counting events in the respective area and in neighboring areas.In this manner, a selective update of feature vectors V relating to different areas of the vision sensor 1010 is possible. This translates the spatial sparsity of events to the feature vectors V, since only those feature vectors V will change, for which the observed scene did change. This allows having a larger number of feature vectors V while keeping the computational burden low, since the spatial sparsity of events guarantees that also feature vector updates will be spatially sparse. Accordingly, an update of a large number of feature vectors V at one time is unlikely, which allows setting up the processing power such as to deal with substantially less feature vectors V than the total number of feature vectors V.As illustrated in Fig. 19, dividing the pixel array 1011 into different areas may come with the necessity to provide the feature vector V with position information that indicates the position of the areas A within the pixel array 1011. Although this position information is in principle included in the data tuples D, it will be lost during the generation of the intermediate vectors I. In particular, in order to treat all areas A on an equal basis for the generation of intermediate vectors I, it might be necessary to transform the global coordinates x, y of the events into local area coordinates by subtracting the area position (e.g. the global coordinates of a lower left pixel) from the global coordinates of the pixel. Further, even when not carrying out such a step, it is not guaranteed that it would be possible to reconstruct the area A from the values of the intermediate vectors I or from the feature vector V.Although possible, the processing device 100 does preferably not simply indicate position information in the form of coordinates. The processing device 100 may instead, preferably by an artificial intelligence model, generate for each area A a positional embedding vector P that can be combined with the respective feature vector V to form an embedding vector E of said area A. How to generate the positional embedding vector P is in principle known to a skilled person. In particular, finding the positional embedding vector P or positional encoding is a problem known from setting up transformer networks / algorithms, used e.g. for natural langue processing. A detailed description thereof will therefore be omitted. The embedding vector E will then be used as input for the further processing steps.In this manner, it is possible to generate input data for further processing steps that is spatially and temporally highly resolved and keeps the computational burden low due to its spatial sparsity, i.e. it is possible to transfer the advantages of event generation to the subsequent processing stage.In order to further reduce the processing load, the processing device 100 may start the generation of feature vectors V only for those areas A in which during a predetermined time interval a number of events occurred that is equal to or higher than an output threshold, and stop the generation of feature vectors V for those areas A in which the number of events that occurred during the predetermined time interval is lower than the output threshold.This means that no feature vector V is provided for a specific area A, if the number of events was too low in this area during a preceding time interval of given length. As areas A with no or only a small number of events contain most probably no important information, respectively generated feature vectors V would not contribute (much) in the subsequent processing stage. Omitting generation of such feature vectors V reduces therefore the processing burden without deteriorating the final outcome of the processing.Fig. 20 schematically illustrates the above principle based on the example of pixel array division of Fig. 19. As shown in Fig. 20 the right area in the top row as well as the middle and the right areas in the middle row contain only few events. Thus, for these areas A no feature vector V is generated. The time interval that is used to determine whether to start and / or stop feature vector generation could be a sliding window, which would emphasize the asynchronous character. However, also a consecutive series of time intervals could be used, at the end of which it is checked, in which areas the number of events is abovethe output threshold. The time interval may be in the ms range, i.e. it may have values of 1 ms, 2 ms, 4 ms or 10 ms. The output threshold may e.g. be between 25% and 50% of the number of pixels in an area A. For example, for areas of 8x8 pixels 51 a threshold of 24 events may be used. The output threshold may also depend on the number of feature vectors V that can be accepted by the further processing stages, in that feature vectors V are only generated for a number of areas equal to the number of acceptable feature vectors V, which areas have the highest numbers of events. Moreover, different thresholds might be used for the start of feature vector generation and for the end of feature vector generation. In particular, the threshold for starting might be higher than the threshold for ending the feature vector generation, meaning that after the start a drop of the event number in that area will not lead to an immediate stop of feature vector generation / updating.As indicated above the output of the processing device 100 serves as input for a further processing stage. This further processing stage could be part of a processing system 1 that comprises next to a sensor device 10 as described above a task processing unit 200 that executes, preferably by using an artificial intelligence model, predetermined processing based on the feature vectors V (or embedding vectors E; in the following the reference to feature vectors V shall be understood to refer equivalently to embedding vectors E) generated by the processing device 100. In particular, the task processing unit 200 may carry out in principle arbitrary tasks based on the feature vectors V, such as e.g. image generation, image / object recognition, image segmentation, image classification, gesture recognition or the like. The task processing unit 200 may carry out said processing e.g. based on a transformer algorithm or other types of neural networks that are in principle known to a skilled person. The artificial intelligence model(s) of the task processing unit 200 and the artificial intelligence models of the processing device 100 are preferable trained together such as to ensure that the feature vectors V generated by the processing device 100 allow the task processing unit 200 achieving of optimal results.A schematic illustration of the sensor system 1 is given in Fig. 21. As can be seen in Fig. 21, the processing device 100 provides and / or stores the feature vectors V in its current form. The task processing unit 200 requests provision of the feature vectors V according to its own processing schedule, which might e.g. be a periodic readout or a readout on demand. Also, the task processing unit 200 may provide feedback to the processing device 100 or the vision sensor 1010 as indicated by the dashed arrows. It should be noted that in principle also the task processing unit 200 could store the feature vectors V, however, storage at the processing device 100 is preferable as explained below.Preferably, the task processing unit 200 is able to update a result of its predetermined processing only based on feature vectors V that have been updated or that have been newly generated since the result of its predetermined processing was determined the last time. This means that only feature vectors V that are not know to the task processing unit 200 must be forwarded to it, which reduces the latency and the computational load of the processing. The task processing unit 200 may comprise a storage or buffer, where it can store feature vectors V that had already been used for the processing. Alternatively or additionally the processing device 100 provides information which feature vectors V did not change to allow the task processing unit 200 to reuse these feature vectors V without the necessity for a renewed transmission. As stated above, the processing device 100 may provide feature vectors V to the taskprocessing unit 200 only upon a request of the task processing unit 200. This is schematically illustrated in Fig. 21, where the request of the task processing unit 200 to the processing device 100 leads to a provision of only updated / new feature vectors V (hatched in Fig. 21).In this manner the task processing unit 200 can be provided with asynchronously updated feature vectors V which ensures that the output of the task processing unit 200 rely on the current state of the observed scene. At the same time, the task processing unit 200 is allowed to reuse information that was already provided and to re-process its result only based on feature vectors V that were updated since the last processing cycle. In this manner, the processing burden and hence the energy consumption is reduced, processing becomes faster, and the latency is reduced.Here, the processing device 100 may apply a sequence analysis based triggering of feature vector output by providing feature vectors V to the task processing unit 200 only if a predetermined number of feature vectors V has been updated or newly generated since the last request from the task processing unit 200. Thus, the processing device 100 may gather and hold back updated feature vectors V until a specific number of feature vectors V has been changed, even upon a request of the task processing unit 200. Only when a sufficient number of new / updated feature vectors has been generated, all these feature vectors V are sent to the task processing unit 200 at once. This ensures that minor changes in the scene do not necessitate re-computation of the predetermined processing of the task processing unit 200, which will lead most probably to the same result. Only if it can be assumed that the changes in the feature vectors V will also lead to a change of the results of the predetermined processing, the processing device 100 will forward the new / updated feature vectors V. In this manner, the computational burden can be decreased further.Additionally or alternatively, the decision which feature vectors V to provide to the task processing unit 200 could be based on other criteria, too, such as e.g. the area A to which the feature vector V belongs (central vs. peripheral), the time since the last provision of feature vectors V, the previous result of the predetermined processing or the like. This makes the processing additionally flexible. Further, also feature vectors V whose values were decreased due to missing updates could be considered updated feature vectors, if the decrease is sufficiently large. Thus, after a certain time, feature vectors V will be provided to the task processing unit 200, even if there are no new events detected. Then, the result of the predetermined processing will reflect the fact that the information contained in the feature vectors V has become less relevant.An exemplary, non-limiting layout of a hardware or software architecture forming the processing device 100 is illustrated schematically in Fig. 22. The processing device 100 contains a patch addressing module 110 and an event to vector processor 115. The patch addressing module 110 contains a raw to relative address module 110a and a global event status memory 110b. The raw to relative module 110a receives the event data tuple D containing the position of the event in absolute coordinates and converts the absolute position coordinates to relative coordinates of the respective area A (also called patch). The raw to relative module 110a may also transform the time coordinate to cyclic time coordinates. The global event status memory 110b also receives the event data tuple D and stores global event information such ase.g. the number of events, the number of events per area and the like.The event to vector processor 115 contains an intermediate vector generation module 120 with a weight memory 125 that stores trained weights of neural networks like MLPs. The intermediate vector generation module 120 carries out the generation of intermediate vectors I. The generated intermediate vectors I are processed to feature vectors V in the feature vector generation module 140 that also stores the feature vectors V. Decrease of values of feature vectors V is controlled by decrease module 130 and updating of feature vectors V is performed by a loop between intermediate vector generation module 120 and feature vector generation module 140.The event to vector processor 115 further contains a positional embedding vector module 150 that generates and stores the positional embedding vector P for all areas A. Control logic 160 controls the functioning of event to vector processor 115, in particular by controlling the decrease module 130, the feature vector generation module 140 and the positional embedding vector module 150. The control logic 160 also receives requests for feature vectors V from the task processing unit 200.The event to vector processor 115 further contains an output formatter 170. The output formatter 170 combines in adder 170a the feature vectors V with their corresponding positional embedding vector P to form the embedding vector E that is provided to the task processing unit 200 as input. The output formatter 170 also contains active area memory 170b that stores the addresses of the areas A for which feature / embedding vectors have been generated or the feature / embedding vectors that have been notified by the control logic 160 to be allowed to be provided to the task processing unit 200.It goes without saying that the layout of Fig. 22 is purely exemplary and that any other architecture that allows carrying out the above functions will also be possible.Fig. 23 shows a schematic process flow of a method executed by a processing device 100 as described above for converting a series of asynchronously generated data tuples D into a feature vector V.Preferably, as described above, the method is carried out by a sensor device 10 that detects events. In this case the method commences at S101 with forming event data with the sensor device 10 based on the generated electrical signals, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel 51 above an event detection threshold, and continues at SI 02 with generating one data tuple D for each event, the data tuple D indicating at least the time of event generation and the coordinates of the event on the pixel array 1011.However, in general these parts of the method are omissible, and the method commences at SI 03 with receiving a number of data tuples D. At SI 04 for each data tuple D thus received an intermediate vector I is generated. At SI 05 a feature vector V is generated based on all intermediate vectors I. And at SI 06 the feature vector V is updated based on the reception of additional data tuples D.Here, as indicated at S 107 when a new data tuple D’ is received, an intermediate vector I’ is generated forsaid new data tuple D’ and a new feature vector V’ is generated based on the previous feature vector V and the intermediate vector I’ of the new data tuple D’. Further, at S108 for each entry of the feature vector V it is monitored whether a change of value occurs, and if for one of the entries of the feature vector V no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.This allows an asynchronous generation and updating of feature vectors V suitable for further processing by machine learning models, which generation and updating has a high temporal resolution and a low latency. This in turn improves the results of the further processing since the high temporal resolution and low latency will be transferred to this processing step via the feature vectors V.The technology according to the above (i.e. the present technology) is applicable to various products. For example, the technology according to the present disclosure may be realized as a device that is installed on any kind of moving bodies, for example, vehicles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobilities, airplanes, drones, ships, and robots.Fig. 24 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied.The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In the example depicted in Fig. 24, the vehicle control system 12000 includes a driving system control unit 12010, a body system control unit 12020, an outside -vehicle information detecting unit 12030, an in-vehicle information detecting unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output section 12052, and a vehicle-mounted network interface (I / F) 12053 are illustrated as a functional configuration of the integrated control unit 12050.The driving system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the driving system control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like.The body system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 12020. The body systemcontrol unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle.The outside-vehicle information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000. For example, the outside-vehicle information detecting unit 12030 is connected with an imaging section 12031. The outside-vehicle information detecting unit 12030 makes the imaging section 12031 image an image of the outside of the vehicle, and receives the imaged image. On the basis of the received image, the outside-vehicle information detecting unit 12030 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto.The imaging section 12031 is an optical sensor that receives light, and which outputs an electric signal corresponding to a received light amount of the light. The imaging section 12031 can output the electric signal as an image, or can output the electric signal as information about a measured distance. In addition, the light received by the imaging section 12031 may be visible light, or may be invisible light such as infrared rays or the like.The in-vehicle information detecting unit 12040 detects information about the inside of the vehicle. The in-vehicle information detecting unit 12040 is, for example, connected with a driver state detecting section 12041 that detects the state of a driver. The driver state detecting section 12041, for example, includes a camera that images the driver. On the basis of detection information input from the driver state detecting section 12041, the in-vehicle information detecting unit 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver, or may determine whether the driver is dozing.The microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device on the basis of the information about the inside or outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040, and output a control command to the driving system control unit 12010. For example, the microcomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like.In addition, the microcomputer 12051 can perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040.In addition, the microcomputer 12051 can output a control command to the body system control unit12020 on the basis of the information about the outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030. For example, the microcomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detecting unit 12030.The sound / image output section 12052 transmits an output signal of at least one of a sound and an image to an output device capable of visually or auditorily notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of Fig. 24, an audio speaker 12061, a display section 12062, and an instrument panel 12063 are illustrated as the output device. The display section 12062 may, for example, include at least one of an on-board display and a head-up display.Fig. 25 is a diagram depicting an example of the installation position of the imaging section 12031.In Fig. 25, the imaging section 12031 includes imaging sections 12101, 12102, 12103, 12104, and 12105.The imaging sections 12101, 12102, 12103, 12104, and 12105 are, for example, disposed at positions on a front nose, sideview mirrors, a rear bumper, and a back door of the vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle. The imaging section 12101 provided to the front nose and the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle 12100. The imaging sections 12102 and 12103 provided to the sideview mirrors obtain mainly an image of the sides of the vehicle 12100. The imaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle 12100. The imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.Incidentally, Fig. 25 depicts an example of photographing ranges of the imaging sections 12101 to 12104. An imaging range 12111 represents the imaging range of the imaging section 12101 provided to the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging sections 12102 and 12103 provided to the sideview mirrors. An imaging range 12114 represents the imaging range of the imaging section 12104 provided to the rear bumper or the back door. A bird’s-eye image of the vehicle 12100 as viewed from above is obtained by superimposing image data imaged by the imaging sections 12101 to 12104, for example.At least one of the imaging sections 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.For example, the microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect tothe vehicle 12100) on the basis of the distance information obtained from the imaging sections 12101 to 12104, and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of the vehicle 12100 and which travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, equal to or more than 0 km / hour). Further, the microcomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance, and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automatic driving that makes the vehicle travel autonomously without depending on the operation of the driver or the like.For example, the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a largesized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance information obtained from the imaging sections 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that the driver of the vehicle 12100 can recognize visually and obstacles that are difficult for the driver of the vehicle 12100 to recognize visually. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle. In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062, and performs forced deceleration or avoidance steering via the driving system control unit 12010. The microcomputer 12051 can thereby assist in driving to avoid collision.At least one of the imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 can, for example, recognize a pedestrian by determining whether or not there is a pedestrian in imaged images of the imaging sections 12101 to 12104. Such recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of the imaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether or not it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there is a pedestrian in the imaged images of the imaging sections 12101 to 12104, and thus recognizes the pedestrian, the sound / image output section 12052 controls the display section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian. The sound / image output section 12052 may also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position.An example of the vehicle control system to which the technology according to the present disclosure is applicable has been described above. The technology according to the present disclosure is applicable to the imaging section 12031 among the above-mentioned configurations. Specifically, the sensor device 10 is applicable to the imaging section 12031. The imaging section 12031 to which the technology according to the present disclosure has been applied flexibly acquires event data and performs dataprocessing on the event data, thereby being capable of providing appropriate driving assistance.Further possible implementations of the sensor device 10 are mobile devices 3000 such as cell phones, tablets, smart watches and the like as shown in Fig. 26A or head-mounted displays 4000 as shown in Fig. 26B. Further, the sensor device 10 is useable in augmented and / or virtual reality applications / cameras or in surveillance systems like 360° cameras.Note that, the embodiments of the present technology are not limited to the above-mentioned embodiment, and various modifications can be made without departing from the gist of the present technology.Further, the effects described herein are only exemplary and not limited, and other effects may be provided.Note that, the present technology can also take the following configurations.[1] A processing device (100) for converting a series of asynchronously generated data tuples (D) into a feature vector (V), the processing device (100) being configured to receive a number of data tuples (D); generate for each data tuple (D) an intermediate vector (I); generate the feature vector (V) based on all intermediate vectors (I); and update the feature vector (V) based on the reception of additional data tuples (D’); wherein when a new data tuple (D’) is received, an intermediate vector (F) is generated for said new data tuple (D’) and a new feature vector (V’) is generated based on the previous feature vector (V) and the intermediate vector (F) of the new data tuple (D’); and for each entry of the feature vector (V) it is monitored whether a change of value occurs, and if for one of the entries of the feature vector (V) no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.[2] A sensor device (10) comprising: a vision sensor (1010) that comprises a pixel array (1011) having a plurality of pixels (51) each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, based on which electrical signals event data are generated, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel (51) above an event detection threshold; and the processing device (100) according [1]; wherein the vision sensor (1010) is configured to generate one data tuple (D) for each event, the data tuple (D) indicating at least the time of event generation and the coordinates of the event on the pixel array (1011).[3] The sensor device (10) according [2], wherein the vision sensor (1010) is configured to determine the occurrence of a positive polarity event when the intensity change of the received light indicates an increase in intensity and a negativepolarity event when the intensity change of the received light indicates a decrease in intensity; and each data tuple (D) indicates also the polarity of the respective event.[4] The sensor device (10) according to [2] or [3], wherein each data tuple (D) indicates time based on two values tx and ty of a sinusoidal representation of time according to the equations tx = a cos (cot) + cp, ty = a sin (cot) + cp, with a, co, and cp being parameters and t being time.[5] The sensor device (10) according to anyone of [2] to [4], wherein the processing device (100) is configured to count for each entry of the feature vector (V) the number of events Ne that occurred since the value of the respective entry did change the last time, and if the value of Ne becomes larger than a predetermined update threshold N, decrease at each occurrence of an event the value of the respective entry by cAwith A. = (Ne - N) / N.[6] The sensor device (10) according to anyone of [2] to [5], wherein the intermediate vectors (I) and the feature vector (V) have the same length; the processing device (100) is configured to generate the feature vector (V) by taking for each entry of the feature vector the maximum value of the corresponding entries of all intermediate vectors (I); and the processing device (100) is configured to generate the new feature vector (V’) by taking for each entry of the new feature vector (V’) the maximum value of the corresponding entries of the previous feature vector (V) and the intermediate vector (T) that was generated for the new data tuple (D’).[7] The sensor device (10) according to anyone of [2] to [6], wherein the processing device (100) is configured to generate the intermediate vectors (I) by using an artificial intelligence model, preferably a neural network and more preferably multi-layer perceptrons, MLP, that receives the data tuples (D) as input and provides the intermediate vectors (I) as output.[8] The sensor device (10) according to [2] to [7], wherein the processing device (100) is configured to divide the pixel array (1011) into different, preferably non-overlapping, areas (A); and the processing device (100) is configured to generate and update one feature vector (V) per area (A) only based on data tuples (D) that are generated for events that occurred at a pixel (51) within said area (A).[9] The sensor device (10) according to [8], wherein the processing device (100) is configured to generate, preferably by an artificial intelligence model, for each area (A) one embedding vector (E) by combining the feature vector (V) of said area (A) with a positional embedding vector (P) of said area (A).

[0010] The sensor device (10) according to [8] or [9], wherein the processing device (100) is configured to start generation of feature vectors (V) for those areas (A) in which during a predetermined time interval a number of events occurred that is equal to or higher than an output threshold; and the processing device (100) is configured to stop generation of feature vectors (V) for those areas (A) in which the number of events that occurred during the predetermined time interval is lower than the output threshold.

[0011] A processing system (1) that comprises the sensor device (10) according to anyone of [8] to

[0010] ; and a task processing unit (200) that is configured to execute, preferably by using an artificial intelligence model, predetermined processing based on the feature vectors (V) generated by the processing device (100); wherein the task processing unit (200) is configured to update a result of its predetermined processing only based on feature vectors (V) that have been updated or that have been newly generated since the result of its predetermined processing was determined the last time.

[0012] The processing system (1) according to

[0011] , wherein the processing device (100) is configured to store the feature vectors (V) and to provide feature vectors to the task processing unit (200) upon a request of the task processing unit (200).

[0013] The processing system (2) according to

[0012] , wherein the processing device (100) is configured to provide feature vectors (V) to the task processing unit (200) only if a predetermined number of feature vectors (V) have been updated or newly generated since the last request from the task processing unit (200).

[0014] A method executed by a processing device (100) for converting a series of asynchronously generated data tuples (D) into a feature vector (V), the method comprising: receiving a number of data tuples (D); generating for each data tuple (D) an intermediate vector (I); generating the feature vector (V) based on all intermediate vectors (I); and updating the feature vector (V) based on the reception of additional data tuples (D); wherein when a new data tuple (D’) is received, an intermediate vector (T) is generated for said new data tuple (D’) and a new feature vector (V’) is generated based on the previous feature vector (V) and the intermediate vector (T) of the new data tuple (D’); and for each entry of the feature vector (V) it is monitored whether a change of value occurs, and if for one of the entries of the feature vector (V) no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.

[0015] The method according to

[0014] , further comprising by a vision sensor (1010) that comprises a pixel array (1011) having a plurality of pixels (51) each being configured to receive light and to perform photoelectric conversion to generate an electricalsignal, forming event data based on the generated electrical signals, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel (51) above an event detection threshold; and generating, by the vision sensor (1010), one data tuple (D) for each event, the data tuple (D) indicating at least the time of event generation and the coordinates of the event on the pixel array (1011).

Claims

CLAIMS1. A processing device for converting a series of asynchronously generated data tuples into a feature vector, the processing device being configured to receive a number of data tuples; generate for each data tuple an intermediate vector; generate the feature vector based on all intermediate vectors; and update the feature vector based on the reception of additional data tuples; wherein when a new data tuple is received, an intermediate vector is generated for said new data tuple and a new feature vector is generated based on the previous feature vector and the intermediate vector of the new data tuple; and for each entry of the feature vector it is monitored whether a change of value occurs, and if for one of the entries of the feature vector no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.

2. A sensor device comprising: a vision sensor that comprises a pixel array having a plurality of pixels each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, based on which electrical signals event data are generated, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel above an event detection threshold; and the processing device according to claim 1 ; wherein the vision sensor is configured to generate one data tuple for each event, the data tuple indicating at least the time of event generation and the coordinates of the event on the pixel array.

3. The sensor device according to claim 2, wherein the vision sensor is configured to determine the occurrence of a positive polarity event when the intensity change of the received light indicates an increase in intensity and a negative polarity event when the intensity change of the received light indicates a decrease in intensity; and each data tuple indicates also the polarity of the respective event.

4. The sensor device according to claim 2, wherein each data tuple indicates time based on at least two values tx and ty of a sinusoidal representation of time according to the equations tx = a cos (cot) + cp, ty = a sin (cot) + cp, with a, co, and cp being parameters, preferably parameters that depend on time and / or space, and t being time.

5. The sensor device according to claim 2, wherein the processing device is configured to count for each entry of the feature vector the number of events Ne that occurred since the value of the respective entry did change the last time, and if the value of Ne becomes larger than a predetermined update threshold N, decrease at each occurrence of an eventthe value of the respective entry by eAwith A. = (Ne - N) / N.

6. The sensor device according to claim 2, wherein the intermediate vectors and the feature vector have the same length; the processing device is configured to generate the feature vector by taking for each entry of the feature vector the maximum value of the corresponding entries of all intermediate vectors; and the processing device is configured to generate the new feature vector by taking for each entry of the new feature vector the maximum value of the corresponding entries of the previous feature vector and the intermediate vector that was generated for the new data tuple.

7. The sensor device according to claim 2, wherein the processing device is configured to generate the intermediate vectors by using an artificial intelligence model, preferably a neural network and more preferably multi-layer perceptrons, that receives the data tuples as input and provides the intermediate vectors as output.

8. The sensor device according to claim 2, wherein the processing device is configured to divide the pixel array into different, preferably nonoverlapping, areas; and the processing device is configured to generate and update one feature vector per area only based on data tuples that are generated for events that occurred at a pixel within said area.

9. The sensor device according to claim 8, wherein the processing device is configured to generate, preferably by an artificial intelligence model, for each area one embedding vector by combining the feature vector of said area with a positional embedding vector of said area.

10. The sensor device according to claim 8, wherein the processing device is configured to start generation of feature vectors for those areas in which during a predetermined time interval a number of events occurred that is equal to or higher than an output threshold; and the processing device is configured to stop generation of feature vectors for those areas in which the number of events that occurred during the predetermined time interval is lower than the output threshold.

11. A processing system that comprises the sensor device according to claim 8; and a task processing unit that is configured to execute, preferably by using an artificial intelligence model, predetermined processing based on the feature vectors generated by the processing device; wherein the task processing unit is configured to update a result of its predetermined processing only based on feature vectors that have been updated or that have been newly generated since the result of its predetermined processing was determined the last time.

12. The processing system according to claim 11, wherein the processing device is configured to store the feature vectors and to provide feature vectors to the task processing unit upon a request of the task processing unit.

13. The processing system according to claim 12, wherein the processing device is configured to provide feature vectors to the task processing unit only if a predetermined number of feature vectors have been updated or newly generated since the last request from the task processing unit.

14. A method executed by a processing device for converting a series of asynchronously generated data tuples into a feature vector, the method comprising: receiving a number of data tuples; generating for each data tuple an intermediate vector; generating the feature vector based on all intermediate vectors; and updating the feature vector based on the reception of additional data tuples; wherein when a new data tuple is received, an intermediate vector is generated for said new data tuple and a new feature vector is generated based on the previous feature vector and the intermediate vector of the new data tuple; and for each entry of the feature vector it is monitored whether a change of value occurs, and if for one of the entries of the feature vector no change of value is observed for a predetermined time, the value of the respective entry is successively decreased with time.

15. The method according to claim 14, further comprising by a vision sensor that comprises a pixel array having a plurality of pixels each being configured to receive light and to perform photoelectric conversion to generate an electrical signal, forming event data based on the generated electrical signals, which event data indicate as an event the occurrence of an intensity change of the light received at one pixel above an event detection threshold; and generating, by the vision sensor, one data tuple for each event, the data tuple indicating at least the time of event generation and the coordinates of the event on the pixel array.

Images