A visual sensor chip

Abstract

This invention provides a visual sensor chip, comprising a pixel array composed of pixel units; wherein each pixel unit has a corresponding temporal difference path and a spatial difference path, or a corresponding intensity path, a temporal difference path, and a spatial difference path. This invention integrates the dual-path characteristics of the human visual system into existing visual sensor chips, thereby significantly improving the chip's ability to perceive spatiotemporal dynamic information and achieving high-precision, high-frame-rate, high-dynamic-range, and highly efficient and robust visual representation.

Description

Technical Field

[0001] This invention relates to the field of optoelectronic imaging technology, and more particularly to a vision sensor chip. Background Technology

[0002] A visual sensor is a device used to sense visible light information in the environment and convert it into electrical signals. It is widely used in digital cameras and other electro-optical devices.

[0003] Currently, common vision sensors are frame-based CMOS image sensors (CIS). These sensors integrate transistors within each pixel to achieve high-performance charge-to-voltage conversion, hence they are also known as active pixel sensors (APS). CIS captures video using a frame-based sampling principle; each frame of the CIS image records the output of all pixel units in the pixel array, and each frame is taken at equal time intervals. Furthermore, CIS senses different wavelengths of visible light through a color filter array (CFA) covering the pixel array, thus obtaining a color image. In short, CIS offers advantages such as high pixel array resolution, high color fidelity, and high image quality. However, it suffers from slow capture speed. This is because CIS retains all pixel information within each frame, resulting in excessive data volume, making it difficult to increase capture speed with limited bandwidth. To overcome this problem, dynamic vision sensors (DVS) have emerged. Unlike CIS pixels, which record the intensity of incident light, Dynamic Vision Sensors (DVS) record the change in light intensity at each pixel location. A DVS only outputs a positive or negative pulse (indicating a decrease or increase in light intensity) when the change exceeds a certain threshold. DVS can output signals asynchronously; as soon as a pixel meets the pulse firing condition, it immediately inputs a signal while other pixels do not output. This significantly reduces data volume and redundancy, achieving extremely high temporal resolution. Furthermore, due to its sensitivity to changes and high-speed recording capabilities, DVS is naturally suited for tasks such as motion monitoring. However, a simple DVS only senses changes in light intensity. While data redundancy is extremely low, it loses a significant amount of color information, and pixel precision is insufficient (it can only output positive and negative pulses, not the degree of light intensity change). In addition, due to the complexity of the circuitry, the area of ​​a single DVS pixel is much larger than that of a CIS pixel, making it difficult to achieve high spatial resolution.

[0004] Therefore, the present invention urgently needs to provide an improved vision sensor. Summary of the Invention

[0005] To overcome the above problems, this invention provides a visual sensor chip that integrates the dual-path characteristics of the human visual system into existing visual sensor chips, thereby significantly improving the visual sensor chip's ability to perceive spatiotemporal dynamic information and achieving high-precision, high-frame-rate, high-dynamic-range, and efficient and robust visual representation.

[0006] In a first aspect, the present invention provides a visual sensor chip, the chip comprising a pixel array composed of pixel units;

[0007] For each pixel unit, the pixel unit has a corresponding temporal difference path and spatial difference path;

[0008] The time difference path is used to output the time difference value of the pixel unit;

[0009] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0010] The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit;

[0011] The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit;

[0012] The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

[0013] According to the visual sensor chip provided by the present invention, the temporal differential path includes the photosensitive subunit, the differential storage subunit deployed inside the pixel unit, and the temporal differential and quantizer;

[0014] The spatial differential path includes the photosensitive subunit, the differential storage subunit, and the spatial differential and quantizer;

[0015] The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0016] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0017] The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output;

[0018] The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node;

[0019] The time difference and quantizer is used to calculate and output the time difference value;

[0020] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

[0021] According to the visual sensor chip provided by the present invention, each pixel unit in the pixel array is provided with a trigger pulse generator;

[0022] or

[0023] All pixel units in the pixel array are connected to a common trigger pulse generator;

[0024] or

[0025] The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator.

[0026] The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

[0027] According to the visual sensor chip provided by the present invention, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0028] Secondly, the present invention provides a visual sensor chip, wherein the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0029] According to the visual sensor chip provided by the present invention, the chip includes a pixel array composed of pixel units;

[0030] For each pixel unit, the pixel unit has a corresponding intensity path, temporal difference path and spatial difference path;

[0031] The intensity path is used to output the quantized value of the current output value of the photosensitive subunit inside the pixel unit;

[0032] The time difference path is used to output the time difference value of the pixel unit;

[0033] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0034] The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit;

[0035] The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit;

[0036] The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

[0037] According to the visual sensor chip provided by the present invention, the intensity path includes the photosensitive subunit, a first unit deployed inside the pixel unit, and an intensity quantizer, or includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, a frequency divider gating unit deployed inside the pixel unit, and an intensity quantizer;

[0038] The time difference path includes the photosensitive subunit, the differential storage subunit, and the time difference and quantizer;

[0039] The spatial difference path includes the photosensitive subunit, the differential storage subunit, and the spatial difference and quantizer;

[0040] The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0041] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0042] The intensity quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0043] The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output.

[0044] The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node;

[0045] The first unit is used to send the current output value of the photosensitive subunit to the intensity quantizer when the photosensitive subunit uses rolling exposure; and to buffer and output the current output value of the photosensitive subunit when the photosensitive subunit does not use rolling exposure.

[0046] The frequency divider is used to perform low-frequency sampling on the current output value of the photosensitive subunit written by the differential storage subunit.

[0047] The intensity quantizer is used to quantize and output the output value of the first unit;

[0048] The time difference and quantizer is used to calculate and output the time difference value;

[0049] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

[0050] According to the visual sensor chip provided by the present invention, the intensity path output is a grayscale value or a color value;

[0051] When the intensity path output is a color value, an externally programmable demosaicer is embedded in both the temporal difference and quantizer and the spatial difference and quantizer. This demosaicer is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity path of the pixel unit and its surrounding units before calculating the temporal difference value / spatial difference value.

[0052] According to the visual sensor chip provided by the present invention, each pixel unit in the pixel array is provided with a trigger pulse generator;

[0053] or

[0054] All pixel units in the pixel array are connected to a common trigger pulse generator;

[0055] or

[0056] The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator.

[0057] The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

[0058] According to the visual sensor chip provided by the present invention, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0059] According to the visual sensor chip provided by the present invention, the exposure mode of each pixel unit in the pixel array is either global exposure or rolling exposure.

[0060] This invention provides a visual sensor chip in which each pixel unit has a unique corresponding temporal difference path and spatial difference path, or a unique corresponding light intensity quantization path, temporal difference path, and spatial difference path. This invention integrates the dual-path characteristics of the human visual system into existing visual sensor chips, thereby significantly improving the chip's ability to perceive spatiotemporal dynamic information and achieving high-precision, high-frame-rate, high-dynamic-range, and highly efficient and robust visual representation. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0062] Figure 1 This is one of the structural schematic diagrams of the visual sensor chip provided by the present invention;

[0063] Figure 2 This is a structural block diagram of the ping-pong cache provided by the present invention;

[0064] Figure 3 This is a design diagram of the ping-pong buffer circuit provided by the present invention;

[0065] Figure 4 This is one of the schematic diagrams of a pixel unit structure provided by the present invention when both the temporal difference and quantizer and the spatial difference and quantizer are arranged within the pixel;

[0066] Figure 5 This is the second schematic diagram of the pixel unit structure when both the temporal difference and quantizer and the spatial difference and quantizer are arranged within the pixel, as provided by the present invention.

[0067] Figure 6This is a schematic diagram of the communication connection between pixel units within a chip provided by the present invention;

[0068] Figure 7 This is one of the schematic diagrams of a pixel unit structure provided by the present invention when both the temporal difference and quantizer and the spatial difference and quantizer are arranged outside the pixel;

[0069] Figure 8 This is a schematic diagram of the trigger pulse signal provided by the present invention;

[0070] Figure 9 This is the second schematic diagram of the structure of the vision sensor chip provided by the present invention;

[0071] Figure 10 This is one of the schematic diagrams of a pixel unit structure provided by the present invention when the temporal difference and quantizer, spatial difference and quantizer and intensity quantizer are all arranged within the pixel;

[0072] Figure 11 This is the second schematic diagram of a pixel unit structure when the temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within a pixel, as provided by the present invention.

[0073] Figure 12 This is a schematic diagram of a pixel unit structure provided by the present invention, in which the temporal difference and quantizer and the spatial difference and quantizer are arranged outside the pixel and the intensity quantizer is arranged inside the pixel.

[0074] Figure 13 This is the third schematic diagram of the pixel unit structure provided by the present invention when the temporal difference and quantizer, spatial difference and quantizer and intensity quantizer are all arranged within the pixel;

[0075] Figure 14 This is the fourth schematic diagram of the pixel unit structure provided by the present invention when the time difference and quantizer, the spatial difference and quantizer, and the intensity quantizer are all arranged within the pixel. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0077] The following is combined with Figures 1-14 The present invention describes a vision sensor chip and a vision sensor.

[0078] First, let's explain the definitions of the abbreviations and key technical terms used in this invention:

[0079] APS: Active Pixel Sensor

[0080] CFA: Color Filter Array

[0081] CIS: CMOS Image Sensor

[0082] DAVIS: Dynamic Active Pixel Vision Sensor

[0083] DVS: Dynamic Vision Sensor

[0084] EVS: Event-based Visual Sensor

[0085] FPS: Frames per second (unit of frame rate)

[0086] PD: Photodiode

[0087] SD: Spatial Difference

[0088] TD: Time Difference

[0089] Firstly, existing DVS technology has the following shortcomings:

[0090] 1. Single-value signals have limited accuracy.

[0091] DVS outputs ±1 bit information (e.g., + indicates an increase in light intensity, - indicates a decrease in light intensity, and 0 indicates no change in light intensity, thus outputting positive and negative pulses through ±1 bit information, but cannot perceive the degree of change in light intensity), making it susceptible to noise interference, with low information content, and unable to adapt to complex environments.

[0092] Second: Lack of spatial difference information

[0093] Meanwhile, the human visual system perceives changes in both time and space, making it more sensitive and robust to external stimuli. In contrast, the DVS (Discrete Visual System) can only output information about the temporal changes in visual signals. This information is easily interfered with (for example, the DVS will fail when there is flickering light in the environment, as it cannot separate changes in light source from changes in signal caused by motion), and it lacks spatial differential information.

[0094] In view of this, the present invention provides a visual sensor chip, the chip comprising a plurality of pixel units arranged in an array;

[0095] For each pixel unit, the pixel unit has a corresponding temporal difference path and spatial difference path;

[0096] The time difference path is used to output the time difference value of the pixel unit;

[0097] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0098] The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit;

[0099] The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit;

[0100] The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

[0101] Specifically, this invention implements a visual sensor with two output channels. The two channels are: a temporal differential channel (TD) and a spatial differential channel (SD).

[0102] Among them, the TD path outputs the current pixel unit (x,y) at t n Time difference fraction TD(x,y,t) n ), expressed by the formula:

[0103] TD(x,y,t n )=Q TD (I(x,y,t n )-I(x,y,t n-1 ))

[0104] In the above formula, I(x,y,t) n ) and I(x,y,t n-1 ) represent the photosensitive sub-units within the current pixel unit (x, y) at time t. n Time and its previous time t n-1 The output value, Q TD The quantization method used for time difference paths.

[0105] The SD path outputs the current pixel unit (x, y) at t n Spatial difference SD at time * (x,y,t n ), expressed by the formula:

[0106] SD * (x,y,t n )=Q SD (I(x,y,t n )-I(x * ,y * ,t n ))

[0107] In the above formula, I(x) * ,y * ,t n ) is the target photosensitive unit at t nOutput value at time (current time);

[0108] Wherein, the pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

[0109] The target photosensitive subunit may be one or multiple, resulting in the following situations:

[0110] (1) When there is only one target photosensitive unit, the SD path only performs differential analysis in a certain direction.

[0111] (2) When there are two target photosensitive sub-units (referred to as the first target photosensitive sub-unit and the second target photosensitive sub-unit respectively) and the pixel unit where the first target photosensitive sub-unit is located, the pixel unit where the second target photosensitive sub-unit is located, and the pixel unit are all on the same straight line, the SD path only performs differential in one direction; at this time, the differential accuracy of the pixel unit is higher than that of (1).

[0112] (3) When there are two target photosensitive sub-units (referred to as the first target photosensitive sub-unit and the second target photosensitive sub-unit respectively) and the pixel unit where the first target photosensitive sub-unit is located, the pixel unit where the second target photosensitive sub-unit is located, and the pixel unit are not in a straight line, the SD path performs differential analysis in two directions; at this time, the pixel unit can obtain spatial differential information in multiple directions.

[0113] (4) When there are multiple (more than two) target photosensitive sub-units and the line connecting the pixel unit where all photosensitive sub-units are located and the pixel unit is a straight line, the SD path only performs differential in one direction. At this time, the differential accuracy of the pixel unit is higher than that of (2).

[0114] (5) When there are multiple (more than two) target photosensitive sub-units and the line connecting the pixel unit where all photosensitive sub-units are located and the pixel unit is not a straight line, the SD path is differentially divided in at least two directions.

[0115] It can be seen that the differential accuracy of the pixel unit is mainly affected by the number of differential directions and the number of target photosensitive sub-units. In fact, the differential accuracy of the pixel unit is also affected by the distance between the pixel unit where the target photosensitive sub-unit is located and the pixel unit. Therefore, the present invention preferably includes a first target photosensitive sub-unit and a second target photosensitive sub-unit in the target photosensitive sub-unit. The pixel unit where the first target photosensitive sub-unit is located (hereinafter referred to as the first pixel unit) and the pixel unit where the first target photosensitive sub-unit is located (hereinafter referred to as the second pixel unit) are both adjacent to the pixel unit, and the lines connecting the first pixel unit, the pixel unit and the second pixel unit are not on the same straight line.

[0116] For example: the first pixel unit and the second pixel unit are pixel unit (x+1, y) and pixel unit (x, y+1), respectively; here 1 refers to the spacing of 1 pixel unit.

[0117] At this time, the SD path output pixel unit (x,y) at t n The pixel value at time t and the pixel unit (x+1, y) in t n Spatial difference SD between pixel values ​​at different times x (x,y,t n ), and pixel unit (x,y) in t n The pixel value at time t and the pixel unit (x, y+1) in t n Spatial difference SD between pixel values ​​at different times y (x,y,t n );

[0118] SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y,t n ))

[0119] SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y+1,t m ))

[0120] I(x,y+1,t n )

[0121] Or, for example, the first pixel unit and the second pixel unit are pixel unit (x-1, y+1) and pixel unit (x+1, y+1) respectively; in this case, the SD path outputs pixel unit (x, y) at t n The pixel value at time t and the pixel unit (x-1, y+1) in t n Spatial difference SD between pixel values ​​at different times ↙ (x,y,t n ), and pixel unit (x,y) in t n The pixel value at time t and the pixel unit (x+1, y+1) in t n Spatial difference SD between pixel values ​​at different times ↘ (x,y,t n );

[0122] SD ↙ (x,y,t n )=Q SD (I(x,y,tn )-I(x+1,y+1,t n ))

[0123] SD ↘ (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y+1,t n ))

[0124] In the above formula, Q SD The quantization method used for spatial difference paths.

[0125] I(x+1,y,t n ), I(x,y+1,t n ), I(x+1,y+1,t n ) and I(x-1,y+1,t n The following are the photosensitive subunits within pixel units (x+1,y), (x,y+1), (x+1,y+1), and I(x-1,y+1) at time t. n Output value at time.

[0126] All the signals mentioned above are three-dimensional quantities, including two-dimensional spatial quantities of x and y and a time dimension t.

[0127] Figure 1 This is a schematic diagram of the corresponding visual sensor chip, with the first pixel unit and the second pixel unit being (x+1, y) and (x, y+1) respectively, as an example.

[0128] The visual sensor chip provided by this invention integrates the dual-path characteristics of the human visual system into existing visual sensor chips, thereby significantly improving the visual sensor chip's ability to perceive spatiotemporal dynamic information and achieving high-precision, high-frame-rate, high-dynamic-range, and highly efficient and robust visual representation.

[0129] Based on the above embodiments, as an optional embodiment, the temporal differential path includes the photosensitive sub-unit, a differential storage sub-unit deployed inside the pixel unit, and a temporal differential and quantizer;

[0130] The spatial differential path includes the photosensitive subunit, the differential storage subunit, and the spatial differential and quantizer;

[0131] The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0132] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0133] The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output;

[0134] It is understandable that the output value of the photosensitive subunit is an electrical signal such as charge, voltage and current, which represents the intensity of the incident light sensed by the current pixel position at the current moment. The greater the light intensity, the higher the pixel value.

[0135] The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node;

[0136] Understandably, in order to construct temporal and spatial differential paths, this invention sets up two storage nodes (a first storage node and a second storage node) within each pixel unit. The first and second storage nodes use a ping-pong buffering method to buffer data. Ping-pong buffering means that if the photosensitive unit is stored in the first storage node at the current moment, it will be stored in the second storage node at the next moment, and so on, alternating between the two. It outputs two time-time signals (I(x,y,t)). n ), I(x,y,t n-1 )).

[0137] Figure 2 This is a structural block diagram of the ping-pong cache. Figure 3 One possible specific circuit design is given, but there are more than one actual circuit schematic.

[0138] The time difference and quantizer is used to calculate and output the time difference value;

[0139] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

[0140] In other words, depending on whether the temporal difference and quantizer and the spatial difference and quantizer are arranged within the pixel, this invention provides four schematic diagrams of pixel unit structures.

[0141] The first method: Each pixel unit is equipped with a temporal difference and quantizer and a spatial difference and quantizer;

[0142] Figure 4 and Figure 5 This is a schematic diagram of a pixel unit structure when both the temporal difference and quantizer and the spatial difference and quantizer are arranged within the pixel. Figure 4 The illustration is illustrated by taking the first pixel unit and the second pixel unit as (x+1, y) and (x, y+1) respectively. Figure 5 The following example illustrates the first and second pixel units as (x-1, y+1) and (x+1, y+1) respectively. The first method uses pixel unit readout, where the temporal and spatial difference values ​​are directly read from each pixel unit.

[0143] It should be noted that, due to the existence of the differential quantization path, a communication connection is established between each pixel unit in the pixel array and the pixel unit containing the corresponding target photosensitive subunit. Figure 6 This is a schematic diagram illustrating the communication connection between pixel units within a chip provided by the present invention; Figure 6 The square boxes represent pixel units, and the connecting lines represent the pixel units at the current time t. n The transmission of the output value of the photosensitive subunits. The left image illustrates this with the first and second pixel units being (x+1, y) and (x, y+1) respectively. The right image illustrates this with the first and second pixel units being (x-1, y+1) and (x+1, y+1) respectively.

[0144] The second method involves setting up a temporal differential quantizer and a spatial differential quantizer for each column of pixel units, so that the pixel units in that column can share the same data.

[0145] Figure 7 This diagram illustrates a pixel unit structure when both the temporal and spatial differential quantizers are located outside the pixel, using (x+1, y) and (x, y+1) as examples of the first and second pixel units. The second method employs a column-level readout approach, where each column shares one temporal and one spatial differential quantizer unit.

[0146] like Figure 7 As shown, the SD differential and quantizer ① calculates sequentially.

[0147] SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y,t n ))

[0148] SD y (y,y,t n )=Q SD (I(x,y,t n )-I(x,y+1,t n ))

[0149] SD x (x,y+1,t n )=Q SD (I(x,y+1,t n )-I(x+1,y+1,t n ))

[0150] SD y (x,y+1,t n )=Q SD (I(x,y+1,t n )-I(x,y+2,t n ))

[0151] SD Differential and Quantizer ② Calculate sequentially

[0152] SD x (x+1,y,t n )=Q SD (I(x+1,t,t n )-I(x+2,y,t n ))

[0153] SD y (x+1,y,t n )=Q SD (I(x+1,y,t n )-I(x+1,y+1,t n ))

[0154] SD x (x+1,y+1,t n )=Q SD (I(x+1,y+1,t n )-I(x+2,y+1,t n ))SD y (x+1,y+1,t n )=Q SD (I(x+1,y+1,t n )-I(x+1,y+2,t n ))

[0155] SD differential and quantizer ③ and so on.

[0156] TD Differential and Quantizer ④ Calculate sequentially

[0157] TD(x,y,t n )=Q TD (I(x,y,t n )-I(x,y,t n-1 ))

[0158] TD(x,y+1,t n )=Q TD (I(x,y+1,t n )-I(x,y+1,t n-1 ))

[0159] TD(x,u+2,t n )=Q TD (I(x,y+2,t n )-I(x,y+2,t n-1 ))

[0160] TD Differential and Quantizer ⑤ Calculate sequentially

[0161] TD(x+1,y,t n )=Q TD (I(x+1,y,t n )-I(x+1,y,t n-1 ))

[0162] TD(x+1,y+1,t n )=Q TD (I(x+1,y+1,t n )-I(x+1,y+1,t n-1 ))TD(x+1,y+2,t n )=Q TD (I(x+1,y+2,t n )-I(x+1,y+2,t n-1 ))

[0163] TD differential and quantizer ⑥ and so on.

[0164] The principle is the same for the example where the first pixel unit and the second pixel unit are (x-1, y+1) and (x+1, y+1) respectively, and will not be repeated here.

[0165] The third method involves setting a temporal difference and quantizer within each pixel unit, and a spatial difference and quantizer for each column of pixel units to be shared by all pixel units in that column.

[0166] The fourth method involves setting a spatial difference and quantizer within each pixel unit, and setting a temporal difference and quantizer for each column of pixel units to be shared by that column of pixel units.

[0167] The third and fourth types evolved from temporal and spatial differential quantizers within a pixel and temporal and spatial differential quantizers outside a pixel, respectively, and will not be elaborated upon here.

[0168] It should be noted that, in the above four methods, the photosensitive sub-units within the pixel unit (x, y) at tn Time and its previous time t n-1 Output value I(x,y,t) n ) and I(x,y,t n-1 Simultaneously, a spatial difference and quantizer was input. The spatial difference and quantizer first selected I(x,y,t) through an internal selector. n And discard I(x,y,t) n-1 Then spatial difference and quantization operations are performed.

[0169] In addition, the time difference quantizer and the spatial difference quantizer use ADC (analog-to-digital converter) quantization.

[0170] Among them, ADC quantization can be single-bit ADC quantization, multi-bit ADC quantization with a sign bit (positive and negative sign bits indicate enhancement / decrease respectively), and multi-bit ADC quantization without a sign bit.

[0171] The present invention preferably uses multi-bit ADC quantization with a sign bit (positive and negative sign bits indicate enhancement / decrease, respectively).

[0172] Multi-bit ADC quantization with a sign bit can measure the magnitude of the difference result and provide the sign of the difference result during the process of quantizing analog signals into digital signals. This allows for the perception of more precise changes in light intensity, thus improving pixel accuracy. In addition, the signal-to-noise ratio is improved because it uses multiple bits.

[0173] Based on the above embodiments, as an optional embodiment, each pixel unit in the pixel array is provided with a trigger pulse generator;

[0174] or

[0175] All pixel units in the pixel array are connected to a common trigger pulse generator;

[0176] or

[0177] The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator.

[0178] The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the exposure time of the corresponding photosensitive subunit.

[0179] The trigger pulse generator is used to generate a trigger signal, which controls the photosensitive subunit to perform exposure, thus determining the time t for signal acquisition. n . Figure 8This is a schematic diagram of the trigger pulse signal. In this diagram, the horizontal axis x represents time, and the vertical axis y represents the amplitude of the digital signal. From Figure 8 As can be seen from this, the trigger pulse generation time is equivalent to the sampling time of the pixel unit, which is t. n , t n-1 , t n-2 …These time intervals can be set not only to the fixed time intervals shown in the left figure, but also to the adaptive, programmable variable intervals shown in the right figure. This adaptive interval can adapt to the changing characteristics of the external visual signal, using a higher sampling frequency when the change is large and the change frequency is high, and using a lower sampling frequency when the signal is low-frequency, in order to reduce the amount of data and energy consumption.

[0180] Based on the above embodiments, as an optional embodiment, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0181] As can be imagined, if a trigger pulse generator is designed within a pixel unit, this generator can independently and adaptively adjust the timing of triggering the spatiotemporal differential signal based on the light intensity level sensed by the pixel unit itself, resulting in a different trigger timing for each pixel. Pixels can output information at any time, increasing flexibility and reducing output latency.

[0182] Therefore, this invention supports all pixel units in the array sharing a single trigger pulse generator, in which case only full array synchronous exposure can be used.

[0183] It also supports using a trigger pulse generator for each pixel unit in the array, and the exposure mode can be set to synchronous exposure or asynchronous exposure as needed.

[0184] It also supports multiple pixels forming a macroblock to share a single pulse trigger generator within a pixel, thereby reducing chip design complexity and area occupied. In this case, pixel units within the same macroblock are exposed synchronously, and the exposure mode between macroblocks can be set to synchronous or asynchronous exposure as needed.

[0185] Based on the above embodiments, as an optional embodiment, the exposure mode of each pixel unit in the pixel array is either global exposure or rolling exposure.

[0186] The present invention provides a time difference and quantizer comprising a time difference calculator and a quantizer, and a spatial difference and quantizer comprising a spatial difference calculator and a quantizer. Each pixel unit outputs an electrical signal representing light intensity through a photosensitive unit. This electrical signal enters a storage node, where time difference calculation / spatial difference calculation and quantization are performed before output. The execution order of the difference calculation and quantization can be interchanged; that is, two analog signals can be quantized into digital signals and then the difference calculation can be performed in the digital domain, or the analog signals can be first differentially calculated in the analog domain, and then the difference result can be quantized into a digital signal.

[0187] Secondly, because DVS alone is insufficient for achieving universal visual perception, existing visual sensors typically combine DVS with high spatial resolution, high image quality CIS. Examples include the DAVIS camera and cameras based on hybrid pixel arrays. The DAVIS camera combines CIS with DVS, where the output current of a single photodiode (PD, used to convert incident light into current) is simultaneously utilized by both the APS and DVS circuits, enabling the recording of both single-frame images (frame-based sampling) and event information (event-based sampling). This allows DAVIS to combine the high image quality of CIS with the high temporal resolution of DVS cameras. However, the DAVIS camera has the following drawbacks:

[0188] First, DAVIS cameras inherited the limitation of DVS single-value signal accuracy.

[0189] Second: Lack of spatial difference information

[0190] When there is a large-scale flash or a drastic change in light intensity in the scene, all TD pixels output events, causing saturation. The DVS path cannot output effective information, and the CIS path, due to frame rate limitations, cannot respond in time. Such extreme situations are very common in autonomous driving and are crucial for driving safety, such as entering and exiting tunnels, or capturing camera flashes at night. In other words, a vision sensor with only two paths, CIS and DVS, is incomplete in its information acquisition from the perspective of visual primitives. In contrast, the human visual system can quickly identify moving targets regardless of midday or dusk, whether in an open scene or partially occluded, achieving robustness and versatility far exceeding that of existing DAVIS. This is because the human eye achieves efficient and robust visual representation by combining different visual primitives.

[0191] Therefore, DVS can only output information about the temporal changes of visual signals, and this information is very easily interfered with, and it lacks spatial difference information.

[0192] Based on this, the present invention provides a visual sensor chip, the chip comprising a pixel array composed of pixel units;

[0193] For each pixel unit, the pixel unit has a corresponding intensity path, temporal difference path and spatial difference path;

[0194] The intensity path is used to output the quantized value of the current output value of the photosensitive subunit inside the pixel unit;

[0195] The time difference path is used to output the time difference value of the pixel unit;

[0196] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0197] The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit;

[0198] The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit;

[0199] The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

[0200] Specifically, this invention implements a visual sensor with three output channels. The three channels are: light intensity quantization channel, temporal difference channel (TD), and spatial difference channel (SD).

[0201] Among them, the light intensity quantization path outputs the photosensitive unit inside the current pixel unit (x,y) at t n The quantization result of the output value at time t is A(x,y,t). n ), expressed by the following formula:

[0202] A(x,y,t n )=Q(I(x,y,t n ))

[0203] Where Q represents the quantization method used in the intensity pathway.

[0204] The signals in the light intensity quantization path are all three-dimensional quantities, including two-dimensional spatial quantities of x and y and a time dimension t.

[0205] The TD and SD pathways are the same as those of the pixel units of the visual sensor chip described in the first aspect, and will not be repeated here.

[0206] Figure 9 This is a schematic diagram of the structure of the corresponding vision sensor chip, with the first pixel unit and the second pixel unit being (x+1,y) and (x,y+1) respectively as examples.

[0207] The visual sensor chip provided by this invention incorporates a spatial difference path, mimicking the human retina, into the existing visual sensor pixel units that only contain a temporal difference path and a color path. This significantly improves the visual sensor's ability to reconstruct spatiotemporal dynamic information with high precision, forming an efficient and robust visual representation.

[0208] Based on the above embodiments, as an optional embodiment, the intensity path includes the photosensitive subunit, a first unit deployed inside the pixel unit, and an intensity quantizer, or includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, a frequency divider gating unit deployed inside the pixel unit, and an intensity quantizer;

[0209] The time difference path includes the photosensitive subunit, the differential storage subunit, and the time difference and quantizer;

[0210] The spatial difference path includes the photosensitive subunit, the differential storage subunit, and the spatial difference and quantizer;

[0211] The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0212] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0213] The intensity quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column;

[0214] The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output.

[0215] The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node;

[0216] The first unit is used to send the current output value of the photosensitive subunit to the intensity quantizer when the photosensitive subunit uses rolling exposure; and to buffer and output the current output value of the photosensitive subunit when the photosensitive subunit does not use rolling exposure.

[0217] The frequency divider gate is used to perform low-frequency sampling on the current output value of the photosensitive sub-unit written by the differential storage sub-unit. It can be understood that the frequency divider gate allows the pixel unit to directly obtain the stored information from the differential storage sub-unit without needing to set up a separate storage node for the intensity path. The function of the frequency divider gate is low-frequency sampling. Assuming the differential path array is 600fps, then because of the ping-pong storage setting, the data update frequency of the first / second storage nodes is 300Hz, and the light intensity quantizer is 30fps. Therefore, the frequency divider gate only needs to reduce the sampling frequency by ten times (equivalent to the frequency divider gate only outputting one signal for every 10 signals received, discarding the other 9).

[0218] The intensity quantizer is used to quantize and output the output value of the first unit;

[0219] The time difference and quantizer is used to calculate and output the time difference value;

[0220] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

[0221] In other words, when the intensity path includes the photosensitive sub-unit, the first unit deployed inside the pixel unit, and the intensity quantizer, the present invention provides eight pixel unit structures depending on whether the temporal difference and quantizer, the spatial difference and quantizer, and the intensity quantizer are arranged inside the pixel.

[0222] A: The temporal difference quantizer, spatial difference quantizer, and intensity quantizer are all arranged within the pixel;

[0223] Figure 10 and Figure 11 This is a schematic diagram of the pixel unit structure when the corresponding temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within the pixel. Figure 10 The illustration is illustrated by taking the first pixel unit and the second pixel unit as (x+1, y) and (x, y+1) respectively. Figure 11 The illustration is illustrated by taking the first pixel unit and the second pixel unit as (x-1, y+1) and (x+1, y+1) respectively.

[0224] B: The intensity is arranged within the pixel using a quantizer, while the temporal difference and quantizer and the spatial difference and quantizer are arranged outside the pixel;

[0225] Figure 12This diagram illustrates a pixel unit structure where the temporal and spatial differential quantizers are located outside the pixel, while the intensity quantizer is located inside the pixel. It uses (x+1, y) and (x, y+1) as examples of first and second pixel units, respectively. The principle is the same when the first and second pixel units are (x+1, y+1) and (x-1, y+1), respectively, and will not be repeated here.

[0226] C: The temporal difference and quantizer are arranged inside the pixel, while the spatial difference and quantizer and the intensity are arranged outside the pixel using quantizers.

[0227] D: Spatial difference and quantizer are arranged inside the pixel, while temporal difference and quantizer and intensity are arranged outside the pixel using quantizer;

[0228] E: The temporal difference and quantizer and the spatial difference and quantizer are arranged inside the pixel, while the intensity is arranged outside the pixel using a quantizer;

[0229] F: Temporal difference and quantizer and intensity are arranged within the pixel using quantizers, while spatial difference and quantizer are arranged outside the pixel;

[0230] G: Spatial difference and quantizer and intensity are arranged in the pixel using quantizer, while temporal difference and quantizer are arranged outside the pixel;

[0231] H: The temporal difference quantizer, spatial difference quantizer, and intensity quantizer are all arranged outside the pixel.

[0232] The pixel unit structures corresponding to C to H are the same as those removed, and will not be elaborated here.

[0233] When the intensity path includes the photosensitive sub-unit, the differential storage sub-unit deployed inside the pixel unit, the frequency divider gate and the intensity quantizer deployed inside the pixel unit, the present invention also provides eight pixel unit structures depending on whether the temporal differential quantizer, the spatial differential quantizer and the intensity quantizer are arranged inside the pixel.

[0234] I: Temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within the pixel;

[0235] Figure 13 and Figure 14 This is a schematic diagram of the pixel unit structure when the corresponding temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within the pixel. Figure 13 The illustration is illustrated by taking the first pixel unit and the second pixel unit as (x+1, y) and (x, y+1) respectively. Figure 14 The illustration is illustrated by taking the first pixel unit and the second pixel unit as (x-1, y+1) and (x+1, y+1) respectively.

[0236] II: The temporal difference and quantizer and the spatial difference and quantizer are arranged inside the pixel, while the intensity quantizer is arranged outside the pixel;

[0237] III: The temporal difference and quantizer and the intensity quantizer are arranged inside the pixel, while the spatial difference and quantizer are arranged outside the pixel;

[0238] IV: Spatial difference and quantizer and intensity are arranged in the pixel using quantizer, while temporal difference and quantizer are arranged outside the pixel;

[0239] V: The temporal difference and quantizer are arranged inside the pixel, while the spatial difference and quantizer and the intensity are arranged outside the pixel using quantizers;

[0240] VI: Spatial difference and quantizer are arranged inside the pixel, while temporal difference and quantizer and intensity are arranged outside the pixel using quantizers;

[0241] VII: The intensity quantizer is placed inside the pixel, while the temporal difference quantizer and spatial difference quantizer are placed outside the pixel;

[0242] VIII: The temporal difference quantizer, spatial difference quantizer, and intensity quantizer are all positioned outside the pixel.

[0243] II through VIII are similar in principle, and will not be elaborated upon here.

[0244] It should be noted that, in the above eight methods, the photosensitive subunits within the pixel unit (x,y) are at t n Time and its previous time t n-1 Output value I(x,y,t) n ) and I(x,y,t n-1 Simultaneously, a frequency divider was input. The frequency divider first selected I(x,y,t) through an internal selector. n And discard I(x,y,t) n-1 Then spatial difference and quantization operations are performed.

[0245] Note that, similar to time difference quantizers and spatial difference quantizers, the intensity quantizer of this invention uses ADC quantization; preferably, it uses multi-bit ADC quantization with a sign bit (positive and negative sign bits indicate enhancement / decrease, respectively).

[0246] Based on the above embodiments, as an optional embodiment, the intensity path output is a grayscale value or a color value;

[0247] When the intensity path output is a color value, an externally programmable demosaicer is embedded in both the temporal difference and quantizer and the spatial difference and quantizer. This demosaicer is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity path of the pixel unit and its surrounding units before calculating the temporal difference value / spatial difference value.

[0248] Color Information System (CIS) uses a color filter array (CFA) covering a pixel array to sense different wavelengths of visible light and thus obtain a color image. A CFA typically contains filters of three colors: red, green, and blue, so the color path is sometimes simply referred to as "RGB." The three color filters are usually arranged in a Bayer array. However, there are other types of CFAs, such as a CMY array based on three complementary colors (cyan, magenta, and yellow), which has higher transmittance. This invention can be performed without a color filter array, in which case the light intensity quantization path outputs grayscale values; or it can be performed with a color filter array (e.g., RGB color filters), in which case it outputs color values.

[0249] If the output of the intensity path is a color value, the corresponding pixel is covered with a color filter. In this case, the output value of the photosensitive sub-unit in the spatiotemporal difference path only contains information from a single color channel. Typically, this includes red, green, and blue channels, in which case the pixels in the array are divided into red, green, and blue color information.

[0250] Therefore, when performing temporal / spatial difference, all colors are first mosaicked. That is, for a pixel of color X, the Y color output at that position is obtained based on the Y color pixels around it, and the Z color output at that position is obtained based on the Z color pixels around it. In this way, each pixel has three color output paths, and then regular spatial difference operations can be performed.

[0251] The present invention embeds an externally programmable demosaicer in both the time difference and quantizer and the spatial difference and quantizer, and uses an internal demosaic algorithm to implement the demosaic operation.

[0252] The mosaic algorithm is not unique; it can be implemented by selecting two, four, or even 16 surrounding points.

[0253] Of course, when the differential and quantization units are distributed in columns, the demosaicing unit can also be set independently within the pixel.

[0254] Alternatively, instead of introducing a demosaicer, you can directly perform spatial difference operations on the color channel corresponding to the current pixel. This difference can be performed on the same color channel (e.g., pixels of color X versus pixels of color Y), or on different color channels (e.g., pixels of color X versus pixels of color Y), followed by additional post-processing algorithms.

[0255] Based on the above embodiments, as an optional embodiment, each pixel unit in the pixel array is provided with a trigger pulse generator;

[0256] or

[0257] All pixel units in the pixel array are connected to a common trigger pulse generator;

[0258] or

[0259] The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator.

[0260] The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the exposure time of the corresponding photosensitive subunit.

[0261] Based on the above embodiments, as an optional embodiment, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0262] Based on the above embodiments, as an optional embodiment, the exposure mode of each pixel unit in the pixel array is either global exposure or rolling exposure.

[0263] The above process is the same as that of the vision sensor chip described in the first aspect, and will not be repeated here.

[0264] The visual sensing chip of the first or second aspect of the present invention is connected to an image processing module, which is integrated with the pixel array in the same chip or can be placed in a computer or other device. It is used to process the chip output signal.

[0265] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0266] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0267] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vision sensor chip, characterized by The chip includes a pixel array composed of pixel units; For each pixel unit, the pixel unit has a corresponding temporal difference path and spatial difference path; The time difference path is used to output the time difference value of the pixel unit; The spatial difference path is used to output the spatial difference value of the pixel unit; The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit; The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit; The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

2. The vision sensor chip of claim 1, wherein, The temporal differential path includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, and a temporal differential and quantizer; The spatial differential path includes the photosensitive subunit, the differential storage subunit, and the spatial differential and quantizer; The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column; The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column; The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output; The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node; The time difference and quantizer is used to calculate and output the time difference value; The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

3. The vision sensor chip of claim 1, wherein, Each pixel unit in the pixel array is equipped with a trigger pulse generator; or All pixel units in the pixel array are connected to a common trigger pulse generator; or The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator. The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

4. The vision sensor chip of claim 1, wherein, Pixel units connected to the same trigger pulse generator are exposed synchronously, while pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

5. The visual sensor chip according to claim 1, characterized in that, The exposure mode for each pixel unit in the pixel array is either global exposure or rolling exposure.

6. A vision sensor chip, characterized by The chip includes a pixel array composed of pixel units; For each pixel unit, the pixel unit has a corresponding intensity path, temporal difference path and spatial difference path; The intensity path is used to output the quantized value of the current output value of the photosensitive subunit inside the pixel unit; The time difference path is used to output the time difference value of the pixel unit; The spatial difference path is used to output the spatial difference value of the pixel unit; The time difference value is the difference and quantization result between the current output value and the previous output value of the photosensitive subunit inside the pixel unit; The spatial difference value is the difference and quantization result between the current output value of the photosensitive subunit and the current output value of the target photosensitive subunit; The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array other than the pixel unit mentioned above.

7. The vision sensor chip of claim 6, wherein, The intensity path includes the photosensitive subunit, a first unit deployed inside the pixel unit, and an intensity quantizer, or includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, a frequency divider gating unit deployed inside the pixel unit, and an intensity quantizer; The time difference path includes the photosensitive subunit, the differential storage subunit, and the time difference and quantizer; The spatial difference path includes the photosensitive subunit, the differential storage subunit, and the spatial difference and quantizer; The time difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column; The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column; The intensity quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and other pixel units in the same column; The photosensitive subunit is used to convert the light intensity of the incident light at the current moment into an electrical signal output. The differential storage sub-unit is used to write the current output value of the photosensitive sub-unit; wherein, the differential storage sub-unit includes a first storage node and a second storage node, and when the previous output value of the photosensitive sub-unit is written into the first storage node / second storage node, the current output value of the photosensitive sub-unit is written into the second storage node / first storage node; The first unit is used to send the current output value of the photosensitive subunit to the intensity quantizer when the photosensitive subunit uses rolling exposure; and to buffer and output the current output value of the photosensitive subunit when the photosensitive subunit does not use rolling exposure. The frequency divider is used to perform low-frequency sampling on the current output value of the photosensitive subunit written by the differential storage subunit. The intensity quantizer is used to quantize and output the output value of the first unit; The time difference and quantizer is used to calculate and output the time difference value; The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

8. The vision sensor chip according to any of claims 6 or 7, characterized in that, The intensity path output is a grayscale value or a color value; When the intensity path output is a color value, an externally programmable demosaicer is embedded in both the temporal difference and quantizer and the spatial difference and quantizer. This demosaicer is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity path of the pixel unit and its surrounding units before calculating the temporal difference value / spatial difference value.

9. The vision sensor chip of claim 6, wherein, Each pixel unit in the pixel array is equipped with a trigger pulse generator; or All pixel units in the pixel array are connected to a common trigger pulse generator; or The pixel array is divided into multiple sub-regions, and all pixel units in each sub-region are connected to a trigger pulse generator. The trigger pulse generator is used to generate a trigger signal at a fixed time interval or at an adaptive, programmable variable interval to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

10. The vision sensor chip of claim 9, wherein, Pixel units connected to the same trigger pulse generator are exposed synchronously, while pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

11. The vision sensor chip of claim 9, wherein, The exposure mode for each pixel unit in the pixel array is either global exposure or rolling exposure.

Images