Visual sensor chip

Abstract

The present application provides a visual sensor chip. The visual sensor chip comprises a pixel array consisting of pixel units, among which each pixel unit has a corresponding time differential path and space differential path or has a corresponding intensity path, time differential path and space differential path. The present application fuses the dual-path feature of a human vision system into existing visual sensor chips, so that the capabilities of the visual sensor chips sensing spatio-temporal dynamic information can be greatly improved, allowing for high-precision, high-frame-rate, high-dynamic-range and high-efficiently robust visual representations.

Description

A visual sensor chip

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311420669.1, filed on October 30, 2023, entitled “A Visual Sensor Chip,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present application relates to the field of optoelectronic imaging technology, and in particular to a visual sensor chip. Background Art

[0004] A visual sensor is a device used to perceive visible light information in the environment and convert it into electrical signals. It is widely used in digital cameras and other electronic optical devices.

[0005] Currently, the most common visual sensor is the frame-based CMOS image sensor (CIS). Its pixels integrate transistors to achieve high-performance charge-to-voltage conversion, hence the name active pixel sensor (APS). CIS captures video using a frame-based sampling principle. Each CIS image records the output of all pixels in the pixel array, and each frame is timed at equal intervals. Furthermore, the CIS uses a color filter array (CFA) overlying the pixel array to sense visible light of varying wavelengths, generating a color image. While CIS offers the advantages of high pixel resolution, excellent color reproduction, and high image quality, it suffers from slow capture speeds. This is because CIS retains all pixel information within each frame, resulting in excessively large data volumes and difficulty increasing capture speeds within limited bandwidth. To overcome this issue, the dynamic vision sensor (DVS) has emerged. Unlike CIS pixels, which record incident light intensity, dynamic vision sensors (DVS) record the change in incident light intensity at each pixel. Each DVS pixel records the change in incident light intensity at its corresponding location and only outputs a positive or negative pulse (indicating a decrease or increase in light intensity) when the change exceeds a certain threshold. DVS can output signals asynchronously: as soon as a pixel meets the pulse emission criteria, it immediately receives an input signal, while other pixels remain silent. This significantly reduces data volume and redundancy, enabling extremely high temporal resolution. Furthermore, DVS's sensitivity to changes and high-speed recording make it naturally suitable for tasks such as motion detection. However, a simple DVS only senses changes in light intensity. While this minimizes data redundancy, it loses significant color information and suffers from limited pixel accuracy (it can only output positive and negative pulses and cannot detect the degree of light intensity change). Furthermore, due to the complex circuitry of DVS pixels, the area of ​​each pixel is much larger than that of CIS pixels, making it difficult to achieve high spatial resolution.

[0006] Therefore, the present application urgently needs to provide an improved visual sensor.

[0007] Summary of the Invention

[0008] To overcome the above problems, the present application provides a visual sensor chip that integrates the dual-channel characteristics of the human visual system into the existing visual sensor chip, thereby greatly improving the visual sensor chip's perception of spatiotemporal dynamic information and achieving high-precision, high frame rate, high dynamic range and efficient and robust visual representation.

[0009] In a first aspect, the present application provides a visual sensor chip, the chip comprising a pixel array composed of pixel units;

[0010] For each pixel unit, the pixel unit has a corresponding time difference path and space difference path;

[0011] The time difference path is used to output the time difference value of the pixel unit;

[0012] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0013] The time difference value is the difference and quantization result between the output value of the photosensitive sub-unit at the current moment and the output value at the previous moment within the pixel unit;

[0014] The spatial difference value is a quantized result of the difference between the output value of the photosensitive sub-unit at the current moment and the output value of the target photosensitive sub-unit at the current moment;

[0015] The pixel unit where the target photosensitive sub-unit is located is any pixel unit in the pixel array except the pixel unit.

[0016] According to the visual sensor chip provided by the present application, the time differential path includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, and a time differential and quantizer;

[0017] The spatial differential path includes the photosensitive subunit, the differential storage subunit and a spatial differential and quantizer;

[0018] The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0019] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0020] The photosensitive subunit is used to convert the intensity of the incident light on the pixel unit at the current moment into an electrical signal for output;

[0021] The differential storage subunit is used to write the current output value of the photosensitive unit; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive unit at the previous moment is written into the first storage node / the second storage node, the current output value of the photosensitive unit is written into the second storage node / the first storage node;

[0022] The time difference and quantizer is used to calculate and output the time difference value;

[0023] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive sub-unit.

[0024] According to the visual sensor chip provided by the present application, a trigger pulse generator is provided in each pixel unit in the pixel array;

[0025] or

[0026] All pixel units in the pixel array are commonly connected to a trigger pulse generator;

[0027] or

[0028] Dividing the pixel array into a plurality of sub-areas, wherein all pixel units in each sub-area are connected to a trigger pulse generator;

[0029] The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

[0030] According to the visual sensor chip provided by the present application, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0031] In a second aspect, the present application provides a visual sensor chip, wherein the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0032] According to the visual sensor chip provided by the present application, the chip includes a pixel array composed of pixel units;

[0033] For each pixel unit, the pixel unit has a corresponding intensity path, a time difference path, and a space difference path;

[0034] The intensity path is used to output the quantized value of the current output value of the photosensitive sub-unit inside the pixel unit;

[0035] The time difference path is used to output the time difference value of the pixel unit;

[0036] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0037] The time difference value is the difference and quantization result between the output value of the photosensitive sub-unit at the current moment and the output value at the previous moment within the pixel unit;

[0038] The spatial difference value is a quantized result of the difference between the output value of the photosensitive sub-unit at the current moment and the output value of the target photosensitive sub-unit at the current moment;

[0039] The pixel unit where the target photosensitive sub-unit is located is any pixel unit in the pixel array except the pixel unit.

[0040] According to the visual sensor chip provided by the present application, the intensity path includes the photosensitive subunit, the first unit deployed inside the pixel unit, and an intensity quantizer, or includes the photosensitive subunit, a differential storage subunit deployed inside the pixel unit, a frequency division gate deployed inside the pixel unit, and an intensity quantizer;

[0041] The time difference path includes the photosensitive subunit, the difference storage subunit and the time difference and quantizer;

[0042] The spatial differential path includes the photosensitive subunit, the differential storage subunit and the spatial differential and quantizer;

[0043] The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0044] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0045] The intensity quantizer is disposed inside the pixel unit, or is disposed outside the pixel unit and shared by the pixel unit and pixel units in the same column;

[0046] The photosensitive unit is used to convert the intensity of the incident light on the pixel unit at the current moment into an electrical signal output;

[0047] The differential storage subunit is used to write the current output value of the photosensitive unit; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive unit at the previous moment is written into the first storage node / the second storage node, the current output value of the photosensitive unit is written into the second storage node / the first storage node;

[0048] The first unit is used to send the current output value of the photosensitive unit to the intensity quantizer when the photosensitive unit adopts rolling exposure; and cache and output the current output value of the photosensitive unit when the photosensitive unit does not adopt rolling exposure;

[0049] The frequency division gate is used to perform low-frequency sampling on the current output value of the photosensitive sub-unit written by the differential storage sub-unit;

[0050] The intensity quantizer is used to quantize and output the output value of the first unit;

[0051] The time difference and quantizer is used to calculate and output the time difference value;

[0052] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive sub-unit.

[0053] According to the visual sensor chip provided by the present application, the intensity path output is a grayscale value or a color value;

[0054] When the intensity path outputs a color value, an external programmable demosaicer is embedded in the temporal difference and quantizer / the spatial difference and quantizer, which is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity path of the pixel unit and its surrounding units before calculating the temporal difference value / the spatial difference value.

[0055] According to the visual sensor chip provided by the present application, a trigger pulse generator is provided in each pixel unit in the pixel array;

[0056] or

[0057] All pixel units in the pixel array are commonly connected to a trigger pulse generator;

[0058] or

[0059] Dividing the pixel array into a plurality of sub-areas, wherein all pixel units in each sub-area are connected to a trigger pulse generator;

[0060] The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

[0061] According to the visual sensor chip provided by the present application, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0062] According to the visual sensor chip provided by the present application, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0063] This application provides a visual sensor chip in which each pixel unit has a unique corresponding temporal differential path and spatial differential path, or a unique corresponding light intensity quantization path, temporal differential path, and spatial differential path. This application incorporates the dual-pathway characteristics of the human visual system into existing visual sensor chips, significantly improving the visual sensor chip's ability to perceive spatiotemporal dynamic information, achieving high-precision, high-frame-rate, high-dynamic-range, and efficient and robust visual representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0065] FIG1 is a schematic diagram of the structure of a visual sensor chip provided by the present application;

[0066] FIG2 is a block diagram of the structure of the ping-pong cache provided by the present application;

[0067] FIG3 is a design diagram of a ping-pong buffer circuit provided by the present application;

[0068] FIG4 is a schematic diagram of a pixel unit structure in which both the temporal difference and quantizer and the spatial difference and quantizer provided by the present application are arranged in a pixel;

[0069] FIG5 is a second schematic diagram of a pixel unit structure in which both the temporal difference and quantizer and the spatial difference and quantizer provided by the present application are arranged in a pixel;

[0070] FIG6 is a schematic diagram of communication connections between pixel units within a chip provided by the present application;

[0071] FIG7 is a schematic diagram of a pixel unit structure in which both the temporal difference and quantizer and the spatial difference and quantizer provided by the present application are arranged outside the pixel;

[0072] FIG8 is a schematic diagram of a trigger pulse signal provided by the present application;

[0073] FIG9 is a second structural diagram of the visual sensor chip provided by the present application;

[0074] FIG10 is a schematic diagram of a pixel unit structure in which a temporal difference and quantizer, a spatial difference and quantizer, and an intensity quantizer are all arranged within a pixel as provided in the present application;

[0075] FIG11 is a second schematic diagram of a pixel unit structure in which the temporal difference and quantizer, the spatial difference and quantizer, and the intensity quantizer provided by the present application are all arranged within a pixel;

[0076] FIG12 is a schematic diagram of a pixel unit structure in which the temporal difference and quantizer and the spatial difference and quantizer provided in the present application are arranged outside the pixel and the intensity quantizer is arranged inside the pixel.

[0077] FIG13 is a third schematic diagram of a pixel unit structure in which the temporal difference and quantizer, the spatial difference and quantizer, and the intensity quantizer provided by the present application are all arranged within a pixel;

[0078] FIG14 is a fourth schematic diagram of the pixel unit structure provided by the present application when the temporal difference and quantizer, the spatial difference and quantizer, and the intensity quantizer are all arranged in a pixel. DETAILED DESCRIPTION

[0079] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0080] The visual sensor chip and visual sensor of the present application are described below with reference to FIG. 1 to FIG. 14 .

[0081] First, explain the abbreviations and key technical terms in this application:

[0082] APS: Active Pixel Sensor

[0083] CFA: Color Filter Array

[0084] CIS: CMOS Image Sensor

[0085] DAVIS: Dynamic Active Pixel Vision Sensor

[0086] DVS: Dynamic Vision Sensor

[0087] EVS: Event-based Vision Sensor

[0088] fps: frames per second (frame rate unit)

[0089] PD: Photodiode

[0090] SD: Spatial Difference

[0091] TD: Time Difference

[0092] First, existing DVS technology has the following defects:

[0093] 1. Limited accuracy of single-valued signals

[0094] DVS outputs ±1-bit information (for example, + indicates an increase in light intensity, - indicates a decrease in light intensity, and 0 indicates that the light intensity remains unchanged). It then outputs positive and negative pulses using ±1-bit information and cannot sense the degree of light intensity change. This makes it susceptible to noise interference, has low information content, and cannot adapt to complex environments.

[0095] 2. Lack of spatial difference information

[0096] The human visual system perceives both temporal and spatial variations, making it more sensitive and robust to the external world. However, DVS can only output information about temporal changes in visual signals, which is highly susceptible to interference (for example, when there is external flickering light, DVS fails and cannot separate signal changes caused by light source changes and motion). It also lacks spatial differential information.

[0097] In view of this, the present application provides a visual sensor chip, wherein the chip includes a plurality of pixel units arranged in an array;

[0098] For each pixel unit, the pixel unit has a corresponding time difference path and space difference path;

[0099] The time difference path is used to output the time difference value of the pixel unit;

[0100] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0101] The time difference value is the difference and quantization result between the output value of the photosensitive sub-unit at the current moment and the output value at the previous moment within the pixel unit;

[0102] The spatial difference value is a quantized result of the difference between the output value of the photosensitive sub-unit at the current moment and the output value of the target photosensitive sub-unit at the current moment;

[0103] The pixel unit where the target photosensitive sub-unit is located is any pixel unit in the pixel array except the pixel unit.

[0104] Specifically, the present application implements a visual sensor with two output channels: a time difference channel TD and a space difference channel SD.

[0105] Among them, the TD path outputs the current pixel unit (x, y) at t n The time difference value TD(x,y,t n ), which can be expressed as: TD(x,y,t n )=Q TD (I(x,y,t n )-I(x,y,t n-1 ))

[0106] In the above formula, I(x,y,t n ) and I(x,y,t n-1 ) are the photosensitive subunits inside the current pixel unit (x, y) at t n time and the previous time t n-1 The output value, Q TD Quantization method used for the time difference path.

[0107] The SD path outputs the current pixel unit (x, y) at t n Spatial difference value SD at time * (x,y,t n ), which can be expressed as: SD * (x,y,t n )=Q SD (I(x,y,t n )-I(x * ,y * ,t n ))

[0108] In the above formula, I(x * ,y * ,t n ) is the target photosensitive unit at t n Output value at the moment (current moment);

[0109] The pixel unit where the target photosensitive sub-unit is located is any pixel unit in the pixel array except the pixel unit.

[0110] The target photosensitive unit may be one or more, resulting in the following situations:

[0111] (1) When there is only one target photosensitive unit, the SD pathway only performs differentiation in a certain direction.

[0112] (2) When there are two target photosensitive units (respectively denoted as the first target photosensitive unit and the second target photosensitive unit) and the pixel unit where the first target photosensitive unit is located, the pixel unit where the second target photosensitive unit is located, and the pixel unit are located on the same straight line, the SD path only performs differentiation in one direction; in this case, the differential accuracy of the pixel unit is higher than (1).

[0113] (3) When there are two target photosensitive units (respectively denoted as the first target photosensitive unit and the second target photosensitive unit) and the pixel unit where the first target photosensitive unit is located, the pixel unit where the second target photosensitive unit is located, and the pixel unit are not in a straight line, the SD path performs differentiation in two directions; at this time, the pixel unit can obtain spatial differential information in multiple directions.

[0114] (4) When there are multiple (greater than two) target photosensitive units and the pixel units where all the photosensitive units are located and the line connecting the pixel units are a straight line, the SD path only performs differentiation in one direction. At this time, the differential accuracy of the pixel units is higher than (2).

[0115] (5) When there are multiple (greater than two) target photosensitive units and the pixel units where all the photosensitive units are located and the lines connecting the pixel units are not straight lines, the SD path performs differentiation in at least two directions.

[0116] It can be seen that the differential accuracy of the pixel unit is mainly affected by the number of differential directions and the number of target photosensitive units. In fact, the differential accuracy of the pixel unit is also affected by the distance between the pixel unit where the target photosensitive unit is located and the pixel unit. Therefore, the present application preferably configures the target photosensitive unit to include a first target photosensitive unit and a second target photosensitive unit; the pixel unit where the first target photosensitive unit is located (hereinafter referred to as the first pixel unit) and the pixel unit where the first target photosensitive unit is located (hereinafter referred to as the second pixel unit) are both adjacent to the pixel unit, and the lines connecting the first pixel unit, the pixel unit and the second pixel unit are not on the same straight line.

[0117] For example, the first pixel unit and the second pixel unit are pixel unit (x+1, y) and pixel unit (x, y+1), respectively; here, 1 refers to the spacing of 1 pixel unit.

[0118] At this time, the SD path output pixel unit (x, y) is at t n The pixel value at the moment and the pixel unit (x+1, y) at t n The spatial difference value SD between the pixel values ​​at each moment x (x,y,t n), and the pixel unit (x, y) at t n The pixel value at the moment is the same as the pixel unit (x, y+1) at t n The spatial difference value SD between the pixel values ​​at each moment y (x,y,t n ); SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y,t n )) SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y+1,t n )) I(x,y+1,t n )

[0119] Or for example, the first pixel unit and the second pixel unit are pixel unit (x-1, y+1) and pixel unit (x+1, y+1) respectively; in this case, the SD path outputs pixel unit (x, y) at t n The pixel value at the moment is the same as the pixel unit (x-1, y+1) at t n The spatial difference value SD between the pixel values ​​at each moment ↙ (x,y,t n ), and the pixel unit (x, y) at t n The pixel value at the moment and the pixel unit (x+1, y+1) at t n The spatial difference value SD between the pixel values ​​at each moment ↘ (x,y,t n ); SD ↙ (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y+1,t n )) SD ↘ (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y+1,t n ))

[0120] In the above formula, Q SD Quantization method used for spatial difference path.

[0121] I(x+1,y,t n )、I(x,y+1,t n )、I(x+1,y+1,t n) and I(x-1,t+1,t n ) are the photosensitive subunits inside the pixel units (x+1,y), (x,y+1), (x+1,y+1) and I(x-1,y+1) at t n Output value at time.

[0122] All the signals mentioned above are three-dimensional quantities, including the spatial two-dimensional quantities x and y and the time dimension t.

[0123] FIG1 is a schematic structural diagram of a corresponding visual sensor chip, which takes the first pixel unit and the second pixel unit as an example, where the first pixel unit is (x+1, y) and the second pixel unit is (x, y+1) respectively.

[0124] The visual sensor chip provided in this application integrates the dual-channel characteristics of the human visual system into existing visual sensor chips, thereby greatly improving the visual sensor chip's ability to perceive spatiotemporal dynamic information, and achieving high-precision, high frame rate, high dynamic range and efficient and robust visual representation.

[0125] Based on the above embodiments, as an optional embodiment, the time difference path includes the photosensitive subunit, a differential storage subunit disposed inside the pixel unit, and a time difference and quantizer;

[0126] The spatial differential path includes the photosensitive subunit, the differential storage subunit and a spatial differential and quantizer;

[0127] The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0128] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0129] The photosensitive subunit is used to convert the intensity of the incident light on the pixel unit at the current moment into an electrical signal for output;

[0130] It can be understood that the output value of the photosensitive unit is an electrical signal such as charge, voltage and current, which represents the intensity of the incident light felt by the current pixel position at the current moment. The greater the light intensity, the higher the pixel value.

[0131] The differential storage subunit is used to write the current output value of the photosensitive unit; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive unit at the previous moment is written into the first storage node / the second storage node, the current output value of the photosensitive unit is written into the second storage node / the first storage node;

[0132] It can be understood that in order to construct a time differential path and a space differential path, the present application sets two storage nodes inside each pixel unit (a first storage node and a second storage node, and the first storage node and the second storage node use a ping-pong cache method to cache data. Ping-pong cache means that if the photosensitive unit is stored in the first storage node at this moment, it will be stored in the second storage node at the next moment, and then in the first storage node at the next moment, and so on alternately), and outputs two time signals (I(x, y, t n ), I(x,y,t n-1 )).

[0133] Figure 2 is a block diagram of the ping-pong buffer. Figure 3 shows a possible specific circuit design, and there are more than one actual circuit schematics.

[0134] The time difference and quantizer is used to calculate and output the time difference value;

[0135] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive sub-unit.

[0136] In other words, depending on whether the temporal difference and quantizer and the spatial difference and quantizer are arranged in the pixel, the present application provides four schematic diagrams of pixel unit structures.

[0137] The first type: a temporal difference and quantizer and a spatial difference and quantizer are set in each pixel unit;

[0138] Figures 4 and 5 are schematic diagrams of the pixel unit structure when both the temporal difference and quantizer and the spatial difference and quantizer are arranged within the pixel. Figure 4 uses the example of a first pixel unit and a second pixel unit with the values ​​of (x+1, y) and (x, y+1), respectively. Figure 5 uses the example of a first pixel unit and a second pixel unit with the values ​​of (x-1, y+1) and (x+1, y+1), respectively. The first method uses a pixel unit readout method, whereby each pixel unit directly reads the temporal difference value and the spatial difference value.

[0139] It should be noted that due to the existence of the differential quantization path, a communication connection is established between each pixel unit in the pixel array and the pixel unit where its corresponding target photosensitive sub-unit is located. FIG6 is a schematic diagram of the communication connection between pixel units in the chip provided by the present application; In FIG6, the square box represents the pixel unit, and the connecting line represents the pixel unit at the current time t n The left figure shows an example where the first pixel unit and the second pixel unit are (x+1, y) and (x, y+1), respectively. The right figure shows an example where the first pixel unit and the second pixel unit are (x-1, y+1) and (x+1, y+1), respectively.

[0140] The second method is to set a time difference and quantizer and a space difference and quantizer for each column of pixel units, so as to be shared by the pixel units in the column.

[0141] Figure 7 shows a schematic diagram of a pixel unit structure where both the temporal and spatial difference and quantization units are located outside the pixel. This diagram uses the example of a first pixel unit with the locations (x+1, y) and a second pixel unit with the locations (x, y+1), respectively. The second approach uses a column-level readout method, where each column shares one temporal difference and quantization unit and one spatial difference and quantization unit.

[0142] As shown in Figure 7, SD difference and quantizer ① calculate SD in turn. x (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y,t n )) SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y+1,t n )) SD x (x,y+1,t n )=Q SD (I(x,y+1,t n )-I(x+1,y+1,t n )) SD y (x,y+1,t n )=Q SD (I(x,y+1,t n )-I(x,y+2,t n ))

[0143] SD difference and quantizer ② calculate SD in turn x (x+1,y,t n )=Q SD (I(x+1,y,t n) - I(x + 2, y, t n )) SD y (x + 1, y, t n ) = Q SD (I(x + 1, y, t n ) - I(x + 1, y + 1, t n )) SD x (x + 1, y + 1, t n ) = Q SD (I(x + 1, y + 1, t n ) - I(x + 2, y + 1, t n )) SD y (x + 1, y + 1, t n ) = Q SD (I(x + 1, y + 1, t n ) - I(x + 1, y + 2, t n ))

[0144] The SD difference and quantizer ③ and so on.

[0145] The TD difference and quantizer ④ calculates TD(x, y, t n ) = Q TD (I(x, y, t n ) - I(x, y, t n-1 )) TD(x, y + 1, t n ) = Q TD (I(x, y + 1, t n ) - I(x, y + 1, t n-1 )) TD(x, y + 2, t n ) = Q TD (I(x, y + 2, t n ) - I(x, y + 2, t n-1 ))

[0146] The TD difference and quantizer ⑤ calculates TD(x + 1, y, t n ) = Q TD (I(x + 1, y, t n ) - I(x + 1, y, t n-1 )) TD(x + 1, y + 1, t n ) = Q TD (I(x + 1, y + 1, t<00001​​​​​​​​​​))

[0147] The same goes for TD difference and quantizer ⑥.

[0148] The principle is the same as the example in which the first pixel unit and the second pixel unit are (x-1, y+1) and (x+1, y+1), and will not be repeated here.

[0149] The third method is to set a temporal difference and quantizer in each pixel unit, and set a spatial difference and quantizer in each column of pixel units to be shared by the pixel units in the column.

[0150] The fourth type: a spatial difference and quantizer is set in each pixel unit, and a temporal difference and quantizer is set in each column of pixel units to be shared by the pixel units in this column.

[0151] The third and fourth types are evolved from the temporal difference and quantizer and the spatial difference and quantizer within the pixel and the temporal difference and quantizer and the spatial difference and quantizer outside the pixel, and will not be described in detail here.

[0152] It should be noted that the photosensitive subunits inside the pixel unit (x, y) in the above four methods are n time and the previous time t n-1 The output value I(x,y,t n ) and I(x,y,t n-1 ) is input into the spatial difference and quantizer at the same time. The spatial difference and quantizer first selects I(x,y,t n ) and abandon I(x,y,t n-1 ), followed by spatial differentiation and quantization operations.

[0153] In addition, the quantization method used by the time difference and quantizer and the spatial difference and quantizer is ADC (analog-to-digital converter) quantization;

[0154] The ADC quantization may be single-bit ADC quantization, multi-bit ADC quantization with a sign bit (positive and negative sign bits represent enhancement / weakening, respectively), or multi-bit ADC quantization without a sign bit.

[0155] The present application prefers multi-bit ADC quantization with a sign bit (positive and negative sign bits represent enhancement / weakening, respectively).

[0156] Multi-bit ADC quantization with a sign bit can not only measure the size of the differential result but also give the positive and negative signs of the differential result during the process of quantizing the analog signal into a digital signal. This allows for more precise perception of light intensity changes and improves pixel accuracy. In addition, because it is represented by multiple bits, the signal-to-noise ratio is improved.

[0157] Based on the above embodiment, as an optional embodiment, a trigger pulse generator is provided in each pixel unit in the pixel array;

[0158] or

[0159] All pixel units in the pixel array are commonly connected to a trigger pulse generator;

[0160] or

[0161] Dividing the pixel array into a plurality of sub-areas, wherein all pixel units in each sub-area are connected to a trigger pulse generator;

[0162] The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the exposure moment of the corresponding photosensitive subunit.

[0163] The trigger pulse generator is used to generate a trigger signal, which controls the photosensitive unit to expose, that is, determines the time t of collecting the signal. n Figure 8 is a schematic diagram of a trigger pulse signal, where the horizontal axis x represents time and the vertical axis y represents the digital signal amplitude. As can be seen from Figure 8, the moment the trigger pulse is generated is equivalent to the sampling moment of the pixel unit, which is t n , t n-1 , t n-2 These moments can be set not only as fixed intervals as shown in the left figure, but also as adaptive, programmable, variable intervals as shown in the right figure. This adaptive interval adapts to the changing nature of external visual signals, using a higher sampling frequency when the amount of change is large and the frequency of change is high, and a lower sampling frequency for low-frequency signals, thereby reducing data volume and energy consumption.

[0164] Based on the above embodiment, as an optional embodiment, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0165] It's conceivable that if a trigger pulse generator is designed into a pixel unit, it can independently and adaptively adjust the triggering moment for calculating the spatiotemporal differential signal based on the light intensity level perceived by the pixel unit. Each pixel has a different triggering moment. This allows the pixel to output information at any time, increasing flexibility and reducing output latency.

[0166] Therefore, this application supports all pixel units in the array sharing one trigger pulse generator, and in this case only full-array synchronous exposure can be used.

[0167] It also supports the use of a trigger pulse generator for each pixel unit in the array, and the exposure mode can be set to synchronous exposure or asynchronous exposure according to needs.

[0168] It also supports multiple pixels forming a macroblock to share an intra-pixel pulse trigger generator to reduce the complexity of chip design and the occupied area. In this case, the pixel units in the same macroblock are exposed synchronously, and the exposure mode between macroblocks can be set to synchronous exposure or asynchronous exposure according to needs.

[0169] Based on the above embodiment, as an optional embodiment, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0170] The time difference and quantizer of the present application includes a time difference calculator and a quantizer, and the space difference and quantizer includes a space difference calculator and a quantizer. Each pixel unit outputs an electrical signal representing the light intensity through a photosensitive unit. After the electrical signal enters the storage node, it is subjected to time difference calculation / spatial difference calculation and quantization before being output. The execution order of the differential calculation and quantization here can be interchangeable, that is, two analog signals are quantized into digital signals, and then differential calculation is performed in the digital domain, or the analog signals are first differentially calculated in the analog domain, and then the differential results are quantized into digital signals.

[0171] Secondly, because it is difficult to achieve universal visual perception through DVS alone, existing visual sensors usually need to combine DVS with CIS with high spatial resolution and high image quality. For example, DAVIS cameras and cameras based on hybrid pixel arrays. Among them, the DAVIS camera combines CIS with DVS, and the output current of a single photodiode (PD, used to convert incident light into current) is simultaneously utilized by the APS circuit and the DVS circuit, which can record both single-frame images (frame-based sampling) and event information (event-based sampling). This allows DAVIS to have the advantages of both the high image quality of CIS and the high temporal resolution of DVS cameras. However, the DAVIS camera has the following defects:

[0172] 1. DAVIS cameras inherit the limited accuracy of DVS single-value signals.

[0173] 2. Lack of spatial difference information

[0174] When there's a large flash or dramatic changes in light intensity in the image, all TD pixels output events, leading to saturation. The DVS pathway is unable to output valid information, and the CIS pathway, due to frame rate limitations, is unable to respond immediately. These extreme conditions are common in autonomous driving and are crucial for driving safety, such as when entering or exiting tunnels or capturing camera flashes at night. This means that a visual sensor with only CIS and DVS pathways, from the perspective of visual primitives, provides incomplete information. In contrast, the human visual system can rapidly recognize moving objects, whether at noon or dusk, in open scenes or partially obscured, achieving far greater robustness and versatility than existing DAVIS. This is because the human eye achieves efficient and robust visual representation by combining different visual primitives.

[0175] Therefore, DVS can only output temporal change information of visual signals, and this information is very easy to be interfered with and lacks spatial difference information.

[0176] On this basis, the present application provides a visual sensor chip, wherein the chip includes a pixel array composed of pixel units;

[0177] For each pixel unit, the pixel unit has a corresponding intensity path, a time difference path, and a space difference path;

[0178] The intensity path is used to output the quantized value of the current output value of the photosensitive sub-unit inside the pixel unit;

[0179] The time difference path is used to output the time difference value of the pixel unit;

[0180] The spatial difference path is used to output the spatial difference value of the pixel unit;

[0181] The time difference value is the difference and quantization result between the output value of the photosensitive sub-unit at the current moment and the output value at the previous moment within the pixel unit;

[0182] The spatial difference value is a quantized result of the difference between the output value of the photosensitive sub-unit at the current moment and the output value of the target photosensitive sub-unit at the current moment;

[0183] The pixel unit where the target photosensitive sub-unit is located is any pixel unit in the pixel array except the pixel unit.

[0184] Specifically, the present application implements a three-channel output visual sensor, which includes a light intensity quantization channel, a time difference channel TD, and a space difference channel SD.

[0185] Among them, the light intensity quantization path outputs the photosensitive unit inside the current pixel unit (x, y) at t n The quantization result of the output value at time A(x,y,t n ), which can be expressed as follows: A(x,y,t n )=Q(I(x,y,t n ))

[0186] Where Q is the quantization method used by the intensity channel.

[0187] The light intensity quantization pathway signals are all three-dimensional quantities, including the spatial two-dimensional quantities of x and y and the time dimension t.

[0188] The relevant parts of the TD path and the SD path are consistent with the TD path and the SD path of the pixel unit of the visual sensor chip described in the first aspect, and will not be repeated here.

[0189] FIG9 is a schematic structural diagram of a corresponding visual sensor chip, taking as an example the first pixel unit and the second pixel unit being (x+1, y) and (x, y+1) respectively.

[0190] The visual sensor chip provided in this application adds a spatial differential pathway that mimics the human retina to the existing visual sensor pixel unit that only contains a temporal differential pathway and a color pathway, thereby significantly improving the visual sensor's ability to reconstruct spatiotemporal dynamic information with high precision, forming an efficient and robust visual representation.

[0191] Based on the above embodiments, as an optional embodiment, the intensity path includes the photosensitive unit, the first unit disposed inside the pixel unit, and an intensity quantizer, or includes the photosensitive unit, the difference storage subunit disposed inside the pixel unit, the frequency division gate disposed inside the pixel unit, and the intensity quantizer;

[0192] The time difference path includes the photosensitive subunit, the difference storage subunit and the time difference and quantizer;

[0193] The spatial differential path includes the photosensitive subunit, the differential storage subunit and the spatial differential and quantizer;

[0194] The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0195] The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column;

[0196] The intensity quantizer is disposed inside the pixel unit, or is disposed outside the pixel unit and shared by the pixel unit and pixel units in the same column;

[0197] The photosensitive unit is used to convert the intensity of the incident light on the pixel unit at the current moment into an electrical signal output;

[0198] The differential storage subunit is used to write the current output value of the photosensitive unit; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive unit at the previous moment is written into the first storage node / the second storage node, the current output value of the photosensitive unit is written into the second storage node / the first storage node;

[0199] The first unit is used to send the current output value of the photosensitive unit to the intensity quantizer when the photosensitive unit adopts rolling exposure; and cache and output the current output value of the photosensitive unit when the photosensitive unit does not adopt rolling exposure;

[0200] The frequency divider gate is used to perform low-frequency sampling on the current output value of the photosensitive subunit written by the differential storage subunit. It can be understood that the configuration of the frequency divider gate eliminates the need for the pixel unit to set up a separate storage node for the intensity path, allowing it to directly access the stored information of the differential storage subunit. The frequency divider gate subunit functions as a low-frequency sampling device. Assuming the differential path array is 600 fps, then because the ping-pong storage is set to update the data of the first and second storage nodes at a frequency of 300 Hz, and the light intensity quantizer is 30 fps, the frequency divider gate only needs to reduce the sampling frequency by ten times (equivalent to the frequency divider gate only outputting one signal for every ten received signals, discarding the other nine).

[0201] The intensity quantizer is used to quantize and output the output value of the first unit;

[0202] The time difference and quantizer is used to calculate and output the time difference value;

[0203] The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive sub-unit.

[0204] In other words, when the intensity path includes the photosensitive unit, the first unit deployed inside the pixel unit and the intensity quantizer, this application provides 8 pixel unit structures based on whether the time difference and quantizer, the spatial difference and quantizer and the intensity quantizer are arranged in the pixel.

[0205] A: Temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged in the pixel;

[0206] Figures 10 and 11 are schematic diagrams of pixel unit structures when the corresponding temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within a pixel. Figure 10 uses the example of a first pixel unit and a second pixel unit being (x+1, y) and (x, y+1), respectively. Figure 11 uses the example of a first pixel unit and a second pixel unit being (x-1, y+1) and (x+1, y+1), respectively.

[0207] B: The intensity quantizer is placed inside the pixel, while the temporal difference quantizer and spatial difference quantizer are placed outside the pixel;

[0208] Figure 12 is a schematic diagram of the pixel unit structure when the temporal difference and quantizer and the spatial difference and quantizer are arranged outside the pixel and the intensity quantizer is arranged inside the pixel. This diagram uses the example of a first pixel unit and a second pixel unit with the values ​​of (x+1, y) and (x, y+1), respectively. The principles are the same when the first pixel unit and the second pixel unit are (x+1, y+1) and (x-1, y+1), respectively, and are not further described here.

[0209] C: The temporal difference and quantizer are placed inside the pixel, while the spatial difference and quantizer and the intensity quantizer are placed outside the pixel;

[0210] D: The spatial difference and quantizer are placed inside the pixel, while the temporal difference and quantizer and the intensity quantizer are placed outside the pixel;

[0211] E: Temporal difference and quantizer and spatial difference and quantizer are arranged inside the pixel, and intensity quantizer is arranged outside the pixel;

[0212] F: The temporal difference and quantizer and the intensity quantizer are arranged inside the pixel, and the spatial difference and quantizer are arranged outside the pixel;

[0213] G: The spatial difference and quantizer and the intensity quantizer are arranged inside the pixel, and the temporal difference and quantizer are arranged outside the pixel;

[0214] H: The temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all placed outside the pixel.

[0215] The pixel unit structures corresponding to C to H are similar to those without any further description.

[0216] When the intensity path includes the photosensitive subunit, the differential storage subunit deployed inside the pixel unit, the frequency division selector and the intensity quantizer deployed inside the pixel unit, this application also provides 8 pixel unit structures according to whether the time differential and quantizer, the spatial differential and quantizer and the intensity quantizer are arranged in the pixel.

[0217] I: Temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged in the pixel;

[0218] Figures 13 and 14 are schematic diagrams of pixel unit structures when the corresponding temporal difference and quantizer, spatial difference and quantizer, and intensity quantizer are all arranged within a pixel. Figure 13 uses the example of a first pixel unit and a second pixel unit being (x+1, y) and (x, y+1), respectively. Figure 14 uses the example of a first pixel unit and a second pixel unit being (x-1, y+1) and (x+1, y+1), respectively.

[0219] II: Temporal difference and quantizer and spatial difference and quantizer are placed inside the pixel, and intensity quantizer is placed outside the pixel;

[0220] III: Temporal difference and quantizer and intensity quantizer are arranged inside the pixel, and spatial difference and quantizer are arranged outside the pixel;

[0221] IV: The spatial difference and quantizer and the intensity quantizer are arranged inside the pixel, and the temporal difference and quantizer are arranged outside the pixel;

[0222] V: Temporal difference and quantizer are placed inside the pixel, spatial difference and quantizer and intensity quantizer are placed outside the pixel;

[0223] VI: The spatial difference and quantizer are placed inside the pixel, while the temporal difference and quantizer and the intensity quantizer are placed outside the pixel;

[0224] VII: The intensity quantizer is placed inside the pixel, while the temporal difference quantizer and the spatial difference quantizer are placed outside the pixel;

[0225] VIII: The temporal difference and quantizer, the spatial difference and quantizer, and the intensity quantizer are all arranged outside the pixel.

[0226] II to VIII have the same purpose and will not be elaborated here.

[0227] It should be noted that the photosensitive subunits inside the pixel unit (x, y) in the above 8 modes are n time and the previous time t n-1 The output value I(x,y,t n ) and I(x,y,t n-1) is input to the frequency divider gate at the same time, and the frequency divider gate first selects I(x,x,t n ) and abandon I(x,y,t n-1 ), followed by spatial differentiation and quantization operations.

[0228] Note that, like the time difference and quantizer and the spatial difference and quantizer, the quantization method used by the intensity quantizer of this application is ADC quantization; preferably, multi-bit ADC quantization with a sign bit (positive and negative sign bits represent enhancement / weakening respectively).

[0229] Based on the above embodiments, as an optional embodiment, the intensity path output is a grayscale value or a color value;

[0230] When the intensity path outputs a color value, an external programmable demosaicer is embedded in the temporal difference and quantizer / the spatial difference and quantizer, which is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity path of the pixel unit and its surrounding units before calculating the temporal difference value / the spatial difference value.

[0231] CIS uses a color filter array (CFA) covered on the pixel array to sense visible light of different wavelengths to obtain a color image. CFA usually contains filters of three colors: red, green and blue, so the color path is sometimes referred to as "RGB". The three color filters are usually arranged in a Bayer array. However, there are other types of CFA, such as the CMY array based on three complementary colors (cyan, magenta, and yellow), which has a higher transmittance. The present application can be applied without adding a color filter array, in which case the light intensity quantization path outputs grayscale values, or a color filter array (such as RGB color filter) can be added, in which case the color value is output.

[0232] If the output of the intensity path is a color value, the corresponding pixel is covered by a color filter. In this case, the output value of the photosensitive unit in the spatiotemporal difference path only contains information of a certain color channel. Typically, such as red, green, and blue channels, the pixels in the array are divided into red, green, and blue color information.

[0233] Therefore, when performing temporal / spatial differencing, all colors are first mosaicked. That is, for an X-colored pixel, the Y color output at that location is first obtained based on the surrounding Y-colored pixels, and the Z color output at that location is obtained based on the surrounding Z-colored pixels. In this way, each pixel has three color output channels, and conventional spatial differencing operations can be performed.

[0234] The present application embeds an external programmable demosaicer in the temporal difference and quantizer / the spatial difference and quantizer, and uses an internal demosaicing algorithm to implement a demosaicing operation.

[0235] Among them, the mosaic algorithm is not unique, and can be implemented by selecting two points, four points, or even 16 points around it.

[0236] Of course, when the differential and quantization units are distributed in columns, the demosaicer can also be independently set inside the pixel.

[0237] Alternatively, you can skip the demosaicer and directly perform a spatial difference operation on the color channels corresponding to the current pixel. This can be done on the same color channels (e.g., pixel X against pixel X) or on different color channels (e.g., pixel X against pixel Y), followed by additional algorithmic post-processing.

[0238] Based on the above embodiments, as an optional embodiment, a trigger pulse generator is provided in each pixel unit in the pixel array;

[0239] or

[0240] All pixel units in the pixel array are commonly connected to a trigger pulse generator;

[0241] or

[0242] Dividing the pixel array into a plurality of sub-areas, wherein all pixel units in each sub-area are connected to a trigger pulse generator;

[0243] The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the exposure moment of the corresponding photosensitive subunit.

[0244] Based on the above embodiments, as an optional embodiment, pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

[0245] Based on the above embodiments, as an optional embodiment, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0246] The above process is the same as that of the visual sensor chip described in the first aspect and will not be repeated here.

[0247] The visual sensor chip of the first aspect or the second aspect of the present application is connected to an image processing module, which is integrated with the pixel array in the same chip and can also be arranged on a computer or other device. It is used to process the chip output signal.

[0248] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0249] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0250] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual sensor chip, the chip comprising a pixel array composed of pixel units; For each pixel unit, the pixel unit has a corresponding time difference path and a space difference path; The time difference path is used to output the time difference value of the pixel unit; The spatial difference path is used to output the spatial difference value of the pixel unit; The time difference value is the difference and quantization result between the output value of the photosensitive subunit at the current moment and the output value at the previous moment inside the pixel unit; The spatial difference value is a difference and quantization result between the output value of the photosensitive subunit at the current moment and the output value of the target photosensitive subunit at the current moment; The pixel unit where the target photosensitive subunit is located is any pixel unit in the pixel array except the pixel unit.

2. The visual sensor chip according to claim 1, wherein: The time difference path includes the photosensitive subunit, a difference storage subunit disposed inside the pixel unit, and a time difference and quantizer; The spatial difference path includes the photosensitive subunit, the difference storage subunit and the spatial difference and quantizer; The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column; The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column; The photosensitive subunit is used to convert the light intensity of the incident light of the pixel unit at the current moment into an electrical signal for output; The differential storage subunit is used to write the output value of the photosensitive subunit at the current moment; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive subunit at the previous moment is written in the first storage node / the second storage node, the output value of the photosensitive subunit at the current moment is written in the second storage node / the first storage node; The time difference and quantizer is used to calculate and output the time difference value; The spatial difference and quantizer is used to calculate and output the spatial difference value based on the current output value of the target photosensitive subunit.

3. The visual sensor chip according to claim 1, wherein: A trigger pulse generator is provided in each pixel unit in the pixel array; or All pixel units in the pixel array are commonly connected to a trigger pulse generator; or Dividing the pixel array into a plurality of sub-areas, all pixel units in each sub-area are connected to a trigger pulse generator; The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

4. The visual sensor chip according to claim 1, wherein: Pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

5. The visual sensor chip according to claim 1, wherein: The exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

6. A visual sensor chip, the chip comprising a pixel array composed of pixel units; For each pixel unit, the pixel unit has a corresponding intensity path, a time difference path and a space difference path; The intensity path is used to output the quantized value of the current output value of the photosensitive subunit inside the pixel unit; The time difference path is used to output the time difference value of the pixel unit; The spatial difference path is used to output the spatial difference value of the pixel unit; in, The time difference value is the difference and quantization result between the output value of the photosensitive subunit in the pixel unit at the current moment and the output value at the previous moment; The spatial difference value is a difference and quantization result between the output value of the photosensitive subunit at the current moment and the output value of the target photosensitive subunit at the current moment; The pixel unit where the target photosensitive subunit is located is the pixel unit in the pixel array except the pixel unit Any pixel unit other than .

7. The visual sensor chip according to claim 6, wherein: The intensity path includes the photosensitive subunit, a first unit disposed inside the pixel unit and an intensity quantizer, or includes the photosensitive subunit, a differential storage subunit disposed inside the pixel unit, a frequency division gate disposed inside the pixel unit and an intensity quantizer; The time difference path includes the photosensitive subunit, the difference storage subunit and the time difference and quantizer; The spatial difference path includes the photosensitive subunit, the difference storage subunit and the spatial difference and quantizer; The time difference and quantizer are deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column; The spatial difference and quantizer is deployed inside the pixel unit, or deployed outside the pixel unit and shared by the pixel unit and the pixel units in the same column; The intensity quantizer is disposed inside the pixel unit, or is disposed outside the pixel unit and shared by the pixel unit and the pixel units in the same column; The photosensitive subunit is used to convert the light intensity of the incident light of the pixel unit at the current moment into an electrical signal output; The differential storage subunit is used to write the output value of the photosensitive subunit at the current moment; wherein the differential storage subunit includes a first storage node and a second storage node, and when the output value of the photosensitive subunit at the previous moment is written in the first storage node / the second storage node, the output value of the photosensitive subunit at the current moment is written in the second storage node / the first storage node; The first unit is used to send the current output value of the photosensitive unit to the intensity quantizer when the photosensitive unit adopts rolling exposure; when the photosensitive unit does not adopt rolling exposure, cache and output the current output value of the photosensitive unit; The frequency division gate is used to perform low-frequency sampling on the current output value of the photosensitive subunit written by the differential storage subunit; The intensity quantizer is used to quantize and output the output value of the first unit; The time difference and quantizer is used to calculate and output the time difference value; The spatial difference and quantizer is used to determine the current moment of the target photosensitive subunit based on the current moment of the target photosensitive subunit. The output value calculates and outputs the spatial difference value.

8. The visual sensor chip according to any one of claims 6 or 7, wherein: The intensity path output is a grayscale value or a color value; When the intensity path outputs a color value, an external programmable demosaicer is embedded in the temporal differential and quantizer / the spatial differential and quantizer, which is used to determine the output values ​​of all color channels of the pixel unit based on the color values ​​output by the intensity paths of the pixel unit and its surrounding units before calculating the temporal differential value / the spatial differential value.

9. The visual sensor chip according to claim 6, wherein: A trigger pulse generator is provided in each pixel unit in the pixel array; or All pixel units in the pixel array are commonly connected to a trigger pulse generator; or Dividing the pixel array into a plurality of sub-areas, all pixel units in each sub-area are connected to a trigger pulse generator; The trigger pulse generator is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable interval, so as to control the start exposure time and exposure duration of the corresponding photosensitive subunit.

10. The visual sensor chip according to claim 9, wherein: Pixel units connected to the same trigger pulse generator are exposed synchronously, and pixel units connected to different trigger pulse generators are exposed synchronously or asynchronously.

11. The visual sensor chip according to claim 9, wherein: The exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

Images