Vision sensor chip based on hybrid array

Abstract

Provided in the present application is a vision sensor chip based on a hybrid array. The vision sensor chip comprises a pixel array and a vision sensing path arranged corresponding to the pixel array, wherein the vision sensing path comprises an intensity path, a time difference path and a space difference path; the pixel array comprises a plurality of multiplexed pixel units and a plurality of single pixel units; each multiplexed pixel unit is a pixel unit multiplexed by two elements from among intensity, time difference and space difference; each single pixel unit is a pixel unit having an element different from the elements of the multiplexed pixel unit; the intensity path is used for determining a quantized value after incident light intensity is converted into an electrical signal; the time difference path is used for determining a time difference value; and the space difference path is used for determining a space difference value. In the present application, by means of a three-path vision sensing chip architecture based on a hybrid array, the sensing capability of a vision sensor chip regarding space-time dynamic information can be greatly improved, and high-precision, high-frame-rate, high-dynamic-range, efficient and robust vision representation is realized.

Description

A visual sensor chip based on hybrid array

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311420673.8, filed on October 30, 2023, entitled “A visual sensor chip based on a hybrid array,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present application relates to the field of optoelectronic imaging technology, and in particular to a hybrid array-based visual sensor chip. Background Art

[0004] A visual sensor is a device used to perceive visible light information in the environment and convert it into electrical signals. It is widely used in digital cameras and other electronic optical devices.

[0005] Currently, the most common visual sensor is the CIS (CMOS Image Sensor). CIS captures video using a frame-based sampling principle, offering advantages such as high pixel array resolution, excellent color reproduction, and high image quality. However, the image signals captured by CIS have a small dynamic range and slow capture speed. Another new type of image sensor, the DVS (Dynamic Vision Sensor), can perceive changes in dynamic scenes in the form of a sparse event stream, offering faster capture speeds and a larger dynamic range. However, these sensors suffer from low spatial resolution and excessive loss of effective information.

[0006] Therefore, the present application urgently needs to provide an improved visual sensor.

[0007] Summary of the Invention

[0008] To overcome the above problems, the present application provides a visual sensor chip based on a hybrid array. Through the three-channel visual sensor chip architecture based on the hybrid array, the visual sensor chip's perception ability of spatiotemporal dynamic information can be greatly improved, and high-precision, high frame rate, high dynamic range and efficient and robust visual representation can be achieved.

[0009] The present application provides a visual sensor chip based on a hybrid array, comprising a pixel array and a corresponding visual sensing path; the visual sensing path comprises an intensity path, a time difference path, and a spatial difference path; the pixel array comprises a plurality of multiplexed pixel units and a plurality of single pixel units; the multiplexed pixel unit is a pixel unit that multiplexes two elements of intensity, time difference, and spatial difference; the single pixel unit is a pixel unit having an element different from that of the multiplexed pixel unit; the intensity path is used to determine the quantized value of an electrical signal converted from the intensity of incident light at the current pixel unit position at the current moment; the time difference path is used to perform a differential and quantized operation on the signal at the current pixel unit position at the current moment and the signal at the current pixel unit position at the previous moment in the charge, analog, or digital domain to obtain a time differential value; the spatial difference path is used to perform a spatial differential and quantized operation on the signal at the current pixel unit position at the current moment and the signal at the spatially associated pixel unit position at the current moment in the charge, analog, or digital domain to obtain a spatial differential value, and the spatially associated pixel unit is any one or more pixel units in the pixel array except the current pixel unit.

[0010] According to a hybrid array-based visual sensor chip provided by the present application, the multiplexed pixel unit is a pixel unit that multiplexes time differences and spatial differences, and the single pixel unit is an intensity pixel unit; the intensity path corresponds to the intensity pixel unit, and the time difference path and the spatial difference path correspond to the pixel unit that multiplexes time differences and spatial differences.

[0011] According to a hybrid array-based visual sensor chip provided by the present application, the intensity path includes an intensity storage module and an intensity quantization module; the intensity storage module is used to store the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment; the intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment, and obtain the quantized value of the electrical signal of the intensity of the incident light at the current pixel unit position at the current moment; the time difference path includes a time difference storage module and a time difference and quantization module; the time difference storage module is used to store the electrical signal of the current pixel unit position at different moments in a ping-pong cache manner; the time difference storage module includes a first time difference storage node and a second time difference storage node; the ping-pong cache manner is to store the electrical signal of the current pixel unit position at the previous moment in a ping-pong cache manner. When the signal is stored in the first time difference storage node / the second time difference storage node, the electrical signal of the pixel unit position at the current moment is stored in the second time difference storage node / the first time difference storage node; the time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the current pixel unit position at the previous moment to obtain a time difference value; the spatial difference path includes a spatial difference storage node and a spatial difference and quantization module that reuse the first time difference storage node; the spatial difference storage node is used to store the electrical signal of the current pixel unit position at the current moment; the spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the spatially associated pixel unit position at the current moment to obtain a spatial difference value.

[0012] According to a hybrid array-based visual sensor chip provided by the present application, the intensity quantization module is arranged in the intensity pixel unit and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; the time difference and quantization module is arranged in the time difference and space difference multiplexing pixel unit, and adopts a pixel-level signal readout method; or the time difference and quantization module is arranged outside the time difference and space difference multiplexing pixel unit, and is shared by the time difference and space difference multiplexing pixel units in the same column, and adopts a column-level signal readout method; the spatial difference and quantization module is arranged in the time difference and space difference multiplexing pixel unit, and adopts a pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the time difference and space difference multiplexing pixel unit, and is shared by the time difference and space difference multiplexing pixel units in the same column, and adopts a column-level signal readout method.

[0013] According to a hybrid array-based visual sensor chip provided by the present application, a pulse generating module is provided in each pixel unit in the pixel array; or, all pixel units in the pixel array are connected to a pulse generating module in common; or, the pixel array is divided into multiple sub-areas, and all pixel units in each sub-area are connected to a pulse generating module in common; wherein, the pulse generating module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable time interval, so as to control the start exposure moment and exposure duration of the photosensitive module, the pixel units connected to the same pulse generating module are exposed synchronously, and the pixel units connected to different pulse generating modules are exposed synchronously or asynchronously; the photosensitive module is provided in the pixel unit, and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

[0014] According to a hybrid array-based visual sensor chip provided by the present application, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0015] According to a hybrid array-based visual sensor chip provided by the present application, when the pixel unit is provided with a color filter, the output color type of the corresponding path of the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the corresponding path of the pixel unit is a grayscale value.

[0016] The present application also provides a visual sensor chip based on a hybrid array, comprising a pixel array and a corresponding visual sensing path; the visual sensing path comprises an intensity path, a time differential path and a space differential path; the pixel array comprises an intensity pixel unit, a time differential pixel unit and a space differential pixel unit; the intensity path corresponds to the intensity pixel unit, the time differential path corresponds to the time differential pixel unit, and the space differential path corresponds to the space differential pixel unit; the intensity path is used to determine the quantized value of the electrical signal converted from the intensity of the incident light at the current moment at the current intensity pixel unit position; The time differential path is used to perform time differential and quantization operations on the signal of the current time differential pixel unit position at the current moment and the signal of the current time differential pixel unit position at the previous moment in the charge, analog or digital domain to obtain a time differential value; the spatial differential path is used to perform spatial differential and quantization operations on the signal of the current spatial differential pixel unit position at the current moment and the signal of the spatially associated spatial differential pixel unit position at the current moment in the charge, analog or digital domain to obtain a spatial differential value, and the spatially associated pixel unit is any one or more pixel units in the pixel array except the current spatial differential pixel unit.

[0017] According to a hybrid array-based visual sensor chip provided by the present application, the intensity path includes an intensity storage module and an intensity quantization module; the intensity storage module is used to store the electrical signal converted from the intensity of the incident light at the current intensity pixel unit position at the current moment; the intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the intensity of the incident light at the current intensity pixel unit position at the current moment, to obtain a quantized value of the electrical signal of the intensity of the incident light at the current intensity pixel unit position at the current moment; the time difference path includes a time difference storage module and a time difference and quantization module; the time difference storage module is used to store the current time difference pixel units at different moments in a ping-pong cache manner. The electrical signal of the position; the time difference storage module includes a first time difference storage node and a second time difference storage node; the ping-pong cache method is to store the electrical signal of the time difference pixel unit position at the current moment in the second time difference storage node / the first time difference storage node when the electrical signal of the current time difference pixel unit position at the previous moment is stored in the first time difference storage node / the second time difference storage node; the time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current time difference pixel unit position at the current moment and the electrical signal of the current time difference pixel unit position at the previous moment to obtain a time difference value; the spatial difference path includes a spatial difference storage node and a spatial difference and quantization module; the spatial difference storage node is used to store the electrical signal of the current spatial difference pixel unit position at the current moment; the spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current spatial difference pixel unit position at the current moment and the electrical signal of the spatially associated spatial difference pixel unit position at the current moment to obtain a spatial difference value.

[0018] According to a hybrid array-based visual sensor chip provided by the present application, the intensity quantization module is arranged in the intensity pixel unit and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; the time difference and quantization module is arranged in the time difference and space difference multiplexing pixel unit, and adopts a pixel-level signal readout method; or the time difference and quantization module is arranged outside the time difference and space difference multiplexing pixel unit, and is shared by the time difference and space difference multiplexing pixel units in the same column, and adopts a column-level signal readout method; the spatial difference and quantization module is arranged in the time difference and space difference multiplexing pixel unit, and adopts a pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the time difference and space difference multiplexing pixel unit, and is shared by the time difference and space difference multiplexing pixel units in the same column, and adopts a column-level signal readout method.

[0019] According to a hybrid array-based visual sensor chip provided by the present application, a pulse generating module is provided in each pixel unit in the pixel array; or, all pixel units in the pixel array are connected to a pulse generating module in common; or, the pixel array is divided into multiple sub-areas, and all pixel units in each sub-area are connected to a pulse generating module in common; wherein, the pulse generating module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable time interval, so as to control the start exposure moment and exposure duration of the photosensitive module, the pixel units connected to the same pulse generating module are exposed synchronously, and the pixel units connected to different pulse generating modules are exposed synchronously or asynchronously; the photosensitive module is provided in the pixel unit, and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

[0020] According to a hybrid array-based visual sensor chip provided by the present application, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0021] According to a hybrid array-based visual sensor chip provided by the present application, when the pixel unit is provided with a color filter, the output color type of the corresponding path of the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the corresponding path of the pixel unit is a grayscale value.

[0022] The present application provides a hybrid array-based visual sensor chip, comprising a pixel array and corresponding visual sensing pathways; the visual sensing pathways comprise an intensity pathway, a time differential pathway, and a spatial differential pathway; the pixel array comprises a plurality of multiplexed pixel units and a plurality of single pixel units; the multiplexed pixel unit is a pixel unit that multiplexes two elements of intensity, time differential, and spatial differential; the single pixel unit is a pixel unit that has an element different from that of the multiplexed pixel unit; the intensity pathway is used to determine the quantized value after the intensity of incident light is converted into an electrical signal; the time differential pathway is used to determine the time differential value; and the time differential pathway is used to determine the spatial differential value. Through a three-pathway visual sensor chip architecture based on a hybrid array, the present application can significantly enhance the visual sensor chip's ability to perceive spatiotemporal dynamic information, achieving high-precision, high-frame-rate, high-dynamic-range, and efficient and robust visual representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] FIG1 is a schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by the present application;

[0025] FIG2a is a schematic diagram showing the principle of reading out the quantized value of the electrical signal converted from the intensity of incident light of the intensity pixel unit provided by the present application;

[0026] FIG2 b is a schematic diagram of a temporal differential value readout principle of a spatiotemporal differential pixel unit provided by the present application;

[0027] FIG2c is a schematic diagram of a spatial differential value readout principle of a spatiotemporal differential pixel unit provided by the present application;

[0028] FIG2 d is a second schematic diagram of the spatial difference value readout principle of the spatiotemporal difference pixel unit provided by the present application;

[0029] FIG3 is a schematic diagram of a specific principle of reading out the time difference value of the spatiotemporal differential pixel unit provided by the present application;

[0030] FIG4 is a schematic diagram of a specific principle of reading out spatial difference values ​​of a spatiotemporal difference pixel unit provided by the present application;

[0031] FIG5 is a second schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by the present application;

[0032] FIG6 is a third schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by the present application;

[0033] FIG7a is a third schematic diagram of the principle of reading out the temporal and spatial differential values ​​of the spatiotemporal differential pixel unit provided by the present application;

[0034] FIG7 b is a fourth schematic diagram of the principle of reading out the temporal and spatial differential values ​​of the spatiotemporal differential pixel unit provided by the present application;

[0035] FIG8 is a fourth schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by the present application;

[0036] FIG9 is a fifth schematic diagram of the architecture of a hybrid array-based visual sensor chip provided in this application. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0038] Currently, CMOS image sensors (CIS) primarily capture video using a frame-based sampling principle. This means that each CIS image frame records the output of all pixels in the pixel array, and each frame is equally spaced. CIS integrates transistors within the pixels to achieve high-performance charge-to-voltage conversion. Through a color filter array overlying the pixel array, CIS can sense visible light of different wavelengths to produce color images. CIS offers the advantages of high pixel resolution, excellent color reproduction, and high image quality. Dynamic Vision Sensors (DVS) are a new type of imaging system. Unlike traditional cameras, which use a shutter to control the frame rate and record light intensity on a per-frame basis, DVS is sensitive to the rate of change of light intensity. Each pixel independently records the change in the logarithm of the light intensity at that pixel, generating a positive or negative pulse when the change exceeds a threshold. The DVS's asynchronous pulse generation makes it unconstrained by the shutter and possesses extremely high temporal resolution. Combined with its sensitivity to change, it is naturally suited for tasks such as motion detection. Another camera called DAVIS combines traditional CIS with DVS, which can record both single-frame images and event information. It has the advantages of high spatial resolution of traditional cameras and high temporal resolution of DVS cameras.

[0039] From the perspective of visual primitives, vision sensors with only CIS and DVS channels still provide incomplete information collection. For example, when there is a large flash or a dramatic change in light intensity in the image, all time-differential pixels will output events, leading to saturation. The DVS channel will be unable to output valid information, and the CIS channel will be unable to respond immediately due to frame rate limitations. Such extreme scenarios are very common in autonomous driving and are crucial to driving safety, such as when entering and exiting tunnels and capturing camera flashes at night.

[0040] Please refer to FIG1 , which is one of the schematic diagrams of the architecture of a hybrid array-based visual sensor chip provided in this application.

[0041] Please refer to FIG. 2 a , which is one of the schematic diagrams of the principle of reading out the quantized value of the electrical signal converted from the intensity of the incident light of the intensity pixel unit provided by the present application.

[0042] Please refer to FIG. 2 b , which is one of the schematic diagrams of the time difference value readout principle of the spatiotemporal difference pixel unit provided in this application.

[0043] Please refer to FIG. 2 c , which is one of the schematic diagrams of the spatial differential value readout principle of the spatiotemporal differential pixel unit provided in this application.

[0044] Please refer to FIG. 2 d , which is a second schematic diagram of the spatial differential value readout principle of the spatiotemporal differential pixel unit provided in this application.

[0045] In order to solve the technical problems existing in the prior art, the present application provides a visual sensor chip based on a hybrid array, including a pixel array and a corresponding visual sensing path; the visual sensing path includes an intensity path, a time difference path and a spatial difference path; the pixel array includes multiple multiplexed pixel units and multiple single pixel units; the multiplexed pixel unit is a pixel unit that multiplexes two elements of intensity, time difference and spatial difference; the single pixel unit is a pixel unit with an element different from that of the multiplexed pixel unit; the intensity path is used to determine the quantized value of the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment; the time difference path is used to perform differential and quantization operations on the signal at the current pixel unit position at the current moment and the signal at the current pixel unit position at the previous moment in the charge, analog or digital domain to obtain a time differential value; the spatial difference path is used to perform spatial differential and quantization operations on the signal at the current pixel unit position at the current moment and the signal at the spatially associated pixel unit position at the current moment in the charge, analog or digital domain to obtain a spatial differential value, and the spatially associated pixel unit is any one or more pixel units in the pixel array except the current pixel unit.

[0046] In contrast, the human visual system can achieve rapid recognition of moving targets, whether at noon or dusk, in an open scene or partially blocked. It achieves robustness and versatility far exceeding that of existing DAVIS or hybrid array systems. This is because in addition to the output intensity path and the time difference path, the human eye also has a spatial difference path. The three are organically integrated and combined into different primitives to form an efficient and robust visual representation. Inspired by human vision, this application adds a spatial difference path that simulates the human retina to the existing single pixel multiplexing or hybrid pixel array solution. That is, the visual sensor has three outputs at the same time: intensity output, time difference (TD, Temporal Difference) output and spatial difference (SD, Spatial Difference) output.

[0047] The output of the intensity path is the current time t n The incident light intensity I(x,y,t n ), that is,

[0048] A(x,y,t n )=Q A (I(x,y,t n ))

[0049] where Q A A quantification method for the intensity pathway.

[0050] The time difference path outputs the time difference value of the current pixel unit position (x, y) at different times. The expression of the obtained time difference path output is

[0051] TD(x,y,t n )=Q TD (I(x,y,t n )-I(x,y,t n-1 ))

[0052] where Q TD A quantification method for the intensity pathway.

[0053] The spatial difference path outputs the current time t n The spatial difference between the current pixel unit position (x, y) and the spatially associated pixel unit position (eg, diagonal or xy direction).

[0054] SD i (x,y,t n )=Q SD (I(x,y,t n )-I(x i ,y i ,t n ))

[0055] where Q SD is the quantification method of the intensity channel, where SD i Indicates the current pixel unit and the associated pixel unit (x i ,y i ) between the spatial differences.

[0056] All signals involved in the three visual sensing pathways are three-dimensional quantities, including the spatial two-dimensional quantities x and y and the time dimension t.

[0057] The pixel units of the pixel array can have the following forms:

[0058] The pixel array is a binary hybrid pixel array composed of multiple multiplexed pixel units and multiple single pixel units;

[0059] Among them, the multiplexed pixel unit is a pixel unit that multiplexes intensity and time differences; the single pixel unit is a spatial difference pixel unit;

[0060] Alternatively, the multiplexed pixel unit is an intensity and spatial difference multiplexed pixel unit; and the single pixel unit is a time difference pixel unit;

[0061] Alternatively, the multiplexed pixel unit is a pixel unit multiplexed with time difference and space difference; and the single pixel unit is an intensity pixel unit.

[0062] Alternatively, the pixel array is a ternary hybrid pixel array composed of a plurality of intensity pixel units, a plurality of time difference pixel units, and a plurality of space difference pixel units.

[0063] The visual sensor chip provided in this application has a three-channel visual sensor chip architecture with intensity output, time differential output and spatial differential output, which can greatly enhance the visual sensor chip's perception ability of spatiotemporal dynamic information and achieve high-precision, high frame rate, high dynamic range and efficient and robust visual representation.

[0064] Based on the above embodiment:

[0065] As a preferred embodiment, the multiplexed pixel unit is a pixel unit multiplexed with time difference and space difference, and the single pixel unit is an intensity pixel unit; the intensity path corresponds to the intensity pixel unit, and the time difference path and the space difference path correspond to the pixel unit multiplexed with time difference and space difference.

[0066] This embodiment provides a three-channel visual sensor with hybrid pixels, in which the pixel array includes intensity pixel units and pixel units multiplexed by time difference and space difference (time-space difference pixel units). The quantized value of the incident light intensity of the intensity pixel unit is read out through the intensity channel, the time difference value of the time-space difference pixel unit is read out through the time difference channel, and the spatial difference value is read out through the spatial difference channel.

[0067] As a preferred embodiment, the intensity path includes an intensity storage module and an intensity quantization module; the intensity storage module is used to store the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment; the intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment, and obtain the quantized value of the electrical signal of the intensity of the incident light at the current pixel unit position at the current moment; the time difference path includes a time difference storage module and a time difference and quantization module; the time difference storage module is used to store the electrical signal of the current pixel unit position at different moments in a ping-pong cache manner; the time difference storage module includes a first time difference storage node and a second time difference storage node; the ping-pong cache manner is to store the electrical signal of the current pixel unit position at the previous moment in the first time difference storage node. In the case of a storage node for time difference / a second storage node for time difference, the electrical signal of the pixel unit position at the current moment is stored in the second storage node for time difference / the first storage node for time difference; the time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the current pixel unit position at the previous moment to obtain a time difference value; the spatial difference path includes a spatial difference storage node and a spatial difference and quantization module that reuse the first storage node for time difference; the spatial difference storage node is used to store the electrical signal of the current pixel unit position at the current moment; the spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the spatially associated pixel unit position at the current moment to obtain a spatial difference value.

[0068] It should be noted that the first time difference storage node and the second time difference storage node can use high-speed, low-precision storage nodes, and the intensity storage module can use low-speed, high-precision storage nodes, so as to achieve complementary advantages between the two storage nodes. With the help of post-processing, it is expected to achieve high speed and high precision of overall performance with smaller generations. The first time difference storage node and the second time difference storage node of the same pixel use a ping-pong alternating method to store information at different times. Ping-pong cache means that if the electrical signal of the pixel unit is currently stored in the first time difference storage node, it will be stored in the second time difference storage node at the next moment, and then in the first time difference storage node at the next moment, and so on, and so on, and output two time signals (I(x, y, t n ), I(x,y,t n-1 )).

[0069] The intensity quantization module can be set outside the intensity pixel unit or inside the intensity pixel unit; the time difference and quantization module and the space difference and quantization module can be set outside the time-space difference pixel unit or inside the time-space difference pixel unit.

[0070] In this embodiment, the intensity quantization module is arranged outside the intensity pixel unit, and the spatiotemporal difference and quantization module and the spatial difference and quantization module are arranged outside the spatiotemporal difference pixel unit as an example.

[0071] Please refer to FIG3 , which is a schematic diagram of the time difference value readout principle of the spatiotemporal difference pixel unit provided in this application.

[0072] When reading out the temporal differential values, considering the number of temporal differential and quantization modules, multiple spatiotemporal differential pixel units share a single temporal differential and quantization module, and thus a single moment can be divided into multiple sub-moments t1, t2, and t3. At sub-moment t1, the temporal differential values ​​of the spatiotemporal differential pixel units in the first region can be read out; at sub-moment t2, the temporal differential values ​​of the spatiotemporal differential pixel units in the second region can be read out; at sub-moment t3, the temporal differential values ​​of the spatiotemporal differential pixel units in the third region can be read out; and so on. The temporal differential values ​​of all spatiotemporal differential pixel units in the pixel array can be read out. The square with a bold border in the figure indicates the pixel being read out at the current sub-moment.

[0073] Please refer to FIG4 , which is a schematic diagram of the spatial differential value readout principle of the spatiotemporal differential pixel unit provided in this application.

[0074] When reading out spatial differential values, considering the number of spatial differential and quantization modules, multiple spatiotemporal differential pixel units share one spatial differential and quantization module, and one moment can be divided into multiple sub-moments t1, t2, t3, and t4. Sub-moment t1 can read the spatial differential values ​​of the spatiotemporal differential pixel units in the first region, sub-moment t2 can read the spatial differential values ​​of the spatiotemporal differential pixel units in the second region, sub-moment t3 can read the spatial differential values ​​of the spatiotemporal differential pixel units in the third region, and sub-moment t4 can read the spatial differential values ​​of the spatiotemporal differential pixel units in the fourth region. And so on, the spatial differential values ​​of all spatiotemporal differential pixel units in the pixel array can be read out. The square with a bold border in the figure indicates the pixel being read out at the current sub-moment.

[0075] There are many ways to select the pixel units at the spatially associated positions. Typically, they can be pixel units at positions adjacent to the pixel unit at the current position in the xy direction, or pixel units at positions adjacent to the pixel unit at the current position in the oblique direction.

[0076] For xy direction difference, the expression of the spatial difference path output is obtained as follows:

[0077] SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y,t n ))

[0078] SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y-1,t n ))

[0079] For oblique difference, the expression of the spatial difference path output is obtained as

[0080] SD ↙ (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y-1,t n ))

[0081] SD ↘ (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y-1,t n ))

[0082] In addition, other forms may be selected, such as only selecting adjacent pixels in a certain direction for differentiation to obtain differential information in that direction, or selecting more than two associated pixels at the same time to improve the accuracy of spatial differentiation.

[0083] The quantization methods of the temporal and spatial differential quantization modules can be either multi-valued (>1 bit) or single-valued (positive and negative pulses). Signal acquisition can be performed synchronously across the entire array with the same time interval, synchronously across the entire array with variable time intervals, or asynchronously across the entire array.

[0084] In the field of digital signal processing, quantization mainly refers to the process of converting analog signals into digital signals. Signal sampling and quantization are usually achieved by analog-to-digital converters (ADCs).

[0085] As a preferred embodiment, the intensity quantization module is arranged in the intensity pixel unit, and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; the time difference and quantization module is arranged in the pixel unit of time difference and space difference multiplexing, and adopts a pixel-level signal readout method; or the time difference and quantization module is arranged outside the pixel unit of time difference and space difference multiplexing, and is shared by the pixel units of time difference and space difference multiplexing in the same column, and adopts a column-level signal readout method; the spatial difference and quantization module is arranged in the pixel unit of time difference and space difference multiplexing, and adopts a pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the pixel unit of time difference and space difference multiplexing, and is shared by the pixel units of time difference and space difference multiplexing in the same column, and adopts a column-level signal readout method.

[0086] In this embodiment, if the temporal difference and quantization module and the spatial difference and quantization module are set outside the spatiotemporal difference pixel unit, the total number of quantization modules is reduced, and hardware resource consumption is reduced. If the temporal difference and quantization module and the spatial difference and quantization module are set within the spatiotemporal difference pixel unit, flexibility is improved and output delay is reduced.

[0087] The present application relates to a hybrid array composed of two different pixels or a hybrid array composed of three different pixels. The arrangement method of the hybrid array is not unique. For example, in addition to FIG. 1 , there is also the arrangement method of FIG. 5 .

[0088] Please refer to FIG5 , which is a second schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by this application.

[0089] Please refer to FIG6 , which is a third schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by this application.

[0090] Please refer to FIG. 7 a , which is a third schematic diagram of the principle of reading out the time difference value and the space difference value of the time-space difference pixel unit provided by the present application.

[0091] Please refer to FIG. 7 b , which is a fourth schematic diagram of the principle of reading out the time difference value and the space difference value of the time-space difference pixel unit provided in the present application.

[0092] The intensity module of the visual sensor chip is arranged outside the intensity pixel unit, and the time difference and quantization module and the space difference and quantization module are both within the time-space difference image unit element.

[0093] Of course, the intensity module of the visual sensor chip of the present application is set outside the intensity pixel unit and the time difference and quantization module, the spatial difference and quantization module are set inside and outside the time and space difference image single element. The situation can be combined arbitrarily, and the present application does not make any special restrictions here.

[0094] As a preferred embodiment, a pulse generating module is provided in each pixel unit in the pixel array;

[0095] Alternatively, all pixel units in the pixel array are connected to a pulse generating module;

[0096] Alternatively, the pixel array is divided into a plurality of sub-regions, and all pixel units in each sub-region are connected to a pulse generating module;

[0097] The pulse generation module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable time interval, to control the start exposure time and exposure duration of the photosensitive module, and the pixel units connected to the same pulse generation module are exposed synchronously, and the pixel units connected to different pulse generation modules are exposed synchronously or asynchronously;

[0098] The photosensitive module is arranged in the pixel unit and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

[0099] In this embodiment, the visual sensor chip further includes a trigger pulse generator, which can generate a trigger signal to realize the exposure control of the photosensitive module, that is, determine the time t of collecting the signal. n . If a trigger pulse generator is designed in each pixel unit, full-array asynchronous exposure can be used. At this time, each trigger pulse generator can independently and adaptively adjust the moment of triggering the calculation of the spatiotemporal differential signal according to the light intensity level perceived by the pixel unit itself. The trigger moment of each pixel unit is different. The pixel unit can output information at any time, which improves flexibility and reduces output delay. Of course, in this case, it can also be set to full-array synchronous exposure as needed. If some pixel units share the same trigger pulse generator, these pixel units are exposed synchronously.

[0100] The trigger pulse generator can generate trigger signals at uniform time intervals or at adaptive, programmable variable intervals.

[0101] If all pixel units in the array share a trigger pulse generator, then all pixel units need to be exposed at the same time, and the output of the pixel units needs to follow a certain rule.

[0102] As a preferred embodiment, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0103] Of course, the intensity path, the time difference path and the space difference path can be arbitrarily combined in a global exposure or rolling exposure manner, and this application does not impose any particular limitation thereto.

[0104] As a preferred embodiment, when the pixel unit is provided with a color filter, the output color type of the path corresponding to the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the path corresponding to the pixel unit is a grayscale value.

[0105] If a pixel unit is covered with a color filter, the information obtained by the pixel is only that of a specific color channel. A typical color filter is a combination of red, green, and blue, known as the RGB type. Other color channels can also be used, such as a CMY array consisting of the three complementary colors (cyan, magenta, and yellow). Pixels of the same type may differ in color channel, such as a spatiotemporal differential pixel for the X color, a spatiotemporal differential pixel for the Y channel, and a spatiotemporal differential pixel for the Z channel. Spatial differentiation can be performed between pixels of the same color or between pixels of different colors (e.g., subtracting an X pixel from a Y pixel).

[0106] In addition, an external programmable demosaicer may be embedded in the pixel unit to obtain the output values ​​of all other color channels at the pixel position of the X color channel through a demosaicing algorithm, that is, by selecting points of surrounding pixels and performing interpolation.

[0107] Please refer to FIG8 , which is a fourth schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by this application.

[0108] Please refer to FIG9 , which is a fifth schematic diagram of the architecture of a hybrid array-based visual sensor chip provided by this application.

[0109] The present application also provides a visual sensor chip based on a hybrid array, a pixel array and a corresponding visual sensing path; the visual sensing path includes an intensity path, a time differential path and a spatial differential path; the pixel array includes an intensity pixel unit, a time differential pixel unit and a spatial differential pixel unit; the intensity path corresponds to the intensity pixel unit, the time differential path corresponds to the time differential pixel unit, and the spatial differential path corresponds to the spatial differential pixel unit; the intensity path is used to determine the quantized value of the electrical signal converted from the intensity of the incident light at the current intensity pixel unit position at the current moment; the time differential path is used to perform time differential and quantization operations on the signal at the current time differential pixel unit position at the current moment and the signal at the current time differential pixel unit position at the previous moment in the charge, analog or digital domain to obtain a time differential value; the spatial differential path is used to perform spatial differential and quantization operations on the signal at the current spatial differential pixel unit position at the current moment and the signal at the spatially associated spatial differential pixel unit position at the current moment in the charge, analog or digital domain to obtain a spatial differential value, and the spatially associated pixel unit is any one or more pixel units in the pixel array except the current spatial differential pixel unit.

[0110] The main difference between the visual sensor chip of this embodiment and the above embodiment is that the pixel array of the visual sensor is divided into three different types of pixel units, namely intensity pixel units, time difference pixel units, and spatial difference pixel units. The arrangement of these three types of pixel units in the pixel array can be arbitrarily combined, and corresponding visual sensing channels can be set for different arrangements to achieve quantization and readout of the quantized value of the incident light intensity of the intensity pixel unit, the time difference value of the time difference pixel unit, and the spatial difference value of the spatial difference pixel unit in a high-precision multi-value format (≥1 bit). Through the three-channel visual sensor chip architecture, this application can greatly improve the visual sensor chip's perception of spatiotemporal dynamic information, and achieve high-precision, high frame rate, high dynamic range, and efficient and robust visual representation.

[0111] As a preferred embodiment, the intensity path includes an intensity storage module and an intensity quantization module; the intensity storage module is used to store the electrical signal converted from the incident light intensity at the current intensity pixel unit position at the current moment; the intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the incident light intensity at the current intensity pixel unit position at the current moment, and obtain the quantized value of the electrical signal of the incident light intensity at the current intensity pixel unit position at the current moment; the time difference path includes a time difference storage module and a time difference and quantization module; the time difference storage module is used to store the electrical signals at the current time difference pixel unit position at different moments in a ping-pong cache manner; the time difference storage module includes a first time difference storage node and a second time difference storage node; the ping-pong cache manner is to store the electrical signal at the current time difference pixel unit position at the previous moment In the case of the first time difference storage node / the second time difference storage node, the electrical signal of the time difference pixel unit position at the current moment is stored in the second time difference storage node / the first time difference storage node; the time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current time difference pixel unit position at the current moment and the electrical signal of the current time difference pixel unit position at the previous moment to obtain a time difference value; the spatial difference path includes a spatial difference storage node and a spatial difference and quantization module; the spatial difference storage node is used to store the electrical signal of the current spatial difference pixel unit position at the current moment; the spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current spatial difference pixel unit position at the current moment and the electrical signal of the spatially associated spatial difference pixel unit position at the current moment to obtain a spatial difference value.

[0112] It should be noted that the first time difference storage node and the second time difference storage node can use high-speed low-precision storage nodes, and the intensity storage module can use low-speed high-precision storage nodes. The first time difference storage node and the second time difference storage node of the same pixel use a ping-pong alternating method to store information at different times. Ping-pong cache means that if the electrical signal of the pixel unit is currently stored in the first time difference storage node, then it will be stored in the second time difference storage node at the next moment, and then in the first time difference storage node at the next moment, and so on, and so on, and outputs two time signals (I(x, y, t n ), I(x,y,t n-1 )).

[0113] The intensity quantization module can be set outside the intensity pixel unit or inside the intensity pixel unit; the time difference and quantization module and the space difference and quantization module can be set outside the time-space difference pixel unit or inside the time-space difference pixel unit.

[0114] This embodiment takes the example of an intensity quantization module being arranged outside the intensity pixel unit, a spatiotemporal difference and quantization module being arranged outside the temporal difference pixel unit, and a spatial difference and quantization module being arranged outside the spatial difference pixel unit.

[0115] There are many ways to select the pixel units at the spatially associated positions. Typically, they can be pixel units at positions adjacent to the pixel unit at the current position in the xy direction, or pixel units at positions adjacent to the pixel unit at the current position in the oblique direction.

[0116] For xy direction difference, the expression of the spatial difference path output is obtained as follows:

[0117] SD x (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y,t n ))

[0118] SD y (x,y,t n )=Q SD (I(x,y,t n )-I(x,y-1,t n ))

[0119] For oblique difference, the expression of the spatial difference path output is obtained as

[0120] SD ↙ (x,y,t n )=Q SD (I(x,y,t n )-I(x-1,y-1,t n ))

[0121] SD ↘ (x,y,t n )=Q SD (I(x,y,t n )-I(x+1,y-1,t n ))

[0122] In addition, other forms may be selected, such as only selecting adjacent pixels in a certain direction for differentiation to obtain differential information in that direction, or selecting more than two associated pixels at the same time to improve the accuracy of spatial differentiation.

[0123] The quantization methods of the temporal and spatial differential quantization modules can be either multi-valued (>1 bit) or single-valued (positive and negative pulses). Signal acquisition can be performed synchronously across the entire array with the same time interval, synchronously across the entire array with variable time intervals, or asynchronously across the entire array.

[0124] In the field of digital signal processing, quantization mainly refers to the process of converting analog signals into digital signals. Signal sampling and quantization are usually achieved by analog-to-digital converters (ADCs).

[0125] As a preferred embodiment, the intensity quantization module is arranged in the intensity pixel unit, and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; the time difference and quantization module is arranged in the pixel unit of time difference and space difference multiplexing, and adopts a pixel-level signal readout method; or the time difference and quantization module is arranged outside the pixel unit of time difference and space difference multiplexing, and is shared by the pixel units of time difference and space difference multiplexing in the same column, and adopts a column-level signal readout method; the spatial difference and quantization module is arranged in the pixel unit of time difference and space difference multiplexing, and adopts a pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the pixel unit of time difference and space difference multiplexing, and is shared by the pixel units of time difference and space difference multiplexing in the same column, and adopts a column-level signal readout method.

[0126] In this embodiment, if the temporal difference and quantization module is disposed outside the temporal difference pixel unit and the spatial difference and quantization module is disposed outside the spatial difference pixel unit, the total number of quantization modules is reduced and the hardware resource consumption is reduced.

[0127] If the temporal difference and quantization module is arranged in the temporal difference pixel unit and the spatial difference and quantization module is arranged in the spatial difference pixel unit, the flexibility is improved and the output delay is reduced.

[0128] Of course, the situations in which the intensity module of the visual sensor chip of the present application is set outside the intensity pixel unit, the time difference and quantization module is set inside and outside the time difference pixel, and the spatial difference and quantization module is set inside and outside the spatial difference image element can be combined arbitrarily, and the present application does not make any special restrictions here.

[0129] As a preferred embodiment, a pulse generating module is provided in each pixel unit in the pixel array;

[0130] Alternatively, all pixel units in the pixel array are connected to a pulse generating module;

[0131] Alternatively, the pixel array is divided into a plurality of sub-regions, and all pixel units in each sub-region are connected to a pulse generating module;

[0132] The pulse generation module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable variable time interval, to control the start exposure time and exposure duration of the photosensitive module, and the pixel units connected to the same pulse generation module are exposed synchronously, and the pixel units connected to different pulse generation modules are exposed synchronously or asynchronously;

[0133] The photosensitive module is arranged in the pixel unit and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

[0134] In this embodiment, the visual sensor chip further includes a trigger pulse generator, which can generate a trigger signal to realize the exposure control of the photosensitive module, that is, determine the time t of collecting the signal. n . If a trigger pulse generator is designed in each pixel unit, full-array asynchronous exposure can be used. At this time, each trigger pulse generator can independently and adaptively adjust the moment of triggering the calculation of the spatiotemporal differential signal according to the light intensity level perceived by the pixel unit itself. The trigger moment of each pixel unit is different. The pixel unit can output information at any time, which improves flexibility and reduces output delay. Of course, in this case, it can also be set to full-array synchronous exposure as needed. If some pixel units share the same trigger pulse generator, these pixel units are exposed synchronously.

[0135] The trigger pulse generator can generate trigger signals at uniform time intervals or at adaptive, programmable variable intervals.

[0136] If all pixel units in the array share a trigger pulse generator, then all pixel units need to be exposed at the same time, and the output of the pixel units needs to follow a certain rule.

[0137] As a preferred embodiment, the exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

[0138] Of course, the intensity path, the time difference path and the space difference path can be arbitrarily combined in a global exposure or rolling exposure manner, and this application does not impose any particular limitation thereto.

[0139] As a preferred embodiment, when the pixel unit is provided with a color filter, the output color type of the path corresponding to the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the path corresponding to the pixel unit is a grayscale value.

[0140] If a pixel unit is covered with a color filter, the information obtained by the pixel is only that of a specific color channel. A typical color filter is a combination of red, green, and blue, known as the RGB type. Other color channels can also be used, such as a CMY array consisting of the three complementary colors (cyan, magenta, and yellow). Pixels of the same type may differ in color channel, such as a spatiotemporal differential pixel for the X color, a spatiotemporal differential pixel for the Y channel, and a spatiotemporal differential pixel for the Z channel. Spatial differentiation can be performed between pixels of the same color or between pixels of different colors (e.g., subtracting an X pixel from a Y pixel).

[0141] In addition, an external programmable demosaicer may be embedded in the pixel unit to obtain the output values ​​of all other color channels at the pixel position of the X color channel through a demosaicing algorithm, that is, by selecting points of surrounding pixels and performing interpolation.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual sensor chip based on a hybrid array, comprising a pixel array and a correspondingly arranged visual sensing path; the visual sensing path comprises an intensity path, a time difference path and a space difference path; the pixel array comprises a plurality of multiplexed pixel units and a plurality of single pixel units; the multiplexed pixel unit is a pixel unit that multiplexes two elements of intensity, time difference and space difference; the single pixel unit is a pixel unit that has an element different from that of the multiplexed pixel unit; the intensity path is used to determine the quantized value of an electrical signal converted from the light intensity of the incident light at the current pixel unit position at the current moment; The time difference path is used to perform a difference and quantization operation on the signal of the current pixel unit position at the current moment and the signal of the current pixel unit position at the previous moment in the charge, analog or digital domain to obtain a time difference value; The spatial differential path is used to perform spatial differential and quantization operations on the signal of the current pixel unit position at the current moment and the signal of the spatially associated pixel unit position at the current moment in the charge, analog or digital domain to obtain a spatial differential value. The spatially associated pixel unit is any one or more pixel units in the pixel array except the current pixel unit.

2. The hybrid array-based visual sensor chip according to claim 1, wherein: The multiplexed pixel unit is a pixel unit multiplexed with time difference and space difference, and the single pixel unit is an intensity pixel unit; the intensity path corresponds to the intensity pixel unit, and the time difference path and the space difference path correspond to the pixel unit multiplexed with time difference and space difference.

3. The hybrid array-based visual sensor chip according to claim 2, wherein: The intensity path includes an intensity storage module and an intensity quantization module; The intensity storage module is used to store the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment; The intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the intensity of the incident light at the current pixel unit position at the current moment, to obtain a quantized value of the electrical signal of the intensity of the incident light at the current pixel unit position at the current moment; The time difference path includes a time difference storage module and a time difference and quantization module; The time difference storage module is used to store the current time at different times in a ping-pong buffer mode. The electrical signal of the pixel unit position; the time difference storage module comprises a first time difference storage node and a second time difference storage node; the ping-pong cache method is that when the electrical signal of the current pixel unit position at the previous moment is stored in the first time difference storage node / the second time difference storage node, the electrical signal of the pixel unit position at the current moment is stored in the second time difference storage node / the first time difference storage node; The time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the current pixel unit position at the previous moment to obtain a time difference value; The spatial difference path includes a spatial difference storage node and a spatial difference and quantization module that reuse the first time difference storage node; The spatial difference storage node is used to store the electrical signal of the current pixel unit position at the current moment; The spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current pixel unit position at the current moment and the electrical signal of the spatially associated pixel unit position at the current moment to obtain a spatial difference value.

4. The hybrid array-based visual sensor chip according to claim 3, wherein: The intensity quantization module is arranged in the intensity pixel unit, and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; The time difference and quantization module is arranged in the pixel unit of the time difference and space difference multiplexing, and adopts the pixel level signal reading method; or the time difference and quantization module is arranged outside the pixel unit of the time difference and space difference multiplexing, and is shared by the pixel units of the time difference and space difference multiplexing in the same column, and adopts the column level signal reading method; The spatial difference and quantization module is arranged in the pixel unit of the time difference and spatial difference multiplexing, and adopts the pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the pixel unit of the time difference and spatial difference multiplexing, and is shared by the pixel units of the time difference and spatial difference multiplexing in the same column, and adopts the column-level signal readout method.

5. The hybrid array-based visual sensor chip according to claim 1, wherein: A pulse generating module is provided in each pixel unit in the pixel array; Or, all pixel units in the pixel array are connected to a pulse generating module; Or, the pixel array is divided into a plurality of sub-areas, and all pixel units in each sub-area are connected to a pulse generating module; The pulse generating module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable and variable time interval, so as to control the exposure start time and exposure duration of the photosensitive module, and to synchronously expose the pixel units connected to the same pulse generating module, and synchronously expose or asynchronously expose the pixel units connected to different pulse generating modules; The photosensitive module is disposed in the pixel unit and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

6. The hybrid array-based visual sensor chip according to claim 1, wherein: The exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

7. The hybrid array-based visual sensor chip according to any one of claims 1 to 6, wherein: When the pixel unit is provided with a color filter, the output color type of the path corresponding to the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the path corresponding to the pixel unit is a gray value.

8. A hybrid array-based visual sensor chip, comprising a pixel array and a correspondingly arranged visual sensing pathway; the visual sensing pathway comprises an intensity pathway, a time difference pathway and a space difference pathway; The pixel array includes an intensity pixel unit, a time difference pixel unit and a space difference pixel unit; the intensity path corresponds to the intensity pixel unit, the time difference path corresponds to the time difference pixel unit, and the space difference path corresponds to the space difference pixel unit; The intensity path is used to determine the quantized value of the electrical signal converted from the light intensity of the incident light at the current intensity pixel unit position at the current moment; The time difference path is used to perform time difference and quantization operations on the signal of the current time difference pixel unit position at the current moment and the signal of the current time difference pixel unit position at the previous moment in the charge, analog or digital domain to obtain a time difference value; The spatial difference path is used to perform spatial difference and quantization operations on the signal of the current spatial difference pixel unit position at the current moment and the signal of the spatially associated spatial difference pixel unit position at the current moment in the charge, analog or digital domain to obtain a spatial difference value. The spatially associated pixel unit is any one or more of the pixel arrays except the current spatial difference pixel unit. Pixel unit.

9. The hybrid array-based visual sensor chip according to claim 8, wherein: The intensity path includes an intensity storage module and an intensity quantization module; The intensity storage module is used to store the electrical signal converted from the intensity of the incident light at the current intensity pixel unit position at the current moment; The intensity quantization module is used to perform analog-to-digital conversion on the electrical signal converted from the incident light intensity at the current intensity pixel unit position at the current moment, to obtain a quantized value of the electrical signal of the incident light intensity at the current intensity pixel unit position at the current moment; The time difference path includes a time difference storage module and a time difference and quantization module; The time difference storage module is used to store the electrical signals of the current time difference pixel unit position at different moments in a ping-pong cache manner; the time difference storage module comprises a first time difference storage node and a second time difference storage node; the ping-pong cache manner is to store the electrical signals of the time difference pixel unit position at the current moment in the second time difference storage node / the first time difference storage node when the electrical signals of the current time difference pixel unit position at the previous moment are stored in the first time difference storage node / the second time difference storage node; The time difference and quantization module is used to perform time difference and quantization operations on the electrical signal of the current time difference pixel unit position at the current moment and the electrical signal of the current time difference pixel unit position at the previous moment to obtain a time difference value; The spatial difference path includes a storage node for spatial difference and a spatial difference and quantization module; The spatial difference storage node is used to store the electrical signal of the current spatial difference pixel unit position at the current moment; The spatial difference and quantization module is used to perform difference and quantization operations on the electrical signal of the current spatial difference pixel unit position at the current moment and the electrical signal of the spatially associated spatial difference pixel unit position at the current moment to obtain a spatial difference value.

10. The hybrid array-based visual sensor chip according to claim 9, wherein: The intensity quantization module is arranged in the intensity pixel unit, and adopts a pixel-level signal readout method; or the intensity quantization module is arranged outside the intensity pixel unit and is shared by the intensity pixel units in the same column, and adopts a column-level signal readout method; The time difference and quantization module is arranged in the pixel unit of the time difference and space difference multiplexing, and adopts the pixel level signal reading method; or the time difference and quantization module is arranged outside the pixel unit of the time difference and space difference multiplexing, and is shared by the pixel units of the time difference and space difference multiplexing in the same column, and adopts the column level signal reading method; The spatial difference and quantization module is arranged in the pixel unit of the time difference and spatial difference multiplexing, and adopts the pixel-level signal readout method; or the spatial difference and quantization module is arranged outside the pixel unit of the time difference and spatial difference multiplexing, and is shared by the pixel units of the time difference and spatial difference multiplexing in the same column, and adopts the column-level signal readout method.

11. The hybrid array-based visual sensor chip according to claim 8, wherein: A pulse generating module is provided in each pixel unit in the pixel array; Or, all pixel units in the pixel array are connected to a pulse generating module; Or, the pixel array is divided into a plurality of sub-areas, and all pixel units in each sub-area are connected to a pulse generating module; The pulse generating module is used to generate a trigger signal at a fixed time interval, or to generate a trigger signal at an adaptive, programmable and variable time interval, so as to control the exposure start time and exposure duration of the photosensitive module, and to synchronously expose the pixel units connected to the same pulse generating module, and synchronously expose or asynchronously expose the pixel units connected to different pulse generating modules; The photosensitive module is disposed in the pixel unit and is used to convert the light signal at the current pixel unit position into an analog electrical signal.

12. The hybrid array-based visual sensor chip according to claim 8, wherein: The exposure mode of each pixel unit in the pixel array is global exposure or rolling exposure.

13. The hybrid array-based visual sensor chip according to any one of claims 8 to 12, wherein: When the pixel unit is provided with a color filter, the output color type of the path corresponding to the pixel unit is a color value; when the pixel unit is not provided with a color filter, the output color type of the path corresponding to the pixel unit is a gray value.

Images