High-speed optical flow sensor chip of variable pixel array

Abstract

The invention discloses a high-speed optical flow sensor chip of a variable pixel array, and an image sensor can work in different working modes including a full-image mode, an undersampling mode and a pixel merging mode, and can adapt to different motion scenes including high speed, high resolution and low illumination. The method can be widely applied to multiple emerging consumption fields including displacement detection in optical mice, sweeping robots, printers and scanners, crown rotation detection in smart watches and image-based positioning of unmanned aerial vehicles.

Description

Technical Field

[0001] The present invention relates to a high-speed optical flow sensor chip with a variable pixel array, which can adapt to different motion scenes including high speed, high resolution and low illumination, and belongs to the field of photoelectric detection technology. Background Art

[0002] The motion sensing device based on the optical flow sensor can convert the texture information on the working surface into a light field pattern with a certain contrast at the image sensor on it, and the light field pattern can move with the relative displacement with the working surface. The image sensor collects the image sequence, uses the optical flow algorithm to calculate the correlation between frames in the image sequence, and combines the frame rate to evaluate the relative displacement between frames. It can extract the motion information on the working surface including displacement, speed, and direction of movement. This motion sensing scheme only requires a certain degree of roughness on the working surface and can adapt to various types of working surfaces. It has a wide range of applications in many emerging consumer fields, including detecting the displacement of optical mice or sweeping robots, detecting the displacement of paper in printers or scanners, detecting the rotation of the crown in smart watches, and positioning drones based on images without a global positioning system (GPS).

[0003] The optical flow sensor relies on the image sequence collected by the image sensor on it, and the refresh rate of the output displacement is limited by the frame rate of the image sensor. The main factors affecting the frame rate of the image sensor are the exposure time and the readout time. Reducing the exposure time of the image sensor can not only effectively improve the frame rate, but also effectively reduce the power consumption of the optical flow sensor for the light source that needs to emit light synchronously with the exposure moment. However, due to the insufficient exposure of each pixel in the image sensor, the output image signal will have a low signal-to-noise ratio, which will affect the image quality and reduce the resolution of the displacement of the working surface. Keeping the exposure of a single pixel in the image sensor unchanged and reducing the readout time are the technical problems to be solved by the present invention. The readout time of the image sensor is related to the pixel resolution. The main methods for reducing the effective pixel resolution are: 1. Undersampling method, that is, turning on one of every m×n pixels on the original pixel array through a dedicated circuit, and the rest of the pixels are in a closed state; 2. Pixel merging method, that is, connecting adjacent m×n pixels on the original pixel array through a dedicated circuit. To implement the above method, the pixel array on the image sensor includes a two-dimensional array formed by macro pixels with a resolution of M×N, each of which includes a two-dimensional array formed by sub pixels with a resolution of m×n. The working modes of the image sensor involved in the present invention mainly include: 1. Full image mode, that is, the m×n sub pixels in each macro pixel are sequentially exposed and read out by means of a rolling shutter to obtain a full resolution image; 2. Undersampling mode, that is, one of the sub pixels in each macro pixel is selected for exposure and read out to increase the frame rate; 3. Pixel merging mode, that is, all m×n sub pixels in each macro pixel are exposed at the same time, and the generated electrical signals are merged and read out to increase the amount of charge accumulated by a single macro pixel and expand the dynamic range. Summary of the invention

[0004] The object of the present invention is to provide a high-speed optical flow sensor chip with a variable pixel array, which can adapt to different motion scenes including high speed, high resolution and low illumination.

[0005] The object of the present invention is achieved in that:

[0006] The system block diagram of the motion sensing device (100) based on the optical flow sensor chip (106) is as follows Figure 1As shown, the motion sensing device (100) further includes a light source (102) and a working surface (104), characterized in that: the light source (102) can be any one of a light emitting diode (LED) and a vertical cavity surface emitting diode (VCSEL), and can be controlled by a driving signal output by an optical flow sensor chip (106) to emit a light beam; the light beam emitted by the light source (102) should have a certain light beam divergence angle, and the half-height full width of the light beam divergence angle is between 5° and 30°; the working surface (104) is a surface that can undergo relative displacement with the light source (102) and the optical flow sensor chip (106); the working surface (104) should have a certain roughness, and the profile arithmetic of the surface roughness is The average deviation (Ra) is greater than 0.5; there should be a certain gap between the working surface (104) and the light source (102) and the working surface (104), and the gap is between 0.5 mm and 20 mm, and 1.5 mm is optimal; the light beam emitted by the light source (102) is modulated by the working surface (104) to generate a light field pattern with a certain contrast, which is projected onto the optical flow sensor chip (106), and the projected pattern can change in time and space with the displacement of the working surface (104); the optical flow sensor chip (106) collects an image sequence and uses an optical flow algorithm to evaluate its relative displacement with the working surface (104), and outputs incremental displacement data ΔX and ΔY, and incremental time data ΔT.

[0007] The block diagram of the optical flow sensor chip (106) in the system is as follows Figure 2As shown, the optical flow sensor chip (106) is composed of a main controller (202), a light source driver (204), an image sensor (206), a data cache (208), an algorithm circuit (210), a register table (212), and an input / output interface (214), and is characterized in that: the main controller (202) can read the configuration parameters in the register table (212), send a light source driving instruction to the light source driver (204), adjust the gain, exposure time, and working modes including full image mode, undersampling mode, and pixel merging mode of the image sensor (206), and send a signal to the image sensor (206). ) issues an image acquisition instruction to switch between different power modes including an active mode, a sleep mode, and a hibernation mode; the light source driver (204) receives the light source drive instruction from the main controller and outputs a drive signal to the light source (102); the image sensor (206) receives the image acquisition instruction issued by the main controller (202), executes image acquisition according to the configuration parameters provided by the main controller (202), generates a sample image, and outputs it to the data cache (208); the data cache (208) is a volatile memory used to store the sample image generated by the image sensor (206) and the sample image generated by the algorithm circuit (206); 10) generates intermediate data including a reference image, reference displacements X0 and Y0, reference time T0, latest displacements X' and Y', and latest time T'; the algorithm circuit (210) compares the reference image and the sample image in the data cache (208) using an optical flow algorithm, calculates image correlation, evaluates absolute displacements X and Y, and absolute time T in combination with the reference displacements X0 and Y0 and the reference time T0, converts them into incremental displacements ΔX=X-X' and ΔY=Y-Y', and incremental time ΔT=T-T' of the current frame relative to the previous frame, and outputs them through an input / output interface (214); the image sensor (206) completes each For each image acquisition, the algorithm circuit (210) needs to update the register table (212) once, and update the latest displacement X' and Y' and the latest time T' thereon to the absolute displacement X and Y and the absolute time T; the register table (212) is used to store configuration parameters including the refresh rate of the displacement of the optical flow sensor chip (106), the power mode, the output data format, and the gain, exposure time, and working mode of the image sensor (206) therein; the input / output interface (214) can read and write the configuration parameters in the register table (212), and output the light source driving signal, the incremental displacement ΔX and ΔY, and the incremental time ΔT.

[0008] It is worth noting that: the optical flow sensor chip (106) that has just completed the initialization process does not have a reference image in its data cache (208); or as the displacement of the working surface (104) continues to accumulate, the displacement of the sample image relative to the reference image may exceed the threshold range; or the movement speed of the working surface (104) is too fast, and the frame rate of the image sensor (206) is too low. The above situations will cause the algorithm circuit (210) to be abnormal when calculating image correlation, resulting in image matching failure and inability to evaluate the displacement. Therefore, the algorithm circuit (210) should include a set of reference image update strategies that can update the sample image to the reference image when the above conditions are met, and update the intermediate data in the data cache (208) including the reference displacements X0 and Y0 and the reference time T0 to reduce the frequency of the above situations.

[0009] Optionally, the motion sensing device (100) may include a group of transmitting optical elements (103) and / or a group of receiving optical elements (105), wherein the transmitting optical elements (103) and / or the receiving optical elements (105) may be one or more of a lens, an attenuation plate, a spectral filter, and a polarizer. The lens may be used to match the light field of the light source (102) and the field of view of the optical flow sensor chip (106), reduce the demand for the output power of the light source (102), and increase the irradiance of the light field pattern projected onto the optical flow sensor chip (106); the attenuation plate may be used to reduce the intensity of the light field to avoid overexposure of the image sensor (206) on the optical flow sensor chip (106); and the spectral filter and the polarizer may reduce the non-signal light component in the light field pattern. The purpose of including the transmitting optical element (103) and / or the receiving optical element (105) is to improve the contrast and / or signal-to-noise ratio of the sample image collected by the motion sensing device (100).

[0010] A block diagram of the image sensor (206) in the system is shown in FIG. Figure 3As shown, the image sensor (206) is composed of a timing control circuit (302), a pixel control circuit (304), a pixel array (306), a pixel readout circuit (308), an analog-to-digital converter (310), and an image signal processor (312), and is characterized in that: the timing control circuit (302) can receive configuration parameters and image acquisition instructions output by the main controller (202), send a transmission instruction to the pixel control circuit (304), generate a reset control signal and a macro selection control signal to the pixel array (306) through the reset control line and the macro selection control line thereon, and send a read instruction to the pixel readout circuit (308) at a specified time; the pixel control circuit (304) is connected to the pixel array (306) through a group of control lines, can receive the transmission instruction issued by the timing control circuit (302), and generate a corresponding transmission control signal according to the required working mode; the pixel array (306) is a two-dimensional array composed of 36×36 pixels of resolution, and can generate a reset potential VR and a transmission potential VT according to the light field pattern irradiated thereon.

[0011] The pixel array (306) in the system is a two-dimensional array formed by macro pixels (400) with a resolution of M×N. Figure 4The circuit diagram is schematically shown, each macro pixel (400) is composed of a two-dimensional array formed by sub-pixels (402) with a resolution of m×n, a floating diffusion region (404), a source follower (406), a reset tube (408), a macro selection tube (410), and an output line (412). Each sub-pixel (402) is connected by a photodiode (414) and a transmission gate (416). It is characterized in that: M, N, m, and n in the pixel array (306) satisfy M×m=36, N×n=36, that is, all pixels are divided into macro pixels and sub-pixels; the reset tube (408), the macro selection tube (410), and the transmission gate (416) are field effect tubes with a gate, a source, and a drain, and the level of the gate can control The source and drain are controlled to be on and off, that is, when the gate is at a high level, the source and drain are turned on, the reset tube (408), the macro selection tube (410), and the transmission gate (416) are in an on state, and when the gate is at a low level, the source and drain are disconnected, the reset tube (408), the macro selection tube (410), and the transmission gate (416) are in a closed state; the photodiode (414) can accumulate charge by receiving light field radiation, and when the full well charge is not reached, the accumulated charge is linearly related to the exposure amount of the light field radiation received, and the higher the exposure amount of the light field radiation received, the more the accumulated charge is; the floating diffusion region (404) is connected to the transmission gates (416) on all sub-pixels (402) in the macro pixel (400) where it is located, and its full well The amount of charge is not less than the sum of the full well charges of the photodiodes (414) on all the sub-pixels (402) in the macro pixel (400) where the photodiodes (414) are located, and the well depth is deeper than that of the photodiodes (414), which is conducive to the merging of the charges accumulated from different photodiodes (414); the floating diffusion region (404) is connected to the reset tube (408), and its well depth is shallower than VDD, which is conducive to the accumulated charges flowing out to VDD; the floating diffusion region (404) is connected to the gate of the source follower (406), and the potential of the drain of the source follower (406) is linearly related to the amount of charge accumulated by the floating diffusion region (404). The more the amount of charge accumulated by the floating diffusion region (404), the greater the potential of the drain of the source follower (406). low; the drain of the source follower (406) is connected to the output line (412) through the macro selection tube (410); the gates of the transmission gates (416) on the sub-pixels (402) at the same position in all macro pixels (400) are connected to each other, that is, the gates of the transmission gates (416) on the sub-pixels (402) at position 1 are connected to each other, and the gates of the transmission gates (416) on the sub-pixels (402) at position 2 are connected to each other, and so on, the interconnected gates are connected to the pixel control circuit (304) as control lines, that is, the pixel control circuit (304) is respectively connected to the gates of the transmission gates (416) on the m×n sub-pixels (402) in all macro pixels (400) through m×n control lines;The gates of the reset tubes (408) in all macro-pixels (400) are connected to each other, and the connected gates are connected to the timing control circuit (302) as reset control lines, that is, the timing control circuit (302) is connected to the gates of the reset tubes (408) on all macro-pixels (400) through one reset control line; the gates of the macro-selection tubes (410) in all macro-pixels (400) are connected to each other, and the connected gates are connected to the timing control circuit (302) as macro-selection control lines, that is, the timing control circuit (302) is connected to the gates of the macro-selection tubes (410) on all macro-pixels (400) through one macro-selection control line; the output line (412) on each macro-pixel (400) is connected to the pixel readout circuit (308) as a data line, that is, the pixel readout circuit (308) is connected to the voltage output lines on the M×N macro-pixels (400) in the pixel array (306) through M×N data lines. ;

[0012] Figure 5 The image acquisition process shown is divided into a reset process, an exposure process, and a readout process, and is characterized in that: during the reset process, all transmission gates (416), reset tubes (408), and macro selection tubes (410) on the pixel array (306) are in an open state to clear the residual charges on all photodiodes (414) and floating diffusion areas (404); before the exposure process begins, all photodiodes (414) do not contain any charge, and all transmission gates (416), reset tubes (408), and macro selection tubes (410) are closed, and the photodiodes (414) begin to accumulate charges due to light field radiation, and the accumulated charges are confined in the photodiodes (414); before the exposure process ends, the macro selection tubes (410) and reset tubes (408) are opened to clear the charges on all floating diffusion areas (404), and when the reset tubes (408) are closed, all floating diffusion areas (404) do not contain any charge. The pixel readout circuit (308) immediately reads the potential on the output line (412), which is recorded as the reset potential VR, and then opens all or part of the transmission gates (416) according to the transmission control signal output by the pixel control circuit (304). At the moment when any transmission gate (416) is opened, the exposure process ends and the readout process begins. At this time, all macro selection tubes (410) are in the open state, and all reset tubes (408) are in the closed state. The charge accumulated by the photodiode (414) on the sub-pixel (402) with the transmission gate (416) turned on is transferred to the floating diffusion region (404) with a deeper potential well, and all charge transfer is completed at the moment when the transmission gate (416) is closed. The corresponding photodiode (414) does not contain any charge. The pixel readout circuit (308) immediately reads the potential on the output line (412), which is recorded as the transmission potential VT, and then the readout process ends.

[0013] Figure 3 The image sensor (206) shown is characterized in that: a pixel readout circuit (308) is connected to the pixel array (306) through a group of data lines, and can receive a read instruction issued by a timing control circuit (302), read the reset potential VR and the transmission potential VT on the pixel array (306), and generate an analog signal associated with the potential difference VR-VT between the two, and the voltage value on the analog signal is linearly related to the exposure amount of the light field radiation on the pixel array (306), and can be used as an input signal of an analog-to-digital converter (310); The analog-to-digital converter (310) is used to convert the analog signal generated by the pixel readout circuit (308) into a digital signal and output it to the image signal processor (312); the image signal processor (312) receives the digital signal generated by the analog-to-digital converter (310), performs image processing processes including brightness and contrast compensation, image fusion, and image filtering on the digital signal, generates a grayscale image whose grayscale value is linearly related to the exposure amount of the light field radiation received by the pixel array (306), and outputs it to the data cache (208) as a sample image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The system block diagram of a motion sensing device (100) based on an optical flow sensor is provided, wherein the system is composed of a light source (102), a working surface (104), and an optical flow sensor chip (106); optionally, the system may include a group of transmitting optical elements (103) and / or a group of receiving optical elements (105).

[0015] Figure 2 It is a block diagram of an optical flow sensor chip (106). The optical flow sensor chip (106) is composed of a main controller (202), a light source driver (204), an image sensor (206), a data cache (208), an algorithm circuit (210), a register table (212), and an input / output interface (214). The arrows in the block diagram show the direction of signal transmission inside the chip.

[0016] Figure 3 It is a block diagram of an image sensor (206), which is composed of a timing control circuit (302), a pixel control circuit (304), a pixel array (306), a pixel readout circuit (308), an analog-to-digital converter (310), and an image signal processor (312). The pixel array (306) is a two-dimensional array formed by a resolution of 36×36 pixels. The arrows in the block diagram indicate the direction of signal transmission inside the chip.

[0017] Figure 4The circuit diagram of the pixel array (306) is schematically shown, wherein each macro pixel (400) is composed of a two-dimensional array formed by multiple sub-pixels (402), a floating diffusion region (404), a source follower (406), a reset tube (408), a macro selection tube (410), and an output line (412). Each sub-pixel (402) is connected by a photodiode (414) and a transmission gate (416). The states of all reset tubes (408) and macro selection tubes (410) are respectively controlled by a reset control signal and a macro selection control signal on a timing control circuit (302), and the states of all transmission gates (416) are controlled by a transmission control signal on a pixel control circuit (304). The potential on the output line (412) can be read by a pixel readout circuit (308).

[0018] Figure 5 A timing diagram of an image acquisition process is schematically shown. An image acquisition process is divided into a reset process, an exposure process, and a readout process. During the exposure process and the readout process, the pixel readout circuit (308) reads the potential on the output line (412) when the reset tube (408) and the macro selection tube (410) are respectively turned off, and is recorded as the reset potential VR and the transmission potential VT respectively. DETAILED DESCRIPTION

[0019] An image sampling process includes a reset process, an exposure process, and a readout process in sequence.

[0020] Embodiment 1:

[0021] During the reset process, the timing control circuit (302) and the pixel control circuit (304) control all reset tubes (408), macro selection tubes (410) and transmission gates (416) to be in an open state to clear the residual charge in the photodiode (414) and the floating diffusion area (404).

[0022] The exposure process starts when all reset tubes (408), macro selection tubes (410), and transmission gates (416) are closed, and all photodiodes (414) begin to accumulate charges when receiving light field radiation. The accumulated charges are limited in the photodiodes (414). When the charge amount of the photodiodes (414) does not exceed the full well charge amount, the accumulated charge amount of the photodiodes (414) is linearly related to the exposure amount of the light field radiation received. Before the end of the exposure process, the timing control circuit (302) controls all reset tubes (408) and macro selection tubes (410) to turn on. At this time, all reset tubes (408) and macro selection tubes (410) are in the open state, and all transmission gates (416) are in the closed state. The charges in the floating diffusion area (404) are pulled out by the electric field to clear the charges remaining in the floating diffusion area (404). All reset tubes (408) are turned off after being turned on for a period of time. At this time, all macro selection tubes (410) are in the turned-on state, and all reset tubes (408) and transmission gates (416) are in the turned-off state. The timing control circuit (302) controls the pixel readout circuit (308) to read the potential on all output lines (412) immediately after the reset tube (408) is turned off, which is recorded as the reset potential VR.

[0023] Embodiment 2:

[0024] In the full pixel mode, during the readout process in an image sampling process, the pixel control circuit (304) controls the transmission gate (416) on the sub-pixel (402) at a certain position in all macro pixels (400) to be turned on, and the remaining transmission gates (416) are in a closed state, all macro selection tubes (410) are in an open state, and all reset tubes (408) are in a closed state.

[0025] At the moment when any transmission gate (416) is turned on, the exposure process ends and the readout process begins. The charges in the photodiodes (414) on the sub-pixels (402) whose transmission gates (416) are turned on are pulled by the electric field to flow to the floating diffusion region (404). At the moment when these transmission gates (416) are turned off, all the charges in the photodiodes (414) flow to the floating diffusion region (404), and the charges in the photodiodes (414) are cleared. The amount of charge in the floating diffusion region (404) is equal to the amount of charge accumulated in the photodiodes (414) before the start of the readout process. At the moment when the transmission gates (416) are turned off, the timing control circuit (302) controls the pixel readout circuit (308) to immediately read the potentials on all output lines (412), which are recorded as the transmission potential VT, and then the readout process ends.

[0026] In particular, in different image sampling processes in the full pixel mode, the pixel control circuit (304) controls the transmission gate (416) at the corresponding position in all macro pixels (400) to be turned on, that is, in the first sampling cycle, the pixel control circuit (304) controls the transmission gate (416) on the sub-pixel (402) at position 1 in all macro pixels (400) to be turned on, that is, in the second sampling cycle, the pixel control circuit (304) controls the transmission gate (416) on the sub-pixel (402) at position 2 in all macro pixels (400) to be turned on, and so on. After completing the image signal acquisition of all m×n sub-pixels (402), the pixel readout circuit (308) generates an analog signal associated with the potential difference VR-VT between the reset potential and the transmission potential, and the digital signal generated by the analog-to-digital converter (310) is then subjected to image fusion by the image signal processor (312) to splice a full pixel image with a resolution of 36×36.

[0027] In full-pixel mode, the image sensor (206) can output a 36×36 full-pixel image. The spatial resolution of the optical flow sensor chip (106) is relatively high. The image sensor (206) requires m×n image sampling processes to generate an image. The frame rate of the optical flow sensor chip (106) is relatively low, and is suitable for low-speed, high-resolution motion scenes.

[0028] Embodiment three:

[0029] In the undersampling mode, during the readout process in any image sampling process, the pixel control circuit (304) controls the transmission gate (416) on one of the sub-pixels (402) in all the macro-pixels (400) to be turned on, the remaining transmission gates (416) are in a closed state, all the macro selection tubes (410) are in an open state, and all the reset tubes (408) are in a closed state.

[0030] At the moment when any transmission gate (416) is turned on, the exposure process ends and the readout process begins. The charges in the photodiodes (414) on the sub-pixels (402) with the transmission gates (416) turned on are pulled by the electric field to flow to the floating diffusion region (404). At the moment when these transmission gates (416) are turned off, all the charges in the photodiodes (414) flow to the floating diffusion region (404), and the charges in the photodiodes (414) are cleared. The amount of charge in the floating diffusion region (404) is equal to the amount of charge accumulated in the photodiodes (414) before the start of the readout process. At the moment when the transmission gates (416) are turned off, the timing control circuit (302) controls the pixel readout circuit (308) to immediately read the potentials on all output lines (412), which are recorded as the transmission potential VT, and then the readout process ends.

[0031] In the undersampling mode, the image sensor (206) can output an undersampled image with a resolution of M×N, the spatial resolution of the optical flow sensor chip (106) is low, the image sensor (206) only needs one image sampling process to generate an image, and the frame rate of the optical flow sensor chip (106) is high, which is suitable for high-speed, low-resolution motion scenes.

[0032] Embodiment 4:

[0033] In the pixel merging mode, during the readout process in any image sampling process, the pixel control circuit (304) controls the transmission gates (416) on all sub-pixels (402) to open, all transmission gates (416) and macro selection tubes (410) are in an open state, and all reset tubes (408) are in a closed state.

[0034] At the moment when any transmission gate (416) is turned on, the exposure process ends and the readout process begins. The charges in all photodiodes (414) are pulled to the floating diffusion region (404) by the electric field. At the moment when the transmission gate (416) is turned off, all the charges in the photodiode (414) flow to the floating diffusion region (404), and the charges in the photodiode (414) are cleared. The amount of charge in the floating diffusion region (404) is equal to the total amount of charge accumulated by all photodiodes (414) on the macro pixel (400) before the start of the readout process. At the moment when the transmission gate (416) is turned off, the timing control circuit (302) controls the pixel readout circuit (308) to immediately read the potential on all output lines (412), which is recorded as the transmission potential VT, and then the readout process ends.

[0035] In the pixel merging mode, the image sensor (206) can output an image with a resolution of M×N, the spatial resolution of the optical flow sensor chip (106) is low, the image sensor (206) only needs one sampling cycle to generate an image, and the frame rate of the optical flow sensor chip (106) is high. Since the floating diffusion area (404) on a single macro pixel (400) can accumulate a larger amount of charge, the image sensor (206) in this working mode has a larger dynamic range and is suitable for high-speed, low-resolution, and low-light motion scenes.

Claims

1. A high-speed optical flow sensor chip with a variable pixel array, characterized by: The motion sensing device (100) based on the optical flow sensor chip (106) also includes a light source (102) and a working surface (104); the light source (102) can be any one of a light emitting diode (LED) and a vertical cavity surface emitting diode (VCSEL), and can be controlled by a driving signal output by the optical flow sensor chip (106) to emit a light beam; the light beam emitted by the light source (102) should have a certain beam divergence angle, and the half-height full width of the beam divergence angle is between 5° and 30°; the working surface (104) is a surface that can undergo relative displacement with the light source (102) and the optical flow sensor chip (106); the working surface (104) should have a certain roughness, and the wheel of its surface roughness The arithmetic mean deviation (Ra) of the profile is greater than 0.5; there should be a certain gap between the working surface (104) and the light source (102) and the working surface (104), and the gap is between 0.5 mm and 20 mm, and 1.5 mm is optimal; the light beam emitted by the light source (102) is modulated by the working surface (104) to generate a light field pattern with a certain contrast and projected onto the optical flow sensor chip (106), and the projected pattern can change in time and space with the displacement of the working surface (104); the optical flow sensor chip (106) collects an image sequence and uses an optical flow algorithm to evaluate its relative displacement with the working surface (104), and outputs incremental displacement data ΔX, ΔY, and incremental time data ΔT.

2. The motion sensing device according to claim 1, characterized in that: The motion sensing device (100) may include a group of transmitting optical elements (103) and / or a group of receiving optical elements (105); the transmitting optical elements (103) and / or the receiving optical elements (105) may be one or more of a lens, an attenuation plate, a spectral filter, and a polarizing plate; the contrast and / or signal-to-noise ratio of the sample image collected by the motion sensing device (100) including the transmitting optical elements (103) and / or the receiving optical elements (105) should not be lower than the contrast and / or signal-to-noise ratio of the sample image collected by the motion sensing device (100) not including the transmitting optical elements (103) and / or the receiving optical elements (105).

3. The optical flow sensor chip according to claim 1, characterized in that: The optical flow sensor chip (106) is composed of a main controller (202), a light source driver (204), an image sensor (206), a data cache (208), an algorithm circuit (210), a register table (212), and an input / output interface (214); the main controller (202) can read the configuration parameters in the register table (212), issue a light source driving instruction to the light source driver (204), adjust the configuration parameters of the image sensor (206), issue an image acquisition instruction to the image sensor (206), and implement power mode switching; the light source driver (204) receives the light source driving instruction from the main controller and outputs a driving signal to the light source (102); the image sensor (206) receives the image acquisition instruction issued by the main controller (202), executes image acquisition according to the configuration parameters provided by the main controller (202), generates a sample image, and outputs it to the data cache (208). ); the data cache (208) is a volatile memory used to store sample images generated by the image sensor (206) and intermediate data generated by the algorithm circuit (210); the algorithm circuit (210) uses an optical flow algorithm to compare the reference image and the sample image in the data cache (208), evaluates the absolute displacements X and Y, and the absolute time T, and outputs the incremental displacements ΔX and ΔY, and the incremental time ΔT through the input / output interface (214); the algorithm circuit (210) should include a set of reference image update strategies, and when conditions are met, the reference image and intermediate data in the data cache (208) are updated; the register table (212) is used to store configuration parameters including the optical flow sensor chip (106) and the image sensor (206) therein; the input / output interface (214) can read and write the configuration parameters in the register table (212), and output the light source driving signal, the incremental displacements ΔX and ΔY, and the incremental time ΔT.

4. The image sensor according to claim 3, characterized in that: The image sensor (206) is composed of a timing control circuit (302), a pixel control circuit (304), a pixel array (306), a pixel readout circuit (308), an analog-to-digital converter (310), and an image signal processor (312); the timing control circuit (302) is capable of receiving configuration parameters and image acquisition instructions output by a main controller (202), issuing a transmission instruction to the pixel control circuit (304), generating a reset control signal and a macro selection control signal to the pixel array (306) through a reset control line and a macro selection control line thereon, and issuing a read instruction to the pixel readout circuit (308) at a specified time; the pixel control circuit (304) is connected to the pixel array (306) through a group of control lines, is capable of receiving a transmission instruction issued by the timing control circuit (302), and generates a corresponding transmission control signal according to a required working mode; the pixel array (306) is a two-dimensional array consisting of 36×36 pixels in resolution , capable of generating a reset potential VR and a transmission potential VT according to a light field pattern irradiated thereon; a pixel readout circuit (308) is connected to a pixel array (306) through a group of data lines, capable of receiving a read instruction issued by a timing control circuit (302), reading the reset potential VR and the transmission potential VT on the pixel array (306), and generating an analog signal associated with a potential difference VR-VT between the two as an input signal of an analog-to-digital converter (310); the analog-to-digital converter (310) is used to convert the analog signal generated by the pixel readout circuit (308) into a digital signal, and output it to an image signal processor (312); the image signal processor (312) receives the digital signal generated by the analog-to-digital converter (310), performs an image processing process on the digital signal, generates a grayscale image whose grayscale value is linearly related to the exposure amount of the light field radiation received by the pixel array (306), and outputs it to a data cache (208) as a sample image.

5. The pixel array according to claim 4, characterized in that: The pixel array (306) is a two-dimensional array formed by macro pixels (400) with a resolution of M×N. Each macro pixel (400) is composed of a two-dimensional array formed by sub pixels (402) with a resolution of m×n, a floating diffusion region (404), a source follower (406), a reset tube (408), a macro selection tube (410), and an output line (412). Each sub pixel (402) is connected by a photodiode (414) and a transmission gate (416), wherein M, N, m, and n satisfy M×m=36 and N×n=36; the reset tube (408), the macro selection tube (410), the transmission gate (416) ) is a field effect transistor having a gate, a source, and a drain, and can be turned on or off by the level control of its gate; the photodiode (414) can accumulate charge when it is irradiated by a light field, and when its full well charge is not reached, the accumulated charge is linearly related to the exposure amount of the light field radiation it is exposed to; the floating diffusion region (404) is connected to the transmission gate (416) on all sub-pixels (402) in the macro pixel (400) where it is located, and its full well charge is not less than the sum of the full well charges of the photodiodes (414) on all sub-pixels (402) in the macro pixel (400) where it is located, and the well depth is deeper than the photodiode (4 14) is deeper, which is conducive to the merging of charges accumulated from different photodiodes (414); the floating diffusion region (404) is connected to the reset tube (408), and its well depth is shallower than VDD, which is conducive to the accumulated charges flowing out to VDD; the floating diffusion region (404) is connected to the gate of the source follower (406), and the potential of the drain of the source follower (406) is linearly related to the amount of charge accumulated in the floating diffusion region (404); the drain of the source follower (406) is connected to the output line (412) through the macro selection tube (410); the sub-pixels (402) at the same position in all macro pixels (400) The gates of the transmission gates (416) on the macro pixels (400) are connected to each other, and the gates connected to each other are connected to the pixel control circuit (304) as a control line; the gates of the reset tubes (408) in all macro pixels (400) are connected to each other, and the gates connected to each other are connected to the timing control circuit (302) as a reset control line; the gates of the macro selection tubes (410) in all macro pixels (400) are connected to each other, and the gates connected to each other are connected to the timing control circuit (302) as a macro selection control line; the output line (412) on each macro pixel (400) is connected to the pixel readout circuit (308) as a data line.

6. The pixel array according to claim 4, characterized in that: The pixel array (306) is capable of receiving control signals output by the timing control circuit (302) and the pixel control circuit (304), performing image acquisition, and outputting voltage signals to the pixel readout circuit (308); an image acquisition process is divided into a reset process, an exposure process, and a readout process; during the reset process, all transmission gates (416), reset tubes (408), and macro selection tubes (410) on the pixel array (306) are in an open state to clear all residual charges on the photodiodes (414) and the floating diffusion region (404); before the exposure process begins, all transmission gates (416), reset tubes (408), and macro selection tubes (410) are closed, and the photodiodes (414) begin to accumulate charges due to light field radiation, and the accumulated charges are confined in the photodiodes (414); before the exposure process ends, the macro selection tubes (410) are opened. 10) A reset tube (408) is turned on to clear all the charges on the floating diffusion area (404), and the pixel readout circuit (308) immediately reads the potential on the output line (412), which is recorded as the reset potential VR; then, according to the transmission control signal output by the pixel control circuit (304), all or part of the transmission gates (416) are turned on; at the moment when any transmission gate (416) is turned on, the exposure process ends and the readout process begins. At this time, all macro selection tubes (410) are in the turned-on state, and all reset tubes (408) are in the turned-off state. All the charges accumulated in the photodiode (414) on the sub-pixel (402) with the turned-on transmission gate (416) are transferred to the floating diffusion area (404). At the moment when the transmission gate (416) is turned off, the pixel readout circuit (308) immediately reads the potential on the output line (412), which is recorded as the transmission potential VT.

7. The image sensor according to claim 3, wherein: In the full pixel mode, during a readout process in an image sampling process, the pixel control circuit (304) controls the transmission gate (416) on the sub-pixel (402) at a certain position in all macro-pixels (400) to be turned on, and the remaining transmission gates (416) are in a closed state, all macro selection tubes (410) are in an open state, and all reset tubes (408) are in a closed state; the floating diffusion region (404) on each macro-pixel (400) obtains the charge in the photodiode on different sub-pixels (402) during different image acquisition processes; After completing the image acquisition process of all m×n sub-pixels (402), the digital signal generated by the pixel readout circuit (308) and the analog-to-digital converter (310) is input into the image signal processor (312) to implement image fusion, and a full-pixel image with a resolution of 36×36 is spliced. In this working mode, the image sensor (206) can output a full-pixel image with a resolution of 36×36. The spatial resolution of the optical flow sensor chip (106) is relatively high. The image sensor (206) requires m×n image sampling processes to generate an image. The frame rate of the optical flow sensor chip (106) is relatively low, and is suitable for low-speed, high-resolution motion scenes.

8. The image sensor according to claim 3, characterized in that: In the undersampling mode, during a readout process in an image sampling process, the pixel control circuit (304) controls the transmission gate (416) on one of the sub-pixels (402) in all the macro-pixels (400) to be turned on, and the remaining transmission gates (416) are in a closed state, all the macro-selection tubes (410) are in an open state, and all the reset tubes (408) are in a closed state; the floating diffusion region (404) on each macro-pixel (400) obtains the charge in the photodiode on the same sub-pixel (402) during different image acquisition processes; After completing an image acquisition process, an image with a resolution of M×N can be generated in sequence through a pixel readout circuit (308), an analog-to-digital converter (310), and an image signal processor (312); in this working mode, the image sensor (206) can output an undersampled image with a resolution of M×N, the spatial resolution of the optical flow sensor chip (106) is relatively low, the image sensor (206) only needs one image sampling process to generate an image, and the frame rate of the optical flow sensor chip (106) is relatively high, which is suitable for high-speed, low-resolution motion scenes.

9. The image sensor according to claim 3, characterized in that: In the pixel merging mode, during a readout process in an image sampling process, the pixel control circuit (304) controls all the transmission gates (416) to be turned on; the floating diffusion region (404) on each macro pixel (400) acquires the charge in the photodiodes on all the sub-pixels (402) thereon during an image acquisition process; After completing an image acquisition process, an image with a resolution of M×N can be generated in sequence through a pixel readout circuit (308), an analog-to-digital converter (310), and an image signal processor (312); in this working mode, each floating diffusion region (404) can accumulate more charge, the dynamic range of the image sensor (206) is larger, the image sensor (206) can output an image with a resolution of M×N, the spatial resolution of the optical flow sensor chip (106) is lower, the image sensor (206) only needs one image sampling process to generate an image, the frame rate of the optical flow sensor chip (106) is higher, and it is suitable for high-speed, low-resolution, and low-light motion scenes.

Images